linaro-image-tools-2014.11/0000775000175000017500000000000012435067365016467 5ustar fboudrafboudra00000000000000linaro-image-tools-2014.11/ChangeLog0000664000175000017500000065112312435067251020243 0ustar fboudrafboudra000000000000002014-11-25 Fathi Boudra Release 2014.11. 2014-09-12 Yongqin Liu linaro-android-media-create: fix problem for deploying data partition 2014-08-25 Fathi Boudra Release 2014.08. 2014-08-11 Koen Kooi media-create: sort devicelist in help output alphabetically. 2014-07-30 Yongqin Liu unpack_binary_tarball.py: add --selinux option for tar command 2014-08-03 Neil Williams Drop all calls to sync 2014-07-31 Fathi Boudra Release 2014.07. 2014-07-31 Yongqin Liu fix error when not specify --userdataimage option 2014-07-15 Fathi Boudra Revert "sgdisk requires a block device" 2014-06-26 Yongqin Liu linaro-android-media-create: add support to use img files 2014-06-14 Fathi Boudra sgdisk requires a block device 2014-06-04 Semen Protsenko linaro-media-create: add GPT support (--part-table option) 2014-06-14 Fathi Boudra Refactor kernel image renaming to avoid to hardcode the kernel name 2014-04-09 Ryan Harkin Add a generic board config 2014-06-14 Fathi Boudra Fix tests regression introduced by multi_arch support 2014-05-27 Fathi Boudra fastmodel: copy initrd to the output directory 2014-05-26 Fathi Boudra copy qemu-aarch64-static binary in the chroot 2014-04-24 Fathi Boudra Release 2014.04. 2014-04-24 Fathi Boudra Revert "Release 2014.04." 2014-04-24 Fathi Boudra Specify gzip as the compression type to use when building dummy package (LP: #1311829) 2014-04-24 Fathi Boudra Add multi-arch support (LP: #1311829) 2014-04-24 Fathi Boudra gitreview: update project 2014-04-23 Fathi Boudra Release 2014.04. 2014-04-07 Ryan Harkin juno: extract juno binaries to the boot_dir 2014-02-27 Fathi Boudra Post-release version bump to 2014.02.1. 2014-02-27 Fathi Boudra Release 2014.02. 2014-02-25 Tushar Behera arndale_octa: Rename the device name to match Android build 2014-02-22 Fathi Boudra Fix files permissions in copy_drop() to avoid empty Image (LP #1259885) 2014-02-20 Tushar Behera Arndale Octa: Add support for Android build 2014-01-23 Milo Casagrande Post-release version bump to 2014.01.1. 2014-01-23 Milo Casagrande Release 2014.01. 2014-01-15 Fathi Boudra Fix network interface not updated (LP: #1268776) 2014-01-10 Fathi Boudra Arndale Octa: add mac address to bootargs to avoid random address 2014-01-08 Fathi Boudra Arndale Octa: update bootloader flavor 2014-01-08 Fathi Boudra Add support for Arndale Octa 2013-12-20 Milo Casagrande Remove not necessary white spaces from boot command. 2013-12-20 Milo Casagrande Post-release version bump to 2013.12.1. 2013-12-20 Milo Casagrande Release 2013.12. 2013-12-20 Milo Casagrande Fixed typo. 2013-12-20 Milo Casagrande Fixed typos, added push command for tag. 2013-12-20 Milo Casagrande Fixed wrong test. 2013-12-16 Milo Casagrande Added new linaro-hwpack-append dcommand line program. 2013-12-13 Fathi Boudra Copy all QEMU ARM binaries (arm, armeb, arm64) 2013-12-13 Fathi Boudra Refactor the code to detect rootfs distribution 2013-12-13 Fathi Boudra Fix typos in comments 2013-12-12 Milo Casagrande Updated do-release script for git. 2013-12-12 Milo Casagrande Make sure we can run l-i-t from the checkout. 2013-12-12 Milo Casagrande Added gitreview file and renamed bzrignore. 2013-11-29 Vishal Bhoj Update the tests wrt to the latest changes in system partition size for Android 2013-11-28 Vishal Bhoj Increase system partition size since we run out of space while creating image that includes native-toolchain. 2013-10-24 Georgy Redkozubov Added command line option --dtb-file 2013-10-14 Georgy Redkozubov Added checks for dtb_files and if found set first entry as default dtb file. 2013-09-26 Fathi Boudra Post-release version bump to 2013.09.1 2013-09-26 Fathi Boudra Release 2013.09 2013-09-23 Ryan Harkin Rename file after copy for FastModel. 2013-09-19 Fathi Boudra Use wildcard to include dtbs 2013-09-19 Fathi Boudra Add support for the new ARM models 2013-08-29 Milo Casagrande Post-release version bump to 2013.08.1 2013-08-29 Milo Casagrande Release 2013.08 2013-08-19 Fathi Boudra Revert Fedora support - regression 2013-08-17 Fathi Boudra Fix populate rootfs test 2013-08-16 Fathi Boudra Add workaround for Arndale bootcmd on OpenEmbedded (LP: #1208815) 2013-08-16 Fathi Boudra Add AA9 tests 2013-08-16 Fathi Boudra Add AA9 board support 2013-08-08 Fathi Boudra Initial support for Fedora rootfs 2013-08-08 Fathi Boudra Fix Highbank GetBootCmd test after bootm fixes 2013-08-06 Fathi Boudra Fix _get_bootcmd() to cover all bootm use cases: * kernel only * kernel + initrd * kernel + dtb * kernel + initrd + dtb 2013-08-06 Fathi Boudra Fix typos 2013-07-25 Milo Casagrande Post-release version bump to 2013.07.1 2013-07-25 Milo Casagrande Release 2013.07 2013-07-25 Milo Casagrande PEP8 fixes. 2013-06-28 Fathi Boudra with fastmodel device type, copy kernel/dtb/initrd in the boot partition (LP:#1192832) 2013-06-27 Milo Casagrande Post-release version bump to 2013.06.1 2013-06-27 Milo Casagrande Release 2013.06 2013-06-27 Milo Casagrande PEP8 fixes. 2013-06-04 Milo Casagrande Fixed missing import statements. 2013-06-04 Milo Casagrande Moved PackageUnpacker into its own file/class. 2013-05-29 Milo Casagrande Post-release version bump to 2013.05.1 2013-05-29 Milo Casagrande Release 2013.05 2013-05-23 Fathi Boudra Add tests for beaglebone support 2013-05-23 Fathi Boudra Add beaglebone support 2013-04-24 Milo Casagrande Post-release version bump to 2013.04.1 2013-04-24 Milo Casagrande Release 2013.04 2013-04-04 Fathi Boudra Highbank: add tests 2013-04-03 James Tunnicliffe PEP8 fixes. 2013-04-03 Milo Casagrande Fixed MMC option for Snowball EMMC board. 2013-04-03 Milo Casagrande Add dconf call to disable automount. 2013-04-02 Fathi Boudra Highbank: load dtb provided by the firmware 2013-03-27 Milo Casagrande Post-release version bump to 2013.03.1 2013-03-27 Milo Casagrande Release 2013.03 2013-03-26 Milo Casagrande Fixed regression when using actual dtb file name, not the general one. 2013-03-26 Fathi Boudra partly revert trunk merge to fix a regression 2013-03-25 Fathi Boudra Refactor Highbank support: - introduce load_interface (default is mmc) - introduce bootfs_type (default is vfat) - get rid of Highbank inherited functions 2013-03-25 Milo Casagrande Added a test for Arndale under Android, fixed other tests. 2013-03-25 Milo Casagrande Fixed problem not picking up correct DTB file name. 2013-03-25 Milo Casagrande Completed support for Arndale in android boards. 2013-03-25 Fathi Boudra Highbank support: - load kernel/initrd from ext2 boot parttion - fix boot_cmd to boot from disk 2013-03-25 Milo Casagrande Added Arndale support for Android boards. 2013-03-22 Fathi Boudra Initial Highbank board config 2013-03-21 Milo Casagrande Bumped value of samsung_env_start for Origen Quad. 2013-03-20 Milo Casagrande Fixed tests, added a new test for exception raised. 2013-03-20 Milo Casagrande Added custom exception. 2013-03-20 Milo Casagrande Fixed samsung_env_start value. 2013-03-19 Milo Casagrande Added overridden method for Origen Quad bootloader support. 2013-02-26 Stevan Radakovic Post-release version bump to 2013.02.1 2013-02-26 Stevan Radakovic Release 2013.02 2013-02-25 Stevan Radakovic Update README file. 2013-02-25 Stevan Radakovic Fix pep8. 2013-02-25 Stevan Radakovic Fix error in tests. 2013-02-25 Stevan Radakovic Sort out exception handling when CommandNotFound is not present. 2013-02-21 James Tunnicliffe Slightly updated apt-get update call to try harder to complete without a net connection 2013-02-20 James Tunnicliffe When installing hardware pack allow apt-get update from internet sources to fail. Provide default for BoardConfig.bootloader_copy_files. 2013-02-18 Stevan Radakovic Another final run through pep8 fixes. 2013-02-18 Stevan Radakovic Another final run through pep8 fixes. 2013-02-18 Stevan Radakovic Final run through pep8 fixes. 2013-02-18 Stevan Radakovic Syntax error fix. 2013-02-18 Stevan Radakovic Fourth run of pep8 fixes. 2013-02-17 Stevan Radakovic Third run of pep8 fixes. 2013-02-14 Stevan Radakovic More PEP8 fixes. 2013-02-07 Stevan Radakovic First pass of pep8 fixes for 12.10. 2013-02-07 Stevan Radakovic Test change for pep8 fixes. 2013-01-30 Milo Casagrande Reworked args checking, added tests. 2013-01-28 Georgy Redkozubov Post-release version bump to 2013.01.1 2013-01-28 Georgy Redkozubov Release 2013.01 2013-01-22 Vishal Bhoj Pick hwpack for android from boottarball if available. 2013-01-15 Fathi Boudra Add test_arndale_make_boot_files_v2 test 2013-01-14 Milo Casagrande Reworked bootloader setting. 2013-01-14 Fathi Boudra Add test_arndale to cover _get_boot_env overridden method 2013-01-14 Fathi Boudra Move *_addr and samsung_* fields in Arndale hwpack config file 2013-01-12 Fathi Boudra Add ethact/ethaddr to Arndale boot_env (LP: #1097265) 2013-01-12 Fathi Boudra Initial support for Arndale BL1 binary (samsung_bl0_*) 2013-01-12 Fathi Boudra Fix samsung_{bl1,bl2,env}_{start,len} metadata initialisation. 2013-01-10 Milo Casagrande Refactored Config attributes board and bootloader. 2013-01-09 Milo Casagrande Fixed regression: missing check on None strings; fixed tests. 2013-01-07 James Tunnicliffe Add python-mock to required packages for development 2013-01-07 Milo Casagrande Refactored serial_opts into serial_options. 2013-01-07 Milo Casagrande Unified variable name between boards and android_boards. 2013-01-03 Milo Casagrande Catch exception on missing file, added test. 2013-01-03 Milo Casagrande Refactored and harmonized variable name. 2013-01-03 Milo Casagrande Fixed problem with StringIO, now use tmp file. 2013-01-02 Milo Casagrande Refactored field name, handled file opening. 2013-01-02 Milo Casagrande Removed wrong method call. 2012-12-28 Milo Casagrande Refactored tests: made YAML string a little bit more readable. 2012-12-28 Milo Casagrande Reverted atexit call. 2012-12-28 Milo Casagrande Added tests for default instance values with Android boards. 2012-12-27 Milo Casagrande Fixed arndale tests, cleaned-up code. 2012-12-27 Milo Casagrande Fixed two tests. 2012-12-27 Milo Casagrande Refactored OrigenQuad to be an instance. 2012-12-27 Milo Casagrande Added comments. 2012-12-27 Milo Casagrande Added couple of tests for mmc_id field handling. 2012-12-23 Fathi Boudra Add Arndale board support 2012-12-20 Milo Casagrande Post-release version bump to 2012.12.1 2012-12-20 Milo Casagrande Release 2012.12 2012-12-12 Tushar Behera Add OrigenQuad board support 2012-12-11 Milo Casagrande Fixed tests with new attribute name. 2012-12-11 Milo Casagrande Fixed wrong attribute name, added missing one. 2012-12-10 Milo Casagrande Added missing copyright header, fixed years. 2012-12-10 Milo Casagrande Fixed PEP8 warning. 2012-12-10 Milo Casagrande Readedd wrongly removed attribute. 2012-12-07 Milo Casagrande Fixed last tests not passing. 2012-12-07 Milo Casagrande Fixed copyright year, fixed indentation. 2012-12-07 Milo Casagrande Fixed almost all tests, 2 to go. 2012-12-07 Tushar Behera boards: origen: Remove duplicate assignments 2012-12-07 Tushar Behera Fix l-m-c issues after refactoring Samsung related code 2012-12-06 Milo Casagrande Added MagicMock import, fixed one test. 2012-12-06 Tushar Behera test: Fix testr errors 2012-12-06 Tushar Behera boards: samsung: Make SamsungConfig generic to be used for newer boards 2012-12-05 Milo Casagrande Refactored tests and fixed almost all. 2012-12-05 Milo Casagrande Added command line options to l-a-m-c. 2012-12-05 Milo Casagrande Refactored android_boards, from class to instance, tests to be fixed. 2012-12-05 Milo Casagrande From class to instace for boards, tests to be fixed. 2012-12-05 Milo Casagrande Refactored HardwarepackHandler class, moved under the hwpack package, fixed tests. 2012-12-05 Milo Casagrande Added new attributes. 2012-12-03 Riku Voipio allow promptless building of quantal images 2012-11-23 Stevan Radakovic Post-release version bump to 2012.11.1 2012-11-23 Stevan Radakovic Release 2012.11 2012-10-22 Milo Casagrande Fixed problem with logger: missing global variable. 2012-10-22 Milo Casagrande Re-applied common logging infrastructure. 2012-10-22 Fathi Boudra Post-release version bump to 2012.10.1 2012-10-22 Fathi Boudra Release 2012.10 2012-10-20 Fathi Boudra Revert the use of common logging infrastructure. 2012-10-18 Fathi Boudra Workaroun IOError in test case. 2012-10-17 Milo Casagrande Added docstrings. 2012-10-17 Milo Casagrande Fixed docstring and code indent. 2012-10-17 Milo Casagrande Added conversion of dtb_file into dtb_files for hwpack converter. 2012-10-17 Milo Casagrande Refactored method to make it more readable. 2012-10-17 Paul Sokolovsky Fix pep8 "E221 multiple spaces before operator" 2012-10-17 Paul Sokolovsky Update tests for actual Snowball values for fdt_high/initrd_high. 2012-10-16 Ricardo Salveti de Araujo linaro-hwpack-replace: exporting the updated manifest file 2012-10-16 Paul Sokolovsky Set actual values of fdt_high & initrd_high for Snowball, per Mathieu Poirier. 2012-10-16 Paul Sokolovsky Update tests for overridable values for 'initrd_high', 'fdt_high'. 2012-10-12 Paul Sokolovsky Allow to override initrd_high and fdt_high uboot params. 2012-10-12 Fathi Boudra Skip symlink-ed or invalid debian package file. (LP: #1065826) 2012-10-11 Fathi Boudra Build-Info support: * manifest file isn't license protected (open). * create an open BUILD-INFO.txt when no protected files are available. 2012-10-09 Ricardo Salveti de Araujo FastModel: also copying the boot wrapper for the foundation model 2012-10-04 fboudra@gmail.com <> PEP8 fixes. 2012-10-04 Milo Casagrande Fixed logger calls. 2012-10-04 Milo Casagrande Added debug option to linaro-media-create. 2012-10-04 Milo Casagrande Use module name for logger. 2012-10-04 Milo Casagrande Use default logger name. 2012-10-04 Milo Casagrande Use new logger method. 2012-10-04 Milo Casagrande Use new logger method. 2012-10-04 Milo Casagrande Fixed new logger usage. 2012-10-04 Milo Casagrande Fixed new logger usage. 2012-10-04 Milo Casagrande Added new logger method, fixed logger usage. 2012-10-04 Ricardo Salveti de Araujo Extract kpkgs should not depend on qemu 2012-10-03 fboudra@gmail.com <> Add some comments and build_info_available flag. 2012-10-03 fboudra@gmail.com <> Add Build-Info support 2012-10-02 Milo Casagrande Removed unnecessary comment. 2012-10-02 Milo Casagrande Fixed default bootloader. 2012-10-01 Milo Casagrande Added default bootloader to option help. 2012-10-01 Milo Casagrande Fixed typo. 2012-10-01 Milo Casagrande Moved logger calls earlier to have it ready. 2012-10-01 Milo Casagrande Default to u_boot when no bootloader is specified. 2012-10-01 Milo Casagrande Fixed tests for dtb_files support. 2012-09-28 Milo Casagrande Fixed syntax for dtb_files field. 2012-09-26 Milo Casagrande Moved coying of dtb_files when temp dir is still there. 2012-09-26 Milo Casagrande Post-release version bump to 2012.09-1.1 2012-09-26 Milo Casagrande Release 2012.09-1 2012-09-25 Milo Casagrande Added dtb_files support in media-create. 2012-09-25 Milo Casagrande Added dtb_files support in hwpack config. 2012-09-24 Milo Casagrande Added tests for Samsung fields. 2012-09-24 Milo Casagrande Fixed typo in v3 Samsung fields definition. 2012-09-20 Georgy Redkozubov Post-release version bump to 2012.09.1 2012-09-20 Georgy Redkozubov Release 2012.09 2012-09-20 Patrik Ryd Adapted tests for Snowball Android for device tree additions 2012-09-20 Patrik Ryd Added device tree support for AndroidSnowballEmmcConfig 2012-09-17 Patrik Ryd android_boards: Changed name of snowball dtb file to board.dtb 2012-09-14 Patrik Ryd DT support for Android Snowball 2012-09-12 James Tunnicliffe Fix V2 style directory layouts with a V3 config 2012-09-11 James Tunnicliffe Removed unused variable EXTRACT_FILES and associated imports. Renamed call_for_all_boards_and_bootloaders to foreach_boards_and_bootloaders. Fixed bug in foreach_boards_and_bootloaders (assumed board wasn't set). Made some code less ugly. Added more documentation to get_last_used_keys. Restored part of test (had been removed during refactoring but became relevant again). 2012-09-10 James Tunnicliffe Fixed up so linaro-media-create, linaro-hwpack-create and unit tests all work. 2012-09-10 James Tunnicliffe Make sure that all packages referenced by a copy_files section are saved in the hardware pack. 2012-09-10 James Tunnicliffe Updated copy_file code to make missing directories and test rename file support. 2012-09-10 James Tunnicliffe Refactored copy_files support. * Don't extract files into hwpack (leave in package) * Extract files to target path. 2012-09-05 James Tunnicliffe Fix incorrect variable name in get_file (bug introduced when refactoring code) 2012-09-04 Paul Sokolovsky Remove TODO comment based on the review. 2012-09-04 Paul Sokolovsky For consistency, use isinstance() instead of classy type() idiom. 2012-09-04 Paul Sokolovsky Add test for proper copy_files fields processing while populating boot part. 2012-09-04 Paul Sokolovsky Add test for HardwarepackHandler.get_file() for v3 fields. 2012-09-04 Paul Sokolovsky Implement 'copy_files' fields support in linaro-media-create. 2012-09-04 James Tunnicliffe Made test clearer. 2012-09-03 James Tunnicliffe Added refactored test for copy_files support in hardware pack create. 2012-09-03 James Tunnicliffe Removed new test files 2012-09-03 Riku Voipio fix fastmodel image build 2012-08-30 James Tunnicliffe Fix for multiple bootloaders in a hardware pack configuration (added missing comment) 2012-08-29 James Tunnicliffe * Added test for creating a hardware pack from V3 metadata containing more than one bootloader * Worked around the logic in find_command that assumes that if you are running from a checkout, file paths will be relative. This isn't true when running in the PyCharm debugger. * Tidied up IsHardwarePack and gave it the option to use a different format string to match in the FORMAT file. 2012-08-29 Paul Sokolovsky Remove _set_new_values(), fully replaced by extract_files(). 2012-08-29 Paul Sokolovsky Add initial integration tests for linaro-hwpack-create tool. 2012-08-29 Paul Sokolovsky Add corpus of mock Debian packages to use in tests. 2012-08-29 Paul Sokolovsky pep8 fix (2 blank lines). 2012-08-29 Paul Sokolovsky Wrap long lines to make pep8 happy. 2012-08-29 Paul Sokolovsky Revamp deletion of boot packages not included in rootfs. 2012-08-29 Paul Sokolovsky Allow to override hwpack output filename, mostly for tests. 2012-08-29 Paul Sokolovsky Make sure that PackageUnpacker uses separate dir for each package. 2012-08-29 Paul Sokolovsky Make sure that empty "sources" section is supported. 2012-08-28 James Tunnicliffe Test for selecting parameters from multiple bootloaders. Fixed bug where if a bootloader wasn't specified, linaro-hwpack-create would fail. 2012-08-28 Paul Sokolovsky Suffix bootloader path with board instead of prefix it, to make it sort nice. 2012-08-28 Paul Sokolovsky Elaborate global vs board-specific bootloader handling. 2012-08-28 Paul Sokolovsky Reimplement extraction of bootloader-related files, with support for copy_files. 2012-08-28 Paul Sokolovsky Using ppa:linaro-maintainers/tools is still required on 12.04. 2012-08-28 Paul Sokolovsky Wrap in 79 chars. 2012-08-28 Paul Sokolovsky Add basic definition and property accessor for "copy_files" field. 2012-08-27 Paul Sokolovsky Elaborate test dependencies in README. 2012-08-26 Ricardo Salveti de Araujo boards.py: checking for argument value before trying to use it 2012-08-23 Stevan Radakovic Post-release version bump to 2012.08.1 2012-08-23 Stevan Radakovic Release 2012.08 2012-08-17 Ricardo Salveti de Araujo linaro-hwpack-install: avoid failing in case depmod also fails 2012-08-09 Ricardo Salveti de Araujo linaro-media-create: fixing os.path.join call 2012-08-09 James Tunnicliffe Fixed PEP8 and minor review quibble 2012-08-05 Ricardo Salveti de Araujo temporarily_overwrite_file_on_dir: don't fail in case the rootfs files doesn't exist 2012-08-05 Ricardo Salveti de Araujo Do not force initrd at the boot cmds in case initrd is not available 2012-08-05 Ricardo Salveti de Araujo Allow booting without initrd, by using root dev path instead of always using UUID 2012-08-05 Ricardo Salveti de Araujo Allow lmc to just extract the kernel pkgs in case it's not a debian/ubuntu based rootfs 2012-08-05 Ricardo Salveti de Araujo linaro-hwpack-install: refactoring code and adding support for --extract-kernel-only 2012-07-31 James Tunnicliffe Moved hwpack_v3_layout structure to hwpack_fields.py 2012-07-30 James Tunnicliffe Add key validation for V3 metadata files. 2012-07-26 James Tunnicliffe Last of the uboot -> bootloader changes 2012-07-26 James Tunnicliffe More uboot -> bootloader changes 2012-07-26 James Tunnicliffe renamed uboot_dd to bootloader_dd 2012-07-26 James Tunnicliffe renamed uboot_flavor to bootloader_flavor 2012-07-26 James Tunnicliffe More u_boot -> bootloader renaming (3) 2012-07-26 James Tunnicliffe More u_boot -> bootloader renaming (3) 2012-07-26 James Tunnicliffe More u_boot -> bootloader renaming (2) 2012-07-26 James Tunnicliffe More u_boot -> bootloader renaming 2012-07-26 James Tunnicliffe renamed u_boot_file to bootloader_file 2012-07-26 James Tunnicliffe renamed u_boot_file to bootloader_file 2012-07-26 James Tunnicliffe renamed u_boot_package to bootloader_package 2012-07-26 Milo Casagrande Added tests for multiple bootlaoders and boards. 2012-07-26 Milo Casagrande Removed comments. 2012-07-26 Milo Casagrande Fixed args name. 2012-07-26 Milo Casagrande Fixed name for --dev option. 2012-07-25 Milo Casagrande Fixed assertEqual tests. 2012-07-25 Milo Casagrande Fixed annoying pyflakes warnings. 2012-07-25 Milo Casagrande Fixed tests and imports. 2012-07-25 Milo Casagrande Fixed PEP8 warning. 2012-07-25 Milo Casagrande Added option check. 2012-07-25 James Tunnicliffe Wired up command line parameters that should allow board and bootloader selection. 2012-07-25 James Tunnicliffe renamed u_boot_in_boot_part to bootloader_file_in_boot_part. 2012-07-25 Milo Casagrande Added initial work for hwpack metadata probing. 2012-07-23 Milo Casagrande Post-release version bump to 2012.07.1 2012-07-23 Milo Casagrande Release 2012.07 2012-07-23 James Tunnicliffe Updated media create tests to cope with changes from last checkin. 2012-07-23 Milo Casagrande Added support for boards in metadata and Config. 2012-07-23 James Tunnicliffe Move parsing of infrormation in hardware pack metadata out of linaro-hwpack-install. 2012-07-23 Milo Casagrande Fixed _get_v3_option. 2012-07-23 Milo Casagrande Fixed method name. 2012-07-23 Milo Casagrande Renamed method. 2012-07-23 Milo Casagrande Fixed method name, typos, removed unnecessary comments. 2012-07-23 Milo Casagrande Renamed method and args. 2012-07-23 Milo Casagrande Removed unnecessary import. 2012-07-20 Milo Casagrande Added loop through multiple bootloaders. 2012-07-20 Milo Casagrande Fixed problem with bash script. 2012-07-20 James Tunnicliffe Tidied up config.py Deleted unused file hwpack_parser.py 2012-07-20 Milo Casagrande Used external file for fields, fixed fields for v3. 2012-07-19 Milo Casagrande Removed un-used parameter. 2012-07-19 Milo Casagrande Removed unecessary files. 2012-07-19 James Tunnicliffe V2 config reader should return None when the looked up result evaluates to False. 2012-07-19 James Tunnicliffe Added logic to allow Config to read V2 metadata files. 2012-07-19 Milo Casagrande Fixed indentation, remove unused import. 2012-07-19 Milo Casagrande Added license header, fixed boolean values. 2012-07-19 Milo Casagrande Fixed typo in dictionary. 2012-07-19 James Tunnicliffe V2 fix: Now using the generic parser, need to patch up for V2 files that don't start with [hwpack]. 2012-07-19 Milo Casagrande Fixed converter and metadata. 2012-07-19 James Tunnicliffe Removed debug prints Fixed a couple more incorrect strings for new lookup function. 2012-07-19 Milo Casagrande Used yaml.dump, cleaned up code and tests. 2012-07-19 Milo Casagrande Fixed converter and metadata creation. 2012-07-19 Milo Casagrande Added hwpack converter from INI to YAML. 2012-07-18 James Tunnicliffe Bug fix to creation using YAML config. 2012-07-18 James Tunnicliffe Added YAML support to linaro-hwpack-install 2012-07-18 Milo Casagrande Added more tests for metadata v3. 2012-07-18 Milo Casagrande Fixed tests. 2012-07-18 Milo Casagrande Added back old metadata format, fixed tests. 2012-07-18 Milo Casagrande Fixed metadata tests. 2012-07-18 Milo Casagrande Create YAML metadata file, fixed coverter. 2012-07-17 James Tunnicliffe linaro-hwpack-install accepts V3 input. Auto-detect default bootloader. more config.py bool handling fixes. boards.py: Changed a decision from "is v2" to "isn't v1" to make it work with v3. 2012-07-17 Mathieu Poirier 400M for vmalloc is still not enough, bumping to 500M. (LP: #1014650) 2012-07-17 Milo Casagrande Added support for bootloaders conversion. 2012-07-17 Milo Casagrande Created functions to create YAML dump. 2012-07-12 James Tunnicliffe Final few v3 test fixes. Partition layout check now generates the correct error message. Other tests that were calling the wrong function were patched up. One test deleted (it didn't test anything useful). 2012-07-12 James Tunnicliffe Updated source validation logic. This used to be called section validation logic, but the sections were only used as sources and thus the name was confusing. 2012-07-12 James Tunnicliffe Cleaned up _get_option_bool. Removed some unncessary configuration from tests. Clearned up tests that weren't calling a specific validation function. 2012-07-12 James Tunnicliffe Moved address validation into a common function to reduce duplication. 2012-07-12 James Tunnicliffe Fixed board specific options. Fixed V3 sources. 2012-07-12 James Tunnicliffe Added dictionary to translate v2 metadata parameters to v3. Started to make error messages more universal by generating them in functions. Added formatter function to generate "address" strings (v2 required these to be 8 character hex with leading 0x - sticking with the same logic for now, but isn't suitable for 64 bit). Added _get_bootloader_option to intelegently handle getting bootloader options depending on the currently chosen bootloader (set_bootloader() or __init__ parameter). Bools always translated to yes/no string. 2012-07-11 James Tunnicliffe Think I have got all the simple YAML dictionary lookups sorted 2012-07-11 Milo Casagrande Fixes to test regressions. 2012-07-11 Milo Casagrande Progress on v3 tests fixing. 2012-07-11 Milo Casagrande Fixed regressions on tests. 2012-07-11 Milo Casagrande Added get boolean function. 2012-07-11 James Tunnicliffe Replaced a few calls to self.parser.get(self.MAIN_SECTION with self._get_option() to improve v3 compatibility 2012-07-11 James Tunnicliffe Use format_as_string to check metadata format 2012-07-11 James Tunnicliffe Set default format for v3 tests to 3.0 2012-07-11 James Tunnicliffe v3 test_architectures now works 2012-07-11 James Tunnicliffe Changed _get_option name to reflect differences in v2 and v3 metadata formats 2012-07-11 Milo Casagrande Fixed problems on the hwpack parser abstraction. 2012-07-10 James Tunnicliffe Adding Tests for v3 metadata 2012-07-10 Milo Casagrande Added an initiali abstraction on the config parser. 2012-07-10 Milo Casagrande Added hwpack v3 format. 2012-07-10 Milo Casagrande Removed not needed value. 2012-07-09 Milo Casagrande Added test for True-False values. 2012-07-09 Milo Casagrande Added list creation for architectures. 2012-07-09 Milo Casagrande Added test for architectures block parsing. 2012-07-09 Milo Casagrande Added first version of the converstion tool. 2012-07-05 Emanuele Aina Fix duplicates detection on missing sources.list.d entries 2012-07-04 Mathieu Poirier Increasing vmalloc pool to accommodate ioremap'ed devices and hwmem allocations. Fixing LP1014650 2012-06-21 Deepti B. Kalakeri Post-release version bump to 2012.06.1 2012-06-21 Deepti B. Kalakeri Release 2012.06 2012-06-21 Vishal Bhoj Add device tree support for panda 2012-06-20 Deepti B. Kalakeri Modify the README to include the correct python-testools dependency information 2012-06-13 Danilo Segan Update the test to not ignore fixed PEP8 problems. 2012-06-13 Danilo Segan More PEP8 fixes on new stuff. 2012-06-13 Danilo Segan Fix remaining PEP8 problems. 2012-06-13 Danilo Segan Fix line-too-longs in hardwarepack.py. 2012-06-13 Danilo Segan Fix line-too-longs in android_boards.py. 2012-06-13 Danilo Segan Fix line-too-longs in boards.py. 2012-06-13 Danilo Segan Line-too-long fixes for test_config.py. 2012-06-13 Danilo Segan Remove trailing whitespace. 2012-06-13 Danilo Segan Continue with PEP-8 fixes. 2012-06-13 Danilo Segan Start with PEP-8 fixes. 2012-06-13 Milo Casagrande Removed round function. 2012-06-13 Milo Casagrande Fixed proposed review changes. 2012-06-12 Milo Casagrande Fixed unit tests, added new unit test. 2012-06-12 Milo Casagrande Fixed bytes rounding to a round-up. 2012-06-11 Milo Casagrande Removed thrown exception, program exits instead. 2012-06-11 Milo Casagrande User friendly messages. 2012-06-11 Milo Casagrande Reenabled stdout and stderr on apt-get install. 2012-06-08 Loïc Minier Also silence E202 for natty users with a comment that this can be re-enabled after natty support is dropped. 2012-06-07 Loïc Minier Document leftover warnings/errors. 2012-06-07 Loïc Minier PEP8: Fix W293 blank line contains whitespace and W391 blank line at end of file. 2012-06-07 Loïc Minier PEP8: Already fixed W292 no newline at end of file. 2012-06-07 Loïc Minier PEP8: Fix E303 too many blank lines (2). 2012-06-07 Loïc Minier PEP8: Fix E302 expected 2 blank lines, found 1. 2012-06-07 Loïc Minier PEP8: Fix E301 expected 1 blank line, found 0. 2012-06-07 Loïc Minier PEP8: Fix E251 no spaces around keyword / parameter equals. 2012-06-07 Loïc Minier PEP8: Fix E241 multiple spaces after ','. 2012-06-07 Loïc Minier PEP8: Fix missing whitespace after ','. 2012-06-07 Loïc Minier PEP8: Fix E225 missing whitespace around operator. 2012-06-07 Loïc Minier PEP8: Fix E222 multiple spaces after operator. 2012-06-07 Loïc Minier PEP8: Fix E201 whitespace after '[' (or after '{'). 2012-06-07 Loïc Minier Add PEP8 compliance test, ignoring all current warnings and errors for now. 2012-06-07 Milo Casagrande Fixed test regression. 2012-06-07 Milo Casagrande Renamed test class: conflicting with another one 2012-06-07 Milo Casagrande First step into fixing the mock object. 2012-06-06 Milo Casagrande Applyed review changes, fixed PEP8 warnings 2012-06-06 Milo Casagrande Applied PEP8 changes. 2012-06-06 Milo Casagrande Fixed test case with new method needing output. 2012-06-06 Milo Casagrande First fix to the test case. 2012-06-05 Milo Casagrande Fixed image size calculation to a multiple of 1MiB. 2012-05-31 Milo Casagrande Added a new test for additional hwpack checks. 2012-05-31 Milo Casagrande Fixed import sorting in alphabetical order. 2012-05-30 Milo Casagrande Fixed typos and test description. 2012-05-30 Milo Casagrande Applied diff from kiko, and modified it. 2012-05-30 Bernhard Rosenkraenzer Don't require qemu-arm-static for linaro-android-media-create 2012-05-28 Milo Casagrande Fixed error: hwpacks is a list. 2012-05-28 Milo Casagrande Remove tmp directory at the end of test. 2012-05-28 Milo Casagrande Added hwpack argument checking. 2012-05-28 Milo Casagrande Added test for hwpack argument 2012-05-25 Fathi Boudra Post-release version bump to 2012.05.1 2012-05-25 Fathi Boudra Release 2012.05 2012-05-17 Riku Voipio FastModel: Make ancillary files optional 2012-05-16 Mathieu Poirier Removing miscellaneous comments as per reviewer's request. https://blueprints.launchpad.net/u-boot-linaro/+spec/snowball-u-boot-linaro-12.02 2012-05-15 Mathieu Poirier Using default value for u-boot-linaro-stable 2012-05-10 Michael Hope Support floating point values such as '1.9G' for --image-size. Add tests. 2012-05-07 Michael Hope Change ownership when copying files root only to the output directory. 2012-04-29 Fathi Boudra Hide password for private PPA access: * strip passwords from apt sources in the generated hwpack * obfuscate passwords leaked by ConfigParser and FetchFailed exceptions 2012-04-23 Luis Araujo Add hwpack version 2 support to the i386 board Add 'pae' kernel to the i386 kernel flavors list 2012-04-20 James Tunnicliffe Post-release version bump to 2012.04.1 2012-04-20 James Tunnicliffe Release 2012.04 2012-04-20 James Tunnicliffe Fixed failing tests due to last merge 2012-04-19 Ricardo Salveti de Araujo hwpacks: avoid printing the repo private key in the log 2012-04-18 Ricardo Salveti de Araujo Adding mmc_device_id to be parsed from mmc_id and be used by flash-kernel 2012-04-18 James Tunnicliffe Removed unused directory variable. 2012-04-18 James Tunnicliffe remove .idea ignore 2012-04-18 James Tunnicliffe Moved additional option checks into their own function (additional_option_checks). 2012-04-18 James Tunnicliffe Moved defaulting of args.device into ArgumentParser set up. 2012-04-18 James Tunnicliffe Renamed --directory option to --output-directory. Set default file name for an image to sd.img 2012-04-18 James Tunnicliffe Fixed spelling of IncompatibleOptions 2012-04-17 Abner Silva Remove vmlinuz and initrd definitions from I386 class. 2012-04-17 James Tunnicliffe Removed outputs_directory flag 2012-04-17 James Tunnicliffe Adding .idea projects to ignore 2012-04-17 James Tunnicliffe --directory can now be specified at the same time as --image-file. If --directory and --image-file are combined, a file, whose name is specified by --image-file will be created in --directory. If --image-file is not specified, a sensible default file name will be used and printed for the user to see. 2012-04-16 James Tunnicliffe Added test for new board config handling 2012-04-16 James Tunnicliffe Patch from https://bugs.launchpad.net/linaro-image-tools/+bug/962147 2012-04-13 Luis Araujo Add a label for booting instead of a root partition. Show output to console too. 2012-04-12 Fathi Boudra Allow FORCE_OPTIONS to be present when APT tries to update its cache: linaro-hwpack-install stops and asks for user interaction when trying to update APT cache from unsigned repositories. (LP: #979178) 2012-04-03 Abner Silva Add i386 support to LMC. Creates a i386 device that allows LMC to create x86 images. The bootloader being used is GRUB2. 2012-03-23 James Tunnicliffe Post-release version bump to 2012.03.1 2012-03-23 James Tunnicliffe Release 2012.03 2012-03-23 James Tunnicliffe Patched up failing test for 2012.03 release. Removed linaro-fetch-image from setup.py 2012-03-23 Fathi Boudra Some clean up from merge request feedback. 2012-03-22 Fathi Boudra Replace and then restore start-stop-daemon and initctl (LP: #98604): Upstart jobs and some init script don't honor policy-rc.d. 2012-03-21 Jon Medhurst (Tixy) Remove 'a9' from VExpress Android board name 2012-03-21 Fathi Boudra Prevent daemons to start in the chroot (LP: #912782) 2012-03-06 Mathieu Poirier snowball: pusing vmalloc size to 300M 2012-03-02 James Tunnicliffe Fix: Removed linaro-fetch-image tests from test list in __init__.py 2012-03-02 James Tunnicliffe Removed linaro-fetch-image related files. 2012-02-21 Mattias Backman Post-release version bump to 2012.02.1 2012-02-21 Mattias Backman Release 2012.02 2012-02-21 Mattias Backman Fix test so that a startupfiles dir is not needed. 2012-02-21 Mattias Backman Change delete_startupfiles to be a classproperty. 2012-02-20 Guilherme Salgado Add a couple tests for AndroidMx6QSabreliteConfig 2012-02-20 Mattias Backman Undo line break and remove new arguments from another call. 2012-02-20 Mattias Backman Revert the call from linaro-android-media-create. 2012-02-20 Mattias Backman Remove the new arguments to populate_raw_partition and get them from a class function instead. Add tests for populate_raw_partition. 2012-02-19 Bernhard Rosenkraenzer Add comment on dtb_name path 2012-02-19 Bernhard Rosenkraenzer Whitespace adjustment 2012-02-19 Bernhard Rosenkraenzer Make sure dtb_name is None for all non-mx6qsabrelite Android builds, the way it used to be 2012-02-19 Bernhard Rosenkraenzer Adjust whitespace to match the rest of the code 2012-02-17 Mattias Backman Start fixing review comments. 2012-02-16 Mattias Backman Add an error message for when the user forgot to supply the startupfiles. 2012-02-16 Mattias Backman Look for the startupfiles in configdir and don't assume that they are in the boot partition. 2012-02-16 Mattias Backman Add separate source dir for startfiles so we can pass in a different dir from l-m-c and l-a-m-c. Add flag for deleting the startupfiles on disk. 2012-02-16 Mattias Backman Switch arguments to Snowball populate_raw_partitions() so that it matches the parent function signature. 2012-02-15 Bernhard Rosenkraenzer Pass DeviceTree file correctly 2012-02-12 Bernhard Rosenkraenzer Add support for Android on mx6qsabrelite 2012-02-09 Mattias Backman The u-boot-env.bin will be in the ./boot folder, not in the boot tarball. 2012-02-09 Mattias Backman Also assume u-boot-env.bin is in the boot tarball. 2012-02-08 Mattias Backman A hack for copying the u-boot.bin file from boot.tar.bz2 for Snowball. It expects the user to have put the startfiles in ./boot . 2012-02-08 Mattias Backman Update tests for Snowball Android configs. 2012-02-08 Mathieu Poirier snowball: android: Increasing mali memory to 64MB. 2012-02-07 Mattias Backman Deprecate the hwpack V1 format. 2012-02-01 Mattias Backman Benn Porscke's fix for Snowball Android emmc boot. 2012-01-25 Fathi Boudra Update fetch_image_settings.yaml to pick up 12.01 release. 2012-01-24 Guilherme Salgado Make sure the new --extra-boot-args argument works when the extra_boot_args_options attribute of the board config is set to None 2012-01-19 Mattias Backman Post-release version bump to 2012.01.1 2012-01-19 Mattias Backman Release 2012.01 2012-01-19 Mattias Backman Add the new code to l-a-m-c as well. 2012-01-18 Mattias Backman Call add_boot_args() from add_boot_args_from_file(). 2012-01-17 Mattias Backman Add reading boot args from file. 2012-01-16 Mattias Backman Add command line option to add to boot args. 2012-01-16 Mattias Backman Remove assignment to unused variable, just to fix the test suite. 2012-01-16 Mattias Backman Add add_boot_args() to BoardConfig. 2011-12-26 Georgy Redkozubov Adjust error output to use with logger 2011-12-23 Georgy Redkozubov Fixes pep8 style 2011-12-22 Georgy Redkozubov Added tests for suggested fix. 2011-12-22 Georgy Redkozubov Removed unnecessary print 2011-12-22 Georgy Redkozubov Added tests for fix 2011-12-21 Jon Medhurst (Tixy) Make Android sdcard partition always FAT32. 2011-12-19 Mattias Backman Add tests for Android mx5 board config. 2011-12-19 James Tunnicliffe Updated fetch_image_settings.yaml to pick up 11.12 release 2011-12-19 Fathi Boudra Remove console=tty0 on mx53loco bootargs. (Closes: LP #900460 2011-12-15 Danilo Segan Post-release version bump to 2011.12.1 2011-12-15 Danilo Segan Release 2011.12 2011-12-15 Danilo Segan Drop console=tty0 from PandaBoard AndroidConfig. 2011-12-15 Danilo Segan Make the test for panda-android not include console=tty0 option. 2011-12-14 Eric Miao Add support for Freescale i.MX6Q SabreLite board 2011-12-13 James Tunnicliffe Padding for grid2 is now calculated using its size (no fixed guess about how big it is in pixels). 2011-12-13 Georgy Redkozubov Proposed fix. Wait in loop for disk being freed and run sfdisk after that. 2011-12-13 Georgy Redkozubov Moved fix for bug 718665 to it's own branch since issue couldn't be reproduced and as a result verified easily. 2011-12-13 Georgy Redkozubov Incorporated fix from Guilhermes branch lp:~salgado/linaro-image-tools/717129 2011-12-12 James Tunnicliffe Having tried many update, refresh and resize calls, forcing the size of the collapsable pane on page 2 (OS selection) seems to be the only way to fix bug #832327. 2011-12-08 Georgy Redkozubov Added loop for wating partitions got created by the udev in function _get_device_file_for_partition_number. Added loop to wait for partition to settle after parted in function create_partitions. 2011-12-06 Georgy Redkozubov Added warning message to the user about unsetting TMPDIR variable. In chroot environment it likely doesn't exist and can lead to error messages 2011-12-05 Guilherme Salgado Increase the size of the system partition for android images from 256MB to 512MB 2011-12-02 Georgy Redkozubov Unset users TMPDIR variable 2011-11-29 Georgy Redkozubov Added python-yaml to the dependency list 2011-11-28 Mattias Backman Add license and origin information for try_import. 2011-11-28 Mattias Backman Steal try_import from testtools to avoid having testtools as a production dependency. 2011-11-23 Jon Medhurst (Tixy) Fix bad addresses in vexpress U-Boot script 2011-11-21 Guilherme Salgado Change the README to list python-testtools as a runtime dep and apt-utils as a test-suite dep 2011-11-21 Guilherme Salgado Add the 11.11 release to fetch_image_settings.yaml 2011-11-21 Guilherme Salgado Use testtools.try_import for CommandNotFound; that way our pyflakes test can expect a clean output 2011-11-17 Guilherme Salgado Change a test to not hard-code a value 2011-11-17 Danilo Segan Post-release version bump to 2011.11.1 2011-11-17 Danilo Segan Release 2011.11 2011-11-17 Mattias Backman Handle whitespace in Snowball startfiles config file. 2011-11-16 Guilherme Salgado In l-m-c, stop before trying to install hwpacks if we detect that we can't execute anything in the chroot 2011-11-16 Jon Medhurst (Tixy) Add vexpress-a9 test case for android. 2011-11-16 Jon Medhurst (Tixy) Add vexpress-a9 platform to linaro-android-media-create 2011-11-14 Mattias Backman Clean up boot args and fix unit test to run with the changes. Also add one test for SnowballSdConfig that was missing. 2011-11-14 Mattias Backman Stop test suite from moving files using sudo. 2011-11-14 Guilherme Salgado Add a workaround for bug 885895 2011-11-09 Bernhard Rosenkraenzer Fix snowball_emmc support as well (not just snowball_sd) 2011-11-09 Bernhard Rosenkraenzer More adaptions to snowball's "unique" uboot 2011-11-09 Bernhard Rosenkraenzer Fix syntax error 2011-11-09 Bernhard Rosenkraenzer Adapt to Android on Snowball using a custom uboot that takes "fat load" instead of "fatload" 2011-11-09 Bernhard Rosenkraenzer Attempt to fix boot args for Android on Snowball SD 2011-11-03 Leonid Suggested fix NOT verified 2011-10-28 James Tunnicliffe Restore global xdg import. 2011-10-27 James Tunnicliffe Updated paths to match the server, not local development. 2011-10-27 James Tunnicliffe Updated settings to give more human names to releases. 2011-10-27 Mattias Backman Clarify an assert, enable logging and experiment with getting both old and new structures. 2011-10-26 Mattias Backman Remove code that was commented out. 2011-10-24 Andy Doan S5VP310 support has been dropped to focus on Origen 2011-10-20 James Tunnicliffe Updated to match new lmc command line params. 2011-10-20 Mattias Backman Post-release version bump to 2011.10.1 2011-10-20 Mattias Backman Release 2011.10 2011-10-19 Mattias Backman Fix a comment typo. 2011-10-19 Mattias Backman Add tests. 2011-10-19 Mattias Backman Add one entry in rootfs /etc/network/interfaces for each interface specified in the hwpack to bring them up automatically with dhcp. 2011-10-19 Mattias Backman Add qemu-arm-static version to l-m-c and l-a-m-c --version output. 2011-10-13 Mattias Backman Don't use Tarfile as a context manager. 2011-10-13 Deepti B. Kalakeri Addressing the review comments for the hwpack name changes 2011-10-13 Deepti B. Kalakeri Adding an optional parameter to specify the build number to be used in new hwpack filename. 2011-10-12 Deepti B. Kalakeri Changing the timestamp on the hwpack to use utc time than the local time and fixing the tempdir cleanup 2011-10-12 Mattias Backman Swap places for the dash and underscore option aliases so that the ones with dashes are displayed first in the help. 2011-10-12 Mattias Backman Add aliases for command line options using dashes instead of underscores. 2011-10-12 Mattias Backman Add path_in_tarfile_exists() and use it to detect new Live image format. 2011-10-11 Paul Larson * Add a -i, --in-place option to modify the specified hwpack rather than making a new one * Cleanup the tempdir 2011-10-06 Mattias Backman Just remove the package if no replacement package passed to linaro-hwpack-replace. 2011-10-05 James Tunnicliffe Enable 11.09 release in linaro-fetch-image 2011-10-04 Mattias Backman Don't execute set_appropriate_tty for V2 hwpacks, since tty will be set explicitly. 2011-10-04 Mattias Backman Change version back to numeric only. 2011-10-04 Mattias Backman Move seds to function. 2011-10-03 Andy Doan l-a-m-c is no longer "experimental" 2011-10-03 Mattias Backman Change setup.py to get version from __version__.py 2011-10-03 Mattias Backman Change do-release to update version in __version__.py 2011-10-03 Mattias Backman Enable -v and get version from __version__.py 2011-09-30 Mattias Backman Restore iMX53 board option and add a deprecation warning. 2011-09-27 Mattias Backman Make boot_script field optional. 2011-09-23 Mattias Backman Post-release version bump to 2011.09.1 2011-09-23 Mattias Backman Release 2011.09 2011-09-22 Mattias Backman Pay attention to the --dev option when using Snowball hwpacks, which can be used for sd (two partitions) and emmc (three partitions with loader population). 2011-09-22 Mattias Backman Change check that a flashable env should be created and dd:d. 2011-09-21 Mattias Backman Set the spl properties from hwpack metadata. 2011-09-21 Mattias Backman Add spl_x properties to BoardConfig. 2011-09-21 Mattias Backman Make presence of spl_file and uboot_file indicate that these files should be put in the hwpack. 2011-09-21 Mattias Backman Correct whitespace. 2011-09-21 Mattias Backman Salvage some more forgotten Samsung code. 2011-09-21 Mattias Backman Create flashable env if this is a Samsung hwpack. 2011-09-21 Mattias Backman Change x-loader to spl in hwpack metadata. 2011-09-21 Mattias Backman Change x-loader to spl in hwpack config. 2011-09-21 Mattias Backman Check for spl file instead of x_loader in install_omap_boot_loader. 2011-09-21 Mattias Backman Get metata fields uboot_dd and spl_dd. 2011-09-21 Mattias Backman Use spl file as Omap x-loader and dd spl file if offset is specified in spl_dd. This removes the need for the Samsung specific code. 2011-09-20 Mattias Backman Change format of mmc_id to match mmc_option '0:1' and calculate mmc_part_offset from that. 2011-09-20 Mattias Backman Remove test for removing found packages. 2011-09-20 Mattias Backman Defer removing packages from hwpack list until they have been used. 2011-09-15 Mattias Backman Set up logging, seems to be needed to enable warn logging in partitions.py. 2011-09-15 Mattias Backman Use the same Samsung bootloader code for both v1 and v2 code paths. 2011-09-15 Mattias Backman Save hwpack_format so we don't have to iterate over the hwpack tarballs every time we need the format version. 2011-09-15 Mattias Backman Fix the broken get_format() fix. 2011-09-15 Mattias Backman Fix glob pattern to accept leading * and fix some pep8 problems. 2011-09-14 Mattias Backman Extract dd_file method to assert that the file exists, that it does not exceed a maximum size and dd it to the target. 2011-09-14 Mattias Backman Change kernel files glob to be relative to chroot instead of chroot/boot/ and fix get_format() bug. 2011-09-13 Mattias Backman Add uboot_dd field to hwpack metadata to be used to request that the uboot file is dd:d to the target. 2011-09-13 Mattias Backman Consolidate _make_boot_files() methods into _make_boot_files_v2() with conditionals. 2011-09-13 Mattias Backman Assert that format is 1.0 if partition_layout is not specified. 2011-09-12 Mattias Backman Assert that the uboot bin file is found instead of asserting a certain set of hwpack v1 fields. 2011-09-06 Andy Doan add ability to provide hwpack descriptions to the user 2011-08-30 Angus Ainslie Add the new spl name for Origen in upsream u-boot 2011-08-30 Mattias Backman Change --dev value for imx53 to mx53loco to be consistent with l-m-c. 2011-08-29 Mattias Backman Fix extra empty line. 2011-08-29 Mattias Backman Move dd(uboot_file) into hardwarepackhandler context. 2011-08-29 Mattias Backman Move Samsung specifics to BoardConfig. 2011-08-26 Mattias Backman Add samsung fields to hwpack metadata. 2011-08-26 Mattias Backman Add Samsung hwpack v2 fields to l-m-c. 2011-08-26 Mattias Backman Add Samsung BL1, BL2 and ENV fields to hwpack config. 2011-08-26 Mattias Backman Accept spl_file config field and have hwpack-create put that file in the hwpack. 2011-08-26 Mattias Backman Get spl_file from hwpack if it's in there. 2011-08-26 Mattias Backman Move Samsung constants from module scope to SamsungConfig class properties. 2011-08-25 Mattias Backman Add load_start field needed by Snowball and make sure partition_layout field sticks to class scope. 2011-08-25 Mattias Backman Add tests for V2 get_sfdisk_cmd functions. 2011-08-25 Mattias Backman Create V2 get_x_sfdisk_cmd functions. 2011-08-25 Mattias Backman Use snowball_startup_files_config from metadata. 2011-08-24 Mattias Backman Make Samsung Android board configs inherit a common sfdisk_command function. Add a couple of unit tests. 2011-08-24 Mattias Backman Add debug printout. 2011-08-23 deeptik Fixing the new hwpack name to contain minutes info instead of seconds 2011-08-23 Mattias Backman Detect absolute path in startfiles.cfg instead of checking that the file exists. 2011-08-22 James Tunnicliffe (Hopefully) final update of UI settings for 11.08. 2011-08-22 Angus Ainslie Add origen supprt 2011-08-22 Angus Ainslie Zero the u-boot env on the SD card so an old one doesn't get used 2011-08-22 James Tunnicliffe Updated YAML for 11.08 release 2011-08-22 James Tunnicliffe Add a log file for linaro-media-create output. Single line change to fetch_image_settings.yaml - removing the now deprecated lt-mx53loco 2011-08-22 Mattias Backman Post-release version bump to 2011.08.1 2011-08-22 Mattias Backman Release 2011.08 2011-08-22 Mattias Backman Correct comment text. 2011-08-22 Mattias Backman Remove hacky extra check. Fix absolute path to be relative to the chroot and not system root. 2011-08-22 James Tunnicliffe Proper handling of the new way of telling the indexer to index a directory: 2011-08-22 Mattias Backman Add support for absolute paths in Snowball startfiles.cfg. Also fallback to a hardcoded path to support u-boot packages that do not link u-boot.bin to /boot. 2011-08-20 James Tunnicliffe * All hardware that is available is shown on page 1 * If you chose hardware that doesn't have a build type available (release or snapshot) that build type is greyed out on page 2 with some help text saying why. 2011-08-19 James Tunnicliffe Updating server indexer. Target is to get rid of all the special casing and make the indexer better at excluding/including files. Have given the URL verifier variable a new structure, so now it is a tuple of two lists. The first element of the tuple is a list of regexps that should match, the second is a list of regexps that shouldn't match. 2011-08-19 James Tunnicliffe Changed default file system to ext4. 2011-08-19 Mattias Backman Change --rootfs default to ext4. 2011-08-19 Mattias Backman Check for u-boot.img to be copied to the boot partition. If u-boot.img is not found, use u-boot.bin. 2011-08-18 James Tunnicliffe Added an OK device choser. Unfortunately most of the information that you can get over dbus (http://hal.freedesktop.org/docs/DeviceKit-disks/Device.html) is mostly useless. My guess is that hard disks are much better supported than finding out about an SD card hanging of a USB writer. 2011-08-18 James Tunnicliffe Trying to get a decent dev choser. Not working so far! 2011-08-18 James Tunnicliffe lmc_settings page updates: * Removed "are you sure" tick box * Put advanced settings in proper UI element 2011-08-18 James Tunnicliffe Fixes three problems shown by unit tests: * Assert always runs - changed to raise AssertionError * Updated test_pyflakes.py so the "redefinition of unused CommandNotFound" error is ignored again * Test test_verify_files now passes (NamedTemporaryFile mocked to be predicatable) 2011-08-18 Mattias Backman Accept globs in kernel file fields. 2011-08-18 Mattias Backman Restore a test that was lost along the way. 2011-08-18 Mattias Backman Fix type of uboot_in_boot_part in comment. 2011-08-18 Mattias Backman Add tests for metadata output. 2011-08-18 James Westby Up the default image size to 3G. 2011-08-18 Mattias Backman Add unit tests for more fields. 2011-08-18 Mattias Backman Add unit tests for more fields. 2011-08-18 James Tunnicliffe Added a couple of module level constants. 2011-08-18 Mattias Backman Make a couple of more fields non required and clean up tests. 2011-08-18 Mattias Backman Add test for kernel_file. 2011-08-17 James Tunnicliffe Added advanced options to stable/dev page. 2011-08-17 James Tunnicliffe If user has selected emulated system, don't let them write to a device (assumption is QEmu will run using an image file). 2011-08-17 James Tunnicliffe switch from using inspect to __file__ to determine the current file. While this is potentially less reliable, it is consistent with the rest of the code. 2011-08-17 Mattias Backman Allow reserved partition option for partition_layout. 2011-08-17 Mattias Backman Rename vmlinuz and initrd fields to kernel_file and initrd_file to be consistent with dtb_file, kernel_addr etc. 2011-08-17 Mattias Backman Rename u-boot_in_boot_part to use underscore as separator. 2011-08-17 Mattias Backman Rename u-boot-file and u-boot-package fields to use underscore like all the hwpack v2 fields. 2011-08-15 James Tunnicliffe GPG interaction now uses --status-file to give us language independent output. 2011-08-15 James Tunnicliffe Re-arranged first two pages according to new UI design. 2011-08-15 James Tunnicliffe Found another message not being passed as a tuple. 2011-08-15 James Tunnicliffe Bug fix: Was not passing a message correctly into the message queue correctly (not sent as tuple) linaro-fetch-image[-ui] copes with GPG signates failing because the GPG key is missing from the users keyring more sensibly. 2011-08-12 James Tunnicliffe Added back some required parts of the yaml that apparently hadn't got into bzr. 2011-08-12 James Tunnicliffe Undo previous change - it was broken. If an URL can not be pulled from the database we now log useful error messages, which are printed to the terminal by default. 2011-08-12 James Tunnicliffe Quick spell check for the release/snapshot CLI parameter. 2011-08-12 James Tunnicliffe Corrected Bug #824733 Added in some new hardware packs. Added note about need to support snowball, but need to investigate. 2011-08-11 James Tunnicliffe Fix possible problem with using a really old database for the test. 2011-08-11 James Tunnicliffe Tests for linaro-fetch-image now use local files, not server files. 2011-08-11 Deepti B. Kalakeri Adding a small comment and including the descrption abt hwpack removal into should_remove() and using the string to directly compare instead of string substitution 2011-08-10 James Westby Create /etc/apt/sources.list.d in IsolatedAptCache. Thanks to Rex Tsai. 2011-08-11 Deepti B. Kalakeri Addressing comments to include a small function (should_remove) to verify if package information needs to be removed from the Packages and manifest file. Also, use the same function to remove the debian packages matching the prefix of the package specified. 2011-08-05 Mattias Backman Make dtb_file not required. 2011-08-05 Mattias Backman Make x-loader hwpack fields not required. 2011-08-05 Mattias Backman Enable reserved_bootfs_rootfs partitioning option. 2011-08-05 Mattias Backman Add snowball_startfiles_config file to hwpack-create config. 2011-08-05 Mattias Backman Make the u-boot fields not required in hwpack metadata. 2011-08-05 Deepti B. Kalakeri Changes to make it work for non linaro hwpack instead of making it work for linaro hwpack only 2011-08-05 Deepti B. Kalakeri Removing un necessary backslash 2011-08-05 Deepti B. Kalakeri This patch addresses review comments with the following changes: 1) Improve the description of the --prefix-pkg-remove arguement of l-h-r 2) Rectified indentation problem and removed un necessary backslash 3) Removed un used debian_pack_found variable 4) Gave a comment on why we need to remove the hwpack-linaro-* info 2011-08-04 Mattias Backman Remove target parameter from get_metadata_field and clean up. 2011-08-04 Mattias Backman Add extra_serial_opts to hwpack metadata. 2011-08-04 Deepti B. Kalakeri This patch fixes the bug Bug #816767 [linaro-hwpack-replace only replace same name deb pkg but in fact it needs a smart match] It now replaces old deb (with a different name) package with a new hwpack, so that l-m-c can determine which package is to be installed in later hwpack installation. The changes adds the new option --prefix-pkg-remove to handle this where the user is needed to provide the prefix of the debian package that needs to be removed while including the new one. 2011-08-04 Mattias Backman Add boot_script and u-boot_in_boot_part field in hwpack metadata. 2011-08-04 Mattias Backman Add extra_boot_options field to hwpack metadata. 2011-08-04 Mattias Backman Rename fdt field to dtb_file to be consistent with dtb_addr. 2011-08-03 Loïc Minier Install initrd-do script. 2011-08-03 Loïc Minier Add new initrd-do script to interactively or automatically edit an initrd. 2011-08-03 Mattias Backman Add dtb_addr field to hwpack metadata. 2011-08-03 Mattias Backman Add vmlinuz, initrd and fdt fields in hwpack metadata. 2011-08-03 Mattias Backman Fix test_install_omap_boot_loader to work with an extra parameter. 2011-08-03 Deepti B. Kalakeri Fix the way l-h-r tar the hwpack 2011-08-03 Mattias Backman Add l-m-c support for having OMAP MLO file in hwpack. 2011-08-03 Mattias Backman Add tests for x-loader config fields. Add one missing test for u-boot-package config. 2011-08-02 James Tunnicliffe Added support for x_loader_file and x_loader_package fields in the hwpack config. Fixed up tests to not fail with new fields. TODO: Test new fields. 2011-07-28 James Westby hwpack-replace: Always add the new info the Packages file. 2011-07-28 Loïc Minier Drop unused gpg_files entirely. 2011-07-28 Loïc Minier Mention optional installation of pyflakes to run tests in the README. 2011-07-28 Loïc Minier pyflakes: update line numbers in allowed warnings. 2011-07-28 Loïc Minier Actually test executed commands in TestMountedPartitionContextManager.test_umount_failure(); has the side effect of fixing a pyflake unused variable warning. 2011-07-28 Loïc Minier pyflakes: drop unused variable redownloaded_files. 2011-07-28 Loïc Minier pyflakes: gpg_files should be self.gpg_files; this is unused in both cases, so should perhaps be removed. 2011-07-28 Loïc Minier Split with statement in two for compatibility with older pythons. 2011-07-27 Angus Ainslie Undo 1 more change 2011-07-27 Angus Ainslie fix whitespce 2011-07-27 Angus Ainslie whitespace change 2011-07-27 Angus Ainslie Fix botched merge 2011-07-28 Bernhard Rosenkraenzer install_boot_loader needs to be called with the boot partition mounted 2011-07-28 Bernhard Rosenkraenzer The iMX53 boot loader needs to be u-boot.imx, not u-boot.bin Current builds have been adapted to provide the correct file. 2011-07-27 James Tunnicliffe Fixing half way through change check in. 2011-07-27 James Tunnicliffe 1. fetch-image-ui now examines the return code of linaro-media-create and reports failures to the UI. 2. The image_size parameter to linaro-media-create wasn't being passed. This resulted in linaro-fetch-image being unable to create Ubuntu Desktop disk images, which now need to be larger than the default size of 2GB. 2011-07-27 Angus Ainslie Add android support for the SMDKV310 board 2011-07-26 Guilherme Salgado Post-release version bump to 2011.07.1 2011-07-26 Guilherme Salgado Release 2011.07 2011-07-26 James Westby Add the new scripts to setup.py. 2011-07-26 James Westby Import the new name of fetch_image in test_fetch_image. 2011-07-26 Bernhard Rosenkranzer Install the imx.53 bootloader when using Android. 2011-07-26 James Westby Avoid an error if the kernel version doesn't match 2.6.*. 2011-07-26 Guilherme Salgado Change the partition_mounted() context manager to only log a warning when umount() fails with a SubcommandNonZeroReturnValue 2011-07-26 James Tunnicliffe This is a temporary workaround for bug #816015. 2011-07-25 Guilherme Salgado A new contextmanager to use when you want to mount a partition, copy something to it and umount when done 2011-07-22 James Tunnicliffe Renamed fetch image tools in line with other applications in the linaro-image-tools repository. 2011-07-22 James Tunnicliffe Added abort condition - if we have a GPG signature for a sha1sum file and even after re-downloading they don't match, something is clearly wrong, so abort. 2011-07-22 James Tunnicliffe If there isn't a GPG signature for a sha1sum file, use sha1sums to validate download, just don't mark packages as signed. 2011-07-22 James Tunnicliffe Moved downloading functions into their own class since there was a lot of shared data being pushed around in variables that could be better implemented as class variables. 2011-07-22 James Tunnicliffe * If a sha1sum check fails and GPG signature for sha1sum file passes, try re-downloading failed files. * Fixed a bug where a build number could be chosen for a day where only the OS image was available for that build number. 2011-07-21 Guilherme Salgado Update the docs of populate_rootfs() to state clearly that it uses an atexit handler and does not umount the partition directly 2011-07-21 Guilherme Salgado Make sure the root partition is umounted if something goes wrong while it's being populated. 2011-07-21 James Tunnicliffe Moved checking the result of verify_file_integrity into its own function that logs the errors, which is now unit tested. 2011-07-21 James Tunnicliffe Added stdout and stderr to SubcommandNonZeroReturnValue.__str__ 2011-07-21 James Tunnicliffe Added communicate function to cmd_runner that will both throw an exception if the function it is communicating with exits with a non-zero exit code and also returns stdout and stderr in the exception object. 2011-07-21 James Tunnicliffe linaro-media-create now aborts on GPG and sha1sum check fails with nice messages. Unit tests currently failing, but manual testing shows that this is working fine. Moved the registration of the atexit handler so it isn't called before /tmp has been used. 2011-07-20 James Tunnicliffe * Changed utils.verify_file_integrity so it won't throw exceptions when either GPG or sha1sum fails * Modified linaro-media-create to cope to new utils.verify_file_integrity return signature * Modified FetchImage to use new utils.verify_file_integrity return information 2011-07-20 James Tunnicliffe Minor tidy up of some truely ugly formatting. 2011-07-19 Andy Doan fix some overo display issues 2011-07-19 James Tunnicliffe Wrapping lines to 79 chars. 2011-07-19 James Tunnicliffe Adding a Policy Kit action definition file, which should live in /usr/share/polkit-1/actions so when pkexec linaro-media-create is called we have more information and less clutter in the Policy Kit auth dialog. 2011-07-19 James Tunnicliffe Moved unit tests out of fetch_image_ui.py into linaro_image_tools/tests/test_fetch_image.py, added tests to linaro_image_tools/tests/__init__.py and fixed bug found by actually running the test in linaro_image_tools/FetchImage.py. 2011-07-19 James Tunnicliffe Tidying up. 2011-07-19 James Tunnicliffe sha1sums and GPG sigs downloaded and checked. 2011-07-18 Bernhard Rosenkraenzer Fix syntax error 2011-07-18 Bernhard Rosenkraenzer Enable VGA port 2011-07-18 Bernhard Rosenkraenzer Don't hardcode ttymxc0 2011-07-18 Guilherme Salgado Make sure rootfs.move_contents() doesn't skip files that are not world-readable 2011-07-18 Deepti B. Kalakeri Addressing the review comments on linaro-inject-debian-inhwpack 2011-07-18 Bernhard Rosenkraenzer mmc_part_offset must be 0 for Android 2011-07-17 Bernhard Rosenkraenzer Initial support for Android on Freescale i.MX53 Quick Start Board 2011-07-14 James Tunnicliffe Now download files required for sig checking in linaro-media-create. Not passing files to l-m-c yet... 2011-07-14 James Tunnicliffe Code clean up. 2011-07-14 James Tunnicliffe GUI download logic excludes already downloaded files from its calculations. 2011-07-13 James Tunnicliffe Fixed lack of download status prints for CLI. 2011-07-13 James Tunnicliffe Now have a download bar in the UI with an ETA read out and current download speed. 2011-07-13 James Tunnicliffe Download files now done by list. Bytes downloaded sent back to UI that now tracks percentage complete. 2011-07-12 James Tunnicliffe GUI code removed from common library. Now GUI runs all downloads and the linaro-media-create call in a separate thread and passes messages in the event queue back to the GUI. 2011-07-12 James Tunnicliffe UI code out of library, but need to run download in separate thread to keep UI responsive. 2011-07-12 Deepti B. Kalakeri Add feature to include/replace a debian package into the given hwpack 2011-07-11 James Tunnicliffe Moved Wizard code out of library. Download status bars to go. 2011-07-09 David A. Long Set the initrd_high and the new fdt_high u-boot environment variables to prevent u-boot from relocating the intial ramdisk and flattened device tree to the end of memory, which would be in HIGHMEM on Panda and would therefore not work. 2011-07-08 James Tunnicliffe Split out linaro-media-create command generation into separate function. 2011-07-07 James Tunnicliffe Moved download function out of create_image. 2011-07-07 James Tunnicliffe Renamed index_server.py to the less generic linaro-image-indexer 2011-07-07 James Tunnicliffe Restored absolute path change (rev 369) and moved file to location where it will work. 2011-07-07 James Tunnicliffe Removed status prints Reverted absolute path change (rev 369). Perhaps I am missing something, but the script wouldn't run. 2011-07-06 Angus Ainslie Fix the BL1 image size check 2011-07-05 Guilherme Salgado Fix the asserts in install_samsung_boot_loader(), which were broken and thus making it impossible to generate images for smvdk310 and origen 2011-07-05 Guilherme Salgado Fix the call to install_samsung_boot_loader() in SamsungConfig._make_boot_files() 2011-07-05 Guilherme Salgado Remove some unneeded arguments from methods of Samsung board configs 2011-07-04 Angus Ainslie Add suport for the Samsung Origen board 2011-07-01 Mattias Backman Only clear metadata defaults if no 1.0 hwpacks have been passed. 2011-07-01 Mattias Backman Minor clean up. 2011-06-30 Markus Andersson Fixed media_create test for 64-bit system and some test parameters more understandable. 2011-06-30 Ola Borgelin Added extra_part parameter to get_sfdisk_cmd in android_boards and moved and renamed populate_startup_partition to populate_raw_partition 2011-06-29 James Westby Post-release version bump to 2011.06-1.1 2011-06-29 James Westby Release 2011.06-1 2011-06-29 James Westby Use the lt- variant of the omap kernel. 2011-06-29 Mattias Backman Add metadata fields boot_min_size, root_min_size and loader_min_size to linaro-media-create. 2011-06-29 Mattias Backman Add tests for metadata output. 2011-06-29 Mattias Backman Add boot_min_size, root_min_size and loader_min_size fields to hwpack-create. 2011-06-29 Mattias Backman Allow mixing hwpack formats and consider metadata format to be the same as the highest format version encountered. 2011-06-29 Ola Borgelin Snowball Android image support and tests 2011-06-29 Mattias Backman Change test to use variable for matching text. 2011-06-29 Mattias Backman Fix the FetchImage import. 2011-06-28 Mattias Backman Use absolute imports to fix problem with setup.py. 2011-06-28 Mattias Backman Clean up and minor fixes. 2011-06-28 Mattias Backman Delay setting extra_serial_opts and live_serial_opts for all boards to make sure that serial_tty is not read from class defaults for V2 format. 2011-06-28 Mattias Backman Support multiple hwpacks. This makes sure that all hwpacks are of the same format version and that metadata fields are not specified in more than one of the hwpacks (except name, maintainter and a few more). 2011-06-28 Mattias Backman Also avoid overriding fat_size. 2011-06-28 Mattias Backman Add asserts for file sizes according to comments. 2011-06-28 Mattias Backman Use uboot file from hwpack for SMDKV310Config. 2011-06-28 Mattias Backman Move SMDKV310 specific methods into SMDKV310Config class. 2011-06-28 Mattias Backman Use hwpack uboot file for Mx5Config. 2011-06-28 Mattias Backman Add get_file() to HardwarepackHandler and use it in populate_boot(). 2011-06-28 Mattias Backman Add tests for HardwarepackHandler. 2011-06-27 Mattias Backman Assert that metadata fields have not already been set to prevent overriding data. 2011-06-23 James Westby Post-release version bump to 2011.06.1 2011-06-23 James Westby Release 2011.06 2011-06-23 Mattias Backman Add a test for Android Panda extra_boot_args. 2011-06-23 Mattias Backman Restore an empty line. 2011-06-23 Mattias Backman Format version comes from the FORMAT file. 2011-06-23 Mattias Backman Duplicate extra_boot_args_options to android_boards.AndroidPandaConfig. 2011-06-23 Mattias Backman Add 2.0 as supported format in hwpack-install. 2011-06-23 Mattias Backman Add unpacking and copying of uboot binary from hwpack. 2011-06-23 Mattias Backman Enable bootfs16_rootfs partition layout in hwpack-create. 2011-06-23 Mattias Backman Add support for new metadata fields. 2011-06-22 Angus Ainslie Check for old SPL file and if it doesn't exist check for new SPL. If neither file exists raise an exception. 2011-06-22 Mattias Backman Change omapfb.vram=0:24M to omapfb.vram=0:24M,1:24M to enable SGX on TI's kernel. 2011-06-21 Mattias Backman Adjust pyflakes test to reflect that a source line has been added. 2011-06-21 Mattias Backman Add basic tests for the file verification. 2011-06-21 Mattias Backman Move file verification from main into a utils function. 2011-06-21 Mattias Backman Add cwd parameter to cmd_runner.run() and use it for the sha1 verification in linaro-media-create. 2011-06-20 Mattias Backman Clean up some ugly formatting. 2011-06-20 Mattias Backman Add tests for find_fetched_package. 2011-06-20 Mattias Backman Store full path to package when fetched using apt. 2011-06-20 Mattias Backman Break apart function that had two purposes. 2011-06-20 Mattias Backman Add option to MockCmdRunnerPopenFixture to disable child finished assertion which is needed to be able to pipe between two cmd_runners. Add stdin property to MockCmdRunnerPopenFixture. Add test_unpack_package test. 2011-06-20 Mattias Backman Remove --wildcards from tar command and add comment. 2011-06-17 James Tunnicliffe Initial merge request of fetch_image_ui.py - a GUI with similar functionality to fetch_image.py. 2011-06-17 James Tunnicliffe Minor code clean up and adding fetch_image_settings.yaml 2011-06-17 James Tunnicliffe Provided sensible default to confirm_device_selection_and_ensure_it_is_ready parameter yes_to_mmc_selection 2011-06-17 Mattias Backman Restore an empty line. 2011-06-17 Mattias Backman Add metadata as string tests. 2011-06-17 Mattias Backman Add tests for PackageUnpacker. 2011-06-17 James Tunnicliffe Modified according to changes proposed by code review. 2011-06-17 James Tunnicliffe Removed trailing whitespace. 2011-06-17 James Tunnicliffe Many changes recommended by code review. 2011-06-17 James Tunnicliffe Updated variable name to match CLI option: nocheck_mmc 2011-06-17 Mattias Backman Clean up a little. 2011-06-17 Mattias Backman Use HardwarePackFormat objects instead of format strings. 2011-06-17 James Tunnicliffe * Changed command line option from --yes-to-mmc-selection to --nocheck-mmc * Wrapped lines to 79 chars. 2011-06-17 Mattias Backman Add PackageFetcher.get_package_path(). 2011-06-17 Mattias Backman Cleanup after review. 2011-06-16 Mattias Backman Break out u-boot code into separate method. 2011-06-16 Mattias Backman Compare floats instead of format strings. 2011-06-16 Mattias Backman Remove not yet supported config fields. 2011-06-16 Mattias Backman Break out regexp matching to separate method. 2011-06-16 Mattias Backman Move package unpacking and tempfile handling to context manager object. 2011-06-15 James Tunnicliffe Adding the initial revision of the Fetch Image CLI tool. 2011-06-15 James Tunnicliffe Some major changes to FetchImage.py since the last merge proposal, all in support of development on lp:~dooferlad/linaro-image-tools/my_dev 2011-06-15 James Tunnicliffe Add command line option to force yes to the "Are you 100% sure on selecting [mmc]" question. 2011-06-14 Mattias Backman Use smaller partitions for testing. 2011-06-13 Mattias Backman Clean up tempdir atexit and verify that the u-boot package actually was unpacked. 2011-06-13 Mattias Backman Add test and validation for mmc_id. 2011-06-13 Mattias Backman Add test for partition_layout. 2011-06-13 Mattias Backman Add tests for network interface fields. Any ideas about naming restrictions for nw interfaces? 2011-06-13 Mattias Backman Add tests and validation for kernel_addr, initrd_addr and load_addr. 2011-06-13 Mattias Backman Add test for serial_tty. 2011-06-13 Mattias Backman Add tests and validation for format, u-boot-package and u-boot-file. 2011-06-13 Mattias Backman Improve comments in tests. 2011-06-10 Mattias Backman Make sure that 2.0 code only is executed when format is at least 2.0. Fix tests for new function signatures. 2011-06-10 Mattias Backman Change test_get_boot_and_root_loopback_devices() and test_calculate_partition_size_and_offset() to use data stored in a variable instead of unconnected magic numbers. 2011-06-10 Mattias Backman Limit the data list to < 80 chars, didn't improve readability though. 2011-06-10 Mattias Backman Clean up tests and put partition size and offsets in variable, after review comments. Thanks for the tip, Guilherme. 2011-06-09 Mattias Backman Only validate new fields if format is at least 2.0. 2011-06-09 Mattias Backman Don't unpack the u-boot package if 2.0 format is not used. 2011-06-09 Mattias Backman Add test for get_android_loopback_devices(). 2011-06-08 Mattias Backman Add test for calculate_android_partition_size_and_offset. 2011-06-07 Mattias Backman Unpack only explicitly specified file from u-boot package. Extract function to add one file from a package to the hwpack in preparation for supporting similar files. 2011-06-07 Mattias Backman Remove hwpack-create board field. 2011-06-01 Mattias Backman Get u-boot package path from PackageFetcher. 2011-06-01 Mattias Backman Add new metadata fields from wiki spec with notes about implementation. 2011-06-01 Mattias Backman Remove placeholders for options that are not being worked on yet. 2011-06-01 Mattias Backman Add support for adding plain files to hwpack and add filenames to metadata. Also use this for file unpacked from u-boot package. 2011-05-31 Mattias Backman Get name of u-boot package from hwpack file, download the package but do not add to hwpack. 2011-05-31 Mattias Backman Get format from config file. 1.0 will be default and require hwpacks to specify format from now on. 2011-05-31 Mattias Backman Ignore the extended partition info. 2011-05-30 Mattias Backman Get partition info for Android image. 2011-05-30 Mattias Backman Replace 512 with SECTOR_SIZE constant. 2011-05-26 James Westby Post-release version bump to 0.4.8.1 2011-05-26 James Westby Release 0.4.8 2011-05-26 James Westby Use single quotes to avoid escaping. Thanks Guilherme. 2011-05-26 James Westby Add a test for the quoting behaviour. 2011-05-26 James Westby Use double quotes around setenv arguments. 2011-05-26 Mattias Backman Assert that all partitions are found. 2011-05-26 Mattias Backman Use loopback devices for image file. 2011-05-26 Mattias Backman Add --image_size option too. 2011-05-26 Mattias Backman Change --no-part check to match command line options. 2011-05-26 Mattias Backman Add --image_file option to linaro-android-media-create 2011-05-25 James Westby Post-release version bump to 0.4.7.1 2011-05-25 James Westby Release 0.4.7 2011-05-25 James Westby Drop the unused doctest import. 2011-05-25 James Westby Cleanups from review. Thanks Guilherme. 2011-05-25 James Westby Allow packages in hwpacks to remove packages from the rootfs. 2011-05-25 James Westby Add a test for the log output. 2011-05-25 James Westby Fix the logging output from linaro-hwpack-create. 2011-05-23 Ricardo Salveti de Araujo linaro_image_tools/media_create/boards.py: adding mpurate=${mpurate} argument for beagle 2011-05-23 Mattias Backman Make do-release script remove possibly remaining .testrepository dir. 2011-05-23 Mattias Backman Post-release version bump to 0.4.6.1 2011-05-23 Mattias Backman Release 0.4.6 2011-05-23 Mattias Backman Change test to match new mem values. Remove quotes to be consistent with Beagle and Overo configs. 2011-05-22 Alexander Sack linaro_image_tools/media_create/boards.py: allocate 48M vram and 0:24M for first screen for 1080p resolution 2011-05-20 Mattias Backman Remove Snowball_Emmc test which is not working. 2011-05-20 Mattias Backman Assert that toc file is not larger than the size reserved for it. Add comment to remember that more sanity checks are needed. 2011-05-20 Mattias Backman Change name of config file to startupfiles.cfg per Tony's suggestion. Remove the example file and assume that the config file unpacks to the same location as startup files. 2011-05-19 Mattias Backman Block --mm option for SnowballEmmcConfig since this config does not make sense to write to an mmc. 2011-05-19 Mattias Backman Change SnowballSdcardConfig to SnowballSdConfig and change SnowballImage to SnowballEmmcConfig. 2011-05-19 Mattias Backman Explicitly specify all toc entries in data file. 2011-05-18 Mattias Backman Break a long line. 2011-05-18 Mattias Backman Move startup files config file name into constant. 2011-05-18 Mattias Backman Make parameters in create_toc match the names of TOC fields according to the spec. 2011-05-16 Mattias Backman Temp add the toc text file for convenience. 2011-05-16 Mattias Backman Break out toc data into file. 2011-05-16 Mattias Backman Use tempfile for temporary toc file. 2011-05-16 Mattias Backman Removed some unused item in install_snowball_boot_loader. 2011-05-16 Mattias Backman Add comment to explain format string. Note spec and code seems to not match. 2011-05-16 Mattias Backman Remove variable item which was only temporary storage. 2011-05-16 Mattias Backman Remove unused tuple item 'sz'. 2011-05-16 Mattias Backman Change iteration in get_file_info a little. 2011-05-16 Mattias Backman Move functions install_snowball_boot_loader, create_toc and get_file_info into SnowballImageConfig since they are specific to Snowball. 2011-05-16 Mattias Backman Rename LOADER_START_S to SNOWBALL_LOADER_START_S 2011-05-13 Tony Mansson Initial Snowball support 2011-05-10 Loïc Minier Run pyflakes if it's installed and the tests are being run from a source tree. 2011-05-10 Loïc Minier Break a new has_command() out of ensure_command(). 2011-05-10 Loïc Minier Assert that the created config isn't None; has the nice side effect of silencing a pyflakes warning. 2011-05-10 Loïc Minier Remove unused proc variable. 2011-05-10 Loïc Minier Add an assert to a test which has the nice side effect of silencing a pyflakes warning. 2011-05-06 Loïc Minier Use nicer construct to test if a char is a digit. 2011-05-06 Loïc Minier Add support for --image_size without any unit; LP: #778426. 2011-05-06 Mattias Backman Extend Config to validate and provide new options. 2011-05-06 Mattias Backman Add new options to metadata file. 2011-05-06 Mattias Backman Extend Metadata object iwith new options. 2011-05-05 Loïc Minier Fix leftover "!= 0:"; thanks Matt Waddel; LP: #777938. 2011-05-05 Mattias Backman Make version option make it into the FORMAT file in hwpack. 2011-05-05 James Tunnicliffe Server indexer updated to scan local file system 2011-05-02 Mattias Backman Remove message and exit that can never be reached. 2011-04-29 Mattias Backman Use os.path.basename() instead of split(). 2011-04-29 Loïc Minier Drop mention of qemu-kvm in required packages in the README; rename qemu-kvm-extras-static to qemu-user-static and kill obsolete links. 2011-04-29 Loïc Minier Convert the testsuite to also use dd instead of qemu-img. 2011-04-29 Loïc Minier Fix missing count=0 in dd calls and a mention of qemu-img in the testsuite's comments. 2011-04-29 Loïc Minier Don't require qemu-img any longer; LP: 772264. 2011-04-29 Loïc Minier Use dd instead of qemu-img to create sparse files. 2011-04-28 James Tunnicliffe Merge of files for generating the server index 2011-04-28 Mattias Backman Set hwpack-force-yes for hwpacks with verified signature. 2011-04-28 Mattias Backman Use communicate() when checking sha1 hashes to get command output. 2011-04-28 Mattias Backman Sort devices before printing them. 2011-04-27 Mattias Backman Post-release version bump to 0.4.5.1 2011-04-27 Mattias Backman Release 0.4.5 2011-04-27 Mattias Backman Include linaro-android-media-create in release. 2011-04-27 Mattias Backman Replace magic number with len(.asc) and change error messages. 2011-04-27 Mattias Backman Remove error printouts. 2011-04-26 Mattias Backman Make signature list default be empty list to avoid having to check for None. 2011-04-22 jeremy.chang@linaro.org <> Share more between the two bootscript 2011-04-22 jeremy.chang@linaro.org <> Fix umount error 2011-04-22 jeremy.chang@linaro.org <> Use import to replace the duplicatd code 2011-04-22 Loïc Minier Fix linaro-android-media-create imports: import media_create.chroot_utils instaed of hwpack, and drop unused and unavailable populate_boot import; LP: #768790. 2011-04-21 Andy Doan allow expansion of variables in boot args 2011-04-21 Mattias Backman Make checking signatures optional. 2011-04-21 Mattias Backman Post-release version bump to 0.4.4.1 2011-04-21 Mattias Backman Release 0.4.4 2011-04-21 Mattias Backman Tests are now in module linaro_image_tools. 2011-04-21 Loïc Minier TestBootSteps.make_boot_files(): only return a d_img_data if the BoardConfig defines a dtb_name. 2011-04-21 Loïc Minier Don't call make_dtb() on Samsung or pass a d_img_data file in the testsuite as this board doesn't set DTB config vars for now. 2011-04-21 Loïc Minier Raise an assertion error when _get_bootcmd() is called with a d_img_data but cls.dtb_addr is None. 2011-04-21 Mattias Backman Check that a signature file for each hwpack is passed. 2011-04-21 Mattias Backman Exit if gpg signature or archive hash cannot be verified. 2011-04-20 Andy Doan set VRAM value for Overo 2011-04-20 Andy Doan revert commit 289. There kernel now has OMAP support for EDID which can probe for the best resolution to use. 2011-04-21 jeremy.chang@linaro.org <> android_board.py: fix boot env 2011-04-19 jeremy.chang@linaro.org <> l-a-m-c: Remove unused ROOTFS_DIR 2011-04-19 jeremy.chang@linaro.org <> linaro-android-media-create: remove the unnecessary and wrong import of install_hwpacks 2011-04-18 jeremy.chang@linaro.org <> Android kernel cmdline support added 2011-04-18 jeremy.chang@linaro.org <> generate boot.scr in android boot partition 2011-04-15 Shawn Guo Fix lines longer than 80 characters introduced by dtb changes 2011-04-15 Shawn Guo Change @classproperty bootcmd to @classmethod _get_bootcmd, and pass full path name of .dtb file, so that we can get the correct bootcmd with .dtb being optional. 2011-04-14 Shawn Guo Change DT_GLOB to DTB_GLOB to include dtb_name, so that the meaning of 'dtb' and 'cls.dtb_name' in _get_kflavor_files can be clear. 2011-04-14 Shawn Guo Improve _get_kflavor_files comment to state that dtb is optional 2011-04-12 Shawn Guo Address the following review comments from Loic 2011-04-12 jeremy.chang@linaro.org <> Create separate config classes for android 2011-04-11 jeremy.chang@linaro.org <> Introduce boottarball to android to replace the roottarball 2011-04-11 loic.minier@linaro.org <> Tweak first RAM size on Panda to be a multiple of 8 MiB; see LP #633227. 2011-04-11 Shawn Guo Add dtb support for omap and mx5 based boards 2011-04-10 Shawn Guo Align mx51 offset of kernel_addr and initrd_addr with mx53 2011-04-09 loic.minier@linaro.org <> Merge install_smdkv310_boot_env(), install_smdkv310_spl(), and install_smdkv310_uboot() into Samsung's make_boot_files(); simplifies reading. 2011-04-09 loic.minier@linaro.org <> make_uImage and make_uInitrd return _run_mkimage() again. 2011-04-09 loic.minier@linaro.org <> Drop count= from the _dd() call in install_smdkv310_boot_env() too as the size of the input file is guaranteed already and this might hide issues. 2011-04-09 loic.minier@linaro.org <> Drop unused Samsung constants and asserts. 2011-04-09 loic.minier@linaro.org <> Drop count= from _dd() call in install_smdkv310_spl(); this was only needed when copying parts of the source file, but now the whole file should be copied. 2011-04-09 Loïc Minier Update comments for Samsung implementation. 2011-04-07 Loïc Minier Drop ip=none from Panda board cmdline args; dangerous option to include by default according to Steve Langasek. 2011-04-06 Angus Ainslie Fix test cases the Samsung smdkv310 2011-04-06 Angus Ainslie SMDKv310 u-boot can load from a FAT partition some remove the old code that wrote directly to the MMC card 2011-04-06 Marcin Juszkiewicz pandaboard: enable 463+512MB ram configuration 2011-04-06 Mattias Backman Remove blank line, sort kernel_flavors and add test for _get_kflavor_files not finding files. 2011-04-06 Mattias Backman Fix deprecation warning for package.isInstalled. 2011-04-06 Mattias Backman Add kernel_flavor 'omap4' for linaro-m hwpack bsp-omap4. 2011-04-05 loic.minier@linaro.org <> Typo: cumalative instead of cumulative. 2011-04-05 Angus Ainslie Fix broken merge and remove redundant line 2011-04-05 Mattias Backman Move looking up kernel and initrd files to make_boot_files(). 2011-04-05 Mattias Backman Fix tests by mocking _get_kflavor_files. 2011-04-05 loic.minier@linaro.org <> Fix _get_kflavor_files tests to pass by creating a temp dir and a derived class rather than temp files and a BoardConfig instance. 2011-04-05 Guilherme Salgado Add a couple XXX/TODO comments for things that need to be fixed. 2011-04-05 loic.minier@linaro.org <> Restore uImage_file and uInitrd_file in Samsung V310 code. 2011-04-05 loic.minier@linaro.org <> Rename _get_kflavor_file to _get_kflavor_files and change it to only take a base directory to search the kernel and initrd files and return these. 2011-04-05 loic.minier@linaro.org <> Move BoardConfig.kernel_base and .initrd_base to global constants named KERNEL_GLOB and INITRD_GLOB. 2011-04-05 loic.minier@linaro.org <> Fix typo: use initrd_base instead of kernel_base for initrd. 2011-04-05 Mattias Backman Make the changes to _make_boot_files. 2011-04-05 Loïc Minier Mock os.getuid() in test_append_to_fstab() to avoid calling sudo in the tests. 2011-04-04 Loïc Minier Run apt-get clean after installing packages. 2011-04-04 matt.waddel@linaro.org <> Fixes number of parameters in call to make_boot_files() in the test code 2011-04-04 matt.waddel@linaro.org <> Removes potentially uninitialized variable, boot_script_path, from calls to make_boot_files() 2011-04-04 Mattias Backman Add get_kflavor_file function. 2011-04-04 Loïc Minier Add newline to all lines appended to etc/fstab; add tests for append_to_fstab; LP: #745728. 2011-04-04 Loïc Minier Silence install_packages() output in tests. 2011-04-04 Loïc Minier Add some message while installing packages in the chroot, and fix the linaro-hwpack-install message to not mention apt-get. 2011-04-04 Loïc Minier Add rootfs_mount_options() helper for the mount options to use in fstab depending on the rootfs; add tests; LP: #745771. 2011-04-04 Mattias Backman Change kernel_suffix to kernel_flavors. 2011-04-04 Loïc Minier Rename linaro_image_tools.media_create.hwpack to chroot_utils as it also contains a helper to install packages. 2011-04-04 Loïc Minier Factor tests for mkfs.fs commands and raise an error for unknown rootfs types. 2011-04-04 Loïc Minier Install btrfs-tools in the chroot when creating a btrfs rootfs; LP: #715932. 2011-04-04 Loïc Minier Add test for media_create.hwpack.install_packages(). 2011-04-04 Loïc Minier Add install_packages function to apt-get install some packages in the chroot. 2011-04-04 Loïc Minier Remove unused vars. 2011-04-04 Loïc Minier Don't repeat the prepare_chroot() testing in test_install_hwpacks(), just check that it was called with proper args; refactor some media_create.hwpack tests to work with arbitrary chroot and tmp dirs. 2011-04-04 Loïc Minier Test whether .wait() (or .communicate()) was run for every Popen() object, not just the last one. 2011-04-04 Loïc Minier Split off generic parts from install_hwpacks() into a new prepare_chroot(); update tests. 2011-04-04 Loïc Minier Add chroot support to cmd_runner.run() and use it to install hwpacks. 2011-04-01 matt.waddel@linaro.org <> Prevents uninitialized string from being passed to make_boot_files() in the vexpress 2011-04-01 jeremy.chang@linaro.org <> Add android_board_configs to indicate the supported boards for android 2011-04-01 jeremy.chang@linaro.org <> Add get_android_sfdisk_cmd function 2011-04-01 jeremy.chang@linaro.org <> Give it a loop over formating same kind partitions 2011-04-01 jeremy.chang@linaro.org <> umount android partitions in cleanup function 2011-03-31 Angus Ainslie Remove debugging code 2011-03-31 Angus Ainslie Update Samsung BL0 and BL1 comments Remove u-boot comment as it is no longer valid 2011-03-30 Angus Ainslie Change the tools to use the new SPL + uboot method Fat and ext2 are enabled to load the kernel 2011-03-31 jeremy.chang@linaro.org <> Add a EXPERIMENTAL banner for linaro-android-media-create 2011-03-31 jeremy.chang@linaro.org <> Recover the unneeded Keyword arguments change. 2011-03-31 jeremy.chang@linaro.org <> Fix regression bug, UnboundLocalError in get_sfdisk_cmd in boards.py 2011-03-30 jeremy.chang@linaro.org <> Clean up the android code 2011-03-29 Loïc Minier Update tests for new mx51 initrd address. 2011-03-24 Loïc Minier Expose default path of sanitize_path() as to help testing and add tests for sanitize_path(). 2011-03-24 Loïc Minier Use cmd_runner instead of subprocess, this ensures that a sane PATH is used. 2011-03-24 Loïc Minier Use cmd_runner.SUDO_ARGS instead of importing it; tidy up imports. 2011-03-24 Loïc Minier Revert parts of r309; use MockCmdRunnerPopenFixture.mock.commands_executed instead of .calls for clarity in the test cases, and simply define sudo_args from SUDO_ARGS. 2011-03-24 Loïc Minier Sanitize path with cmd_runner's helper rather than reinventing it. 2011-03-24 Loïc Minier Bug: find_command() was setting os.environ['PATH'] to ":/bin:usr/bin" if unset, but this means that the current directory was searched first unconditionally even when calling linaro-image-tools system-wide. Later logic in find_command() correctly prepends the current directory when needed, so it was useless and incorrect to include . in the PATH by default. 2011-03-24 Loïc Minier Factor handling of unset PATH and default PATH in the common case. 2011-03-24 Loïc Minier Fix handling of empty PATH; this actually means that the current directory should be searched, and not that PATH is empty. 2011-03-24 Loïc Minier Actually make sure that any of the directories typically found in the PATH are added if missing; this will ensure that commands which we call and which are typically in /usr/bin are also in the PATH, and that commands from linaro-image-tools which might have been installed under /usr/local are also in the PATH. 2011-03-24 Loïc Minier Move PATH sanitization to a function and call that from Popen.__init__(); note that: - this regresses the hwpack testsuite as it uses subprocess.Popen directly, so that a broken PATH might not be fixed up - due to the media_create code doing a os.environ.copy() before os.environ being sanitized, this requires sanitizing both os.environ and the env copy passed to Popen() 2011-03-24 Loïc Minier Set os.environ['PATH'] if unset or empty; append /sbin to it if missing; LP: #709517. 2011-03-24 Loïc Minier Pass --yes to apt-get when automatically installing packages for missing commands, this is needed when more than one package would get installed; LP: #741638. 2011-03-24 Loïc Minier Move duplicated preferred_tools_dir helper to linaro_image_tools.utils; this still works because linaro_image_tools.utils' __file__ is still a relative path. 2011-03-24 Loïc Minier Move TestCaseWithFixtures from linaro_image_tools.hwpack.testing to testing as it's used in more places than hwpack. 2011-03-24 Loïc Minier Fix typo: "tht" instead of "the". 2011-03-24 Loïc Minier Drop unused imports. 2011-03-24 Loïc Minier Move CreateTempDirFixture, MockSomethingFixture, MockCmdRunnerPopen and MockCmdRunnerPopenFixture from linaro_image_tools.media_create.tests.fixtures to tests.fixtures as these are used outside of media_create. 2011-03-24 Loïc Minier Expose cmd_runner's 'sudo -E' args for use in tests; convert most tests to use an array of args rather than a string. 2011-03-24 Loïc Minier Add a test for and fix cmd_runner.run() with tuples. 2011-03-24 Loïc Minier Split linaro_image_tools.cmd_runner tests out of media_create.tests.test_media_create into tests.test_cmd_runner. 2011-03-24 Loïc Minier Move linaro_image_tools.media_create.cmd_runner to linaro_image_tools.cmd_runner as it's used in linaro_image_tools.utils and generally useful anyway. 2011-03-24 Loïc Minier Split linaro_image_tools.utils tests out of media_create.tests.test_media_create into tests.test_utils; some code duplication for now. 2011-03-24 Loïc Minier Drop unused "os" import. 2011-03-24 Loïc Minier Move linaro_image_tools.media_create.utils to linaro_image_tools.utils. 2011-03-24 Loïc Minier Use media_create.find_command instead of custom find_script as it works better to run the testsuite on an installed system; LP: #711312. 2011-03-24 Loïc Minier Rename boot_script to boot_script_path when passing a pathname. 2011-03-24 Loïc Minier Simplify boot_script_path computation. 2011-03-23 Loïc Minier Move populate_boot() from media_create.populate_boot to media_create.boards.BoardConfig; LP: #716469. 2011-03-23 Loïc Minier Shuffle modules to be under the same hierarchy: - linaro_media_create -> linaro_image_tools/media_create - hwpack -> linaro_image_tools.hwpack - tests -> linaro_image_tools.tests this will notably give us a place to share code between linaro_image_tools.media_create and .hwpack. 2011-03-23 jeremy.chang@linaro.org <> Change sdcard partition to vfat format 2011-03-17 John Rigby make initrd_addr for mx5 larger 2011-03-17 Loïc Minier Use HEADS and SECTORS from partitions.py in tests. 2011-03-17 Loïc Minier Create a Mx51Config derived from Mx5Config and move load addresses and kernel flavor specific to i.MX51 to Mx51Config; also introduce a Mx53Config derived from Mx5Config and merge i.MX53 data from Mx53LocoConfig into it. Update tests to use Mx5 or Mx51 correctly. 2011-03-17 Loïc Minier Rename i.MX5 to i.MX5x in comment. 2011-03-17 Loïc Minier TestBootSteps.test_mx5_steps(): create a fake Mx5Config instead of using Mx51evkConfig. 2011-03-17 Loïc Minier Add support for EfikaMX SmartBook (efikasb). 2011-03-16 jeremy.chang@linaro.org <> Change android rootfs partition default format to ext4 2011-03-09 Guilherme Salgado Change cmd_runner.run() to always pass -E to sudo 2011-03-09 jeremy.chang@linaro.org <> Install linaro-android-media-create 2011-03-08 jeremy.chang@linaro.org <> Add Android images deployment support 2011-03-08 Eric Miao Fix names of uboot_flavor and kernel_suffix 2011-03-07 Lee Jones Include support for the new u8500 hardware pack 2011-03-05 Loïc Minier Set kernel_suffix = 'linaro-omap' in OmapConfig instead of children classes. 2011-03-05 Loïc Minier Drop now unused install_mx5_uboot(). 2011-03-05 Loïc Minier Let Mx53 use the Mx5 _make_boot_files() which is now identical. 2011-03-05 Loïc Minier Always require a non-padded u-boot.imx for Mx53. 2011-03-05 Loïc Minier Use cls.uboot_flavor instead of hardcoding 'mx53_loco'. 2011-03-05 Loïc Minier Convert tabs to spaces. 2011-03-05 Eric Miao Make use of Mx5Config as superclass for Mx53LoCoConfig 2011-03-05 Loïc Minier Make sure an assertion is raised if the environment size is smaller than the generated environment; also change the code to do a single struct.pack() call instead of one call per byte to pad. 2011-03-05 Loïc Minier make_flashable_env: sort env_strings to have reliable ordering. 2011-03-05 Loïc Minier Add test for make_flashable_env(). 2011-03-05 Loïc Minier Rename tempfile to tmpfile to allow importing tempfile. 2011-03-05 Loïc Minier Implement size checks in install_smdkv310_uImage and install_smdkv310_initrd and document why they aren't needed in install_smdkv310_boot_env. 2011-03-05 Loïc Minier Simplify _dd docstring a bit. 2011-03-05 Loïc Minier Change the _get_bootcmd classmethod to a bootcmd classproperty; thanks Guilherme Salgado. 2011-03-05 Loïc Minier Remove unused import causing a lot of pyflakes errors. 2011-03-05 Loïc Minier Add testcase for the mx5 bootloader being too big. 2011-03-05 Loïc Minier Fix size check for mx5_boot_loader to use the LOADER constant rather than BOOT. 2011-03-05 Loïc Minier Check size of imx_file is smaller than guaranteed bootloader partition size in install_mx5_boot_loader(); update testsuite to pass a real file to allow os.path.getsize() to work. 2011-03-03 Guilherme Salgado Saving of the plain boot script (boot.txt) on the boot partition was failing with a permission error; this fixes it. 2011-03-02 Angus Ainslie fix review comments from salgado 2011-03-02 Angus Ainslie add the temp_dir logic back in that I removed during the last merge 2011-03-01 Angus Ainslie Fixup partition tests for new partition sizes. Remove duplicate Samsung SMDK v310 test 2011-03-01 Angus Ainslie Remove duplicate EfikaMX classes 2011-03-01 Angus Ainslie Remove duplicate Samsung board definition from merge Add constants reserved length and copy length for the Samsung uinitrd 2011-03-01 Loïc Minier Use proper initrd size of 6 MiB instead of 100 MiB for Samsung, and use that in movi boot commands. 2011-03-01 Loïc Minier Don't hardcode kernel_addr and initrd_addr in Samsung movi commands. 2011-03-01 Loïc Minier Remove a couple of spaces. 2011-03-01 Loïc Minier Don't generate a boot.scr for now as it is unused and might be confusing. 2011-03-01 Loïc Minier Use chroot_dir consistently instead of mixing with root_dir. 2011-03-01 Loïc Minier Instead of passing boot commands around, just pass a boot_env dict. 2011-03-01 Loïc Minier Rename boot_cmd to boot_commands for clarity. 2011-03-01 Loïc Minier SMDKV310Config: make env_size a regular variable in _make_boot_files() instead of a class variable as it's used only once. 2011-03-01 Loïc Minier Merge SMDKV310Config and SamsungConfig since there's only one board for now. 2011-03-01 Loïc Minier Consistent spacing between functions. 2011-03-01 Loïc Minier Drop seeminly useless make_boot_ini() return value. 2011-03-01 Loïc Minier Simplify _get_boot_commands() a bit. 2011-03-01 Loïc Minier Ups, also rename _get_boot_cmd to _get_boot_commands in the testsuite. 2011-03-01 Loïc Minier Rename _get_boot_cmd to _get_boot_commands to avoid confusion with _get_bootcmd. 2011-03-01 Loïc Minier Don't align bootloader partition to 4 MiB boundary, instead start on the first sector possible, +1s. 2011-03-01 Loïc Minier Remove trailing spaces. 2011-02-28 Angus Ainslie incorporated Loic's conastant changes 2011-02-28 Angus Ainslie Update bootcmd and bootargs Fixed tests for v310 board 2011-02-28 Guilherme Salgado Rename the plain-text boot script saved on the boot partition from boot.script to boot.txt 2011-02-28 Eric Miao Add preliminary support for Freescale i.MX53 LoCo board 2011-02-25 Loïc Minier Be consistent and use "uboot" rather than "u_boot". 2011-02-25 Loïc Minier populate_boot(): assert that uboot_flavor is set when uboot_in_boot_part is. 2011-02-25 Loïc Minier Fix TestPopulateBoot.prepare_config to create derived classes rather than using the passed class. 2011-02-25 Loïc Minier TestPopulateBoot: instead of testing with representative boards, test with specially crafted test configs. 2011-02-25 Loïc Minier Share expected_call across TestPopulateBoot tests. 2011-02-25 Loïc Minier Share expected_args across TestPopulateBoot tests. 2011-02-25 Loïc Minier Add TestPopulateBoot.call_populate_boot() to avoid code repeating the populate_boot args. 2011-02-24 Loïc Minier Test install_omap_boot_loader(). 2011-02-24 Loïc Minier Test install_mx5_boot_loader(). 2011-02-24 Loïc Minier Don't instanciate BoardConfig classes. 2011-02-24 Loïc Minier Use Mx5Config() instead of mx51evk's config. 2011-02-24 Loïc Minier Fix TestPopulateBoot name clash which was hiding some tests. 2011-02-24 Loïc Minier More consistent spacing. 2011-02-24 Loïc Minier Test populate_boot() with more board configs. 2011-02-24 Loïc Minier Add tests for linaro_media_create.populate_boot.populate_boot(). 2011-02-24 Loïc Minier Drop unused errno import. 2011-02-24 Loïc Minier Run "mkdir -p" instead of using os.makedirs(); this might be less elegant, but it makes the code much simpler and easier to test as we don't need to create a place to run the os.makedirs() or to mock anything more than cmd_runner. 2011-02-24 Loïc Minier Add a uboot_in_boot_part BoardConfig flag to decide whether u-boot.bin should be copied to the boot partition instead of just checking uboot_flavor; this allows using uboot_flavor consistently on mx5 boards. 2011-02-22 Andy Doan update boot.scr for overo to support 1024x768 resolution 2011-02-22 Loïc Minier Add TODOs to add missing checks. 2011-02-22 Loïc Minier Define and use a _dd() function to wrap dd. 2011-02-22 Loïc Minier Rework Samsung V310 partitioning: - don't use a boot partition since u-boot currently reads uImage/uInitrd from fixed offsets - avoid hardcoding partition lengths/offsets, use constants instead - limit dd's count when writing uImage and uInitrd 2011-02-21 Loïc Minier Use SECTOR_SIZE in install_mx51evk_boot_loader() instead of hardcoding 1024. 2011-02-21 Loïc Minier Use SECTOR_SIZE instead of hardcoding 512 in dd calls. 2011-02-19 Loïc Minier We try to align partition start/end offsets to 4 MiB, but we had stopped making any effort to align to CHS boundaries and were always using 255 heads and 63 sectors. Since this causes warnings with partitioning tools and since we can use any geometry we like, use 128 heads and 32 sectors as geometry which gives us 2 MiB cylinders and means partitions will end/start on cylinder boundaries most of the time. 2011-02-17 Angus Ainslie Remove commented out code and old comments 2011-02-17 Angus Ainslie Finished fixing changes to partitons Remove use_fat_partiton flag Added vfat partition to Samsung v310 build All boot binaries are generated to the boot partition but they are currently still loaded from raw MMC offsets. 2011-02-17 Loïc Minier Add support for EfikaMX smarttops. - rename mx51evk-related logic to share it across mx5-based boards - EfikamxConfig and Mx51evkConfig just set uboot_name (NB: uboot_flavor has a special omap-only meaning and hence wasn't suitable) 2011-02-15 Angus Ainslie fix broken smdkv310 tests 2011-02-15 Angus Ainslie more review changes 2011-02-15 Angus Ainslie more review changes 2011-02-14 Angus Ainslie fix changes due to sfdisk command changes 2011-02-14 Loïc Minier Fix typos in comments. 2011-02-14 Loïc Minier Closing triple double-quotes need to stand on their own line. 2011-02-14 Loïc Minier Convert align_up to a real function and add docstrings to align_partition() and align_up(). 2011-02-12 Loïc Minier Add tests for IGEP. 2011-02-11 Loïc Minier Make sure boot partition has an even length in sectors (partition length is an entire number of KiBs) to please OMAP3 ROMs; these apparently don't care for the start sector, but do care for the length of the partition... Update comments as well. 2011-02-11 Loïc Minier Add --align-boot-part flag to force (4 MiB) alignment of boot partition. 2011-02-11 Loïc Minier Add tests for align_up and align_partition. 2011-02-11 Loïc Minier Rework sfdisk commands again; completely bypass CHS calculations as they are irrelevant (see LP #626907 for discussion). - enforce some sensible sizes of >= 1 MiB, >= 50 MiB, and >= 50 MiB for (optional) bootloader partition, boot partition and root partition respectively - document partition layout in get_sfdisk_cmd() - update testsuite for new values 2011-02-11 Loïc Minier Factorize the creation of a relatively common disk tempfile. 2011-02-11 Loïc Minier Add messages to asserts. 2011-02-10 Angus Ainslie fix bootm line so that initrd mounts 2011-02-10 Angus Ainslie Get rid of whitespace change 2011-02-10 Angus Ainslie Merge review changes 2011-02-10 Angus Ainslie added tests for the SMDKv310 board 2011-02-10 Loïc Minier Add comment to test_partition_numbering(). 2011-02-10 Loïc Minier Add test to make sure first partition after vfat is selected as the linux partition. 2011-02-10 Loïc Minier Drop tests which are probably useless after discussion -- let's not bloat the testsuite. 2011-02-10 Loïc Minier Remove debug print. 2011-02-10 Loïc Minier Add some tests to make sure rootfs is after bootfs and that partitions don't overlap. 2011-02-10 Loïc Minier Add an explicit break when we find the linux partition as to not use a partition found afterwards. 2011-02-10 Loïc Minier Add some helpful message in the "normal partition" assert. 2011-02-10 Loïc Minier Fix weird line-wrapping of mine. 2011-02-10 Guilherme Salgado When --consoles is passed to l-m-c, append (instead of prepend) them to the list of serial options in the boot cmd 2011-02-10 Loïc Minier Align partitions at 4 MiB; this requires switching to sector addressing for sfdisk commands and is fairly intrusive. 2011-02-10 Loïc Minier Rework processing of parted partitions; when using partitions not aligned on cylinder boundaries, some "free space" partition is returned by the call to nextPartition(); this partition isn't the one we were actually looking for. This will be exercized by the testsuite once we move to 4 MiB aligned partitions. 2011-02-09 Loïc Minier Don't reset image_size_in_bytes to a cylinder-aligned size: we want to honor the requested image size, even if that means some padding. 2011-02-09 Loïc Minier Fix use of image_size instead of image_size_in_bytes when calling qemu-img. 2011-02-09 Angus Ainslie Fix kernel load address and u-boot env 2011-02-09 Guilherme Salgado l-m-c now stores the original boot script (before we feed it to mkimage) in the boot partition 2011-02-08 Angus Ainslie Added Samsung SMDK V310 - doesn't quite boot yet 2011-02-02 Loïc Minier Factor the prefer_dir computation which had to be repeated in the previous merge. 2011-02-02 Loïc Minier Default to os.getcwd() instead of "." as "." breaks with some ChdirToTempdirFixture based tests. 2011-02-02 Loïc Minier More explicit handling of empty PATH components as "." in find_command. 2011-02-02 Loïc Minier Pass a tools dir (preferred directory) to install_hwpacks; should be the directory of linaro-media-create when it's run with an explicit directory, or "." when running from a bzr checkout. 2011-02-02 Loïc Minier Add an optional prefer_dir arg to find_command to allow searching a specific directory first. 2011-02-02 Loïc Minier Always try to unmount ROOT_DISK, even if BOOT_DISK didn't unmount successfully. 2011-02-02 Loïc Minier Use cmd_runner.run() instead of .Popen() as the latter doesn't accept as_root. 2011-02-02 Loïc Minier Add missing cmd_runner module before SubcommandNonZeroReturnValue; LP: #711573. 2011-02-01 Loïc Minier Fix testsuite breakage with reldir; was running the wrong testsuite. :-/ 2011-02-01 Guilherme Salgado Add a test for hwpack.run_local_atexit_funcs() and fix an undefined name in it 2011-02-01 Loïc Minier Post-release version bump to 0.4.3.1 2011-02-01 Loïc Minier Release 0.4.3 2011-02-01 Loïc Minier Bump version after release. 2011-02-01 Loïc Minier Make names "tarball" and "tarball.gz" configurable and move mkdir() to setUp() instead of __init__(). 2011-02-01 Loïc Minier Test find_command(). 2011-02-01 Loïc Minier Add and use find_command() which will search a linaro-image-tools command in the current directory if running from a bzr checkout or in PATH otherwise; this helps with running the testsuite on an installed system; LP: #709518. 2011-02-01 Guilherme Salgado Some tweaks to the test added in the previous commit and add a safety net to avoid bugs similar to #710971 2011-02-01 Loïc Minier Rework tarball creation to avoid these warnings: tar: Removing leading `/' from member names tar: /tmp/tmpNhHGL5: file changed as we read it 2011-01-31 Andy Doan add test case for atexit fix 2011-01-31 Guilherme Salgado A test for the fix in the previous commit 2011-01-31 Guilherme Salgado Fix the bug, but a test is tricky to write 2011-01-31 Guilherme Salgado Mention sfdisk on the requirements to run the test suite 2011-01-29 Loïc Minier Mock getuid() when testing cmd_runner and test both the sudo needed and sudo not needed cases. 2011-01-29 Loïc Minier linaro-media-create: Use linaro_media_create.cmd_runner's Popen rather than the one from subprocess; avoids hardcoding sudo in the commands. 2011-01-29 Loïc Minier Remove useless import. 2011-01-29 Loïc Minier linaro-hwpack-install: only call sudo when running as non-root. 2011-01-29 Loïc Minier cmd_runner: Don't prepend sudo if we're root. 2011-01-28 Loïc Minier Post-release bump to 0.4.2.1. 2011-01-28 Loïc Minier Release 0.4.2 2011-01-28 Guilherme Salgado Update the URL of l-i-t's Launchpad home to use https 2011-01-28 Guilherme Salgado s/2010,2011/2010, 2011/ on all copyright notices 2011-01-28 Guilherme Salgado Use linaro-dev@linaro.org instead of infrastructure@ as the latter is internal only 2011-01-28 Guilherme Salgado Add a GPL2 header to all hwpack files 2011-01-28 Guilherme Salgado Add license headers to all media-create files 2011-01-28 Loïc Minier GPLv3+, not v2+ as some bits are only v3+. 2011-01-28 Loïc Minier Sign tarball. 2011-01-28 Loïc Minier Update setup.py and commit the version change. 2011-01-28 Loïc Minier Run testsuite before release. 2011-01-28 Loïc Minier No need to list the packages explicitly. 2011-01-28 Guilherme Salgado Change OMAP configs to use the correct serial tty depending on the kernel version that is installed 2011-01-28 Guilherme Salgado Make it possible to change the serial_tty of OmapConfig at run time 2011-01-28 Loïc Minier Switch to DistutilsExtra, this has the nice side effect of shipping tests. 2011-01-28 Loïc Minier Drop unused imports. 2011-01-28 Guilherme Salgado Update tests/integration.txt to match the current state of the code. We should remember to run them more often. 2011-01-28 Loïc Minier Check whether tree is dirty with bzr status before rolling a release. 2011-01-28 Loïc Minier Drop packaging; will be maintained in Ubuntu. 2011-01-28 Loïc Minier Add misc metadata: description, license, URL, author information... 2011-01-28 Loïc Minier Install scripts. 2011-01-28 Loïc Minier Harcode version in setup.py instead of reading it from the packaging which will go away. Use version 0.4.1.1 as to distinguish people using bzr checkouts from the 0.4.1 release. 2011-01-28 Loïc Minier Rename to linaro-image-tools. 2011-01-28 Loïc Minier Add simple script to roll a release; needs some setup.py fixes before it can be used though. 2011-01-28 Guilherme Salgado Add list of dependencies for running l-m-c to README 2011-01-28 Loïc Minier Add shebang and make setup.py +x. 2011-01-27 Guilherme Salgado Fix install_package_providing() to wait() for its child 2011-01-27 Guilherme Salgado Add a test to make sure ensure_partition_is_not_mounted is not called when generating image files 2011-01-27 Guilherme Salgado Only ensure a partition is not mounted before running mkfs when we're burning an SD card. 2011-01-26 Andy Doan add exception handling to hwpack cleanup logic 2011-01-26 Loïc Minier Use blkid -p to force probing and ignore the cache in this case since it's useless; call blkid within sudo as it needs direct device access. 2011-01-26 Guilherme Salgado Use the board's mmc_part_offset (instead of the media's partition count) to figure out the boot/root partition numbers 2011-01-26 Loïc Minier Test that get_uuid() runs blkid correctly. 2011-01-26 Loïc Minier Only lookup rootfs UUID when needed. 2011-01-26 Loïc Minier fdisk isn't used anymore. 2011-01-26 Loïc Minier Don't pass -U to mkfs helpers; instead get the UUID from blkid after creating the root filesystem; LP: #705454. 2011-01-25 Guilherme Salgado Get rid of the mx51evk-specific code in create_partitions 2011-01-25 Andy Doan ensure local_atexit functions always run 2011-01-25 Loïc Minier TestGetBootCmd.test_overo(): update for new OveroConfig console settings; fixes testsuite. 2011-01-25 Guilherme Salgado Do not try to clean stuff up when there's nothing to cleanup. 2011-01-25 Loïc Minier Move docstring in proper place. 2011-01-25 Loïc Minier TestBuildSteps: also move self.mock_all_boards_funcs() inside setUp() 2011-01-25 Loïc Minier TestBootSteps: move make_boot_files() logic in a wrapper function. 2011-01-25 Loïc Minier TestBootSteps(): only set funcs_calls = [] in setUp() rather than duplicating this in all tests. 2011-01-24 Andy Doan update serial settings for overo to match natty kernel 2011-01-24 Guilherme Salgado Make sure we create an extra (non-Data) partition for mx51evk boards 2011-01-23 Loïc Minier Test steps (functions) of linaro_media_create.boards called by various boards' configs with expected sequence. 2011-01-22 Loïc Minier Drop spaces around env keyword argument. 2011-01-22 Loïc Minier Add boot_device_or_file parameter to _make_boot_files() class method in VexpressConfig() class; was added specifically for imx51, but not added to all other boards. 2011-01-22 Loïc Minier Rename rootfs to rootfs_dev to avoid shadowing rootfs import; rename bootfs to bootfs_dev for consistency. 2011-01-22 Loïc Minier Remove unused code. 2011-01-22 Loïc Minier Set NO_PKG_MANGLE=1 when calling dpkg-deb; LP: #706144. 2011-01-22 Loïc Minier Set LC_ALL=C when running dpkg-deb; LP: #706140. 2011-01-19 Andy Doan change overo's sub_arch to kernel_suffix as per commit 247 2011-01-19 Andy Doan add test case for overo board 2011-01-19 Loïc Minier 0.4.1 changelog 2011-01-19 Loïc Minier Build-dep on qemu-kvm for qemu-img. Build-dep and Recommend on command-not-found for CommandNotFound. Build-dep on u-boot-tools | uboot-mkimage for mkimage. 2011-01-19 Loïc Minier Drop util-linux from Depends as it's Essential. 2011-01-19 Loïc Minier Uppercase Python in python-linaro-media-create long description. 2011-01-19 Loïc Minier Disable pkgbinarymangler when running the testsuite as it might cause some stderr output which confuses it. 2011-01-19 Loïc Minier linaro_media_create.rootfs.move_contents(): Sort files as to get a reliable set of commands which we can test. 2011-01-19 Loïc Minier Update changelog for 0.4 2011-01-19 Loïc Minier Remove omapfb.debug=y from kernel cmdline as the DSS2 code is mature now. 2011-01-18 Andy Doan support for overo boards 2011-01-18 Guilherme Salgado Rename BoardConfig.sub_arch to BoardConfig.kernel_suffix and fix PandaConfig.kernel_suffix 2011-01-18 Guilherme Salgado Mention the bug number in an XXX comment and create an OmapConfig which is extended by both Beagle and Panda configs 2011-01-18 Guilherme Salgado Fix the console boot arg for the beagle config 2011-01-18 Guilherme Salgado Make l-m-c find the MLO file to install on omap boards regardless of where x-loader puts them 2011-01-18 Dave Martin Remove spaces from around '=' in kernel command-lines. 2011-01-18 Guilherme Salgado Add the new utils.py module which somehow was not included in the previous commit 2011-01-18 Guilherme Salgado Recommends: qemu-kvm* and btrfs-tools and stop depending on btrfs-tools (not always needed) and e2fsprogs (which is essential and thus we can assume to always be installed) 2011-01-17 Guilherme Salgado Fix Depends: and Suggests: to match the real requirements and also change ensure_command to lookup the package which provides the given command 2011-01-14 Guilherme Salgado Get rid of remove_binary_dir.py as the binary/ dir is now created in a temp directory that is always removed when the script exits. 2011-01-14 Guilherme Salgado Fix 3 tests that my last commit broke 2011-01-14 Guilherme Salgado Add a comment mentioning the UDisks bug I've reported 2011-01-13 Guilherme Salgado Use UDisks to lookup the correct device files for the boot/root partitions 2011-01-12 Guilherme Salgado Move board-specific code from populate_boot.py to boards.py 2011-01-12 Guilherme Salgado Use classes for board config 2011-01-11 Guilherme Salgado Rename the media_create python package to linaro_media_create, to match the name of the deb package (python-linaro-media-create) 2011-01-11 Guilherme Salgado Make the --binary argument of l-m-c required 2011-01-11 Guilherme Salgado Finish tests for get_board_config 2011-01-11 Guilherme Salgado Update integration.txt to call the functions directly instead of using python -m as we used to do 2011-01-11 Guilherme Salgado Remove some debug code I'd left behind 2011-01-11 Guilherme Salgado Remove l-m-c and rename l-m-c.py to l-m-c 2011-01-11 Guilherme Salgado Update linaro-media-create.py to call get_board_config() correctly 2011-01-11 Guilherme Salgado Tests for get_board_config 2011-01-10 Guilherme Salgado A few more unrelated tweaks 2011-01-10 Guilherme Salgado Fix test_make_boot_script, which broke after the signature of make_boot_script was changed 2011-01-10 Guilherme Salgado Fix media_create.hwpack which was relying on the atexit funcs being called right after its job was done. That's no longer the case as it's no longer executed as a separate python process 2011-01-10 Guilherme Salgado Fix a few minor issues introduced in l-m-c.py 2011-01-10 Guilherme Salgado Implement the cleanup function in l-m-c.py and call confirm_device_selection_and_ensure_it_is_ready, which was missing in l-m-c.py 2011-01-08 Loïc Minier u-boot -> u-boot-tools (natty). 2011-01-07 Guilherme Salgado Checkpoint commit; there's still lots of TODOs and FIXMEs 2011-01-07 Guilherme Salgado Make the --hwpack-force-yes argument to l-m-c work again. No tests as this code is being killed on another branch of mine. 2011-01-06 Guilherme Salgado Show progress when running apt-get update in linaro-hwpack-install 2011-01-06 Guilherme Salgado Once the user confirms the device selection on l-m-c, ensure all of its partitions are umounted to prevent failures further on 2011-01-05 Guilherme Salgado Use the UDisks dbus interface to check whether or not a given partition is mounted 2011-01-05 Guilherme Salgado Minor tweaks and a couple tests for ensure_filesystem_is_not_mounted 2011-01-05 Guilherme Salgado Unmount bootfs/rootfs before attempting to create a new filesystem on them. Needed because KDE apparently triggers the automounter after a disk is repartitioned 2011-01-05 Guilherme Salgado Add tests for install_hwpacks and co 2011-01-05 Guilherme Salgado Add some docstrings and fix some minor issues 2011-01-05 Loïc Minier Fix misc issue with python-linaro-media-create package: - add missing dep of linaro-image-tools on python-linaro-media-create - uppercase Python in description - drop superfluous python-apt dep 2011-01-05 Loïc Minier Wrap build-deps and deps 2011-01-04 Guilherme Salgado Port install_hwpacks to python 2011-01-04 Guilherme Salgado Move some remaining board-specific configs to the one block where they should all be defined 2011-01-04 Guilherme Salgado Use a temp file rather than stdout to pass data from a python script to l-m-c 2011-01-04 Loïc Minier Revert r220 as media_create/partitions.py still uses parted (along python-parted). 2011-01-03 Guilherme Salgado Escape a percent character in a string of check_device._select_device. (fix provided by David Epping, for bug 695662) 2011-01-03 Guilherme Salgado Print the correct device size in check_device._print_devices. (fix provided by David Epping, for bug 695659) 2010-12-23 Loïc Minier Prepare changelog for 0.4 2010-12-23 Loïc Minier Drop parted dependency. 2010-12-22 Loïc Minier Uppercase Python in description (lintian) 2010-12-22 Loïc Minier Add a python >= 2.6.5-1~ build-dep for dh_python2. 2010-12-22 Loïc Minier m 2010-12-22 Loïc Minier Use python2 sequence rather than python-support -- python-support build-dep was missing anyway. 2010-12-22 Loïc Minier Use a stricter dependency on python-hwpack as to avoid unhappy mixups 2010-12-22 Loïc Minier Wrap python-hwpack's Depends to get cleaner diffs 2010-12-22 Loïc Minier Update runtime Depends per README 2010-12-22 Loïc Minier Fix empty Vcs-Bzr 2010-12-22 Loïc Minier Build-dep on python-dbus, python-debian (>= 0.1.16ubuntu1), and python-parted per README 2010-12-22 Loïc Minier Bump python-testtools build-dep to >= 0.9.6 per README 2010-12-22 Loïc Minier Wrap build-deps to get cleaner diffs 2010-12-20 Martin Ohlsson Refactoring of check_device.py. 2010-12-20 Loïc Minier Depends on u-boot instead of uboot-mkimage as this is where mkimage is shipped in natty. 2010-12-17 Paul Larson Don't fail if there are spaces in the path. 2010-12-17 Martin Ohlsson Replace StdoutToDevnull fixture with a MockSomething fixture. 2010-12-17 Martin Ohlsson Improve formatting of device printout. 2010-12-17 Guilherme Salgado Port prepare_partitions to python 2010-12-17 Michael Hudson create an external manifest 2010-12-17 Michael Hudson support different extensions in HardwarePack.filename 2010-12-17 Michael Hudson WARNING is a more appropriate level 2010-12-17 Michael Hudson warn if a package supplied with --local-deb does not end up in the hwpack 2010-12-17 Michael Hudson document & give a default to --local-deb 2010-12-17 Michael Hudson finish making test_builder tests less repetitive 2010-12-17 Michael Hudson start to make test_builder tests less repetitive 2010-12-17 Michael Hudson include local debs in the hwpack even if the config file does not include them 2010-12-17 Michael Hudson remove special casing of None keys in HardwarePack.sources 2010-12-17 Michael Hudson de-XXX 2010-12-17 Michael Hudson rename destination of --local-deb to indicate pluralness 2010-12-16 Guilherme Salgado Fix arg parsing in rootfs.py 2010-12-16 Guilherme Salgado Port populate_rootfs to python 2010-12-16 Loïc Minier Drop a bunch of unused dependencies. 2010-12-16 Martin Ohlsson Replace fdisk command with the UDisks dbus interface. 2010-12-16 Michael Hudson don't mark the dependency package as automatically installed; hide stderr from dpkg --get-selections 2010-12-16 Michael Hudson grr 2010-12-16 Michael Hudson test the prefer_label stuff 2010-12-16 Michael Hudson make sources_entry_for_debs label support less hacky & tested 2010-12-16 Michael Hudson small tidy 2010-12-16 Michael Hudson make test pass, some hacking though 2010-12-16 Michael Hudson the final test! 2010-12-16 Michael Hudson consider the local debs as a package source when building the hwpack 2010-12-16 Michael Hudson make sure that the Packages file created by sources_entry_for_debs has Filename: entries that work 2010-12-16 Michael Hudson implement sources_entry_for_debs 2010-12-16 Michael Hudson split TemporaryDirectoryManager out of PackageMaker, add LocalArchiveMaker 2010-12-16 Michael Hudson if you pass a source with a key of "None" to add_apt_sources(), it is not added to the hwpack 2010-12-16 Michael Hudson allow matching the "content" (i.e. the file listing) of directories in tarfiles 2010-12-16 Michael Hudson add a modicum more logging 2010-12-16 Michael Hudson add local-debs option, which currently does not do anything 2010-12-15 Martin Ohlsson Add TODO comment about using UDisk dbus interface to avoid subprocess.Popen. 2010-12-15 Guilherme Salgado Update two python modules that were using subprocess.Popen() to use cmd_runner.run() instead 2010-12-15 Martin Ohlsson Refactor check_device.py. - sys.stdout.write replaced with print. - check_device() returns boolean instead of exit code. 2010-12-15 Michael Hudson this probably allows you to pass --hwpack to linaro-media-create more than once 2010-12-15 Michael Hudson copy StartsWith from testtools, as that is only present in testtools 0.9.7+ and reenable test 2010-12-14 Martin Ohlsson Move get_extended_env() from check_device to cmd_runner. 2010-12-14 Martin Ohlsson Refactor for PEP-8 compliance. 2010-12-14 Guilherme Salgado Merge create_partitions.py with partitions.py 2010-12-14 Guilherme Salgado Disable a test which has a non-documented requirement and thus fails on Maverick 2010-12-14 Guilherme Salgado A few tweaks a bug fixes suggested by reviewers 2010-12-14 Martin Ohlsson Add test for deselecting a device. 2010-12-14 Martin Ohlsson Add python port of check_device. 2010-12-14 Michael Hudson remove patch from james again 2010-12-14 Michael Hudson the fix 2010-12-14 Michael Hudson tests that are the core of the issue 2010-12-14 Michael Hudson patch from james 2010-12-14 Michael Hudson this is probably more robust 2010-12-14 Michael Hudson only mark packages newly installed with the hwpack as automatically installed 2010-12-14 Michael Hudson add a small essay to linaro-hwpack-install explaining why it does what it does 2010-12-14 Michael Hudson redo linaro-hwpack-install to use markauto on the contents of the hwpack 2010-12-14 Michael Hudson mostly revert r199 2010-12-13 Guilherme Salgado Update a few cmd_runner.run() callsites that were not wait()ing 2010-12-13 Guilherme Salgado Fix remaining tests, make sure everything is tested and clean up some bits 2010-12-13 Guilherme Salgado Fix some tests and add more XXXs/TODOs 2010-12-13 Michael Hudson de-magic a test 2010-12-13 Michael Hudson explain a test 2010-12-13 Michael Hudson docstring & test for MatchesPackageRelationshipList 2010-12-10 Guilherme Salgado Checkpoint commit 2010-12-10 Michael Hudson have the generated packages break/replace each other 2010-12-10 Michael Hudson duh 2010-12-10 Michael Hudson rollback format bump & make linaro-hwpack-install detect new-style hwpacks differently 2010-12-10 Michael Hudson create two dependency packages, one which depends on the contents of the hwpack, one which depends on the spec 2010-12-10 Michael Hudson classic 180 degree wrong logic bug 2010-12-10 Michael Hudson doh 2010-12-10 Michael Hudson support --hwpack-force-yes in linaro-media-create 2010-12-10 Michael Hudson oops 2010-12-10 Michael Hudson add --force-yes option to linaro-hwpack-install, rewriting its argument parsing on the way 2010-12-09 Guilherme Salgado Use the new cmd_runner.Popen instead of subprocess.Popen 2010-12-09 Guilherme Salgado Create a subprocess.Popen subclass which raises an error when the process exits with a non-zero return code. This is useful in cases when cmd_runner can't be used but we still want to raise an error on non-zero exit. 2010-12-09 Michael Hudson rename dependency package to be prefixed with hwpack- 2010-12-09 Michael Hudson doh! 2010-12-09 Michael Hudson add a docstring 2010-12-08 Guilherme Salgado Port more bits of create_partitions to python 2010-12-08 Guilherme Salgado Port create_partitions to python 2010-12-08 Guilherme Salgado Fix test_run_mkimage so that it doesn't run a subprocess using sudo 2010-12-08 Michael Hudson sort test module names in hwpacks/tests/__init__.py & add test_testing 2010-12-08 Michael Hudson disable test_run_mkimage because it keeps asking for my sudo password 2010-12-08 Michael Hudson more vertical separation 2010-12-08 Michael Hudson support format 2.0 in linaro-media-create 2010-12-08 Michael Hudson bump hwpack format to 2 2010-12-08 Michael Hudson test add_dependency_package 2010-12-08 Michael Hudson oops, PackageMaker.make_package did not get the file name right when arch != all 2010-12-08 Michael Hudson simplifications 2010-12-08 Michael Hudson add AfterPreprocessing.__str__ 2010-12-08 Michael Hudson more documentation, some duplication removed 2010-12-08 Michael Hudson rename ZipMatchers to EachOf, document it and (doc)test it 2010-12-08 Michael Hudson replace Packages file "parsing" code with a real parser from debian.deb822 2010-12-08 Michael Hudson thoroughly rewrite the reporting part of MatchesSetwise and test that side of things 2010-12-08 Michael Hudson maybe this is a bit clearer 2010-12-07 Martin Ohlsson Remove import of shutil in test_media_create.py. 2010-12-07 Martin Ohlsson Use tempdir fixture as param when creating ChangeCurrentWorkingDirFixture. 2010-12-07 Martin Ohlsson Add check for absolute path. 2010-12-07 Martin Ohlsson Unpack tarball in system tempdir. 2010-12-07 Michael Hudson build a dependency package as part of building a hwpack 2010-12-07 Michael Hudson test that hardware matching is independent of package order & last fix to actually make it so 2010-12-07 Michael Hudson use MatchesAsPackagesFile in IsHardwarePack 2010-12-07 Michael Hudson generalize how TarfileHasFile matches content 2010-12-07 Michael Hudson wth, actual working impl of last commit 2010-12-07 Michael Hudson kill off MatchesPackage.fromPackage constructor 2010-12-07 Michael Hudson finally, MatchesAsPackagesFile 2010-12-07 Michael Hudson implement MatchesSetwise 2010-12-07 Michael Hudson ok, this is a bit crazy: add a MatchesStructure matcher that is a bit reminscent of destructuring-bind in common lisp make explicit the attributes which must be compared to consider FetchedPackages equal use the above two things to make a MatchesPackage matcher 2010-12-06 Guilherme Salgado A few tweaks 2010-12-03 Guilherme Salgado Port the meat of populate_boot to python 2010-12-03 Guilherme Salgado Implement a new fixture which uses MockSomethingFixture to mock cmd_runner.do_run 2010-12-03 Guilherme Salgado Fix the mocking of do_run in tests to not use the context manager protocol as that may fail to undo the mocking in some circumstances 2010-12-03 Guilherme Salgado REAME now states the specific version of python-debian that is needed to run the test suite 2010-12-02 Guilherme Salgado Update README with more packages that are needed to run the test suite 2010-12-01 Guilherme Salgado Refactor a bunch of things taking into account Peter's and James' comments 2010-12-01 Martin Ohlsson Fix sudo for remove_binary_dir.py. 2010-12-01 Michael Hudson don't compare content when comparing FetchedPackages -- the md5 is enough 2010-12-01 Michael Hudson michael will always run the tests before committing to trunk 2010-11-30 Guilherme Salgado Add an XXX for a question to the reviewer 2010-11-30 Guilherme Salgado Create a new run() function to be used when running external commands as subprocesses 2010-11-30 Guilherme Salgado Refactor the meat of populate_boot() into smaller functions that can be reused for all boards 2010-11-30 Martin Ohlsson Rename test fixtures and organize imports. 2010-11-30 Michael Hudson add a FetchedPackage.__hash__ method 2010-11-30 Michael Hudson test and fix for bug found by pyflakes 2010-11-30 Michael Hudson fix lint in test_tarfile_matchers 2010-11-30 Michael Hudson michael will always run the tests before committing to trunk 2010-11-30 Michael Hudson another comment 2010-11-30 Michael Hudson better comment 2010-11-29 Martin Ohlsson Move fixtures to separate module. 2010-11-29 Michael Hudson changes following review 2010-11-28 Martin Ohlsson Use fixtures for setUp and tearDown. 2010-11-27 Martin Ohlsson Apply subprocess args with shell=False. 2010-11-26 Martin Ohlsson Add tests for unpack_binary_dir. 2010-11-26 Michael Hudson allow the architecture to be supplied 2010-11-26 Michael Hudson test that created package had architecture: all 2010-11-25 Martin Ohlsson Add test for remove_binary_dir. 2010-11-25 Martin Ohlsson Port unpack_binary_tarball to python (includes porting of remove_binary_dir). 2010-11-25 Michael Hudson more tests & implementation for from_deb 2010-11-24 Michael Hudson first test passing for PackageFetcher.from_deb 2010-11-24 Michael Hudson create debs named like dpkg-name would move them too 2010-11-23 Guilherme Salgado First attempt at packaging media_create. Works but has a few things that need fixing 2010-11-23 Guilherme Salgado Port ensure_command to python 2010-11-23 Michael Hudson write out some more tests 2010-11-23 Michael Hudson depends and pre-depends fields work and invalid fields do not 2010-11-23 Michael Hudson use python-debian instead of python-apt for handling .deb files 2010-11-23 Michael Hudson test version too 2010-11-23 Michael Hudson check something about the created package 2010-11-23 Michael Hudson first test that creates a .deb 2010-11-23 Michael Hudson start with a fixture/context manager that provides temporary directories this must have been done so many times before... 2010-11-22 Guilherme Salgado Add instructions for running the tests and create a new top-level test_suite, used to run the whole test suite with subunit 2010-11-18 Guilherme Salgado Port the meat of create_boot_cmd to python and call that from l-m-c 2010-11-18 Martin Ohlsson Emphasize the hwpack installation in target rootfs. 2010-11-17 Guilherme Salgado Refactor create_boot_cmd to make it not aware of 2010-11-17 Martin Ohlsson Notify user that "apt-get" will be run in target roofs. 2010-11-17 Martin Ohlsson Replace "sfdisk -l" with "fdisk -l". 2010-11-12 Guilherme Salgado Remove an unnecessary call to get_device_by_id and fix some indentation 2010-11-12 Guilherme Salgado Properly indent code. I wish we had some form of watchdog to at least prevent s in the code 2010-11-11 Guilherme Salgado Add a test for HardwarePack.FORMAT this time. 2010-11-09 Steve Langasek releasing version 0.3 2010-11-09 Steve Langasek document bug #651905 in the changelog 2010-11-09 Dave Martin Implement proper cleanup handling on signal / exit, so that we don't leave clutter behind on error. This also fixes the message at the end of a successful install about being unable to unmount proc. LP: #616385, #660196, #667394, #643601, #627383. 2010-11-09 Dave Martin Remove suprious call to cleanup_sd before $DEPLOY_STEPS. 2010-11-04 James Westby Ensure that recommends are installed in the hwpack. 2010-11-04 James Westby Add INFO output in normal operation. 2010-11-04 James Westby Add --debug option to hwpack-create. 2010-11-04 Alexander Sack add support for ux500; inspired by Frederick Gueydon 2010-11-02 Steve Langasek Use --numeric-owner argument to tar when unpacking, so that files get the right owner on boot of the image. LP: #652121. 2010-11-02 Dave Martin Only run losetup -d on loop devices, not random partitions 2010-11-02 Dave Martin Simplify shell exit from the cleanup / signal handlers 2010-11-02 Dave Martin Fixed harmless error message typo 2010-11-01 Dave Martin Remove incorrect, overzealous sanity-check when cleaning up $TMP_DIR 2010-10-27 Loïc Minier Drop spurious single quote character in boot_args_options for beagle|igep; thanks Peter Maydell; LP: #667400. 2010-10-22 Jamie Bennett Consolidate code in create_boot_cmd() and fix serial port typo on Panda 2010-10-22 Jamie Bennett Add TI OMAP4 Pandaboard support 2010-10-22 Dave Martin Remove unnecessary extra unset IMAGE_FILE 2010-10-21 Dave Martin Actually trap interrupts to signal_handler, as originally intended. 2010-10-21 Dave Martin Tidy cleanup code formatting to be consistent with the rest of the script. 2010-10-21 Dave Martin Use sudo to remove loop devs so this works if not running as root. 2010-10-21 Dave Martin Try to clean up mess on exit, error, ^C, and return meaningful exit status. 2010-10-19 matt.waddel@linaro.org <> Fixed typo in comment 2010-10-18 matt.waddel@linaro.org <> Script style and comment corrections from Loic's review. 2010-10-18 matt.waddel@linaro.org <> Combine create_boot_cmd() func into BOOTFS_STEP step. 2010-10-18 matt.waddel@linaro.org <> Since the script isn't MMC/SD specific rename parts to usage: MMC1 -> BOOTFS MMC2 -> ROOTFS get_mmcs_by_id() -> get_device_by_id() 2010-10-18 matt.waddel@linaro.org <> Added ability to not repartition device 2010-10-18 matt.waddel@linaro.org <> Add the --no-rootfs and --no-bootfs options. Refactored the deploy actions so they can be run independently 2010-10-18 Guilherme Salgado Ensure the format version of hwpacks don't contain white spaces as that is not necessary and could break linaro-hwpack-install 2010-10-17 matt.waddel@linaro.org <> Add the --no-rootfs and --no-bootfs options. Refactored the deploy actions so they can be run independently 2010-10-17 matt.waddel@linaro.org <> Remove unused MMC3 from get_mmcs_by_id() 2010-10-17 matt.waddel@linaro.org <> Make check_mmc() a generic device scanner, switch name to check_device() 2010-10-15 Steve Langasek Fix lookup of linaro-hwpack-install to work correctly when l-m-c is on the path. Closes: #652669. 2010-10-14 James Westby Invert the --install-latest check so that it has the right meaning. 2010-10-14 Dave Martin Don't truncate the image file when installing U-Boot for mx51evk 2010-10-14 Dave Martin Fix mx51evk U-Boot installation to work with --image_file 2010-10-14 Dave Martin Shawn Guo's fixes for mx51evk 2010-10-13 Tom Gall add warning for when --hwpack (HWPACK_FILE) isn't set 2010-10-13 matt.waddel@linaro.org <> With the image alignment bug (lp:658511) fixed it's not required to have a special boot.cmd script for the qemu image anymore. 2010-10-13 matt.waddel@linaro.org <> Round the size of the raw disk image up to a multiple of 256K so it is an exact number of SD card erase blocks in length. Otherwise Linux under qemu cannot access the last part of the card and is likely to complain that the last partition on the disk has been truncated. 2010-10-13 Guilherme Salgado Tell users to try a newer hwpack-install script when the format is not supported 2010-10-12 Steve Langasek drop the convoluted dirname handling, sh always gives us the full path in $0 so dirname "$0" is always what we want on the path - thanks to Peter Maydell for noticing! 2010-10-11 matt.waddel@linaro.org <> Remove the final gzip file creation step in an --image_file run 2010-10-10 Steve Langasek fix a bug in how we look up linaro-hwpack-install when l-m-c is both on the path and in a bzr checkout 2010-10-09 Steve Langasek fix sfdisk invocation: - suppress stderr, which is quite noisy on any system that uses LVM - fix the grep commandline to actually match on the string 'Disk' instead of on the set of letters [Disk]... 2010-10-09 Steve Langasek if linaro-media-create is not on the path, add it there; then look for linaro-hwpack-install on the path 2010-10-08 Steve Langasek revert revision 134 due to numerous issues; see https://code.launchpad.net/~tom-gall/linaro-image-tools/rsync-speedup/+merge/37791 for more information 2010-10-08 matt.waddel@linaro.org <> Added 2 quoting suggestions from James Westby's review. 2010-10-08 Alexander Sack linaro-media-create: use correct boot and root devices for target device - fix regression from rev 133 2010-10-08 Dave Martin Fix off-by-one-sector error determining partition sizes from fdisk output. 2010-10-08 Dave Martin Fixed misparse of fdisk output. 2010-10-08 Dave Martin Satisfy missing command dependency for qemu-img. 2010-10-07 Tom Gall adjust tar to use --numeric-owner for UID/GID when untaring 2010-10-07 Dave Martin Fix some more straightforward vulnerbilities regarding argument quoting. 2010-10-06 Tom Gall copy the rootfs in parallel with rsync 2010-10-06 Guilherme Salgado Check that the hwpack's format is supported before trying to do anything with it. 2010-10-06 Alexander Sack add MMC_PART_OFFSET to fix part id for non-fs-data part layouts like mx51evk. In turn use MMC2 for root= cmdline opt and MMC1 for UBOOT_PART flash-kernel.conf value 2010-09-30 Loïc Minier Drop non-FS data partition for everything but mx51evk since the builtin u-boot wont find the boot script otherwise. 2010-09-30 Loïc Minier Set LC_ALL=C when entering the chroot as it has no locales. 2010-09-30 Loïc Minier Fix fatload partition since non-FS data addition. 2010-09-30 Loïc Minier Unset swap related vars at the top. 2010-09-30 Loïc Minier Refactor some ugly grep usages into case statements. 2010-09-30 Loïc Minier Pass LC_ALL=C to commands of which we parse the possibly localized output. 2010-09-30 Loïc Minier Avoid repeating the kernel and initrd physical load addresses. 2010-09-30 Loïc Minier Drop "mmc init" since boot.cmd/.ini is loaded from the MMC by u-boot, and "mmc init" has necessarily been run already. 2010-09-30 Loïc Minier Add non-FS data partition and factor the mx51evk case in the general case. 2010-09-30 Loïc Minier Misc syntax tweaks. 2010-09-30 Shawn Guo Address review comments 2010-09-29 Shawn Guo Add --dev option imx51 support 2010-09-29 Steve Langasek DEVIMAGE, not DEVNAME. 2010-09-28 Loïc Minier Quote uses of BOOT_DISK and ROOT_DISK. 2010-09-28 Loïc Minier Drop CODENAME and use "boot script" as the description of the boot script. 2010-09-28 James Westby Add python-argparse to dependencies. Thanks Guilherme. 2010-09-28 Loïc Minier Drop broken test (DEVNAME instead of DEVIMAGE) and require uboot-mkimage unconditionally. 2010-09-28 Loïc Minier Require a valid DEVIMAGE before starting; this is required by populate_boot later anyway, so best to fail early and have a single copy of the user-visible error-messages. 2010-09-28 Loïc Minier Consistently check for vmlinuz-*-$flavor and initrd-*-$flavor and for linaro- in the flavor name. 2010-09-28 Loïc Minier Fix indentation of cat and exit in usage(). 2010-09-28 Loïc Minier Drop unreachable code. 2010-09-28 Loïc Minier Fix typo: paramater vs. parameter. 2010-09-28 Loïc Minier Fix indentation to 1 or 3 spaces to be 0 or 2 spaces. 2010-09-28 Loïc Minier Factorize the mkfs.extN checks and fix ext2 support. 2010-09-28 Loïc Minier Drop useless FS_TYPE var. 2010-09-28 Loïc Minier Refactor check_fs_type into a case statement. 2010-09-28 Loïc Minier Fold "10.05" in constants at the top. 2010-09-28 Loïc Minier Drop all uses of "cd" since it's always to "${DIR}" and DIR is PWD anyway. 2010-09-28 Loïc Minier Drop "cd" before "sync" as sync is system-wide anyway. 2010-09-28 Loïc Minier Vexpress: fix missing sudo before cp-ing the bootloader. 2010-09-28 Loïc Minier Harmonize shell syntax across boards. 2010-09-28 Loïc Minier Drop incorrect "|| true" guard after mkdir -p; mkdir -p wont fail if the directory already exists, but only if it can't be created, which should fail this script. 2010-09-28 Loïc Minier Drop useless MLO and UBOOT vars; used only once. 2010-09-28 Loïc Minier Beagle: don't test for MLO and u-boot presence; just copy them unconditionally, it's a bug in the images if these are missing. In theory, this might prevent writing older images if these files ever went missing, but these images are unsupported anyway. 2010-09-28 Loïc Minier Set MLO and UBOOT in beagle-specific section as these are specific to beagle. 2010-09-24 James Westby Package the hardware pack tools. 2010-09-24 James Westby Tweaks from review. Thanks Guilherme. 2010-09-23 Guilherme Salgado Refactor l-m-c so that we don't need extra unpack/repack steps when installing hwpacks. 2010-09-23 Guilherme Salgado Fix some places in l-m-c that were using ROOT_DIR when they meant ROOT_DISC and some changes suggested by Peter Maydell 2010-09-21 Guilherme Salgado First round of refactoring: use constants for a couple things and remove some unneeded code 2010-09-20 Guilherme Salgado Ignore empty/commented lines when comparing a hwpack's apt sources with the system ones 2010-09-20 Guilherme Salgado Re-add a sudo I accidentally removed 2010-09-20 Guilherme Salgado When installing an hwpack as part of linaro-media-create, make sure we unmount the chroot's /proc before trying to remove it. 2010-09-17 Guilherme Salgado Change linaro-hwpack-install to only install the sources.list.d files that have at least one line not present in the existing apt sources 2010-09-17 Guilherme Salgado Make sure the temp dir is used to store things copied from the tarball when installing hwpacks 2010-09-17 Guilherme Salgado Make sure the temp directory is removed when the script exits, plus a few minor tweaks 2010-09-16 Guilherme Salgado Fix a bug that was causing the newly generated tarball to have a './' prefix in all paths 2010-09-16 Guilherme Salgado Add a new --hwpack argument to linaro-media-create 2010-09-16 Guilherme Salgado Make sure we don't add duplicate apt sources when installing a hardware pack 2010-09-15 Steve Langasek split out u-boot handling into common code, instead of using two completely different board-specific approaches 2010-09-15 Steve Langasek add sudo to a couple more commands in linaro-media-create, so the script itself can run as non-root as intended. 2010-09-15 Steve Langasek only enforce the presence of commands we're actually using; this lets us automatically pull in btrfs-tools when btrfs is specified. 2010-09-15 Steve Langasek more shuffling; a handful of argument handling options require external commands 2010-09-15 Steve Langasek make sure we unpack uboot and x-loader in time to use them for the boot partition 2010-09-15 Steve Langasek shuffle the file around so we can be more selective with ensure_command after argument parsing 2010-09-15 Steve Langasek fix the paths used for looking up x-loader and u-boot binaries in the tarball; these don't belong in /boot at all, pull them from /usr/lib. 2010-09-15 James Westby Use a raw string for the regex, thanks Michael. 2010-09-15 James Westby Some improvements to PackageFetcher. 2010-09-15 James Westby Thread Provides, Replaces and Breaks through to the Packages files. 2010-09-15 Guilherme Salgado Standardize indentation of linaro-media-create 2010-09-14 Steve Langasek changelog entry 2010-09-14 Steve Langasek no longer using the 'omap' uboot flavor support in live-helper; instead, run the mkimage commands directly for omap 2010-09-14 James Westby Add __ne__ and remove __hash__ from the Mismatches. Thanks Zygmunt. 2010-09-14 James Westby Define __ne__ and remove __hash__ on FetchedPackage. Thanks Zygmunt. 2010-09-14 James Westby Actually exclude the packages from the hardware pack. 2010-09-14 James Westby Add assume-installed to the configuration. 2010-09-14 James Westby Simplify the implementation of fetch_packages. 2010-09-14 Guilherme Salgado Make sure the packages contained in the hwpack take precedence over the ones coming from other sources. Also fixes handling of the --install-latest argument. 2010-09-13 James Westby Further improvements to fetch_packages. 2010-09-13 James Westby Use a parameter rather than a separate method. 2010-09-13 James Westby Make FetchedPackage.content optional. 2010-09-13 James Westby Have the HardwarePackBuilder obey the include-debs config setting. 2010-09-13 James Westby Add a method to not fetch the content, just get the versions. 2010-09-13 James Westby Also remove the .debs we have no content for from the Packages file. 2010-09-13 James Westby Leave out debs that we don't have content for. 2010-09-13 James Westby Make the content of the FetchedPackage optional. 2010-09-13 James Westby Allow setting the installed packages seen by an IsolatedAptCache. 2010-09-13 James Westby Simplify mtime_skew handling. Thanks Zygmunt. 2010-09-13 James Westby Specify a literal "-" in the regex. Thanks Zygmunt. 2010-09-13 James Westby Allow specifying extra text for each stanza in a Packages file. 2010-09-13 James Westby Document ignore_packages. 2010-09-13 James Westby Document and test set_installed_packages. 2010-09-13 James Westby Test and document extra_text. 2010-09-13 James Westby Drop the underscore from the parameter. 2010-09-13 James Westby Ensure we get an error if we can't install an ignored package. 2010-09-13 James Westby Ensure an error is raised if we ask to ignore an unknown package. 2010-09-13 James Westby Test that we ignore dependencies too. 2010-09-13 James Westby Allow ignoring packages in the PackageFetcher. 2010-09-13 James Westby Set the mtime of all the files in the hwpack. 2010-09-13 James Westby Allow specifying a skew for the mtime for TarfileHasFile. 2010-09-13 James Westby Use argparse. Thanks Zygmunt. 2010-09-13 Guilherme Salgado Make sure the loops to add sources.list.d entries and gpg keys don't fail when there's no sources.list.d entries or gpg keys 2010-09-13 Guilherme Salgado Change parsing of the manifest file in linaro-hwpack-install as it now uses an equal sign to separate packages from their version 2010-09-13 James Westby Fix the package and hwpack name regex. 2010-09-13 Guilherme Salgado Use mktemp as suggested by James 2010-09-10 James Westby Allow apt to parallelise the download 2010-09-10 James Westby Download all the dependencies of the target packages too. 2010-09-10 James Westby Add a docstring to from_apt. Thanks Zygmunt. 2010-09-10 James Westby Move stringify_relationship to be a top-level function and test it. 2010-09-10 James Westby Add a test that PackageFetcher fills in the relationships. 2010-09-10 James Westby Have from_apt translate the other relationships too. 2010-09-10 James Westby Put the relationships in the Packages file. 2010-09-10 James Westby Allow specifying other relationships in FetchedPackage. 2010-09-10 James Westby Split out an IsolatedAptCache class which can be used by other code. 2010-09-10 James Westby Add a from_apt method that knows how to translate dependencies. 2010-09-10 James Westby Allow specifying the dependencies of a package in the constructor. 2010-09-10 James Westby Pull IsolatedAptCache in to it's own class so that tests can use it. 2010-09-10 James Westby Include Depends in the Packages file we generate. 2010-09-10 James Westby Ensure that sources files have .list extension. 2010-09-10 James Westby Allow the PackageFetcher to be used as a context manager. 2010-09-10 James Westby Put the architecture in the hwpack filename. 2010-09-10 James Westby Have the manifest use "=" rather than " " to split package and version. 2010-09-08 Guilherme Salgado Drop a couple things that are not really needed and clean other things up 2010-09-07 Matt Domsch Quote uses of $device as required for some weirdly named /dev/disk/by-id/* devices; thanks Matt Domsch. 2010-09-07 Loïc Minier Release 0.2 2010-09-07 Loïc Minier Document hwpack changes by James Westby: * Add a module that provides some improvements to tarfile. * Add some matchers for testing the content of tarfiles. * Add the skeleton code to create empty hardware packs. 2010-09-07 Loïc Minier Fix deps in control as well: depend on uuid-runtime rather than uuidgen-runtime and do not depend on util-linux, e2fsprogs and coreutils as these are Essential: yes; thanks Colin Watson. 2010-09-06 Guilherme Salgado A few extra improvements 2010-09-06 Guilherme Salgado Use a trap to cleanup things in linaro-hwpack-install and a few other improvements 2010-09-06 Guilherme Salgado A few more improvements 2010-09-05 Loïc Minier Close LP: #627611. 2010-09-05 Loïc Minier Add changelog entry for IGEP support. 2010-09-04 Torez Smith updated format and igep board support 2010-09-03 Guilherme Salgado checkpoint commit 2010-09-03 Torez Smith support for igep board 2010-09-03 James Westby Use add_apt_sources instead of the old add_sources. Thanks Guilherme. 2010-09-02 James Westby Use textwrap to avoid a multiline string breaking formatting. 2010-09-02 James Westby Correct another typo. Thanks Guilherme. 2010-09-02 James Westby Fix typos. Thanks Guilherme. 2010-09-02 James Westby Avoid a double join by storing strings not tuples. 2010-09-02 James Westby Rename add_sources to add_apt_sources and update the docstring. 2010-09-02 James Westby Check the passed arguments before trying to use them. 2010-09-02 James Westby Have the missing config file exception printed nicely. 2010-09-02 James Westby Raise a specific exception if the config file is not there. 2010-09-02 James Westby Clean up some repeated hardcoding in a test. 2010-09-02 James Westby Add a rudimentary script to allow creating hwpacks. 2010-09-02 James Westby Add a builder class that puts the pieces together. 2010-09-02 James Westby Include the architecture in the filename. 2010-09-02 James Westby Add a simple test that PackageFetcher is a context manager. 2010-09-02 James Westby Add a docstring for from_config. 2010-09-02 James Westby Add a docstring for the sources property. 2010-09-02 James Westby Add a method to create a Metadata from a Config. 2010-09-02 James Westby Also add a property for getting the sources info. 2010-09-02 James Westby Add architectures to the configuration file. 2010-09-02 James Westby Specify the version external to the config. 2010-09-02 James Westby Use the new name of AptSourceFixture. 2010-09-02 James Westby Add a docstring to get_packages_file. 2010-09-02 James Westby Tweak docstrings in hwpack.packages. Thanks Michael. 2010-09-02 James Westby Add more documentation to testing.py and rename AptSource to AptSourceFixture 2010-09-02 James Westby Make Package in to DummyFetchedPackage to share an interface. 2010-09-02 James Westby Add some documentation for the Package object. 2010-09-02 James Westby Add a test for the new Metadata.from_config constructor. 2010-09-01 James Westby Process the architectures property. 2010-09-01 James Westby Ensure that architectures is specified in the config file. 2010-09-01 James Westby Add a sources property to config to get that information. 2010-09-01 James Westby Filter duplicate package names in the configuration. 2010-09-01 James Westby Add Architecture to Metadata. 2010-09-01 James Westby Add a way to specify the architecture to the fetcher. 2010-09-01 James Westby Put the architecture in the indices. 2010-09-01 James Westby Add architecture to our package classes. 2010-09-01 James Westby Rename FetchedPackageFixture to DummyFetchedPackage as it isn't a fixture 2010-09-01 James Westby Write a correct Packages file. 2010-09-01 James Westby Use the __eq__ method to get better checking in the tests. 2010-09-01 James Westby Add __eq__ to FetchedPackage. 2010-09-01 James Westby Make the generation of a rudimentary Packages file usable by application code 2010-09-01 James Westby Add the packages to the manifest too. 2010-09-01 James Westby Put the packges in the hardware pack. 2010-09-01 James Westby Consolidate the two Package implementations somewhat. 2010-09-01 James Westby Use a class as the return value such that we can return name, version and filename. 2010-09-01 James Westby Add a docstring. 2010-09-01 James Westby Add sources to the hwpack. 2010-09-01 James Westby Use a subclass of the matcher, rather than a custom assertion method. 2010-09-01 James Westby Add a TODO, thanks Guilherme. 2010-09-01 James Westby Rename to_f to to_file, thanks Guilherme. 2010-09-01 James Westby Remove the unneeded ugly code. 2010-09-01 James Westby Use apt rather than bzrlib, with our own method to avoid the stdout output. 2010-09-01 James Westby Create TestCaseWithFixtures. 2010-09-01 James Westby Move the helpers to testing.py. 2010-09-01 James Westby Add some tests for cleanup/prepare. 2010-09-01 James Westby Add some tests and docstrings. 2010-09-01 James Westby Test that the newer package is taken regardless of the source. 2010-09-01 James Westby Test that the newest is fetched. 2010-09-01 James Westby Some more simple tests for PackageFetcher. 2010-09-01 James Westby Implement basic package fetching support. 2010-09-01 Loïc Minier Release 0.1 2010-09-01 Loïc Minier Copyright 2010 Robert Nelson 2010-09-01 Loïc Minier Remove useless DISK_NAME var altogether 2010-09-01 Loïc Minier Don't test for German translation in sfdisk output; instead, run sfdisk with LC_ALL=C. 2010-08-31 matt.waddel@linaro.org <> Adds copy stage of uInitrd to the SD card for vexpress. Incorporates Loic's review feedback. 2010-08-31 Loïc Minier Fix whitespaces 2010-08-31 Loïc Minier Fix package for uuidgen (uuid-runtime) 2010-08-31 Loïc Minier Add proper depends 2010-08-31 James Westby Add a packages property to get the list of packages. 2010-08-31 Loïc Minier Add initial packaging 2010-08-31 James Westby Packages are global rather than per-archive now. 2010-08-31 Loïc Minier Fix copyright/licensing header to state Copyright Linaro and GPLv3 as the license. 2010-08-31 James Westby Add docstrings. 2010-08-31 James Westby Extract out a Metadata object to make testing easier. 2010-08-31 James Westby Also test the values set on TarfileMissingPathMismatch. 2010-08-31 James Westby Take advantage of the new __eq__ when checking TarfileWrongValueMismatch. 2010-08-31 James Westby Add __eq__ and __hash__ on the mismatches. 2010-08-31 Loïc Minier Kill insecure /tmp/rebuild_uinitrd.sh creation and the /rebuild_uinitrd.sh clutter, this should be shipped in a package if that's still useful, but really flash-kernel should refresh the uInitrd. 2010-08-31 Loïc Minier Don't copy over a vmlinuz into the boot partition (NB: NOT /boot) and use "uImage" instead of "uImage.vxp". 2010-08-31 James Westby Reduce duplication in the tests. Thanks Michael. 2010-08-31 Loïc Minier Use regular "u-boot.bin" path instead of ubootca9.bin and save one cp. 2010-08-31 James Westby Less magic in the tests. Thanks Michael. 2010-08-31 Loïc Minier Use POSIX function declarations rather than function keyword. 2010-08-31 Loïc Minier Simplify tar calls, relying on the fact that tar will guess that archives are compressed depending on their extensions. 2010-08-31 Loïc Minier Use sudo for mkimage in the vexpress case too. 2010-08-31 James Westby Add a module docstring to better_tarfile. Thanks Michael. 2010-08-31 James Westby Add a docstring to _get_main_option and change the name. Thanks Michael. 2010-08-30 James Westby Set the defaults for created files. 2010-08-30 James Westby Use dict.pop instead of the custom implementation. Thanks Michael. 2010-08-30 James Westby Use the matchers in the tests. 2010-08-30 James Westby Factor out some common code in the tests. 2010-08-30 James Westby Add some documentation. 2010-08-30 James Westby Consolidate more code in to the common method. 2010-08-30 James Westby Add documentation. 2010-08-30 James Westby Allow checking content in the matcher. 2010-08-30 James Westby Allow testing the uname and gname in the matcher. 2010-08-30 James Westby Allow testing the uid and gid in the matcher. 2010-08-30 James Westby Allow testing the linkname in the matcher. 2010-08-30 James Westby Allow testing the mode in the matcher. 2010-08-30 James Westby Allow checking the mtime of a path. 2010-08-30 James Westby Allowing checking the size in the matcher. 2010-08-30 James Westby Simple tarfile matchers. 2010-08-30 James Westby Move the helper context managers out of the test code. 2010-08-30 James Westby Merge better-tarfile. 2010-08-30 James Westby Use the defaults for directories too. 2010-08-30 James Westby Tests for the standard attributes. 2010-08-30 James Westby Call the methods create_* rather than add_*. 2010-08-30 James Westby Allow creating directories. 2010-08-30 James Westby Add default gname support 2010-08-30 James Westby Add default uname support. 2010-08-30 James Westby Add default uid. 2010-08-30 James Westby Allow specifying the default mtime for files. 2010-08-30 matt.waddel@linaro.org <> Set the default vfat partition type to 16 for the Versatile Express platform. 2010-08-30 matt.waddel@linaro.org <> Pull u-boot from the /usr/lib/u-boot/ca9x4_ct_vxp/ directory and put it in the vfat boot partition. 2010-08-30 James Westby Add docstrings for the config properties. Thanks Michael. 2010-08-30 James Westby Use a helper method to reduce duplicated code. Thanks Michael. 2010-08-29 Alexander Sack linaro-media-create: don't mount our vfat partition as /boot; we have flash-kernel + UBOOT_PART magic for that 2010-08-29 Alexander Sack linaro-media-create: bump requirements to FAT_SIZE=32 by default 2010-08-28 James Westby Test that linkname is set correctly. 2010-08-28 James Westby Test that the type is set correctly. 2010-08-28 James Westby Test that the mode is set correctly. 2010-08-28 James Westby Actually add the content. 2010-08-28 James Westby Basics of a better tarfile wrapper. 2010-08-27 James Westby Add the sources list dirs. 2010-08-27 James Westby Create a Packages file. 2010-08-27 James Westby Create an empty manifest file. 2010-08-27 James Westby Generate the full metadata file. 2010-08-27 James Westby Basic metadata file creation. 2010-08-27 James Westby Start the support for generating tarballs. 2010-08-27 James Westby Add a filename method. 2010-08-27 James Westby Test basic attributes of HardwarePack. 2010-08-27 James Westby Use a helper assertion to reduce code duplication. 2010-08-27 James Westby Use a method to avoid hardcoding StringIO everywhere. 2010-08-27 James Westby Add a .testr.conf for the project. 2010-08-27 James Westby Add a testsuite function to run the tests. 2010-08-27 James Westby Support property. 2010-08-27 Tom Gall Fix to add fstab entries for rootfs and boot 2010-08-27 James Westby maintainer property. 2010-08-27 James Westby Origin property. 2010-08-27 James Westby Basic properties for the main values. 2010-08-27 James Westby Test that validate allows a valid configuration. 2010-08-27 James Westby Small refactoring. 2010-08-27 James Westby Further validation of the packages value. 2010-08-27 James Westby Validation of the packages value. 2010-08-27 James Westby Futher validation of the source-entry value. 2010-08-27 James Westby Check the sources-entry of each section. 2010-08-27 James Westby Validate support and that there is more than one section. 2010-08-27 James Westby Use several methods. 2010-08-27 James Westby Test the value of the include-debs key. 2010-08-27 James Westby Check the name in the config against the rules. 2010-08-27 James Westby Basics of the hwpack.config module. 2010-08-27 Tom Gall Fix to respect FAT_SIZE and not to use sfdisk for MMC targets - lp:NUMBER 2010-08-23 matt.waddel@linaro.org <> Add vexpress image creation option 2010-08-23 matt.waddel@linaro.org <> Refactored for adding new boards 2010-08-20 Scott Bambrough New file. Bare bones. 2010-08-05 John Rigby Create /etc/flash-kernel.conf so flash-kernel works 2010-07-02 John Rigby use sfdisk for partition images and devices 2010-07-02 John Rigby split setup_image into two functions 2010-07-01 Alexander Sack remove binary/ tree from workdir after done to avoid pollution 2010-07-01 Alexander Sack only extract binary/${parts_dir} in prepare_sources to get the boot bits 2010-07-01 Alexander Sack drop the --chessy flag - we always use this code path 2010-07-01 mwaddel@ubuntu.com <> Addes the binary file selection option 2010-07-01 Alexander Sack create SWAP file in / rather than /mnt 2010-06-30 mwaddel@ubuntu.com <> Add file compression and fix swap size calculation 2010-06-18 matt.waddel@ubuntu.com <> Added an image creation capability for use with qemu 2010-06-14 Alexander Sack * rename setup_sdcard to linaro-media-create 2010-06-03 Alexander Sack do not require mkfs.reiserfs 2010-06-01 Zygmunt Krynicki Added checks for missing commands 2010-06-01 Alexander Sack add splash and quiet for non-dev live images 2010-05-31 Alexander Sack hide error messages if binary-tar.tar.lzma does not exist 2010-05-31 Alexander Sack add nocompcache to boot.cmd 2010-05-31 Alexander Sack * make setup_sdcard.sh executable 2010-05-24 Alexander Sack * introduce --live-256m option for low-mem systems * add -dev option that aut sets serial console ports * add --console option to explicitly add consoles to boot options 2010-05-21 Alexander Sack add copyright, license info and fix comment indent 2010-05-21 Alexander Sack * install all boot parts from proper directory (casper, boot) * support lzma as well as gz compressed binary-tar tarballs * use boot=casper for IS_LIVE and root=UUID=... for not IS_LIVE 2010-05-20 Alexander Sack support new boot parts (including uimage and uninitrd add --live parameter to set the boot=casper boot option 2010-05-20 Alexander Sack parse mmc ids before cleaning up sdcard, which requires details for unmounting 2010-05-19 Alexander Sack * swap gets holes when using seek for dd'ing swap file 2010-05-19 Alexander Sack allow CODENAME to be dynamic for uImage name; make swap image creation fast using dd seek and add fixrtc to bootargs for beagle without rtc battery 2010-05-18 Alexander Sack don't rely on device node syntax, but detect the device parts through /dev/disk; in turn split create_partitions (partition) and prepare_partitions (add files to boot and rootf) 2010-05-07 Jamie Bennett Remove download support and use local u-boot and x-loader files instead 2010-05-07 Jamie Bennett Remove x-loader support, its not needed 2010-05-05 John Rigby Initial checking with Jamie Bennet's additions linaro-image-tools-2014.11/do-release0000775000175000017500000000356712400111666020432 0ustar fboudrafboudra00000000000000#!/bin/sh # tag tree, generate ChangeLog and roll a release tarball set -e self="$(basename "$0")" usage() { echo "Usage: $self []" } log() { echo "$*" >&2 } log_i() { log "I:" "$@" } die() { log "E:" "$@" exit 1 } set_version() { log_i "Setting version to $1 in __version__.py" sed -i "s/^__version__ =.*/__version__ = \"$1\"/" linaro_image_tools/__version__.py } version="$1" old_version="$2" if [ -z "$version" ]; then usage >&2 exit 1 fi log_i "Checking tree status" status=`git status --short` if [ -n "$status" ]; then die "Tree is dirty according to git status" fi log_i "Running tests" if ! python -m testtools.run linaro_image_tools.tests.test_suite; then die "Testsuite doesn't pass" fi log_i "Removing test repository data" rm -rf .testrepository/ if git tag | awk '{print $1}' | grep -qFx "$version"; then die "Tag $version already exists" fi set_version $version log_i "Committing $version" git commit -a -m "Release $version." log_i "Creating tag $version" git tag "$version" log_i "Pushing changes and tag" git push origin master git push --tags log_i "Generating ChangeLog" if [ -z "$old_version"]; then git log --date=short --no-merges --format=format:"%ad %aN <%aE>%n%n %s%n" >ChangeLog else # If we have also the old revision, we can generate changelog only for # those tags, instead of a full changelog since the beginning of time. git log --date=short --no-merges --format=format:"%ad %aN <%aE>%n%n %s%n" >ChangeLog "$old_version".."$version" fi log_i "Creating release tarball in parent directory" ./setup.py sdist -d .. log_i "Cleaning up" rm -f ChangeLog MANIFEST log_i "Signing tarball" gpg --armor --sign --detach-sig "../linaro-image-tools-$version.tar.gz" set_version "$version.1" log_i "Committing $version.1" git commit -a -m "Post-release version bump to $version.1." linaro-image-tools-2014.11/linaro-android-media-create0000775000175000017500000001570012435051314023623 0ustar fboudrafboudra00000000000000#!/usr/bin/env python # Copyright (C) 2011 Linaro # # Author: Jeremy Chang # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import atexit import os import sys import tempfile from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.android_boards import ( get_board_config, ) from linaro_image_tools.media_create.check_device import ( confirm_device_selection_and_ensure_it_is_ready) from linaro_image_tools.media_create.partitions import ( Media, setup_android_partitions, partition_mounted, ) from linaro_image_tools.media_create.rootfs import populate_partition from linaro_image_tools.media_create.unpack_binary_tarball import ( unpack_android_binary_tarball ) from linaro_image_tools.media_create import get_android_args_parser from linaro_image_tools.utils import ( additional_android_option_checks, android_hwpack_in_boot_tarball, ensure_command, get_logger, disable_automount, enable_automount, ) # Just define the global variables TMP_DIR = None BOOT_DISK = None CACHE_DISK = None SDCARD_DISK = None # Registered as the first atexit handler as we want this to be the last # handler to execute. @atexit.register def cleanup_tempdir(): """Remove TEMP_DIR with all its contents. Before doing so, make sure DISKs are not mounted. """ devnull = open('/dev/null', 'w') # ignore non-zero return codes for disk in BOOT_DISK, CACHE_DISK, \ SDCARD_DISK: if disk is not None: try: cmd_runner.run(['umount', disk], stdout=devnull, stderr=devnull, as_root=True).wait() except cmd_runner.SubcommandNonZeroReturnValue: pass # Remove TMP_DIR as root because some files written there are # owned by root. if TMP_DIR is not None: cmd_runner.run(['rm', '-rf', TMP_DIR], as_root=True).wait() def ensure_required_commands(args): """Ensure we have the commands that we know are going to be used.""" required_commands = [ 'mkfs.vfat', 'sfdisk', 'mkimage', 'parted'] for command in required_commands: ensure_command(command) if __name__ == '__main__': parser = get_android_args_parser() args = parser.parse_args() logger = get_logger(debug=args.debug) additional_android_option_checks(args) # If --help was specified this won't execute. # Create temp dir and initialize rest of path vars. TMP_DIR = tempfile.mkdtemp() BOOT_DIR = os.path.join(TMP_DIR, 'boot') SYSTEM_DIR = os.path.join(TMP_DIR, 'system') DATA_DIR = os.path.join(TMP_DIR, 'data') BOOT_DISK = os.path.join(TMP_DIR, 'boot-disc') CACHE_DISK = os.path.join(TMP_DIR, 'cache-disc') SDCARD_DISK = os.path.join(TMP_DIR, 'sdcard-disc') if args.dev == 'iMX53': # XXX: remove this and the corresponding entry in android_board_configs logger.warning("DEPRECATION WARNING: iMX53 is deprecated, please " "use mx53loco.") ensure_required_commands(args) # Do this by default, disable automount options and re-enable them at exit. disable_automount() atexit.register(enable_automount) media = Media(args.device) if media.is_block_device: if not confirm_device_selection_and_ensure_it_is_ready(args.device): sys.exit(1) elif not args.should_create_partitions: logger.error("Do not use --no-part in conjunction with --image_file.") sys.exit(1) cmd_runner.run(['mkdir', '-p', BOOT_DIR]).wait() cmd_runner.run(['mkdir', '-p', SYSTEM_DIR]).wait() cmd_runner.run(['mkdir', '-p', DATA_DIR]).wait() unpack_android_binary_tarball(args.boot, BOOT_DIR) board_config = get_board_config(args.dev) hwpack_exists, config_file = android_hwpack_in_boot_tarball(BOOT_DIR) if not args.hwpack and not hwpack_exists: # No hwpack in the boot tarball nor provided on the command line. logger.warning("No hwpack found in the boot tarball nor passed on " "the command line. Default values will be used.") elif not args.hwpack and hwpack_exists: board_config.from_file(config_file) elif args.hwpack: logger.warning("Values from the hwpack provided on the command line " "will be used.") board_config.from_file(args.hwpack) board_config.add_boot_args(args.extra_boot_args) board_config.add_boot_args_from_file(args.extra_boot_args_file) # Create partitions boot_partition, system_partition, cache_partition, \ data_partition, sdcard_partition = setup_android_partitions( \ board_config, media, args.image_size, args.boot_label, args.should_create_partitions, args.should_align_boot_part) board_config.populate_raw_partition(args.device, BOOT_DIR) populate_partition(BOOT_DIR + "/boot", BOOT_DISK, boot_partition) board_config.populate_boot_script(boot_partition, BOOT_DISK, args.consoles) with partition_mounted(boot_partition, BOOT_DISK): board_config.install_boot_loader(args.device, BOOT_DISK) if args.system: with partition_mounted(system_partition, SYSTEM_DIR): unpack_android_binary_tarball(args.system, TMP_DIR) elif args.systemimage : cmd_runner.run( [ 'e2label', args.systemimage, "system"], stderr=open('/dev/null', 'w'), as_root=True).wait() cmd_runner.run( [ 'dd', 'if=%s' % args.systemimage, 'of=%s' % system_partition], stderr=open('/dev/null', 'w'), as_root=True).wait() else: #should not reach here pass if args.userdata: with partition_mounted(data_partition, DATA_DIR): unpack_android_binary_tarball(args.userdata, TMP_DIR) elif args.userdataimage: cmd_runner.run( [ 'e2label', args.userdataimage, "userdata"], stderr=open('/dev/null', 'w'), as_root=True).wait() cmd_runner.run( [ 'dd', 'if=%s' % args.userdataimage, 'of=%s' % data_partition], stderr=open('/dev/null', 'w'), as_root=True).wait() else: #should not reach here pass print "Done creating Linaro Android image on %s" % args.device linaro-image-tools-2014.11/COPYING0000664000175000017500000000073112400111666017505 0ustar fboudrafboudra00000000000000Format-Specification: http://svn.debian.org/wsvn/dep/web/deps/dep5.mdwn?op=file&rev=135 Name: Linaro Image Tools Maintainer: Linaro Infrastructure team Source: https://launchpad.net/linaro-image-tools Files: linaro-hwpack-create linaro-hwpack-install linaro_image_tools/hwpack/* Copyright: 2010, 2011 Linaro Limited License: GPL-2+ Files: linaro-media-create linaro_image_tools/media_create/* Copyright: 2010, 2011 Linaro Limited License: GPL-3+ linaro-image-tools-2014.11/linaro-media-create0000775000175000017500000002424312400111666022206 0ustar fboudrafboudra00000000000000#!/usr/bin/env python # Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import atexit import os import sys import tempfile from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.boards import get_board_config from linaro_image_tools.media_create.check_device import ( confirm_device_selection_and_ensure_it_is_ready) from linaro_image_tools.media_create.chroot_utils import ( install_hwpacks, install_packages, ) from linaro_image_tools.hwpack.hwpack_reader import ( HwpackReader, HwpackReaderError, ) from linaro_image_tools.media_create.partitions import ( Media, setup_partitions, get_uuid, ) from linaro_image_tools.media_create.rootfs import populate_rootfs from linaro_image_tools.media_create.unpack_binary_tarball import ( unpack_binary_tarball, ) from linaro_image_tools.media_create import get_args_parser from linaro_image_tools.utils import ( additional_option_checks, check_file_integrity_and_log_errors, check_required_args, ensure_command, IncompatibleOptions, is_arm_host, MissingRequiredOption, path_in_tarfile_exists, prep_media_path, get_logger, UnableToFindPackageProvidingCommand, disable_automount, enable_automount, ) # Just define the global variables TMP_DIR = None ROOTFS_DIR = None BOOT_DISK = None ROOT_DISK = None # Registered as the first atexit handler as we want this to be the last # handler to execute. def cleanup_tempdir(): """Remove TEMP_DIR with all its contents. Before doing so, make sure BOOT_DISK and ROOT_DISK are not mounted. """ devnull = open('/dev/null', 'w') # ignore non-zero return codes for disk in BOOT_DISK, ROOT_DISK: if disk is not None: try: cmd_runner.run(['umount', disk], stdout=devnull, stderr=devnull, as_root=True).wait() except cmd_runner.SubcommandNonZeroReturnValue: pass # Remove TMP_DIR as root because some files written there are # owned by root. if TMP_DIR is not None: cmd_runner.run(['rm', '-rf', TMP_DIR], as_root=True).wait() def ensure_required_commands(args): """Ensure we have the commands that we know are going to be used.""" required_commands = [ 'mkfs.vfat', 'sfdisk', 'mkimage', 'parted', 'gpg', 'sha1sum', 'sgdisk'] if not is_arm_host(): required_commands.append('qemu-arm-static') if args.rootfs in ['btrfs', 'ext2', 'ext3', 'ext4']: required_commands.append('mkfs.%s' % args.rootfs) else: raise AssertionError('Unsupported rootfs type %s' % args.rootfs) for command in required_commands: try: ensure_command(command) except UnableToFindPackageProvidingCommand: logger.error("Could not look up command %s. Please ensure that command %s is installed." % (command, command)) raise if __name__ == '__main__': parser = get_args_parser() args = parser.parse_args() logger = get_logger(debug=args.debug) try: additional_option_checks(args) except IncompatibleOptions as e: parser.print_help() logger.error(e.value) sys.exit(1) if args.readhwpack: try: reader = HwpackReader(args.hwpacks) logger.info(reader.get_supported_boards()) sys.exit(0) except HwpackReaderError as e: logger.error(e.value) sys.exit(1) try: check_required_args(args) except MissingRequiredOption as e: parser.print_help() logger.error(e.value) sys.exit(1) # Do this by default, disable automount options and re-enable them at exit. disable_automount() atexit.register(enable_automount) board_config = get_board_config(args.dev) board_config.set_metadata(args.hwpacks, args.bootloader, args.dev, args.dtb_file) board_config.add_boot_args(args.extra_boot_args) board_config.add_boot_args_from_file(args.extra_boot_args_file) media = Media(prep_media_path(args)) if media.is_block_device: if not board_config.supports_writing_to_mmc: logger.error("The board '%s' does not support the --mmc option. " "Please use --image_file to create an image file for " "this board." % args.dev) sys.exit(1) if not confirm_device_selection_and_ensure_it_is_ready( args.device, args.nocheck_mmc): sys.exit(1) elif not args.should_format_rootfs or not args.should_format_bootfs: logger.error("Do not use --no-boot or --no-part in conjunction with " "--image_file.") sys.exit(1) # If --help was specified this won't execute. # Create temp dir and initialize rest of path vars. TMP_DIR = tempfile.mkdtemp() BOOT_DISK = os.path.join(TMP_DIR, 'boot-disc') ROOT_DISK = os.path.join(TMP_DIR, 'root-disc') BIN_DIR = os.path.join(TMP_DIR, 'rootfs') os.mkdir(BIN_DIR) logger.info('Searching correct rootfs path') # Identify the correct path for the rootfs filesystem_dir = '' if path_in_tarfile_exists('binary/etc', args.binary): filesystem_dir = 'binary' elif path_in_tarfile_exists('binary/boot/filesystem.dir', args.binary): # The binary image is in the new live format. filesystem_dir = 'binary/boot/filesystem.dir' ROOTFS_DIR = os.path.join(BIN_DIR, filesystem_dir) try: ensure_required_commands(args) except UnableToFindPackageProvidingCommand: sys.exit(1) sig_file_list = args.hwpacksigs[:] if args.binarysig is not None: sig_file_list.append(args.binarysig) # Check that the signatures that we have been provided (if any) match # the hwpack and OS binaries we have been provided. If they don't, quit. files_ok, verified_files = check_file_integrity_and_log_errors( sig_file_list, args.binary, args.hwpacks) if not files_ok: sys.exit(1) atexit.register(cleanup_tempdir) unpack_binary_tarball(args.binary, BIN_DIR) # if compatible system, extract all packages os_release_id = 'linux' os_release_file = '%s/etc/os-release' % ROOTFS_DIR if os.path.exists(os_release_file): for line in open(os_release_file): if line.startswith('ID='): os_release_id = line[(len('ID=')):] os_release_id = os_release_id.strip('\"\n') break if os_release_id == 'debian' or os_release_id == 'ubuntu' or \ os.path.exists('%s/etc/debian_version' % ROOTFS_DIR): extract_kpkgs = False elif os_release_id == 'fedora': extract_kpkgs = False else: extract_kpkgs = True hwpacks = args.hwpacks lmc_dir = os.path.dirname(__file__) if lmc_dir == '': lmc_dir = None install_hwpacks(ROOTFS_DIR, TMP_DIR, lmc_dir, args.hwpack_force_yes, verified_files, extract_kpkgs, *hwpacks) if args.rootfs == 'btrfs': if not extract_kpkgs: logger.info("Desired rootfs type is 'btrfs', trying to " "auto-install the 'btrfs-tools' package") install_packages(ROOTFS_DIR, TMP_DIR, "btrfs-tools") else: logger.info("Desired rootfs type is 'btrfs', please make sure the " "rootfs also includes 'btrfs-tools'") boot_partition, root_partition = setup_partitions( board_config, media, args.image_size, args.boot_label, args.rfs_label, args.rootfs, args.should_create_partitions, args.should_format_bootfs, args.should_format_rootfs, args.should_align_boot_part, args.part_table) uuid = get_uuid(root_partition) # In case we're only extracting the kernel packages, avoid # using uuid because we don't have a working initrd if extract_kpkgs: # XXX: workaround https://bugs.launchpad.net/bugs/1208815 # When we use OE, we don't have initrd/UUID and fallback to pass # root=/dev/mmcblk0p3 to the kernel. It's based on mmc_option value # provided by the hardware configuration (mmc_id: '0:2'). # At U-Boot stage, the value is correct and we load from mmc 0:2. # At the kernel stage, the value becomes incorrect because Arndale # has eMMC and rootfs can be found on /dev/mmcblk1p3. # Since the boot commands are calculated based on the same mmc_id # parameter, Arndale can't boot in this use case. if board_config.board == 'arndale': board_config.mmc_device_id = board_config.mmc_device_id + 1 # XXX: this needs to be smarter as we can't always assume mmcblk devices rootfs_id = '/dev/mmcblk%dp%s' % ( board_config.mmc_device_id, 2 + board_config.mmc_part_offset) else: rootfs_id = "UUID=%s" % uuid if args.should_format_bootfs: board_config.populate_boot( ROOTFS_DIR, rootfs_id, boot_partition, BOOT_DISK, media.path, args.is_live, args.is_lowmem, args.consoles) if args.should_format_rootfs: create_swap = False if args.swap_file is not None: create_swap = True populate_rootfs(ROOTFS_DIR, ROOT_DISK, root_partition, args.rootfs, rootfs_id, create_swap, str(args.swap_file), board_config.mmc_device_id, board_config.mmc_part_offset, os_release_id, board_config) logger.info("Done creating Linaro image on %s" % media.path) linaro-image-tools-2014.11/README0000664000175000017500000000340612400111666017334 0ustar fboudrafboudra00000000000000This project contains a collection of tools to help creating Linaro bootable images to be used in development boards. = Running linaro-media-create = To run linaro-media-create you'll need to make sure the following packages are installed: - parted - dosfstools - u-boot-tools or uboot-mkimage - python-argparse - python-dbus - python-debian >= 0.1.16ubuntu1 - python-parted - qemu-user-static >= 0.13.0 (only if you're running on x86) - btrfs-tools - command-not-found - python-yaml = Running tests = Before running any tests you need to install the following packages: - testrepository - python-testtools >= 0.9.12 (available at https://launchpad.net/~linaro-maintainers/+archive/tools) - python-debian >= 0.1.16ubuntu1 - python-argparse - dpkg-dev - python-parted - python-dbus (and dbus, udisks) - python-apt - qemu-kvm - sfdisk (from util-linux) - apt-utils - pep8 - python-mock - python-commandnotfound (as of Ubuntu 12.10) Also consider installing pyflakes, which is optional but will enable more tests. Some of the packages above require fixes, which are as of Ubuntu 12.04 are not yet in upstream, so using Linaro PPA is required: $ sudo add-apt-repository ppa:linaro-maintainers/tools Then, you can install all packages listed above in one go with the following command: $ sudo apt-get install testrepository python-testtools python-debian \ python-argparse dpkg-dev python-parted dbus udisks python-dbus \ python-apt qemu-kvm util-linux apt-utils pep8 pyflakes \ python-commandnotfound To initialized testsuite, run the following command: $ testr init After that you can run the whole test suite with: $ testr run If you want to learn more about testrepository: $ testr quickstart linaro-image-tools-2014.11/linaro-hwpack-replace0000775000175000017500000002204012400111666022545 0ustar fboudrafboudra00000000000000#!/usr/bin/env python # Copyright (C) 2010, 2011 Linaro # # Author: Deepti B. Kalakeri # # This file is part of Linaro Image Tools. It adds the feature # to include/replace a debian package into the given hwpack. # We might need to change the manifest and Packages file in the # future to match the hardware pack v2 changes when available. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . # import os import sys import shutil import glob import tarfile import tempfile import argparse import datetime import fileinput from debian.deb822 import Packages from linaro_image_tools.hwpack.packages import get_packages_file from linaro_image_tools.hwpack.packages import FetchedPackage from linaro_image_tools.utils import get_logger parser = argparse.ArgumentParser() parser.add_argument("-t", "--hwpack-name", dest="hwpack_name", help="Specific hwpack-name to use (default: None)") parser.add_argument("-p", "--deb-pack", dest="deb_pack", help="Specific debian package to replace (default: None).") parser.add_argument("-r", "--prefix-pkg-remove", dest="prefix_pkg_remove", help="Specify the prefix of the old debian package to "\ "replace (default: None).") parser.add_argument("-n", "--append-build-number", dest="build_number", help="Specify the build number if any to be used in new "\ "hwpack name (default: None).") parser.add_argument("-i", "--in-place", action="store_true", dest="inplace", help="Modify the hwpack rather than creating a new one") parser.add_argument("-d", "--debug-output", action="store_true", dest="debug", help="Verbose messages are displayed when specified") logger = None class DummyStanza(object): def __init__(self, info): self.info = info def dump(self, fd): fd.write(get_packages_file([self.info])) def get_hwpack_name(old_hwpack, build_number): # The build_number would be the job build number. # Valid value for the build_number would be available for ex # when l-h-r is used in the jenkins. timestamp = [datetime.datetime.utcnow().strftime("%Y%m%d-%H%M")] hwpack_name_parts = (old_hwpack.split('_', 3)) new_hwpack_name = [('_'.join(hwpack_name_parts[:2] + timestamp))] if build_number is not None: job_build_number = [''.join('b' + build_number)] new_hwpack_name = [('_'.join(new_hwpack_name + job_build_number))] return('_'.join(new_hwpack_name + hwpack_name_parts[3:])) def should_remove(package_name, prefix_pkg_remove): # hwpack-* Package is a metadata package that contain reference to the # linux-linaro-omap that was previously present in the hwpack. # We need to make sure we dont write the hwpack-* related # package information into Package, otherwise it would try to download the old # kernel package that was present in the hwpack than installing the new one. if (package_name.startswith(prefix_pkg_remove) or package_name.startswith("hwpack-")): return True return False def verify_existing_debians(debpack_dirname, prefix_pkg_remove): """ Find if the debian file with the same name exists, if it exists then remove it and replace with the new deb file If similar debian file exists then remove it """ deb_file_to_remove = None try: for deb_filename in os.listdir(debpack_dirname): root, ext = os.path.splitext(deb_filename) if should_remove(root, prefix_pkg_remove) and ext == '.deb': deb_file_to_remove = os.path.join(debpack_dirname, deb_filename) os.remove(deb_file_to_remove) except Exception, details: logger.error("Error Details: %s", details) def modify_manifest_info(tempdir, new_debpack_info, prefix_pkg_remove): """ Modify the manifest file to include the new debian information """ debpack_manifest_fname = os.path.join(tempdir, "manifest") if new_debpack_info is not None: new_debpack_line = '%s=%s\n' % (new_debpack_info.name, new_debpack_info.version) for line in fileinput.FileInput(debpack_manifest_fname, inplace=1): if not should_remove(line, prefix_pkg_remove): sys.stdout.write(line) if new_debpack_info is not None: logger.debug("Adding the new debian package info to manifest") fout = open(debpack_manifest_fname, "a") fout.write(new_debpack_line) fout.close() else: logger.debug("Removed the debian package info from manifest") def modify_Packages_info(debpack_dirname, new_debpack_info, prefix_pkg_remove): """ Modify the Packages file to include the new debian information """ debpack_Packages_fname = os.path.join(debpack_dirname, "Packages") try: output = [] f = open(debpack_Packages_fname, "r+") for stanza in Packages.iter_paragraphs(f): if not should_remove(stanza["Package"], prefix_pkg_remove): output.append(stanza) if new_debpack_info is not None: output.append(DummyStanza(new_debpack_info)) f.seek(0,0) f.truncate() for stanza in output: stanza.dump(f) f.write("\n") finally: f.close() def main(): # Validate that all the required information is passed on the command line args = parser.parse_args() if (args.hwpack_name == None or args.prefix_pkg_remove == None): parser.print_help() parser.error("You must specify both hwpack name "\ "and the debian package information\n") return 1 global logger logger = get_logger(debug=args.debug) old_hwpack = args.hwpack_name new_deb_file_to_copy = args.deb_pack prefix_pkg_remove = args.prefix_pkg_remove build_number = args.build_number status = 0 tempdir = "" try: # Get the new hardware pack name hwpack_name = get_hwpack_name(old_hwpack, build_number) if hwpack_name == None: logger.error("Did not get a valid hwpack name, exiting") return status # untar the hardware pack and extract all the files in it tar = tarfile.open(old_hwpack, "r:gz") tempdir = tempfile.mkdtemp() tar.extractall(tempdir) tar.close() # Search if a similar package with the same name exists, if yes then # replace it. IF the old and new debian have the same name then we # are still replacing the old one with the new one. debpack_dirname = os.path.join(tempdir, 'pkgs/') if not os.path.exists(debpack_dirname): logger.error("Failed to extract the hwpack: %s ", old_hwpack) return status new_debpack_info = None if new_deb_file_to_copy is not None: new_debpack_info = FetchedPackage.from_deb(new_deb_file_to_copy) verify_existing_debians(debpack_dirname, prefix_pkg_remove) # Copy the new debian file to the pkgs dir, if new_deb_file_to_copy is not None: shutil.copy2(new_deb_file_to_copy, debpack_dirname) modify_manifest_info(tempdir, new_debpack_info, prefix_pkg_remove) modify_Packages_info(debpack_dirname, new_debpack_info, prefix_pkg_remove) # Compress the hardware pack with the new debian file included in it tar = tarfile.open(hwpack_name , "w:gz") origdir = os.getcwd() os.chdir(tempdir) for file_name in glob.glob('*'): tar.add(file_name, recursive=True) tar.close() # Retain old hwpack name instead of using a new name os.chdir(origdir) if args.inplace: os.rename(hwpack_name, old_hwpack) hwpack_name = old_hwpack # Export the updated manifest file manifest_name = hwpack_name.replace('.tar.gz', '.manifest.txt') shutil.copy2(os.path.join(tempdir, 'manifest'), manifest_name) except Exception, details: logger.error("Error Details: %s", details) status = 1 finally: if os.path.exists(tempdir): shutil.rmtree(tempdir) if status == 0: logger.info("The debian package '%s' has been been included in '%s'", new_deb_file_to_copy, hwpack_name) print hwpack_name else: logger.error("Injecting the debian package '%s' failed", new_deb_file_to_copy) return status if __name__ == '__main__': sys.exit(main()) linaro-image-tools-2014.11/initrd-do0000775000175000017500000000763212400111666020300 0ustar fboudrafboudra00000000000000#!/bin/sh # initrd-do - run a command within an initrd # Copyright (C) 2008 Loïc Minier # Copyright (C) 2011 Linaro Limited # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), # to deal in the Software without restriction, including without limitation # the rights to use, copy, modify, merge, publish, distribute, sublicense, # and/or sell copies of the Software, and to permit persons to whom the # Software is furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in # all copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL # SOFTWARE IN THE PUBLIC INTEREST, INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR # OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER # DEALINGS IN THE SOFTWARE. # # Except as contained in this notice, the name of the author shall not be used # in advertising or otherwise to promote the sale, use or other dealings in # this Software without prior written authorization from the author. # # depends: sudo set -e self="$(basename "$0")" work_dir="" new_initrd="" log() { echo "$*" >&2 } log_i() { log "I:" "$@" } log_w() { log "W:" "$@" } die() { log "E:" "$@" exit 1 } usage() { log "Usage: $self --initrd [-c [ ...]]" } getshell() { if [ -n "$SHELL" ]; then echo "$SHELL" return fi PASSWD="$(getent passwd $(id -u) | sed -n 's/.*://p')" if [ -n "$PASSWD" ]; then echo "$PASSWD" return fi echo "/bin/sh" } escape() { echo "$*" | sed "s/'/'\"'\"'/g; s/.*/'&'/" } # unused; use eval unescape () { eval "echo" "$*" } cleanup() { if [ -n "$new_initrd" ]; then rm -f "$new_initrd" fi if [ -n "$work_dir" ]; then rm -rf "$work_dir" fi } trap "cleanup" 0 1 2 3 9 11 13 15 initrd="" command="$(escape "$(getshell)")" while [ $# -gt 0 ]; do case $1 in --help) usage exit 0 ;; -i|--initrd) shift initrd="$1" if ! shift; then die "Need an initrd file after --initrd" fi ;; -c) shift if [ $# -le 0 ]; then die "Need a command after -c" fi command="" while :; do command="$command $(escape "$1")" shift if [ $# -eq 0 ]; then break fi done break ;; *) # clever handling of args: set initrd first if unset, otherwise set # command if [ -z "$initrd" ]; then initrd="$1" shift else command="" while :; do command="$command $(escape "$1")" shift if [ $# -eq 0 ]; then break fi done fi ;; esac done if [ -z "$initrd" ]; then usage exit 1 fi work_dir="$(mktemp -dt "$self.XXXXXXXXXX")" # subshell to not touch cwd as we will nuke it ( cd "$work_dir" log_i "Unpacking initrd $initrd" gunzip -c <"$initrd" | cpio -i --quiet log_i "Running command:" $command eval $command ) err=$? if [ 0 != $err ]; then log_w "Command exited with $err; aborting" exit 1 fi log_i "Generating new initrd" new_initrd="$(mktemp -t "$self.XXXXXXXXXX")" # subshell to not touch cwd as we will nuke it ( cd "$work_dir" find . | cpio -o -H newc -R 0:0 --quiet | gzip -c >"$new_initrd" ) log_i "Moving new initrd to $initrd" mv -f "$new_initrd" "$initrd" linaro-image-tools-2014.11/linaro-hwpack-install0000775000175000017500000002537712400111666022620 0ustar fboudrafboudra00000000000000#!/bin/sh # linaro-hwpack-install - Install a Linaro Hardware Pack. # This script is meant to run inside a chroot. It must not depend on anything # that's not in there. # TODO: When upgrading to a newer hwpack, make sure packages and apt sources # that are no longer needed are removed. # Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. set -e if [ -n "${TMPDIR+x}" ]; then echo -e "\nWARNING: TMPDIR variable is set. It will be unset because in chroot environment it likely doesn't exist and can lead to error messages.\n" unset TMPDIR fi LOCKFILE="/var/lock/hwpack" TEMP_DIR=$(mktemp -d) HWPACK_DIR="${TEMP_DIR}/unpacked" INSTALL_LATEST="no" FORCE_YES="no" SOURCES_LIST_FILE="${TEMP_DIR}/sources.list" APT_GET_OPTIONS="Dir::Etc::SourceList=${SOURCES_LIST_FILE}" SUPPORTED_FORMATS="1.0 2.0 3.0" # A space-separated list of hwpack formats. FLASH_KERNEL_SKIP="true" export FLASH_KERNEL_SKIP # skip attempting to run flash-kernel-hooks DISTRIBUTION=`grep '^ID=' /etc/os-release | sed 's/ID=//'` sudo="sudo" if [ $(id -u) -eq 0 ]; then sudo="" fi die() { echo -e "$@" exit 1 } usage_msg="Usage: $(basename $0) [--install-latest] [--force-yes] [--extract-kernel-only] --hwpack-version --hwpack-arch --hwpack-name HWPACK_TARBALL" if [ $# -eq 0 ]; then die $usage_msg fi HWPACK_TARBALL_FOUND="no" HWPACK_VERSION="" HWPACK_ARCH="" HWPACK_NAME="" EXTRACT_KERNEL_ONLY="no" while [ $# -gt 0 ]; do case "$1" in --install-latest) INSTALL_LATEST="yes" shift;; --force-yes) FORCE_YES="yes" shift;; --hwpack-version) HWPACK_VERSION=$2 shift; shift;; --hwpack-arch) HWPACK_ARCH=$2 shift; shift;; --hwpack-name) HWPACK_NAME=$2 shift; shift;; --extract-kernel-only) EXTRACT_KERNEL_ONLY="yes" shift;; --*) die $usage_msg "\nUnrecognized option: \"$1\"";; *) [ "$HWPACK_TARBALL_FOUND" = "yes" ] && die $usage_msg HWPACK_TARBALL="$1" HWPACK_TARBALL_FOUND="yes" shift;; esac done [ "$HWPACK_TARBALL_FOUND" = "no" ] && die $usage_msg [ "$HWPACK_VERSION" = "" ] && die $usage_msg [ "$HWPACK_ARCH" = "" ] && die $usage_msg [ "$HWPACK_NAME" = "" ] && die $usage_msg setup_hwpack() { # This creates all the directories we need. mkdir -p "$HWPACK_DIR" # Unpack the hwpack tarball. We don't download it here because the chroot may # not contain any tools that would allow us to do that. echo -n "Unpacking hardware pack ..." tar zxf "$HWPACK_TARBALL" -C "$HWPACK_DIR" echo "Done" # Check the format of the hwpack is supported. hwpack_format=$(cat ${HWPACK_DIR}/FORMAT) supported="false" for format in $SUPPORTED_FORMATS; do if [ "x$hwpack_format" = "x$format" ]; then supported="true" break fi done [ $supported = "true" ] || \ die "Unsupported hwpack format: $hwpack_format. "\ "Try using a newer version of $(basename $0)." # Check the architecture of the hwpack matches that of the host system. if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then # TODO: create a generic way to identify the architecture, without depending on dpkg [ "$HWPACK_ARCH" = `dpkg --print-architecture` ] || \ die "Hardware pack architecture ($HWPACK_ARCH) does not match the host's architecture" fi } setup_apt_sources() { # Install the apt sources that contain the packages we need. for filename in $(ls "${HWPACK_DIR}"/sources.list.d/); do file="${HWPACK_DIR}"/sources.list.d/$filename should_install=0 stripped_file=${TEMP_DIR}/$filename grep -v "\(^#\|^\s*$\)" $file > $stripped_file while read line; do # Only install files that have at least one line not present in the # existing sources lists. grep -qF "$line" $(find /etc/apt/sources.list.d/ -name '*.list') /etc/apt/sources.list \ || should_install=1 done < $stripped_file if [ $should_install -eq 1 ]; then $sudo cp $file /etc/apt/sources.list.d/hwpack.$filename fi done # Import the OpenPGP keys for the files installed above. for filename in $(ls "${HWPACK_DIR}"/sources.list.d.gpg/); do file="${HWPACK_DIR}"/sources.list.d.gpg/$filename $sudo apt-key add $file done # Add one extra apt source for the packages included in the hwpack and make # sure it's the first on the list of sources so that it gets precedence over # the others. echo "deb file:${HWPACK_DIR}/pkgs ./" > "$SOURCES_LIST_FILE" cat /etc/apt/sources.list >> "$SOURCES_LIST_FILE" if [ "$FORCE_YES" = "yes" ]; then FORCE_OPTIONS="--yes --force-yes" else FORCE_OPTIONS="" fi # Do two updates. The first doesn't try to download package lists: # * First update doesn't access net # - not allowed to fail. Image file + hwpack should contain all packages # needed to create image. If this update fails we have problems. # * Second update may fail # - If can't download package updates (the only difference between the two # commands), we should still be OK. echo "Updating apt package lists ..." $sudo apt-get $FORCE_OPTIONS -o "$APT_GET_OPTIONS" update -q --no-download --ignore-missing $sudo apt-get $FORCE_OPTIONS -o "$APT_GET_OPTIONS" update -q || true } setup_ubuntu_rootfs() { # Prevent daemons to start in the chroot echo "exit 101" > /usr/sbin/policy-rc.d chmod a+x /usr/sbin/policy-rc.d mv -f /sbin/start-stop-daemon /sbin/start-stop-daemon.REAL cat > /sbin/start-stop-daemon << EOF #!/bin/sh echo "Warning: Fake start-stop-daemon called, doing nothing" EOF chmod 755 /sbin/start-stop-daemon if [ -x /sbin/initctl ]; then mv -f /sbin/initctl /sbin/initctl.REAL cat > /sbin/initctl << EOF #!/bin/sh echo "Warning: Fake initctl called, doing nothing" EOF chmod 755 /sbin/initctl fi } install_deb_packages() { echo -n "Installing packages ..." # "newer" hwpacks contain a dependency package whose Depends is the # same as the packages config setting from the file the hwpack was # build from. But if we just installed that, a newer version of a # package than that in the hwpack might have made it to the main # archive and apt-get would install that instead. So we install the # specific package versions that make up the hwpack. /That/ however # would leave all the packages from the hwpack marked as manually # installed, so if a newer hwpack was installed over the top which no # longer depended on one of the packages the older one did, the # package would not be eligible for autoremoval. So we mark the all # packages newly installed as part of hwpack installed (apart from the # dependency package) as automatically installed with apt-get # markauto. # # For "older" hwpacks that don't have a dependency package, we just # manually install the contents of the hwpack. dependency_package="hwpack-${HWPACK_NAME}" if grep -q "^${dependency_package}=${HWPACK_VERSION}\$" "${HWPACK_DIR}"/manifest; then DEP_PACKAGE_PRESENT="yes" else DEP_PACKAGE_PRESENT="no" fi packages_without_versions=`sed 's/=.*//' "${HWPACK_DIR}"/manifest` packages_with_versions=`cat "${HWPACK_DIR}"/manifest` if [ "$INSTALL_LATEST" = "yes" ]; then packages="${packages_without_versions}" else packages="${packages_with_versions}" fi if [ "$DEP_PACKAGE_PRESENT" = "yes" ]; then to_be_installed= for package in $packages_without_versions; do if [ "${package}" != "${dependency_package}" ]; then { dpkg --get-selections $package 2>/dev/null| grep -qw 'install$'; } || to_be_installed="$to_be_installed $package" fi done fi $sudo apt-get $FORCE_OPTIONS -o "$APT_GET_OPTIONS" install ${packages} if [ "$DEP_PACKAGE_PRESENT" = "yes" ]; then if [ -n "${to_be_installed}" ]; then $sudo apt-get $FORCE_OPTIONS -o "$APT_GET_OPTIONS" markauto ${to_be_installed} fi fi } extract_kernel_packages() { echo "Extracting all kernel packages ..." # We assume the hwpack is always available at the rootfs ROOTFS_DIR=$(dirname $HWPACK_TARBALL) ls ${HWPACK_DIR}/pkgs/linux-[ih]*.deb | while read pkg; do echo "Extracting package `basename $pkg`" dpkg-deb -x ${pkg} $ROOTFS_DIR done ls ${HWPACK_DIR}/pkgs/*-pre-boot_*.deb | while read pkg; do echo "Extracting package `basename $pkg`" dpkg-deb -x ${pkg} $ROOTFS_DIR done ls ${HWPACK_DIR}/pkgs/uefi-image-*.deb | while read pkg; do echo "Extracting package `basename $pkg`" dpkg-deb -x ${pkg} $ROOTFS_DIR done # manually generate modules.dep ls $ROOTFS_DIR/lib/modules | while read kernel; do depmod -b $ROOTFS_DIR ${kernel} || true done; } cleanup() { # Ensure our temp dir and apt sources are removed. echo -n "Cleaning up ..." rm -rf $TEMP_DIR if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then rm -f /usr/sbin/policy-rc.d mv -f /sbin/start-stop-daemon.REAL /sbin/start-stop-daemon if [ -x /sbin/initctl.REAL ]; then mv -f /sbin/initctl.REAL /sbin/initctl fi # Do two updates. The first doesn't try to download package lists: # * First update doesn't access net # - not allowed to fail. Image file + hwpack should contain all packages # needed to create image. If this update fails we have problems. # * Second update may fail # - If can't download package updates (the only difference between the two # commands), we should still be OK. $sudo apt-get update -qq --no-download --ignore-missing $sudo apt-get update -qq || true fi echo "Done" } ## main # Try to acquire fd #9 (i.e. /var/lock/hwpack) # Using 9 as the file descriptor because of https://launchpad.net/bugs/249620 exec 9>$LOCKFILE flock -n 9 || die "Could not acquire lock: $LOCKFILE" # From now on we'll be making changes to the system, so we need to clean # things up when the script exits. trap cleanup EXIT # Extract and set up the hwpack at the rootfs setup_hwpack # In case we only care about the kernel, don't mess up with the system if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then setup_apt_sources setup_ubuntu_rootfs install_deb_packages else extract_kernel_packages fi echo "Done" linaro-image-tools-2014.11/.testr.conf0000664000175000017500000000015612400111666020541 0ustar fboudrafboudra00000000000000[DEFAULT] test_command=python -m subunit.run $IDLIST test_id_list_default=linaro_image_tools.tests.test_suite linaro-image-tools-2014.11/PKG-INFO0000664000175000017500000000075612435067365017574 0ustar fboudrafboudra00000000000000Metadata-Version: 1.1 Name: linaro-image-tools Version: 2014.11 Summary: Tools to create and write Linaro images Home-page: https://launchpad.net/linaro-image-tools Author: Linaro Infrastructure team Author-email: linaro-dev@lists.linaro.org License: GPL v3 or later Description: UNKNOWN Platform: UNKNOWN Requires: apt.cache Requires: apt.package Requires: apt_pkg Requires: dbus Requires: debian Requires: mock Requires: parted Requires: testtools Requires: yaml Provides: linaro_image_tools linaro-image-tools-2014.11/linaro-hwpack-convert0000775000175000017500000000413612400111666022620 0ustar fboudrafboudra00000000000000#!/usr/bin/python # Copyright (C) 2010, 2011, 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import argparse import sys from linaro_image_tools.hwpack.hwpack_convert import ( HwpackConverter, HwpackConverterException, check_and_validate_args, ) from linaro_image_tools.utils import get_logger from linaro_image_tools.__version__ import __version__ if __name__ == '__main__': parser = argparse.ArgumentParser(version='%(prog)s ' + __version__) parser.add_argument("CONFIG_FILE", help="The configuration file to convert.") parser.add_argument("--out", help="The output file name to write. If none is " "given, the input file name (and path) will be " "used with the '.yaml' suffix.") parser.add_argument("--debug", action="store_true") args = parser.parse_args() logger = get_logger(debug=args.debug) try: input_file, output_file = check_and_validate_args(args) logger.info("Converting '%s' into new YAML format..." % input_file) converter = HwpackConverter(input_file, output_file) except HwpackConverterException, e: logger.error(str(e)) sys.exit(1) converter.convert() logger.info("File '%s' converted in '%s'." % (input_file, output_file)) linaro-image-tools-2014.11/linaro-hwpack-append0000775000175000017500000002035312400111666022406 0ustar fboudrafboudra00000000000000#!/usr/bin/env python # Copyright (C) 2013 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. It adds the feature # to append a debian package into the given hwpack. # We might need to change the manifest and Packages file in the # future to match the hardware pack v2 changes when available. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . # import argparse import atexit import os import sys import tarfile import tempfile import shutil from debian.arfile import ArError from linaro_image_tools.hwpack.packages import ( get_packages_file, FetchedPackage ) from linaro_image_tools.utils import get_logger from linaro_image_tools.__version__ import __version__ logger = None def setup_args_parser(): """Setup the argument parsing. :return The parsed arguments. """ description = "Adds new packages inside a hardware pack." parser = argparse.ArgumentParser(version=__version__, description=description) parser.add_argument("-d", "--debug", action="store_true") parser.add_argument("-t", "--hwpack", required=True, help="The hardware pack to modify") parser.add_argument("-p", "--package", action="append", required=True, help="The debian package to append. Can be repeated " "multiple times.") parser.add_argument("-i", "--inplace", action="store_true", help="Add the packages in place, without creating a " "new hardware pack.") return parser.parse_args() def validate_args(args): """Verify the arguments passed on the command line. :param args: The arguments passed. """ hwpack_path = os.path.abspath(args.hwpack) if not os.path.isfile(hwpack_path): logger.error("Error: provided hardware pack file does not exists, or " "is not a file: {0}.".format(args.hwpack)) sys.exit(1) if not tarfile.is_tarfile(hwpack_path): logger.error("Error: cannot read hardware pack file. Make sure it " "is a supported tar archive.") sys.exit(1) for package in args.package: if not os.path.isfile(os.path.abspath(package)): logger.error("Error: provided package to add does not exists, or " "is not a file: {0}.".format(package)) sys.exit(1) def modify_manifest_file(debpackage_info, hwpack_dir): """Modify the manifest file to include a new package entry. :param debpackage_info: The info to write. :param hwpack_dir: Where the manifest file is located. """ debpack_manifest = os.path.join(hwpack_dir, "manifest") new_debpack_line = '{0}={1}\n'.format(debpackage_info.name, debpackage_info.version) logger.debug("Manifest line: {0}".format(new_debpack_line)) with open(debpack_manifest, "a") as manifest: manifest.write(new_debpack_line) def modify_packages_file(debpack_info, pkgs_dir): """Modify the Packages file to include a new package entry. :param debpack_info: The info to be written. :param pkgs_dir: The directory with the Packages file. """ debpack_Packages_fname = os.path.join(pkgs_dir, "Packages") package_info = get_packages_file([debpack_info]).strip() logger.debug("Packages line:\n{0}".format(package_info)) with open(debpack_Packages_fname, "a") as packages_file: packages_file.write("{0}\n\n".format(package_info)) def has_matching_package(pkg_to_search, dir_to_search): """Search for a matching file name in the provided directory. :param pkg_to_search: The package whose name will be matched. :param dir_to_search: Where to search for a matching name. """ logger.debug("Searching matching packages") package_found = False for pkg in os.listdir(dir_to_search): if os.path.basename(pkg_to_search) == os.path.basename(pkg): package_found = True break return package_found def add_packages_to_hwpack(hwpack, packages_to_add, inplace): """Add the provided packages to the hardware pack. Each package to add will be checked against the already available packages: if a similar one is found (it just need to have the same name), it will be skipped. :param hwpack: The hardware pack where to add the new files. :param packagess_to_add: List of package to add. """ hwpack = os.path.abspath(hwpack) tempdir = tempfile.mkdtemp() pkgs_dir = os.path.join(tempdir, 'pkgs') # Cleanup once done. atexit.register(shutil.rmtree, tempdir) # Unfortunately we cannot operate in memory, Python tar library does not # allow adding files with compressed tarballs. We have to extract it. logger.info("Opening hardware pack {0}...".format(hwpack)) logger.debug("Extracting hardware pack in {0}".format(tempdir)) with tarfile.open(hwpack, "r|gz") as tar_file: tar_file.extractall(tempdir) if not os.path.isdir(pkgs_dir): logger.error("Error: tar file does not include packages directory.") sys.exit(1) # Flag to check if we really need to save the new hwpack. save_hwpack = False for debpackage in packages_to_add: debpackage_path = os.path.abspath(debpackage) if has_matching_package(debpackage_path, pkgs_dir): logger.warning("Found similar package in the tar archive: file " "will not be added.") continue if os.path.isfile(debpackage_path): logger.info("Adding file {0}...".format(debpackage)) try: debpackage_info = FetchedPackage.from_deb(debpackage_path) except ArError: logger.warning("File {0} is invalid, skipping " "it.".format(debpackage)) continue if debpackage_info: logger.debug("Package info data:\n{0}".format(debpackage_info)) modify_manifest_file(debpackage_info, tempdir) modify_packages_file(debpackage_info, pkgs_dir) shutil.copy2(debpackage_path, pkgs_dir) save_hwpack |= True else: logger.warning("Unable to find valid info for package " "{0}.".format(debpackage)) else: logger.warning("File {0} does not exists, skipping " "it.".format(debpackage)) if save_hwpack: if inplace: logger.info("Saving hardware pack {0}...".format(hwpack)) with tarfile.open(hwpack, "w|gz") as tar_file: tar_file.add(tempdir, arcname="") else: save_dir = os.path.dirname(hwpack) # Retrieve the file name without the extensions, and create a new # file name. root_ext, ext1 = os.path.splitext(os.path.basename(hwpack)) root, ext2 = os.path.splitext(root_ext) root += "_new" new_file_name = root + ext2 + ext1 save_file = os.path.join(save_dir, new_file_name) logger.info("Saving new hardware pack {0}...".format(save_file)) with tarfile.open(save_file, "w|gz") as tar_file: tar_file.add(tempdir, arcname="") logger.info("New packages added successfully.") else: logger.info("No packages added. Exiting.") def hwpack_append(): args = setup_args_parser() global logger logger = get_logger(debug=args.debug) validate_args(args) add_packages_to_hwpack(args.hwpack, args.package, args.inplace) if __name__ == '__main__': hwpack_append() linaro-image-tools-2014.11/COPYING.GPL-20000664000175000017500000004325412400111666020274 0ustar fboudrafboudra00000000000000 GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. linaro-image-tools-2014.11/linaro_image_tools/0000775000175000017500000000000012435067365022335 5ustar fboudrafboudra00000000000000linaro-image-tools-2014.11/linaro_image_tools/utils.py0000664000175000017500000003571212400111666024041 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import os import platform import subprocess import re import logging import tempfile import tarfile import sys from linaro_image_tools import cmd_runner DEFAULT_LOGGER_NAME = 'linaro_image_tools' # The boot path in the boot tarball. BOOT_DIR_IN_TARBALL = "boot" # The name of the hwpack file found in the boot tarball. HWPACK_NAME = "config" # dconf keys to disable automount options. AUTOMOUNT_DCONF_KEY = '/org/gnome/desktop/media-handling/automount' AUTOMOUNT_OPEN_DCONF_KEYU = '/org/gnome/desktop/media-handling/automount-open' # try_import was copied from python-testtools 0.9.12 and was originally # licensed under a MIT-style license but relicensed under the GPL in Linaro # Image Tools. # Copyright (c) 2011 Jonathan M. Lange . def try_import(name, alternative=None, error_callback=None): """Attempt to import ``name``. If it fails, return ``alternative``. When supporting multiple versions of Python or optional dependencies, it is useful to be able to try to import a module. :param name: The name of the object to import, e.g. ``os.path`` or ``os.path.join``. :param alternative: The value to return if no module can be imported. Defaults to None. :param error_callback: If non-None, a callable that is passed the ImportError when the module cannot be loaded. """ module_segments = name.split('.') last_error = None while module_segments: module_name = '.'.join(module_segments) try: module = __import__(module_name) except ImportError: last_error = sys.exc_info()[1] module_segments.pop() continue else: break else: if last_error is not None and error_callback is not None: error_callback(last_error) return alternative nonexistent = object() for segment in name.split('.')[1:]: module = getattr(module, segment, nonexistent) if module is nonexistent: if last_error is not None and error_callback is not None: error_callback(last_error) return alternative return module CommandNotFound = try_import('CommandNotFound.CommandNotFound') def path_in_tarfile_exists(path, tar_file): exists = True try: tarinfo = tarfile.open(tar_file, 'r:*') tarinfo.getmember(path) tarinfo.close() except KeyError: exists = False except (tarfile.ReadError, tarfile.CompressionError): exists = False # Fallback to tar command cmd = ['tar', '-tf', tar_file, '--wildcards', '*' + path] proc = cmd_runner.run(cmd, stdout=open('/dev/null', 'w'), stderr=open('/dev/null', 'w')) proc.wait() if proc.returncode == 0: exists = True finally: return exists def verify_file_integrity(sig_file_list): """Verify a list of signature files. The parameter is a list of filenames of gpg signature files which will be verified using gpg. For each of the files it is assumed that there is an sha1 hash file with the same file name minus the '.asc' extension. Each of the sha1 files will be checked using sha1sums. All files listed in the sha1 hash file must be found in the same directory as the hash file. """ gpg_sig_ok = True gpg_out = "" verified_files = [] for sig_file in sig_file_list: hash_file = sig_file[0:-len('.asc')] tmp = tempfile.NamedTemporaryFile() try: cmd_runner.run(['gpg', '--status-file={0}'.format(tmp.name), '--verify', sig_file]).wait() except cmd_runner.SubcommandNonZeroReturnValue: gpg_sig_ok = False gpg_out = gpg_out + tmp.read() tmp.close() if os.path.dirname(hash_file) == '': sha_cwd = None else: sha_cwd = os.path.dirname(hash_file) try: sha1sums_out, _ = cmd_runner.Popen( ['sha1sum', '-c', hash_file], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=sha_cwd ).communicate() except cmd_runner.SubcommandNonZeroReturnValue as inst: sha1sums_out = inst.stdout for line in sha1sums_out.splitlines(): sha1_check = re.search(r'^(.*):\s+OK', line) if sha1_check: verified_files.append(sha1_check.group(1)) return verified_files, gpg_sig_ok, gpg_out def check_file_integrity_and_log_errors(sig_file_list, binary, hwpacks): """ Wrapper around verify_file_integrity that prints error messages to stderr if verify_file_integrity finds any problems. """ verified_files, gpg_sig_pass, _ = verify_file_integrity(sig_file_list) # Check the outputs from verify_file_integrity # Abort if anything fails. logger = logging.getLogger(__name__) if len(sig_file_list): if not gpg_sig_pass: logger.error("GPG signature verification failed.") return False, [] if not os.path.basename(binary) in verified_files: logger.error("OS Binary verification failed") return False, [] for hwpack in hwpacks: if not os.path.basename(hwpack) in verified_files: logger.error("Hwpack {0} verification failed".format(hwpack)) return False, [] for verified_file in verified_files: logger.info('Hash verification of file {0} OK.'.format( verified_file)) return True, verified_files def install_package_providing(command): """Install a package which provides the given command. If we can't find any package which provides it, raise UnableToFindPackageProvidingCommand. If the user denies installing the package, the program exits. """ if CommandNotFound is None: raise UnableToFindPackageProvidingCommand( "CommandNotFound python module does not exist.") packages = CommandNotFound().getPackages(command) if len(packages) == 0: raise UnableToFindPackageProvidingCommand( "Unable to find any package providing %s" % command) # TODO: Ask the user to pick a package when there's more than one that # provides the given command. package, _ = packages[0] output, _ = cmd_runner.run(['apt-get', '-s', 'install', package], stdout=subprocess.PIPE).communicate() to_install = [] for line in output.splitlines(): if line.startswith("Inst"): to_install.append(line.split()[1]) if not to_install: raise UnableToFindPackageProvidingCommand( "Unable to find any package to be installed.") try: print ("In order to use the '%s' command, the following package/s " "have to be installed: %s" % (command, " ".join(to_install))) resp = raw_input("Install? (Y/n) ") if resp.lower() != 'y': print "Package installation is necessary to continue. Exiting." sys.exit(1) print ("Installing required command '%s' from package '%s'..." % (command, package)) cmd_runner.run(['apt-get', '--yes', 'install', package], as_root=True).wait() except EOFError: raise PackageInstallationRefused( "Package installation interrupted: input error.") except KeyboardInterrupt: raise PackageInstallationRefused( "Package installation interrupted by the user.") def has_command(command): """Check the given command is available.""" try: cmd_runner.run( ['which', command], stdout=open('/dev/null', 'w')).wait() return True except cmd_runner.SubcommandNonZeroReturnValue: return False def ensure_command(command): """Ensure the given command is available. If it's not, look up a package that provides it and install that. """ if not has_command(command): install_package_providing(command) def find_command(name, prefer_dir=None): """Finds a linaro-image-tools command. Prefers specified directory, otherwise searches only the current directory when running from a checkout, or only PATH when running from an installed version. """ assert name != "" assert os.path.dirname(name) == "" cmd_runner.sanitize_path(os.environ) # default to searching in current directory when running from a bzr # checkout dirs = [os.getcwd(), ] if os.path.isabs(__file__): dirs = os.environ["PATH"].split(os.pathsep) # empty dir in PATH means current directory dirs = map(lambda x: x == '' and '.' or x, dirs) if prefer_dir is not None: dirs.insert(0, prefer_dir) for dir in dirs: path = os.path.join(dir, name) if os.path.exists(path) and os.access(path, os.X_OK): return path return None def is_arm_host(): return platform.machine().startswith('arm') def preferred_tools_dir(): prefer_dir = None # running from bzr checkout? if not os.path.isabs(__file__): prefer_dir = os.getcwd() return prefer_dir def prep_media_path(args): if args.directory is not None: loc = os.path.abspath(args.directory) try: os.makedirs(loc) except OSError: # Directory exists. pass path = os.path.join(loc, args.device) else: path = args.device return path class UnableToFindPackageProvidingCommand(Exception): """We can't find a package which provides the given command.""" class PackageInstallationRefused(Exception): """User has chosen not to install a package.""" class InvalidHwpackFile(Exception): """The hwpack parameter is not a regular file.""" class MissingRequiredOption(Exception): """A required option from the command line is missing.""" def __init__(self, value): self.value = value def __str__(self): return repr(self.value) class IncompatibleOptions(Exception): def __init__(self, value): self.value = value def __str__(self): return repr(self.value) def additional_option_checks(args): if args.directory is not None: # If args.device is a path to a device (/dev/) then this is an error if "--mmc" in sys.argv: raise IncompatibleOptions("--directory option incompatible with " "option --mmc") # If directory is used as well as having a full path (rather than just # a file name or relative path) in args.device, this is an error. if re.search(r"^/", args.device): raise IncompatibleOptions("--directory option incompatible with " "a full path in --image-file") for hwpack in args.hwpacks: if not os.path.isfile(hwpack): raise InvalidHwpackFile( "--hwpack argument (%s) is not a regular file" % hwpack) def additional_android_option_checks(args): """Checks that some of the args passed to l-a-m-c are valid.""" if args.hwpack: if not os.path.isfile(args.hwpack): raise InvalidHwpackFile( "--hwpack argument (%s) is not a regular file" % args.hwpack) def android_hwpack_in_boot_tarball(boot_dir): """Simple check for existence of a path. Needed to make cli command testable in some way. :param boot_dir: The path where the boot tarball has been extracted. :type str :return A tuple with a bool if the path exists, and the path to the config file. """ conf_file = os.path.join(boot_dir, BOOT_DIR_IN_TARBALL, HWPACK_NAME) return os.path.exists(conf_file), conf_file def check_required_args(args): """Check that the required args are passed.""" if args.dev is None: raise MissingRequiredOption("--dev option is required") if args.binary is None: raise MissingRequiredOption("--binary option is required") def get_logger(name=DEFAULT_LOGGER_NAME, debug=False): """ Retrieves a named logger. Default name is set in the variable DEFAULT_LOG_NAME. Debug is set to False by default. :param name: The name of the logger. :param debug: If debug level should be turned on :return: A logger instance. """ logger = logging.getLogger(name) ch = logging.StreamHandler() if debug: ch.setLevel(logging.DEBUG) formatter = logging.Formatter( "%(asctime)s - %(name)s - %(levelname)s - %(message)s") ch.setFormatter(formatter) logger.setLevel(logging.DEBUG) else: ch.setLevel(logging.INFO) formatter = logging.Formatter("%(message)s") ch.setFormatter(formatter) logger.setLevel(logging.INFO) logger.addHandler(ch) return logger def disable_automount(): """Disables the desktop environment automount option. This will work only under GNOME with dconf installed. """ logger = logging.getLogger(DEFAULT_LOGGER_NAME) if has_command('dconf'): logger.info("Disabling desktop environment automount option.") try: cmd_runner.run( ['dconf', 'write', AUTOMOUNT_DCONF_KEY, 'false'], stdout=open('/dev/null', 'w')).wait() cmd_runner.run( ['dconf', 'write', AUTOMOUNT_OPEN_DCONF_KEYU, 'false'], stdout=open('/dev/null', 'w')).wait() except cmd_runner.SubcommandNonZeroReturnValue: logger.error("Error disabling desktop environemnt automount.") def enable_automount(): """Re-enables back the desktop environment automount option. This will work only under GNOME with dconf installed. It should be run as an atexit function. """ logger = logging.getLogger(DEFAULT_LOGGER_NAME) if has_command('dconf'): try: cmd_runner.run( ['dconf', 'write', AUTOMOUNT_DCONF_KEY, 'true'], stdout=open('/dev/null', 'w')).wait() cmd_runner.run( ['dconf', 'write', AUTOMOUNT_OPEN_DCONF_KEYU, 'true'], stdout=open('/dev/null', 'w')).wait() except cmd_runner.SubcommandNonZeroReturnValue: logger.error("Error enabling back desktop environemnt automount.") linaro-image-tools-2014.11/linaro_image_tools/hwpack/0000775000175000017500000000000012435067365023612 5ustar fboudrafboudra00000000000000linaro-image-tools-2014.11/linaro_image_tools/hwpack/__init__.py0000664000175000017500000000207312400111666025707 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import logging from linaro_image_tools.utils import DEFAULT_LOGGER_NAME class NullHandler(logging.Handler): def emit(self, record): pass h = NullHandler() logging.getLogger(DEFAULT_LOGGER_NAME).addHandler(h) linaro-image-tools-2014.11/linaro_image_tools/hwpack/hwpack_reader.py0000664000175000017500000001531112400111666026746 0ustar fboudrafboudra00000000000000# Copyright (C) 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . from linaro_image_tools.hwpack.handler import HardwarepackHandler from linaro_image_tools.hwpack.hwpack_fields import ( FORMAT_FIELD, NAME_FIELD, BOARDS_FIELD, BOOTLOADERS_FIELD, ) from os import linesep as LINE_SEP # Fields necessary for the string representation of the hardware pack supported # boards and bootlaoders. HALF_SEPARATOR = '+--------------------------------------' ENDING = '+' SEPARATOR = HALF_SEPARATOR * 2 + ENDING FORMAT = '{:<80}' CENTER_ALIGN = '{:^80}' ELEMENT_FORMAT = '{:<39}| {:<39}' class HwpackReaderError(Exception): """General error raised by HwpackReader.""" def __init__(self, value): self.value = value def __str__(self): return repr(self.value) class Hwpack(object): """A simple representation of an hardware pack and its value.""" def __init__(self): self._hwpack = None self._boards = None self._bootloaders = None self._name = None @property def hwpack(self): """The hardware pack it refers to.""" return self._hwpack def sethwpack(self, value): """Sets the hwpack field.""" self._hwpack = value @property def boards(self): """The boards field of this hardware pack.""" return self._boards def setboards(self, value): """Sets the boards field.""" self._boards = value @property def name(self): """The name field of this hardware pack.""" return self._name def setname(self, value): """Sets the name field.""" self._name = value @property def bootloaders(self): """The bootlaoders field of this hardware pack.""" return self._bootloaders def setbootloaders(self, value): """Sets the bootlaoders field.""" self._bootloaders = value def __eq__(self, other): """Equality method.""" equal = False if isinstance(other, Hwpack): equal = (self.name == other.name and self.boards == other.boards and self.hwpack == other.hwpack and self.bootloaders == other.bootloaders) return equal def __hash__(self): return hash(frozenset(self.bootloaders), frozenset(self.boards), self.name, self.hwpack) def __str__(self): """String representation of this hwapack supported elements.""" string = FORMAT.format("Read hardware pack: %s" % self.hwpack) string += LINE_SEP string += FORMAT.format(SEPARATOR) string += LINE_SEP string += ELEMENT_FORMAT.format("Supported boards", "Supported bootloaders") string += LINE_SEP string += FORMAT.format(SEPARATOR) string += LINE_SEP if self.boards: for key, value in self.boards.iteritems(): if value.get(BOOTLOADERS_FIELD, None) is not None: bootloaders = value.get(BOOTLOADERS_FIELD) supported_bootloaders = bootloaders.keys() else: supported_bootloaders = self.bootloaders.keys() string += ELEMENT_FORMAT.format( key, ",".join(supported_bootloaders)) string += LINE_SEP else: # If we pass a converted file with just a single board, we do not # have the boards section, and we default to the name of the hwpack if self.bootloaders: supported_bootloaders = self.bootloaders.keys() string += ELEMENT_FORMAT.format( self.name, ",".join(supported_bootloaders)) string += LINE_SEP else: string += CENTER_ALIGN.format("No supported boards and " "bootloaders") string += LINE_SEP string += FORMAT.format(SEPARATOR) return string + LINE_SEP class HwpackReader(object): """Reads the information contained in a hwpack """ def __init__(self, hwpacks): """Create a new instance. :param hwpacks: The list of hardware packs to read from.""" self.hwpacks = hwpacks # Where we store all the info from the hwpack. self._supported_elements = [] @property def supported_elements(self): """Gets the supported elements of by all the hardwapare packs.""" return self._supported_elements def _read_hwpacks_metadata(self): """Reads the hardware pack metadata file, and prints information about the supported boards and bootloaders.""" for tarball in self.hwpacks: with HardwarepackHandler([tarball]) as handler: hwpack_format = handler.get_field(FORMAT_FIELD)[0] if hwpack_format.format_as_string == "3.0": local_hwpack = Hwpack() local_hwpack.sethwpack(tarball) local_hwpack.setname(handler.get_field(NAME_FIELD)[0]) local_hwpack.setboards(handler.get_field(BOARDS_FIELD)[0]) local_hwpack.setbootloaders( handler.get_field(BOOTLOADERS_FIELD)[0]) self.supported_elements.append(local_hwpack) else: raise HwpackReaderError("Hardwarepack '%s' cannot be " "read, unsupported format." % (tarball)) def get_supported_boards(self): """Prints the necessary information. :return A string representation of the information.""" self._read_hwpacks_metadata() return str(self) def __str__(self): """The string representation of this reader. It is a printable representation of the necessary information.""" hwpack_reader = "" for element in self.supported_elements: hwpack_reader += str(element) return hwpack_reader linaro-image-tools-2014.11/linaro_image_tools/hwpack/hwpack_convert.py0000664000175000017500000003106612400111666027171 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011, 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import ConfigParser import logging import os import os.path import re import yaml from hwpack_fields import ( ARCHITECTURES_FIELD, ASSUME_INSTALLED_FIELD, BOOTLOADERS_FIELD, EXTRA_BOOT_OPTIONS_FIELD, EXTRA_SERIAL_OPTIONS_FIELD, SOURCES_FIELD, FORMAT_FIELD, PACKAGES_FIELD, PACKAGE_FIELD, FILE_FIELD, IN_BOOT_PART_FIELD, DD_FIELD, DTB_FILE_FIELD, DTB_FILES_FIELD, ENV_DD_FIELD, SPL_IN_BOOT_PART_FIELD, SPL_DD_FIELD, SPL_PACKAGE_FIELD, SPL_FILE_FIELD, WIRED_INTERFACES_FIELD, WIRELESS_INTERFACES_FIELD, INCLUDE_DEBS_FIELD, ) # This is the main section of an INI-style hwpack config file. MAIN_SECTION = 'hwpack' # The suffix for the new file NEW_FILE_SUFFIX = '.yaml' # How many spaces should be used for indentation. INDENT_STEP = 1 # Regular expression to convert from Yes/No values into Boolean. YES_REGEX = '[Yy]es' NO_REGEX = '[Nn]o' # The default format number. DEFAULT_FORMAT = '3.0' # Old INI style u_boot keys name. UBOOT_PACKAGE_KEY = "u_boot_package" UBOOT_FILE_KEY = "u_boot_file" UBOOT_IN_BOOT_PART_KEY = 'u_boot_in_boot_part' UBOOT_DD_KEY = 'u_boot_dd' # All the u_boot defined keys in a list. UBOOT_KEYS = [UBOOT_PACKAGE_KEY, UBOOT_FILE_KEY, UBOOT_IN_BOOT_PART_KEY, UBOOT_DD_KEY] # Old field, the only one with a dash: since the format is new, convert it. ASSUME_INSTALLED_OLD = 'assume-installed' INCLUDE_DEBS_OLD = 'include-debs' # The default bootloader for the bootloaders section. DEFAULT_BOOTLOADER = 'u_boot' # All the SPL keys SPL_KEYS = [SPL_IN_BOOT_PART_FIELD, SPL_DD_FIELD, SPL_PACKAGE_FIELD, SPL_FILE_FIELD, ENV_DD_FIELD] # The default name used for renaming dtb file DEFAULT_DTB_NAME = 'board.dtb' logger = logging.getLogger(__name__) class HwpackConverterException(Exception): """General exception class for the converter.""" class HwpackConverter(object): """Simple and basic class that converts an INI-style format file into the new YAML format. The old format number is maintained. :param input_file: the input file to parse, has to be an INI-style file. :param output_file: where to write the new file, if not given, the name of the input file will be used adding the 'yaml' suffix. """ def __init__(self, input_file=None, output_file=None): """Initializie the class.""" self.input_file = input_file self.output_file = output_file # Where we store the list of sources. self.sources = {} # Where we store all the information of the hwpack config file # In this case we have one board per hwpack config file. self.hwpack = {} # List of supported architectures. self.architectures = [] # Where we hold bootloaders info self.bootloaders = {} # List to store extra boot options. self.extra_boot_options = [] # The list of packages. self.packages = [] # List of the extra_serial_options. self.extra_serial_options = [] # Lists for network interfaces. self.wired_interfaces = [] self.wireless_interfaces = [] # SPL entries self.spl = {} # The list of packages that should be installed. self.assume_installed = [] # The dtb_files section self.dtb_files = [] def _parse(self): """Parses the config file and stores its values.""" if self.input_file is not None: parser = ConfigParser.RawConfigParser() with open(self.input_file, 'r') as fp: parser.readfp(fp) # Iterate through all the file sections. for section in parser.sections(): if section == MAIN_SECTION: for key, value in parser.items(section): if value is not None: if re.match("[Yy]es", value): value = True elif re.match("[Nn]o", value): value = False if key == ARCHITECTURES_FIELD: self.parse_list_string( self.architectures, value) continue elif key == EXTRA_BOOT_OPTIONS_FIELD: self.parse_list_string( self.extra_boot_options, value) continue elif key == EXTRA_SERIAL_OPTIONS_FIELD: self.parse_list_string( self.extra_serial_options, value) continue elif key == WIRED_INTERFACES_FIELD: self.parse_list_string(self.wired_interfaces, value) continue elif key == WIRELESS_INTERFACES_FIELD: self.parse_list_string( self.wireless_interfaces, value) continue elif key in SPL_KEYS: self.spl[key] = value continue elif key == FORMAT_FIELD: value = DEFAULT_FORMAT elif key == PACKAGES_FIELD: self.parse_list_string(self.packages, value) continue elif key in UBOOT_KEYS: self._set_bootloaders(key, value) continue # Create list. elif key == ASSUME_INSTALLED_OLD: self.parse_list_string( self.assume_installed, value) continue elif key == DTB_FILE_FIELD: self.dtb_files.append({DEFAULT_DTB_NAME: value}) continue elif key == INCLUDE_DEBS_OLD: key = INCLUDE_DEBS_FIELD self.hwpack[key] = value else: # Here we have only sources sections. for _, value in parser.items(section): if value is not None: self.sources[section] = value def _set_bootloaders(self, key, value): """Sets the bootloaders dictionary of a new YAML file. Converts from the old INI keys name into the new ones. :param key: The key of the bootloader. :param value: The key value.""" if key == UBOOT_PACKAGE_KEY: self.bootloaders[PACKAGE_FIELD] = value elif key == UBOOT_FILE_KEY: self.bootloaders[FILE_FIELD] = value elif key == UBOOT_IN_BOOT_PART_KEY: self.bootloaders[IN_BOOT_PART_FIELD] = value elif key == UBOOT_DD_KEY: self.bootloaders[DD_FIELD] = value def parse_list_string(self, store, string, split=" "): """Parses a string of listed values, and stores the single splitted value in the provided list. :param store: The list where to store the values. :param string: The string that should be splitted. :param split: The separator to use, defaults to empty space. """ if not isinstance(store, list): raise HwpackConverterException("Can use this method only with " "list.") store.extend(string.split(" ")) def _to_file(self): """Writes the converted hwpack to file.""" with open(self.output_file, 'w') as fp: fp.write(str(self)) def convert(self): """Converts the input file into the output file with the new format. """ self._parse() self._to_file() def __str__(self): """Readable representation of the converted hwpack. :return A YAML-string representation of the hwpack configuration. """ converted = '' if self.hwpack: converted += dump(self.hwpack) if self.architectures: archs = {ARCHITECTURES_FIELD: self.architectures} converted += dump(archs) if self.assume_installed: installed = {ASSUME_INSTALLED_FIELD: self.assume_installed} converted += dump(installed) if self.extra_serial_options: serial_options = {EXTRA_SERIAL_OPTIONS_FIELD: self.extra_serial_options} converted += dump(serial_options) if self.packages: packages = {PACKAGES_FIELD: self.packages} converted += dump(packages) if self.wired_interfaces: wired = {WIRED_INTERFACES_FIELD: self.wired_interfaces} converted += dump(wired) if self.wireless_interfaces: converted += dump(self.wireless_interfaces) if self.sources: sources = {SOURCES_FIELD: self.sources} converted += dump(sources) if self.dtb_files: dtb = {DTB_FILES_FIELD: self.dtb_files} converted += dump(dtb) if self.bootloaders or self.extra_boot_options or self.spl: # The bootloaders section in the new YAML file is a dictionary # containing a dictionary which can contains also other # dictionaries. In this case we only have list and normal values. nested_value = {} if self.bootloaders: for key, value in self.bootloaders.iteritems(): nested_value[key] = value if self.extra_boot_options: nested_value[EXTRA_BOOT_OPTIONS_FIELD] = \ self.extra_boot_options if self.spl: for key, value in self.spl.iteritems(): nested_value[key] = value default_bootloader = {DEFAULT_BOOTLOADER: nested_value} bootloaders = {BOOTLOADERS_FIELD: default_bootloader} converted += dump(bootloaders) return converted def dump(python_object): """Serialize a Python object in a YAML string format. :param python_object: The object to serialize. """ return yaml.dump(python_object, default_flow_style=False) def check_and_validate_args(args): """Assures that the args passed are valid. :param args: the args as defined in linaro-hwpack-convert. """ input_file = args.CONFIG_FILE output_file = args.out if not os.path.exists(input_file) or not os.path.isfile(input_file): raise HwpackConverterException("The configuration file '%s' is not a " "regular file." % input_file) if output_file is not None: if os.path.exists(output_file) or os.path.isdir(output_file): raise HwpackConverterException("The output file name provided " "'%s' already exists, or is a " "directory." % output_file) elif not os.path.isabs(output_file): # If we output file is just a name, write it in the current dir. output_file = os.path.join(os.getcwd(), output_file) else: output_file = input_file + NEW_FILE_SUFFIX return (input_file, output_file) linaro-image-tools-2014.11/linaro_image_tools/hwpack/hardwarepack.py0000664000175000017500000006045212400111666026611 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import time import os import urlparse from linaro_image_tools.hwpack.better_tarfile import writeable_tarfile from linaro_image_tools.hwpack.packages import ( FetchedPackage, get_packages_file, PackageMaker, ) from linaro_image_tools.hwpack.hardwarepack_format import ( HardwarePackFormatV1, ) from linaro_image_tools.hwpack.hwpack_convert import ( dump, ) from hwpack_fields import ( BOARDS_FIELD, BOOTLOADERS_FIELD, BOOT_MIN_SIZE_FIELD, BOOT_SCRIPT_FIELD, DTB_ADDR_FIELD, DTB_FILE_FIELD, DTB_FILES_FIELD, EXTRA_SERIAL_OPTIONS_FIELD, FORMAT_FIELD, INITRD_ADDR_FIELD, INITRD_FILE_FIELD, KERNEL_ADDR_FIELD, KERNEL_FILE_FIELD, LOAD_ADDR_FIELD, LOADER_MIN_SIZE_FIELD, LOADER_START_FIELD, MAINTAINER_FIELD, METADATA_ARCH_FIELD, METADATA_VERSION_FIELD, MMC_ID_FIELD, NAME_FIELD, ORIGIN_FIELD, PARTITION_LAYOUT_FIELD, ROOT_MIN_SIZE_FIELD, SAMSUNG_BL1_LEN_FIELD, SAMSUNG_BL1_START_FIELD, SAMSUNG_BL2_LEN_FIELD, SAMSUNG_BL2_START_FIELD, SAMSUNG_ENV_LEN_FIELD, SAMSUNG_ENV_START_FIELD, SERIAL_TTY_FIELD, SNOWBALL_STARTUP_FILES_CONFIG_FIELD, SUPPORT_FIELD, WIRED_INTERFACES_FIELD, WIRELESS_INTERFACES_FIELD, ) class Metadata(object): """Metadata for a hardware pack. This metadata is what is required and optional for the metadata file that ends up in the hardware pack. In addition str() can be used to create the contents of the metdata file for the hardware pack that this metadata file refers to. :ivar name: the name of the hardware pack. :type name: str :ivar version: the version of the hardware pack. It must not contain white spaces. :type version: str :ivar origin: the origin of the hardware pack, or None if the origin is not known. :type origin: str or None :ivar maintainer: the maintainer of the hardware pack, or None if not known. :type maintainer: str or None :ivar support: the support status of this hardware pack, or None if not know. :type support: str or None """ def __init__(self, name, version, architecture, origin=None, maintainer=None, support=None, format=HardwarePackFormatV1()): """Create the Metadata for a hardware pack. See the instance variables for a description of the arguments. """ self.format = format self.name = name if ' ' in version: raise AssertionError( 'Hardware pack version must not contain white ' 'spaces: "%s"' % version) self.version = version self.origin = origin self.maintainer = maintainer self.support = support self.architecture = architecture @classmethod def add_v2_config(self, serial_tty=None, kernel_addr=None, initrd_addr=None, load_addr=None, dtb_file=None, wired_interfaces=[], wireless_interfaces=[], partition_layout=None, mmc_id=None, boot_min_size=None, root_min_size=None, loader_min_size=None, vmlinuz=None, initrd=None, dtb_addr=None, extra_boot_options=None, env_dd=None, boot_script=None, bootloader_file_in_boot_part=None, uboot_dd=None, spl_in_boot_part=None, spl_dd=None, extra_serial_options=None, loader_start=None, snowball_startup_files_config=None, samsung_bl1_start=None, samsung_bl1_len=None, samsung_env_start=None, samsung_env_len=None, samsung_bl2_start=None, samsung_bl2_len=None): """Add fields that are specific to the new format. These fields are not present in earlier config files. """ self.u_boot = None self.spl = None self.serial_tty = serial_tty self.kernel_addr = kernel_addr self.initrd_addr = initrd_addr self.load_addr = load_addr self.wired_interfaces = wired_interfaces self.wireless_interfaces = wireless_interfaces self.partition_layout = partition_layout self.mmc_id = mmc_id self.boot_min_size = boot_min_size self.root_min_size = root_min_size self.loader_min_size = loader_min_size self.loader_start = loader_start self.vmlinuz = vmlinuz self.initrd = initrd self.dtb_file = dtb_file self.dtb_addr = dtb_addr self.extra_boot_options = extra_boot_options self.boot_script = boot_script self.bootloader_file_in_boot_part = bootloader_file_in_boot_part self.uboot_dd = uboot_dd self.spl_in_boot_part = spl_in_boot_part self.spl_dd = spl_dd self.env_dd = env_dd self.extra_serial_options = extra_serial_options self.snowball_startup_files_config = snowball_startup_files_config self.samsung_bl1_start = samsung_bl1_start self.samsung_bl1_len = samsung_bl1_len self.samsung_env_start = samsung_env_start self.samsung_env_len = samsung_env_len self.samsung_bl2_start = samsung_bl2_start self.samsung_bl2_len = samsung_bl2_len @classmethod def add_v3_config(self, boards=None, bootloaders=None, dtb_files=None): """Add fields that are specific to the v3 config format. These fields are not present in the earlier config files. :param boards: The boards section of the hwpack. :param bootloaders: The bootloaders section of the hwpack. :param dtb_files: The dtb_files section of the hwpack.""" self.boards = boards self.bootloaders = bootloaders self.dtb_files = dtb_files @classmethod def from_config(cls, config, version, architecture): """Create a Metadata from a Config object. As a Config will contain most of the information needed for a Metadata, we can provide this convenient way to construct one. Information that is not in the config has to be provided by the caller. :param config: the config to take values from. :type config: Config :param version: the version to record in the metadata. :type version: str :param architecture: the architecture that the hardware pack is targetting. :type architecture: str """ metadata = cls( config.name, version, architecture, origin=config.origin, maintainer=config.maintainer, support=config.support, format=config.format) if config.format.has_v2_fields: # Helper variable to adhere to the line length limit. snowball_startup_config = config.snowball_startup_files_config metadata.add_v2_config( boot_min_size=config.boot_min_size, boot_script=config.boot_script, dtb_addr=config.dtb_addr, dtb_file=config.dtb_file, env_dd=config.env_dd, extra_boot_options=config.extra_boot_options, extra_serial_options=config.extra_serial_options, initrd_addr=config.initrd_addr, initrd=config.initrd, kernel_addr=config.kernel_addr, load_addr=config.load_addr, loader_min_size=config.loader_min_size, loader_start=config.loader_start, mmc_id=config.mmc_id, partition_layout=config.partition_layout, root_min_size=config.root_min_size, samsung_bl1_len=config.samsung_bl1_len, samsung_bl1_start=config.samsung_bl1_start, samsung_bl2_len=config.samsung_bl2_len, samsung_bl2_start=config.samsung_bl2_start, samsung_env_len=config.samsung_env_len, samsung_env_start=config.samsung_env_start, serial_tty=config.serial_tty, snowball_startup_files_config=snowball_startup_config, spl_dd=config.spl_dd, spl_in_boot_part=config.spl_in_boot_part, uboot_dd=config.bootloader_dd, bootloader_file_in_boot_part=config. bootloader_file_in_boot_part, vmlinuz=config.vmlinuz, wired_interfaces=config.wired_interfaces, wireless_interfaces=config.wireless_interfaces, ) if config.format.format_as_string == '3.0': metadata.add_v3_config(boards=config.boards, bootloaders=config.bootloaders, dtb_files=config.dtb_files) return metadata def __str__(self): if self.format.format_as_string == '3.0': return self.create_metadata_new() else: return self.create_metadata_old() def create_metadata_new(self): """Get the contents of the metadata file. The metadata file is almost an identical copy of the hwpack configuration file. Only a couple of fields are different, and some are missing. :return A string. """ metadata = "" metadata += dump({FORMAT_FIELD: self.format.format_as_string}) metadata += dump({NAME_FIELD: self.name}) metadata += dump({METADATA_VERSION_FIELD: self.version}) # This is a single 'architecture' hwpack, each arch will get its own, # it is not retrieved from the Config. metadata += dump({METADATA_ARCH_FIELD: self.architecture}) if self.origin is not None: metadata += dump({ORIGIN_FIELD: self.origin}) if self.maintainer is not None: metadata += dump({MAINTAINER_FIELD: self.maintainer}) if self.support is not None: metadata += dump({SUPPORT_FIELD: self.support}) if self.boards is not None: metadata += dump({BOARDS_FIELD: self.boards}) if self.bootloaders is not None: metadata += dump({BOOTLOADERS_FIELD: self.bootloaders}) if self.serial_tty is not None: metadata += dump({SERIAL_TTY_FIELD: self.serial_tty}) if self.kernel_addr is not None: metadata += dump({KERNEL_ADDR_FIELD: self.kernel_addr}) if self.initrd_addr is not None: metadata += dump({INITRD_ADDR_FIELD: self.initrd_addr}) if self.load_addr is not None: metadata += dump({LOAD_ADDR_FIELD: self.load_addr}) if self.dtb_addr is not None: metadata += dump({DTB_ADDR_FIELD: self.dtb_addr}) if self.wired_interfaces != []: eth_interfaces = " ".join(self.wired_interfaces) metadata += dump({WIRED_INTERFACES_FIELD: eth_interfaces}) if self.wireless_interfaces != []: wifi_interfaces = " ".join(self.wireless_interfaces) metadata += dump({WIRELESS_INTERFACES_FIELD: wifi_interfaces}) if self.partition_layout is not None: metadata += dump({PARTITION_LAYOUT_FIELD: self.partition_layout}) if self.mmc_id is not None: metadata += dump({MMC_ID_FIELD: self.mmc_id}) if self.boot_min_size is not None: metadata += dump({BOOT_MIN_SIZE_FIELD: self.boot_min_size}) if self.root_min_size is not None: metadata += dump({ROOT_MIN_SIZE_FIELD: self.root_min_size}) if self.loader_min_size is not None: metadata += dump({LOADER_MIN_SIZE_FIELD: self.loader_min_size}) if self.loader_start is not None: metadata += dump({LOADER_START_FIELD: self.loader_start}) if self.vmlinuz is not None: metadata += dump({KERNEL_FILE_FIELD: self.vmlinuz}) if self.initrd is not None: metadata += dump({INITRD_FILE_FIELD: self.initrd}) if self.dtb_file is not None: # XXX In V3 this one should be a list, called dtb_files. metadata += dump({DTB_FILE_FIELD: self.dtb_file}) if self.dtb_files is not None: metadata += dump({DTB_FILES_FIELD: self.dtb_files}) if self.boot_script is not None: metadata += dump({BOOT_SCRIPT_FIELD: self.boot_script}) if self.extra_serial_options is not None: # XXX Check why and where once we get a list once a string. if isinstance(self.extra_serial_options, list): extra_serial_options = " ".join(self.extra_serial_options) else: extra_serial_options = self.extra_serial_options metadata += dump({ EXTRA_SERIAL_OPTIONS_FIELD: extra_serial_options}) if self.snowball_startup_files_config is not None: metadata += dump({SNOWBALL_STARTUP_FILES_CONFIG_FIELD: self.snowball_startup_files_config}) if self.samsung_bl1_start is not None: metadata += dump({SAMSUNG_BL1_START_FIELD: self.samsung_bl1_start}) if self.samsung_bl1_len is not None: metadata += dump({SAMSUNG_BL1_LEN_FIELD: self.samsung_bl1_len}) if self.samsung_env_start is not None: metadata += dump({SAMSUNG_ENV_START_FIELD: self.samsung_env_start}) if self.samsung_env_len is not None: metadata += dump({SAMSUNG_ENV_LEN_FIELD: self.samsung_env_len}) if self.samsung_bl2_start is not None: metadata += dump({SAMSUNG_BL2_START_FIELD: self.samsung_bl2_start}) if self.samsung_bl2_len is not None: metadata += dump({SAMSUNG_BL2_LEN_FIELD: self.samsung_bl2_len}) return metadata def create_metadata_old(self): """Get the contents of the metadata file. Creates a metadata file for v1 and v2 of the hwpack config file. """ metadata = "NAME=%s\n" % self.name metadata += "VERSION=%s\n" % self.version metadata += "ARCHITECTURE=%s\n" % self.architecture if self.origin is not None: metadata += "ORIGIN=%s\n" % self.origin if self.maintainer is not None: metadata += "MAINTAINER=%s\n" % self.maintainer if self.support is not None: metadata += "SUPPORT=%s\n" % self.support if not self.format.has_v2_fields: return metadata if self.u_boot is not None: metadata += "U_BOOT=%s\n" % self.u_boot if self.spl is not None: metadata += "SPL=%s\n" % self.spl if self.serial_tty is not None: metadata += "SERIAL_TTY=%s\n" % self.serial_tty if self.kernel_addr is not None: metadata += "KERNEL_ADDR=%s\n" % self.kernel_addr if self.initrd_addr is not None: metadata += "INITRD_ADDR=%s\n" % self.initrd_addr if self.load_addr is not None: metadata += "LOAD_ADDR=%s\n" % self.load_addr if self.dtb_addr is not None: metadata += "DTB_ADDR=%s\n" % self.dtb_addr if self.wired_interfaces != []: metadata += "WIRED_INTERFACES=%s\n" % " ".join( self.wired_interfaces) if self.wireless_interfaces != []: metadata += "WIRELESS_INTERFACES=%s\n" % " ".join( self.wireless_interfaces) if self.partition_layout is not None: metadata += "PARTITION_LAYOUT=%s\n" % self.partition_layout if self.mmc_id is not None: metadata += "MMC_ID=%s\n" % self.mmc_id if self.boot_min_size is not None: metadata += "BOOT_MIN_SIZE=%s\n" % self.boot_min_size if self.root_min_size is not None: metadata += "ROOT_MIN_SIZE=%s\n" % self.root_min_size if self.loader_min_size is not None: metadata += "LOADER_MIN_SIZE=%s\n" % self.loader_min_size if self.loader_start is not None: metadata += "LOADER_START=%s\n" % self.loader_start if self.vmlinuz is not None: metadata += "KERNEL_FILE=%s\n" % self.vmlinuz if self.initrd is not None: metadata += "INITRD_FILE=%s\n" % self.initrd if self.dtb_file is not None: metadata += "DTB_FILE=%s\n" % self.dtb_file if self.extra_boot_options is not None: metadata += "EXTRA_BOOT_OPTIONS=%s\n" % self.extra_boot_options if self.boot_script is not None: metadata += "BOOT_SCRIPT=%s\n" % self.boot_script if self.bootloader_file_in_boot_part is not None: metadata += ("U_BOOT_IN_BOOT_PART=%s\n" % self.bootloader_file_in_boot_part) if self.spl_in_boot_part is not None: metadata += "SPL_IN_BOOT_PART=%s\n" % self.spl_in_boot_part if self.uboot_dd is not None: metadata += "U_BOOT_DD=%s\n" % self.uboot_dd if self.spl_dd is not None: metadata += "SPL_DD=%s\n" % self.spl_dd if self.env_dd is not None: metadata += "ENV_DD=%s\n" % self.env_dd if self.extra_serial_options is not None: metadata += "EXTRA_SERIAL_OPTIONS=%s\n" % self.extra_serial_options if self.snowball_startup_files_config is not None: metadata += "SNOWBALL_STARTUP_FILES_CONFIG=%s\n" % ( self.snowball_startup_files_config) if self.samsung_bl1_start is not None: metadata += "SAMSUNG_BL1_START=%s\n" % self.samsung_bl1_start if self.samsung_bl1_len is not None: metadata += "SAMSUNG_BL1_LEN=%s\n" % self.samsung_bl1_len if self.samsung_env_start is not None: metadata += "SAMSUNG_ENV_START=%s\n" % self.samsung_env_start if self.samsung_env_len is not None: metadata += "SAMSUNG_ENV_LEN=%s\n" % self.samsung_env_len if self.samsung_bl2_start is not None: metadata += "SAMSUNG_BL2_START=%s\n" % self.samsung_bl2_start if self.samsung_bl2_len is not None: metadata += "SAMSUNG_BL2_LEN=%s\n" % self.samsung_bl2_len return metadata class HardwarePack(object): """The representation of a hardware pack. :ivar metadata: the metadata of this hardware pack. :type metadata: Metadata :ivar FORMAT: the format of hardware pack that should be created. :type FORMAT: str """ FORMAT_FILENAME = "FORMAT" METADATA_FILENAME = "metadata" MANIFEST_FILENAME = "manifest" PACKAGES_DIRNAME = "pkgs" PACKAGES_FILENAME = "%s/Packages" % PACKAGES_DIRNAME SOURCES_LIST_DIRNAME = "sources.list.d" SOURCES_LIST_GPG_DIRNAME = "sources.list.d.gpg" U_BOOT_DIR = "u-boot" SPL_DIR = "spl" BOOT_DIR = "boot" def __init__(self, metadata): """Create a HardwarePack. :param metadata: the metadata to use. :type metadata: Metadata """ self.metadata = metadata self.sources = {} self.packages = [] self.format = metadata.format self.files = [] def filename(self, extension=".tar.gz"): """The filename that this hardware pack should have. Returns the filename that the hardware pack should have, according to the convention used. :return: the filename that should be used. :rtype: str """ if self.metadata.support is None: support_suffix = "" else: support_suffix = "_%s" % self.metadata.support return "hwpack_%s_%s_%s%s%s" % ( self.metadata.name, self.metadata.version, self.metadata.architecture, support_suffix, extension) def add_apt_sources(self, sources): """Add APT sources to the hardware pack. Given a dict of names and the source lines this will add them to the hardware pack. The names should be an identifier for the source, and the source lines should be what is put in sources.list for that source, minus the "deb" part. If you pass an identifier that has already been passed to this method, then the previous value will be replaced with the new value. :param sources: the sources to use as a dict mapping identifiers to sources entries. :type sources: a dict mapping str to str """ if sources: self.sources.update(sources) def add_packages(self, packages): """Add packages to the hardware pack. Given a list of packages this will add them to the hardware pack. :param packages: the packages to add :type packages: FetchedPackage """ self.packages += packages def add_dependency_package(self, packages_spec): """Add a packge that depends on packages_spec to the hardware pack. :param packages_spec: A list of apt package specifications, e.g. ``['foo', 'bar (>= 1.2)']``. """ with PackageMaker() as maker: dep_package_name = 'hwpack-' + self.metadata.name relationships = {} if packages_spec: relationships = {'Depends': ', '.join(packages_spec)} deb_file_path = maker.make_package( dep_package_name, self.metadata.version, relationships, self.metadata.architecture) self.packages.append(FetchedPackage.from_deb(deb_file_path)) def add_file(self, dir, file): target_file = os.path.join(dir, os.path.basename(file)) self.files.append((file, target_file)) return target_file def manifest_text(self): manifest_content = "" for package in self.packages: manifest_content += "%s=%s\n" % ( package.name, package.version) return manifest_content def to_file(self, fileobj): """Write the hwpack to a file object. The full hardware pack will be written to the file object in gzip compressed tarball form as the spec requires. :param fileobj: the file object to write to. :type fileobj: a file-like object :return: None """ kwargs = {} kwargs["default_uid"] = 1000 kwargs["default_gid"] = 1000 kwargs["default_uname"] = "user" kwargs["default_gname"] = "group" kwargs["default_mtime"] = time.time() with writeable_tarfile(fileobj, mode="w:gz", **kwargs) as tf: tf.create_file_from_string( self.FORMAT_FILENAME, "%s\n" % self.format) tf.create_file_from_string( self.METADATA_FILENAME, str(self.metadata)) for fs_file_name, arc_file_name in self.files: tf.add(fs_file_name, arcname=arc_file_name) tf.create_dir(self.PACKAGES_DIRNAME) for package in self.packages: if package.content is not None: tf.create_file_from_string( self.PACKAGES_DIRNAME + "/" + package.filename, package.content.read()) tf.create_file_from_string( self.MANIFEST_FILENAME, self.manifest_text()) tf.create_file_from_string( self.PACKAGES_FILENAME, get_packages_file( [p for p in self.packages if p.content is not None])) tf.create_dir(self.SOURCES_LIST_DIRNAME) for source_name, source_info in self.sources.items(): url_parsed = urlparse.urlsplit(source_info) # Don't output sources with passwords in them if not url_parsed.password: tf.create_file_from_string( (self.SOURCES_LIST_DIRNAME + "/" + source_name + ".list"), "deb " + source_info + "\n") # TODO: include sources keys etc. tf.create_dir(self.SOURCES_LIST_GPG_DIRNAME) linaro-image-tools-2014.11/linaro_image_tools/hwpack/better_tarfile.py0000664000175000017500000001115212400111666027141 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from contextlib import contextmanager from StringIO import StringIO from tarfile import DIRTYPE, TarFile as StandardTarFile, TarInfo """Improvements to the standard library's tarfile module. In particular this module provides a tarfile.TarFile subclass that aids in adding paths to the tarfile that aren't present on the filesystem, with the ability to specify file content as strings, and provide default values for the mtime, uid, etc. of the created paths. """ @contextmanager def writeable_tarfile(backing_file, mode="w", **kwargs): """A context manager to get a writeable better tarfile. :param backing_file: a file object to write the tarfile contents to. :param mode: the mode to open the tarfile with. Default is "w". :param kwargs: other keyword arguments to pass to the TarFile constructor. """ tf = TarFile.open(mode=mode, fileobj=backing_file, **kwargs) try: yield tf finally: tf.close() class TarFile(StandardTarFile): """An improvement to tarfile that can add paths not on the filesystem. With the standard tarfile implementation adding paths that are not present on the filesystem is convoluted. This subclass adds methods to create paths in the tarfile that are not present on the filesystem. In addition, it can take constructor parameters to set the defaults of various attributes of the paths that it adds. """ def __init__(self, *args, **kwargs): """Create a TarFile. :param default_mtime: the default mtime to create paths with, an int or None to use the stdlib default. :param default_uid: the default user id to set as the owner of created paths, an int or None to use the stdlib default. :param default_gid: the default group id to set as the owner of created paths, an int or None to use the stdlib default. :param default_uname: the default user name to set as the owner of created paths, a string, or None to use the stdlib default. :param default_gname: the default group name ot set as the owner of created paths, a string, or None to use the stdlib default. """ self.default_mtime = kwargs.pop("default_mtime", None) self.default_uid = kwargs.pop("default_uid", None) self.default_gid = kwargs.pop("default_gid", None) self.default_uname = kwargs.pop("default_uname", None) self.default_gname = kwargs.pop("default_gname", None) super(TarFile, self).__init__(*args, **kwargs) def _set_defaults(self, tarinfo): if self.default_mtime is not None: tarinfo.mtime = self.default_mtime if self.default_uid is not None: tarinfo.uid = self.default_uid if self.default_gid is not None: tarinfo.gid = self.default_gid if self.default_uname is not None: tarinfo.uname = self.default_uname if self.default_gname is not None: tarinfo.gname = self.default_gname def create_file_from_string(self, filename, content): """Create a file with the contents passed as a string. :param filename: the path to put the file at inside the tarfile. :param content: the content to put in the created file. """ tarinfo = TarInfo(name=filename) tarinfo.size = len(content) self._set_defaults(tarinfo) fileobj = StringIO(content) self.addfile(tarinfo, fileobj=fileobj) def create_dir(self, path): """Create a directory within the tarfile. :param path: the path to put the directory at. """ tarinfo = TarInfo(name=path) tarinfo.type = DIRTYPE tarinfo.mode = 0755 self._set_defaults(tarinfo) self.addfile(tarinfo) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/0000775000175000017500000000000012435067365024754 5ustar fboudrafboudra00000000000000linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/__init__.py0000664000175000017500000000322512400111666027051 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011, 2012 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import unittest def test_suite(): module_names = [ 'linaro_image_tools.hwpack.tests.test_better_tarfile', 'linaro_image_tools.hwpack.tests.test_builder', 'linaro_image_tools.hwpack.tests.test_config', 'linaro_image_tools.hwpack.tests.test_config_v3', 'linaro_image_tools.hwpack.tests.test_hardwarepack', 'linaro_image_tools.hwpack.tests.test_hwpack_converter', 'linaro_image_tools.hwpack.tests.test_hwpack_reader', 'linaro_image_tools.hwpack.tests.test_packages', 'linaro_image_tools.hwpack.tests.test_script', 'linaro_image_tools.hwpack.tests.test_tarfile_matchers', 'linaro_image_tools.hwpack.tests.test_testing', ] loader = unittest.TestLoader() suite = loader.loadTestsFromNames(module_names) return suite linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_hardwarepack.py0000664000175000017500000006470312400111666031015 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from StringIO import StringIO import re import tarfile from testtools import TestCase from testtools.matchers import Equals, MismatchError from linaro_image_tools.hwpack.hardwarepack import HardwarePack, Metadata from linaro_image_tools.hwpack.packages import get_packages_file from linaro_image_tools.hwpack.testing import ( DummyFetchedPackage, HardwarePackHasFile, MatchesAsPackagesFile, MatchesAsPackageContent, MatchesPackageRelationshipList, MatchesStructure, Not, ) from linaro_image_tools.hwpack.hardwarepack_format import ( HardwarePackFormatV1, HardwarePackFormatV2, HardwarePackFormatV3, ) class MetadataTests(TestCase): def setUp(self): super(MetadataTests, self).setUp() self.metadata = Metadata("ahwpack", "3", "armel") def test_name(self): self.assertEqual("ahwpack", self.metadata.name) def test_version(self): self.assertEqual("3", self.metadata.version) def test_version_with_whitespace(self): self.assertRaises( AssertionError, Metadata, "ahwpack", "3 (with extras)", "armel") def test_architecture(self): self.assertEqual("armel", self.metadata.architecture) def test_default_origin_is_None(self): self.assertEqual(None, self.metadata.origin) def test_origin(self): metadata = Metadata("ahwpack", "4", "armel", origin="linaro") self.assertEqual("linaro", metadata.origin) def test_default_maintainer_is_None(self): metadata = Metadata("ahwpack", "4", "armel") self.assertEqual(None, metadata.maintainer) def test_maintainer(self): metadata = Metadata( "ahwpack", "4", "armel", maintainer="Some maintainer") self.assertEqual("Some maintainer", metadata.maintainer) def test_default_support_is_None(self): metadata = Metadata("ahwpack", "4", "armel") self.assertEqual(None, metadata.support) def test_support(self): metadata = Metadata("ahwpack", "4", "armel", support="supported") self.assertEqual("supported", metadata.support) def test_str(self): metadata = Metadata("ahwpack", "4", "armel") expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_origin(self): metadata = Metadata("ahwpack", "4", "armel", origin="linaro") expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "ORIGIN=linaro\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_maintainer(self): metadata = Metadata( "ahwpack", "4", "armel", maintainer="Some Maintainer") expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "MAINTAINER=Some Maintainer\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_support(self): metadata = Metadata("ahwpack", "4", "armel", support="unsupported") expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "SUPPORT=unsupported\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_serial_tty(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(serial_tty='ttyO2') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "SERIAL_TTY=ttyO2\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_kernel_addr(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(kernel_addr='0x80000000') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "KERNEL_ADDR=0x80000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_initrd_addr(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(initrd_addr='0x80000000') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "INITRD_ADDR=0x80000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_load_addr(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(load_addr='0x80000000') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "LOAD_ADDR=0x80000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_dtb_addr(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(dtb_addr='0x80000000') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "DTB_ADDR=0x80000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_wired_interfaces(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(wired_interfaces=['eth0', 'usb0']) expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "WIRED_INTERFACES=eth0 usb0\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_wireless_interfaces(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(wireless_interfaces=['wlan0', 'wl0']) expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "WIRELESS_INTERFACES=wlan0 wl0\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_partition_layout(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(partition_layout='bootfs_rootfs') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "PARTITION_LAYOUT=bootfs_rootfs\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_mmc_id(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(mmc_id='1') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "MMC_ID=1\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_boot_min_size(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(boot_min_size='50') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "BOOT_MIN_SIZE=50\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_root_min_size(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(root_min_size='100') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "ROOT_MIN_SIZE=100\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_loader_min_size(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(loader_min_size='1') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "LOADER_MIN_SIZE=1\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_kernel_file(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(vmlinuz='boot/vmlinuz-3.0.0-1002-linaro-omap') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "KERNEL_FILE=boot/vmlinuz-3.0.0-1002-linaro-omap\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_initrd_file(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(initrd='boot/initrd.img-3.0.0-1002-linaro-omap') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "INITRD_FILE=boot/initrd.img-3.0.0-1002-linaro-omap\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_dtb_file(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config( dtb_file='boot/dt-3.0.0-1002-linaro-omap/omap4-panda.dtb') expected_out = ("NAME=ahwpack\nVERSION=4\n" "ARCHITECTURE=armel\nDTB_FILE=" "boot/dt-3.0.0-1002-linaro-omap/omap4-panda.dtb\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_boot_script(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(boot_script='boot.scr') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "BOOT_SCRIPT=boot.scr\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_extra_boot_options(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config( extra_boot_options=( 'earlyprintk fixrtc nocompcache vram=48M omapfb.vram=0:24M ' 'mem=456M@0x80000000 mem=512M@0xA0000000')) expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "EXTRA_BOOT_OPTIONS=earlyprintk fixrtc nocompcache " "vram=48M omapfb.vram=0:24M " "mem=456M@0x80000000 mem=512M@0xA0000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_extra_serial_options(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config( extra_serial_options='console=tty0 console=ttyO2,115200n8') expected_out = ("NAME=ahwpack\nVERSION=4\n" "ARCHITECTURE=armel\nEXTRA_SERIAL_OPTIONS=" "console=tty0 console=ttyO2,115200n8\n") self.assertEqual(expected_out, str(metadata)) def test_from_config(self): class Config: name = "foo" origin = "linaro" maintainer = "someone" support = "supported" format = HardwarePackFormatV1() config = Config() metadata = Metadata.from_config(config, "2.0", "i386") self.assertEqual(config.name, metadata.name) self.assertEqual(config.origin, metadata.origin) self.assertEqual(config.maintainer, metadata.maintainer) self.assertEqual(config.support, metadata.support) self.assertEqual("2.0", metadata.version) self.assertEqual("i386", metadata.architecture) class NewMetadataTests(TestCase): def setUp(self): super(NewMetadataTests, self).setUp() def test_format(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) # Need to call also this one! metadata.add_v2_config() metadata.add_v3_config(bootloaders=None) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\n") self.assertEqual(expected_out, str(metadata)) def test_section_bootloaders(self): bootloaders = {'u_boot': {'file': 'a_file'}} metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) # Need to call also this one! metadata.add_v2_config() metadata.add_v3_config(bootloaders=bootloaders) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\nbootloaders:\n u_boot:\n" " file: a_file\n") self.assertEqual(expected_out, str(metadata)) def test_section_wireless(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) wireless_list = ['wlan0', 'wl0'] # Need to call also this one! metadata.add_v2_config(wireless_interfaces=wireless_list) metadata.add_v3_config(bootloaders=None) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\nwireless_interfaces: wlan0 " "wl0\n") self.assertEqual(expected_out, str(metadata)) def test_section_wired(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) wired_list = ['eth0', 'usb0'] # Need to call also this one! metadata.add_v2_config(wired_interfaces=wired_list) metadata.add_v3_config(bootloaders=None) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\nwired_interfaces: eth0 usb0\n") self.assertEqual(expected_out, str(metadata)) def test_section_extra_serial_options(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) options = ['option1', 'option2,option3'] # Need to call also this one! metadata.add_v2_config(extra_serial_options=options) metadata.add_v3_config(bootloaders=None) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\nextra_serial_options: option1 " "option2,option3\n") self.assertEqual(expected_out, str(metadata)) class HardwarePackTests(TestCase): def setUp(self): super(HardwarePackTests, self).setUp() self.metadata = Metadata("ahwpack", "4", "armel") def test_format_is_correct(self): format = '1.0' hwpack = HardwarePack(self.metadata) self.assertEqual(format, hwpack.format.__str__()) def test_format_has_no_spaces(self): hwpack = HardwarePack(self.metadata) self.assertIs(None, re.search('\s', hwpack.format.__str__()), "hwpack.format contains spaces.") def test_filename(self): hwpack = HardwarePack(self.metadata) self.assertEqual("hwpack_ahwpack_4_armel.tar.gz", hwpack.filename()) def test_filename_with_support(self): metadata = Metadata("ahwpack", "4", "armel", support="supported") hwpack = HardwarePack(metadata) self.assertEqual( "hwpack_ahwpack_4_armel_supported.tar.gz", hwpack.filename()) def test_filename_with_extension(self): hwpack = HardwarePack(self.metadata) self.assertEqual( "hwpack_ahwpack_4_armel.txt", hwpack.filename('.txt')) def get_tarfile(self, hwpack): fileobj = StringIO() hwpack.to_file(fileobj) fileobj.seek(0) tf = tarfile.open(mode="r:gz", fileobj=fileobj) self.addCleanup(tf.close) return tf def test_creates_FORMAT_file(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("FORMAT", content=hwpack.format.__str__() + "\n")) def test_creates_metadata_file(self): metadata = Metadata( "ahwpack", "4", "armel", origin="linaro", maintainer="Some Maintainer", support="unsupported") hwpack = HardwarePack(metadata) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("metadata", content=str(metadata))) def test_creates_manifest_file(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("manifest")) def test_manifest_file_empty_with_no_packages(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("manifest", content="")) def test_manifest_contains_package_info(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.2") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("manifest", content="foo=1.1\nbar=1.2\n")) def test_creates_pkgs_dir(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("pkgs", type=tarfile.DIRTYPE)) def test_adds_packages(self): package = DummyFetchedPackage("foo", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package.filename, content=package.content.read())) def test_adds_multiple_packages_at_once(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package1.filename, content=package1.content.read())) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package2.filename, content=package2.content.read())) def test_adds_multiple_in_multiple_steps(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1]) hwpack.add_packages([package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package1.filename, content=package1.content.read())) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package2.filename, content=package2.content.read())) def test_add_packages_without_content_leaves_out_debs(self): package1 = DummyFetchedPackage("foo", "1.1", no_content=True) hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1]) tf = self.get_tarfile(hwpack) self.assertThat( tf, Not(HardwarePackHasFile("pkgs/%s" % package1.filename))) def test_add_dependency_package_adds_package(self): hwpack = HardwarePack(self.metadata) hwpack.add_dependency_package([]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/%s_%s_%s.deb" % ( 'hwpack-' + self.metadata.name, self.metadata.version, self.metadata.architecture), content_matcher=MatchesAsPackageContent( MatchesStructure( name=Equals('hwpack-' + self.metadata.name), architecture=Equals(self.metadata.architecture), depends=Equals(None), version=Equals(self.metadata.version))))) def test_add_dependency_package_adds_package_with_dependency(self): hwpack = HardwarePack(self.metadata) hwpack.add_dependency_package(["foo", "bar (= 1.0)"]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/%s_%s_%s.deb" % ( 'hwpack-' + self.metadata.name, self.metadata.version, self.metadata.architecture), content_matcher=MatchesAsPackageContent( MatchesStructure( name=Equals('hwpack-' + self.metadata.name), architecture=Equals(self.metadata.architecture), depends=MatchesPackageRelationshipList( [Equals("foo"), Equals("bar (= 1.0)")]), version=Equals(self.metadata.version))))) def test_add_dependency_package_adds_package_to_Packages(self): hwpack = HardwarePack(self.metadata) hwpack.add_dependency_package(["foo", "bar (= 1.0)"]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/Packages", content_matcher=MatchesAsPackagesFile( MatchesStructure( name=Equals('hwpack-' + self.metadata.name), architecture=Equals(self.metadata.architecture), depends=MatchesPackageRelationshipList( [Equals("foo"), Equals("bar (= 1.0)")]), version=Equals(self.metadata.version))))) def test_creates_Packages_file(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("pkgs/Packages")) def test_Packages_file_empty_with_no_packages(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("pkgs/Packages", content="")) def test_Packages_file_correct_contents_with_packages(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/Packages", content=get_packages_file([package1, package2]))) def test_Packages_file_empty_with_no_deb_content(self): package1 = DummyFetchedPackage("foo", "1.1", no_content=True) package2 = DummyFetchedPackage("bar", "1.1", no_content=True) hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("pkgs/Packages", content="")) def test_Packages_file_correct_content_with_some_deb_content(self): package1 = DummyFetchedPackage("foo", "1.1", no_content=True) package2 = DummyFetchedPackage("bar", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/Packages", content=get_packages_file([package2]))) def test_creates_sources_list_dir(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d", type=tarfile.DIRTYPE)) def test_adds_sources_list_file(self): hwpack = HardwarePack(self.metadata) source = 'http://example.org/ ubuntu' hwpack.add_apt_sources({'ubuntu': source}) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d/ubuntu.list", content="deb " + source + "\n")) def test_adds_multiple_sources_list_files(self): hwpack = HardwarePack(self.metadata) source1 = 'http://example.org/ ubuntu main universe' source2 = 'http://example.org/ linaro' hwpack.add_apt_sources({'ubuntu': source1, 'linaro': source2}) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d/ubuntu.list", content="deb " + source1 + "\n")) self.assertThat( tf, HardwarePackHasFile("sources.list.d/linaro.list", content="deb " + source2 + "\n")) def test_overwrites_sources_list_file(self): hwpack = HardwarePack(self.metadata) old_source = 'http://example.org/ ubuntu' hwpack.add_apt_sources({'ubuntu': old_source}) new_source = 'http://example.org/ ubuntu main universe' hwpack.add_apt_sources({'ubuntu': new_source}) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d/ubuntu.list", content="deb " + new_source + "\n")) def test_creates_sources_list_gpg_dir(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d.gpg", type=tarfile.DIRTYPE)) def test_password_removed_from_urls(self): hwpack = HardwarePack(self.metadata) url = "https://username:password@hostname/url precise main" hwpack.add_apt_sources({"protected": url}) tf = self.get_tarfile(hwpack) try: self.assertThat( tf, HardwarePackHasFile("sources.list.d/protected.list", content="deb " + url + "\n")) except MismatchError: pass # Expect to not find the password protected URL else: self.assertTrue(False, "Found password protected URL in hwpack") linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_packages.py0000664000175000017500000014732712400111666030143 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import os import re import shutil from StringIO import StringIO import subprocess import tempfile import textwrap import apt_pkg from debian.debfile import DebFile from debian import deb822 from testtools import TestCase from testtools.matchers import Equals from linaro_image_tools.hwpack.packages import ( DependencyNotSatisfied, DummyProgress, FetchedPackage, get_packages_file, IsolatedAptCache, LocalArchiveMaker, PackageFetcher, PackageMaker, stringify_relationship, TemporaryDirectoryManager, ) from linaro_image_tools.hwpack.testing import ( AptSourceFixture, ContextManagerFixture, DummyFetchedPackage, MatchesPackage, ) from linaro_image_tools.testing import TestCaseWithFixtures class GetPackagesFileTests(TestCase): def test_single_stanza(self): package = DummyFetchedPackage("foo", "1.1", architecture="armel") self.assertEqual(textwrap.dedent("""\ Package: foo Version: 1.1 Filename: %(filename)s Size: %(size)d Architecture: armel MD5sum: %(md5)s \n""" % {'filename': package.filename, 'size': package.size, 'md5': package.md5, }), get_packages_file([package])) def test_two_stanzas(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.2") self.assertEqual( get_packages_file([package1]) + get_packages_file([package2]), get_packages_file([package1, package2])) def get_stanza(self, package, relationships=""): stanza = textwrap.dedent("""\ Package: foo Version: 1.1 Filename: %(filename)s Size: %(size)d Architecture: all """ % { 'filename': package.filename, 'size': package.size, }) stanza += relationships stanza += "MD5sum: %s\n\n" % package.md5 return stanza def test_with_depends(self): package = DummyFetchedPackage("foo", "1.1", depends="bar | baz") self.assertEqual( self.get_stanza(package, "Depends: bar | baz\n"), get_packages_file([package])) def test_with_pre_depends(self): package = DummyFetchedPackage("foo", "1.1", pre_depends="bar | baz") self.assertEqual( self.get_stanza(package, "Pre-Depends: bar | baz\n"), get_packages_file([package])) def test_with_conflicts(self): package = DummyFetchedPackage("foo", "1.1", conflicts="bar | baz") self.assertEqual( self.get_stanza(package, "Conflicts: bar | baz\n"), get_packages_file([package])) def test_with_recommends(self): package = DummyFetchedPackage("foo", "1.1", recommends="bar | baz") self.assertEqual( self.get_stanza(package, "Recommends: bar | baz\n"), get_packages_file([package])) def test_with_provides(self): package = DummyFetchedPackage("foo", "1.1", provides="bar") self.assertEqual( self.get_stanza(package, "Provides: bar\n"), get_packages_file([package])) def test_with_replaces(self): package = DummyFetchedPackage("foo", "1.1", replaces="bar (<< 2.0)") self.assertEqual( self.get_stanza(package, "Replaces: bar (<< 2.0)\n"), get_packages_file([package])) def test_with_breaks(self): package = DummyFetchedPackage("foo", "1.1", breaks="bar (<< 2.0)") self.assertEqual( self.get_stanza(package, "Breaks: bar (<< 2.0)\n"), get_packages_file([package])) def test_with_extra_text(self): package = DummyFetchedPackage("foo", "1.1") self.assertEqual(textwrap.dedent("""\ Package: foo Status: bar Version: 1.1 Filename: %(filename)s Size: %(size)d Architecture: all MD5sum: %(md5)s \n""" % {'filename': package.filename, 'size': package.size, 'md5': package.md5, }), get_packages_file([package], extra_text="Status: bar")) class StringifyRelationshipTests(TestCaseWithFixtures): def test_no_relationship(self): target_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( None, stringify_relationship(candidate, "Depends")) def test_single_package(self): target_package = DummyFetchedPackage("foo", "1.0", depends="bar") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "bar", stringify_relationship(candidate, "Depends")) def test_multiple_package(self): target_package = DummyFetchedPackage("foo", "1.0", depends="bar, baz") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "bar, baz", stringify_relationship(candidate, "Depends")) def test_alternative_packages(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="bar | baz") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "bar | baz", stringify_relationship(candidate, "Depends")) def test_package_with_le(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (<= 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (<= 2.0)", stringify_relationship(candidate, "Depends")) def test_package_with_lt(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (<< 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (<< 2.0)", stringify_relationship(candidate, "Depends")) def test_package_with_eq(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (= 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (= 2.0)", stringify_relationship(candidate, "Depends")) def test_package_with_gt(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (>> 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (>> 2.0)", stringify_relationship(candidate, "Depends")) def test_package_with_ge(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (>= 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (>= 2.0)", stringify_relationship(candidate, "Depends")) class TemporaryDirectoryManagerTests(TestCaseWithFixtures): def test_enter_twice_fails(self): maker = TemporaryDirectoryManager() maker.__enter__() self.assertRaises(AssertionError, maker.__enter__) def test_exit_without_enter_silent(self): maker = TemporaryDirectoryManager() maker.__exit__() def test_make_temporary_directory_without_enter_fails(self): maker = TemporaryDirectoryManager() self.assertRaises(AssertionError, maker.make_temporary_directory) def test_make_temporary_directory_makes_directory(self): maker = TemporaryDirectoryManager() self.useFixture(ContextManagerFixture(maker)) tmpdir = maker.make_temporary_directory() self.assertTrue(os.path.isdir(tmpdir)) def test_exit_removes_temporary_directory(self): maker = TemporaryDirectoryManager() self.useFixture(ContextManagerFixture(maker)) tmpdir = maker.make_temporary_directory() maker.__exit__() self.assertFalse(os.path.isdir(tmpdir)) class LocalArchiveMakerTests(TestCaseWithFixtures): def test_sources_entry_for_debs(self): package = DummyFetchedPackage("foo", "1.0") local_archive_maker = LocalArchiveMaker() self.useFixture(ContextManagerFixture(local_archive_maker)) entry = local_archive_maker.sources_entry_for_debs([package]) with IsolatedAptCache([entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, package.filename) self.assertThat(created_package, MatchesPackage(package)) def test_packages_from_sources_entry_for_debs_are_fetchable(self): local_archive_maker = LocalArchiveMaker() package_maker = PackageMaker() self.useFixture(ContextManagerFixture(local_archive_maker)) self.useFixture(ContextManagerFixture(package_maker)) deb_file_path = package_maker.make_package("foo", "1.0", {}) target_package = FetchedPackage.from_deb(deb_file_path) entry = local_archive_maker.sources_entry_for_debs( [target_package]) tmpdir = tempfile.mkdtemp() self.addCleanup(shutil.rmtree, tmpdir) with IsolatedAptCache([entry]) as cache: candidate = cache.cache['foo'].candidate acq = apt_pkg.Acquire(DummyProgress()) base = os.path.basename(candidate.filename) acqfile = apt_pkg.AcquireFile( acq, candidate.uri, candidate.md5, candidate.size, base, destfile=os.path.join(tmpdir, 'deb')) acq.run() self.assertThat(acqfile.status, Equals(acqfile.STAT_DONE)) def test_sources_entry_for_debs_creates_release_with_label(self): package = DummyFetchedPackage("foo", "1.0") local_archive_maker = LocalArchiveMaker() self.useFixture(ContextManagerFixture(local_archive_maker)) label_text = 'random-label' entry = local_archive_maker.sources_entry_for_debs( [package], label=label_text) loc = re.match('file://([^ ]*).*', entry).group(1) release = deb822.Release(open(os.path.join(loc, 'Release'))) self.assertThat(release['Label'], Equals(label_text)) class PackageMakerTests(TestCaseWithFixtures): def test_make_package_creates_file(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}) self.assertTrue(os.path.exists(deb_path)) def test_make_package_creates_deb_with_given_package_name(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}) deb_pkg = DebFile(deb_path) self.assertEqual('foo', deb_pkg.control.debcontrol()['Package']) def test_make_package_creates_deb_with_correct_file_name(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}) proc = subprocess.Popen( ['dpkg-name', deb_path], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) output = proc.communicate()[0] # dpkg-name prints 'dpkg-name: warning: skipping $path' when you give # it a package that is already correctly named. self.assertTrue( 'skipping' in output, "'skipping' was not found in dpkg-name output:\n%s" % output) def test_make_package_creates_deb_with_correct_file_name_arch(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}, 'armel') proc = subprocess.Popen( ['dpkg-name', deb_path], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) output = proc.communicate()[0] self.assertTrue( 'skipping' in output, "'skipping' was not found in dpkg-name output:\n%s" % output) def test_make_package_creates_deb_with_given_version(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0ubuntu1', {}) deb_pkg = DebFile(deb_path) self.assertEqual( '1.0ubuntu1', deb_pkg.control.debcontrol()['Version']) def create_package_and_assert_control_fields_preserved(self, fields): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package( 'foo', '1.0', fields) deb_pkg = DebFile(deb_path) for key, value in fields.items(): self.assertEqual( value, deb_pkg.control.debcontrol()[key]) def test_make_package_creates_deb_with_given_depends(self): self.create_package_and_assert_control_fields_preserved( {'Depends': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_predepends(self): self.create_package_and_assert_control_fields_preserved( {'Pre-Depends': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_conflicts(self): self.create_package_and_assert_control_fields_preserved( {'Conflicts': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_recommends(self): self.create_package_and_assert_control_fields_preserved( {'Recommends': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_provides(self): self.create_package_and_assert_control_fields_preserved( {'Provides': 'bar, baz (= 1.0)'}) def test_make_package_creates_deb_with_given_replaces(self): self.create_package_and_assert_control_fields_preserved( {'Replaces': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_breaks(self): self.create_package_and_assert_control_fields_preserved( {'Breaks': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_all_given_fields(self): self.create_package_and_assert_control_fields_preserved({ 'Depends': 'bar, baz (>= 1.0)', 'Pre-Depends': 'bar, baz (>= 1.0)', 'Conflicts': 'bar, baz (>= 1.0)', 'Recommends': 'bar, baz (>= 1.0)', 'Provides': 'bar, baz (= 1.0)', 'Replaces': 'bar, baz (>= 1.0)', 'Breaks': 'bar, baz (>= 1.0)', }) def test_unknown_field_name_fails(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) self.assertRaises( ValueError, maker.make_package, 'foo', '1.0', {'InvalidField': 'value'}) def test_arch_all_by_default(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}) deb_pkg = DebFile(deb_path) self.assertEqual("all", deb_pkg.control.debcontrol()['Architecture']) def test_custom_architecture(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}, 'armel') deb_pkg = DebFile(deb_path) self.assertEqual('armel', deb_pkg.control.debcontrol()['Architecture']) class FetchedPackageTests(TestCaseWithFixtures): def test_attributes(self): package = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") self.assertEqual("foo", package.name) self.assertEqual("1.1", package.version) self.assertEqual("foo_1.1.deb", package.filename) self.assertEqual(None, package.content) self.assertEqual(4, package.size) self.assertEqual("aaaa", package.md5) self.assertEqual("armel", package.architecture) def test_equal(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") self.assertEqual(package1, package2) self.assertFalse(package1 != package2) def test_not_equal_different_name(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2 = FetchedPackage( "bar", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_version(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2 = FetchedPackage( "foo", "1.2", "foo_1.1.deb", 4, "aaaa", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_filename(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2 = FetchedPackage( "foo", "1.1", "afoo_1.1.deb", 4, "aaaa", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_size(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 5, "aaaa", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_md5(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "bbbb", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_architecture(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "i386") self.assertNotEqual(package1, package2) def test_not_equal_different_depends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", depends="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_depends_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", depends=None) self.assertNotEqual(package1, package2) def test_equal_same_depends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", depends="bar") self.assertEqual(package1, package2) def test_not_equal_different_pre_depends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", pre_depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", pre_depends="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_pre_depends_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", pre_depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", pre_depends=None) self.assertNotEqual(package1, package2) def test_equal_same_pre_depends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", pre_depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", pre_depends="bar") self.assertEqual(package1, package2) def test_not_equal_different_conflicts(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", conflicts="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", conflicts="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_conflicts_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", conflicts="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", conflicts=None) self.assertNotEqual(package1, package2) def test_equal_same_conflicts(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", conflicts="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", conflicts="bar") self.assertEqual(package1, package2) def test_not_equal_different_recommends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", recommends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", recommends="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_recommends_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", recommends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", recommends=None) self.assertNotEqual(package1, package2) def test_equal_same_recommends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", recommends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", recommends="bar") self.assertEqual(package1, package2) def test_not_equal_different_provides(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", provides="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", provides="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_provides_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", provides="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", provides=None) self.assertNotEqual(package1, package2) def test_equal_same_provides(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", provides="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", provides="bar") self.assertEqual(package1, package2) def test_not_equal_different_replaces(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", replaces="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", replaces="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_replaces_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", replaces="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", replaces=None) self.assertNotEqual(package1, package2) def test_equal_same_replaces(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", replaces="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", replaces="bar") self.assertEqual(package1, package2) def test_not_equal_different_breaks(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", breaks="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", breaks="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_breaks_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", breaks="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", breaks=None) self.assertNotEqual(package1, package2) def test_equal_same_breaks(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", breaks="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel", breaks="bar") self.assertEqual(package1, package2) def test_equal_different_contents(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package1.content = StringIO("xxxx") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2.content = StringIO("yyyy") self.assertEqual(package1, package2) def test_equal_one_with_contents_one_not(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package1.content = StringIO("xxxx") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") self.assertEqual(package1, package2) def test_equal_packages_hash_the_same(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "armel") self.assertEqual(hash(package1), hash(package2)) def test_from_apt(self): target_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, target_package.filename, content=target_package.content) self.assertEqual(target_package, created_package) def create_package_and_assert_from_apt_translates_relationship( self, relationship): kwargs = {} kwargs[relationship] = "bar | baz (>= 1.0), zap" target_package = DummyFetchedPackage("foo", "1.0", **kwargs) source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, target_package.filename, content=target_package.content) self.assertEqual(target_package, created_package) def test_from_apt_with_depends(self): self.create_package_and_assert_from_apt_translates_relationship( 'depends') def test_from_apt_with_pre_depends(self): self.create_package_and_assert_from_apt_translates_relationship( 'pre_depends') def test_from_apt_with_conflicts(self): self.create_package_and_assert_from_apt_translates_relationship( 'conflicts') def test_from_apt_with_recommends(self): self.create_package_and_assert_from_apt_translates_relationship( 'recommends') def test_from_apt_with_replaces(self): self.create_package_and_assert_from_apt_translates_relationship( 'replaces') def test_from_apt_with_breaks(self): self.create_package_and_assert_from_apt_translates_relationship( 'breaks') def test_from_apt_with_provides(self): target_package = DummyFetchedPackage("foo", "1.0", provides="bar") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, target_package.filename, content=target_package.content) self.assertEqual(target_package, created_package) def test_from_apt_without_content(self): target_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, target_package.filename) self.assertEqual(None, created_package.content) def test_from_deb(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_file_path = maker.make_package('foo', '1.0', {}) target_package = DummyFetchedPackage( "foo", "1.0", content=open(deb_file_path).read()) created_package = FetchedPackage.from_deb(deb_file_path) self.assertEqual(target_package, created_package) def create_package_and_assert_from_deb_translates_relationships( self, relationships): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_file_path = maker.make_package('foo', '1.0', relationships) dummy_relationships = {} for relationship, value in relationships.items(): dummy_relationships[relationship.lower().replace('-', '_')] = value target_package = DummyFetchedPackage( "foo", "1.0", content=open(deb_file_path).read(), **dummy_relationships) created_package = FetchedPackage.from_deb(deb_file_path) self.assertThat(created_package, MatchesPackage(target_package)) def test_from_deb_with_depends(self): self.create_package_and_assert_from_deb_translates_relationships( {'Depends': 'bar, baz (>= 1.0)'}) def test_from_deb_with_pre_depends(self): self.create_package_and_assert_from_deb_translates_relationships( {'Pre-Depends': 'bar, baz (>= 1.0)'}) def test_from_deb_with_conflicts(self): self.create_package_and_assert_from_deb_translates_relationships( {'Conflicts': 'bar, baz (>= 1.0)'}) def test_from_deb_with_recommends(self): self.create_package_and_assert_from_deb_translates_relationships( {'Recommends': 'bar, baz (>= 1.0)'}) def test_from_deb_with_replaces(self): self.create_package_and_assert_from_deb_translates_relationships( {'Replaces': 'bar, baz (>= 1.0)'}) def test_from_deb_with_breaks(self): self.create_package_and_assert_from_deb_translates_relationships( {'breaks': 'bar, baz (>= 1.0)'}) class AptCacheTests(TestCaseWithFixtures): def test_cleanup_removes_tempdir(self): cache = IsolatedAptCache([]) cache.prepare() tempdir = cache.tempdir cache.cleanup() self.assertFalse(os.path.exists(tempdir)) def test_cleanup_ignores_missing_tempdir(self): cache = IsolatedAptCache([]) cache.prepare() cache.cleanup() # Check that there is no problem removing it again cache.cleanup() def test_cleanup_before_prepare(self): cache = IsolatedAptCache([]) # Check that there is no problem cleaning up before we start cache.cleanup() def test_prepare_creates_tempdir(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertTrue(os.path.isdir(cache.tempdir)) def test_prepare_creates_var_lib_dpkg_status_file(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertEqual( '', open(os.path.join( cache.tempdir, "var", "lib", "dpkg", "status")).read()) def test_prepare_creates_var_cache_apt_archives_partial_dir(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertTrue( os.path.isdir(os.path.join( cache.tempdir, "var", "cache", "apt", "archives", "partial"))) def test_prepare_creates_var_lib_apt_lists_partial_dir(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertTrue( os.path.isdir(os.path.join( cache.tempdir, "var", "lib", "apt", "lists", "partial"))) def test_prepare_creates_etc_apt_sources_list_file(self): source1 = self.useFixture(AptSourceFixture([])) source2 = self.useFixture(AptSourceFixture([])) cache = IsolatedAptCache( [source1.sources_entry, source2.sources_entry]) self.addCleanup(cache.cleanup) cache.prepare() self.assertEqual( "deb %s\ndeb %s\n" % ( source1.sources_entry, source2.sources_entry), open(os.path.join( cache.tempdir, "etc", "apt", "sources.list")).read()) def test_prepare_creates_etc_apt_sources_list_dot_d_dir(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertTrue( os.path.isdir(os.path.join( cache.tempdir, "etc", "apt", "sources.list.d"))) def test_prepare_with_arch_creates_etc_apt_apt_conf(self): cache = IsolatedAptCache([], architecture="arch") self.addCleanup(cache.cleanup) cache.prepare() self.assertEqual( 'Apt {\nArchitecture "arch";\nInstall-Recommends "true";\n}\n', open(os.path.join( cache.tempdir, "etc", "apt", "apt.conf")).read()) def test_prepare_with_prefer_label_creates_etc_apt_preferences(self): label_text = 'random-label' cache = IsolatedAptCache([], prefer_label=label_text) self.addCleanup(cache.cleanup) cache.prepare() self.assertIn( label_text, open(os.path.join( cache.tempdir, "etc", "apt", "preferences")).read()) def test_context_manager(self): # A smoketest that IsolatedAptCache can be used as a context # manager with IsolatedAptCache([]) as cache: tempdir = cache.tempdir self.assertTrue(os.path.isdir(tempdir)) self.assertFalse(os.path.exists(tempdir)) def test_set_installed_packages(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() packages = [DummyFetchedPackage("foo", "1.0")] cache.set_installed_packages(packages) self.assertEqual( get_packages_file( packages, extra_text="Status: install ok installed"), open(os.path.join( cache.tempdir, "var", "lib", "dpkg", "status")).read()) def test_set_installed_packages_empty_list(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() cache.set_installed_packages([]) self.assertEqual( "", open(os.path.join( cache.tempdir, "var", "lib", "dpkg", "status")).read()) def test_package_list_has_no_passwords(self): source1 = self.useFixture(AptSourceFixture([])) path = re.sub("file:/", "file://user:pass@", source1.sources_entry) cache = IsolatedAptCache([path]) self.addCleanup(cache.cleanup) cache.prepare() cache.set_installed_packages([]) sources_list_location = os.path.join(cache.tempdir, "etc", "apt", "sources.list") sources_list = open(sources_list_location).read() self.assertEqual("deb %s\n" % source1.sources_entry, sources_list) class PackageFetcherTests(TestCaseWithFixtures): def test_context_manager(self): # A smoketest that PackageFetcher can be used as a context # manager with PackageFetcher([]) as fetcher: tempdir = fetcher.cache.tempdir self.assertTrue(os.path.isdir(tempdir)) self.assertFalse(os.path.exists(tempdir)) def get_fetcher(self, sources, architecture=None, prefer_label=None): fetcher = PackageFetcher( [s.sources_entry for s in sources], architecture=architecture, prefer_label=prefer_label) self.addCleanup(fetcher.cleanup) fetcher.prepare() return fetcher def test_fetch_packages_not_found_because_no_sources(self): fetcher = self.get_fetcher([]) self.assertRaises(KeyError, fetcher.fetch_packages, ["nothere"]) def test_fetch_packages_not_found_because_not_in_sources(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertRaises(KeyError, fetcher.fetch_packages, ["nothere"]) def test_fetch_packages_not_found_one_of_two_missing(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertRaises( KeyError, fetcher.fetch_packages, ["foo", "nothere"]) def test_fetch_packages_fetches_no_packages(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertEqual(0, len(fetcher.fetch_packages([]))) def test_fetch_packages_fetches_single_package(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertEqual(1, len(fetcher.fetch_packages(["foo"]))) def test_fetch_packages_fetches_correct_package(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertEqual( available_package, fetcher.fetch_packages(["foo"])[0]) def test_fetch_packages_fetches_preferred_label(self): lower_package = DummyFetchedPackage("foo", "1.0") higher_package = DummyFetchedPackage("foo", "2.0") label_text = 'random-label' source1 = self.useFixture(AptSourceFixture([higher_package])) source2 = self.useFixture( AptSourceFixture([lower_package], label=label_text)) fetcher = self.get_fetcher( [source1, source2], prefer_label=label_text) self.assertEqual( lower_package, fetcher.fetch_packages(["foo"])[0]) def test_fetch_packages_fetches_multiple_packages(self): available_packages = [ DummyFetchedPackage("bar", "1.0"), DummyFetchedPackage("foo", "1.0"), ] source = self.useFixture(AptSourceFixture(available_packages)) fetcher = self.get_fetcher([source]) self.assertEqual(2, len(fetcher.fetch_packages(["foo", "bar"]))) def test_fetch_packages_fetches_multiple_packages_correctly(self): available_packages = [ DummyFetchedPackage("foo", "1.0"), DummyFetchedPackage("bar", "1.0"), ] source = self.useFixture(AptSourceFixture(available_packages)) fetcher = self.get_fetcher([source]) fetched = fetcher.fetch_packages(["foo", "bar"]) self.assertEqual(available_packages[0], fetched[0]) self.assertEqual(available_packages[1], fetched[1]) def test_fetch_packages_fetches_newest(self): available_packages = [ DummyFetchedPackage("bar", "1.0"), DummyFetchedPackage("bar", "1.1"), ] source = self.useFixture(AptSourceFixture(available_packages)) fetcher = self.get_fetcher([source]) fetched = fetcher.fetch_packages(["bar"]) self.assertEqual(available_packages[1], fetched[0]) def test_fetch_packages_fetches_newest_from_multiple_sources(self): old_source_packages = [DummyFetchedPackage("bar", "1.0")] new_source_packages = [DummyFetchedPackage("bar", "1.1")] old_source = self.useFixture(AptSourceFixture(old_source_packages)) new_source = self.useFixture(AptSourceFixture(new_source_packages)) fetcher = self.get_fetcher([old_source, new_source]) fetched = fetcher.fetch_packages(["bar"]) self.assertEqual(new_source_packages[0], fetched[0]) def test_fetch_packages_records_correct_architecture(self): available_package = DummyFetchedPackage( "foo", "1.0", architecture="nonexistant") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source], architecture="nonexistant") self.assertEqual( "nonexistant", fetcher.fetch_packages(["foo"])[0].architecture) def test_fetch_packages_fetches_from_correct_architecture(self): wanted_package = DummyFetchedPackage( "foo", "1.0", architecture="arch1") unwanted_package = DummyFetchedPackage( "foo", "1.1", architecture="arch2") source = self.useFixture( AptSourceFixture([wanted_package, unwanted_package])) fetcher = self.get_fetcher([source], architecture="arch1") self.assertEqual( wanted_package, fetcher.fetch_packages(["foo"])[0]) def test_fetch_packages_fetches_with_relationships(self): depends = "foo" pre_depends = "bar (>= 1.0)" conflicts = "baz | zap" recommends = "zing, zang" dependency_packages = [ DummyFetchedPackage("foo", "1.0"), DummyFetchedPackage("bar", "1.0"), DummyFetchedPackage("baz", "1.0"), DummyFetchedPackage("zap", "1.0"), DummyFetchedPackage("zing", "1.0"), DummyFetchedPackage("zang", "1.0"), ] wanted_package = DummyFetchedPackage( "top", "1.0", depends=depends, pre_depends=pre_depends, conflicts=conflicts, recommends=recommends) source = self.useFixture( AptSourceFixture([wanted_package] + dependency_packages)) fetcher = self.get_fetcher([source]) self.assertIn( wanted_package, fetcher.fetch_packages(["top"])) def test_fetch_packages_download_content_False_doesnt_set_content(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) fetched_package = fetcher.fetch_packages( ["foo"], download_content=False)[0] self.assertIs(None, fetched_package.content) def test_fetches_dependencies(self): wanted_package1 = DummyFetchedPackage("foo", "1.0", depends="bar") wanted_package2 = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([wanted_package1, wanted_package2])) fetcher = self.get_fetcher([source]) self.assertEqual( [wanted_package1, wanted_package2], fetcher.fetch_packages(["foo"])) def test_fetches_recommends(self): wanted_package1 = DummyFetchedPackage("foo", "1.0", recommends="bar") wanted_package2 = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([wanted_package1, wanted_package2])) fetcher = self.get_fetcher([source]) self.assertEqual( [wanted_package1, wanted_package2], fetcher.fetch_packages(["foo"])) def test_broken_dependencies(self): wanted_package = DummyFetchedPackage("foo", "1.0", depends="bar") source = self.useFixture(AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) e = self.assertRaises( DependencyNotSatisfied, fetcher.fetch_packages, ["foo"]) self.assertEqual( "Unable to satisfy dependencies of foo", str(e)) def test_fetch_packages_leaves_no_marked_changes(self): wanted_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) fetcher.fetch_packages(["foo"]) self.assertEqual([], list(fetcher.cache.cache.get_changes())) def test_fetch_packages_without_content_leaves_no_marked_changes(self): wanted_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) fetcher.fetch_packages(["foo"], download_content=False) self.assertEqual([], list(fetcher.cache.cache.get_changes())) def test_fetch_packages_can_remove_package(self): """Check that removed packages aren't included in the hwpack When installing the hwpack would cause an "assume-installed" package to be removed the hwpack shouldn't contain that package. """ wanted_package = DummyFetchedPackage( "foo", "1.0", conflicts="provided", provides="provided", depends="zoo", replaces="provided") top_package = DummyFetchedPackage( "top", "1.0", recommends="bar, baz") dep_package = DummyFetchedPackage( "bar", "1.0", depends="baz") conflict_package = DummyFetchedPackage( "baz", "1.0", recommends="bar", provides="provided", conflicts="provided", replaces="provided") extra_package = DummyFetchedPackage("zoo", "1.0") source = self.useFixture(AptSourceFixture( [wanted_package, dep_package, conflict_package, extra_package, top_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["top"]) fetched = fetcher.fetch_packages(["foo"]) self.assertEqual([wanted_package, extra_package], fetched) def test_ignore_with_provides(self): ignored_package = DummyFetchedPackage( "ubuntu-minimal", "1.0", depends="apt-utils") middle_package = DummyFetchedPackage( "apt-utils", "1.0", depends="libapt-pkg") provides_package = DummyFetchedPackage( "apt", "1.0", provides="libapt-pkg", replaces="someotherpackage") source = self.useFixture( AptSourceFixture( [ignored_package, middle_package, provides_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["ubuntu-minimal"]) def test_download_content_False_fetches_no_dependencies(self): wanted_package1 = DummyFetchedPackage("foo", "1.0", depends="bar") wanted_package2 = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([wanted_package1, wanted_package2])) fetcher = self.get_fetcher([source]) self.assertEqual( 1, len(fetcher.fetch_packages(["foo"], download_content=False))) def test_ignore_packages(self): wanted_package = DummyFetchedPackage("foo", "1.0", depends="bar") ignored_package = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([wanted_package, ignored_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["bar"]) self.assertEqual( [wanted_package], fetcher.fetch_packages(["foo"])) def test_ignore_dependency_of_ignored(self): wanted_package = DummyFetchedPackage("foo", "1.0", depends="baz") ignored_package = DummyFetchedPackage("bar", "1.0", depends="baz") dependency = DummyFetchedPackage("baz", "1.0") source = self.useFixture( AptSourceFixture([wanted_package, ignored_package, dependency])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["bar"]) self.assertEqual( [wanted_package], fetcher.fetch_packages(["foo"])) def test_ignore_unknown_package(self): source = self.useFixture(AptSourceFixture([])) fetcher = self.get_fetcher([source]) self.assertRaises(KeyError, fetcher.ignore_packages, ["unknown"]) def test_ignore_cant_satisfy_depenencies(self): wanted_package = DummyFetchedPackage("foo", "1.0", depends="bar") source = self.useFixture(AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) e = self.assertRaises( DependencyNotSatisfied, fetcher.ignore_packages, ["foo"]) self.assertEqual( "Unable to satisfy dependencies of foo", str(e)) def test_ignored_arent_fetched_even_if_explicitly_requested(self): wanted_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture( AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["foo"]) self.assertEqual( [], fetcher.fetch_packages(["foo"])) def test_no_metadata_for_ignored_even_if_explicitly_requested(self): wanted_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture( AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["foo"]) self.assertEqual( [], fetcher.fetch_packages(["foo"], download_content=False)) def test_ignore_is_cumulative(self): package1 = DummyFetchedPackage("foo", "1.0") package2 = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([package1, package2])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["foo"]) fetcher.ignore_packages(["bar"]) self.assertEqual( [], fetcher.fetch_packages(["foo", "bar"], download_content=False)) def test_ignore_leaves_no_marked_changes(self): package1 = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([package1])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["foo"]) self.assertEqual( [], list(fetcher.cache.cache.get_changes())) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_script.py0000664000175000017500000002052612400111666027660 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import subprocess from testtools.matchers import DocTestMatches from linaro_image_tools.hwpack.hardwarepack import Metadata from linaro_image_tools.hwpack.testing import ( AptSourceFixture, ChdirToTempdirFixture, ConfigFileFixture, DummyFetchedPackage, IsHardwarePack, ) from linaro_image_tools.hwpack.config import Config from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.utils import find_command import os from StringIO import StringIO class ScriptTests(TestCaseWithFixtures): """Tests that execute the linaro-hwpack-create script.""" def setUp(self): super(ScriptTests, self).setUp() # Work out root of checkout. # We do this here because when running in PyCharm # the assumption in find_command that os.path.isabs(__file__) is # only true when not running from a Bazaar checkout is broken. # Thankfully find_command allows us to work around this by specifying # prefer_dir. dir = os.path.dirname(__file__) while True: path = os.path.join(dir, "linaro-hwpack-create") if dir == "/" or dir == "": # Didn't find linaro-media-create. Continue as if we haven't # tried to work out prefer_dir. dir = None break if os.path.exists(path) and os.access(path, os.X_OK): break dir = os.path.split(dir)[0] self.script_path = find_command("linaro-hwpack-create", prefer_dir=dir) self.useFixture(ChdirToTempdirFixture()) def run_script(self, args, expected_returncode=0): cmdline = [self.script_path] + args proc = subprocess.Popen( cmdline, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = proc.communicate() self.assertEqual( expected_returncode, proc.returncode, "%s exited with code %d. stdout: %s\nstderr: %s\n" % (str(cmdline), proc.returncode, stdout, stderr)) return stdout, stderr def test_errors_on_missing_configfile(self): stdout, stderr = self.run_script( ["nonexistant", "1.0"], expected_returncode=1) self.assertEqual("No such config file: 'nonexistant'\n", stderr) self.assertEqual("", stdout) def test_errors_on_missing_configfile_argument(self): stdout, stderr = self.run_script([], expected_returncode=2) self.assertEqual( "linaro-hwpack-create: error: too few arguments", stderr.splitlines()[-1]) self.assertEqual("", stdout) def test_errors_on_missing_version_argument(self): stdout, stderr = self.run_script(["somefile"], expected_returncode=2) self.assertEqual( "linaro-hwpack-create: error: too few arguments", stderr.splitlines()[-1]) self.assertEqual("", stdout) def test_builds_a_hwpack(self): package_name = 'foo' available_package = DummyFetchedPackage( package_name, "1.1", architecture="armel") source = self.useFixture(AptSourceFixture([available_package])) config = self.useFixture(ConfigFileFixture( '[hwpack]\nname=ahwpack\npackages=%s\narchitectures=armel\n' '\n[ubuntu]\nsources-entry=%s\n' % ( package_name, source.sources_entry))) stdout, stderr = self.run_script([config.filename, "1.0"]) metadata = Metadata("ahwpack", "1.0", "armel") self.assertThat( "hwpack_ahwpack_1.0_armel.tar.gz", IsHardwarePack( metadata, [available_package], {"ubuntu": source.sources_entry}, package_spec=package_name)) def test_builds_a_v3_hwpack_from_config_with_2_bootloaders(self): config_v3 = ("format: 3.0\n" "name: ahwpack\n" "architectures: armel\n" "serial_tty: ttySAC1\n" "partition_layout:\n" " - bootfs_rootfs\n" "boot_script: boot.scr\n" "extra_serial_options:\n" " - console=tty0\n" " - console=ttyO2,115200n8\n" "mmc_id: 0:1\n" "kernel_file: boot/vmlinuz-*-linaro-omap\n" "initrd_file: boot/initrd.img-*-linaro-omap\n" "dtb_file: boot/dt-*-linaro-omap/omap4-panda.dtb\n" "packages:\n" " - %s\n" " - %s\n") bootloader_config = (" package: %s\n" " in_boot_part: %s\n" " extra_boot_options:\n" " - earlyprintk\n" " - fixrtc\n" " - nocompcache\n" " - vram=48M\n" " - omapfb.vram=0:24M\n" " - mem=456M@0x80000000\n" " - mem=512M@0xA0000000\n") config_v3 += ("bootloaders:\n" " u_boot:\n" + bootloader_config + " u_boot_2:\n" + bootloader_config) config_v3 += ("sources:\n" " ubuntu: %s\n") package_names = ['foo', 'bar'] available_packages = [] for package_name in package_names: available_packages.append( DummyFetchedPackage(package_name, "1.1", architecture="armel")) source = self.useFixture(AptSourceFixture(available_packages)) config_v3 = config_v3 % (package_names[0], package_names[1], package_names[0], "True", package_names[1], "False", source.sources_entry) config_file_fixture = self.useFixture(ConfigFileFixture(config_v3)) self.run_script([config_file_fixture.filename, "1.0"]) # We now need a real config object to test against the configuration # in the hardware pack we have created. config = Config(StringIO(config_v3)) config.bootloader = "u_boot" metadata = Metadata.from_config(config, "1.0", "armel") self.assertThat( "hwpack_ahwpack_1.0_armel.tar.gz", IsHardwarePack( metadata, available_packages, {"ubuntu": source.sources_entry}, package_spec=",".join(package_names), format="3.0")) def test_log_output(self): package_name = 'foo' architecture = 'armel' available_package = DummyFetchedPackage( package_name, "1.1", architecture=architecture) source = self.useFixture(AptSourceFixture([available_package])) config = self.useFixture(ConfigFileFixture( '[hwpack]\nname=ahwpack\npackages=%s\narchitectures=armel\n' '\n[ubuntu]\nsources-entry=%s\n' % ( package_name, source.sources_entry))) stdout, stderr = self.run_script([config.filename, "1.0"]) # XXX Adding in the format deprecation message below is just a hack # until the test can be fixed up to test a new format hwpack. self.assertThat( stderr, DocTestMatches( "Building for %(arch)s\nFetching packages\n" "The format '1.0' is deprecated, please update your hardware " "pack configuration.\n" "Wrote hwpack_ahwpack_1.0_%(arch)s.tar.gz" "\n" % dict(arch=architecture))) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_config_v3.py0000664000175000017500000010623412400111666030232 0ustar fboudrafboudra00000000000000# Copyright (C) 2010 - 2012 Linaro # # Author: James Tunnicliffe # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import re from StringIO import StringIO from testtools import TestCase from linaro_image_tools.hwpack.config import Config, HwpackConfigError from linaro_image_tools.hwpack.hwpack_fields import ( DEFINED_PARTITION_LAYOUTS, SAMSUNG_BL1_LEN_FIELD, SAMSUNG_BL1_START_FIELD, SAMSUNG_BL2_LEN_FIELD, SAMSUNG_BL2_START_FIELD, SAMSUNG_ENV_LEN_FIELD, SAMSUNG_ENV_START_FIELD, ) class ConfigTests(TestCase): valid_start = ( "name: ahwpack\npackages: foo\narchitectures: armel\n") valid_start_v3 = valid_start + "format: 3.0\n" valid_complete_v3 = (valid_start_v3 + "serial_tty: ttySAC1\n" "partition_layout:\n" " - bootfs_rootfs\n" "boot_script: boot.scr\n" "extra_serial_options:\n" " - console=tty0\n" " - console=ttyO2,115200n8\n" "mmc_id: 0:1\n" "kernel_file: boot/vmlinuz-*-linaro-omap\n" "initrd_file: boot/initrd.img-*-linaro-omap\n" "dtb_file: boot/dt-*-linaro-omap/omap4-panda.dtb\n" "bootloaders:\n" " u_boot:\n" " package: u-boot-linaro-s5pv310\n" " file: usr/lib/u-boot/smdkv310/u-boot.bin\n" " spl_package: x-loader-omap4-panda\n" " spl_file: usr/lib/x-loader/omap4430panda/MLO\n" " in_boot_part: True\n" " extra_boot_options:\n" " - earlyprintk\n" " - fixrtc\n" " - nocompcache\n" " - vram=48M\n" " - omapfb.vram=0:24M\n" " - mem=456M@0x80000000\n" " - mem=512M@0xA0000000\n") valid_end = "sources:\n sources-entry: foo bar\n" def test_create(self): config = Config(StringIO()) self.assertTrue(config is not None) def get_config(self, contents): if not re.search("\s*format\s*:", contents): contents = "format: 3.0\n" + contents return Config(StringIO(contents), bootloader="u_boot") def assertConfigError(self, contents, f, *args, **kwargs): e = self.assertRaises(HwpackConfigError, f, *args, **kwargs) self.assertEqual(contents, str(e)) def assertValidationError(self, contents, validate_function): self.assertConfigError(contents, validate_function) def test_validate_empty_name(self): config = self.get_config("name: ") self.assertValidationError("Empty value for name", config._validate_name) def test_validate_invalid_name(self): config = self.get_config("name: ~~\n") self.assertValidationError("Invalid name: ~~", config._validate_name) def test_validate_invalid_include_debs(self): config = self.get_config( "name: ahwpack\n" "include_debs: if you don't mind\n") self.assertValidationError( "Invalid value for include-debs: if you don't mind", config._validate_include_debs) def test_validate_invalid_supported(self): config = self.get_config( "name: ahwpack\nsupport: if you pay us\n") self.assertValidationError( "Invalid value for support: if you pay us", config._validate_support) def test_validate_no_packages(self): config = self.get_config( "name: ahwpack\n\n") self.assertValidationError( "No packages found in the metadata", config._validate_packages) def test_validate_empty_packages(self): config = self.get_config( "name: ahwpack\npackages: \n") self.assertValidationError( "No packages found in the metadata", config._validate_packages) def test_validate_invalid_package_name(self): config = self.get_config( "name: ahwpack\npackages: foo ~~ bar\n") self.assertValidationError( "Invalid value in packages in the metadata: ~~", config._validate_packages) def test_validate_no_architectures(self): config = self.get_config( "name: ahwpack\npackages: foo\n") self.assertValidationError( "No architectures found in the metadata", config._validate_architectures) def test_validate_empty_architectures(self): config = self.get_config( "name: ahwpack\npackages: foo\n" "architectures: \n") self.assertValidationError( "No architectures found in the metadata", config._validate_architectures) def test_validate_invalid_package_name_in_assume_installed(self): config = self.get_config( "name: ahwpack\npackages: foo\n" "architectures: armel\nassume_installed:\n - bar\n - ~~\n") self.assertValidationError( "Invalid value in assume-installed in the metadata: ~~", config._validate_assume_installed) def test_validate_other_section_empty_sources_entry(self): config = self.get_config( self.valid_start + "sources:\n ubuntu: \n") self.assertValidationError( "The sources-entry, ubuntu is missing the URI", config._validate_sources) def test_validate_other_section_only_uri_in_sources_entry(self): config = self.get_config( self.valid_start + "sources:\n ubuntu: foo\n") self.assertValidationError( "The sources-entry, ubuntu is missing the distribution", config._validate_sources) def test_validate_other_section_sources_entry_starting_with_deb(self): config = self.get_config( self.valid_start + "sources:\n ubuntu: deb http://example.org/ foo main\n") self.assertValidationError( "The sources-entry, ubuntu shouldn't start with 'deb'", config._validate_sources) def test_validate_other_section_sources_entry_starting_with_deb_src(self): config = self.get_config( self.valid_start + "sources:\n ubuntu: deb-src http://example.org/ foo main\n") self.assertValidationError( "The sources-entry, ubuntu shouldn't start with 'deb'", config._validate_sources) def test_validate_valid_config(self): config = self.get_config(self.valid_complete_v3) self.assertEqual(None, config.validate()) def test_validate_supported_format(self): config = self.get_config(self.valid_start + "format: 0.9\n") self.assertValidationError( "Format version '0.9' is not supported.", config._validate_format) def test_validate_invalid_u_boot_package_name(self): config = self.get_config(self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " package: ~~\n") self.assertValidationError( "Invalid value in u_boot_package in the metadata: ~~", config._validate_bootloader_package) def test_validate_invalid_bootloader_file(self): config = self.get_config(self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " file: ~~\n") self.assertValidationError("Invalid path: ~~", config._validate_bootloader_file) def test_validate_invalid_kernel_file(self): config = self.get_config(self.valid_start_v3 + "kernel_file: ~~\n") self.assertValidationError("Invalid path: ~~", config._validate_vmlinuz) def test_validate_empty_kernel_file(self): config = self.get_config(self.valid_start_v3 + "kernel_file: \n") self.assertValidationError("No kernel_file found in the metadata", config._validate_vmlinuz) def test_validate_invalid_initrd_file(self): config = self.get_config(self.valid_start_v3 + "initrd_file: ~~\n") self.assertValidationError("Invalid path: ~~", config._validate_initrd) def test_validate_empty_initrd_file(self): config = self.get_config(self.valid_start_v3 + "kernel_file: \n") self.assertValidationError("No initrd_file found in the metadata", config._validate_initrd) def test_validate_invalid_boot_script(self): config = self.get_config(self.valid_start_v3 + "boot_script: ~~") self.assertValidationError("Invalid path: ~~", config._validate_boot_script) def test_validate_invalid_dtb_file(self): config = self.get_config(self.valid_start_v3 + "dtb_file: ~~\n") self.assertValidationError("Invalid path: ~~", config._validate_dtb_file) def test_validate_invalid_spl_package_name(self): config = self.get_config(self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " spl_package: ~~\n") config.board = "panda" self.assertValidationError( "Invalid value in spl_package in the metadata: ~~", config._validate_spl_package) def test_validate_invalid_spl_file(self): config = self.get_config(self.valid_start_v3 + "boards:\n" " panda:\n" " bootloaders:\n" " u_boot:\n" " spl_file: ~~\n") config.board = "panda" self.assertValidationError("Invalid path: ~~", config._validate_spl_file) def test_validate_partition_layout(self): partition_layout = 'apafs_bananfs' config = self.get_config(self.valid_start_v3 + "partition_layout: " + partition_layout) self.assertValidationError( "Undefined partition layout %s. " "Valid partition layouts are %s." % (partition_layout, ", ".join(DEFINED_PARTITION_LAYOUTS)), config._validate_partition_layout) def test_validate_wired_interfaces(self): self.assertTrue("XXX What is an invalid interface name?") def test_validate_wireless_interfaces(self): self.assertTrue("XXX What is an invalid interface name?") def test_validate_bootloader_in_boot_part_bool(self): config = self.get_config( self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " in_boot_part: Nope\n") self.assertValidationError( "Invalid value for bootloader_in_boot_part: Nope", config._validate_bootloader_file_in_boot_part) def test_find_board_specific_variable(self): config = self.get_config( self.valid_start_v3 + "boards:\n" " panda:\n" " bootloaders:\n" " u_boot:\n" " in_boot_part: Yes\n") config.bootloader = "u_boot" config.board = "panda" config._validate_bootloader_file_in_boot_part() self.assertEqual(config.bootloader_file_in_boot_part, "yes") def test_board_specific_overwrites_global(self): config = self.get_config( self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " in_boot_part: No\n" "boards:\n" " panda:\n" " bootloaders:\n" " u_boot:\n" " in_boot_part: Yes\n") config.bootloader = "u_boot" config.board = "panda" config._validate_bootloader_file_in_boot_part() self.assertEqual(config.bootloader_file_in_boot_part, "yes") def test_multiple_bootloaders(self): config = self.get_config( self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " in_boot_part: No\n" " anotherboot:\n" " in_boot_part: Yes\n") config.bootloader = "u_boot" config._validate_bootloader_file_in_boot_part() self.assertEqual(config.bootloader_file_in_boot_part, "no") config.bootloader = "anotherboot" config._validate_bootloader_file_in_boot_part() self.assertEqual(config.bootloader_file_in_boot_part, "yes") def test_validate_serial_tty(self): config = self.get_config(self.valid_start_v3 + "serial_tty: tty\n") self.assertValidationError("Invalid serial tty: tty", config._validate_serial_tty) config = self.get_config(self.valid_start_v3 + "serial_tty: ttxSAC1\n") self.assertValidationError("Invalid serial tty: ttxSAC1", config._validate_serial_tty) def test_validate_mmc_id_wrong(self): # The mmc_id value, if coming from a yaml file, has to be quoted. # Make sure the test does not accept a valid-unquoted value. config = self.get_config(self.valid_complete_v3 + "mmc_id: 1:1\n") self.assertRaises(HwpackConfigError, config._validate_mmc_id) def test_validate_mmc_id(self): config = self.get_config(self.valid_complete_v3 + "mmc_id: x\n") self.assertValidationError("Invalid mmc_id x", config._validate_mmc_id) def test_validate_boot_min_size(self): config = self.get_config(self.valid_complete_v3 + "boot_min_size: x\n") self.assertValidationError("Invalid boot min size x", config._validate_boot_min_size) def test_validate_root_min_size(self): config = self.get_config(self.valid_complete_v3 + "root_min_size: x\n") self.assertValidationError("Invalid root min size x", config._validate_root_min_size) def test_validate_loader_min_size(self): config = self.get_config(self.valid_complete_v3 + "loader_min_size: x\n") self.assertValidationError("Invalid loader min size x", config._validate_loader_min_size) def test_validate_kernel_addr(self): # V3 change: All numerical inputs are good addresses (since YAML # converts them to ingegers and we convert them back to the correct # format). We don't need 8 digit hex values for addresses. config = self.get_config(self.valid_complete_v3 + "kernel_addr: 0x8000000\n") config._validate_kernel_addr() config = self.get_config(self.valid_complete_v3 + "kernel_addr: 0x8000000x\n") self.assertValidationError( "Invalid kernel address: 0x8000000x", config._validate_kernel_addr) config = self.get_config(self.valid_complete_v3 + "kernel_addr: 80000000\n") config._validate_kernel_addr() def test_validate_initrd_addr(self): # V3 change: All numerical inputs are good addresses (since YAML # converts them to ingegers and we convert them back to the correct # format). We don't need 8 digit hex values for addresses. config = self.get_config(self.valid_complete_v3 + "initrd_addr: 0x8000000\n") config._validate_initrd_addr() config = self.get_config(self.valid_complete_v3 + "initrd_addr: 0x8000000x\n") self.assertValidationError( "Invalid initrd address: 0x8000000x", config._validate_initrd_addr) config = self.get_config(self.valid_complete_v3 + "initrd_addr: 80000000\n") config._validate_initrd_addr() def test_validate_load_addr(self): # V3 change: All numerical inputs are good addresses (since YAML # converts them to ingegers and we convert them back to the correct # format). We don't need 8 digit hex values for addresses. config = self.get_config(self.valid_complete_v3 + "load_addr: 0x8000000\n") config._validate_load_addr() config = self.get_config(self.valid_complete_v3 + "load_addr: 0x8000000x\n") self.assertValidationError("Invalid load address: 0x8000000x", config._validate_load_addr) config = self.get_config(self.valid_complete_v3 + "load_addr: 80000000\n") config._validate_load_addr() def test_validate_dtb_addr(self): # V3 change: All numerical inputs are good addresses (since YAML # converts them to ingegers and we convert them back to the correct # format). We don't need 8 digit hex values for addresses. config = self.get_config(self.valid_complete_v3 + "dtb_addr: 0x8000000\n") config._validate_dtb_addr() config = self.get_config(self.valid_complete_v3 + "dtb_addr: 0x8000000x\n") self.assertValidationError("Invalid dtb address: 0x8000000x", config._validate_dtb_addr) config = self.get_config(self.valid_complete_v3 + "dtb_addr: 80000000\n") config._validate_dtb_addr() def test_wired_interfaces(self): config = self.get_config(self.valid_complete_v3 + "wired_interfaces:\n - eth0\n" + self.valid_end) config.validate() self.assertEqual(["eth0"], config.wired_interfaces) config = self.get_config(self.valid_complete_v3 + "wired_interfaces:\n" " - eth0\n" " - eth1\n" " - usb2\n" + self.valid_end) config.validate() self.assertEqual(["eth0", "eth1", "usb2"], config.wired_interfaces) def test_wireless_interfaces(self): config = self.get_config(self.valid_complete_v3 + "wireless_interfaces:\n" " - wlan0\n" + self.valid_end) config.validate() self.assertEqual(["wlan0"], config.wireless_interfaces) config = self.get_config(self.valid_complete_v3 + "wireless_interfaces:\n" " - wlan0\n" " - wl1\n" " - usb2\n" + self.valid_end) config.validate() self.assertEqual(["wlan0", "wl1", "usb2"], config.wireless_interfaces) def test_partition_layout(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("bootfs_rootfs", config.partition_layout) def test_bootloader_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("usr/lib/u-boot/smdkv310/u-boot.bin", config.bootloader_file) def test_u_boot_package(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("u-boot-linaro-s5pv310", config.bootloader_package) def test_spl_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("usr/lib/x-loader/omap4430panda/MLO", config.spl_file) def test_kernel_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("boot/vmlinuz-*-linaro-omap", config.vmlinuz) def test_initrd_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("boot/initrd.img-*-linaro-omap", config.initrd) def test_dtb_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("boot/dt-*-linaro-omap/omap4-panda.dtb", config.dtb_file) def test_extra_boot_options(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual( "earlyprintk fixrtc nocompcache vram=48M " "omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000", config.extra_boot_options) def test_extra_serial_options(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual('console=tty0 console=ttyO2,115200n8', config.extra_serial_options) def test_boot_script(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("boot.scr", config.boot_script) def test_u_boot_in_boot_part(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("yes", config.bootloader_file_in_boot_part) def test_spl_package(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("x-loader-omap4-panda", config.spl_package) def test_serial_tty(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("ttySAC1", config.serial_tty) def test_mmc_id(self): config = self.get_config(self.valid_complete_v3 + "mmc_id: 0:1\n" + self.valid_end) config.validate() self.assertEqual("0:1", config.mmc_id) def test_boot_min_size(self): config = self.get_config(self.valid_complete_v3 + "boot_min_size: 50\n" + self.valid_end) config.validate() self.assertEqual("50", config.boot_min_size) def test_root_min_size(self): config = self.get_config(self.valid_complete_v3 + "root_min_size: 50\n" + self.valid_end) config.validate() self.assertEqual("50", config.root_min_size) def test_loader_min_size(self): config = self.get_config(self.valid_complete_v3 + "loader_min_size: 2\n" + self.valid_end) config.validate() self.assertEqual("2", config.loader_min_size) def test_kernel_addr(self): config = self.get_config(self.valid_complete_v3 + "kernel_addr: 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.kernel_addr) config = self.get_config(self.valid_complete_v3 + "kernel_addr: 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8abcdeff", config.kernel_addr) def test_initrd_addr(self): config = self.get_config(self.valid_complete_v3 + "initrd_addr: 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.initrd_addr) config = self.get_config(self.valid_complete_v3 + "initrd_addr: 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8abcdeff", config.initrd_addr) def test_load_addr(self): config = self.get_config(self.valid_complete_v3 + "load_addr: 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.load_addr) config = self.get_config(self.valid_complete_v3 + "load_addr: 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8abcdeff", config.load_addr) def test_dtb_addr(self): config = self.get_config(self.valid_complete_v3 + "dtb_addr: 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.dtb_addr) config = self.get_config(self.valid_complete_v3 + "dtb_addr: 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8abcdeff", config.dtb_addr) def test_name(self): config = self.get_config( "name: ahwpack\n" "packages: foo\n" "architectures: armel\n") self.assertEqual("ahwpack", config.name) def test_include_debs(self): config = self.get_config(self.valid_start + "include_debs: false\n") self.assertEqual(False, config.include_debs) def test_include_debs_defaults_true(self): config = self.get_config(self.valid_start) self.assertEqual(True, config.include_debs) def test_include_debs_defaults_true_on_empty(self): config = self.get_config(self.valid_start + "include_debs: \n") self.assertEqual(True, config.include_debs) def test_origin(self): config = self.get_config(self.valid_start + "origin: linaro\n") self.assertEqual("linaro", config.origin) def test_origin_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.origin) def test_origin_None_on_empty(self): config = self.get_config(self.valid_start + "origin: \n") self.assertEqual(None, config.origin) def test_maintainer(self): maintainer = "Linaro Developers " config = self.get_config( self.valid_start + "maintainer: %s\n" % maintainer) self.assertEqual(maintainer, config.maintainer) def test_maintainer_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.maintainer) def test_maintainer_None_on_empty(self): config = self.get_config(self.valid_start + "maintainer: \n") self.assertEqual(None, config.maintainer) def test_support_supported(self): config = self.get_config(self.valid_start + "support: supported\n") self.assertEqual("supported", config.support) def test_support_unsupported(self): config = self.get_config(self.valid_start + "support: unsupported\n") self.assertEqual("unsupported", config.support) def test_support_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.support) def test_support_None_on_empty(self): config = self.get_config(self.valid_start + "support: \n") self.assertEqual(None, config.support) def test_packages(self): config = self.get_config( "name: ahwpack\n" "packages:\n" " - foo\n" " - bar\n" "architectures: armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_packages_filters_duplicates(self): config = self.get_config( "name: ahwpack\n" "packages:\n" " - foo\n" " - bar\n" " - foo\n" "architectures: armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_sources_single(self): config = self.get_config( self.valid_start + "sources:\n" " ubuntu: http://example.org foo\n") self.assertEqual({"ubuntu": "http://example.org foo"}, config.sources) def test_sources_multiple(self): config = self.get_config( self.valid_start + "sources:\n" " ubuntu: http://example.org foo\n" " linaro: http://example.org bar\n") self.assertEqual( {"ubuntu": "http://example.org foo", "linaro": "http://example.org bar"}, config.sources) def test_architectures(self): config = self.get_config( "name: ahwpack\n" "packages: foo\n" "architectures:\n" " - foo\n" " - bar\n") self.assertEqual(["foo", "bar"], config.architectures) def test_architectures_filters_duplicates(self): config = self.get_config( "name: ahwpack\n" "packages: foo\n" "architectures:\n" " - foo\n" " - bar\n" " - foo\n") self.assertEqual(["foo", "bar"], config.architectures) def test_assume_installed(self): config = self.get_config( "name: ahwpack\n" "packages:\n" " - foo\n" "architectures:\n" " - armel\n" "assume_installed:\n" " - foo\n" " - bar\n") self.assertEqual(["foo", "bar"], config.assume_installed) def test_assume_installed_filters_duplicates(self): config = self.get_config( "name: ahwpack\n" "packages:\n" " - foo\n" "architectures:\n" " - armel\n" "assume_installed:\n" " - foo\n" " - bar\n" " - foo\n") self.assertEqual(["foo", "bar"], config.assume_installed) def test_invalid_key_in_root(self): config = self.get_config("foo: bar") self.assertValidationError("Unknown key in metadata: 'foo'", config._validate_keys) def test_invalid_key_value_root(self): config = self.get_config("bootloaders: bar") self.assertValidationError("Invalid structure in metadata. Expected " "key: value pairs, found: 'bootloaders: " "bar'", config._validate_keys) def test_invalid_key_value_bootloaders(self): config = self.get_config("\n".join([ "bootloaders:", " u_boot:", " foo: bar" ])) self.assertValidationError("Unknown key in metadata: 'bootloaders: " "u_boot: foo'", config._validate_keys) def test_invalid_key_in_board(self): config = self.get_config("\n".join([ "boards:", " pandaboard:", " foo: bar" ])) self.assertValidationError("Unknown key in metadata: " "'boards: pandaboard: foo'", config._validate_keys) def test_invalid_key_in_board_2(self): config = self.get_config("\n".join([ "boards:", " pandaboard:", " name: bar", " snowball:", " foo: bar", ])) self.assertValidationError("Unknown key in metadata: " "'boards: snowball: foo'", config._validate_keys) def test_valid_samsung_bl1_len_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_BL1_LEN_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_bl1_start_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_BL1_START_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_bl2_len_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_BL2_LEN_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_bl2_start_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_BL2_START_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_env_len_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_ENV_LEN_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_env_start_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_ENV_START_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_samsung_field_wrong(self): config = self.get_config(self.valid_start_v3 + 'samsung_wrong_field: 1\n') self.assertRaises(HwpackConfigError, config._validate_keys) # Tests for dtb_files support def test_dtb_files(self): dtb_files = ('dtb_files:\n' + ' - adest.dtb : boot/dt-*-linaro-omap/omap4-panda.dtb\n' + ' - bdest.dtb : ' + 'boot/dt-*-linaro-omap2/omap4-panda2.dtb\n') expected = [{'adest.dtb': 'boot/dt-*-linaro-omap/omap4-panda.dtb'}, {'bdest.dtb': 'boot/dt-*-linaro-omap2/omap4-panda2.dtb'}] config = self.get_config(self.valid_complete_v3 + dtb_files) config.validate() self.assertEqual(expected, config.dtb_files) def test_dtb_files_one_wrong(self): dtb_files = ('dtb_files:\n' + ' - adest.dtb : boot/dt-*-linaro-omap/omap4-panda.dtb\n' + ' - bdest.dtb : ~~~\n') config = self.get_config(self.valid_start_v3 + dtb_files) self.assertRaises(HwpackConfigError, config._validate_dtb_files) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_hwpack_converter.py0000664000175000017500000002006312400111666031714 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011, 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import tempfile from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( CreateTempDirFixture, CreateTempFileFixture, ) from linaro_image_tools.hwpack.hwpack_convert import ( HwpackConverter, HwpackConverterException, check_and_validate_args, ) class Args(): """Defines the args for the command line options.""" def __init__(self, input_file, output_file=None): self.CONFIG_FILE = input_file self.out = output_file class HwpackConverterTests(TestCaseWithFixtures): """Test class for the hwpack converter.""" def setUp(self): super(HwpackConverterTests, self).setUp() def test_wrong_input_file(self): """Pass a non-existing file.""" input_file = '/tmp/foobaz' self.assertRaises( HwpackConverterException, check_and_validate_args, Args(input_file=input_file)) def test_wrong_input_dir(self): """Pass a directory instead of file.""" temp_file = tempfile.NamedTemporaryFile() temp_dir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( HwpackConverterException, check_and_validate_args, Args(input_file=temp_file.name, output_file=temp_dir)) def test_same_input_output_file(self): """Pass the same existing file path to the two arguments.""" temp_file = self.useFixture(CreateTempFileFixture()).get_file_name() self.assertRaises( HwpackConverterException, check_and_validate_args, Args(input_file=temp_file, output_file=temp_file)) def test_basic_parse(self): ini_format = '[hwpack]\nformat=2.0\nsupport=supported' output_format = "format: '3.0'\nsupport: supported\n" input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(output_format, str(converter)) def test_architectures_section_creation(self): """Tests that we create the correct architectures list in the converted file. """ ini_format = '[hwpack]\nformat=2.0\narchitectures=armhf armel' output_format = "format: '3.0'\narchitectures:\n- armhf\n- armel\n" input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(output_format, str(converter)) def test_bootloaders(self): """Tests the correct creation of the bootloaders part.""" ini_format = ("[hwpack]\nformat=2.0\nu_boot_package=a_package\n" "u_boot_file=a_file\nu_boot_in_boot_part=Yes\n" "u_boot_dd=33") out_format = ("format: '3.0'\nbootloaders:\n u_boot:\n dd: '33'" "\n file: a_file\n in_boot_part: true\n" " package: a_package\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_extra_boot_options(self): """Tests the correct creation of the extra_boot_options part.""" ini_format = ("[hwpack]\nformat=2.0\nu_boot_package=a_package\n" "extra_boot_options=opt1 opt2") out_format = ("format: '3.0'\nbootloaders:\n u_boot:\n " " extra_boot_options:\n - opt1\n " "- opt2\n package: a_package\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_extra_serial_options(self): """Tests the correct creation of the extra_serial_options part.""" ini_format = ("[hwpack]\nformat=2.0\nextra_serial_options=opt1 opt2") out_format = ("format: '3.0'\nextra_serial_options:\n- opt1\n- opt2\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_assume_installed(self): """Tests the correct creation of the extra_serial_options part.""" ini_format = ("[hwpack]\nformat=2.0\nassume-installed=install1 " "install2") out_format = ("format: '3.0'\nassume_installed:\n- install1\n- " "install2\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_include_debs(self): """Tests the correct creation of the extra_serial_options part.""" ini_format = ("[hwpack]\nformat=2.0\ninclude-debs=yes") out_format = ("format: '3.0'\ninclude_debs: true\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_dtb_file(self): """Test the dtb_file conversion.""" ini_format = ("[hwpack]\nformat=2.0\ndtb_file=boot/a-*-path/file.dtb") out_format = ("format: '3.0'\ndtb_files:\n- board.dtb: " "boot/a-*-path/file.dtb\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_mmc_id(self): """Test correct handling of mmc_id field. The mmc_id field has to be quoted coming out from the converter otherwise when reading the yaml file the value is read as a number, not a string.""" ini_format = ("[hwpack]\nformat=2.0\nmmc_id=1:1") out_format = ("format: '3.0'\nmmc_id: '1:1'\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_hwpack_reader.py0000664000175000017500000001050012400111666031142 0ustar fboudrafboudra00000000000000# Copyright (C) 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import tarfile from StringIO import StringIO from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( CreateTempDirFixture, ) from linaro_image_tools.media_create.tests.fixtures import ( CreateTarballFixture, ) from linaro_image_tools.hwpack.hwpack_reader import ( Hwpack, HwpackReader, HwpackReaderError, ) class HwpackReaderTests(TestCaseWithFixtures): """Test class for the hwpack reader.""" def setUp(self): super(HwpackReaderTests, self).setUp() self.metadata = ("format: 3.0\nversion: '1'\nname: test-hwpack\n" "architecture: armel\norigin: Linaro") self.hwpack = Hwpack() self.hwpack.setname('test-hwpack') self.tar_dir_fixture = CreateTempDirFixture() self.useFixture(self.tar_dir_fixture) self.tarball_fixture = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir()) self.useFixture(self.tarball_fixture) def tearDown(self): super(HwpackReaderTests, self).tearDown() self.hwpack = None self.metadata = "" def add_to_tarball(self, files, tarball=None): if tarball is None: tarball = self.tarball_fixture.get_tarball() tar_file = tarfile.open(tarball, mode='w:gz') for filename, data in files: tarinfo = tarfile.TarInfo(filename) tarinfo.size = len(data) tar_file.addfile(tarinfo, StringIO(data)) tar_file.close() return tarball def test_hwpack_class(self): hwpack = Hwpack() hwpack.setname('test-hwpack') hwpack.sethwpack('a_hwpack') self.hwpack.sethwpack('a_hwpack') self.assertEqual(self.hwpack, hwpack) def test_hwpack_class_not_equal(self): hwpack = Hwpack() hwpack.setname('test-hwpack') hwpack.sethwpack('a_hwpack') self.hwpack.sethwpack('b_hwpack') self.assertNotEqual(self.hwpack, hwpack) def test_hwpack_metadata_read(self): tarball = self.add_to_tarball([('metadata', self.metadata)]) reader = HwpackReader([tarball]) reader._read_hwpacks_metadata() self.hwpack.sethwpack(tarball) self.assertEqual(self.hwpack, reader.supported_elements[0]) def test_raise_exception(self): new_metadata = ("format=2.0\nversion=4") tarball = self.add_to_tarball([('metadata', new_metadata)]) reader = HwpackReader([tarball]) self.assertRaises(HwpackReaderError, reader._read_hwpacks_metadata) def test_hwpack_metadata_read_with_boards(self): self.metadata += "\nboards:\n panda:\n support: supported\n" tarball = self.add_to_tarball([('metadata', self.metadata)]) reader = HwpackReader([tarball]) reader._read_hwpacks_metadata() self.hwpack.sethwpack(tarball) self.hwpack.setboards({'panda': {'support': 'supported'}}) self.assertEqual(self.hwpack, reader.supported_elements[0]) def test_hwpack_metadata_read_with_bootloaders(self): self.metadata += ("\nboards:\n panda:\n support: supported\n " "bootloaders:\n u_boot:\n file: a_file\n") tarball = self.add_to_tarball([('metadata', self.metadata)]) reader = HwpackReader([tarball]) reader._read_hwpacks_metadata() self.hwpack.sethwpack(tarball) self.hwpack.setboards({'panda': {'support': 'supported', 'bootloaders': {'u_boot': {'file': 'a_file'}}}}) self.assertEqual(self.hwpack, reader.supported_elements[0]) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_tarfile_matchers.py0000664000175000017500000002245012400111666031666 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import tarfile from testtools import TestCase from linaro_image_tools.hwpack.tarfile_matchers import ( TarfileHasFile, TarfileMissingPathMismatch, TarfileWrongValueMismatch, ) from linaro_image_tools.hwpack.testing import test_tarfile class TarfileMissingPathMismatchTests(TestCase): def test_describe(self): mismatch = TarfileMissingPathMismatch("foo", "bar") self.assertEqual('"foo" has no path "bar"', mismatch.describe()) def test_eq(self): mismatch1 = TarfileMissingPathMismatch("foo", "bar") mismatch2 = TarfileMissingPathMismatch("foo", "bar") self.assertEqual(mismatch1, mismatch2) self.assertFalse(mismatch1 != mismatch2) def test_no_eq_different_tarball(self): mismatch1 = TarfileMissingPathMismatch("foo", "bar") mismatch2 = TarfileMissingPathMismatch("baz", "bar") self.assertNotEqual(mismatch1, mismatch2) def test_no_eq_different_path(self): mismatch1 = TarfileMissingPathMismatch("foo", "bar") mismatch2 = TarfileMissingPathMismatch("foo", "baz") self.assertNotEqual(mismatch1, mismatch2) class TarfileWrongTypeMismatchTests(TestCase): def test_describe(self): mismatch = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) self.assertEqual( 'The path "bar" in "foo" has type 2, expected 1', mismatch.describe()) def test_eq(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) self.assertEqual(mismatch1, mismatch2) self.assertFalse(mismatch1 != mismatch2) def test_not_eq_different_attribute(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("size", "foo", "bar", 1, 2) self.assertNotEqual(mismatch1, mismatch2) def test_not_eq_different_tarball(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "baz", "bar", 1, 2) self.assertNotEqual(mismatch1, mismatch2) def test_not_eq_different_path(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "foo", "baz", 1, 2) self.assertNotEqual(mismatch1, mismatch2) def test_not_eq_different_expected(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "foo", "bar", 3, 2) self.assertNotEqual(mismatch1, mismatch2) def test_not_eq_different_actual(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 3) self.assertNotEqual(mismatch1, mismatch2) class TarfileHasFileTests(TestCase): def test_str(self): matcher = TarfileHasFile("foo") self.assertEqual('tarfile has file "foo"', str(matcher)) def test_matches(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo") self.assertIs(None, matcher.match(tf)) def test_matches_directory(self): with test_tarfile(contents=[("foo/", "")]) as tf: matcher = TarfileHasFile("foo", type=tarfile.DIRTYPE) self.assertIs(None, matcher.match(tf)) def test_matches_directory_content(self): with test_tarfile(contents=[("foo/", ""), ("foo/bar", "")]) as tf: matcher = TarfileHasFile( "foo", type=tarfile.DIRTYPE, content=["bar"]) self.assertIs(None, matcher.match(tf)) def test_mismatches_missing_path(self): with test_tarfile() as tf: matcher = TarfileHasFile("foo") mismatch = matcher.match(tf) self.assertIsInstance(mismatch, TarfileMissingPathMismatch) self.assertEqual(TarfileMissingPathMismatch(tf, "foo"), mismatch) def assertValueMismatch(self, mismatch, tarball, path, attribute, expected, actual): self.assertIsInstance(mismatch, TarfileWrongValueMismatch) expected_mismatch = TarfileWrongValueMismatch( attribute, tarball, path, expected, actual) self.assertEqual(expected_mismatch, mismatch) def test_mismatches_wrong_type(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo", type=tarfile.DIRTYPE) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "type", tarfile.DIRTYPE, tarfile.REGTYPE) def test_mismatches_wrong_size(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo", size=1235) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "size", 1235, 0) def test_mismatches_wrong_mtime(self): with test_tarfile(contents=[("foo", "")], default_mtime=12345) as tf: matcher = TarfileHasFile("foo", mtime=54321) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "mtime", 54321, 12345) def test_mismatches_wrong_mode(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo", mode=0000) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "mode", 0000, 0644) def test_mismatches_wrong_linkname(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo", linkname="somelink") mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "linkname", "somelink", "") def test_mismatches_wrong_uid(self): with test_tarfile(contents=[("foo", "")], default_uid=100) as tf: matcher = TarfileHasFile("foo", uid=99) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "uid", 99, 100) def test_mismatches_wrong_gid(self): with test_tarfile(contents=[("foo", "")], default_gid=100) as tf: matcher = TarfileHasFile("foo", gid=99) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "gid", 99, 100) def test_mismatches_wrong_uname(self): with test_tarfile( contents=[("foo", "")], default_uname="someuser") as tf: matcher = TarfileHasFile("foo", uname="otheruser") mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "uname", "otheruser", "someuser") def test_mismatches_wrong_gname(self): with test_tarfile( contents=[("foo", "")], default_gname="somegroup") as tf: matcher = TarfileHasFile("foo", gname="othergroup") mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "gname", "othergroup", "somegroup") def test_mismatches_wrong_content(self): with test_tarfile(contents=[("foo", "somecontent")]) as tf: matcher = TarfileHasFile("foo", content="othercontent") mismatch = matcher.match(tf) self.assertEquals( "'othercontent' != 'somecontent': The content of " "path \"foo\" did not match", mismatch.describe()) def test_mismatches_wrong_directory_content(self): with test_tarfile(contents=[("foo/", ""), ("foo/bar", "")]) as tf: matcher = TarfileHasFile( "foo", type=tarfile.DIRTYPE, content=["baz"]) mismatch = matcher.match(tf) self.assertEquals( "['baz'] != ['bar']: The content of " "path \"foo\" did not match", mismatch.describe()) def test_matches_mtime_with_skew(self): with test_tarfile(contents=[("foo", "")], default_mtime=12345) as tf: matcher = TarfileHasFile("foo", mtime=12346, mtime_skew=1) mismatch = matcher.match(tf) self.assertIs(None, mismatch) def test_matches_mtime_without_skew(self): with test_tarfile(contents=[("foo", "")], default_mtime=12345) as tf: matcher = TarfileHasFile("foo", mtime=12345) mismatch = matcher.match(tf) self.assertIs(None, mismatch) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_testing.py0000664000175000017500000002205112400111666030024 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import doctest import re from StringIO import StringIO import sys from testtools import TestCase from testtools.matchers import ( Annotate, Equals, Mismatch, NotEquals, ) from linaro_image_tools.hwpack.testing import ( DummyFetchedPackage, EachOf, MatchesAsPackagesFile, MatchesPackage, MatchesPackageRelationshipList, MatchesStructure, MatchesSetwise, parse_packages_file_content, ) from linaro_image_tools.hwpack.packages import ( get_packages_file, ) def run_doctest(obj, name): p = doctest.DocTestParser() t = p.get_doctest( obj.__doc__, sys.modules[obj.__module__].__dict__, name, '', 0) r = doctest.DocTestRunner() output = StringIO() r.run(t, out=output.write) return r.failures, output.getvalue() class TestEachOf(TestCase): def test_docstring(self): failure_count, output = run_doctest(EachOf, "EachOf") if failure_count: self.fail("Doctest failed with %s" % output) class TestMatchesPackageRelationshipList(TestCase): def test_docstring(self): failure_count, output = run_doctest( MatchesPackageRelationshipList, "MatchesPackageRelationshipList") if failure_count: self.fail("Doctest failed with %s" % output) class TestMatchesStructure(TestCase): class SimpleClass: def __init__(self, x): self.x = x def test_matches(self): self.assertThat( self.SimpleClass(1), MatchesStructure(x=Equals(1))) def test_mismatch(self): self.assertRaises( AssertionError, self.assertThat, self.SimpleClass(1), MatchesStructure(x=NotEquals(1))) def test_fromExample(self): self.assertThat( self.SimpleClass(1), MatchesStructure.fromExample(self.SimpleClass(1), 'x')) def test_update(self): self.assertThat( self.SimpleClass(1), MatchesStructure(x=NotEquals(1)).update(x=Equals(1))) def test_update_none(self): self.assertThat( self.SimpleClass(1), MatchesStructure(x=Equals(1), y=NotEquals(42)).update( y=None)) class TestMatchesPackage(TestCase): def test_simple(self): observed = DummyFetchedPackage("foo", "1.1", architecture="armel") expected = DummyFetchedPackage("foo", "1.1", architecture="armel") self.assertThat( observed, MatchesPackage(expected)) def test_mismatch(self): observed = DummyFetchedPackage("foo", "1.1", depends="bar") expected = DummyFetchedPackage("foo", "1.1", depends="baz") self.assertRaises(AssertionError, self.assertThat, observed, MatchesPackage(expected)) def test_skip_one_attribute(self): observed = DummyFetchedPackage("foo", "1.1", depends="bar") expected = DummyFetchedPackage("foo", "1.1", depends="baz") self.assertThat( observed, MatchesPackage(expected).update(depends=None)) class MatchesRegex(object): def __init__(self, pattern, flags=0): self.pattern = pattern self.flags = flags def match(self, value): if not re.match(self.pattern, value, self.flags): return Mismatch("%r did not match %r" % (self.pattern, value)) class TestMatchesSetwise(TestCase): def assertMismatchWithDescriptionMatching(self, value, matcher, description_matcher): mismatch = matcher.match(value) if mismatch is None: self.fail("%s matched %s" % (matcher, value)) actual_description = mismatch.describe() self.assertThat( actual_description, Annotate( "%s matching %s" % (matcher, value), description_matcher)) def test_matches(self): self.assertIs( None, MatchesSetwise(Equals(1), Equals(2)).match([2, 1])) def test_mismatches(self): self.assertMismatchWithDescriptionMatching( [2, 3], MatchesSetwise(Equals(1), Equals(2)), MatchesRegex('.*There was 1 mismatch$', re.S)) def test_too_many_matchers(self): self.assertMismatchWithDescriptionMatching( [2, 3], MatchesSetwise(Equals(1), Equals(2), Equals(3)), Equals('There was 1 matcher left over: Equals(1)')) def test_too_many_values(self): self.assertMismatchWithDescriptionMatching( [1, 2, 3], MatchesSetwise(Equals(1), Equals(2)), Equals('There was 1 value left over: [3]')) def test_two_too_many_matchers(self): self.assertMismatchWithDescriptionMatching( [3], MatchesSetwise(Equals(1), Equals(2), Equals(3)), MatchesRegex( 'There were 2 matchers left over: Equals\([12]\), ' 'Equals\([12]\)')) def test_two_too_many_values(self): self.assertMismatchWithDescriptionMatching( [1, 2, 3, 4], MatchesSetwise(Equals(1), Equals(2)), MatchesRegex( 'There were 2 values left over: \[[34], [34]\]')) def test_mismatch_and_too_many_matchers(self): self.assertMismatchWithDescriptionMatching( [2, 3], MatchesSetwise(Equals(0), Equals(1), Equals(2)), MatchesRegex( '.*There was 1 mismatch and 1 extra matcher: Equals\([01]\)', re.S)) def test_mismatch_and_too_many_values(self): self.assertMismatchWithDescriptionMatching( [2, 3, 4], MatchesSetwise(Equals(1), Equals(2)), MatchesRegex( '.*There was 1 mismatch and 1 extra value: \[[34]\]', re.S)) def test_mismatch_and_two_too_many_matchers(self): self.assertMismatchWithDescriptionMatching( [3, 4], MatchesSetwise( Equals(0), Equals(1), Equals(2), Equals(3)), MatchesRegex( '.*There was 1 mismatch and 2 extra matchers: ' 'Equals\([012]\), Equals\([012]\)', re.S)) def test_mismatch_and_two_too_many_values(self): self.assertMismatchWithDescriptionMatching( [2, 3, 4, 5], MatchesSetwise(Equals(1), Equals(2)), MatchesRegex( '.*There was 1 mismatch and 2 extra values: \[[145], [145]\]', re.S)) class TestParsePackagesFileContent(TestCase): def test_one(self): observed = DummyFetchedPackage("foo", "1.1") packages_content = get_packages_file([observed]) parsed = parse_packages_file_content(packages_content) self.assertThat(len(parsed), Equals(1)) self.assertThat(parsed[0], MatchesPackage(observed)) def test_several(self): observed1 = DummyFetchedPackage("foo", "1.1") observed2 = DummyFetchedPackage("bar", "1.2") observed3 = DummyFetchedPackage("baz", "1.5") packages_content = get_packages_file( [observed1, observed2, observed3]) parsed = parse_packages_file_content(packages_content) self.assertThat(parsed, MatchesSetwise( MatchesPackage(observed3), MatchesPackage(observed2), MatchesPackage(observed1))) class TestMatchesAsPackagesFile(TestCase): def test_one(self): observed = DummyFetchedPackage("foo", "1.1") packages_content = get_packages_file([observed]) self.assertThat( packages_content, MatchesAsPackagesFile( MatchesPackage(observed))) def test_ignore_one_md5(self): # This is what I actually care about: being able to specify that a # packages file matches a set of packages, ignoring just a few # details on just one package. observed1 = DummyFetchedPackage("foo", "1.1") observed2 = DummyFetchedPackage("bar", "1.2") observed3 = DummyFetchedPackage("baz", "1.5") packages_content = get_packages_file( [observed1, observed2, observed3]) oldmd5 = observed3.md5 observed3._content = ''.join(reversed(observed3._content_str())) self.assertNotEqual(oldmd5, observed3.md5) self.assertThat(packages_content, MatchesAsPackagesFile( MatchesPackage(observed1), MatchesPackage(observed2), MatchesPackage(observed3).update(md5=None))) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_config.py0000664000175000017500000007624612400111666027633 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from StringIO import StringIO from testtools import TestCase from linaro_image_tools.hwpack.config import Config, HwpackConfigError from linaro_image_tools.hwpack.hwpack_fields import ( DEFINED_PARTITION_LAYOUTS, ) class ConfigTests(TestCase): valid_start = ( "[hwpack]\nname = ahwpack\npackages = foo\narchitectures = armel\n") valid_start_v2 = valid_start + "format = 2.0\n" valid_complete_v2 = (valid_start_v2 + "u_boot_package = u-boot-linaro-s5pv310\n" "u_boot_file = usr/lib/u-boot/smdkv310/" "u-boot.bin\nserial_tty=ttySAC1\n" "partition_layout = bootfs_rootfs\n" "spl_package = x-loader-omap4-panda\n" "spl_file = usr/lib/x-loader/omap4430panda/MLO\n" "kernel_file = boot/vmlinuz-*-linaro-omap\n" "initrd_file = boot/initrd.img-*-linaro-omap\n" "dtb_file = boot/dt-*-linaro-omap/omap4-panda.dtb\n" "boot_script = boot.scr\n" + ("extra_serial_options = console=tty0 " "console=ttyO2,115200n8\n") + ("extra_boot_options = earlyprintk fixrtc " "nocompcache vram=48M omapfb.vram=0:24M " "mem=456M@0x80000000 mem=512M@0xA0000000\n") + "boot_script = boot.scr\n" "mmc_id = 0:1\n" "u_boot_in_boot_part = Yes\n") valid_end = "[ubuntu]\nsources-entry = foo bar\n" def test_create(self): config = Config(StringIO()) self.assertTrue(config is not None) def get_config(self, contents): return Config(StringIO(contents)) def assertConfigError(self, contents, f, *args, **kwargs): e = self.assertRaises(HwpackConfigError, f, *args, **kwargs) self.assertEqual(contents, str(e)) def assertValidationError(self, contents, config, function="validate"): self.assertConfigError(contents, config.get_option(function)) def test_validate_no_hwpack_section(self): config = self.get_config("") self.assertValidationError("No [hwpack] section", config) def test_validate_no_name(self): config = self.get_config("[hwpack]\n") self.assertValidationError("Empty value for name", config) def test_validate_empty_name(self): config = self.get_config("[hwpack]\nname = \n") self.assertValidationError("Empty value for name", config) def test_validate_invalid_name(self): config = self.get_config("[hwpack]\nname = ~~\n") self.assertValidationError("Invalid name: ~~", config) def test_validate_invalid_include_debs(self): config = self.get_config( "[hwpack]\nname = ahwpack\n" "include-debs = if you don't mind\n") self.assertValidationError( "Invalid value for include-debs: Not a boolean: if you don't mind", config) def test_validate_invalid_supported(self): config = self.get_config( "[hwpack]\nname = ahwpack\nsupport = if you pay us\n") self.assertValidationError( "Invalid value for support: if you pay us", config) def test_validate_no_packages(self): config = self.get_config( "[hwpack]\nname = ahwpack\n\n") self.assertValidationError( "No packages in the [hwpack] section", config) def test_validate_empty_packages(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = \n") self.assertValidationError( "No packages in the [hwpack] section", config) def test_validate_invalid_package_name(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo ~~ bar\n") self.assertValidationError( "Invalid value in packages in the [hwpack] section: ~~", config) def test_validate_no_architectures(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n") self.assertValidationError( "No architectures in the [hwpack] section", config) def test_validate_empty_architectures(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n" "architectures = \n") self.assertValidationError( "No architectures in the [hwpack] section", config) def test_validate_invalid_package_name_in_assume_installed(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n" "architectures = armel\nassume-installed = bar ~~\n") self.assertValidationError( "Invalid value in assume-installed in the [hwpack] section: ~~", config) def test_validate_no_other_sections(self): config = self.get_config(self.valid_start + "\n") self.assertValidationError( "No sections other than [hwpack]", config) def test_validate_other_section_no_sources_entry(self): config = self.get_config(self.valid_start + "\n[ubuntu]\n") self.assertValidationError( "No sources-entry in the [ubuntu] section", config) def test_validate_other_section_empty_sources_entry(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = \n") self.assertValidationError( "The sources-entry in the [ubuntu] section is missing the URI", config) def test_validate_other_section_only_uri_in_sources_entry(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = foo\n") self.assertValidationError( "The sources-entry in the [ubuntu] section is missing the " "distribution", config) def test_validate_other_section_sources_entry_starting_with_deb(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = deb http://example.org/ " "foo main\n") self.assertValidationError( "The sources-entry in the [ubuntu] section shouldn't start " "with 'deb'", config) def test_validate_other_section_sources_entry_starting_with_deb_src(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = deb-src http://example.org/ " "foo main\n") self.assertValidationError( "The sources-entry in the [ubuntu] section shouldn't start " "with 'deb'", config) def test_validate_valid_config(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = foo bar\n") self.assertEqual(None, config.validate()) def test_validate_valid_config_with_dash_in_package_name(self): config = self.get_config( "[hwpack]\nname = ahwpack\n" "packages = u-boot\n" "architectures = armel\n\n" "[ubuntu]\nsources-entry = foo bar\n") self.assertEqual(None, config.validate()) def test_validate_supported_format(self): contents = self.valid_start + "format = 0.9\n" config = Config(StringIO(contents)) self.assertRaises(HwpackConfigError, config.validate) def test_validate_invalid_u_boot_package_name(self): config = self.get_config( self.valid_start_v2 + "u_boot_package = ~~\n") self.assertValidationError( "Invalid value in u_boot_package in the [hwpack] section: ~~", config) def test_validate_invalid_u_boot_file(self): config = self.get_config(self.valid_start_v2 + "u_boot_package = u-boot-linaro-s5pv310\n" "u_boot_file = ~~\n") self.assertValidationError("Invalid path: ~~", config) def test_validate_invalid_kernel_file(self): config = self.get_config(self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = ~~\n") self.assertValidationError("Invalid path: ~~", config, "_validate_vmlinuz") def test_validate_empty_kernel_file(self): config = self.get_config(self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = \n") self.assertValidationError("No kernel_file in the [hwpack] section", config, "_validate_vmlinuz") def test_validate_invalid_initrd_file(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = ~~\n") self.assertValidationError("Invalid path: ~~", config, "_validate_initrd") def test_validate_empty_initrd_file(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = \n") self.assertValidationError("No initrd_file in the [hwpack] section", config, "_validate_initrd") def test_validate_invalid_boot_script(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "mmc_id = 0:1\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = boot/initrd.img-3.0.0-1002-linaro-omap\n" "u_boot_in_boot_part = No\n" "boot_script = ~~\n") self.assertValidationError("Invalid path: ~~", config) def test_validate_invalid_dtb_file(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = boot/initrd.img-3.0.0-1002-linaro-omap\n" "boot_script = boot.scr\n" "u_boot_in_boot_part = No\n" "mmc_id = 0:1\n" "dtb_file = ~~\n") self.assertValidationError("Invalid path: ~~", config) def test_validate_invalid_spl_package_name(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = usr/bin/version/MLO\n" "partition_layout = bootfs_rootfs\n" "mmc_id = 0:1\n" "spl_package = ~~\n") self.assertValidationError( "Invalid value in spl_package in the [hwpack] section: ~~", config) def test_validate_invalid_spl_file(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = usr/bin/version/MLO\n" "partition_layout = bootfs_rootfs\n" "spl_package = x-loader--linaro-s5pv310\n" "spl_file = ~~\n") self.assertValidationError("Invalid path: ~~", config) def test_validate_partition_layout(self): partition_layout = 'apafs_bananfs' config = self.get_config(self.valid_start_v2 + "u_boot_package = " "u-boot-linaro-s5pv310\nu_boot_file = " "u-boot.bin\npartition_layout = %s\n" % partition_layout) self.assertValidationError( "Undefined partition layout %s in the [%s] section. " "Valid partition layouts are %s." % (partition_layout, 'hwpack', ", ".join(DEFINED_PARTITION_LAYOUTS)), config, "_validate_partition_layout") def test_validate_wired_interfaces(self): self.assertTrue("XXX What is an invalid interface name?") def test_validate_wireless_interfaces(self): self.assertTrue("XXX What is an invalid interface name?") def test_validate_u_boot_in_boot_part(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = boot/initrd.img-3.0.0-1002-linaro-omap\n" "boot_script = boot.scr\n" "mmc_id = 0:1\n" "u_boot_in_boot_part = Nope\n") self.assertValidationError( "Invalid value for u_boot_in_boot_part: Nope", config) def test_validate_u_boot_in_boot_part_bool(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = boot/initrd.img-3.0.0-1002-linaro-omap\n" "boot_script = boot.scr\n" "mmc_id = 0:1\n" "u_boot_in_boot_part = True\n") self.assertValidationError( "Invalid value for u_boot_in_boot_part: True", config) def test_validate_serial_tty(self): config = self.get_config( self.valid_start_v2 + "u_boot_package = u-boot-linaro-s5pv310\n" "u_boot_file = u-boot.bin\nserial_tty=tty\n") self.assertValidationError("Invalid serial tty: tty", config, "_validate_serial_tty") config = self.get_config( self.valid_start_v2 + "u_boot_package = u-boot-linaro-s5pv310\n" "u_boot_file = u-boot.bin\n" "serial_tty=ttxSAC1\n") self.assertValidationError("Invalid serial tty: ttxSAC1", config, "_validate_serial_tty") def test_validate_mmc_id(self): config = self.get_config(self.valid_complete_v2 + "mmc_id = x\n") self.assertValidationError("Invalid mmc_id x", config) def test_validate_boot_min_size(self): config = self.get_config(self.valid_complete_v2 + "boot_min_size = x\n") self.assertValidationError("Invalid boot min size x", config) def test_validate_root_min_size(self): config = self.get_config(self.valid_complete_v2 + "root_min_size = x\n") self.assertValidationError("Invalid root min size x", config) def test_validate_loader_min_size(self): config = self.get_config(self.valid_complete_v2 + "loader_min_size = x\n") self.assertValidationError("Invalid loader min size x", config) def test_validate_kernel_addr(self): config = self.get_config(self.valid_complete_v2 + "kernel_addr = 0x8000000\n") self.assertValidationError("Invalid kernel address: 0x8000000", config) config = self.get_config(self.valid_complete_v2 + "kernel_addr = 0x8000000x\n") self.assertValidationError( "Invalid kernel address: 0x8000000x", config) config = self.get_config(self.valid_complete_v2 + "kernel_addr = 80000000\n") self.assertValidationError("Invalid kernel address: 80000000", config) def test_validate_initrd_addr(self): config = self.get_config(self.valid_complete_v2 + "initrd_addr = 0x8000000\n") self.assertValidationError("Invalid initrd address: 0x8000000", config) config = self.get_config(self.valid_complete_v2 + "initrd_addr = 0x8000000x\n") self.assertValidationError( "Invalid initrd address: 0x8000000x", config) config = self.get_config(self.valid_complete_v2 + "initrd_addr = 80000000\n") self.assertValidationError("Invalid initrd address: 80000000", config) def test_validate_load_addr(self): config = self.get_config(self.valid_complete_v2 + "load_addr = 0x8000000\n") self.assertValidationError("Invalid load address: 0x8000000", config) config = self.get_config(self.valid_complete_v2 + "load_addr = 0x8000000x\n") self.assertValidationError("Invalid load address: 0x8000000x", config) config = self.get_config(self.valid_complete_v2 + "load_addr = 80000000\n") self.assertValidationError("Invalid load address: 80000000", config) def test_validate_dtb_addr(self): config = self.get_config(self.valid_complete_v2 + "dtb_addr = 0x8000000\n") self.assertValidationError("Invalid dtb address: 0x8000000", config) config = self.get_config(self.valid_complete_v2 + "dtb_addr = 0x8000000x\n") self.assertValidationError("Invalid dtb address: 0x8000000x", config) config = self.get_config(self.valid_complete_v2 + "dtb_addr = 80000000\n") self.assertValidationError("Invalid dtb address: 80000000", config) def test_wired_interfaces(self): config = self.get_config(self.valid_complete_v2 + "wired_interfaces = eth0\n" + self.valid_end) config.validate() self.assertEqual(["eth0"], config.wired_interfaces) config = self.get_config(self.valid_complete_v2 + "wired_interfaces = eth0 eth1 usb2\n" + self.valid_end) config.validate() self.assertEqual(["eth0", "eth1", "usb2"], config.wired_interfaces) def test_wireless_interfaces(self): config = self.get_config(self.valid_complete_v2 + "wireless_interfaces = wlan0\n" + self.valid_end) config.validate() self.assertEqual(["wlan0"], config.wireless_interfaces) config = self.get_config(self.valid_complete_v2 + "wireless_interfaces = wlan0 wl1 usb2\n" + self.valid_end) config.validate() self.assertEqual(["wlan0", "wl1", "usb2"], config.wireless_interfaces) def test_partition_layout(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("bootfs_rootfs", config.partition_layout) def test_u_boot_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("usr/lib/u-boot/smdkv310/u-boot.bin", config.bootloader_file) def test_u_boot_package(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("u-boot-linaro-s5pv310", config.bootloader_package) def test_spl_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("usr/lib/x-loader/omap4430panda/MLO", config.spl_file) def test_kernel_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("boot/vmlinuz-*-linaro-omap", config.vmlinuz) def test_initrd_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("boot/initrd.img-*-linaro-omap", config.initrd) def test_dtb_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("boot/dt-*-linaro-omap/omap4-panda.dtb", config.dtb_file) def test_extra_boot_options(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual( "earlyprintk fixrtc nocompcache vram=48M " "omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000", config.extra_boot_options) def test_extra_serial_options(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("console=tty0 console=ttyO2,115200n8", config.extra_serial_options) def test_boot_script(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("boot.scr", config.boot_script) def test_u_boot_in_boot_part(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("Yes", config.bootloader_file_in_boot_part) def test_spl_package(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("x-loader-omap4-panda", config.spl_package) def test_serial_tty(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("ttySAC1", config.serial_tty) def test_mmc_id(self): config = self.get_config(self.valid_complete_v2 + "mmc_id = 0:1\n" + self.valid_end) config.validate() self.assertEqual("0:1", config.mmc_id) def test_boot_min_size(self): config = self.get_config(self.valid_complete_v2 + "boot_min_size = 50\n" + self.valid_end) config.validate() self.assertEqual("50", config.boot_min_size) def test_root_min_size(self): config = self.get_config(self.valid_complete_v2 + "root_min_size = 50\n" + self.valid_end) config.validate() self.assertEqual("50", config.root_min_size) def test_loader_min_size(self): config = self.get_config(self.valid_complete_v2 + "loader_min_size = 2\n" + self.valid_end) config.validate() self.assertEqual("2", config.loader_min_size) def test_kernel_addr(self): config = self.get_config(self.valid_complete_v2 + "kernel_addr = 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.kernel_addr) config = self.get_config(self.valid_complete_v2 + "kernel_addr = 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8aBcdEFf", config.kernel_addr) def test_initrd_addr(self): config = self.get_config(self.valid_complete_v2 + "initrd_addr = 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.initrd_addr) config = self.get_config(self.valid_complete_v2 + "initrd_addr = 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8aBcdEFf", config.initrd_addr) def test_load_addr(self): config = self.get_config(self.valid_complete_v2 + "load_addr = 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.load_addr) config = self.get_config(self.valid_complete_v2 + "load_addr = 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8aBcdEFf", config.load_addr) def test_dtb_addr(self): config = self.get_config(self.valid_complete_v2 + "dtb_addr = 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.dtb_addr) config = self.get_config(self.valid_complete_v2 + "dtb_addr = 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8aBcdEFf", config.dtb_addr) def test_name(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n" "architectures = armel\n") self.assertEqual("ahwpack", config.name) def test_include_debs(self): config = self.get_config(self.valid_start + "include-debs = false\n") self.assertEqual(False, config.include_debs) def test_include_debs_defaults_true(self): config = self.get_config(self.valid_start) self.assertEqual(True, config.include_debs) def test_include_debs_defaults_true_on_empty(self): config = self.get_config(self.valid_start + "include-debs = \n") self.assertEqual(True, config.include_debs) def test_origin(self): config = self.get_config(self.valid_start + "origin = linaro\n") self.assertEqual("linaro", config.origin) def test_origin_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.origin) def test_origin_None_on_empty(self): config = self.get_config(self.valid_start + "origin = \n") self.assertEqual(None, config.origin) def test_maintainer(self): maintainer = "Linaro Developers " config = self.get_config( self.valid_start + "maintainer = %s\n" % maintainer) self.assertEqual(maintainer, config.maintainer) def test_maintainer_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.maintainer) def test_maintainer_None_on_empty(self): config = self.get_config(self.valid_start + "maintainer = \n") self.assertEqual(None, config.maintainer) def test_support_supported(self): config = self.get_config(self.valid_start + "support = supported\n") self.assertEqual("supported", config.support) def test_support_unsupported(self): config = self.get_config(self.valid_start + "support = unsupported\n") self.assertEqual("unsupported", config.support) def test_support_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.support) def test_support_None_on_empty(self): config = self.get_config(self.valid_start + "support = \n") self.assertEqual(None, config.support) def test_packages(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo bar\n" "architectures=armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_packages_with_newline(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\n bar\n" "architectures=armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_packages_filters_duplicates(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo bar foo\n" "architectures=armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_sources_single(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = http://example.org foo\n") self.assertEqual({"ubuntu": "http://example.org foo"}, config.sources) def test_sources_multiple(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = http://example.org foo\n" + "\n[linaro]\nsources-entry = http://example.org bar\n") self.assertEqual( {"ubuntu": "http://example.org foo", "linaro": "http://example.org bar"}, config.sources) def test_architectures(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=foo bar\n") self.assertEqual(["foo", "bar"], config.architectures) def test_architectures_with_newline(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=foo\n bar\n") self.assertEqual(["foo", "bar"], config.architectures) def test_architectures_filters_duplicates(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\n" "architectures=foo bar foo\n") self.assertEqual(["foo", "bar"], config.architectures) def test_assume_installed(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=armel\n" "assume-installed=foo bar\n") self.assertEqual(["foo", "bar"], config.assume_installed) def test_assume_installed_with_newline(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=armel\n" "assume-installed=foo\n bar\n") self.assertEqual(["foo", "bar"], config.assume_installed) def test_assume_installed_filters_duplicates(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=armel\n" "assume-installed=foo bar foo\n") self.assertEqual(["foo", "bar"], config.assume_installed) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_better_tarfile.py0000664000175000017500000001722312400111666031347 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from contextlib import contextmanager from StringIO import StringIO import tarfile from testtools import TestCase from linaro_image_tools.hwpack.better_tarfile import writeable_tarfile @contextmanager def standard_tarfile(backing_file, mode="r", seek=True): """A context manager to open a stdlib tarfile. :param backing_file: the file object to take the tarfile contents from. :param mode: the mode to open the tarfile with. :param seek: whether to seek the backing file to 0 before opening. """ if seek: backing_file.seek(0) tf = tarfile.TarFile.open(mode=mode, fileobj=backing_file) try: yield tf finally: tf.close() class TarFileTests(TestCase): def test_creates_empty_tarfile(self): backing_file = StringIO() with writeable_tarfile(backing_file): pass with standard_tarfile(backing_file, seek=False) as tf: self.assertEqual([], tf.getnames()) def create_simple_tarball(self, contents, **kwargs): backing_file = StringIO() with writeable_tarfile(backing_file, **kwargs) as tf: for path, content in contents: if path[-1] == '/': tf.create_dir(path) else: tf.create_file_from_string(path, content) return backing_file def test_create_file_from_string_adds_path(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual(["foo"], tf.getnames()) def test_create_file_from_string_uses_content(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual("bar", tf.extractfile("foo").read()) def test_create_file_from_string_sets_size(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual(3, tf.getmember("foo").size) def test_create_file_from_string_sets_mode(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual(0644, tf.getmember("foo").mode) def test_create_file_from_string_sets_type(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual(tarfile.REGTYPE, tf.getmember("foo").type) def test_create_file_from_string_sets_linkname(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual('', tf.getmember("foo").linkname) def test_create_file_uses_default_mtime(self): now = 126793 backing_file = self.create_simple_tarball( [("foo", "bar")], default_mtime=now) with standard_tarfile(backing_file) as tf: self.assertEqual(now, tf.getmember("foo").mtime) def test_create_file_uses_default_uid(self): uid = 1259 backing_file = self.create_simple_tarball( [("foo", "bar")], default_uid=uid) with standard_tarfile(backing_file) as tf: self.assertEqual(uid, tf.getmember("foo").uid) def test_create_file_uses_default_gid(self): gid = 2259 backing_file = self.create_simple_tarball( [("foo", "bar")], default_gid=gid) with standard_tarfile(backing_file) as tf: self.assertEqual(gid, tf.getmember("foo").gid) def test_create_file_uses_default_uname(self): uname = "someperson" backing_file = self.create_simple_tarball( [("foo", "bar")], default_uname=uname) with standard_tarfile(backing_file) as tf: self.assertEqual(uname, tf.getmember("foo").uname) def test_create_file_uses_default_gname(self): gname = "somegroup" backing_file = self.create_simple_tarball( [("foo", "bar")], default_gname=gname) with standard_tarfile(backing_file) as tf: self.assertEqual(gname, tf.getmember("foo").gname) def test_create_dir_adds_path(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual(["foo"], tf.getnames()) def test_create_dir_sets_name(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual("foo", tf.getmember("foo").name) def test_create_dir_sets_type(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual(tarfile.DIRTYPE, tf.getmember("foo").type) def test_create_dir_sets_size(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual(0, tf.getmember("foo").size) def test_create_dir_sets_mode(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual(0755, tf.getmember("foo").mode) def test_create_dir_sets_linkname(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual('', tf.getmember("foo").linkname) def test_create_dir_uses_default_mtime(self): now = 126793 backing_file = self.create_simple_tarball( [("foo/", "")], default_mtime=now) with standard_tarfile(backing_file) as tf: self.assertEqual(now, tf.getmember("foo").mtime) def test_create_dir_uses_default_uid(self): uid = 1259 backing_file = self.create_simple_tarball( [("foo/", "")], default_uid=uid) with standard_tarfile(backing_file) as tf: self.assertEqual(uid, tf.getmember("foo").uid) def test_create_dir_uses_default_gid(self): gid = 2259 backing_file = self.create_simple_tarball( [("foo/", "")], default_gid=gid) with standard_tarfile(backing_file) as tf: self.assertEqual(gid, tf.getmember("foo").gid) def test_create_dir_uses_default_uname(self): uname = "someperson" backing_file = self.create_simple_tarball( [("foo/", "")], default_uname=uname) with standard_tarfile(backing_file) as tf: self.assertEqual(uname, tf.getmember("foo").uname) def test_create_dir_uses_default_gname(self): gname = "somegroup" backing_file = self.create_simple_tarball( [("foo/", "")], default_gname=gname) with standard_tarfile(backing_file) as tf: self.assertEqual(gname, tf.getmember("foo").gname) linaro-image-tools-2014.11/linaro_image_tools/hwpack/tests/test_builder.py0000664000175000017500000005305212400111666030002 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import os import tarfile from testtools import TestCase from testtools.matchers import Equals from linaro_image_tools.hwpack.builder import ( ConfigFileMissing, HardwarePackBuilder, logger as builder_logger, ) from linaro_image_tools.hwpack.package_unpacker import PackageUnpacker from linaro_image_tools.hwpack.config import HwpackConfigError from linaro_image_tools.hwpack.hardwarepack import Metadata from linaro_image_tools.hwpack.packages import ( FetchedPackage, PackageMaker, ) from linaro_image_tools.hwpack.tarfile_matchers import TarfileHasFile from linaro_image_tools.hwpack.testing import ( AppendingHandler, AptSourceFixture, ChdirToTempdirFixture, ConfigFileFixture, ContextManagerFixture, DummyFetchedPackage, EachOf, IsHardwarePack, MatchesStructure, Not, ) from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( MockSomethingFixture, MockCmdRunnerPopenFixture, ) class ConfigFileMissingTests(TestCase): def test_str(self): exc = ConfigFileMissing("path") self.assertEqual("No such config file: 'path'", str(exc)) class PackageUnpackerTests(TestCaseWithFixtures): def test_creates_tempdir(self): with PackageUnpacker() as package_unpacker: self.assertTrue(os.path.exists(package_unpacker.tempdir)) def test_tempfiles_are_removed(self): tempdir = None with PackageUnpacker() as package_unpacker: tempdir = package_unpacker.tempdir self.assertFalse(os.path.exists(tempdir)) def test_unpack_package(self): fixture = MockCmdRunnerPopenFixture(assert_child_finished=False) self.useFixture(fixture) package_file_name = "package-to-unpack" with PackageUnpacker() as package_unpacker: package_unpacker.unpack_package(package_file_name) package_dir = package_unpacker.get_path(package_file_name) self.assertEquals( ["tar -C %s -xf -" % package_dir, "dpkg --fsys-tarfile %s" % package_file_name], fixture.mock.commands_executed) def test_get_file_returns_tempfile(self): package = 'package' file = 'dummyfile' with PackageUnpacker() as package_unpacker: self.useFixture(MockSomethingFixture( package_unpacker, 'unpack_package', lambda package: None)) self.useFixture(MockSomethingFixture( os.path, 'exists', lambda file: True)) tempfile = package_unpacker.get_file(package, file) self.assertEquals(tempfile, os.path.join(package_unpacker.get_path(package), file)) def test_get_file_raises(self): package = 'package' file = 'dummyfile' with PackageUnpacker() as package_unpacker: self.useFixture(MockSomethingFixture( package_unpacker, 'unpack_package', lambda package: None)) self.assertRaises(AssertionError, package_unpacker.get_file, package, file) def test_get_file_no_clash(self): # Test that PackageUnpacker, asked to get the same file path # from 2 different packages, return reference to *different* # temporary files package1 = 'package1' package2 = 'package2' file = 'dummyfile' with PackageUnpacker() as package_unpacker: self.useFixture(MockSomethingFixture( package_unpacker, 'unpack_package', lambda package: None)) self.useFixture(MockSomethingFixture( os.path, 'exists', lambda file: True)) tempfile1 = package_unpacker.get_file(package1, file) tempfile2 = package_unpacker.get_file(package2, file) self.assertNotEquals(tempfile1, tempfile2) class HardwarePackBuilderTests(TestCaseWithFixtures): config_v3 = "\n".join(["format: 3.0", "name: ahwpack", "architectures: armel", "serial_tty: ttySAC1", "partition_layout:", " - bootfs_rootfs", "boot_script: boot.scr", "mmc_id: 0:1", "kernel_file: boot/vmlinuz-*-linaro-omap", "initrd_file: boot/initrd.img-*-linaro-omap", "dtb_file: boot/dt-*-linaro-omap/omap4-panda.dtb", "packages:", " - %s", " - %s", ""]) bootloader_config = "\n".join([" package: %s", " in_boot_part: %s", ""]) def setUp(self): super(HardwarePackBuilderTests, self).setUp() self.useFixture(ChdirToTempdirFixture()) self.extra_config = { 'format': '2.0', 'u-boot-package': 'wanted-package', 'u-boot-file': 'wanted-file', 'partition_layout': 'bootfs_rootfs', 'x_loader_package': 'x-loader-omap4-panda', 'x_loader_file': 'usr/lib/x-loader/omap4430panda/MLO', 'kernel_file': 'boot/vmlinuz-3.0.0-1002-linaro-omap', 'initrd_file': 'boot/initrd.img-3.0.0-1002-linaro-omap', 'boot_script': 'boot.scr', 'mmc_id': '0:1', 'u_boot_in_boot_part': 'no'} def test_raises_on_missing_configuration(self): e = self.assertRaises( ConfigFileMissing, HardwarePackBuilder, "nonexistant", "1.0", []) self.assertEqual("nonexistant", e.filename) def test_validates_configuration(self): config = self.useFixture(ConfigFileFixture('')) self.assertRaises( HwpackConfigError, HardwarePackBuilder, config.filename, "1.0", []) def makeMetaDataAndConfigFixture( self, packages, sources, hwpack_name="ahwpack", hwpack_version="1.0", architecture="armel", extra_config={}): config_text = ( '[hwpack]\n' 'name=%s\n' 'packages=%s\n' 'architectures=%s\n' % (hwpack_name, ' '.join(packages), architecture)) for key, value in extra_config.iteritems(): config_text += '%s=%s\n' % (key, value) config_text += '\n' for source_id, source in sources.iteritems(): config_text += '\n' config_text += '[%s]\n' % source_id config_text += 'sources-entry=%s\n' % source config = self.useFixture(ConfigFileFixture(config_text)) return Metadata(hwpack_name, hwpack_version, architecture), config def test_find_fetched_package_finds(self): package_name = "dummy-package" wanted_package_name = "wanted-package" available_package = DummyFetchedPackage(package_name, "1.1") wanted_package = DummyFetchedPackage(wanted_package_name, "1.1") sources_dict = self.sourcesDictForPackages([available_package, wanted_package]) _, config = self.makeMetaDataAndConfigFixture( [package_name, wanted_package_name], sources_dict, extra_config=self.extra_config) builder = HardwarePackBuilder(config.filename, "1.0", []) found_package = builder.find_fetched_package( [available_package, wanted_package], wanted_package_name) self.assertEquals(wanted_package, found_package) def test_find_fetched_package_raises(self): package_name = "dummy-package" wanted_package_name = "wanted-package" available_package = DummyFetchedPackage(package_name, "1.1") sources_dict = self.sourcesDictForPackages([available_package]) _, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict, extra_config=self.extra_config) builder = HardwarePackBuilder(config.filename, "1.0", []) packages = [available_package] self.assertRaises(AssertionError, builder.find_fetched_package, packages, wanted_package_name) def test_creates_external_manifest(self): available_package = DummyFetchedPackage("foo", "1.1") sources_dict = self.sourcesDictForPackages([available_package]) metadata, config = self.makeMetaDataAndConfigFixture( ["foo"], sources_dict) builder = HardwarePackBuilder(config.filename, "1.0", []) builder.build() self.assertTrue( os.path.isfile("hwpack_ahwpack_1.0_armel.manifest.txt")) def sourcesDictForPackages(self, packages): source = self.useFixture(AptSourceFixture(packages)) return {'ubuntu': source.sources_entry} def test_builds_one_pack_per_arch(self): available_package = DummyFetchedPackage("foo", "1.1") sources_dict = self.sourcesDictForPackages([available_package]) metadata, config = self.makeMetaDataAndConfigFixture( ["foo"], sources_dict, architecture="i386 armel") builder = HardwarePackBuilder(config.filename, "1.0", []) builder.build() self.assertTrue(os.path.isfile("hwpack_ahwpack_1.0_i386.tar.gz")) self.assertTrue(os.path.isfile("hwpack_ahwpack_1.0_armel.tar.gz")) def test_builds_correct_contents(self): package_name = "foo" available_package = DummyFetchedPackage(package_name, "1.1") sources_dict = self.sourcesDictForPackages([available_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict) builder = HardwarePackBuilder(config.filename, metadata.version, []) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [available_package], sources_dict, package_spec=package_name)) def test_builds_correct_contents_multiple_packages(self): package_name1 = "foo" package_name2 = "goo" available_package1 = DummyFetchedPackage(package_name1, "1.1") available_package2 = DummyFetchedPackage(package_name2, "1.2") sources_dict = self.sourcesDictForPackages( [available_package1, available_package2]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name1, package_name2], sources_dict) builder = HardwarePackBuilder(config.filename, metadata.version, []) builder.build() hwpack_filename = "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture) self.assertThat( hwpack_filename, IsHardwarePack( metadata, [available_package1, available_package2], sources_dict, package_spec='%s, %s' % (package_name1, package_name2))) self.assertThat( hwpack_filename, IsHardwarePack( metadata, [available_package2, available_package1], sources_dict, package_spec='%s, %s' % (package_name1, package_name2))) def test_obeys_include_debs(self): package_name = "foo" available_package = DummyFetchedPackage(package_name, "1.1") sources_dict = self.sourcesDictForPackages([available_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict, extra_config={'include-debs': 'no'}) builder = HardwarePackBuilder(config.filename, metadata.version, []) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [available_package], sources_dict, packages_without_content=[available_package], package_spec=package_name)) def test_obeys_assume_installed(self): package_name = "foo" assume_installed = "bar" available_package = DummyFetchedPackage( package_name, "1.1", depends=assume_installed) dependency_package = DummyFetchedPackage(assume_installed, "1.1") sources_dict = self.sourcesDictForPackages( [available_package, dependency_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict, extra_config={'assume-installed': assume_installed}) builder = HardwarePackBuilder(config.filename, metadata.version, []) builder.build() filename = "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture) self.assertThat( filename, IsHardwarePack( metadata, [available_package], sources_dict, package_spec=package_name)) tf = tarfile.open(filename, mode="r:gz") try: self.assertThat( tf, Not(TarfileHasFile("pkgs/%s" % dependency_package.filename))) finally: tf.close() def test_includes_local_debs(self): package_name = "foo" maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) local_path = maker.make_package(package_name, "1.2", {}) available_package = FetchedPackage.from_deb(local_path) sources_dict = self.sourcesDictForPackages([]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict) builder = HardwarePackBuilder( config.filename, metadata.version, [local_path]) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [available_package], sources_dict, package_spec=package_name)) def test_prefers_local_debs(self): package_name = "foo" maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) # The point here is that remote_package has a later version than # local_package, but local_package is still preferred. remote_package = DummyFetchedPackage(package_name, "1.1") local_path = maker.make_package(package_name, "1.0", {}) local_package = FetchedPackage.from_deb(local_path) sources_dict = self.sourcesDictForPackages([remote_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict) builder = HardwarePackBuilder( config.filename, metadata.version, [local_path]) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [local_package], sources_dict, package_spec=package_name)) def test_includes_local_debs_even_if_not_in_config(self): package_name = "foo" local_name = "bar" maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) remote_package = DummyFetchedPackage(package_name, "1.1") local_path = maker.make_package(local_name, "1.0", {}) local_package = FetchedPackage.from_deb(local_path) sources_dict = self.sourcesDictForPackages([remote_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict) builder = HardwarePackBuilder( config.filename, metadata.version, [local_path]) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [remote_package, local_package], sources_dict, package_spec=package_name)) def test_warn_if_not_including_local_deb(self): package_name = "foo" local_name = "bar" maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) remote_package = DummyFetchedPackage(package_name, "1.1") local_path = maker.make_package(local_name, "1.0", {}) sources_dict = self.sourcesDictForPackages([remote_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict, extra_config={'assume-installed': local_name}) builder = HardwarePackBuilder( config.filename, metadata.version, [local_path]) handler = AppendingHandler() builder_logger.addHandler(handler) self.addCleanup(builder_logger.removeHandler, handler) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [remote_package], sources_dict, package_spec=package_name)) self.assertThat( handler.messages, EachOf([MatchesStructure(levelname=Equals('WARNING'))])) self.assertThat( handler.messages[0].getMessage(), Equals("Local package 'bar' not included")) def test_global_and_board_bootloader(self): package_names = ['package0', 'package1', 'package2', 'package3'] files = { package_names[0]: ["usr/lib/u-boot/omap4_panda/u-boot.img", "usr/share/doc/u-boot-linaro-omap4-panda/copyright"], package_names[1]: ["usr/lib/u-boot/omap4_panda/u-boot.img", "some/path/file"], package_names[2]: [], package_names[3]: [], } config_v3 = self.config_v3 + "\n".join([ "bootloaders:", " u_boot:", self.bootloader_config, " file: " + files[package_names[0]][0], " copy_files:", " " + package_names[2] + ":", " - some_file", "boards:", " board1:", " bootloaders:", " u_boot:", " package: %s", " file: " + files[package_names[1]][0], " copy_files:", " " + package_names[3] + ":", " - some_file", " in_boot_part: true", "sources:", " ubuntu: %s"]) # Generate some test packages available_packages = [] maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) for package_name in package_names: # The files parameter to make_package is a list of files to create. # These files are text files containing package_name and their # path. Since package_name is different for each package, this # gives each file a unique content. deb_file_path = maker.make_package(package_name, '1.0', {}, files=files[package_name]) dummy_package = DummyFetchedPackage( package_name, "1.0", content=open(deb_file_path).read()) available_packages.append(dummy_package) source = self.useFixture(AptSourceFixture(available_packages)) # Generate a V3 config config_v3 = config_v3 % (package_names[0], package_names[1], package_names[0], "True", package_names[1], source.sources_entry) config_file_fixture = self.useFixture(ConfigFileFixture(config_v3)) # Build a hardware pack builder = HardwarePackBuilder( config_file_fixture.filename, "1.0", [os.path.join(source.rootdir, package.filepath) for package in available_packages]) builder.build() stored_package_names = [p.name for p in builder.packages] for package_name in package_names: self.assertIn(package_name, stored_package_names) # Read the contents of the hardware pack, making sure it is as expected tf = tarfile.open("hwpack_ahwpack_1.0_armel.tar.gz", mode="r:gz") # We check the content of each file when content != None. For our test # files this is " " so they can be # uniquely identified. expected_files = [ ("u_boot/" + files[package_names[0]][0], package_names[0] + " " + files[package_names[0]][0]), ("board1/u_boot/" + files[package_names[1]][0], package_names[1] + " " + files[package_names[1]][0])] for expected_file, contents in expected_files: self.assertThat( tf, TarfileHasFile(expected_file, content=contents)) linaro-image-tools-2014.11/linaro_image_tools/hwpack/package_unpacker.py0000664000175000017500000000476112400111666027441 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011, 2013 Linaro # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import logging import os import tempfile from subprocess import PIPE from shutil import rmtree from linaro_image_tools import cmd_runner logger = logging.getLogger(__name__) class PackageUnpacker(object): def __enter__(self): self.tempdir = tempfile.mkdtemp() return self def __exit__(self, type, value, traceback): if self.tempdir is not None and os.path.exists(self.tempdir): rmtree(self.tempdir) def get_path(self, package_file_name, file_name=''): """Get package or file path in unpacker tmp dir.""" package_dir = os.path.basename(package_file_name) return os.path.join(self.tempdir, package_dir, file_name) def unpack_package(self, package_file_name): # We could extract only a single file, but since dpkg will pipe # the entire package through tar anyway we might as well extract all. unpack_dir = self.get_path(package_file_name) if not os.path.isdir(unpack_dir): os.mkdir(unpack_dir) p = cmd_runner.run(["tar", "-C", unpack_dir, "-xf", "-"], stdin=PIPE) cmd_runner.run(["dpkg", "--fsys-tarfile", package_file_name], stdout=p.stdin).communicate() p.communicate() def get_file(self, package, file): # File path passed here must not be absolute, or file from # real filesystem will be referenced. assert file and file[0] != '/' self.unpack_package(package) logger.debug("Unpacked package %s." % package) temp_file = self.get_path(package, file) assert os.path.exists(temp_file), "The file '%s' was " \ "not found in the package '%s'." % (file, package) return temp_file linaro-image-tools-2014.11/linaro_image_tools/hwpack/packages.py0000664000175000017500000007474012400111666025740 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import hashlib import logging import os import re import shutil from string import Template import subprocess import tempfile import urlparse from apt.cache import Cache from apt.cache import FetchFailedException from apt.package import FetchError import apt_pkg from debian.debfile import DebFile from linaro_image_tools import cmd_runner logger = logging.getLogger(__name__) def get_packages_file(packages, extra_text=None, rel_to=None): """Get the Packages file contents indexing `packages`. :param packages: the packages to index. :type packages: an iterable of FetchedPackages. :param extra_text: extra text to insert in to each stanza. Should not end with a newline. :type extra_text: str or None :param rel_to: If present, generate the Filename: parts of the Packages file as paths relative to this location. If not present, Filename: will just include the file name (not the path). :return: the Packages file contents indexing `packages`. :rtype: str """ content = "" for package in packages: parts = [] parts.append('Package: %s' % package.name) if extra_text is not None: parts.append(extra_text) parts.append('Version: %s' % package.version) if rel_to is not None: filename = os.path.relpath(package.filepath, rel_to) else: filename = package.filename parts.append('Filename: %s' % filename) parts.append('Size: %d' % package.size) parts.append('Architecture: %s' % package.architecture) if package.depends: parts.append('Depends: %s' % package.depends) if package.pre_depends: parts.append('Pre-Depends: %s' % package.pre_depends) if package.multi_arch: parts.append('Multi-Arch: %s' % package.multi_arch) if package.conflicts: parts.append('Conflicts: %s' % package.conflicts) if package.recommends: parts.append('Recommends: %s' % package.recommends) if package.provides: parts.append('Provides: %s' % package.provides) if package.replaces: parts.append('Replaces: %s' % package.replaces) if package.breaks: parts.append('Breaks: %s' % package.breaks) parts.append('MD5sum: %s' % package.md5) content += "\n".join(parts) content += "\n\n" return content def stringify_relationship(pkg, relationship): """Given a Package, return a string of the specified relationship. apt.package.Version stores the relationship information of the package as objects. This function will convert those objects in to the string form that we are used to from debian/control or Packages files. :param pkg: the package to take the relationship information from. :type pkg: apt.package.Version :param relationship: the relationship to stringify, as understood by apt.package.Package.get_dependencies, e.g. "Depends", "PreDepends". :type relationship: str or None if the package has no relationships of that type. """ relationship_str = None pkg_dependencies = pkg.get_dependencies(relationship) if pkg_dependencies: relationship_list = [] for or_dep in pkg_dependencies: or_list = [] for or_alternative in or_dep.or_dependencies: suffix = "" if or_alternative.relation: relation = or_alternative.relation if relation in ('<', '>'): # The choice made here by python-apt is to report the # relationship in a Python spelling; as far as apt # knows, < is a deprecated spelling of <=; << is the # spelling of "strictly less than". Similarly for >. relation *= 2 suffix = " (%s %s)" % (relation, or_alternative.version) or_list.append("%s%s" % (or_alternative.name, suffix)) relationship_list.append(" | ".join(or_list)) relationship_str = ", ".join(relationship_list) return relationship_str class DummyProgress(object): """An AcquireProgress that silences all output. This can be used to ensure that apt produces no output when fetching files. """ def start(self): pass def ims_hit(self, item): pass def fail(self, item): pass def fetch(self, item): pass def pulse(self, owner): return True def media_change(self): return False def stop(self): pass class TemporaryDirectoryManager(object): def __init__(self): self._temporary_directories = None def __enter__(self): if self._temporary_directories is not None: raise AssertionError("__enter__ must not be called twice") self._temporary_directories = [] return self def __exit__(self, exc_type=None, exc_value=None, traceback=None): if self._temporary_directories is None: return for tmpdir in self._temporary_directories: shutil.rmtree(tmpdir) self._temporary_directories = None return False def make_temporary_directory(self): """Create a temporary directory and return its path. The created directory will be deleted on __exit__. """ if self._temporary_directories is None: raise AssertionError("__enter__ must be called") tmpdir = tempfile.mkdtemp() self._temporary_directories.append(tmpdir) return tmpdir class LocalArchiveMaker(TemporaryDirectoryManager): def sources_entry_for_debs(self, local_debs, label=None): tmpdir = self.make_temporary_directory() with open(os.path.join(tmpdir, 'Packages'), 'w') as packages_file: packages_file.write(get_packages_file(local_debs, rel_to=tmpdir)) if label: cmd_runner.run( ['apt-ftparchive', '-oAPT::FTPArchive::Release::Label=%s' % label, 'release', tmpdir], stdout=open(os.path.join(tmpdir, 'Release'), 'w')).wait() return 'file://%s ./' % (tmpdir, ) class PackageMaker(TemporaryDirectoryManager): """An object that can create binary debs on the fly. PackageMakers implement the context manager protocol to manage the temporary directories the debs are created in. """ # This template (and the code that uses it) is made more awkward by the # fact that blank lines are invalid in control files -- so in particular # when there are no relationships, there must be no blank line between the # Maintainer and the Description. control_file_template = Template('''\ Package: ${name} Version: ${version} Architecture: ${architecture} Maintainer: Nobody ${relationships}\ Description: Dummy package to install a hwpack This package was created automatically by linaro-media-create ''') def make_package(self, name, version, relationships, architecture='all', files=[]): tmp_dir = self.make_temporary_directory() filename = '%s_%s_%s' % (name, version, architecture) packaging_dir = os.path.join(tmp_dir, filename) os.mkdir(packaging_dir) os.mkdir(os.path.join(packaging_dir, 'DEBIAN')) relationship_strs = [] for relationship_name, relationship_value in relationships.items(): relationship_strs.append( '%s: %s\n' % (relationship_name, relationship_value)) subst_vars = dict( architecture=architecture, name=name, relationships=''.join(relationship_strs), version=version, ) control_file_text = self.control_file_template.safe_substitute( subst_vars) # If any files have been specified, create them for file_path in files: os.makedirs(os.path.join(packaging_dir, os.path.dirname(file_path))) with open(os.path.join(packaging_dir, file_path), 'w') as new_file: new_file.write(name + " " + file_path) with open(os.path.join( packaging_dir, 'DEBIAN', 'control'), 'w') as control_file: control_file.write(control_file_text) env = os.environ env['LC_ALL'] = 'C' env['NO_PKG_MANGLE'] = '1' proc = cmd_runner.Popen( ['dpkg-deb', '-b', '-Zgzip', packaging_dir], env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdoutdata, stderrdata) = proc.communicate() if proc.returncode: raise ValueError("dpkg-deb failed!\n%s" % stderrdata) if stderrdata: raise ValueError("dpkg-deb had warnings:\n%s" % stderrdata) deb_file_path_match = re.match( "dpkg-deb: building package `.*' in `(.*)'", stdoutdata) if not deb_file_path_match: raise ValueError( "failed to find filename in dpkg-deb output:\n%s" % stdoutdata) return deb_file_path_match.group(1) class FetchedPackage(object): """The result of fetching packages. :ivar name: the name of the fetched package. :type name: str :ivar version: the version of the fetched package. :type version: str :ivar filename: the filename that the package has. :type filename: str :ivar content: a file that the content of the package can be read from, or None if the content is not known. :type content: a file-like object or None :ivar size: the size of the package :type size: int :ivar md5: the hex representation of the md5sum of the contents of the package. :type md5: str :ivar architecture: the architecture that the package is for, may be 'all'. :type architecture: str :ivar depends: the Depends string that the package has, i.e. the dependencies as specified in debian/control. May be None if the package has none. :type depends: str or None :ivar pre_depends: the Pre-Depends string that the package has, i.e. the pre-dependencies as specified in debian/control. May be None if the package has none. :type pre_depends: str or None :ivar multi_arch: the Multi-Arch string that the package has. May be None if the package has none. :type multi_arch: str or None :ivar conflicts: the Conflicts string that the package has, i.e. the conflicts as specified in debian/control. May be None if the package has none. :type conflicts: str or None :ivar recommends: the Recommends string that the package has, i.e. the recommends as specified in debian/control. May be None if the package has none. :type recommends: str or None :ivar provides: the Provides string that the package has, i.e. the provides as specified in debian/control. May be None if the package has none. :type provides: str or None :ivar replaces: the Replaces string that the package has, i.e. the replaces as specified in debian/control. May be None if the package has none. :type replaces: str or None :ivar breaks: the Breaks string that the package has, i.e. the breaks as specified in debian/control. May be None if the package has none. :type breaks: str or None """ def __init__(self, name, version, filename, size, md5, architecture, depends=None, pre_depends=None, multi_arch=None, conflicts=None, recommends=None, provides=None, replaces=None, breaks=None): """Create a FetchedPackage. See the instance variables for the arguments. """ self.name = name self.version = version self.filename = filename self.size = size self.md5 = md5 self.architecture = architecture self.depends = depends self.pre_depends = pre_depends self.multi_arch = multi_arch self.conflicts = conflicts self.recommends = recommends self.provides = provides self.replaces = replaces self.breaks = breaks self.content = None self._file_path = None @property def filepath(self): if self._file_path is not None: return self._file_path else: return self.filename @classmethod def from_apt(cls, pkg, filename, content=None): """Create a FetchedPackage from a python-apt Version (package). This is an alternative constructor for FetchedPackages that takes most of the information from an apt.package.Version object (i.e. a single version of a package), with some additional information supplied by tha caller. :param pkg: the python-apt package to take the information from. :type pkg: apt.package.Version instance :param filename: the filename that the package has. :type filename: str :param content: the content of the package. :type content: file-like object """ depends = stringify_relationship(pkg, "Depends") pre_depends = stringify_relationship(pkg, "PreDepends") multi_arch = pkg.record.get("Multi-Arch") or None conflicts = stringify_relationship(pkg, "Conflicts") recommends = stringify_relationship(pkg, "Recommends") replaces = stringify_relationship(pkg, "Replaces") breaks = stringify_relationship(pkg, "Breaks") provides = ", ".join([a[0] for a in pkg._cand.provides_list]) or None pkg = cls( pkg.package.name, pkg.version, filename, pkg.size, pkg.md5, pkg.architecture, depends=depends, pre_depends=pre_depends, multi_arch=multi_arch, conflicts=conflicts, recommends=recommends, provides=provides, replaces=replaces, breaks=breaks) if content is not None: pkg.content = content return pkg @classmethod def from_deb(cls, deb_file_path): """Create a FetchedPackage from a binary package on disk.""" debcontrol = DebFile(deb_file_path).control.debcontrol() name = debcontrol['Package'] version = debcontrol['Version'] filename = os.path.basename(deb_file_path) size = os.path.getsize(deb_file_path) md5sum = hashlib.md5(open(deb_file_path).read()).hexdigest() architecture = debcontrol['Architecture'] depends = debcontrol.get('Depends') pre_depends = debcontrol.get('Pre-Depends') multi_arch = debcontrol.get('Multi-Arch') conflicts = debcontrol.get('Conflicts') recommends = debcontrol.get('Recommends') provides = debcontrol.get('Provides') replaces = debcontrol.get('Replaces') breaks = debcontrol.get('Breaks') pkg = cls( name, version, filename, size, md5sum, architecture, depends, pre_depends, multi_arch, conflicts, recommends, provides, replaces, breaks) pkg.content = open(deb_file_path) pkg._file_path = deb_file_path return pkg # A list of attributes that are compared to determine equality. Note that # we don't include the contents here -- we assume that comparing the md5 # checksum is enough (more philosophically, FetchedPackages are equal if # they represent the same underlying package, even if they represent it in # slightly different ways) _equality_attributes = ( 'name', 'version', 'filename', 'size', 'md5', 'architecture', 'depends', 'pre_depends', 'multi_arch', 'conflicts', 'recommends', 'provides', 'replaces', 'breaks') @property def _equality_data(self): return tuple( getattr(self, attr) for attr in self._equality_attributes) def __eq__(self, other): return self._equality_data == other._equality_data def __ne__(self, other): return not self.__eq__(other) def __hash__(self): return hash(self._equality_data) def __repr__(self): has_content = self.content and "yes" or "no" return ( '<%s name=%s version=%s size=%s md5=%s architecture=%s ' 'depends="%s" pre_depends="%s" multi_arch="%s" conflicts="%s" ' 'recommends="%s" provides="%s" replaces="%s" breaks="%s" ' 'has_content=%s>' % (self.__class__.__name__, self.name, self.version, self.size, self.md5, self.architecture, self.depends, self.pre_depends, self.multi_arch, self.conflicts, self.recommends, self.provides, self.replaces, self.breaks, has_content)) class IsolatedAptCache(object): """A apt.cache.Cache wrapper that isolates it from the system it runs on. :ivar cache: the isolated cache. :type cache: apt.cache.Cache """ def __init__(self, sources, architecture=None, prefer_label=None): """Create an IsolatedAptCache. :param sources: a list of sources such that they can be prefixed with "deb " and fed to apt. :type sources: an iterable of str :param architecture: the architecture to fetch packages for. :type architecture: str """ self.sources = sources self.architecture = architecture self.tempdir = None self.prefer_label = prefer_label def prepare(self): """Prepare the IsolatedAptCache for use. Should be called before use, and after any modification to the list of sources. """ self.cleanup() logger.debug("Writing apt configs") self.tempdir = tempfile.mkdtemp(prefix="hwpack-apt-cache-") dirs = ["var/lib/dpkg", "etc/apt/sources.list.d", "var/cache/apt/archives/partial", "var/lib/apt/lists/partial", ] for d in dirs: os.makedirs(os.path.join(self.tempdir, d)) self.set_installed_packages([], reopen=False) sources_list = os.path.join( self.tempdir, "etc", "apt", "sources.list") with open(sources_list, 'w') as f: for source in self.sources: # To make a file URL look like an HTTP one (for urlparse) # We do this to use urlparse, which is probably more robust # than any regexp we come up with. mangled_source = source if re.search("file:/[^/]", source): mangled_source = re.sub("file:/", "file://", source) url_parsed = urlparse.urlsplit(mangled_source) # If the source uses authentication, don't put in sources.list if url_parsed.password: url_parts_without_user_pass = [url_parsed.scheme, url_parsed.hostname, url_parsed.path, url_parsed.query, url_parsed.fragment] auth_name = os.path.join( self.tempdir, "etc", "apt", "auth.conf") with open(auth_name, 'w') as auth: auth.write( "machine " + url_parsed.hostname + "\n" + "login " + url_parsed.username + "\n" + "password " + url_parsed.password + "\n") source = urlparse.urlunsplit(url_parts_without_user_pass) # Get rid of extra / in file URLs source = re.sub("file://", "file:/", source) f.write("deb %s\n" % source) if self.architecture is not None: apt_conf = os.path.join(self.tempdir, "etc", "apt", "apt.conf") with open(apt_conf, 'w') as f: f.write( 'Apt {\nArchitecture "%s";\n' 'Install-Recommends "true";\n}\n' % self.architecture) if self.prefer_label is not None: apt_preferences = os.path.join( self.tempdir, "etc", "apt", "preferences") with open(apt_preferences, 'w') as f: f.write( 'Package: *\n' 'Pin: release l=%s\n' 'Pin-Priority: 1001\n' % self.prefer_label) # XXX: This is a temporary workaround for bug 885895. apt_pkg.config.set("Dir::bin::dpkg", "/bin/false") self.cache = Cache(rootdir=self.tempdir, memonly=True) logger.debug("Updating apt cache") try: self.cache.update() except FetchFailedException, e: obfuscated_e = re.sub(r"([^ ]https://).+?(@)", r"\1***\2", str(e)) raise FetchFailedException(obfuscated_e) self.cache.open() return self def set_installed_packages(self, packages, reopen=True): """Set a list of packages as those installed on the system. This does no installing, just changes dpkg's database to have the tools think the packages are installed. :param packages: a list of packages to "install" on the system, replacing any others. :type packages: an iterable of FetchedPackages. :param reopen: whether to reopen the apt cache after doing the operation. Default is to do so. Note that if it is not done, then the changes will not be visible in the cache until it is reopened. """ with open( os.path.join(self.tempdir, "var/lib/dpkg/status"), "w") as f: f.write( get_packages_file( packages, extra_text="Status: install ok installed")) if reopen: self.cache.open() __enter__ = prepare def cleanup(self): """Cleanup any remaining artefacts. Should be called on all IsolatedAptCache when they are finished with. """ if self.tempdir is not None and os.path.exists(self.tempdir): shutil.rmtree(self.tempdir) def __exit__(self, exc_type, exc_value, traceback): self.cleanup() return False class DependencyNotSatisfied(Exception): pass class PackageFetcher(object): """A class to fetch packages from a defined list of sources.""" def __init__(self, sources, architecture=None, prefer_label=None): """Create a PackageFetcher. Once created a PackageFetcher should have its `prepare` method called before use. :param sources: a list of sources such that they can be prefixed with "deb " and fed to apt. :type sources: an iterable of str :param architecture: the architecture to fetch packages for. :type architecture: str """ self.cache = IsolatedAptCache( sources, architecture=architecture, prefer_label=prefer_label) def prepare(self): """Prepare the PackageFetcher for use. Should be called before use. """ self.cache.prepare() return self __enter__ = prepare def cleanup(self): """Cleanup any remaining artefacts. Should be called on all PackageFetchers when they are finished with. """ self.cache.cleanup() def __exit__(self, exc_type, exc_value, traceback): self.cleanup() return False def ignore_packages(self, packages): """Ignore packages such that they will not be fetched. If a package is ignored then neither it or any of its recursive dependencies will be fetched by fetch_packages. :param packages: the list of package names to ignore. :type packages: an iterable of str """ logger.debug("Ignoring %s" % packages) for package in packages: self.cache.cache[package].mark_install(auto_fix=False) if self.cache.cache.broken_count: raise DependencyNotSatisfied( "Unable to satisfy dependencies of %s" % ", ".join([p.name for p in self.cache.cache if p.is_inst_broken])) installed = [] for package in self.cache.cache.get_changes(): candidate = package.candidate base = os.path.basename(candidate.filename) installed.append(FetchedPackage.from_apt(candidate, base)) for package in self.cache.cache: if not package.is_installed: continue candidate = package.installed base = os.path.basename(candidate.filename) installed.append(FetchedPackage.from_apt(candidate, base)) logger.debug("Ignored %s" % package.name) self.cache.set_installed_packages(installed) broken = [p.name for p in self.cache.cache if p.is_inst_broken or p.is_now_broken] if broken: # If this happens then there is a bug, as we should have # caught this problem earlier raise AssertionError( "Weirdly unable to satisfy dependencies of %s" % ", ".join(broken)) def _filter_ignored(self, package_dict): seen_packages = set() for package in self.cache.cache.get_changes(): if package.name in package_dict: seen_packages.add(package.name) all_packages = set(package_dict.keys()) for unseen_package in all_packages.difference(seen_packages): logger.debug("%s is ignored, skipping" % unseen_package) del package_dict[unseen_package] def fetch_packages(self, packages, download_content=True): """Fetch the files for the given list of package names. The files, and all their dependencies are download, and the metadata and content returned as FetchedPackage objects. If download_content is False then only the metadata is returned (i.e. the FetchedPackages will have None for their content attribute), and only information about the specified packages will be returned, no dependencies. No packages that have been ignored, or are recursive dependencies of ignored packages will be returned. :param packages: a list of package names to install :type packages: an iterable of str :param download_content: whether to download the content of the packages. Default is to do so. :type download_content: bool :return: a list of the packages that were fetched, with relevant metdata and the contents of the files available. :rtype: an iterable of FetchedPackages. :raises KeyError: if any of the package names in the list couldn't be found. """ fetched = {} for package in packages: candidate = self.cache.cache[package].candidate base = os.path.basename(candidate.filename) result_package = FetchedPackage.from_apt(candidate, base) fetched[package] = result_package def check_no_broken_packages(): if self.cache.cache.broken_count: raise DependencyNotSatisfied( "Unable to satisfy dependencies of %s" % ", ".join([p.name for p in self.cache.cache if p.is_inst_broken])) for package in packages: try: self.cache.cache[package].mark_install(auto_fix=True) except SystemError: # Either we raise a DependencyNotSatisfied error # if some packages are broken, or we raise the original # error if there was another cause check_no_broken_packages() raise # Check that nothing was broken, even if mark_install didn't # raise SystemError, just to make sure. check_no_broken_packages() self._filter_ignored(fetched) if not download_content: self.cache.cache.clear() return fetched.values() acq = apt_pkg.Acquire(DummyProgress()) acqfiles = [] # re to remove the repo private key deb_url_auth_re = re.compile( r"(?P.*://)(?P.*):.*@(?P.*$)") for package in self.cache.cache.get_changes(): if (package.marked_delete or package.marked_keep): continue logger.debug("Fetching %s ..." % package) candidate = package.candidate base = os.path.basename(candidate.filename) if package.name not in fetched: result_package = FetchedPackage.from_apt(candidate, base) fetched[package.name] = result_package result_package = fetched[package.name] destfile = os.path.join(self.cache.tempdir, base) acqfile = apt_pkg.AcquireFile( acq, candidate.uri, candidate.md5, candidate.size, base, destfile=destfile) acqfiles.append((acqfile, result_package, destfile)) # check if we have a private key in the pkg url deb_url_auth = deb_url_auth_re.match(acqfile.desc_uri) if deb_url_auth: logger.debug(" ... from %s%s:***@%s" % deb_url_auth.groups()) else: logger.debug(" ... from %s" % acqfile.desc_uri) self.cache.cache.clear() acq.run() for acqfile, result_package, destfile in acqfiles: if acqfile.status != acqfile.STAT_DONE: raise FetchError( "The item %r could not be fetched: %s" % (acqfile.destfile, acqfile.error_text)) result_package.content = open(destfile) result_package._file_path = destfile return fetched.values() linaro-image-tools-2014.11/linaro_image_tools/hwpack/handler.py0000664000175000017500000002675712400111666025604 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011, 2012 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . from StringIO import StringIO import ConfigParser import logging import os import re import shutil import tarfile import tempfile from linaro_image_tools.hwpack.config import Config from linaro_image_tools.hwpack.package_unpacker import PackageUnpacker from linaro_image_tools.utils import DEFAULT_LOGGER_NAME logger = logging.getLogger(DEFAULT_LOGGER_NAME) class HardwarepackHandler(object): FORMAT_1 = '1.0' FORMAT_2 = '2.0' FORMAT_3 = '3.0' FORMAT_MIXED = '1.0and2.0' metadata_filename = 'metadata' format_filename = 'FORMAT' main_section = 'main' hwpack_tarfiles = [] tempdir = None def __init__(self, hwpacks, bootloader=None, board=None): self.hwpacks = hwpacks self.hwpack_tarfiles = [] self.bootloader = bootloader self.board = board self.tempdirs = {} # Used to store the config created from the metadata. self.config = None class FakeSecHead(object): """ Add a fake section header to the metadata file. This is done so we can use ConfigParser to parse the file. """ def __init__(self, fp): self.fp = fp self.sechead = '[%s]\n' % HardwarepackHandler.main_section def readline(self): if self.sechead: try: return self.sechead finally: self.sechead = None else: return self.fp.readline() def __enter__(self): self.tempdir = tempfile.mkdtemp() for hwpack in self.hwpacks: hwpack_tarfile = tarfile.open(hwpack, mode='r:gz') self.hwpack_tarfiles.append(hwpack_tarfile) return self def __exit__(self, type, value, traceback): for hwpack_tarfile in self.hwpack_tarfiles: if hwpack_tarfile is not None: hwpack_tarfile.close() self.hwpack_tarfiles = [] if self.tempdir is not None and os.path.exists(self.tempdir): shutil.rmtree(self.tempdir) for name in self.tempdirs: tempdir = self.tempdirs[name] if tempdir is not None and os.path.exists(tempdir): shutil.rmtree(tempdir) def _get_config_from_metadata(self, metadata): """ Retrieves a Config object associated with the metadata. :param metadata: The metadata to parse. :return: A Config instance. """ if not self.config: lines = metadata.readlines() if re.search("=", lines[0]) and not re.search(":", lines[0]): # Probably V2 hardware pack without [hwpack] on the first line lines = ["[hwpack]\n"] + lines self.config = Config(StringIO("".join(lines))) self.config.board = self.board self.config.bootloader = self.bootloader return self.config def get_field(self, field, return_keys=False): data = None hwpack_with_data = None keys = None for hwpack_tarfile in self.hwpack_tarfiles: metadata = hwpack_tarfile.extractfile(self.metadata_filename) parser = self._get_config_from_metadata(metadata) try: new_data = parser.get_option(field) if new_data is not None: assert data is None, "The metadata field '%s' is set to " \ "'%s' and new value '%s' is found" % (field, data, new_data) data = new_data hwpack_with_data = hwpack_tarfile if return_keys: keys = parser.get_last_used_keys() except ConfigParser.NoOptionError: continue if return_keys: return data, hwpack_with_data, keys return data, hwpack_with_data def get_format(self): format = None supported_formats = [self.FORMAT_1, self.FORMAT_2, self.FORMAT_3] for hwpack_tarfile in self.hwpack_tarfiles: format_file = hwpack_tarfile.extractfile(self.format_filename) format_string = format_file.read().strip() if not format_string in supported_formats: raise AssertionError( "Format version '%s' is not supported." % format_string) if format is None: format = format_string elif format != format_string: return self.FORMAT_MIXED return format def get_file(self, file_alias): """Get file(s) from a hwpack. :param file_alias: Property name (not field name) which contains file reference(s) :return: path to a file or list of paths to files """ file_names, hwpack_tarfile, keys = self.get_field(file_alias, return_keys=True) if not file_names: return file_names single = False if not isinstance(file_names, list): single = True file_names = [file_names] out_files = [] # Depending on if board and/or bootloader were used to look up the # file we are getting, we need to prepend those names to the path # to get the correct extracted file from the hardware pack. config_names = [("board", "boards"), ("bootloader", "bootloaders")] base_path = "" if keys: # If keys is non-empty, we have a V3 config option that was # modified by the bootloader and/or boot option... for name, key in config_names: if self.get_field(name): value = self.get_field(name)[0] if keys[0] == key: base_path = os.path.join(base_path, value) keys = keys[1:] for f in file_names: # Check that the base path is needed. If the file doesn't exist, # try without it (this provides fallback to V2 style directory # layouts with a V3 config). path_inc_board_and_bootloader = os.path.join(base_path, f) if path_inc_board_and_bootloader in hwpack_tarfile.getnames(): f = path_inc_board_and_bootloader hwpack_tarfile.extract(f, self.tempdir) f = os.path.join(self.tempdir, f) out_files.append(f) if single: return out_files[0] return out_files def list_packages(self): """Return list of (package names, TarFile object containing them)""" packages = [] for tf in self.hwpack_tarfiles: for name in tf.getnames(): if name.startswith("pkgs/") and name.endswith(".deb"): packages.append((tf, name)) return packages def find_package_for(self, name, version=None, revision=None, architecture=None): """Find a package that matches the name, version, rev and arch given. Packages are named according to the debian specification: http://www.debian.org/doc/manuals/debian-faq/ch-pkg_basics.en.html _-_.deb DebianRevisionNumber seems to be optional. Use this spec to return a package matching the requirements given. """ for tar_file, package in self.list_packages(): file_name = os.path.basename(package) dpkg_chunks = re.search("^(.+)_(.+)_(.+)\.deb$", file_name) assert dpkg_chunks, "Could not split package file name into"\ "__.deb" pkg_name = dpkg_chunks.group(1) pkg_version = dpkg_chunks.group(2) pkg_architecture = dpkg_chunks.group(3) ver_chunks = re.search("^(.+)-(.+)$", pkg_version) if ver_chunks: pkg_version = ver_chunks.group(1) pkg_revision = ver_chunks.group(2) else: pkg_revision = None if name != pkg_name: continue if version is not None and str(version) != pkg_version: continue if revision is not None and str(revision) != pkg_revision: continue if (architecture is not None and str(architecture) != pkg_architecture): continue # Got a matching package - return its path inside the tarball return tar_file, package # Failed to find a matching package - return None return None def get_file_from_package(self, file_path, package_name, package_version=None, package_revision=None, package_architecture=None): """Extract named file from package specified by name, ver, rev, arch. File is extracted from the package matching the given specification to a temporary directory. The absolute path to the extracted file is returned. """ package_info = self.find_package_for(package_name, package_version, package_revision, package_architecture) if package_info is None: return None tar_file, package = package_info # Avoid unpacking hardware pack more than once by assigning each one # its own tempdir to unpack into. # TODO: update logic that uses self.tempdir so we can get rid of this # by sharing nicely. if not package in self.tempdirs: self.tempdirs[package] = tempfile.mkdtemp() tempdir = self.tempdirs[package] # We extract everything in the hardware pack so we don't have to worry # about chasing links (extract a link, find where it points to, extract # that...). This is slower, but more reliable. tar_file.extractall(tempdir) package_path = os.path.join(tempdir, package) with PackageUnpacker() as self.package_unpacker: extracted_file = self.package_unpacker.get_file(package_path, file_path) after_tmp = re.sub(self.package_unpacker.tempdir, "", extracted_file).lstrip("/\\") extract_dir = os.path.join(tempdir, "extracted", os.path.dirname(after_tmp)) os.makedirs(extract_dir) shutil.move(extracted_file, extract_dir) extracted_file = os.path.join(extract_dir, os.path.basename(extracted_file)) return extracted_file linaro-image-tools-2014.11/linaro_image_tools/hwpack/testing.py0000664000175000017500000005501412400111666025630 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from contextlib import contextmanager import hashlib import logging import os import shutil import subprocess import tempfile from StringIO import StringIO import tarfile import time from debian.deb822 import Packages from testtools.matchers import Annotate, Equals, Matcher, Mismatch from linaro_image_tools.hwpack.better_tarfile import writeable_tarfile from linaro_image_tools.hwpack.tarfile_matchers import TarfileHasFile from linaro_image_tools.hwpack.packages import ( get_packages_file, FetchedPackage, ) @contextmanager def test_tarfile(contents=[], **kwargs): """Create a tarfile with the given contents, then re-open it for reading. This context manager creates a tarfile with the given contents, then reopens it for reading and yields it for use in a with block. When the block ends the tarfile will be closed. The contents can be specified as a list of tuples of (path, contents), where if the path ends with '/' it is considered to be a directory and the contents ignored. :param contents: the contents to put in the tarball, defaults to the empty list. :type contents: a list of tuples of (str, str) :param kwargs: keyword arguments for the better_tarfile.TarFile constructor. """ backing_file = StringIO() with writeable_tarfile(backing_file, **kwargs) as tf: for path, content in contents: if path[-1] == "/": tf.create_dir(path) else: tf.create_file_from_string(path, content) if contents: backing_file.seek(0) tf = tarfile.TarFile.open(mode="r", fileobj=backing_file) try: yield tf finally: tf.close() class DummyFetchedPackage(FetchedPackage): """A FetchedPackage with dummy information. See FetchedPackage for the instance variables. """ def __init__(self, name, version, architecture="all", depends=None, pre_depends=None, multi_arch=None, conflicts=None, recommends=None, provides=None, replaces=None, breaks=None, no_content=False, content=None): self.name = name self.version = version self.architecture = architecture self.depends = depends self.pre_depends = pre_depends self.multi_arch = multi_arch self.conflicts = conflicts self.recommends = recommends self.provides = provides self.replaces = replaces self.breaks = breaks self._no_content = no_content self._content = content self._file_path = None @property def filename(self): return "%s_%s_all.deb" % (self.name, self.version) def _content_str(self): return "Content of %s" % self.filename @property def content(self): if self._no_content: return None elif self._content is not None: return StringIO(self._content) return StringIO(self._content_str()) @property def size(self): return len(self.content.read()) @property def md5(self): md5sum = hashlib.md5() md5sum.update(self.content.read()) return md5sum.hexdigest() class AptSourceFixture(object): """A fixture that provides an apt source, with packages and indices. An apt source provides a set of package files, and a Packages file that allows apt to determine the contents of the source. :ivar sources_entry: the URI and suite to give to apt to view the source (i.e. a sources.list line without the "deb" prefix :type sources_entry: str """ def __init__(self, packages, label=None): """Create an AptSourceFixture. :param packages: a list of packages to add to the source and index. :type packages: an iterable of FetchedPackages """ self.packages = packages self.label = label def setUp(self): self.rootdir = tempfile.mkdtemp(prefix="hwpack-apt-source-") for package in self.packages: with open(os.path.join(self.rootdir, package.filename), 'wb') as f: f.write(package.content.read()) with open(os.path.join(self.rootdir, "Packages"), 'wb') as f: f.write(get_packages_file(self.packages)) if self.label is not None: subprocess.check_call( ['apt-ftparchive', '-oAPT::FTPArchive::Release::Label=%s' % self.label, 'release', self.rootdir], stdout=open(os.path.join(self.rootdir, 'Release'), 'w')) def tearDown(self): if os.path.exists(self.rootdir): shutil.rmtree(self.rootdir) @property def sources_entry(self): return "file:" + os.path.abspath(self.rootdir) + " ./" class ConfigFileFixture(object): def __init__(self, contents): self.contents = contents self.filename = None def setUp(self): fh, self.filename = tempfile.mkstemp(prefix="hwpack-test-config-") with os.fdopen(fh, 'w') as f: f.write(self.contents) def tearDown(self): if self.filename is not None and os.path.exists(self.filename): os.unlink(self.filename) class ContextManagerFixture(object): """Adapt a context manager to be a usable with `useFixture`.""" def __init__(self, context_manager): self.manager = context_manager def setUp(self): self.manager.__enter__() def tearDown(self): # It might be nice to pass exc_type, exc_value, traceback in here in # the failure case, if that's possible. self.manager.__exit__(None, None, None) class ChdirToTempdirFixture(object): def __init__(self): self._orig_dir = None self.tempdir = None def setUp(self): self.tearDown() self._orig_dir = os.getcwd() self.tempdir = tempfile.mkdtemp(prefix="hwpack-tests-") os.chdir(self.tempdir) def tearDown(self): if self._orig_dir is not None: os.chdir(self._orig_dir) self._orig_dir = None if self.tempdir is not None and os.path.exists(self.tempdir): shutil.rmtree(self.tempdir) self.tempdir = None class MismatchesAll(Mismatch): """A mismatch with many child mismatches.""" def __init__(self, mismatches): self.mismatches = mismatches def describe(self): descriptions = ["Differences: ["] for mismatch in self.mismatches: descriptions.append(mismatch.describe()) descriptions.append("]\n") return '\n'.join(descriptions) class MatchesAll(object): def __init__(self, *matchers): self.matchers = matchers def __str__(self): return 'MatchesAll(%s)' % ', '.join(map(str, self.matchers)) def match(self, matchee): results = [] for matcher in self.matchers: mismatch = matcher.match(matchee) if mismatch is not None: results.append(mismatch) if results: return MismatchesAll(results) else: return None class Not: """Inverts a matcher.""" def __init__(self, matcher): self.matcher = matcher def __str__(self): return 'Not(%s)' % (self.matcher,) def match(self, other): mismatch = self.matcher.match(other) if mismatch is None: return MatchedUnexpectedly(self.matcher, other) else: return None class MatchedUnexpectedly: """A thing matched when it wasn't supposed to.""" def __init__(self, matcher, other): self.matcher = matcher self.other = other def describe(self): return "%r matches %s" % (self.other, self.matcher) class HardwarePackHasFile(TarfileHasFile): """A subclass of TarfileHasFile specific to hardware packs. We default to a set of attributes expected for files in a hardware pack. """ def __init__(self, path, **kwargs): """Create a HardwarePackHasFile matcher. The kwargs are the keyword arguments taken by TarfileHasFile. If they are not given then defaults will be checked: - The type should be a regular file - If the content is given then the size will be checked to ensure it indicates the length of the content correctly. - the mode is appropriate for the type. If the type is regular file this is 0644, otherwise if it is a directory then it is 0755. - the linkname should be the empty string. - the uid and gid should be 1000 - the uname and gname should be "user" and "group" respectively. :param path: the path that should be present. :type path: str """ kwargs.setdefault("type", tarfile.REGTYPE) if "content" in kwargs and kwargs['type'] != tarfile.DIRTYPE: kwargs.setdefault("size", len(kwargs["content"])) if kwargs["type"] == tarfile.DIRTYPE: kwargs.setdefault("mode", 0755) else: kwargs.setdefault("mode", 0644) kwargs.setdefault("linkname", "") kwargs.setdefault("uid", 1000) kwargs.setdefault("gid", 1000) kwargs.setdefault("uname", "user") kwargs.setdefault("gname", "group") kwargs.setdefault("mtime", time.time()) # Enough that it won't fail if the test is slow to execute, but # not enough that we can have a wildly wrong timestamp. kwargs.setdefault("mtime_skew", 100) super(HardwarePackHasFile, self).__init__(path, **kwargs) class IsHardwarePack(Matcher): def __init__(self, metadata, packages, sources, packages_without_content=None, package_spec=None, format="1.0"): self.metadata = metadata self.packages = packages self.sources = sources self.packages_without_content = packages_without_content or [] self.package_spec = package_spec self.format = format + "\n" def match(self, path): with tarfile.open(name=path, mode="r:gz") as tf: matchers = [] matchers.append(HardwarePackHasFile("FORMAT", content=self.format)) matchers.append(HardwarePackHasFile( "metadata", content=str(self.metadata))) manifest_lines = [] for package in self.packages: manifest_lines.append( "%s=%s" % (package.name, package.version)) manifest_lines.append( "%s=%s" % ( 'hwpack-' + self.metadata.name, self.metadata.version)) matchers.append( HardwarePackHasFile( "manifest", content_matcher=AfterPreproccessing( str.splitlines, MatchesSetwise(*map(Equals, manifest_lines))))) matchers.append(HardwarePackHasFile("pkgs", type=tarfile.DIRTYPE)) packages_with_content = [p for p in self.packages if p not in self.packages_without_content] for package in packages_with_content: matchers.append(HardwarePackHasFile( "pkgs/%s" % package.filename, content=package.content.read())) package_matchers = [ MatchesPackage(p) for p in packages_with_content] dep_package_matcher = MatchesStructure( name=Equals('hwpack-' + self.metadata.name), version=Equals(self.metadata.version), architecture=Equals(self.metadata.architecture), filename=Equals('hwpack-%s_%s_%s.deb' % ( self.metadata.name, self.metadata.version, self.metadata.architecture))) if self.package_spec: dep_package_matcher = dep_package_matcher.update( depends=MatchesPackageRelationshipList( [Equals(p.strip()) for p in self.package_spec.split(',')])) package_matchers.append(dep_package_matcher) matchers.append(HardwarePackHasFile( "pkgs/Packages", content_matcher=MatchesAsPackagesFile( *package_matchers))) matchers.append(HardwarePackHasFile( "sources.list.d", type=tarfile.DIRTYPE)) for source_id, sources_entry in self.sources.items(): matchers.append(HardwarePackHasFile( "sources.list.d/%s.list" % source_id, content="deb " + sources_entry + "\n")) matchers.append(HardwarePackHasFile( "sources.list.d.gpg", type=tarfile.DIRTYPE)) return MatchesAll(*matchers).match(tf) def __str__(self): return "Is a valid hardware pack." class EachOf(object): """Matches if each matcher matches the corresponding value. More easily explained by example than in words: >>> EachOf([Equals(1)]).match([1]) >>> EachOf([Equals(1), Equals(2)]).match([1, 2]) >>> EachOf([Equals(1), Equals(2)]).match([2, 1]) #doctest: +ELLIPSIS <...Mismatch...> """ def __init__(self, matchers): self.matchers = matchers def match(self, values): mismatches = [] length_mismatch = Annotate( "Length mismatch", Equals(len(self.matchers))).match(len(values)) if length_mismatch: mismatches.append(length_mismatch) for matcher, value in zip(self.matchers, values): mismatch = matcher.match(value) if mismatch: mismatches.append(mismatch) if mismatches: return MismatchesAll(mismatches) class MatchesStructure(object): """Matcher that matches an object structurally. 'Structurally' here means that attributes of the object being matched are compared against given matchers. `fromExample` allows the creation of a matcher from a prototype object and then modified versions can be created with `update`. """ def __init__(self, **kwargs): self.kws = kwargs @classmethod def fromExample(cls, example, *attributes): kwargs = {} for attr in attributes: kwargs[attr] = Equals(getattr(example, attr)) return cls(**kwargs) def update(self, **kws): new_kws = self.kws.copy() for attr, matcher in kws.iteritems(): if matcher is None: new_kws.pop(attr, None) else: new_kws[attr] = matcher return type(self)(**new_kws) def match(self, value): matchers = [] values = [] for attr, matcher in self.kws.iteritems(): matchers.append(Annotate(attr, matcher)) values.append(getattr(value, attr)) return EachOf(matchers).match(values) def MatchesPackage(example): """Create a `MatchesStructure` object from a `FetchedPackage`.""" return MatchesStructure.fromExample( example, *example._equality_attributes) class MatchesSetwise(object): """Matches if all the matchers match elements of the value being matched. The difference compared to `EachOf` is that the order of the matchings does not matter. """ def __init__(self, *matchers): self.matchers = matchers def match(self, observed): remaining_matchers = set(self.matchers) not_matched = [] for value in observed: for matcher in remaining_matchers: if matcher.match(value) is None: remaining_matchers.remove(matcher) break else: not_matched.append(value) if not_matched or remaining_matchers: remaining_matchers = list(remaining_matchers) # There are various cases that all should be reported somewhat # differently. # There are two trivial cases: # 1) There are just some matchers left over. # 2) There are just some values left over. # Then there are three more interesting cases: # 3) There are the same number of matchers and values left over. # 4) There are more matchers left over than values. # 5) There are more values left over than matchers. if len(not_matched) == 0: if len(remaining_matchers) > 1: msg = "There were %s matchers left over: " % ( len(remaining_matchers),) else: msg = "There was 1 matcher left over: " msg += ', '.join(map(str, remaining_matchers)) return Mismatch(msg) elif len(remaining_matchers) == 0: if len(not_matched) > 1: return Mismatch( "There were %s values left over: %s" % ( len(not_matched), not_matched)) else: return Mismatch( "There was 1 value left over: %s" % ( not_matched, )) else: common_length = min(len(remaining_matchers), len(not_matched)) if common_length == 0: raise AssertionError("common_length can't be 0 here") if common_length > 1: msg = "There were %s mismatches" % (common_length,) else: msg = "There was 1 mismatch" if len(remaining_matchers) > len(not_matched): extra_matchers = remaining_matchers[common_length:] msg += " and %s extra matcher" % (len(extra_matchers), ) if len(extra_matchers) > 1: msg += "s" msg += ': ' + ', '.join(map(str, extra_matchers)) elif len(not_matched) > len(remaining_matchers): extra_values = not_matched[common_length:] msg += " and %s extra value" % (len(extra_values), ) if len(extra_values) > 1: msg += "s" msg += ': ' + str(extra_values) return Annotate( msg, EachOf(remaining_matchers[:common_length]) ).match(not_matched[:common_length]) def parse_packages_file_content(file_content): packages = [] for para in Packages.iter_paragraphs(StringIO(file_content)): args = {} for key, value in para.iteritems(): key = key.lower() if key == 'md5sum': key = 'md5' elif key == 'package': key = 'name' elif key == 'size': value = int(value) if key in FetchedPackage._equality_attributes: args[key] = value packages.append(FetchedPackage(**args)) return packages class AfterPreproccessing(object): """Matches if the value matches after passing through a function.""" def __init__(self, preprocessor, matcher): self.preprocessor = preprocessor self.matcher = matcher def __str__(self): return "AfterPreproccessing(%s, %s)" % ( self.preprocessor, self.matcher) def match(self, value): value = self.preprocessor(value) return self.matcher.match(value) def MatchesAsPackagesFile(*package_matchers): """Matches the contents of a Packages file against the given matchers. The contents of the Packages file is turned into a list of FetchedPackages using `parse_packages_file_content` above. """ return AfterPreproccessing( parse_packages_file_content, MatchesSetwise(*package_matchers)) def MatchesAsPackageContent(package_matcher): """Match a package on disk against `package_matcher`.""" def load_from_disk(content): fd, path = tempfile.mkstemp() try: os.write(fd, content) os.close(fd) return FetchedPackage.from_deb(path) finally: os.remove(path) return AfterPreproccessing(load_from_disk, package_matcher) class DoesNotStartWith(Mismatch): def __init__(self, matchee, expected): """Create a DoesNotStartWith Mismatch. :param matchee: the string that did not match. :param expected: the string that `matchee` was expected to start with. """ self.matchee = matchee self.expected = expected def describe(self): return "'%s' does not start with '%s'." % ( self.matchee, self.expected) class StartsWith(Matcher): """Checks whether one string starts with another.""" def __init__(self, expected): """Create a StartsWith Matcher. :param expected: the string that matchees should start with. """ self.expected = expected def __str__(self): return "Starts with '%s'." % self.expected def match(self, matchee): if not matchee.startswith(self.expected): return DoesNotStartWith(matchee, self.expected) return None def MatchesPackageRelationshipList(relationship_matchers): """Matches a set of matchers against a package relationship specification. >>> MatchesPackageRelationshipList( ... [Equals('foo'), StartsWith('bar (')]).match('bar (= 1.0), foo') >>> """ def process(relationships): if relationships is None: return [] return [rel.strip() for rel in relationships.split(',')] return AfterPreproccessing( process, MatchesSetwise(*relationship_matchers)) class AppendingHandler(logging.Handler): """A logging handler that simply appends messages to a list.""" def __init__(self): logging.Handler.__init__(self) self.messages = [] def emit(self, message): self.messages.append(message) linaro-image-tools-2014.11/linaro_image_tools/hwpack/builder.py0000664000175000017500000004156412400111666025606 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import logging import errno import subprocess import os import shutil from glob import iglob from debian.debfile import DebFile from debian.arfile import ArError from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.config import Config from linaro_image_tools.hwpack.hardwarepack import HardwarePack, Metadata from linaro_image_tools.hwpack.packages import ( FetchedPackage, LocalArchiveMaker, PackageFetcher, ) from linaro_image_tools.hwpack.package_unpacker import PackageUnpacker from linaro_image_tools.hwpack.hwpack_fields import ( PACKAGE_FIELD, SPL_PACKAGE_FIELD, ) # The fields that hold packages to be installed. PACKAGE_FIELDS = [PACKAGE_FIELD, SPL_PACKAGE_FIELD] logger = logging.getLogger(__name__) LOCAL_ARCHIVE_LABEL = 'hwpack-local' class ConfigFileMissing(Exception): def __init__(self, filename): self.filename = filename super(ConfigFileMissing, self).__init__( "No such config file: '%s'" % self.filename) class HardwarePackBuilder(object): def __init__(self, config_path, version, local_debs, out_name=None): try: with open(config_path) as fp: self.config = Config(fp, allow_unset_bootloader=True) except IOError, e: if e.errno == errno.ENOENT: raise ConfigFileMissing(config_path) raise self.config.validate() self.format = self.config.format self.version = version self.local_debs = local_debs self.package_unpacker = None self.hwpack = None self.packages = None self.packages_added_to_hwpack = [] self.out_name = out_name def find_fetched_package(self, packages, wanted_package_name): wanted_package = None for package in packages: if package.name == wanted_package_name: wanted_package = package break else: raise AssertionError("Package '%s' was not fetched." % wanted_package_name) return wanted_package def add_file_to_hwpack(self, package, wanted_file, target_path): if (package.name, wanted_file) in self.packages_added_to_hwpack: # Don't bother adding the same package more than once. return tempfile_name = self.package_unpacker.get_file( package.filepath, wanted_file) self.packages_added_to_hwpack.append((package.name, target_path)) return self.hwpack.add_file(target_path, tempfile_name) def find_bootloader_packages(self, bootloaders_config): """Loop through the bootloaders dictionary searching for packages that should be installed, based on known keywords. :param bootloaders_config: The bootloaders dictionary to loop through. :return A list of packages, without duplicates.""" boot_packages = [] for key, value in bootloaders_config.iteritems(): if isinstance(value, dict): boot_packages.extend(self.find_bootloader_packages(value)) else: if key in PACKAGE_FIELDS: boot_packages.append(value) # Eliminate duplicates. return list(set(boot_packages)) def do_extract_file(self, package, source_path, dest_path): """Extract specified file from package to dest_path.""" package_ref = self.find_fetched_package(self.packages, package) return self.add_file_to_hwpack(package_ref, source_path, dest_path) def do_extract_files(self): """Go through a bootloader config, search for files to extract.""" base_dest_path = "" if self.config.board: base_dest_path = self.config.board base_dest_path = os.path.join(base_dest_path, self.config.bootloader) # Extract bootloader file if self.config.bootloader_package and self.config.bootloader_file: dest_path = os.path.join( base_dest_path, os.path.dirname(self.config.bootloader_file)) self.do_extract_file(self.config.bootloader_package, self.config.bootloader_file, dest_path) # Extract SPL file if self.config.spl_package and self.config.spl_file: dest_path = os.path.join(base_dest_path, os.path.dirname(self.config.spl_file)) self.do_extract_file(self.config.spl_package, self.config.spl_file, dest_path) def foreach_boards_and_bootloaders(self, function): """Call function for each board + bootloader combination in metadata""" if self.config.bootloaders is not None: for bootloader in self.config.bootloaders: self.config.board = None self.config.bootloader = bootloader function() if self.config.boards is not None: for board in self.config.boards: if self.config.bootloaders is not None: for bootloader in self.config.bootloaders: self.config.board = board self.config.bootloader = bootloader function() def extract_files(self): """Find bootloaders in config that may contain files to extract.""" if float(self.config.format.format_as_string) < 3.0: # extract files was introduced in version 3 configurations and is # a null operation for earlier configuration files return self.foreach_boards_and_bootloaders(self.do_extract_files) def do_find_copy_files_packages(self): """Find packages referenced by copy_files (single board, bootloader)""" copy_files = self.config.bootloader_copy_files if copy_files: self.copy_files_packages.extend(copy_files.keys()) def find_copy_files_packages(self): """Find all packages referenced by copy_files sections in metadata.""" self.copy_files_packages = [] self.foreach_boards_and_bootloaders( self.do_find_copy_files_packages) packages = self.copy_files_packages del self.copy_files_packages return packages def build(self): for architecture in self.config.architectures: logger.info("Building for %s" % architecture) metadata = Metadata.from_config( self.config, self.version, architecture) self.hwpack = HardwarePack(metadata) sources = self.config.sources with LocalArchiveMaker() as local_archive_maker: self.hwpack.add_apt_sources(sources) if sources: sources = sources.values() else: sources = [] self.packages = self.config.packages[:] # Loop through multiple bootloaders. # In V3 of hwpack configuration, all the bootloaders info and # packages are in the bootloaders section. if self.format.format_as_string == '3.0': if self.config.bootloaders is not None: self.packages.extend(self.find_bootloader_packages( self.config.bootloaders)) if self.config.boards is not None: self.packages.extend(self.find_bootloader_packages( self.config.boards)) self.packages.extend(self.find_copy_files_packages()) else: if self.config.bootloader_package is not None: self.packages.append(self.config.bootloader_package) if self.config.spl_package is not None: self.packages.append(self.config.spl_package) local_packages = [ FetchedPackage.from_deb(deb) for deb in self.local_debs] sources.append( local_archive_maker.sources_entry_for_debs( local_packages, LOCAL_ARCHIVE_LABEL)) self.packages.extend([lp.name for lp in local_packages]) logger.info("Fetching packages") fetcher = PackageFetcher( sources, architecture=architecture, prefer_label=LOCAL_ARCHIVE_LABEL) with fetcher: with PackageUnpacker() as self.package_unpacker: fetcher.ignore_packages(self.config.assume_installed) self.packages = fetcher.fetch_packages( self.packages, download_content=self.config.include_debs) if self.format.format_as_string == '3.0': self.extract_files() else: self._old_format_extract_files() self._add_packages_to_hwpack(local_packages) out_name = self.out_name if not out_name: out_name = self.hwpack.filename() manifest_name = os.path.splitext(out_name)[0] if manifest_name.endswith('.tar'): manifest_name = os.path.splitext(manifest_name)[0] manifest_name += '.manifest.txt' self._write_hwpack_and_manifest(out_name, manifest_name) cache_dir = fetcher.cache.tempdir self._extract_build_info(cache_dir, out_name, manifest_name) def _write_hwpack_and_manifest(self, out_name, manifest_name): """Write the real hwpack file and its manifest file. :param out_name: The name of the file to write. :type out_name: str :param manifest_name: The name of the manifest file. :type manifest_name: str """ logger.debug("Writing hwpack file") with open(out_name, 'w') as f: self.hwpack.to_file(f) logger.info("Wrote %s" % out_name) logger.debug("Writing manifest file content") with open(manifest_name, 'w') as f: f.write(self.hwpack.manifest_text()) def _old_format_extract_files(self): """Extract files for hwpack versions < 3.0.""" bootloader_package = None if self.config.bootloader_file is not None: assert(self.config.bootloader_package is not None) bootloader_package = self.find_fetched_package( self.packages, self.config.bootloader_package) self.hwpack.metadata.u_boot = self.add_file_to_hwpack( bootloader_package, self.config.bootloader_file, self.hwpack.U_BOOT_DIR) spl_package = None if self.config.spl_file is not None: assert self.config.spl_package is not None spl_package = self.find_fetched_package(self.packages, self.config.spl_package) self.hwpack.metadata.spl = self.add_file_to_hwpack( spl_package, self.config.spl_file, self.hwpack.SPL_DIR) # bootloader_package and spl_package can be identical if (bootloader_package is not None and bootloader_package in self.packages): self.packages.remove(bootloader_package) if (spl_package is not None and spl_package in self.packages): self.packages.remove(spl_package) def _add_packages_to_hwpack(self, local_packages): """Adds the packages to the hwpack. :param local_packages: The packages to add. :type local_packages: list """ logger.debug("Adding packages to hwpack") self.hwpack.add_packages(self.packages) for local_package in local_packages: if local_package not in self.packages: logger.warning("Local package '%s' not included", local_package.name) self.hwpack.add_dependency_package(self.config.packages) def _extract_build_info(self, cache_dir, out_name, manifest_name): """Extracts build-info from the packages. :param cache_dir: The cache directory where build-info should be located. :type cache_dir: str :param out_name: The name of the hwpack file. :type out_name: str :param manifest_name: The name of the manifest file. :type manifest_name: str """ logger.debug("Extracting build-info") build_info_dir = os.path.join(cache_dir, 'build-info') build_info_available = 0 for deb_pkg in self.packages: deb_pkg_file_path = deb_pkg.filepath # FIXME: test deb_pkg_dir to work around # https://bugs.launchpad.net/bugs/1067786 deb_pkg_dir = os.path.dirname(deb_pkg_file_path) if deb_pkg_dir != cache_dir: continue if os.path.islink(deb_pkg_file_path): # Skip symlink-ed debian package file # e.g. fetched package with dummy information continue try: # Extract Build-Info attribute from debian control deb_control = DebFile(deb_pkg_file_path).control.debcontrol() build_info = deb_control.get('Build-Info') except ArError: # Skip invalid debian package file # e.g. fetched package with dummy information continue if build_info is not None: build_info_available += 1 # Extract debian packages with build information env = os.environ env['LC_ALL'] = 'C' env['NO_PKG_MANGLE'] = '1' proc = cmd_runner.Popen(['dpkg-deb', '-x', deb_pkg_file_path, build_info_dir], env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdoutdata, stderrdata) = proc.communicate() if proc.returncode: raise ValueError('dpkg-deb extract failed!\n%s' % stderrdata) if stderrdata: raise ValueError('dpkg-deb extract had warnings:\n%s' % stderrdata) self._concatenate_build_info(build_info_available, build_info_dir, out_name, manifest_name) def _concatenate_build_info(self, build_info_available, build_info_dir, out_name, manifest_name): """Concatenates the build-info text if more than one is available. :param build_info_available: The number of available build-info. :type build_info_available: int :param build_info_dir: Where build-info files should be. :type build_info_dir: str :param out_name: The name of the hwpack file. :type out_name: str :param manifest_name: The name of the manifest file. :type manifest_name: str """ logger.debug("Concatenating build-info files") dst_file = open('BUILD-INFO.txt', 'wb') if build_info_available > 0: build_info_path = (r'%s/usr/share/doc/*/BUILD-INFO.txt' % build_info_dir) for src_file in iglob(build_info_path): with open(src_file, 'rb') as f: dst_file.write('\nFiles-Pattern: %s\n' % out_name) shutil.copyfileobj(f, dst_file) dst_file.write('\nFiles-Pattern: %s\nLicense-Type: open\n' % manifest_name) else: dst_file.write('Format-Version: 0.1\n' 'Files-Pattern: %s, %s\n' 'License-Type: open\n' % (out_name, manifest_name)) dst_file.close() linaro-image-tools-2014.11/linaro_image_tools/hwpack/config.py0000664000175000017500000014151412400111666025421 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import ConfigParser from operator import attrgetter import os import re import string import yaml from linaro_image_tools.hwpack.hardwarepack_format import ( HardwarePackFormatV1, HardwarePackFormatV2, HardwarePackFormatV3, ) from hwpack_fields import ( ARCHITECTURES_FIELD, ARCHITECTURE_FIELD, ASSUME_INSTALLED_FIELD, BOARDS_FIELD, BOOTLOADERS_FIELD, BOOT_MIN_SIZE_FIELD, BOOT_SCRIPT_FIELD, COPY_FILES_FIELD, DD_FIELD, DTB_ADDR_FIELD, DTB_FILE_FIELD, DEFAULT_BOOTLOADER, DTB_FILES_FIELD, ENV_DD_FIELD, EXTRA_BOOT_OPTIONS_FIELD, EXTRA_SERIAL_OPTIONS_FIELD, FILE_FIELD, FORMAT_FIELD, INCLUDE_DEBS_FIELD, IN_BOOT_PART_FIELD, INITRD_ADDR_FIELD, INITRD_FILE_FIELD, KERNEL_ADDR_FIELD, KERNEL_FILE_FIELD, LOAD_ADDR_FIELD, LOADER_MIN_SIZE_FIELD, LOADER_START_FIELD, MAINTAINER_FIELD, MMC_ID_FIELD, NAME_FIELD, ORIGIN_FIELD, PACKAGE_FIELD, PACKAGES_FIELD, PARTITION_LAYOUT_FIELD, ROOT_MIN_SIZE_FIELD, SAMSUNG_BL1_LEN_FIELD, SAMSUNG_BL1_START_FIELD, SAMSUNG_BL2_LEN_FIELD, SAMSUNG_BL2_START_FIELD, SAMSUNG_ENV_LEN_FIELD, SAMSUNG_ENV_START_FIELD, SERIAL_TTY_FIELD, SNOWBALL_STARTUP_FILES_CONFIG_FIELD, SOURCES_FIELD, SPL_DD_FIELD, SPL_FILE_FIELD, SPL_IN_BOOT_PART_FIELD, SPL_PACKAGE_FIELD, SUPPORT_FIELD, WIRED_INTERFACES_FIELD, WIRELESS_INTERFACES_FIELD, DEFINED_PARTITION_LAYOUTS, VERSION_FIELD, hwpack_v3_layout, ) import logging class HwpackConfigError(Exception): pass class Config(object): """Encapsulation of a hwpack-create configuration.""" translate_v2_to_v3 = {} translate_v2_metadata = {} MAIN_SECTION = "hwpack" NAME_REGEX = r"[a-z0-9][a-z0-9+\-.]+$" SOURCES_ENTRY_KEY = "sources-entry" PACKAGE_REGEX = NAME_REGEX PATH_REGEX = r"\w[\w+\-./_]+$" GLOB_REGEX = r"[\w+\-./_\*]+$" INCLUDE_DEBS_KEY = "include-debs" translate_v2_to_v3[INCLUDE_DEBS_KEY] = INCLUDE_DEBS_FIELD translate_v2_metadata[ARCHITECTURES_FIELD] = "ARCHITECTURE" ASSUME_INSTALLED_KEY = "assume-installed" translate_v2_to_v3[ASSUME_INSTALLED_KEY] = ASSUME_INSTALLED_FIELD BOOTLOADER_PACKAGE_KEY = "u_boot_package" translate_v2_to_v3[BOOTLOADER_PACKAGE_KEY] = PACKAGE_FIELD BOOTLOADER_FILE_KEY = "u_boot_file" translate_v2_to_v3[BOOTLOADER_FILE_KEY] = FILE_FIELD translate_v2_metadata[BOOTLOADER_FILE_KEY] = "U_BOOT" SPL_FILE_KEY = "spl_file" translate_v2_metadata[SPL_FILE_KEY] = "SPL" BOOTLOADER_IN_BOOT_PART_KEY = 'u_boot_in_boot_part' translate_v2_to_v3[BOOTLOADER_IN_BOOT_PART_KEY] = IN_BOOT_PART_FIELD BOOTLOADER_DD_KEY = 'u_boot_dd' translate_v2_to_v3[BOOTLOADER_DD_KEY] = DD_FIELD last_used_keys = [] board = None def __init__(self, fp, bootloader=None, board=None, allow_unset_bootloader=False): """Create a Config. :param fp: a file-like object containing the configuration. :param allow_unset_bootloader: Bool. If you have more than 1 bootloader in the config object and don't set which one to use, accessing bootloader related parameters will throw an exception. By setting this None will be returned instead. """ # This Config class is used in two places: # 1. Generating hardware packs # 2. Combining a hardware pack with an OS image to create a bootable # disk image. # # In both cases we are providing a file format independant interface # to configuration data to the rest of Linaro Image Tools. # # In case 1 we want all information to be put in the hardware pack and # there is no possibility of picking a booloader (all bootloaders are # put in the hardware pack and one is picked later). In this case we # don't want to trip up other code by throwing an exception when a # bootloader dependant parameter is queried so we return None. In # reality this information isn't used, but sometimes gets queried by # tests. This flag allows us to keep things simple. # # In case 2 we may have multiple bootloaders specified, but only one # can be used by the OS image so we need to pick one. If a bootloader # isn't specified we want to throw an error when a choice would make a # difference to what is returned when querying the object. # # self.allow_unset_bootloader allows for both modes of operation. self.logger = logging.getLogger('linaro_image_tools') self.allow_unset_bootloader = allow_unset_bootloader self.board = board self._bootloader = bootloader obfuscated_e = None obfuscated_yaml_e = "" try: self.parser = ConfigParser.RawConfigParser() self.parser.readfp(fp) except ConfigParser.Error, e: obfuscated_e = re.sub(r"([^ ]https://).+?(@)", r"\1***\2", str(e)) if obfuscated_e: # obfuscated_e being set indicates that something went wrong. # It could be that the input is in fact YAML. Try the YAML # parser. try: fp.seek(0) self.parser = yaml.safe_load(fp) except yaml.YAMLError, e: obfuscated_yaml_e = re.sub(r"([^ ]https://).+?(@)", r"\1***\2", str(e)) else: # If YAML parsed OK, we don't have an error. obfuscated_e = None if obfuscated_e: # If INI parsing from ConfigParser or YAML parsing failed, # print both error messages. msg = ("Failed to parse hardware pack configuration. Tried to " "parse as both INI and YAML. INI parsing error:\n" + obfuscated_e + "\n" + "YAML parser error:\n" + obfuscated_yaml_e) raise ConfigParser.Error(msg) def _get_bootloader(self): """Returns the bootloader associated with this config. If bootloader is None / empty and there is only one bootloader available, use that.""" bootloader = self._bootloader if not bootloader: # Auto-detect bootloader. If there is a single bootloader specified # then use it, else, error. bootloaders = self.bootloaders if isinstance(bootloaders, dict): # We have a list of bootloaders in the expected format bootloaders = bootloaders.keys() bootloader = bootloaders[0] if len(bootloaders) > 1: # We have more than one bootloader, use 'u_boot'. if DEFAULT_BOOTLOADER in bootloaders: bootloader = DEFAULT_BOOTLOADER self.logger.warning('WARNING: no bootloader specified ' 'on the command line. Defaulting ' 'to \'%s\'.' % DEFAULT_BOOTLOADER) self.logger.warning('WARNING: specify another ' 'bootloader if this is not the ' 'correct one to use.') else: self.logger.warning('Default bootloader \'%s\' not ' 'found. Will try to use \'%s\'. ' 'instead.' % (DEFAULT_BOOTLOADER, bootloader)) # bootloader is None: since we are here, set it so we do not # have to go through all the config retrieval again. self._bootloader = bootloader return bootloader def _set_bootloader(self, value): """Set bootloader used to look up configuration in bootloader section. """ self._bootloader = value bootloader = property(_get_bootloader, _set_bootloader) def get_bootloader_list(self): if isinstance(self.bootloaders, dict): # We have a list of bootloaders in the expected format return self.bootloaders.keys() return [] def validate_bootloader_fields(self): self._validate_bootloader_package() self._validate_bootloader_file() self._validate_spl_package() self._validate_spl_file() self._validate_bootloader_file_in_boot_part() self._validate_bootloader_dd() self._validate_spl_in_boot_part() self._validate_spl_dd() self._validate_env_dd() def validate(self): """Check that this configuration follows the schema. :raises HwpackConfigError: if it does not. """ if isinstance(self.parser, ConfigParser.RawConfigParser): if not self.parser.has_section(self.MAIN_SECTION): raise HwpackConfigError("No [%s] section" % self.MAIN_SECTION) self._validate_keys() self._validate_format() self._validate_name() self._validate_include_debs() self._validate_support() self._validate_packages() self._validate_architectures() self._validate_assume_installed() if self.format.has_v2_fields: # Check config for all bootloaders if one isn't specified. if not self.bootloader and self._is_v3: for bootloader in self.get_bootloader_list(): self.bootloader = bootloader self.validate_bootloader_fields() else: self.validate_bootloader_fields() self._validate_serial_tty() self._validate_kernel_addr() self._validate_initrd_addr() self._validate_load_addr() self._validate_dtb_addr() self._validate_wired_interfaces() self._validate_wireless_interfaces() self._validate_partition_layout() self._validate_boot_min_size() self._validate_root_min_size() self._validate_loader_min_size() self._validate_loader_start() self._validate_vmlinuz() self._validate_initrd() self._validate_dtb_file() self._validate_dtb_files() self._validate_mmc_id() self._validate_extra_boot_options() self._validate_boot_script() self._validate_extra_serial_options() self._validate_snowball_startup_files_config() self._validate_samsung_bl1_start() self._validate_samsung_bl1_len() self._validate_samsung_env_start() self._validate_samsung_env_len() self._validate_samsung_bl2_start() self._validate_samsung_bl2_len() self._validate_sources() @property def format(self): """The format of the hardware pack. A subclass of HardwarePackFormat. """ if isinstance(self.parser, ConfigParser.RawConfigParser): try: format_string = self.parser.get(self.MAIN_SECTION, FORMAT_FIELD) except (ConfigParser.NoOptionError, ConfigParser.NoSectionError): # Default to 1.0 to aviod breaking existing hwpack files. # When this code no longer supports 1.0, it effectively makes # explicitly specifying format in hwpack files mandatory. format_string = "1.0" else: format_string = self.parser.get(FORMAT_FIELD) if format_string == '1.0': return HardwarePackFormatV1() elif format_string == '2.0': return HardwarePackFormatV2() elif format_string == 3.0 or format_string == '3.0': return HardwarePackFormatV3() else: raise HwpackConfigError("Format version '%s' is not supported." % format_string) @property def name(self): """The name of the hardware pack. A str.""" return self._get_option(NAME_FIELD) @property def version(self): return self._get_option(VERSION_FIELD) @property def include_debs(self): """Whether the hardware pack should contain .debs. A bool.""" try: if self._get_option(self.INCLUDE_DEBS_KEY) is None: return True try: return self._get_option_bool(self.INCLUDE_DEBS_KEY) except ValueError as e: raise HwpackConfigError("Invalid value for include-debs: %s" % e) except ConfigParser.NoOptionError: return True @property def bootloaders(self): """Bootloaders available in the hardware pack""" return self._get_option(BOOTLOADERS_FIELD) @property def boards(self): """Multiple boards available in the hardware pack.""" return self._get_option(BOARDS_FIELD) @property def bootloader_file_in_boot_part(self): """Whether uboot binary should be put in the boot partition. A str.""" return self._get_bootloader_option(self.BOOTLOADER_IN_BOOT_PART_KEY) @property def bootloader_dd(self): """If the uboot binary should be dd:d to the boot partition this field specifies the offset. An int.""" return self._get_bootloader_option(self.BOOTLOADER_DD_KEY) @property def spl_in_boot_part(self): """Whether spl binary should be put in the boot partition. A str.""" return self._get_bootloader_option(SPL_IN_BOOT_PART_FIELD) @property def bootloader_copy_files(self): """Extra files to copy to boot partition. This can be stored in several formats. We always present in a common one: {source_package: [{source_file_path: dest_file_path}]. dest_file_path (in the above example) is always absolute. """ #copy_files: # source_package: # - source_file_path : dest_file_path # - source_file_without_explicit_destination #copy_files: # - file1 # - file2: dest_path # # Note that the list of files is always that - a list. copy_files = self._get_bootloader_option(COPY_FILES_FIELD) if copy_files is None: return None if not isinstance(copy_files, dict): copy_files = {self.bootloader_package: copy_files} for package in copy_files: new_list = [] for value in copy_files[package]: if not isinstance(value, dict): dest_path = "/boot" source_path = value else: if len(value.keys()) > 1: raise HwpackConfigError("copy_files entry found with" "more than one destination") source_path = value.keys()[0] dest_path = value[source_path] if not dest_path.startswith("/boot"): # Target path should be relative, or start with /boot - we # don't support to copying to anywhere other than /boot. if dest_path[0] == "/": raise HwpackConfigError( "copy_files destinations must" "be relative to /boot or start with /boot.") dest_path = os.path.join("/boot", dest_path) new_list.append({source_path: dest_path}) copy_files[package] = new_list return copy_files @property def spl_dd(self): """If the spl binary should be dd:d to the boot partition this field specifies the offset. An int.""" return self._get_bootloader_option(SPL_DD_FIELD) @property def env_dd(self): """If the env should be dd:d to the boot partition. 'Yes' or 'No'.""" return self._get_bootloader_option(ENV_DD_FIELD) def _get_option_bool(self, key): """Gets a boolean value from the key.""" if self.format.format_as_string == '3.0': value = self._get_option(key, convert_to="disable") if isinstance(value, bool): return value else: raise ValueError(value) else: try: return self.parser.getboolean(self.MAIN_SECTION, key) except ConfigParser.NoOptionError: return None def _get_bootloader_option(self, key, join_list_with=False, convert_to=None): """Get an option inside the current bootloader section.""" if self._is_v3: if not self.bootloader: if self.allow_unset_bootloader: return None raise ValueError("bootloader not set.") if not isinstance(key, list): keys = [key] keys = [BOOTLOADERS_FIELD, self.bootloader] + keys else: keys = key return self._get_option(keys, join_list_with, convert_to) def _bool_to_string(self, value): """Convert value, treated as boolean, to string "yes" or "no".""" if value: return "yes" else: return "no" def _hex_addrress(self, value): """Convert value to 8 character hex string""" converted_value = value if not isinstance(value, str): converted_value = "0x%08x" % value return converted_value def _v2_key_to_v3(self, key): """Convert V2 key to a V3 key""" if key in self.translate_v2_to_v3: key = self.translate_v2_to_v3[key] return key def _get_v3_option(self, keys): """Find value in config dictionary based on supplied list (keys).""" result = self.parser for key in keys: key = self._v2_key_to_v3(key) if result is not None: result = result.get(key, None) self.last_used_keys = keys return result def get_last_used_keys(self): """Used so you can work out which boards + boot loader was used. Configuration data is stored in a dictionary. This returns a list of keys used to traverse into the dictionary the last time an item was looked up. This can be used to see where a bit of information came from - we store data that may be indexed differently depending on which board and bootloader are set. """ return self.last_used_keys def get_option(self, name): """Return the value of an attribute by name. Used when you can't use a property. """ return attrgetter(name)(self) def _get_option(self, key, join_list_with=False, convert_to=None): """Return value for the given key. Precedence to board specific values. :param key: the key to return the value for. :type key: str. :param join_list_with: Used to convert lists to strings. :type join_list_with: str :param convert_to: Used to convert stored value to another type. :type convert_to: type or function. :return: the value for that key, or None if the key is not present or the value is empty. :rtype: str or None. """ if self.format.format_as_string == "3.0": if not isinstance(key, list): keys = [key] else: keys = key result = None # Just mark result as not set yet... # If board is set, search board specific keys first if self.board: result = self._get_v3_option([BOARDS_FIELD, self.board] + keys) # If a board specific value isn't found, look for a global one if result is None: result = self._get_v3_option(keys) # If no value is found, bail early (return None) if result is None: return None # 0 except: raise HwpackConfigError( "Invalid root min size %s" % (root_min_size)) def _validate_boot_min_size(self): boot_min_size = self.boot_min_size if boot_min_size is None: return try: assert int(boot_min_size) > 0 except: raise HwpackConfigError( "Invalid boot min size %s" % (boot_min_size)) def _validate_loader_min_size(self): loader_min_size = self.loader_min_size if loader_min_size is None: return try: assert int(loader_min_size) > 0 except: raise HwpackConfigError( "Invalid loader min size %s" % (loader_min_size)) def _validate_loader_start(self): loader_start = self.loader_start if loader_start is None: return try: assert int(loader_start) > 0 except: raise HwpackConfigError( "Invalid loader start %s" % (loader_start)) def _validate_include_debs(self): try: self.include_debs except ValueError: raise HwpackConfigError( "Invalid value for include-debs: %s" % self.include_debs) @property def _is_v3(self): """Checks if format is 3.0.""" return self.format.format_as_string == '3.0' def _validate_bool(self, value): """Checks if a value is boolean or not, represented by "yes" or "no". """ if not isinstance(value, str): return False return string.lower(value) in ['yes', 'no'] def _validate_bootloader_file_in_boot_part(self): if not self._validate_bool(self.bootloader_file_in_boot_part): if self._is_v3: name = "bootloader" else: name = "u_boot" raise HwpackConfigError( "Invalid value for %s_in_boot_part: %s" % (name, self.bootloader_file_in_boot_part)) def _validate_spl_in_boot_part(self): spl_in_boot_part = self.spl_in_boot_part if spl_in_boot_part is None: return if string.lower(spl_in_boot_part) not in ['yes', 'no']: raise HwpackConfigError( "Invalid value for spl_in_boot_part: %s" % self.spl_in_boot_part) def _validate_env_dd(self): env_dd = self.env_dd if env_dd is None: return if string.lower(env_dd) not in ['yes', 'no']: raise HwpackConfigError( "Invalid value for env_dd: %s" % self.env_dd) def _validate_bootloader_dd(self): bootloader_dd = self.bootloader_dd if bootloader_dd is None: return try: assert int(bootloader_dd) > 0 except: if self._is_v3: name = "bootloader" else: name = "u_boot" raise HwpackConfigError( "Invalid %s_dd %s" % (name, bootloader_dd)) def _validate_spl_dd(self): spl_dd = self.spl_dd if spl_dd is None: return try: assert int(spl_dd) > 0 except: raise HwpackConfigError( "Invalid spl_dd %s" % (spl_dd)) def _validate_support(self): support = self.support if support not in (None, "supported", "unsupported"): raise HwpackConfigError( "Invalid value for support: %s" % support) def _invalid_package_message(self, package_name, section_name, value): if self._is_v3: message = ("Invalid value in %s in the metadata: %s" % (package_name, value)) else: message = ("Invalid value in %s in the [%s] section: %s" % (package_name, section_name, value)) return message def _validate_packages(self): packages = self.packages if not packages: raise HwpackConfigError(self._not_found_message(PACKAGES_FIELD)) for package in packages: self._assert_matches_pattern( self.PACKAGE_REGEX, package, self._invalid_package_message( PACKAGES_FIELD, self.MAIN_SECTION, package)) def _validate_bootloader_package(self): bootloader_package = self.bootloader_package if bootloader_package is not None: self._assert_matches_pattern( self.PACKAGE_REGEX, bootloader_package, self._invalid_package_message( self.BOOTLOADER_PACKAGE_KEY, self.MAIN_SECTION, bootloader_package)) def _validate_spl_package(self): spl_package = self.spl_package if spl_package is not None: self._assert_matches_pattern( self.PACKAGE_REGEX, spl_package, self._invalid_package_message(SPL_PACKAGE_FIELD, self.MAIN_SECTION, spl_package)) def _validate_samsung_bl1_start(self): samsung_bl1_start = self.samsung_bl1_start if samsung_bl1_start is None: return try: assert int(samsung_bl1_start) > 0 except: raise HwpackConfigError( "Invalid samsung_bl1_start %s" % (samsung_bl1_start)) def _validate_samsung_bl1_len(self): samsung_bl1_len = self.samsung_bl1_len if samsung_bl1_len is None: return try: assert int(samsung_bl1_len) > 0 except: raise HwpackConfigError( "Invalid samsung_bl1_len %s" % (samsung_bl1_len)) def _validate_samsung_env_start(self): samsung_env_start = self.samsung_env_start if samsung_env_start is None: return try: assert int(samsung_env_start) > 0 except: raise HwpackConfigError( "Invalid samsung_env_start %s" % (samsung_env_start)) def _validate_samsung_env_len(self): samsung_env_len = self.samsung_env_len if samsung_env_len is None: return try: assert int(samsung_env_len) > 0 except: raise HwpackConfigError( "Invalid samsung_env_len %s" % (samsung_env_len)) def _validate_samsung_bl2_start(self): samsung_bl2_start = self.samsung_bl2_start if samsung_bl2_start is None: return try: assert int(samsung_bl2_start) > 0 except: raise HwpackConfigError( "Invalid samsung_bl2_start %s" % (samsung_bl2_start)) def _validate_samsung_bl2_len(self): samsung_bl2_len = self.samsung_bl2_len if samsung_bl2_len is None: return try: assert int(samsung_bl2_len) > 0 except: raise HwpackConfigError( "Invalid samsung_bl2_len %s" % (samsung_bl2_len)) def _validate_architectures(self): architectures = self.architectures if not architectures: raise HwpackConfigError( self._not_found_message(ARCHITECTURES_FIELD)) def _validate_assume_installed(self): assume_installed = self.assume_installed for package in assume_installed: self._assert_matches_pattern( self.PACKAGE_REGEX, package, self._invalid_package_message(self.ASSUME_INSTALLED_KEY, self.MAIN_SECTION, package)) def _message_start(self, key, section_name): if self._is_v3: message = "The %s, %s " % (key, section_name) else: message = "The %s in the [%s] section " % (key, section_name) return message def _validate_source(self, section_name): if self._is_v3: sources_entry = self._get_option([SOURCES_FIELD] + [section_name]) else: try: sources_entry = self.parser.get( section_name, self.SOURCES_ENTRY_KEY) except ConfigParser.NoOptionError: raise HwpackConfigError( "No %s in the [%s] section" % (self.SOURCES_ENTRY_KEY, section_name)) if not sources_entry: raise HwpackConfigError( self._message_start(self.SOURCES_ENTRY_KEY, section_name) + "is missing the URI") if len(sources_entry.split(" ", 1)) < 2: raise HwpackConfigError( self._message_start(self.SOURCES_ENTRY_KEY, section_name) + "is missing the distribution") if sources_entry.startswith("deb"): raise HwpackConfigError( self._message_start(self.SOURCES_ENTRY_KEY, section_name) + "shouldn't start with 'deb'") def _validate_sources(self): if self._is_v3: source_dict = self.parser.get(SOURCES_FIELD) if not source_dict: return if isinstance(source_dict, dict): sources = source_dict.keys() else: raise HwpackConfigError( "The %s in the [%s] section is missing the URI" % (self.SOURCES_ENTRY_KEY, source_dict)) else: sources = self.parser.sections() found = False for source_name in sources: if source_name == self.MAIN_SECTION: continue self._validate_source(source_name) found = True if not found: raise HwpackConfigError( "No sections other than [%s]" % self.MAIN_SECTION) def _validate_keys(self): """Check the dictionary created by the YAML parser for unknown keys""" if not self._is_v3: # We don't check V1 or V2 configurations in this way return self._validate_keys_layout = hwpack_v3_layout self._do_validate_keys_prefix = [] self._do_validate_keys(self._validate_keys_layout, self.parser) def _do_validate_keys_push_prefix(self, prefix): self._do_validate_keys_prefix.append(prefix) prefix = ": ".join(self._do_validate_keys_prefix)[2:] if prefix: prefix += ": " return prefix def _do_validate_keys(self, expected, config, prefix=""): prefix = self._do_validate_keys_push_prefix(prefix) if not isinstance(config, dict): raise HwpackConfigError("Invalid structure in metadata. Expected " "key: value pairs, found: '%s'" % (prefix + str(config))) for key in config.keys(): # If expected == {"*": {...}} then we can accept any key if("*" in expected and expected.keys() == ["*"] and isinstance(expected["*"], dict)): # Have found a sub-dictionary to check. Recurse. self._do_validate_keys(expected["*"], config[key], key) continue # Check to see if the key is valid if key not in expected: raise HwpackConfigError("Unknown key in metadata: '%s'" % (prefix + str(key))) # Have a valid key. If it should point to a dictionary, recurse if expected[key]: if isinstance(expected[key], dict): # Have found a sub-dictionary to check. Recurse. self._do_validate_keys(expected[key], config[key], key) continue if expected[key] == "root": config = config[key] prefix = self._do_validate_keys_push_prefix(key) for key in config.keys(): self._do_validate_keys(self._validate_keys_layout, config[key], key) self._do_validate_keys_prefix.pop() self._do_validate_keys_prefix.pop() linaro-image-tools-2014.11/linaro_image_tools/hwpack/tarfile_matchers.py0000664000175000017500000001625212400111666027470 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import tarfile from testtools.matchers import Annotate, Equals, Matcher, Mismatch class TarfileMissingPathMismatch(Mismatch): """A Mismatch indicating that a required path was missing from a tarfile. """ def __init__(self, tarball, path): """Create a TarfileMissingPathMismatch Mismatch. :param tarball: the tarfile that was checked. :param path: the path that was expected to be present. """ self.tarball = tarball self.path = path def describe(self): return '"%s" has no path "%s"' % (self.tarball, self.path) def __eq__(self, other): return self.tarball == other.tarball and self.path == other.path def __ne__(self, other): return not self.__eq__(other) class TarfileWrongValueMismatch(Mismatch): """A Mismatch indicating that an entry in a tarfile was not as expected. """ def __init__(self, attribute, tarball, path, expected, actual): """Create a TarfileWrongValueMismatch Mismatch. :param attribute: the attribute that was not as expected. :type attribute: str :param tarball: the tarfile that was checked. :param path: the path that was checked. :param expected: the expected value of the attribute. :param actual: the value that was found. """ self.attribute = attribute self.tarball = tarball self.path = path self.expected = expected self.actual = actual def describe(self): return 'The path "%s" in "%s" has %s %s, expected %s' % ( self.path, self.tarball, self.attribute, self.actual, self.expected) def __eq__(self, other): return (self.attribute == other.attribute and self.tarball == other.tarball and self.path == other.path and self.expected == other.expected and self.actual == other.actual) def __ne__(self, other): return not self.__eq__(other) class TarfileHasFile(Matcher): """Check that a tarfile has an entry with certain values.""" def __init__(self, path, type=None, size=None, mtime=None, mtime_skew=None, mode=None, linkname=None, uid=None, gid=None, uname=None, gname=None, content=None, content_matcher=None): """Create a TarfileHasFile Matcher. :param path: the path that must be present. :type path: str :param type: the type of TarInfo that must be at `path`, or None to not check. :param size: the size that the entry at `path` must have, or None to not check. :param mtime: the mtime that the entry at `path` must have, or None to not check. :param mtime_skew: the number of seconds that the file mtime can be different to the required. :param mode: the mode that the entry at `path` must have, or None to not check. :param linkname: the linkname that the entry at `path` must have, or None to not check. :param uid: the user id that the entry at `path` must have, or None to not check. :param gid: the group id that the entry at `path` must have, or None to not check. :param uname: the username that the entry at `path` must have, or None to not check. :param gname: the group name that the entry at `path` must have, or None to not check. :param content: the content that `path` must have when extracted, or None to not check. :param content_matcher: a matcher to match the content that `path` has when extracted, or None to not check. You can't specify both content_matcher and content. """ self.path = path self.type = type self.size = size self.mtime = mtime self.mtime_skew = mtime_skew self.mode = mode self.linkname = linkname self.uid = uid self.gid = gid self.uname = uname self.gname = gname if content is not None: if content_matcher is not None: raise ValueError( "doesn't make sense to specify content and " "content_matcher") content_matcher = Equals(content) if content_matcher is not None: self.content_matcher = Annotate( 'The content of path "%s" did not match' % path, content_matcher) else: self.content_matcher = None def match(self, tarball): """Match a tarfile.TarFile against the expected values.""" if self.path not in tarball.getnames(): return TarfileMissingPathMismatch(tarball, self.path) info = tarball.getmember(self.path) for attr in ("type", "size", "mode", "linkname", "uid", "gid", "uname", "gname"): expected = getattr(self, attr, None) if expected is not None: actual = getattr(info, attr) if expected != actual: return TarfileWrongValueMismatch( attr, tarball, self.path, expected, actual) if self.mtime is not None: mtime_skew = self.mtime_skew or 0 if abs(self.mtime - info.mtime) > mtime_skew: return TarfileWrongValueMismatch( "mtime", tarball, self.path, self.mtime, info.mtime) if self.content_matcher is not None: if info.type == tarfile.DIRTYPE: contents = [] path_frags = self.path.split('/') for name in tarball.getnames(): name_frags = name.split('/') if (len(name_frags) == len(path_frags) + 1 and name_frags[:-1] == path_frags): contents.append(name_frags[-1]) content_mismatch = self.content_matcher.match(contents) if content_mismatch: return content_mismatch else: actual = tarball.extractfile(self.path).read() content_mismatch = self.content_matcher.match(actual) if content_mismatch: return content_mismatch return None def __str__(self): return 'tarfile has file "%s"' % (self.path, ) linaro-image-tools-2014.11/linaro_image_tools/hwpack/hwpack_fields.py0000664000175000017500000001244112400111666026753 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011, 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. # This file contains all the valid fields for an hwpack v3. # Reference wiki page: https://wiki.linaro.org/HardwarePacksV3 # # Try to keep it alphabetically sorted per section. # ARCHITECTURES_FIELD = 'architectures' ARCHITECTURE_FIELD = 'architecture' ASSUME_INSTALLED_FIELD = 'assume_installed' BOARDS_FIELD = 'boards' BOOTLOADERS_FIELD = 'bootloaders' BOOT_MIN_SIZE_FIELD = 'boot_min_size' BOOT_SCRIPT_FIELD = 'boot_script' COPY_FILES_FIELD = 'copy_files' DTB_ADDR_FIELD = 'dtb_addr' DTB_FILE_FIELD = 'dtb_file' DTB_FILES_FIELD = 'dtb_files' EXTRA_SERIAL_OPTIONS_FIELD = 'extra_serial_options' FORMAT_FIELD = 'format' INCLUDE_DEBS_FIELD = 'include_debs' INITRD_ADDR_FIELD = 'initrd_addr' INITRD_FILE_FIELD = 'initrd_file' KERNEL_ADDR_FIELD = 'kernel_addr' KERNEL_FILE_FIELD = 'kernel_file' LOAD_ADDR_FIELD = 'load_addr' LOADER_MIN_SIZE_FIELD = 'loader_min_size' LOADER_START_FIELD = 'loader_start' MAINTAINER_FIELD = 'maintainer' MMC_ID_FIELD = 'mmc_id' NAME_FIELD = 'name' ORIGIN_FIELD = 'origin' PACKAGES_FIELD = 'packages' PARTITION_LAYOUT_FIELD = 'partition_layout' ROOT_MIN_SIZE_FIELD = 'root_min_size' SERIAL_TTY_FIELD = 'serial_tty' SOURCES_FIELD = 'sources' SUPPORT_FIELD = 'support' WIRED_INTERFACES_FIELD = 'wired_interfaces' WIRELESS_INTERFACES_FIELD = 'wireless_interfaces' VERSION_FIELD = 'version' # Bootloaders specific fields COPY_FILES_FIELD = 'copy_files' DD_FIELD = 'dd' ENV_DD_FIELD = 'env_dd' EXTRA_BOOT_OPTIONS_FIELD = 'extra_boot_options' FILE_FIELD = 'file' IN_BOOT_PART_FIELD = 'in_boot_part' PACKAGE_FIELD = 'package' SPL_DD_FIELD = 'spl_dd' SPL_FILE_FIELD = 'spl_file' SPL_IN_BOOT_PART_FIELD = 'spl_in_boot_part' SPL_PACKAGE_FIELD = 'spl_package' # Samsung fields SAMSUNG_BL1_LEN_FIELD = 'samsung_bl1_len' SAMSUNG_BL1_START_FIELD = 'samsung_bl1_start' SAMSUNG_BL2_LEN_FIELD = 'samsung_bl2_len' SAMSUNG_BL2_START_FIELD = 'samsung_bl2_start' SAMSUNG_ENV_LEN_FIELD = 'samsung_env_len' SAMSUNG_ENV_START_FIELD = 'samsung_env_start' # Snowball fields SNOWBALL_STARTUP_FILES_CONFIG_FIELD = 'snowball_startup_files_config' # Fields that might be necessary for the metadata file METADATA_ARCH_FIELD = 'architecture' METADATA_VERSION_FIELD = 'version' # The allowed partition layouts. BOOTFS16 = 'bootfs16_rootfs' BOOTFS = 'bootfs_rootfs' RESERVED_BOOTFS = 'reserved_bootfs_rootfs' DEFINED_PARTITION_LAYOUTS = [ BOOTFS16, BOOTFS, RESERVED_BOOTFS, ] # Supported bootloaders U_BOOT = 'u_boot' UEFI = 'uefi' DEFAULT_BOOTLOADER = U_BOOT # Define where fields are valid, so we can test them. # If a key has a value None, this indicates there is either a value or # list of values that can be associated with it. # If a key contains a dictionary, this means that the key can # contain a dictionary. # The string "root" indicates that the key can contain the root # structure. This is used for the boards section, where each # board can contain the full or partial layout, overwriting the global # settings. hwpack_v3_layout = { FORMAT_FIELD: None, NAME_FIELD: None, ARCHITECTURES_FIELD: None, ORIGIN_FIELD: None, MAINTAINER_FIELD: None, SUPPORT_FIELD: None, ASSUME_INSTALLED_FIELD: None, INCLUDE_DEBS_FIELD: None, DTB_FILE_FIELD: None, DTB_FILES_FIELD: None, DTB_ADDR_FIELD: None, SERIAL_TTY_FIELD: None, EXTRA_SERIAL_OPTIONS_FIELD: None, MMC_ID_FIELD: None, PACKAGES_FIELD: None, PARTITION_LAYOUT_FIELD: None, KERNEL_FILE_FIELD: None, KERNEL_ADDR_FIELD: None, INITRD_FILE_FIELD: None, INITRD_ADDR_FIELD: None, LOAD_ADDR_FIELD: None, BOOT_SCRIPT_FIELD: None, LOADER_START_FIELD: None, WIRED_INTERFACES_FIELD: None, WIRELESS_INTERFACES_FIELD: None, BOOT_MIN_SIZE_FIELD: None, ROOT_MIN_SIZE_FIELD: None, LOADER_MIN_SIZE_FIELD: None, SAMSUNG_BL1_LEN_FIELD: None, SAMSUNG_BL1_START_FIELD: None, SAMSUNG_ENV_LEN_FIELD: None, SAMSUNG_ENV_START_FIELD: None, SAMSUNG_BL2_LEN_FIELD: None, SAMSUNG_BL2_START_FIELD: None, SNOWBALL_STARTUP_FILES_CONFIG_FIELD: None, SOURCES_FIELD: None, BOOTLOADERS_FIELD: { "*": { PACKAGE_FIELD: None, FILE_FIELD: None, IN_BOOT_PART_FIELD: None, COPY_FILES_FIELD: None, DD_FIELD: None, EXTRA_BOOT_OPTIONS_FIELD: None, SPL_PACKAGE_FIELD: None, SPL_FILE_FIELD: None, SPL_IN_BOOT_PART_FIELD: None, SPL_DD_FIELD: None, ENV_DD_FIELD: None, } }, BOARDS_FIELD: "root", } linaro-image-tools-2014.11/linaro_image_tools/hwpack/hardwarepack_format.py0000664000175000017500000000430412400111666030153 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import logging logger = logging.getLogger(__name__) class HardwarePackFormat(object): def __init__(self): self.format_as_string = None self.is_deprecated = False self.is_supported = False self.has_v2_fields = False def __str__(self): if self.format_as_string is None: raise NotImplementedError() if self.is_deprecated: logger.warning("The format '%s' is deprecated, please update " "your hardware pack configuration." % self.format_as_string) return self.format_as_string class HardwarePackFormatV1(HardwarePackFormat): def __init__(self): super(HardwarePackFormatV1, self).__init__() self.format_as_string = "1.0" self.is_supported = True self.is_deprecated = True class HardwarePackFormatV2(HardwarePackFormat): def __init__(self): super(HardwarePackFormatV2, self).__init__() self.format_as_string = "2.0" self.is_supported = True self.is_deprecated = False self.has_v2_fields = True class HardwarePackFormatV3(HardwarePackFormat): def __init__(self): super(HardwarePackFormatV3, self).__init__() self.format_as_string = "3.0" self.is_supported = True self.is_deprecated = False self.has_v2_fields = True linaro-image-tools-2014.11/linaro_image_tools/__init__.py0000664000175000017500000000000012400111666024416 0ustar fboudrafboudra00000000000000linaro-image-tools-2014.11/linaro_image_tools/__version__.py0000664000175000017500000000003012435051422025144 0ustar fboudrafboudra00000000000000__version__ = "2014.11" linaro-image-tools-2014.11/linaro_image_tools/tests/0000775000175000017500000000000012435067365023477 5ustar fboudrafboudra00000000000000linaro-image-tools-2014.11/linaro_image_tools/tests/__init__.py0000664000175000017500000000335712400111666025602 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011, 2012 Linaro # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os import unittest from linaro_image_tools.hwpack.tests import test_suite as hwpack_suite from linaro_image_tools.media_create.tests import ( test_suite as media_create_suite, ) from linaro_image_tools.utils import has_command def test_suite(): module_names = [ 'linaro_image_tools.tests.test_cmd_runner', 'linaro_image_tools.tests.test_utils', ] # if pyflakes is installed and we're running from a bzr checkout... if has_command('pyflakes') and not os.path.isabs(__file__): # ...also run the pyflakes test module_names.append('linaro_image_tools.tests.test_pyflakes') # if pep8 is installed and we're running from a bzr checkout... if has_command('pep8') and not os.path.isabs(__file__): # ...also run the pep8 test module_names.append('linaro_image_tools.tests.test_pep8') loader = unittest.TestLoader() suite = loader.loadTestsFromNames(module_names) suite.addTests(hwpack_suite()) suite.addTests(media_create_suite()) return suite linaro-image-tools-2014.11/linaro_image_tools/tests/fixtures.py0000664000175000017500000001133712400111666025711 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os import shutil import tempfile from StringIO import StringIO from linaro_image_tools import cmd_runner class CreateTempDirFixture(object): def __init__(self): self.tempdir = None def setUp(self): self.tempdir = tempfile.mkdtemp() def tearDown(self): if os.path.exists(self.tempdir): shutil.rmtree(self.tempdir) def get_temp_dir(self): return self.tempdir class CreateTempFileFixture(object): """Class to create a temporary file to be used in a test.""" def __init__(self, string=None): """Initialize the fixture. :param string: the string to write in the file. """ self.temp_file = None self.string = string def setUp(self): self.temp_file = tempfile.NamedTemporaryFile() if self.string is not None: self.temp_file.write(self.string) # Go back to the initial position, we just need to write something # and be able to read from the beginning of the file. self.temp_file.seek(0) def tearDown(self): # We don't need to do anything, file is automatically deleted. pass def get_file_name(self): return self.temp_file.name class MockSomethingFixture(object): """A fixture which mocks something on the given object. Replaces attr_name on obj with the given mock, undoing that upon tearDown(). """ def __init__(self, obj, attr_name, mock): self.obj = obj self.attr_name = attr_name self.mock = mock self.orig_attr = getattr(obj, attr_name) def setUp(self): setattr(self.obj, self.attr_name, self.mock) def tearDown(self): setattr(self.obj, self.attr_name, self.orig_attr) class MockCmdRunnerPopen(object): """A mock for cmd_runner.Popen() which stores the args given to it.""" calls = None # A variable that is set to False in __call__() and only set back to True # when wait() is called, to indicate that the subprocess has finished. Is # used in tests to make sure all callsites wait for their child. child_finished = True def __init__(self, output_string='', assert_child_finished=True): self.assert_child_finished = assert_child_finished self.output_string = output_string def __call__(self, cmd, *args, **kwargs): if self.assert_child_finished and not self.child_finished: raise AssertionError( "You should call wait() or communicate() to ensure " "the subprocess is finished before proceeding.") self.child_finished = False if self.calls is None: self.calls = [] if isinstance(cmd, basestring): all_args = [cmd] else: all_args = cmd all_args.extend(args) self.calls.append(all_args) self.returncode = 0 return self def communicate(self, input=None): self.wait() return self.output_string, '' def wait(self): self.child_finished = True return self.returncode @property def commands_executed(self): return [' '.join(args) for args in self.calls] @property def stdin(self): return StringIO() class MockCmdRunnerPopenFixture(MockSomethingFixture): """A test fixture which mocks cmd_runner.do_run with the given mock. If no mock is given, a MockCmdRunnerPopen instance is used. """ def __init__(self, output_string='', assert_child_finished=True): super(MockCmdRunnerPopenFixture, self).__init__( cmd_runner, 'Popen', MockCmdRunnerPopen(output_string, assert_child_finished)) def tearDown(self): super(MockCmdRunnerPopenFixture, self).tearDown() if self.mock.assert_child_finished and not self.mock.child_finished: raise AssertionError( "You should call wait() or communicate() to ensure " "the subprocess is finished before proceeding.") linaro-image-tools-2014.11/linaro_image_tools/tests/test_cmd_runner.py0000664000175000017500000001027512400111666027233 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os from linaro_image_tools import cmd_runner from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( MockCmdRunnerPopenFixture, MockSomethingFixture, ) sudo_args = " ".join(cmd_runner.SUDO_ARGS) chroot_args = " ".join(cmd_runner.CHROOT_ARGS) class TestSanitizePath(TestCaseWithFixtures): def setUp(self): super(TestSanitizePath, self).setUp() self.env = {} def test_path_unset(self): cmd_runner.sanitize_path(self.env) self.assertEqual(cmd_runner.DEFAULT_PATH, self.env['PATH']) def test_path_missing_dirs(self): path = '/bin:/sbin:/foo:/usr/local/sbin' self.env['PATH'] = path cmd_runner.sanitize_path(self.env) expected = '%s:/usr/local/bin:/usr/sbin:/usr/bin' % path self.assertEqual(expected, self.env['PATH']) def test_idempotent(self): self.env['PATH'] = cmd_runner.DEFAULT_PATH cmd_runner.sanitize_path(self.env) self.assertEqual(cmd_runner.DEFAULT_PATH, self.env['PATH']) class TestCmdRunner(TestCaseWithFixtures): def test_run(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) proc = cmd_runner.run(['foo', 'bar', 'baz']) # Call wait or else MockCmdRunnerPopenFixture() raises an # AssertionError(). proc.wait() self.assertEqual(0, proc.returncode) self.assertEqual(['foo bar baz'], fixture.mock.commands_executed) def test_run_as_root_with_sudo(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture(os, 'getuid', lambda: 1000)) cmd_runner.run(['foo', 'bar'], as_root=True).wait() self.assertEqual( ['%s foo bar' % sudo_args], fixture.mock.commands_executed) def test_run_as_root_as_root(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture(os, 'getuid', lambda: 0)) cmd_runner.run(['foo', 'bar'], as_root=True).wait() self.assertEqual(['foo bar'], fixture.mock.commands_executed) def test_tuple_with_sudo(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture(os, 'getuid', lambda: 1000)) cmd_runner.run(('foo', 'bar',), as_root=True).wait() self.assertEqual( ['%s foo bar' % sudo_args], fixture.mock.commands_executed) def test_chrooted(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) cmd_runner.run(['foo', 'bar'], chroot='chroot_dir').wait() self.assertEqual( ['%s %s chroot_dir foo bar' % (sudo_args, chroot_args)], fixture.mock.commands_executed) def test_run_succeeds_on_zero_return_code(self): proc = cmd_runner.run(['true']) # Need to wait() here as we're using the real Popen. proc.wait() self.assertEqual(0, proc.returncode) def test_run_raises_exception_on_non_zero_return_code(self): def run_and_wait(): proc = cmd_runner.run(['false']) proc.wait() self.assertRaises( cmd_runner.SubcommandNonZeroReturnValue, run_and_wait) def test_run_must_be_given_list_as_args(self): self.assertRaises(AssertionError, cmd_runner.run, 'true') def test_Popen(self): proc = cmd_runner.Popen('true') returncode = proc.wait() self.assertEqual(0, returncode) linaro-image-tools-2014.11/linaro_image_tools/tests/integration.txt0000664000175000017500000000247712400111666026557 0ustar fboudrafboudra00000000000000A few integration tests we can run to test things on a higher level. These will probably require root or access to specific block devices so they are not meant to be automated. # This should print nothing to stdout but will unpack the given binary # tarball under the given directory. >>> from linaro_image_tools.media_create.unpack_binary_tarball import unpack_binary_tarball >>> unpack_binary_tarball(, ) # Partition (for real!) /dev/sdb for a beagle board and return the devices # for the boot and root partitions. >>> from linaro_image_tools.media_create.partitions import Media, setup_partitions >>> from linaro_image_tools.media_create.boards import board_configs >>> setup_partitions( ... board_configs['beagle'], Media('/dev/sdb'), None, 'boot', ... 'root', 'ext3', True, True, True) Checking that no-one is using this disk right now ... ('/dev/sdb1', '/dev/sdb2') # Partition /tmp/beagle.img for a beagle board and return the loopback # devices for the boot and root partitions. >>> setup_partitions( ... board_configs['beagle'], Media('/tmp/beagle.img'), '2G', 'boot', ... 'root', 'ext3', True, True, True) Warning: /tmp/beagle.img is not a block device ... ('/dev/loop0', '/dev/loop1') linaro-image-tools-2014.11/linaro_image_tools/tests/test_pyflakes.py0000664000175000017500000000225312400111666026712 0ustar fboudrafboudra00000000000000# Copyright (C) 2011 Linaro # # Author: Loic Minier # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import subprocess from testtools import TestCase class TestPyflakes(TestCase): def test_pyflakes(self): # ignore return code proc = subprocess.Popen(['pyflakes', '.'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = proc.communicate() self.assertEquals('', stdout) self.assertEquals('', stderr) linaro-image-tools-2014.11/linaro_image_tools/tests/test_utils.py0000664000175000017500000003764512400111666026251 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os import stat import subprocess import sys import logging import tempfile import tarfile from StringIO import StringIO from linaro_image_tools import cmd_runner, utils from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( CreateTempDirFixture, MockCmdRunnerPopenFixture, MockSomethingFixture, ) from linaro_image_tools.utils import ( IncompatibleOptions, InvalidHwpackFile, UnableToFindPackageProvidingCommand, additional_android_option_checks, additional_option_checks, android_hwpack_in_boot_tarball, check_file_integrity_and_log_errors, ensure_command, find_command, install_package_providing, path_in_tarfile_exists, preferred_tools_dir, prep_media_path, try_import, verify_file_integrity, ) sudo_args = " ".join(cmd_runner.SUDO_ARGS) class TestPathInTarfile(TestCaseWithFixtures): def setUp(self): super(TestPathInTarfile, self).setUp() tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.tarfile_name = os.path.join(tempdir, 'test_tarfile.tar.gz') self.tempfile_added = self.createTempFileAsFixture() self.tempfile_unused = self.createTempFileAsFixture() with tarfile.open(self.tarfile_name, 'w:gz') as tar: tar.add(self.tempfile_added) def test_file_exists(self): self.assertTrue(path_in_tarfile_exists(self.tempfile_added[1:], self.tarfile_name)) def test_file_does_not_exist(self): self.assertFalse(path_in_tarfile_exists(self.tempfile_unused[1:], self.tarfile_name)) class TestVerifyFileIntegrity(TestCaseWithFixtures): filenames_in_shafile = ['verified-file1', 'verified-file2'] class MockCmdRunnerPopen(object): def __call__(self, cmd, *args, **kwargs): self.returncode = 0 return self def communicate(self, input=None): self.wait() return ': OK\n'.join( TestVerifyFileIntegrity.filenames_in_shafile) + ': OK\n', '' def wait(self): return self.returncode class MockCmdRunnerPopen_sha1sum_fail(object): def __call__(self, cmd, *args, **kwargs): self.returncode = 0 return self def communicate(self, input=None): self.wait() return ': ERROR\n'.join( TestVerifyFileIntegrity.filenames_in_shafile) + ': ERROR\n', '' def wait(self): return self.returncode class MockCmdRunnerPopen_wait_fails(object): def __call__(self, cmd, *args, **kwargs): self.returncode = 0 return self def communicate(self, input=None): self.wait() return ': OK\n'.join( TestVerifyFileIntegrity.filenames_in_shafile) + ': OK\n', '' def wait(self): stdout = ': OK\n'.join( TestVerifyFileIntegrity.filenames_in_shafile) + ': OK\n' raise cmd_runner.SubcommandNonZeroReturnValue([], 1, stdout, None) class FakeTempFile(): name = "/tmp/1" def close(self): pass def read(self): return "" def test_verify_files(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture(tempfile, 'NamedTemporaryFile', self.FakeTempFile)) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" verify_file_integrity([signature_filename]) self.assertEqual( ['gpg --status-file=%s --verify %s' % (self.FakeTempFile.name, signature_filename), 'sha1sum -c %s' % hash_filename], fixture.mock.commands_executed) def test_verify_files_returns_files(self): self.useFixture(MockSomethingFixture(cmd_runner, 'Popen', self.MockCmdRunnerPopen())) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" verified_files, _, _ = verify_file_integrity([signature_filename]) self.assertEqual(self.filenames_in_shafile, verified_files) def test_check_file_integrity_and_print_errors(self): self.useFixture(MockSomethingFixture(cmd_runner, 'Popen', self.MockCmdRunnerPopen())) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" result, verified_files = check_file_integrity_and_log_errors( [signature_filename], self.filenames_in_shafile[0], [self.filenames_in_shafile[1]]) self.assertEqual(self.filenames_in_shafile, verified_files) # The sha1sums are faked as passing and all commands return 0, so # it should look like GPG passed self.assertTrue(result) def test_check_file_integrity_and_print_errors_fail_sha1sum(self): logging.getLogger().setLevel(100) # Disable logging messages to screen self.useFixture(MockSomethingFixture( cmd_runner, 'Popen', self.MockCmdRunnerPopen_sha1sum_fail())) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" result, verified_files = check_file_integrity_and_log_errors( [signature_filename], self.filenames_in_shafile[0], [self.filenames_in_shafile[1]]) self.assertEqual([], verified_files) # The sha1sums are faked as failing and all commands return 0, so # it should look like GPG passed self.assertFalse(result) logging.getLogger().setLevel(logging.WARNING) def test_check_file_integrity_and_print_errors_fail_gpg(self): logging.getLogger().setLevel(100) # Disable logging messages to screen self.useFixture(MockSomethingFixture( cmd_runner, 'Popen', self.MockCmdRunnerPopen_wait_fails())) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" result, verified_files = check_file_integrity_and_log_errors( [signature_filename], self.filenames_in_shafile[0], [self.filenames_in_shafile[1]]) self.assertEqual([], verified_files) # The sha1sums are faked as passing and all commands return 1, so # it should look like GPG failed self.assertFalse(result) logging.getLogger().setLevel(logging.WARNING) class TestEnsureCommand(TestCaseWithFixtures): install_pkg_providing_called = False def setUp(self): super(TestEnsureCommand, self).setUp() self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) def test_command_already_present(self): self.mock_install_package_providing() ensure_command('apt-get') self.assertFalse(self.install_pkg_providing_called) def test_command_not_present(self): self.mock_install_package_providing() ensure_command('apt-get-two-o') self.assertTrue(self.install_pkg_providing_called) def mock_install_package_providing(self): def mock_func(command): self.install_pkg_providing_called = True self.useFixture(MockSomethingFixture( utils, 'install_package_providing', mock_func)) class TestFindCommand(TestCaseWithFixtures): def test_preferred_dir(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() lmc = 'linaro-media-create' path = os.path.join(tempdir, lmc) open(path, 'w').close() os.chmod(path, stat.S_IXUSR) self.assertEquals(path, find_command(lmc, tempdir)) def test_existing_command(self): lmc = 'linaro-media-create' prefer_dir = preferred_tools_dir() if prefer_dir is None: expected, _ = cmd_runner.run( ['which', lmc, ], stdout=subprocess.PIPE).communicate() expected = expected.strip() else: expected = os.path.join(prefer_dir, lmc) self.assertEquals(expected, find_command(lmc)) def test_nonexisting_command(self): self.assertEquals(find_command('linaro-moo'), None) class TestInstallPackageProviding(TestCaseWithFixtures): # This is the package we need to fake the installation of, it is a # slightly changed version of 'apt-get -s install' output. # It is used as an argument to MockCmdRunnerPopenFixture in order to # pass a string that should be expected as output from the command that # is being executed. output_string = 'Inst dosfstools (3.0.12-1ubuntu1 Ubuntu:12.04)' def test_package_installation_accepted(self): self.useFixture(MockSomethingFixture(sys, 'stdout', open('/dev/null', 'w'))) # We need this since we are getting user input via raw_input # and we need a 'Y' to proceed with the operations. self.useFixture(MockSomethingFixture(sys, 'stdin', StringIO('Y'))) fixture = self.useFixture( MockCmdRunnerPopenFixture(self.output_string)) try: install_package_providing('mkfs.vfat') except UnableToFindPackageProvidingCommand as inst: self.assertEqual("CommandNotFound python module does not exist.", inst.args[0]) else: self.assertEqual( ['apt-get -s install dosfstools', '%s apt-get --yes install dosfstools' % sudo_args], fixture.mock.commands_executed) def test_package_installation_refused(self): self.useFixture(MockSomethingFixture(sys, 'stdout', open('/dev/null', 'w'))) # We need this since we are getting user input via raw_input # and we need a 'n' to mimic a refused package installation. self.useFixture(MockSomethingFixture(sys, 'stdin', StringIO('n'))) self.useFixture(MockCmdRunnerPopenFixture(self.output_string)) CommandNotFound = try_import('CommandNotFound.CommandNotFound') if CommandNotFound is None: self.assertRaises(UnableToFindPackageProvidingCommand, install_package_providing, 'mkfs.vfat') else: self.assertRaises(SystemExit, install_package_providing, 'mkfs.vfat') def test_not_found_package(self): self.assertRaises(UnableToFindPackageProvidingCommand, install_package_providing, 'mkfs.lean') class Args(): def __init__(self, directory, device, board): self.directory = directory self.device = device self.board = board class TestPrepMediaPath(TestCaseWithFixtures): def test_prep_media_path(self): self.useFixture(MockSomethingFixture(os.path, 'abspath', lambda x: x)) self.useFixture(MockSomethingFixture(os, "makedirs", lambda x: x)) self.assertEqual("testdevice", prep_media_path(Args(directory=None, device="testdevice", board="testboard"))) self.assertEqual("/foo/bar/testdevice", prep_media_path(Args(directory="/foo/bar", device="testdevice", board="testboard"))) class TestAdditionalOptionChecks(TestCaseWithFixtures): def test_additional_option_checks(self): self.useFixture(MockSomethingFixture(os.path, 'abspath', lambda x: x)) self.useFixture(MockSomethingFixture(os, "makedirs", lambda x: x)) self.assertRaises(IncompatibleOptions, additional_option_checks, Args(directory="/foo/bar", device="/testdevice", board="testboard")) sys.argv.append("--mmc") self.assertRaises(IncompatibleOptions, additional_option_checks, Args(directory="/foo/bar", device="testdevice", board="testboard")) sys.argv.remove("--mmc") class TestAndroidOptionChecks(TestCaseWithFixtures): def test_hwpack_is_file(self): class HwPacksArgs: def __init__(self, hwpack): self.hwpack = hwpack try: tmpdir = tempfile.mkdtemp() self.assertRaises(InvalidHwpackFile, additional_android_option_checks, HwPacksArgs(tmpdir)) finally: os.rmdir(tmpdir) def test_android_hwpack_in_boot(self): """Test presence of config file in boot directory.""" try: tmpdir = tempfile.mkdtemp() boot_dir = os.path.join(tmpdir, "boot") os.mkdir(boot_dir) config_file = os.path.join(boot_dir, "config") expected = (True, config_file) with open(config_file, "w"): self.assertEqual(expected, android_hwpack_in_boot_tarball(tmpdir)) finally: os.unlink(config_file) os.removedirs(boot_dir) def test_android_hwpack_not_in_boot(self): """Test missing config file.""" try: tmpdir = tempfile.mkdtemp() boot_dir = os.path.join(tmpdir, "boot") os.mkdir(boot_dir) config_file = os.path.join(boot_dir, "config") expected = (False, config_file) self.assertEqual(expected, android_hwpack_in_boot_tarball(tmpdir)) finally: os.removedirs(boot_dir) class TestHwpackIsFile(TestCaseWithFixtures): """Testing '--hwpack' option only allows regular files.""" def test_hwpack_is_file(self): class HwPackArgs: def __init__(self, hwpack): self.hwpacks = [hwpack] self.directory = None try: tmpdir = tempfile.mkdtemp() self.assertRaises(InvalidHwpackFile, additional_option_checks, HwPackArgs(hwpack=tmpdir)) finally: os.rmdir(tmpdir) def test_hwpacks_are_files(self): """ Tests that multiple hwpacks are regular files. Tests against a file and a directory, to avoid circumstances in which 'additional_option_checks' is tweaked. """ class HwPacksArgs: def __init__(self, hwpacks): self.hwpacks = hwpacks self.directory = None try: tmpdir = tempfile.mkdtemp() _, tmpfile = tempfile.mkstemp() self.assertRaises(InvalidHwpackFile, additional_option_checks, HwPacksArgs([tmpfile, tmpdir])) finally: os.rmdir(tmpdir) os.remove(tmpfile) linaro-image-tools-2014.11/linaro_image_tools/tests/test_pep8.py0000664000175000017500000000273412400111666025754 0ustar fboudrafboudra00000000000000# Copyright (C) 2012 Linaro Ltd. # # Author: Loic Minier # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import subprocess from testtools import TestCase class TestPep8(TestCase): def test_pep8(self): # Errors we have to ignore for now: # * E202 whitespace before ')' or ']' # E202 is actually only reported with the natty version of pep8 and # can be re-enabled once we drop support for natty. ignore = ['E202'] # Ignore return code. proc = subprocess.Popen( ['pep8', '--repeat', '--ignore=%s' % ','.join(ignore), '.'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = proc.communicate() self.assertEquals('', stdout) self.assertEquals('', stderr) linaro-image-tools-2014.11/linaro_image_tools/cmd_runner.py0000664000175000017500000001026512400111666025031 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import os import subprocess DEFAULT_PATH = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' CHROOT_ARGS = ['chroot'] SUDO_ARGS = ['sudo', '-E'] def sanitize_path(env): """Makes sure PATH is set and has important directories""" dirs = env.get('PATH', DEFAULT_PATH).split(os.pathsep) for d in DEFAULT_PATH.split(os.pathsep): if d not in dirs: dirs.append(d) env['PATH'] = os.pathsep.join(dirs) def run(args, as_root=False, chroot=None, stdin=None, stdout=None, stderr=None, cwd=None): """Run the given command as a sub process. Return a Popen instance. Callsites must wait() or communicate() with the returned Popen instance. :param command: A list or tuple containing the command to run and the arguments that should be passed to it. :param as_root: Should the given command be run as root (with sudo)? :param chroot: A directory to chroot into (implies as_root). :param stdin: Same as in subprocess.Popen(). :param stdout: Same as in subprocess.Popen(). :param stderr: Same as in subprocess.Popen(). """ assert isinstance(args, (list, tuple)), ( "The command to run must be a list or tuple, found: %s" % type(args)) if isinstance(args, tuple): args = list(args) if chroot is not None: args = CHROOT_ARGS + [chroot] + args as_root = True if as_root and os.getuid() != 0: args = SUDO_ARGS + args return Popen(args, stdin=stdin, stdout=stdout, stderr=stderr, cwd=cwd) class Popen(subprocess.Popen): """A version of Popen which raises an error on non-zero returncode. Once the subprocess completes we check its returncode and raise SubcommandNonZeroReturnValue if it's non-zero. """ def __init__(self, args, env=None, **kwargs): self._my_args = args self.except_on_cmd_fail = True if env is None: env = os.environ.copy() env['LC_ALL'] = 'C' # ensure a proper PATH before calling Popen sanitize_path(os.environ) # and for subcommands sanitize_path(env) super(Popen, self).__init__(args, env=env, **kwargs) def communicate(self, input=None): self.except_on_cmd_fail = False stdout, stderr = super(Popen, self).communicate(input) self.except_on_cmd_fail = True if self.returncode != 0: raise SubcommandNonZeroReturnValue(self._my_args, self.returncode, stdout, stderr) return stdout, stderr def wait(self): returncode = super(Popen, self).wait() if returncode != 0 and self.except_on_cmd_fail: raise SubcommandNonZeroReturnValue(self._my_args, returncode) return returncode class SubcommandNonZeroReturnValue(Exception): def __init__(self, command, return_value, stdout=None, stderr=None): self.command = command self.retval = return_value self.stdout = stdout self.stderr = stderr def __str__(self): message = 'Sub process "%s" returned a non-zero value: %d' % ( self.command, self.retval) if self.stdout: message += '\nstdout was\n{0}'.format(self.stdout) if self.stderr: message += '\nstderr was\n{0}'.format(self.stderr) return message linaro-image-tools-2014.11/linaro_image_tools/media_create/0000775000175000017500000000000012435067365024737 5ustar fboudrafboudra00000000000000linaro-image-tools-2014.11/linaro_image_tools/media_create/__init__.py0000664000175000017500000002527612400111666027046 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import argparse import subprocess import os from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.boards import board_configs from linaro_image_tools.media_create.android_boards import ( android_board_configs) from linaro_image_tools.__version__ import __version__ from linaro_image_tools.hwpack.hwpack_fields import ( DEFAULT_BOOTLOADER ) KNOWN_BOARDS = sorted(board_configs.keys()) ANDROID_KNOWN_BOARDS = sorted(android_board_configs.keys()) class Live256MegsAction(argparse.Action): """A custom argparse.Action for the --live-256m option. It is a store_true action for the given dest plus a store_true action for 'is_live'. """ def __init__(self, option_strings, dest, default=None, required=False, help=None, metavar=None): super(Live256MegsAction, self).__init__( option_strings=option_strings, dest=dest, nargs=0, default=False, required=required, help=help) def __call__(self, parser, namespace, values, option_string=None): setattr(namespace, self.dest, True) setattr(namespace, 'is_live', True) def get_version(): qemu_path = '/usr/bin/qemu-arm-static' p = cmd_runner.run(["head", "-n", "1"], stdin=subprocess.PIPE, stdout=subprocess.PIPE) if os.path.exists(qemu_path): try: # qemu-arm-static has no --version option so it fails, # but still prints its version plus usage cmd_runner.run(["/usr/bin/qemu-arm-static", "--version"], stdout=p.stdin).communicate() p.communicate() except: qemu_version = p.stdout.read() else: qemu_version = "Cannot find %s." % qemu_path return "%s\n: %s" % (__version__, qemu_version) def add_common_options(parser): parser.add_argument( '--extra-boot-args', dest='extra_boot_args', required=False, help='Extra boot args.') parser.add_argument( '--extra-boot-args-file', dest='extra_boot_args_file', required=False, help=('File containing extra boot arguments.')) parser.add_argument("--debug", action="store_true") def get_args_parser(): """Get the ArgumentParser for the arguments given on the command line.""" parser = argparse.ArgumentParser(version='%(prog)s ' + get_version()) group = parser.add_mutually_exclusive_group() group.add_argument( '--mmc', dest='device', default="sd.img", help='The storage device to use.') group.add_argument( '--image-file', '--image_file', dest='device', default="sd.img", help='File where we should write an image file (defaults to sd.img ' 'if neither --image-file or --mmc are specified.)') parser.add_argument( '--output-directory', dest='directory', help='Directory where image and accessories should be written to.') parser.add_argument( '--read-hwpack', dest='readhwpack', action='store_true', help=('Read the hardware pack and print information about the ' 'supported boards and bootloaders.')) parser.add_argument( '--dev', dest='dev', choices=KNOWN_BOARDS, help='Generate an SD card or image for the given board.') parser.add_argument( '--part-table', default='mbr', choices=['mbr', 'gpt'], help='Type of partition table to use for the MMC image') parser.add_argument( '--rootfs', default='ext4', choices=['ext2', 'ext3', 'ext4', 'btrfs'], help='Type of filesystem to use for the rootfs') parser.add_argument( '--rfs-label', '--rfs_label', default='rootfs', help='Label to use for the root filesystem.') parser.add_argument( '--boot-label', '--boot_label', default='boot', help='Label to use for the boot filesystem.') parser.add_argument( '--swap-file', '--swap_file', type=int, help='Create a swap file of the given size (in MBs).') group = parser.add_mutually_exclusive_group() group.add_argument( '--live', dest='is_live', action='store_true', help=('Create boot command for casper/live images; if this is not ' 'provided the UUID for the rootfs is used as the root= option')) group.add_argument( '--live-256m', dest='is_lowmem', action=Live256MegsAction, help=('Create boot command for casper/live images; adds ' 'only-ubiquity option to allow use of live installer on ' 'boards with 256M memory - like beagle.')) parser.add_argument( '--console', action='append', dest='consoles', default=[], help=('Add a console to kernel boot parameter; this parameter can be ' 'defined multiple times.')) parser.add_argument( '--hwpack', action='append', dest='hwpacks', required=True, help=('A hardware pack that should be installed in the rootfs; this ' 'parameter can be defined multiple times.')) parser.add_argument( '--hwpack-sig', action='append', dest='hwpacksigs', required=False, default=[], help=('Signature file for verifying a hwpack; this ' 'parameter can be defined multiple times.')) parser.add_argument( '--hwpack-force-yes', action='store_true', help='Pass --force-yes to linaro-hwpack-install') parser.add_argument( '--image-size', '--image_size', default='3G', help=('The image size, specified in mega/giga bytes (e.g. 3000M or ' '3G); use with --image_file only')) parser.add_argument( '--binary', default='binary-tar.tar.gz', required=False, help=('The tarball containing the rootfs used to create the bootable ' 'system.')) parser.add_argument( '--binary-sig', dest='binarysig', required=False, help=('Signature file used for verifying the binary tarball.')) parser.add_argument( '--no-rootfs', dest='should_format_rootfs', action='store_false', help='Do not deploy the root filesystem.') parser.add_argument( '--no-bootfs', dest='should_format_bootfs', action='store_false', help='Do not deploy the boot filesystem.') parser.add_argument( '--no-part', dest='should_create_partitions', action='store_false', help='Reuse existing partitions on the given media.') parser.add_argument( '--align-boot-part', dest='should_align_boot_part', action='store_true', help='Align boot partition too (might break older x-loaders).') parser.add_argument( '--nocheck-mmc', dest='nocheck_mmc', action='store_true', help=('Assume yes to the question "Are you 100%% sure, ' 'on selecting [mmc]"')) parser.add_argument( '--bootloader', help="Select a bootloader from a hardware pack that contains more " "than one. If not specified, it will default to '%s'." % DEFAULT_BOOTLOADER) parser.add_argument( '--dtb-file', help="Select a DTB file from a hardware pack that contains more " "than one. If not specified, it will default to the first " "entry in 'dtb_files' list.") add_common_options(parser) return parser def get_android_args_parser(): """Get the ArgumentParser for the arguments given on the command line.""" parser = argparse.ArgumentParser(version='%(prog)s ' + get_version()) group = parser.add_mutually_exclusive_group(required=True) group.add_argument( '--mmc', dest='device', help='The storage device to use.') group.add_argument( '--image-file', '--image_file', dest='device', help='File where we should write the image file.') parser.add_argument( '--hwpack', required=False, help=('An Android hardware pack file with the board configuration.')) parser.add_argument( '--image-size', '--image_size', default='2G', help=('The image size, specified in mega/giga bytes (e.g. 3000M or ' '3G); use with --image_file only')) parser.add_argument( '--dev', required=True, dest='dev', choices=ANDROID_KNOWN_BOARDS, help='Generate an SD card or image for the given board.') parser.add_argument( '--boot-label', '--boot_label', default='boot', help='Label to use for the boot filesystem.') parser.add_argument( '--console', action='append', dest='consoles', default=[], help=('Add a console to kernel boot parameter; this parameter can be ' 'defined multiple times.')) #group for system partition content specification group = parser.add_mutually_exclusive_group(required=True) group.add_argument( '--system', dest="system", help=('The tarball containing the Android system paritition.' 'Like system.tar.bz2')) group.add_argument( '--systemimage', dest="systemimage", help=('The ext4 filesystem data file containing the Android ' 'system paritition. Like system.img')) #group for userdata partition content specification group = parser.add_mutually_exclusive_group(required=True) group.add_argument( '--userdata', dest="userdata", help=('The tarball containing the Android data paritition.' 'Like userdata.tar.bz2')) group.add_argument( '--userdataimage', dest="userdataimage", help=('The ext4 filesystem data containing the Android ' 'data paritition. Like userdata.img')) parser.add_argument( '--boot', default='boot.tar.bz2', required=True, help=('The tarball containing the Android root partition')) parser.add_argument( '--no-part', dest='should_create_partitions', action='store_false', help='Reuse existing partitions on the given media.') parser.add_argument( '--align-boot-part', dest='should_align_boot_part', action='store_true', help='Align boot partition too (might break older x-loaders).') add_common_options(parser) return parser linaro-image-tools-2014.11/linaro_image_tools/media_create/chroot_utils.py0000664000175000017500000002272612400111666030022 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import os import sys from linaro_image_tools import cmd_runner from linaro_image_tools.utils import ( is_arm_host, find_command, ) from linaro_image_tools.hwpack.handler import HardwarepackHandler # It'd be nice if we could use atexit here, but all the things we need to undo # have to happen right after install_hwpacks completes and the atexit # functions would only be called after l-m-c.py exits. local_atexit = [] class ChrootException(Exception): """Base class for chroot exceptions.""" def prepare_chroot(chroot_dir, tmp_dir): """Prepares a chroot to run commands in it (networking and QEMU setup).""" chroot_etc = os.path.join(chroot_dir, 'etc') temporarily_overwrite_file_on_dir('/etc/resolv.conf', chroot_etc, tmp_dir) temporarily_overwrite_file_on_dir('/etc/hosts', chroot_etc, tmp_dir) if not is_arm_host(): for root, dirs, files in os.walk('/usr/bin'): for file in files: # Copy all the QEMU ARM binaries if file.startswith('qemu-arm') or \ file.startswith('qemu-aarch64'): file_name = os.path.join(root, file) copy_file(file_name, os.path.join(chroot_dir, 'usr', 'bin')) def install_hwpacks( rootfs_dir, tmp_dir, tools_dir, hwpack_force_yes, verified_files, extract_kpkgs=False, *hwpack_files): """Install the given hwpacks onto the given rootfs.""" install_command = 'linaro-hwpack-install' linaro_hwpack_install_path = find_command( install_command, prefer_dir=tools_dir) if not linaro_hwpack_install_path: raise ChrootException("The program linaro-hwpack-install could not " "be found found: cannot proceed.") else: linaro_hwpack_install_path = os.path.abspath( linaro_hwpack_install_path) # In case we just want to extract the kernel packages, don't force qemu # with chroot, as we could have archs without qemu support if not extract_kpkgs: prepare_chroot(rootfs_dir, tmp_dir) # FIXME: shouldn't use chroot/usr/bin as this might conflict with # installed packages; would be best to use some custom directory like # chroot/linaro-image-tools/bin copy_file(linaro_hwpack_install_path, os.path.join(rootfs_dir, 'usr', 'bin')) mount_chroot_proc(rootfs_dir) try: # Sometimes the host will have qemu-user-static installed but # another package (i.e. scratchbox) will have mangled its config # and thus we won't be able to chroot and install the hwpack, so # we fail here and tell the user to ensure qemu-arm-static is # setup before trying again. cmd_runner.run(['true'], as_root=True, chroot=rootfs_dir).wait() except: print ("Cannot proceed with hwpack installation because " "there doesn't seem to be a binfmt interpreter registered " "to execute arm binaries in the chroot. Please check " "that qemu-user-static is installed and properly " "configured before trying again.") raise else: # We are not in the chroot, we do not copy the linaro-hwpack-install # file, but we might not have l-i-t installed, so we need the full path # of the linaro-hwpack-install program to run. install_command = linaro_hwpack_install_path try: for hwpack_file in hwpack_files: hwpack_verified = False if os.path.basename(hwpack_file) in verified_files: hwpack_verified = True install_hwpack(rootfs_dir, hwpack_file, extract_kpkgs, hwpack_force_yes or hwpack_verified, install_command) finally: run_local_atexit_funcs() def install_hwpack(rootfs_dir, hwpack_file, extract_kpkgs, hwpack_force_yes, install_command): """Install an hwpack on the given rootfs. Copy the hwpack file to the rootfs and run linaro-hwpack-install passing that hwpack file to it. If hwpack_force_yes is True, also pass --force-yes to linaro-hwpack-install. In case extract_kpkgs is True, it will not install all the packages, but just extract the kernel ones. """ hwpack_basename = os.path.basename(hwpack_file) copy_file(hwpack_file, rootfs_dir) print "-" * 60 print "Installing (linaro-hwpack-install) %s in target rootfs." % ( hwpack_basename) # Get information required by linaro-hwpack-install with HardwarepackHandler([hwpack_file]) as hwpack: version, _ = hwpack.get_field("version") architecture, _ = hwpack.get_field("architecture") name, _ = hwpack.get_field("name") args = [install_command, '--hwpack-version', version, '--hwpack-arch', architecture, '--hwpack-name', name] if hwpack_force_yes: args.append('--force-yes') if extract_kpkgs: args.append('--extract-kernel-only') args.append(os.path.join(rootfs_dir, hwpack_basename)) chroot_dir = None else: args.append('/%s' % hwpack_basename) chroot_dir = rootfs_dir cmd_runner.run(args, as_root=True, chroot=chroot_dir).wait() print "-" * 60 def install_packages(chroot_dir, tmp_dir, *packages): """Install packages in the given chroot. This does not run apt-get update before hand.""" prepare_chroot(chroot_dir, tmp_dir) try: # TODO: Use the partition_mounted() contextmanager here and get rid of # mount_chroot_proc() altogether. mount_chroot_proc(chroot_dir) print "-" * 60 print "Installing (apt-get) %s in target rootfs." % " ".join(packages) args = ("apt-get", "--yes", "install") + packages cmd_runner.run(args, as_root=True, chroot=chroot_dir).wait() print "Cleaning up downloaded packages." args = ("apt-get", "clean") cmd_runner.run(args, as_root=True, chroot=chroot_dir).wait() print "-" * 60 finally: run_local_atexit_funcs() def mount_chroot_proc(chroot_dir): """Mount a /proc filesystem on the given chroot. Also register a function in local_atexit to unmount that /proc filesystem. """ chroot_proc = os.path.join(chroot_dir, 'proc') def umount_chroot_proc(): cmd_runner.run(['umount', '-v', chroot_proc], as_root=True).wait() local_atexit.append(umount_chroot_proc) proc = cmd_runner.run( ['mount', 'proc', chroot_proc, '-t', 'proc'], as_root=True) proc.wait() def copy_file(filepath, directory): """Copy the given file to the given directory. The copying of the file is done in a subprocess and run using sudo. We also register a function in local_atexit to remove the file from the given directory. """ cmd_runner.run(['cp', filepath, directory], as_root=True).wait() def undo(): new_path = os.path.join(directory, os.path.basename(filepath)) cmd_runner.run(['rm', '-f', new_path], as_root=True).wait() local_atexit.append(undo) def temporarily_overwrite_file_on_dir(filepath, directory, tmp_dir): """Temporarily replace a file on the given directory. We'll move the existing file on the given directory to a temp dir, then copy over the given file to that directory and register a function in local_atexit to move the orig file back to the given directory. """ basename = os.path.basename(filepath) path_to_orig = os.path.join(tmp_dir, basename) # Move the existing file from the given directory to the temp dir. oldpath = os.path.join(directory, basename) if os.path.exists(oldpath): cmd_runner.run( ['mv', '-f', oldpath, path_to_orig], as_root=True).wait() # Now copy the given file onto the given directory. cmd_runner.run(['cp', filepath, directory], as_root=True).wait() def undo(): if os.path.exists(path_to_orig): cmd_runner.run( ['mv', '-f', path_to_orig, directory], as_root=True).wait() else: cmd_runner.run( ['rm', '-f', oldpath], as_root=True).wait() local_atexit.append(undo) def run_local_atexit_funcs(): # Run the funcs in LIFO order, just like atexit does. exc_info = None while len(local_atexit) > 0: func = local_atexit.pop() try: func() except SystemExit: exc_info = sys.exc_info() except: import traceback print >> sys.stderr, "Error in local_atexit:" traceback.print_exc() exc_info = sys.exc_info() if exc_info is not None: raise exc_info[0], exc_info[1], exc_info[2] linaro-image-tools-2014.11/linaro_image_tools/media_create/unpack_binary_tarball.py0000664000175000017500000000571612400111666031632 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import re import subprocess from linaro_image_tools import cmd_runner def unpack_android_binary_tarball(tarball, unpack_dir, as_root=True): if is_tar_support_selinux(): tar_cmd = ['tar', '--selinux', '--numeric-owner', '-C', unpack_dir, '-jxf', tarball] else: tar_cmd = ['tar', '--numeric-owner', '-C', unpack_dir, '-jxf', tarball] proc = cmd_runner.run(tar_cmd, as_root=as_root, stderr=subprocess.PIPE) stderr = proc.communicate()[1] selinux_warn_outputted = False selinux_warn1 = "tar: Ignoring unknown extended header keyword" selinux_warn2 = "tar: setfileconat: Cannot set SELinux context" for line in stderr.splitlines(): # following 2 messages will not occur at the same time index = line.find(selinux_warn1) index2 = line.find(selinux_warn2) if index == -1 and index2 == -1: print line continue elif not selinux_warn_outputted: # either index != -1 or index2 != -1 print line print ("WARNING: selinux will not work correctly since the\n" " --selinux option of tar command in this OS\n" " is not fully supported\n") selinux_warn_outputted = True else: # same line of selinux_warn1 or selinux_warn2 continue return proc.returncode def unpack_binary_tarball(tarball, unpack_dir, as_root=True): extract_opt = '-xf' if tarball.endswith('.xz'): extract_opt = '-Jxf' proc = cmd_runner.run( ['tar', '--numeric-owner', '-C', unpack_dir, extract_opt, tarball], as_root=as_root) proc.wait() return proc.returncode def is_tar_support_selinux(): try: tar_help, _ = cmd_runner.Popen( ['tar', '--help'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, ).communicate() except cmd_runner.SubcommandNonZeroReturnValue as inst: return False for line in tar_help.splitlines(): selinux_support = re.search('--selinux', line) if selinux_support: return True return False linaro-image-tools-2014.11/linaro_image_tools/media_create/partitions.py0000664000175000017500000005743612400111666027506 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . from contextlib import contextmanager from math import ceil import atexit import dbus import glob import logging import re import subprocess import time from parted import ( Device, Disk, PARTITION_NORMAL, PARTITION_EXTENDED, ) from linaro_image_tools import cmd_runner logger = logging.getLogger(__name__) HEADS = 128 SECTORS = 32 SECTOR_SIZE = 512 # bytes CYLINDER_SIZE = HEADS * SECTORS * SECTOR_SIZE DBUS_PROPERTIES = 'org.freedesktop.DBus.Properties' UDISKS = "org.freedesktop.UDisks" # Max number of attempts to sleep (total sleep time in seconds = # 1+2+...+MAX_TTS) MAX_TTS = 10 # Image size should be a multiple of 1MiB, expressed in bytes. This is also # the minimum image size possible. ROUND_IMAGE_TO = 2 ** 20 MIN_IMAGE_SIZE = ROUND_IMAGE_TO def setup_android_partitions(board_config, media, image_size, bootfs_label, should_create_partitions, should_align_boot_part=False): cylinders = None if not media.is_block_device: image_size_in_bytes = get_partition_size_in_bytes(image_size) cylinders = image_size_in_bytes / CYLINDER_SIZE proc = cmd_runner.run( ['dd', 'of=%s' % media.path, 'bs=1', 'seek=%s' % image_size_in_bytes, 'count=0'], stderr=open('/dev/null', 'w')) proc.wait() if should_create_partitions: create_partitions( board_config, media, HEADS, SECTORS, cylinders, should_align_boot_part=should_align_boot_part) if media.is_block_device: bootfs, system, cache, data, sdcard = \ get_android_partitions_for_media(media, board_config) ensure_partition_is_not_mounted(bootfs) ensure_partition_is_not_mounted(system) ensure_partition_is_not_mounted(cache) ensure_partition_is_not_mounted(data) ensure_partition_is_not_mounted(sdcard) else: partitions = get_android_loopback_devices(media.path) bootfs = partitions[0] system = partitions[1] cache = partitions[2] data = partitions[3] sdcard = partitions[4] print "\nFormating boot partition\n" proc = cmd_runner.run( ['mkfs.vfat', '-F', str(board_config.fat_size), bootfs, '-n', bootfs_label], as_root=True) proc.wait() ext4_partitions = {"system": system, "cache": cache, "userdata": data} for label, dev in ext4_partitions.iteritems(): mkfs = 'mkfs.%s' % "ext4" proc = cmd_runner.run( [mkfs, dev, '-L', label], as_root=True) proc.wait() proc = cmd_runner.run( ['mkfs.vfat', '-F32', sdcard, '-n', "sdcard"], as_root=True) proc.wait() return bootfs, system, cache, data, sdcard # I wonder if it'd make sense to convert this into a small shim which calls # the appropriate function for the given type of device? I think it's still # small enough that there's not much benefit in doing that, but if it grows we # might want to do it. def setup_partitions(board_config, media, image_size, bootfs_label, rootfs_label, rootfs_type, should_create_partitions, should_format_bootfs, should_format_rootfs, should_align_boot_part=False, part_table="mbr"): """Make sure the given device is partitioned to boot the given board. :param board_config: A BoardConfig class. :param media: The Media we should partition. :param image_size: The size of the image file, in case we're setting up a QEMU image. :param bootfs_label: Label for the boot partition. :param rootfs_label: Label for the root partition. :param rootfs_type: Filesystem for the root partition. :param should_create_partitions: Whether or not we should erase existing partitions and create new ones. :param should_format_bootfs: Whether to reuse the filesystem on the boot partition. :param should_format_rootfs: Whether to reuse the filesystem on the root partition. :param should_align_boot_part: Whether to align the boot partition too. :param part_table: Type of partition table, either 'mbr' or 'gpt'. """ cylinders = None if not media.is_block_device: image_size_in_bytes = get_partition_size_in_bytes(image_size) cylinders = image_size_in_bytes / CYLINDER_SIZE proc = cmd_runner.run( ['dd', 'of=%s' % media.path, 'bs=1', 'seek=%s' % image_size_in_bytes, 'count=0'], stderr=open('/dev/null', 'w')) proc.wait() if should_create_partitions: create_partitions( board_config, media, HEADS, SECTORS, cylinders, should_align_boot_part=should_align_boot_part, part_table=part_table) if media.is_block_device: bootfs, rootfs = get_boot_and_root_partitions_for_media( media, board_config) # It looks like KDE somehow automounts the partitions after you # repartition a disk so we need to unmount them here to create the # filesystem. ensure_partition_is_not_mounted(bootfs) ensure_partition_is_not_mounted(rootfs) else: bootfs, rootfs = get_boot_and_root_loopback_devices(media.path) if should_format_bootfs: print "\nFormating boot partition\n" mkfs = 'mkfs.%s' % board_config.bootfs_type if board_config.bootfs_type == 'vfat': proc = cmd_runner.run( [mkfs, '-F', str(board_config.fat_size), bootfs, '-n', bootfs_label], as_root=True) else: proc = cmd_runner.run( [mkfs, bootfs, '-L', bootfs_label], as_root=True) proc.wait() if should_format_rootfs: print "\nFormating root partition\n" mkfs = 'mkfs.%s' % rootfs_type proc = cmd_runner.run( [mkfs, rootfs, '-L', rootfs_label], as_root=True) proc.wait() return bootfs, rootfs def umount(path): # The old code used to ignore failures here, but I don't think that's # desirable so I'm using cmd_runner.run()'s standard behaviour, which will # fail on a non-zero return value. cmd_runner.run(['umount', path], as_root=True).wait() @contextmanager def partition_mounted(device, path, *args): """A context manager that mounts the given device and umounts when done. We use a try/finally to make sure the device is umounted even if there's an uncaught exception in the with block. :param *args: Extra arguments to the mount command. """ subprocess_args = ['mount', device, path] subprocess_args.extend(args) cmd_runner.run(subprocess_args, as_root=True).wait() try: yield finally: try: umount(path) except cmd_runner.SubcommandNonZeroReturnValue, e: logger.warn("Failed to umount %s, but ignoring it because of a " "previous error" % path) logger.warn(e) def get_uuid(partition): """Find UUID of the given partition.""" proc = cmd_runner.run( ['blkid', '-o', 'udev', '-p', '-c', '/dev/null', partition], as_root=True, stdout=subprocess.PIPE) blkid_output, _ = proc.communicate() return _parse_blkid_output(blkid_output) def _parse_blkid_output(output): for line in output.splitlines(): uuid_match = re.match("ID_FS_UUID=(.*)", line) if uuid_match: return uuid_match.group(1) return None def ensure_partition_is_not_mounted(partition): """Ensure the given partition is not mounted, umounting if necessary.""" if is_partition_mounted(partition): cmd_runner.run(['umount', partition], as_root=True).wait() def is_partition_mounted(partition): """Is the given partition mounted?""" device_path = _get_udisks_device_path(partition) device = dbus.SystemBus().get_object(UDISKS, device_path) return device.Get( device_path, 'DeviceIsMounted', dbus_interface=DBUS_PROPERTIES) def get_boot_and_root_loopback_devices(image_file): """Return the boot and root loopback devices for the given image file. Register the loopback devices as well. """ vfat_size, vfat_offset, linux_size, linux_offset = ( calculate_partition_size_and_offset(image_file)) boot_device = register_loopback(image_file, vfat_offset, vfat_size) root_device = register_loopback(image_file, linux_offset, linux_size) return boot_device, root_device def get_android_loopback_devices(image_file): """Return the loopback devices for the given image file. Assumes a particular order of devices in the file. Register the loopback devices as well. """ devices = [] device_info = calculate_android_partition_size_and_offset(image_file) for device_offset, device_size in device_info: devices.append(register_loopback(image_file, device_offset, device_size)) return devices def register_loopback(image_file, offset, size): """Register a loopback device with an atexit handler to de-register it.""" def undo(device): cmd_runner.run(['losetup', '-d', device], as_root=True).wait() proc = cmd_runner.run( ['losetup', '-f', '--show', image_file, '--offset', str(offset), '--sizelimit', str(size)], stdout=subprocess.PIPE, as_root=True) device, _ = proc.communicate() device = device.strip() atexit.register(undo, device) return device def calculate_partition_size_and_offset(image_file): """Return the size and offset of the boot and root partitions. Both the size and offset are in sectors. :param image_file: A string containing the path to the image_file. :return: A 4-tuple containing the offset and size of the boot partition followed by the offset and size of the root partition. """ # Here we can use parted.Device to read the partitions because we're # reading from a regular file rather than a block device. If it was a # block device we'd need root rights. disk = Disk(Device(image_file)) vfat_partition = None linux_partition = None for partition in disk.partitions: assert partition.type == PARTITION_NORMAL, ( "Parted should only return normal partitions but got type %i" % partition.type) if 'boot' in partition.getFlagsAsString(): geometry = partition.geometry vfat_offset = geometry.start * SECTOR_SIZE vfat_size = geometry.length * SECTOR_SIZE vfat_partition = partition elif vfat_partition is not None: # next partition after boot partition is the root partition # NB: don't use vfat_partition.nextPartition() as that might return # a partition of type PARTITION_FREESPACE; it's much easier to # iterate disk.partitions which only returns # parted.PARTITION_NORMAL partitions geometry = partition.geometry linux_offset = geometry.start * SECTOR_SIZE linux_size = geometry.length * SECTOR_SIZE linux_partition = partition break assert vfat_partition is not None, ( "Couldn't find boot partition on %s" % image_file) assert linux_partition is not None, ( "Couldn't find root partition on %s" % image_file) return vfat_size, vfat_offset, linux_size, linux_offset def calculate_android_partition_size_and_offset(image_file): """Return the size and offset of the android partitions. Both the size and offset are in bytes. :param image_file: A string containing the path to the image_file. :return: A list of (offset, size) pairs. """ # Here we can use parted.Device to read the partitions because we're # reading from a regular file rather than a block device. If it was a # block device we'd need root rights. vfat_partition = None disk = Disk(Device(image_file)) partition_info = [] for partition in disk.partitions: # Will ignore any partitions before boot and of type EXTENDED if 'boot' in partition.getFlagsAsString(): vfat_partition = partition geometry = partition.geometry partition_info.append((geometry.start * SECTOR_SIZE, geometry.length * SECTOR_SIZE)) elif (vfat_partition is not None and partition.type != PARTITION_EXTENDED): geometry = partition.geometry partition_info.append((geometry.start * SECTOR_SIZE, geometry.length * SECTOR_SIZE)) # NB: don't use vfat_partition.nextPartition() as that might return # a partition of type PARTITION_FREESPACE; it's much easier to # iterate disk.partitions which only returns # parted.PARTITION_NORMAL partitions assert vfat_partition is not None, ( "Couldn't find boot partition on %s" % image_file) assert len(partition_info) == 5 return partition_info def get_android_partitions_for_media(media, board_config): """Return the device files for all the Android partitions of media. For boot we use partition number 1 plus the board's defined partition offset and for root we use partition number 2 plus the board's offset. This function must only be used for block devices. """ assert media.is_block_device, ( "This function must only be used for block devices") boot_partition = _get_device_file_for_partition_number( media.path, 1 + board_config.mmc_part_offset) system_partition = _get_device_file_for_partition_number( media.path, 2 + board_config.mmc_part_offset) if board_config.mmc_part_offset != 1: cache_partition = _get_device_file_for_partition_number( media.path, 3 + board_config.mmc_part_offset) else: # In the current setup, partition 4 is always the # extended partition container, so we need to skip 4 cache_partition = _get_device_file_for_partition_number( media.path, 5) data_partition = _get_device_file_for_partition_number( media.path, 5 + board_config.mmc_part_offset) sdcard_partition = _get_device_file_for_partition_number( media.path, 6 + board_config.mmc_part_offset) assert boot_partition is not None, ( "Could not find boot partition for %s" % media.path) assert system_partition is not None, ( "Could not find system partition for %s" % media.path) assert cache_partition is not None, ( "Could not find cache partition for %s" % media.path) assert data_partition is not None, ( "Could not find data partition for %s" % media.path) assert sdcard_partition is not None, ( "Could not find sdcard partition for %s" % media.path) return boot_partition, system_partition, cache_partition, \ data_partition, sdcard_partition def get_boot_and_root_partitions_for_media(media, board_config): """Return the device files for the boot and root partitions of media. For boot we use partition number 1 plus the board's defined partition offset and for root we use partition number 2 plus the board's offset. This function must only be used for block devices. """ assert media.is_block_device, ( "This function must only be used for block devices") boot_partition = _get_device_file_for_partition_number( media.path, 1 + board_config.mmc_part_offset) root_partition = _get_device_file_for_partition_number( media.path, 2 + board_config.mmc_part_offset) assert boot_partition is not None and root_partition is not None, ( "Could not find boot/root partition for %s" % media.path) return boot_partition, root_partition def _get_device_file_for_partition_number(device, partition): """Return the device file for the partition number on the given device. e.g. /dev/sda1 for the first partition on device /dev/sda or /dev/mmcblk0p3 for the third partition on /dev/mmcblk0. """ # This could be simpler but UDisks doesn't make it easy for us: # https://bugs.freedesktop.org/show_bug.cgi?id=33113. time_to_sleep = 1 dev_files = glob.glob("%s?*" % device) i = 0 while i < len(dev_files): dev_file = dev_files[i] try: device_path = _get_udisks_device_path(dev_file) partition_str = _get_udisks_device_file(device_path, partition) if partition_str: return partition_str i += 1 except dbus.exceptions.DBusException, e: if time_to_sleep > MAX_TTS: print "We've waited long enough..." raise print "*" * 60 print "UDisks doesn't know about %s: %s" % (dev_file, e) bus = dbus.SystemBus() manager = dbus.Interface( bus.get_object(UDISKS, "/org/freedesktop/UDisks"), UDISKS) print "This is what UDisks know about: %s" % ( manager.EnumerateDevices()) print "Sleeping for %d seconds" % time_to_sleep time.sleep(time_to_sleep) time_to_sleep += 1 print "*" * 60 return None def _get_udisks_device_path(device): """Return the UDisks path for the given device.""" bus = dbus.SystemBus() udisks = dbus.Interface( bus.get_object(UDISKS, "/org/freedesktop/UDisks"), UDISKS) return udisks.get_dbus_method('FindDeviceByDeviceFile')(device) def _get_udisks_device_file(path, part): """Return the UNIX special device file for the given partition.""" udisks_dev = dbus.SystemBus().get_object(UDISKS, path) part_number = udisks_dev.Get( path, 'PartitionNumber', dbus_interface=DBUS_PROPERTIES) if part_number == part: return str(udisks_dev.Get( path, 'DeviceFile', dbus_interface=DBUS_PROPERTIES)) def get_partition_size_in_bytes(size): """Convert a size string in Kbytes, Mbytes or Gbytes to bytes. The conversion rounds-up the size to the nearest MiB, considering a minimum size of MIN_IMAGE_SIZE bytes. The conversion always assures to have a big enough size for the partition. """ unit = size[-1].upper() real_size = float(size[:-1]) # no unit? (ends with a digit) if unit in '0123456789': real_size = float(size) elif unit == 'K': real_size = real_size * 1024 elif unit == 'M': real_size = real_size * 1024 * 1024 elif unit == 'G': real_size = real_size * 1024 * 1024 * 1024 else: raise ValueError("Unknown size format: %s. Use K[bytes], M[bytes] " "or G[bytes]" % size) # Guarantee that is a multiple of ROUND_IMAGE_TO real_size = _check_min_size(int(ceil(real_size / ROUND_IMAGE_TO) * ROUND_IMAGE_TO)) return real_size def _check_min_size(size): """Check that the image size is at least MIN_IMAGE_SIZE bytes. :param size: The size of the image to check, as a number. """ if (size < MIN_IMAGE_SIZE): size = MIN_IMAGE_SIZE return size def run_sfdisk_commands(commands, heads, sectors, cylinders, device, as_root=True, stderr=None): """Run the given commands under sfdisk. Every time sfdisk is invoked it will repartition the device so to create multiple partitions you should craft a list of newline-separated commands to be executed in a single sfdisk run. :param commands: A string of sfdisk commands; each on a separate line. :return: A 2-tuple containing the subprocess' stdout and stderr. """ # --force is unfortunate, but a consequence of having partitions not # starting on cylinder boundaries: sfdisk will abort with "Warning: # partition 2 does not start at a cylinder boundary" args = ['sfdisk', '--force', '-D', '-uS', '-H', str(heads), '-S', str(sectors)] if cylinders is not None: args.extend(['-C', str(cylinders)]) args.append(device) proc = cmd_runner.run( args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=stderr, as_root=as_root) return proc.communicate("%s\n" % commands) def run_sgdisk_commands(commands, device, as_root=True, stderr=None): args = ['sgdisk', device] args.extend(commands.split()) proc = cmd_runner.run(args, stderr=stderr, as_root=as_root) proc.wait() def create_partitions(board_config, media, heads, sectors, cylinders=None, should_align_boot_part=False, part_table="mbr"): """Partition the given media according to the board requirements. :param board_config: A BoardConfig class. :param media: A setup_partitions.Media object to partition. :param heads: Number of heads to use in the disk geometry of partitions. :param sectors: Number of sectors to use in the disk geometry of partitions. :param cylinders: The number of cylinders to pass to sfdisk's -C argument. If None the -C argument is not passed. :param should_align_boot_part: Whether to align the boot partition too. :param part_table Type of partition table, either 'mbr' or 'gpt'. """ label = 'msdos' if part_table == 'gpt': label = part_table if media.is_block_device: # Overwrite any existing partition tables with a fresh one. proc = cmd_runner.run( ['parted', '-s', media.path, 'mklabel', label], as_root=True) proc.wait() wait_partition_to_settle(media, part_table) if part_table == 'gpt': sgdisk_cmd = board_config.get_sgdisk_cmd( should_align_boot_part=should_align_boot_part) run_sgdisk_commands(sgdisk_cmd, media.path) else: # default partition table to mbr sfdisk_cmd = board_config.get_sfdisk_cmd( should_align_boot_part=should_align_boot_part) run_sfdisk_commands(sfdisk_cmd, heads, sectors, cylinders, media.path) # sleep to wait for the partition to settle. wait_partition_to_settle(media, part_table) def wait_partition_to_settle(media, part_table): """Sleep in a loop to wait partition to settle :param media: A setup_partitions.Media object to partition. """ tts = 1 while (tts > 0) and (tts <= MAX_TTS): try: logger.info("Sleeping for %s second(s) to wait " "for the partition to settle" % tts) time.sleep(tts) args = ['sfdisk', '-l', media.path] if part_table == 'gpt': args = ['sgdisk', '-L', media.path] proc = cmd_runner.run(args, as_root=True, stdout=open('/dev/null', 'w')) proc.wait() return 0 except cmd_runner.SubcommandNonZeroReturnValue: logger.info("Partition table is not available " "for device %s" % media.path) tts += 1 logger.error("Couldn't read partition table " "for a reasonable time for device %s" % media.path) raise class Media(object): """A representation of the media where Linaro will be installed.""" def __init__(self, path): self.path = path self.is_block_device = path.startswith('/dev/') linaro-image-tools-2014.11/linaro_image_tools/media_create/tests/0000775000175000017500000000000012435067365026101 5ustar fboudrafboudra00000000000000linaro-image-tools-2014.11/linaro_image_tools/media_create/tests/test_android_boards.py0000664000175000017500000005622212400111666032455 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from testtools import TestCase from linaro_image_tools.media_create.boards import ( BoardConfigException, ) from linaro_image_tools.media_create.android_boards import ( AndroidBeagleConfig, get_board_config, ) from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import CreateTempFileFixture class TestAndroidBoards(TestCase): """Class to test small things in android_boards.""" def test_get_board_config(self): instance = get_board_config('beagle') self.assertIsInstance(instance, AndroidBeagleConfig) def test_get_board_config_wrong(self): self.assertRaises(BoardConfigException, get_board_config, 'notadevice') def test_hwpack_not_exists(self): instance = get_board_config('beagle') self.assertRaises(BoardConfigException, instance.from_file, 'a_file') class TestAndroidBoardsHwpack(TestCaseWithFixtures): """Class to test the new Android hwpack configuration file.""" # All the necessary Android hwpack fields for the tests. hwpack_format = 'format: 3.0\n' hwpack_dtb_name = 'dtb_name: %(dtb_name)s\n' hwpack_fdt_high = "fdt_high: '%(fdt_high)s'\n" hwpack_fat_size = 'fat_size: 16\n' hwpack_android_args = 'android_specific_args: %(android_specific_args)s\n' hwpack_extra_serial = 'extra_serial_options: %(extra_serial_options)s\n' hwpack_extra_boot = ('extra_boot_args_options: ' '%(extra_boot_args_options)s\n') hwpack_bootloader_flavor = 'bootloader_flavor: %(bootloader_flavor)s\n' hwpack_initrd_addr = 'initrd_addr: \'%(initrd_addr)s\'\n' hwpack_initrd_high = 'initrd_high: \'%(initrd_high)s\'\n' hwpack_kernel_addr = 'kernel_addr: \'%(kernel_addr)s\'\n' hwpack_load_addr = 'load_addr: \'%(load_addr)s\'\n' hwpack_dtb_addr = 'dtb_addr: \'%(dtb_addr)s\'\n' hwpack_boot_script = 'boot_script: %(boot_script)s\n' hwpack_mmc_option = 'mmc_option: \'%(mmc_option)s\'\n' # Some defaults YAML-like strings to use for the tests. android_hwpack_base = (hwpack_format + hwpack_dtb_name) android_hwpack_simple = (android_hwpack_base + hwpack_fdt_high + hwpack_fat_size) android_hwpack_android_args = (android_hwpack_base + hwpack_android_args) android_hwpack_extra_serial = (android_hwpack_base + hwpack_extra_serial) android_hwpack_extra_boot = (android_hwpack_base + hwpack_extra_boot) android_hwpack_panda = ( hwpack_format + hwpack_android_args + hwpack_bootloader_flavor + hwpack_dtb_addr + hwpack_dtb_name + hwpack_extra_boot + hwpack_extra_serial ) android_hwpack_mx6 = ( android_hwpack_panda + hwpack_initrd_addr + hwpack_kernel_addr + hwpack_load_addr ) android_hwpack_snowball_sd = ( hwpack_android_args + hwpack_boot_script + hwpack_dtb_addr + hwpack_dtb_name + hwpack_extra_boot + hwpack_extra_serial + hwpack_fdt_high + hwpack_format + hwpack_initrd_addr ) android_hwpack_snowball_emmc = ( android_hwpack_snowball_sd + hwpack_initrd_high + hwpack_mmc_option) def setUp(self): super(TestAndroidBoardsHwpack, self).setUp() # Pick a default board. self.config = get_board_config('beagle') def _get_tmp_file_name(self, content=None): name = self.useFixture(CreateTempFileFixture(content)).get_file_name() return name def assertBootEnv(self, expected, config=None, board='beagle'): """Helper function the boot env parameters. :param config: The string containing the YAML configuration. :type str :param expected: The expected configuration. :type dict :param board: The name of the board to test. Defaults to beagle. :type str """ board_conf = get_board_config(board) if config: name = self.useFixture(CreateTempFileFixture(config)).\ get_file_name() board_conf.from_file(name) self.assertEqual(expected, board_conf._get_boot_env(consoles=[])) def test_read_from_file(self): values = {'fdt_high': '0xFFFFFFFF', 'dtb_name': 'a_name'} expected = {'format': 3.0, 'dtb_name': 'a_name', 'fdt_high': '0xFFFFFFFF', 'fat_size': 16} yaml_conf = self.android_hwpack_simple % values name = self._get_tmp_file_name(yaml_conf) conf = self.config.from_file(name) self.assertEqual(expected, conf) def test_android_specific_args(self): """The field android_specific_args should be a concatenated string.""" values = {'android_specific_args': ['init=/init', 'androidboot.console=ttyO2'], 'dtb_name': 'a_name'} yaml_conf = self.android_hwpack_android_args % values name = self._get_tmp_file_name(yaml_conf) self.config.from_file(name) expected = 'init=/init androidboot.console=ttyO2' self.assertEqual(expected, self.config.android_specific_args) def test_extra_serial_options(self): """The field extra_serial_options should be a concatenated string.""" values = {'dtb_name': 'a_name', 'extra_serial_options': ['console=tty0', 'console=ttyO2,115200n8']} yaml_conf = self.android_hwpack_extra_serial % values name = self._get_tmp_file_name(yaml_conf) self.config.from_file(name) expected = 'console=tty0 console=ttyO2,115200n8' self.assertEqual(expected, self.config.extra_serial_options) def test_extra_boot_args_options(self): """The field extra_boot_args_options should be a concatenated string. Testing presence of a field defined in the parent class.""" values = { 'dtb_name': 'a_name', 'extra_boot_args_options': ['earlyprintk', 'mem=128M@0', 'mali.mali_mem=64M@128M'] } yaml_conf = self.android_hwpack_extra_boot % values name = self._get_tmp_file_name(yaml_conf) self.config.from_file(name) expected = 'earlyprintk mem=128M@0 mali.mali_mem=64M@128M' self.assertEqual(expected, self.config.extra_boot_args_options) def test_android_mx6(self): values = { "android_specific_args": ["init=/init", "androidboot.console=%s"], "bootloader_flavor": "mx6qsabrelite", "dtb_addr": '0x11ff0000', "dtb_name": "board.dtb", "extra_boot_args_options": ["earlyprintk", "rootdelay=1", "fixrtc", "nocompcache", "di1_primary", "tve"], "extra_serial_options": ["console=%s,115200n8"], "initrd_addr": '0x12000000', "kernel_addr": '0x10000000', "load_addr": '0x10008000', } expected = { 'bootargs': 'console=ttymxc0,115200n8 ' 'rootwait ro earlyprintk rootdelay=1 fixrtc ' 'nocompcache di1_primary tve init=/init ' 'androidboot.console=ttymxc0', 'bootcmd': 'fatload mmc 0:2 0x10000000 uImage; ' 'fatload mmc 0:2 0x12000000 uInitrd; ' 'fatload mmc 0:2 0x11ff0000 board.dtb; ' 'bootm 0x10000000 0x12000000 0x11ff0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = self.android_hwpack_mx6 % values self.assertBootEnv(expected, config=config, board='mx6qsabrelite') def test_android_mx6_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttymxc0,115200n8 ' 'rootwait ro earlyprintk rootdelay=1 fixrtc ' 'nocompcache di1_primary tve init=/init ' 'androidboot.console=ttymxc0', 'bootcmd': 'fatload mmc 0:2 0x10000000 uImage; ' 'fatload mmc 0:2 0x12000000 uInitrd; ' 'fatload mmc 0:2 0x11ff0000 board.dtb; ' 'bootm 0x10000000 0x12000000 0x11ff0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='mx6qsabrelite') def test_panda(self): values = { "android_specific_args": ["init=/init", "androidboot.console=ttyO2"], "bootloader_flavor": "omap4_panda", "dtb_addr": '0x815f0000', "dtb_name": "board.dtb", "extra_boot_args_options": ["earlyprintk", "fixrtc", "nocompcache", "vram=48M", "omapfb.vram=0:24M,1:24M", "mem=456M@0x80000000", "mem=512M@0xA0000000"], "extra_serial_options": ["console=ttyO2,115200n8"], } expected = { 'bootargs': 'console=ttyO2,115200n8 ' 'rootwait ro earlyprintk fixrtc ' 'nocompcache vram=48M omapfb.vram=0:24M,1:24M ' 'mem=456M@0x80000000 mem=512M@0xA0000000 ' 'init=/init androidboot.console=ttyO2', 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80200000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = self.android_hwpack_panda % values self.assertBootEnv(expected, config=config, board='panda') def test_panda_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttyO2,115200n8 ' 'rootwait ro earlyprintk fixrtc ' 'nocompcache vram=48M omapfb.vram=0:24M,1:24M ' 'mem=456M@0x80000000 mem=512M@0xA0000000 ' 'init=/init androidboot.console=ttyO2', 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80200000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='panda') def test_android_snowball_sd(self): values = { "android_specific_args": ["init=/init", "androidboot.console=ttyAMA2"], "boot_script": "boot.scr", "dtb_addr": '0x8000000', "dtb_name": "board.dtb", "extra_boot_args_options": ["earlyprintk", "mem=128M@0", "mali.mali_mem=64M@128M", "hwmem=168M@192M", "mem=22M@360M", "mem_issw=1M@383M", "mem=640M@384M", "vmalloc=500M"], "extra_serial_options": ["console=ttyAMA2,115200n8"], "fdt_high": '0x05000000', "initrd_addr": '0x05000000', "initrd_high": '0x06000000', } expected = { 'bootargs': 'console=ttyAMA2,115200n8 ' 'rootwait ro earlyprintk ' 'mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M ' 'vmalloc=500M init=/init androidboot.console=ttyAMA2', 'bootcmd': 'fatload mmc 1:1 0x00100000 uImage; ' 'fatload mmc 1:1 0x05000000 uInitrd; ' 'fatload mmc 1:1 0x8000000 board.dtb; ' 'bootm 0x00100000 0x05000000 0x8000000', 'fdt_high': '0x05000000', 'initrd_high': '0x06000000'} config = self.android_hwpack_snowball_sd % values self.assertBootEnv(expected, config=config, board='snowball_sd') def test_android_snowball_sd_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttyAMA2,115200n8 ' 'rootwait ro earlyprintk ' 'mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M ' 'vmalloc=500M init=/init androidboot.console=ttyAMA2', 'bootcmd': 'fatload mmc 1:1 0x00100000 uImage; ' 'fatload mmc 1:1 0x05000000 uInitrd; ' 'fatload mmc 1:1 0x8000000 board.dtb; ' 'bootm 0x00100000 0x05000000 0x8000000', 'fdt_high': '0x05000000', 'initrd_high': '0x06000000'} self.assertBootEnv(expected, board='snowball_sd') def test_android_snowball_emmc(self): values = { "android_specific_args": ["init=/init", "androidboot.console=ttyAMA2"], "boot_script": "boot.scr", "dtb_addr": '0x8000000', "dtb_name": "board.dtb", "extra_boot_args_options": ["earlyprintk", "mem=128M@0", "mali.mali_mem=64M@128M", "hwmem=168M@192M", "mem=22M@360M", "mem_issw=1M@383M", "mem=640M@384M", "vmalloc=500M"], "extra_serial_options": ["console=ttyAMA2,115200n8"], "fdt_high": '0x05000000', "initrd_addr": '0x05000000', "initrd_high": '0x06000000', "mmc_option": '0:2' } expected = { 'bootargs': 'console=ttyAMA2,115200n8 ' 'rootwait ro earlyprintk ' 'mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M ' 'vmalloc=500M init=/init androidboot.console=ttyAMA2', 'bootcmd': 'fatload mmc 0:2 0x00100000 uImage; ' 'fatload mmc 0:2 0x05000000 uInitrd; ' 'fatload mmc 0:2 0x8000000 board.dtb; ' 'bootm 0x00100000 0x05000000 0x8000000', 'fdt_high': '0x05000000', 'initrd_high': '0x06000000'} config = self.android_hwpack_snowball_emmc % values self.assertBootEnv(expected, config, board='snowball_emmc') def test_android_snowball_emmc_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttyAMA2,115200n8 ' 'rootwait ro earlyprintk ' 'mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M ' 'vmalloc=500M init=/init androidboot.console=ttyAMA2', 'bootcmd': 'fatload mmc 0:2 0x00100000 uImage; ' 'fatload mmc 0:2 0x05000000 uInitrd; ' 'fatload mmc 0:2 0x8000000 board.dtb; ' 'bootm 0x00100000 0x05000000 0x8000000', 'fdt_high': '0x05000000', 'initrd_high': '0x06000000'} self.assertBootEnv(expected, board='snowball_emmc') def test_android_origen(self): values = { "extra_serial_options": ["console=tty0", "console=ttySAC2,115200n8"], "android_specific_args": ["init=/init", "androidboot.console=ttySAC2"] } expected = { 'bootargs': 'console=tty0 console=ttySAC2,115200n8 ' 'rootwait ro init=/init androidboot.console=ttySAC2', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = ((self.hwpack_format + self.hwpack_extra_serial + self.hwpack_android_args) % values) self.assertBootEnv(expected, config=config, board='origen') def test_android_origen_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=tty0 console=ttySAC2,115200n8 ' 'rootwait ro init=/init androidboot.console=ttySAC2', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='origen') def test_android_origen_quad(self): values = { "extra_serial_options": ["console=tty0", "console=ttySAC2,115200n8"], "android_specific_args": ["init=/init", "androidboot.console=ttySAC2"] } expected = { 'bootargs': 'console=tty0 console=ttySAC2,115200n8 ' 'rootwait ro init=/init androidboot.console=ttySAC2', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = ((self.hwpack_format + self.hwpack_extra_serial + self.hwpack_android_args) % values) self.assertBootEnv(expected, config=config, board='origen_quad') def test_android_origen_quad_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=tty0 console=ttySAC2,115200n8 ' 'rootwait ro init=/init androidboot.console=ttySAC2', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='origen_quad') def test_android_vexpress(self): values = { "extra_serial_options": ["console=tty0", "console=ttyAMA0,38400n8"], "android_specific_args": ["init=/init", "androidboot.console=ttyAMA0"] } expected = { 'bootargs': 'console=tty0 console=ttyAMA0,38400n8 ' 'rootwait ro init=/init androidboot.console=ttyAMA0', 'bootcmd': 'fatload mmc 0:1 0x60000000 uImage; ' 'fatload mmc 0:1 0x62000000 uInitrd; ' 'bootm 0x60000000 0x62000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = ((self.hwpack_format + self.hwpack_extra_serial + self.hwpack_android_args) % values) self.assertBootEnv(expected, config=config, board='vexpress') def test_android_vexpress_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=tty0 console=ttyAMA0,38400n8 ' 'rootwait ro init=/init androidboot.console=ttyAMA0', 'bootcmd': 'fatload mmc 0:1 0x60000000 uImage; ' 'fatload mmc 0:1 0x62000000 uInitrd; ' 'bootm 0x60000000 0x62000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='vexpress') def test_android_mx5(self): values = { "extra_boot_args_options": ["earlyprintk", "rootdelay=1", "fixrtc", "nocompcache", "di1_primary", "tve"], "extra_serial_options": ["console=%s,115200n8"], "android_specific_args": ["init=/init", "androidboot.console=%s"] } expected = { 'bootargs': 'console=ttymxc0,115200n8 ' 'rootwait ro earlyprintk rootdelay=1 fixrtc ' 'nocompcache di1_primary tve init=/init ' 'androidboot.console=ttymxc0', 'bootcmd': 'fatload mmc 0:2 0x70000000 uImage; ' 'fatload mmc 0:2 0x72000000 uInitrd; ' 'bootm 0x70000000 0x72000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = ((self.hwpack_format + self.hwpack_extra_boot + self.hwpack_extra_serial + self.hwpack_android_args) % values) self.assertBootEnv(expected, config=config, board='mx53loco') def test_android_mx5_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttymxc0,115200n8 ' 'rootwait ro earlyprintk rootdelay=1 fixrtc ' 'nocompcache di1_primary tve init=/init ' 'androidboot.console=ttymxc0', 'bootcmd': 'fatload mmc 0:2 0x70000000 uImage; ' 'fatload mmc 0:2 0x72000000 uInitrd; ' 'bootm 0x70000000 0x72000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='mx53loco') def test_android_arndale_old(self): """Test that uses values taken directly from the class. """ expected = { 'bootargs': 'ttySAC2,115200n8 rootwait ro rootdelay=3 ' 'init=/init androidboot.console=ttySAC2 ' 'console=ttySAC2 initrd=0x41000000', 'bootcmd': 'fatload mmc 0:1 0x40007000 uImage; fatload mmc 0:1 ' '0x41000000 uInitrd; fatload mmc 0:1 0x41f00000 ' 'exynos5250-arndale.dtb; bootm 0x40007000 0x41000000 ' '0x41f00000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff', } self.assertBootEnv(expected, board='arndale') linaro-image-tools-2014.11/linaro_image_tools/media_create/tests/__init__.py0000664000175000017500000000201512400111666030172 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011, 2012 Linaro # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest def test_suite(): module_names = [ 'linaro_image_tools.media_create.tests.test_media_create', 'linaro_image_tools.media_create.tests.test_android_boards', ] loader = unittest.TestLoader() suite = loader.loadTestsFromNames(module_names) return suite linaro-image-tools-2014.11/linaro_image_tools/media_create/tests/test_media_create.py0000664000175000017500000054331112400111666032105 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import atexit import glob import os import random import string import subprocess import sys import tempfile import textwrap import time import types import struct import tarfile import dbus import shutil from mock import MagicMock from StringIO import StringIO from testtools import TestCase from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.handler import HardwarepackHandler from linaro_image_tools.hwpack.packages import PackageMaker import linaro_image_tools.media_create from linaro_image_tools.media_create import ( android_boards, boards, check_device, partitions, rootfs, ) from linaro_image_tools.media_create.boards import ( SECTOR_SIZE, align_up, align_partition, get_plain_boot_script_contents, make_flashable_env, install_mx5_boot_loader, install_omap_boot_loader, make_boot_script, make_uImage, make_uInitrd, make_dtb, _get_file_matching, _get_mlo_file, _run_mkimage, BoardConfig, get_board_config, ) from linaro_image_tools.media_create.android_boards import ( AndroidSnowballEmmcConfig, ) from linaro_image_tools.media_create.chroot_utils import ( copy_file, install_hwpack, install_hwpacks, install_packages, mount_chroot_proc, prepare_chroot, run_local_atexit_funcs, temporarily_overwrite_file_on_dir, ) from linaro_image_tools.media_create.partitions import ( HEADS, MIN_IMAGE_SIZE, Media, SECTORS, _check_min_size, _get_device_file_for_partition_number, _parse_blkid_output, calculate_android_partition_size_and_offset, calculate_partition_size_and_offset, create_partitions, ensure_partition_is_not_mounted, get_android_loopback_devices, get_boot_and_root_loopback_devices, get_boot_and_root_partitions_for_media, get_partition_size_in_bytes, get_uuid, partition_mounted, run_sfdisk_commands, setup_partitions, wait_partition_to_settle, ) from linaro_image_tools.media_create.rootfs import ( append_to_fstab, create_flash_kernel_config, has_space_left_for_swap, move_contents, populate_rootfs, rootfs_mount_options, update_network_interfaces, write_data_to_protected_file, ) from linaro_image_tools.media_create.tests.fixtures import ( CreateTarballFixture, MockRunSfdiskCommandsFixture, ) from linaro_image_tools.media_create.unpack_binary_tarball import ( unpack_binary_tarball, ) from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( CreateTempDirFixture, MockCmdRunnerPopenFixture, MockSomethingFixture, ) from linaro_image_tools.utils import find_command, preferred_tools_dir from linaro_image_tools.hwpack.testing import ContextManagerFixture chroot_args = " ".join(cmd_runner.CHROOT_ARGS) sudo_args = " ".join(cmd_runner.SUDO_ARGS) class TestHardwarepackHandler(TestCaseWithFixtures): def setUp(self): super(TestHardwarepackHandler, self).setUp() self.tar_dir_fixture = CreateTempDirFixture() self.useFixture(self.tar_dir_fixture) self.tarball_fixture = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir()) self.useFixture(self.tarball_fixture) self.metadata = ( "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\nORIGIN=linaro\n") def test_hardwarepack_bootloaders(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") metadata += ("bootloaders:\n u_boot:\n file: a_file\n uefi:\n file: " "b_file\n") data = '3.0' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', metadata)]) hp = HardwarepackHandler([tarball], bootloader='u_boot') with hp: self.assertEquals(hp.get_field('bootloader_file')[0], 'a_file') def test_hardwarepack_boards(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") metadata += ("bootloaders:\n u_boot:\n file: a_file\n uefi:\n file: " "b_file\n") metadata += ("boards:\n panda:\n bootloaders:\n u_boot:\n " "file: panda_file") data = '3.0' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', metadata)]) hp = HardwarepackHandler([tarball], board='panda') with hp: self.assertEquals(hp.get_field('bootloader_file')[0], 'panda_file') def test_hardwarepack_boards_and_bootloaders(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") metadata += ("bootloaders:\n u_boot:\n file: a_file\n uefi:\n file: " "b_file\n") metadata += ("boards:\n panda:\n bootloaders:\n u_boot:\n " "file: panda_file\n uefi:\n file: " "uefi_panda_file\n") metadata += (" panda-lt:\n bootloaders:\n u_boot:\n " "file: panda_lt_file") data = '3.0' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', metadata)]) hp = HardwarepackHandler([tarball], board='panda', bootloader='uefi') with hp: self.assertEquals(hp.get_field('bootloader_file')[0], 'uefi_panda_file') def add_to_tarball(self, files, tarball=None): if tarball is None: tarball = self.tarball_fixture.get_tarball() tar_file = tarfile.open(tarball, mode='w:gz') for filename, data in files: tarinfo = tarfile.TarInfo(filename) tarinfo.size = len(data) tar_file.addfile(tarinfo, StringIO(data)) tar_file.close() return tarball def test_get_format_1(self): data = HardwarepackHandler.FORMAT_1 format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: self.assertEquals(hp.get_format(), data) def test_get_format_2(self): data = '2.0' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: self.assertEquals(hp.get_format(), data) def test_get_unknown_format_raises(self): data = '9.9' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: self.assertRaises(AssertionError, hp.get_format) def test_mixed_formats(self): format1 = "%s\n" % HardwarepackHandler.FORMAT_1 format2 = "%s\n" % HardwarepackHandler.FORMAT_2 tarball1 = self.add_to_tarball( [('FORMAT', format1), ('metadata', self.metadata)], tarball=self.tarball_fixture.get_tarball()) tarball_fixture2 = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir(), reldir='tarfile2', filename='secondtarball.tar.gz') self.useFixture(tarball_fixture2) tarball2 = self.add_to_tarball( [('FORMAT', format2), ('metadata', self.metadata)], tarball=tarball_fixture2.get_tarball()) hp = HardwarepackHandler([tarball2, tarball1]) with hp: self.assertEquals(hp.get_format(), '1.0and2.0') def test_identical_formats_ok(self): format1 = "%s\n" % HardwarepackHandler.FORMAT_2 format2 = "%s\n" % HardwarepackHandler.FORMAT_2 tarball1 = self.add_to_tarball( [('FORMAT', format1), ('metadata', self.metadata)], tarball=self.tarball_fixture.get_tarball()) tarball_fixture2 = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir(), reldir='tarfile2', filename='secondtarball.tar.gz') self.useFixture(tarball_fixture2) tarball2 = self.add_to_tarball( [('FORMAT', format2), ('metadata', self.metadata)], tarball=tarball_fixture2.get_tarball()) hp = HardwarepackHandler([tarball1, tarball2]) with hp: self.assertEquals(hp.get_format(), '2.0') def test_get_metadata(self): data = 'data to test' metadata = self.metadata + "U_BOOT=%s\n" % data tarball = self.add_to_tarball( [('metadata', metadata)]) hp = HardwarepackHandler([tarball]) with hp: test_data, _ = hp.get_field('bootloader_file') self.assertEqual(test_data, data) def test_preserves_formatters(self): data = '%s%d' metadata = self.metadata + "U_BOOT=%s\n" % data tarball = self.add_to_tarball( [('metadata', metadata)]) hp = HardwarepackHandler([tarball]) with hp: test_data, _ = hp.get_field('bootloader_file') self.assertEqual(test_data, data) def test_creates_tempdir(self): tarball = self.add_to_tarball( [('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: self.assertTrue(os.path.exists(hp.tempdir)) def test_tempfiles_are_removed(self): tempdir = None tarball = self.add_to_tarball( [('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: tempdir = hp.tempdir self.assertFalse(os.path.exists(tempdir)) def test_get_file(self): data = 'test file contents\n' file_in_archive = 'testfile' metadata = self.metadata + "%s=%s\n" % ('U_BOOT', file_in_archive) tarball = self.add_to_tarball( [('metadata', metadata), (file_in_archive, data)]) hp = HardwarepackHandler([tarball]) with hp: test_file = hp.get_file('bootloader_file') self.assertEquals(data, open(test_file, 'r').read()) def test_list_packages(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") format = "3.0\n" tarball = self.add_to_tarball([ ("FORMAT", format), ("metadata", metadata), ("pkgs/foo_1-1_all.deb", ''), ("pkgs/bar_1-1_all.deb", ''), ]) hp = HardwarepackHandler([tarball], board='panda', bootloader='uefi') with hp: packages = hp.list_packages() names = [p[1] for p in packages] self.assertIn('pkgs/foo_1-1_all.deb', names) self.assertIn('pkgs/bar_1-1_all.deb', names) self.assertEqual(len(packages), 2) def test_find_package_for(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") format = "3.0\n" tarball = self.add_to_tarball([ ("FORMAT", format), ("metadata", metadata), ("pkgs/foo_1-3_all.deb", ''), ("pkgs/foo_2-5_arm.deb", ''), ("pkgs/bar_1-3_arm.deb", ''), ]) hp = HardwarepackHandler([tarball], board='panda', bootloader='uefi') with hp: self.assertEqual(hp.find_package_for("foo")[1], "pkgs/foo_1-3_all.deb") self.assertEqual(hp.find_package_for("bar")[1], "pkgs/bar_1-3_arm.deb") self.assertEqual(hp.find_package_for("foo", version=2)[1], "pkgs/foo_2-5_arm.deb") self.assertEqual(hp.find_package_for("foo", version=2, revision=5)[1], "pkgs/foo_2-5_arm.deb") self.assertEqual(hp.find_package_for("foo", version=2, revision=5, architecture="arm")[1], "pkgs/foo_2-5_arm.deb") self.assertEqual(hp.find_package_for("foo", architecture="arm")[1], "pkgs/foo_2-5_arm.deb") self.assertEqual(hp.find_package_for("foo", architecture="all")[1], "pkgs/foo_1-3_all.deb") def test_get_file_from_package(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") format = "3.0\n" names = ['package0', 'package1', 'package2'] files = { names[0]: ["usr/lib/u-boot/omap4_panda/u-boot.img", "usr/share/doc/u-boot-linaro-omap4-panda/copyright"], names[1]: ["usr/lib/u-boot/omap4_panda/u-boot2.img", "foo/bar", "flim/flam"], names[2]: ["some/path/config"]} # Generate some test packages maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) tarball_content = [("FORMAT", format), ("metadata", metadata)] package_names = [] for package_name in names: # The files parameter to make_package is a list of files to create. # These files are text files containing package_name and their # path. Since package_name is different for each package, this # gives each file a unique content. deb_file_path = maker.make_package(package_name, '1.0', {}, files=files[package_name]) name = os.path.basename(deb_file_path) tarball_content.append((os.path.join("pkgs", name), open(deb_file_path).read())) package_names.append(name) tarball = self.add_to_tarball(tarball_content) hp = HardwarepackHandler([tarball], board='panda', bootloader='uefi') with hp: path = hp.get_file_from_package("some/path/config", "package2") self.assertTrue(path.endswith("some/path/config")) class TestSetMetadata(TestCaseWithFixtures): class MockHardwarepackHandler(HardwarepackHandler): metadata_dict = {} def __enter__(self): return self def get_field(self, field): try: return self.metadata_dict[field], None except: return None, None def get_format(self): return '2.0' def get_file(self, file_alias): return None def test_does_not_set_if_old_format(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(None, board_conf.kernel_addr) def test_sets_kernel_addr(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'kernel_addr' data_to_set = '0x8123ABCD' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.kernel_addr) def test_sets_initrd_addr(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'initrd_addr' data_to_set = '0x8123ABCD' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.initrd_addr) def test_sets_load_addr(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'load_addr' data_to_set = '0x8123ABCD' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.load_addr) def test_sets_serial_tty(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'serial_tty' data_to_set = 'ttyAA' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.serial_tty) def test_sets_wired_interfaces(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'wired_interfaces' data_to_set = 'eth0 eth1' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.wired_interfaces) def test_sets_wireless_interfaces(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'wireless_interfaces' data_to_set = 'wlan0 wl1' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.wireless_interfaces) def test_sets_mmc_id(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'mmc_id' data_to_set = '0:1' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.mmc_option) self.assertEquals(0, board_conf.mmc_device_id) self.assertEquals(0, board_conf.mmc_part_offset) def test_sets_boot_min_size(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'boot_min_size' data_to_set = '100' expected = align_up(int(data_to_set) * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(expected, board_conf.BOOT_MIN_SIZE_S) def test_sets_root_min_size(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'root_min_size' data_to_set = '3' expected = align_up(int(data_to_set) * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(expected, board_conf.ROOT_MIN_SIZE_S) def test_sets_loader_min_size(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'loader_min_size' data_to_set = '2' expected = align_up(int(data_to_set) * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(expected, board_conf.LOADER_MIN_SIZE_S) def test_sets_partition_layout_32(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'partition_layout' data_to_set = 'bootfs_rootfs' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(32, board_conf.fat_size) def test_sets_partition_layout_16(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'partition_layout' data_to_set = 'bootfs16_rootfs' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(16, board_conf.fat_size) def test_sets_partition_layout_raises(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'partition_layout' data_to_set = 'bootfs_bogus_rootfs' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() self.assertRaises( AssertionError, board_conf.set_metadata, 'ahwpack.tar.gz') def test_sets_copy_files(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'bootloader_copy_files' data_to_set = {'package': [{"source1": "dest1"}, {"source2": "dest2"}]} self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.bootloader_copy_files) class TestGetMLOFile(TestCaseWithFixtures): def test_mlo_from_new_xloader(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = os.path.join( tempdir, 'usr', 'lib', 'x-loader', 'omap3530beagle') os.makedirs(path) mlo_path = os.path.join(path, 'MLO') open(mlo_path, 'w').close() self.assertEquals( mlo_path, _get_mlo_file(tempdir)) def test_mlo_from_old_xloader(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = os.path.join(tempdir, 'usr', 'lib', 'x-loader-omap4') os.makedirs(path) mlo_path = os.path.join(path, 'MLO') open(mlo_path, 'w').close() self.assertEquals( mlo_path, _get_mlo_file(tempdir)) def test_no_mlo_found(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, _get_mlo_file, tempdir) def test_more_than_one_mlo_found(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() for directory in ['x-loader-omap4', 'x-loader-omap3']: path = os.path.join(tempdir, 'usr', 'lib', directory) os.makedirs(path) mlo_path = os.path.join(path, 'MLO') open(mlo_path, 'w').close() self.assertRaises( AssertionError, _get_mlo_file, tempdir) def _create_uboot_dir(root, flavor): path = os.path.join(root, 'usr', 'lib', 'u-boot', flavor) os.makedirs(path) return path class TestGetSMDKSPL(TestCaseWithFixtures): def setUp(self): super(TestGetSMDKSPL, self).setUp() self.config = boards.SMDKV310Config() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_no_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, self.config._get_samsung_spl, tempdir) def test_old_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'v310_mmc_spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) def test_new_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'u-boot-mmc-spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) def test_prefers_old_path(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) old_spl_path = os.path.join(path, 'v310_mmc_spl.bin') new_spl_path = os.path.join(path, 'u-boot-mmc-spl.bin') open(old_spl_path, 'w').close() open(new_spl_path, 'w').close() self.assertEquals(old_spl_path, self.config._get_samsung_spl(tempdir)) class TestGetSMDKUboot(TestCaseWithFixtures): def setUp(self): super(TestGetSMDKUboot, self).setUp() self.config = boards.SMDKV310Config() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_uses_uboot_flavour(self): chroot_dir = "chroot" uboot_file = os.path.join(chroot_dir, 'usr', 'lib', 'u-boot', self.config.bootloader_flavor, 'u-boot.bin') self.assertEquals( uboot_file, self.config._get_samsung_bootloader(chroot_dir)) class TestGetOrigenSPL(TestCaseWithFixtures): def setUp(self): super(TestGetOrigenSPL, self).setUp() self.config = boards.OrigenConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_no_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, self.config._get_samsung_spl, tempdir) def test_new_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'u-boot-mmc-spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) class TestGetOrigenUboot(TestGetSMDKUboot): def setUp(self): super(TestGetOrigenUboot, self).setUp() self.config = boards.OrigenConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 class TestGetOrigenQuadSPL(TestCaseWithFixtures): def setUp(self): super(TestGetOrigenQuadSPL, self).setUp() self.config = boards.OrigenQuadConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_no_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, self.config._get_samsung_spl, tempdir) def test_new_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'origen_quad-spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) class TestGetOrigenQuadUboot(TestGetSMDKUboot): def setUp(self): super(TestGetOrigenQuadUboot, self).setUp() self.config = boards.OrigenQuadConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 class TestGetArndaleSPL(TestCaseWithFixtures): def setUp(self): super(TestGetArndaleSPL, self).setUp() self.config = boards.ArndaleConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_no_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, self.config._get_samsung_spl, tempdir) def test_new_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'smdk5250-spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) class TestGetArndaleUboot(TestGetSMDKUboot): config = boards.ArndaleConfig class TestArndaleBootFiles(TestCaseWithFixtures): def test_arndale_make_boot_files_v2(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) board_conf = boards.ArndaleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.load_addr = '0x40008000' board_conf.boot_script = None board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) self.tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.temp_bootdir_path = os.path.join(self.tempdir, 'boot') self.temp_bl0_path = os.path.join(self.tempdir, 'lib', 'firmware', 'arndale') k_img_file = os.path.join(self.tempdir, 'vmlinuz-1-arndale') i_img_file = os.path.join(self.tempdir, 'initrd.img-1-arndale') bl0_file = os.path.join(self.temp_bl0_path, 'arndale-bl1.bin') os.makedirs(self.temp_bl0_path) open(bl0_file, 'w').close() boot_env = {'ethact': 'smc911x-0', 'initrd_high': '0xffffffff', 'ethaddr': '00:40:5c:26:0a:5b', 'fdt_high': '0xffffffff', 'bootcmd': 'fatload mmc 0:2 None uImage; bootm None', 'bootargs': 'root=UUID=test_boot_env_uuid rootwait ro'} board_conf._make_boot_files_v2( boot_env=boot_env, chroot_dir=self.tempdir, boot_dir=self.temp_bootdir_path, boot_device_or_file='boot_device_or_file', k_img_data=k_img_file, i_img_data=i_img_file, d_img_data=None) expected_commands = [ ('sudo -E dd if=%s of=boot_device_or_file bs=512 conv=notrunc ' 'seek=1' % bl0_file), ('sudo -E mkimage -A arm -O linux -T kernel -C none -a %s -e %s ' '-n Linux -d %s %s/uImage' % (board_conf.load_addr, board_conf.load_addr, k_img_file, self.temp_bootdir_path)), ('sudo -E mkimage -A arm -O linux -T ramdisk -C none -a 0 -e 0 ' '-n initramfs -d %s %s/uInitrd' % (i_img_file, self.temp_bootdir_path))] self.assertEqual(expected_commands, popen_fixture.mock.commands_executed) shutil.rmtree(self.tempdir) class TestCreateToc(TestCaseWithFixtures): ''' Tests boards.SnowballEmmcConfig.create_toc()''' def setUp(self): ''' Create a temporary directory to work in''' super(TestCreateToc, self).setUp() self.tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() #Create the test's input data structures zero = '\x00\x00\x00\x00' line1 = zero + zero + zero + zero + zero + 'b' + zero + zero + \ '\x00\x00\x00' maxint = '\xFF\xFF\xFF\x7F' minint = '\xFF\xFF\xFF\xFF' line2 = maxint + maxint + zero + minint + minint + \ 'hello' + zero + '\x00\x00\x00' line3 = '\x01\x00\x00\x00' '\x64\x00\x00\x00' + zero + \ '\x05\x00\x00\x00' '\x05\x00\x00\x00' \ 'hello' + zero + '\x00\x00\x00' self.expected = line1 + line2 + line3 self.board_conf = boards.SnowballEmmcConfig() def create_files_structure(self, src_data): ''' Creates the data structure that the tested function needs as input''' files = [] for line in src_data: files.append({'section_name': line[5], 'filename': 'N/A', 'align': line[3], 'offset': line[0], 'size': line[1], 'load_adress': 'N/A'}) return files def test_create_toc_normal_case(self): ''' Creates a toc file, and then reads the created file and compares it to precomputed data''' correct_data = [(0, 0, 0, 0, 0, 'b'), (0x7FFFFFFF, 0x7FFFFFFF, 0x7FFFFFFF, -1, -1, 'hello'), (1, 100, 1000, 5, 10, 'hello')] files = self.create_files_structure(correct_data) filename = os.path.join(self.tempdir, 'toc') with open(filename, 'w') as f: self.board_conf.create_toc(f, files) with open(filename, 'r') as f: actual = f.read() self.assertEquals(96, len(actual)) for i in range(len(actual)): self.assertEquals( self.expected[i], actual[i], 'Mismatch at ix' ' %d, ref=%c, actual=%c' % (i, self.expected[i], actual[i])) def test_create_toc_error_too_large_section_name(self): '''Verify that trying to write past the end of the section name field raises an exception''' illegal_name_data = [(0, 0, 0, 0, 0, 'Too_longName')] files = self.create_files_structure(illegal_name_data) with open(os.path.join(self.tempdir, 'toc'), 'w') as f: self.assertRaises(AssertionError, self.board_conf.create_toc, f, files) def test_create_toc_error_negative_unsigned(self): '''Verify that trying to write a negative number to an unsigned field raises an exception''' illegal_unsigned_data = [(-3, 0, 0, 0, 0, 'xxx')] files = self.create_files_structure(illegal_unsigned_data) with open(os.path.join(self.tempdir, 'toc'), 'w') as f: self.assertRaises(struct.error, self.board_conf.create_toc, f, files) class TestSnowballBootFiles(TestCaseWithFixtures): ''' Tests boards.SnowballEmmcConfig.install_snowball_boot_loader()''' ''' Tests boards.SnowballEmmcConfig._make_boot_files()''' ''' Tests boards.SnowballEmmcConfig.get_file_info()''' def setUp(self): ''' Create temporary directory to work in''' super(TestSnowballBootFiles, self).setUp() self.tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.temp_bootdir_path = os.path.join(self.tempdir, 'boot') self.temp_configdir_path = os.path.join(self.tempdir, 'startfiles') if not os.path.exists(self.temp_bootdir_path): os.makedirs(self.temp_bootdir_path) if not os.path.exists(self.temp_configdir_path): os.makedirs(self.temp_configdir_path) self.snowball_config = get_board_config('snowball_emmc') self.snowball_config.hwpack_format = HardwarepackHandler.FORMAT_1 def setupFiles(self): return self.create_test_files(self.temp_bootdir_path) def setupAndroidFiles(self): return self.create_test_files(self.temp_configdir_path) def create_test_files(self, path): ''' Adds some files in the temp dir that the tested function can use as input: * A config file, which the tested function reads to discover which binary files should be written to the loader partition. * Test versions of the binary files themselves, containing dummy data. Returns the expected value that the tested function should return, given these input files. ''' src_data = [('ISSW', 'boot_image_issw.bin', -1, 0, '5'), ('X-LOADER', 'boot_image_x-loader.bin', -1, 0, '6'), ('MEM_INIT', 'mem_init.bin', 0, 0x160000, '7'), ('PWR_MGT', 'power_management.bin', 0, 0x170000, '8'), ('NORMAL', 'u-boot.bin', 0, 0xBA0000, '9'), ('UBOOT_ENV', 'u-boot-env.bin', 0, 0x00C1F000, '10')] # Create a config file cfg_file = os.path.join( path, self.snowball_config.snowball_startup_files_config) with open(cfg_file, 'w') as f: for line in src_data: # Write comments, so we test that the parser can read them f.write('#Yet another comment\n') # Write whitespace, so we test that the parser can handle them f.write(' \n\t\n \t \t \n') f.write('%s %s %i %#x %s\n' % line) expected = [] # Define dummy binary files, containing nothing but their own # section names. for line in src_data: with open(os.path.join(path, line[1]), 'w') as f: f.write(line[0]) #define the expected values read from the config file expected = [] ofs = [self.snowball_config.TOC_SIZE, self.snowball_config.TOC_SIZE + len('ISSW'), 0x160000, 0x170000, 0xBA0000, 0xC1F000] size = [len('ISSW'), len('X-LOADER'), len('MEM_INIT'), len('PWR_MGT'), len('NORMAL'), len('UBOOT_ENV')] i = 0 for line in src_data: filename = os.path.join(path, line[1]) expected.append({'section_name': line[0], 'filename': filename, 'align': int(line[2]), 'offset': ofs[i], 'size': long(size[i]), 'load_adress': line[4]}) i += 1 return expected def test_get_file_info_relative_path(self): # Create a config file cfg_file = os.path.join( self.temp_bootdir_path, self.snowball_config.snowball_startup_files_config) uboot_file = 'u-boot.bin' with open(cfg_file, 'w') as f: f.write('%s %s %i %#x %s\n' % ('NORMAL', uboot_file, 0, 0xBA0000, '9')) with open(os.path.join(self.temp_bootdir_path, uboot_file), 'w') as f: file_info = self.snowball_config.get_file_info( self.tempdir, self.temp_bootdir_path) self.assertEquals(file_info[0]['filename'], os.path.join(self.temp_bootdir_path, uboot_file)) def test_get_file_info_abs_path(self): # Create a config file cfg_file = os.path.join( self.temp_bootdir_path, self.snowball_config.snowball_startup_files_config) uboot_dir = tempfile.mkdtemp(dir=self.tempdir) uboot_file = os.path.join(uboot_dir, 'u-boot.bin') uboot_relative_file = uboot_file.replace(self.tempdir, '') with open(cfg_file, 'w') as f: f.write('%s %s %i %#x %s\n' % ( 'NORMAL', uboot_relative_file, 0, 0xBA0000, '9')) with open(uboot_file, 'w') as f: file_info = self.snowball_config.get_file_info( self.tempdir, self.temp_bootdir_path) self.assertEquals(file_info[0]['filename'], uboot_file) def test_get_file_info_raises(self): # Create a config file cfg_file = os.path.join( self.temp_bootdir_path, self.snowball_config.snowball_startup_files_config) with open(cfg_file, 'w') as f: f.write('%s %s %i %#x %s\n' % ('NORMAL', 'u-boot.bin', 0, 0xBA0000, '9')) self.assertRaises( AssertionError, self.snowball_config.get_file_info, self.tempdir, self.temp_bootdir_path) def test_file_name_size(self): ''' Test using a to large toc file ''' _, toc_filename = tempfile.mkstemp() atexit.register(os.unlink, toc_filename) filedata = 'X' bytes = self.snowball_config.TOC_SIZE + 1 tmpfile = open(toc_filename, 'wb') for n in xrange(bytes): tmpfile.write(filedata) tmpfile.close() files = self.setupFiles() self.assertRaises( AssertionError, self.snowball_config.install_snowball_boot_loader, toc_filename, files, "boot_device_or_file", self.snowball_config.SNOWBALL_LOADER_START_S) def test_install_snowball_boot_loader_toc_dont_delete(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) toc_filename = self.createTempFileAsFixture() files = self.setupFiles() self.snowball_config.install_snowball_boot_loader( toc_filename, files, "boot_device_or_file", self.snowball_config.SNOWBALL_LOADER_START_S) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 conv=notrunc' ' seek=%s' % (sudo_args, toc_filename, self.snowball_config.SNOWBALL_LOADER_START_S), '%s dd if=%s/boot_image_issw.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=257' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot_image_x-loader.bin of=boot_device_or_file' ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/mem_init.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3072' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/power_management.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3200' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=24064' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot-env.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=25080' % (sudo_args, self.temp_bootdir_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_snowball_boot_loader_toc_delete(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) toc_filename = self.createTempFileAsFixture() files = self.setupFiles() self.snowball_config.install_snowball_boot_loader( toc_filename, files, "boot_device_or_file", self.snowball_config.SNOWBALL_LOADER_START_S, True) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 conv=notrunc' ' seek=%s' % (sudo_args, toc_filename, self.snowball_config.SNOWBALL_LOADER_START_S), '%s dd if=%s/boot_image_issw.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=257' % (sudo_args, self.temp_bootdir_path), '%s rm %s/boot_image_issw.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot_image_x-loader.bin of=boot_device_or_file' ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.temp_bootdir_path), '%s rm %s/boot_image_x-loader.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/mem_init.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3072' % (sudo_args, self.temp_bootdir_path), '%s rm %s/mem_init.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/power_management.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3200' % (sudo_args, self.temp_bootdir_path), '%s rm %s/power_management.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=24064' % (sudo_args, self.temp_bootdir_path), '%s rm %s/u-boot.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot-env.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=25080' % (sudo_args, self.temp_bootdir_path), '%s rm %s/u-boot-env.bin' % (sudo_args, self.temp_bootdir_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_snowball_boot_loader_toc_android(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) toc_filename = self.createTempFileAsFixture() files = self.setupFiles() board_conf = AndroidSnowballEmmcConfig() board_conf.install_snowball_boot_loader( toc_filename, files, "boot_device_or_file", board_conf.SNOWBALL_LOADER_START_S) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 conv=notrunc' ' seek=%s' % (sudo_args, toc_filename, board_conf.SNOWBALL_LOADER_START_S), '%s dd if=%s/boot_image_issw.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=257' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot_image_x-loader.bin of=boot_device_or_file' ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/mem_init.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3072' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/power_management.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3200' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=24064' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot-env.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=25080' % (sudo_args, self.temp_bootdir_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_snowball_make_boot_files(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture( MockSomethingFixture(tempfile, 'mkstemp', lambda: (-1, '/tmp/temp_snowball_make_boot_files'))) self.setupFiles() k_img_file = os.path.join(self.tempdir, 'vmlinuz-1-ux500') i_img_file = os.path.join(self.tempdir, 'initrd.img-1-ux500') boot_env = self.snowball_config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=test_boot_env_uuid", i_img_data=None, d_img_data=None) self.snowball_config._make_boot_files(boot_env, self.tempdir, self.temp_bootdir_path, 'boot_device_or_file', k_img_file, i_img_file, None) expected = [ '%s mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e' ' 0x00008000 -n Linux -d %s %s/boot/uImage' % (sudo_args, k_img_file, self.tempdir), '%s cp /tmp/temp_snowball_make_boot_files %s/boot/boot.txt' % (sudo_args, self.tempdir), '%s mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n boot' ' script -d %s/boot/boot.txt %s/boot/flash.scr' % (sudo_args, self.tempdir, self.tempdir), '%s dd if=/tmp/temp_snowball_make_boot_files' ' of=boot_device_or_file bs=512 conv=notrunc seek=256' % (sudo_args), '%s dd if=%s/boot/boot_image_issw.bin of=boot_device_or_file' ' bs=512 conv=notrunc seek=257' % (sudo_args, self.tempdir), '%s rm %s/boot_image_issw.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/boot_image_x-loader.bin of=boot_device_or_file' ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.tempdir), '%s rm %s/boot_image_x-loader.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/mem_init.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3072' % (sudo_args, self.tempdir), '%s rm %s/mem_init.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/power_management.bin of=boot_device_or_file' ' bs=512 conv=notrunc seek=3200' % (sudo_args, self.tempdir), '%s rm %s/power_management.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/u-boot.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=24064' % (sudo_args, self.tempdir), '%s rm %s/u-boot.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/u-boot-env.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=25080' % (sudo_args, self.tempdir), '%s rm %s/u-boot-env.bin' % (sudo_args, self.temp_bootdir_path), '%s rm /tmp/temp_snowball_make_boot_files' % (sudo_args), '%s rm %s/startfiles.cfg' % (sudo_args, self.temp_bootdir_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_missing_files(self): '''When the files cannot be read, an IOError should be raised''' self.assertRaises(IOError, self.snowball_config.get_file_info, self.tempdir, self.temp_bootdir_path) def test_normal_case(self): expected = self.setupFiles() actual = self.snowball_config.get_file_info( self.tempdir, self.temp_bootdir_path) self.assertEquals(expected, actual) class TestBootSteps(TestCaseWithFixtures): def setUp(self): super(TestBootSteps, self).setUp() self.funcs_calls = [] self.mock_all_boards_funcs() def mock_all_boards_funcs(self): """Mock functions of boards module with a call tracer.""" def mock_func_creator(name): return lambda *args, **kwargs: self.funcs_calls.append(name) for name in dir(boards): attr = getattr(boards, name) if isinstance(attr, types.FunctionType): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, name, mock_func_creator(name))) def mock_set_appropriate_serial_tty(self, config): def set_appropriate_serial_tty_mock(chroot_dir): config.serial_tty = config._serial_tty config.set_appropriate_serial_tty = MagicMock( side_effect=set_appropriate_serial_tty_mock) def make_boot_files(self, config): def _get_kflavor_files_mock(path): if config.dtb_name is None: return (path, path, None) return (path, path, path) config._get_kflavor_files = MagicMock( side_effect=_get_kflavor_files_mock) config.make_boot_files('', False, False, [], '', '', '', '') def test_vexpress_steps(self): board_conf = boards.VexpressConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.make_boot_files(board_conf) expected = ['make_uImage', 'make_uInitrd'] self.assertEqual(expected, self.funcs_calls) def test_vexpress_a9_steps(self): board_conf = boards.VexpressA9Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.make_boot_files(board_conf) expected = ['make_uImage', 'make_uInitrd'] self.assertEqual(expected, self.funcs_calls) def test_mx5_steps(self): board_conf = boards.Mx5Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.bootloader_flavor = 'bootloader_flavor' board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) self.make_boot_files(board_conf) expected = [ 'install_mx5_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_smdkv310_steps(self): board_conf = boards.SMDKV310Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.install_samsung_boot_loader = MagicMock() board_conf.install_samsung_boot_loader.return_value = \ self.funcs_calls.append('install_samsung_boot_loader') self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) self.make_boot_files(board_conf) expected = [ 'install_samsung_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_origen_steps(self): board_conf = boards.OrigenConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.install_samsung_boot_loader = MagicMock() board_conf.install_samsung_boot_loader.return_value = \ self.funcs_calls.append('install_samsung_boot_loader') self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) self.make_boot_files(board_conf) expected = [ 'install_samsung_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_origen_quad_steps(self): board_conf = boards.OrigenQuadConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.install_samsung_boot_loader = MagicMock() board_conf.install_samsung_boot_loader.return_value = \ self.funcs_calls.append('install_samsung_boot_loader') self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) self.make_boot_files(board_conf) expected = [ 'install_samsung_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_arndale_steps(self): board_conf = boards.ArndaleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.install_samsung_boot_loader = MagicMock() board_conf.install_samsung_boot_loader.return_value = \ self.funcs_calls.append('install_samsung_boot_loader') self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: '1.0') self.make_boot_files(board_conf) expected = [ 'install_samsung_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_ux500_steps(self): board_conf = boards.Ux500Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.make_boot_files(board_conf) expected = ['make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_snowball_sd_steps(self): board_conf = boards.SnowballSdConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.make_boot_files(board_conf) expected = ['make_uImage', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_panda_steps(self): board_conf = boards.PandaConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'install_omap_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_beagle_steps(self): board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'install_omap_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_igep_steps(self): board_conf = boards.IgepConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_overo_steps(self): board_conf = boards.OveroConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'install_omap_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_highbank_steps(self): board_conf = boards.HighBankConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 expected = [] self.assertEqual(expected, self.funcs_calls) def test_beaglbone_steps(self): board_conf = boards.BeagleBoneConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'install_omap_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_aa9_steps(self): board_conf = boards.Aa9Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 expected = [] self.assertEqual(expected, self.funcs_calls) class TestPopulateRawPartition(TestCaseWithFixtures): def setUp(self): super(TestPopulateRawPartition, self).setUp() self.funcs_calls = [] self.mock_all_boards_funcs() def mock_all_boards_funcs(self): """Mock functions of boards module with a call tracer.""" def mock_func_creator(name): return lambda *args, **kwargs: self.funcs_calls.append(name) for name in dir(boards): attr = getattr(boards, name) if isinstance(attr, types.FunctionType): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, name, mock_func_creator(name))) def populate_raw_partition(self, config): config.populate_raw_partition('', '') def test_snowball_config_raises(self): self.assertRaises(NotImplementedError, boards.SnowballSdConfig().snowball_config, '') def test_beagle_raw(self): self.populate_raw_partition(android_boards.AndroidBeagleConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_panda_raw(self): self.populate_raw_partition(android_boards.AndroidPandaConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_snowball_sd_raw(self): self.populate_raw_partition(boards.SnowballSdConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_snowball_emmc_raw(self): def mock_func_creator(name): return classmethod( lambda *args, **kwargs: self.funcs_calls.append(name)) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards.SnowballEmmcConfig, 'get_file_info', mock_func_creator('get_file_info'))) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards.SnowballEmmcConfig, 'create_toc', mock_func_creator('create_toc'))) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards.SnowballEmmcConfig, 'install_snowball_boot_loader', mock_func_creator('install_snowball_boot_loader'))) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards.SnowballEmmcConfig, 'delete_file', mock_func_creator('delete_file'))) self.populate_raw_partition(boards.SnowballEmmcConfig()) expected = ['get_file_info', 'create_toc', 'install_snowball_boot_loader', 'delete_file', 'delete_file'] # Test that we run the Snowball populate_raw_partition() and # delete both the toc and startfiles. self.assertEqual(expected, self.funcs_calls) def test_smdkv310_raw(self): self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(boards.SMDKV310Config()) expected = ['_dd', '_dd', '_dd'] self.assertEqual(expected, self.funcs_calls) def test_mx53loco_raw(self): self.populate_raw_partition(boards.Mx53LoCoConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_origen_raw(self): self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(boards.OrigenConfig()) expected = ['_dd', '_dd', '_dd'] self.assertEqual(expected, self.funcs_calls) def test_origen_quad_raw(self): # Need to mock this since files do not exist here, and # an Exception is raised. self.useFixture( MockSomethingFixture(os.path, 'exists', lambda exists: True)) self.populate_raw_partition(boards.OrigenQuadConfig()) expected = ['_dd', '_dd', '_dd', '_dd', '_dd'] self.assertEqual(expected, self.funcs_calls) def test_origen_quad_raises(self): board_conf = boards.OrigenQuadConfig() self.assertRaises( boards.BoardException, board_conf.populate_raw_partition, '', '') def test_arndale_raw(self): self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(boards.ArndaleConfig()) expected = ['_dd', '_dd', '_dd'] self.assertEqual(expected, self.funcs_calls) def test_vexpress_a9_raw(self): self.populate_raw_partition(boards.VexpressA9Config()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_highbank_raw(self): self.populate_raw_partition(boards.HighBankConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_beaglebone_raw(self): self.populate_raw_partition(boards.BeagleBoneConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_aa9_raw(self): self.populate_raw_partition(boards.Aa9Config()) expected = [] self.assertEqual(expected, self.funcs_calls) class TestPopulateRawPartitionAndroid(TestCaseWithFixtures): def setUp(self): super(TestPopulateRawPartitionAndroid, self).setUp() self.funcs_calls = [] def populate_raw_partition(self, config): config.populate_raw_partition('', '') def test_beagle_raw(self): self.populate_raw_partition(android_boards.AndroidBeagleConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_panda_raw(self): self.populate_raw_partition(android_boards.AndroidPandaConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_snowball_sd_raw(self): self.populate_raw_partition(android_boards.AndroidSnowballSdConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_snowball_emmc_raw(self): def mock_func_creator(name): return classmethod( lambda *args, **kwargs: self.funcs_calls.append(name)) self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) expected_commands = ['sudo -E cp boot/u-boot.bin ./startupfiles'] self.useFixture(MockSomethingFixture( android_boards.AndroidSnowballEmmcConfig, 'get_file_info', mock_func_creator('get_file_info'))) self.useFixture(MockSomethingFixture( android_boards.AndroidSnowballEmmcConfig, 'create_toc', mock_func_creator('create_toc'))) self.useFixture(MockSomethingFixture( android_boards.AndroidSnowballEmmcConfig, 'install_snowball_boot_loader', mock_func_creator('install_snowball_boot_loader'))) self.useFixture(MockSomethingFixture( android_boards.AndroidSnowballEmmcConfig, 'delete_file', mock_func_creator('delete_file'))) self.populate_raw_partition(android_boards.AndroidSnowballEmmcConfig()) expected_calls = ['get_file_info', 'create_toc', 'install_snowball_boot_loader', 'delete_file'] # Test that we copy the u-boot files to the local startupfiles dir. self.assertEqual(expected_commands, fixture.mock.commands_executed) # Test that we run the Snowball populate_raw_partition() and only # delete the toc. self.assertEqual(expected_calls, self.funcs_calls) def test_smdkv310_raw(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) expected_commands = [ 'sudo -E dd if=/dev/zero of= bs=512 conv=notrunc count=32 seek=33', ('sudo -E dd if=boot/u-boot-mmc-spl.bin of= ' 'bs=512 conv=notrunc seek=1'), 'sudo -E dd if=boot/u-boot.bin of= bs=512 conv=notrunc seek=65'] self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(android_boards.AndroidSMDKV310Config()) expected_calls = [] # Test that we dd the files self.assertEqual(expected_commands, fixture.mock.commands_executed) self.assertEqual(expected_calls, self.funcs_calls) def test_mx53loco_raw(self): self.populate_raw_partition(android_boards.AndroidMx53LoCoConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_origen_raw(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) expected_commands = [ 'sudo -E dd if=/dev/zero of= bs=512 conv=notrunc count=32 seek=33', ('sudo -E dd if=boot/u-boot-mmc-spl.bin of= bs=512 ' 'conv=notrunc seek=1'), 'sudo -E dd if=boot/u-boot.bin of= bs=512 conv=notrunc seek=65'] self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(android_boards.AndroidOrigenConfig()) expected = [] # Test that we dd the files self.assertEqual(expected_commands, fixture.mock.commands_executed) self.assertEqual(expected, self.funcs_calls) def test_origen_quad_raw(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) expected_commands = [ ('sudo -E dd if=/dev/zero of= bs=512 conv=notrunc count=32 ' 'seek=1601'), ('sudo -E dd if=boot/origen_quad.bl1.bin of= bs=512 ' 'conv=notrunc seek=1'), ('sudo -E dd if=boot/origen_quad-spl.bin.signed of= bs=512 ' 'conv=notrunc seek=31'), ('sudo -E dd if=boot/u-boot.bin of= bs=512 conv=notrunc ' 'seek=63'), ('sudo -E dd if=boot/exynos4x12.tzsw.signed.img of= bs=512 ' 'conv=notrunc seek=761') ] self.useFixture( MockSomethingFixture(os.path, 'exists', lambda exists: True)) self.populate_raw_partition(android_boards.AndroidOrigenQuadConfig()) expected = [] # Test that we dd the files self.assertEqual(expected_commands, fixture.mock.commands_executed) self.assertEqual(expected, self.funcs_calls) def test_vexpress_raw(self): self.populate_raw_partition(android_boards.AndroidVexpressConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) class TestAlignPartition(TestCase): def test_align_up_none(self): self.assertEqual(1024, align_up(1024, 1)) def test_align_up_no_rounding(self): self.assertEqual(512, align_up(512, 512)) def test_align_up_rounding(self): self.assertEqual(512, align_up(1, 512)) def test_align_partition_4_mib_4_mib(self): expected = (4 * 1024 * 1024, 8 * 1024 * 1024 - 1, 4 * 1024 * 1024) self.assertEqual(expected, align_partition(1, 1, 4 * 1024 * 1024, 4 * 1024 * 1024)) def test_align_partition_none_4_mib(self): expected = (1, 4 * 1024 * 1024 - 1, 4 * 1024 * 1024 - 1) self.assertEqual(expected, align_partition(1, 1, 1, 4 * 1024 * 1024)) class TestFixForBug697824(TestCaseWithFixtures): def mock_set_appropriate_serial_tty(self, config): def set_appropriate_serial_tty_mock(arg): self.set_appropriate_serial_tty_called = True # Need to mock all the calls done from make_boot_files in order # to be able to correctly call it. config._get_kflavor_files = MagicMock(return_value=('', '', '')) config._get_boot_env = MagicMock(return_value=None) config._make_boot_files = MagicMock() config._make_boot_files_v2 = MagicMock() config.set_appropriate_serial_tty = MagicMock( side_effect=set_appropriate_serial_tty_mock) def test_omap_make_boot_files(self): self.set_appropriate_serial_tty_called = False board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) # We don't need to worry about what's passed to make_boot_files() # because we mock the method which does the real work above and here # we're only interested in ensuring that OmapConfig.make_boot_files() # calls set_appropriate_serial_tty(). board_conf.make_boot_files( None, None, None, None, None, None, None, None) self.assertTrue( self.set_appropriate_serial_tty_called, "make_boot_files didn't call set_appropriate_serial_tty") def test_omap_make_boot_files_v2(self): self.set_appropriate_serial_tty_called = False board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_2 self.mock_set_appropriate_serial_tty(board_conf) # We don't need to worry about what's passed to make_boot_files() # because we mock the method which does the real work above and here # we're only interested in ensuring that OmapConfig.make_boot_files() # does not call set_appropriate_serial_tty(). board_conf.make_boot_files( None, None, None, None, None, None, None, None) self.assertFalse( self.set_appropriate_serial_tty_called, "make_boot_files called set_appropriate_serial_tty") def test_set_appropriate_serial_tty_old_kernel(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir boot_dir = os.path.join(tempdir, 'boot') os.makedirs(boot_dir) open(os.path.join(boot_dir, 'vmlinuz-2.6.35-23-foo'), 'w').close() board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.set_appropriate_serial_tty(tempdir) self.assertEquals('ttyS2', board_conf.serial_tty) def test_set_appropriate_serial_tty_new_kernel(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir boot_dir = os.path.join(tempdir, 'boot') os.makedirs(boot_dir) open(os.path.join(boot_dir, 'vmlinuz-2.6.36-13-foo'), 'w').close() board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.set_appropriate_serial_tty(tempdir) self.assertEquals('ttyO2', board_conf.serial_tty) def test_set_appropriate_serial_tty_three_dot_oh_kernel(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir boot_dir = os.path.join(tempdir, 'boot') os.makedirs(boot_dir) open(os.path.join(boot_dir, 'vmlinuz-3.0-13-foo'), 'w').close() board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.set_appropriate_serial_tty(tempdir) self.assertEquals('ttyO2', board_conf.serial_tty) class TestGetSfdiskCmd(TestCase): def set_up_config(self, config): config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_default(self): board_conf = BoardConfig() self.set_up_config(board_conf) self.assertEqual( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_default_aligned(self): board_conf = BoardConfig() self.set_up_config(board_conf) self.assertEqual( '8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd(should_align_boot_part=True)) def test_mx5(self): board_conf = boards.Mx5Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.assertEqual( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_snowball_sd(self): board_conf = boards.SnowballSdConfig() self.set_up_config(board_conf) self.assertEqual( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_snowball_emmc(self): board_conf = boards.SnowballEmmcConfig() self.set_up_config(board_conf) self.assertEqual( '256,7936,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_smdkv310(self): board_conf = get_board_config('smdkv310') self.set_up_config(board_conf) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_origen(self): board_conf = get_board_config('origen') self.set_up_config(board_conf) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_origen_quad(self): board_conf = get_board_config('origen_quad') self.set_up_config(board_conf) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_arndale(self): board_conf = get_board_config('arndale') self.set_up_config(board_conf) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_highbank(self): board_conf = get_board_config('highbank') self.set_up_config(board_conf) self.assertEquals( '63,106432,0x83,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_beaglebone(self): board_conf = get_board_config('beaglebone') self.set_up_config(board_conf) self.assertEquals( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_aa9(self): board_conf = get_board_config('aa9') self.set_up_config(board_conf) self.assertEquals( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_panda_android(self): self.assertEqual( '63,270272,0x0C,*\n270336,1572864,L\n1843200,524288,L\n' '2367488,-,E\n2367488,1179648,L\n3547136,,,-', android_boards.AndroidPandaConfig().get_sfdisk_cmd()) def test_origen_android(self): self.assertEqual( '1,8191,0xDA\n8253,270274,0x0C,*\n278528,1572864,L\n' '1851392,-,E\n1851392,524288,L\n2375680,1179648,L\n3555328,,,-', android_boards.AndroidOrigenConfig().get_sfdisk_cmd()) def test_origen_quad_android(self): self.assertEqual( '1,8191,0xDA\n8253,270274,0x0C,*\n278528,1572864,L\n' '1851392,-,E\n1851392,524288,L\n2375680,1179648,L\n3555328,,,-', android_boards.AndroidOrigenQuadConfig().get_sfdisk_cmd()) def test_snowball_emmc_android(self): self.assertEqual( '256,7936,0xDA\n8192,262144,0x0C,*\n270336,1572864,L\n' '1843200,-,E\n1843200,524288,L\n2367488,1179648,L\n3547136,,,-', android_boards.AndroidSnowballEmmcConfig().get_sfdisk_cmd()) def test_vexpress_android(self): self.assertEqual( '63,270272,0x0E,*\n270336,1572864,L\n1843200,524288,L\n' '2367488,-,E\n2367488,1179648,L\n3547136,,,-', android_boards.AndroidVexpressConfig().get_sfdisk_cmd()) def test_mx5_android(self): self.assertEqual( '1,8191,0xDA\n8192,262144,0x0C,*\n270336,1572864,L\n' '1843200,-,E\n1843200,524288,L\n2367488,1179648,L\n3547136,,,-', android_boards.AndroidMx53LoCoConfig().get_sfdisk_cmd()) def test_mx6_android(self): self.assertEqual( '1,8191,0xDA\n8192,262144,0x0C,*\n270336,1572864,L\n' '1843200,-,E\n1843200,524288,L\n2367488,1179648,L\n3547136,,,-', android_boards.AndroidMx6QSabreliteConfig().get_sfdisk_cmd()) class TestGetSfdiskCmdV2(TestCase): def test_mx5(self): board_conf = boards.Mx5Config() board_conf.partition_layout = 'reserved_bootfs_rootfs' self.assertEqual( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_snowball_sd(self): board_conf = boards.SnowballSdConfig() board_conf.partition_layout = 'bootfs_rootfs' self.assertEqual( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_snowball_emmc(self): board_conf = boards.SnowballEmmcConfig() board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.loader_start_s = (128 * 1024) / SECTOR_SIZE self.assertEqual( '256,7936,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_smdkv310(self): board_conf = get_board_config('smdkv310') board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.LOADER_MIN_SIZE_S = ( board_conf.samsung_bl1_start + board_conf.samsung_bl1_len + board_conf.samsung_bl2_len + board_conf.samsung_env_len) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_origen(self): board_conf = get_board_config('origen') board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.LOADER_MIN_SIZE_S = ( board_conf.samsung_bl1_start + board_conf.samsung_bl1_len + board_conf.samsung_bl2_len + board_conf.samsung_env_len) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_origen_quad(self): board_conf = get_board_config('origen_quad') board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.LOADER_MIN_SIZE_S = ( board_conf.samsung_bl1_start + board_conf.samsung_bl1_len + board_conf.samsung_bl2_len + board_conf.samsung_env_len) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_arndale(self): board_conf = get_board_config('arndale') board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.LOADER_MIN_SIZE_S = ( board_conf.samsung_bl1_start + board_conf.samsung_bl1_len + board_conf.samsung_bl2_len + board_conf.samsung_env_len) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_highbank(self): board_conf = get_board_config('highbank') board_conf.partition_layout = 'bootfs_rootfs' self.assertEquals( '63,106432,0x83,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_beaglebone(self): board_conf = get_board_config('beaglebone') board_conf.partition_layout = 'bootfs_rootfs' self.assertEquals( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_aa9(self): board_conf = get_board_config('aa9') board_conf.partition_layout = 'bootfs_rootfs' self.assertEquals( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) class TestGetBootCmd(TestCase): def test_vexpress(self): board_conf = get_board_config('vexpress') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=['ttyXXX'], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=tty0 console=ttyAMA0,38400n8 ' 'console=ttyXXX root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:1 0x60000000 uImage; ' 'fatload mmc 0:1 0x62000000 uInitrd; ' 'bootm 0x60000000 0x62000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_vexpress_a9(self): board_conf = get_board_config('vexpress-a9') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=['ttyXXX'], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=tty0 console=ttyAMA0,38400n8 ' 'console=ttyXXX root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:1 0x60000000 uImage; ' 'fatload mmc 0:1 0x62000000 uInitrd; ' 'bootm 0x60000000 0x62000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_mx51(self): board_conf = boards.Mx51Config() boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="mx51.dtb") expected = { 'bootargs': 'console=tty0 console=ttymxc0,115200n8 ' 'root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:2 0x90000000 uImage; ' 'fatload mmc 0:2 0x92000000 uInitrd; ' 'fatload mmc 0:2 0x91ff0000 board.dtb; ' 'bootm 0x90000000 0x92000000 0x91ff0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_smdkv310(self): board_conf = get_board_config('smdkv310') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=ttySAC1,115200n8 root=UUID=deadbeef ' 'rootwait ro', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'ethact': 'smc911x-0', 'ethaddr': '00:40:5c:26:0a:5b', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_origen(self): board_conf = get_board_config('origen') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=ttySAC2,115200n8 root=UUID=deadbeef ' 'rootwait ro', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_origen_quad(self): board_conf = get_board_config('origen_quad') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=ttySAC2,115200n8 root=UUID=deadbeef ' 'rootwait ro', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_arndale(self): board_conf = get_board_config('arndale') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:2 None uImage; ' 'fatload mmc 0:2 None uInitrd; ' 'bootm None None', 'ethact': 'smc911x-0', 'ethaddr': '00:40:5c:26:0a:5b', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_ux500(self): board_conf = get_board_config('ux500') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=tty0 console=ttyAMA2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk ' 'rootdelay=1 fixrtc nocompcache mem=96M@0 ' 'mem_modem=32M@96M mem=44M@128M pmem=22M@172M ' 'mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M ' 'hwmem=48M@302M mem=152M@360M', 'bootcmd': 'fatload mmc 1:1 0x00100000 uImage; ' 'fatload mmc 1:1 0x08000000 uInitrd; ' 'bootm 0x00100000 0x08000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_snowball_emmc(self): board_conf = get_board_config('snowball_emmc') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=tty0 console=ttyAMA2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk ' 'rootdelay=1 fixrtc nocompcache mem=96M@0 ' 'mem_modem=32M@96M mem=44M@128M pmem=22M@172M ' 'mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M ' 'hwmem=48M@302M mem=152M@360M', 'bootcmd': 'fatload mmc 0:2 0x00100000 uImage; ' 'fatload mmc 0:2 0x08000000 uInitrd; ' 'bootm 0x00100000 0x08000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_panda(self): # XXX: To fix bug 697824 we have to change class attributes of our # OMAP board configs, and some tests do that so to make sure they # don't interfere with us we'll reset that before doing anything. config = get_board_config('panda') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="panda.dtb") expected = { 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk fixrtc ' 'nocompcache vram=48M omapfb.vram=0:24M ' 'mem=456M@0x80000000 mem=512M@0xA0000000', 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80200000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_beagle(self): # XXX: To fix bug 697824 we have to change class attributes of our # OMAP board configs, and some tests do that so to make sure they # don't interfere with us we'll reset that before doing anything. config = get_board_config('beagle') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="beagle.dtb") expected = { 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk fixrtc ' 'nocompcache vram=12M ' 'omapfb.mode=dvi:1280x720MR-16@60 mpurate=${mpurate}', 'bootcmd': 'fatload mmc 0:1 0x80000000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80000000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_igep(self): # XXX: To fix bug 697824 we have to change class attributes of our # OMAP board configs, and some tests do that so to make sure they # don't interfere with us we'll reset that before doing anything. config = boards.IgepConfig() config.serial_tty = config._serial_tty boot_cmd = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="igep.dtb") expected = { 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk fixrtc ' 'nocompcache vram=12M ' 'omapfb.mode=dvi:1280x720MR-16@60 mpurate=${mpurate}', 'bootcmd': 'fatload mmc 0:1 0x80000000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80000000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_cmd) def test_overo(self): # XXX: To fix bug 697824 we have to change class attributes of our # OMAP board configs, and some tests do that so to make sure they # don't interfere with us we'll reset that before doing anything. config = get_board_config('overo') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="overo.dtb") expected = { 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk ' 'mpurate=${mpurate} vram=12M ' 'omapdss.def_disp=${defaultdisplay} ' 'omapfb.mode=dvi:${dvimode}', 'bootcmd': 'fatload mmc 0:1 0x80000000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80000000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_highbank(self): board_conf = get_board_config('highbank') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="board.dtb") expected = { 'bootargs': 'root=UUID=deadbeef rootwait ro', 'bootcmd': 'ext2load scsi 0:1 0x00800000 uImage; ' 'ext2load scsi 0:1 0x01000000 uInitrd; ' 'ext2load scsi 0:1 0x00001000 board.dtb; ' 'bootm 0x00800000 0x01000000 0x00001000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_beaglebone(self): config = get_board_config('beaglebone') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="board.dtb") expected = { 'bootargs': 'console=ttyO0,115200n8 ' 'root=UUID=deadbeef rootwait ro fixrtc', 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80200000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_aa9(self): config = get_board_config('aa9') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="board.dtb") expected = { 'bootargs': 'console=ttyS0,115200n8 ' 'root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:1 0x40000000 uImage; ' 'fatload mmc 0:1 0x41100000 uInitrd; ' 'fatload mmc 0:1 0x41000000 board.dtb; ' 'bootm 0x40000000 0x41100000 0x41000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) class TestExtraBootCmd(TestCaseWithFixtures): def setUp(self): super(TestExtraBootCmd, self).setUp() self.board_conf = BoardConfig() def test_extra_boot_args_options_is_picked_by_get_boot_env(self): boot_args = 'whatever' self.board_conf.extra_boot_args_options = boot_args boot_commands = self.board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=['ttyXXX'], rootfs_id="UUID=deadbeef", i_img_data=None, d_img_data=None) expected = ( 'console=ttyXXX root=UUID=deadbeef rootwait ro %s' % boot_args) self.assertEqual(expected, boot_commands['bootargs']) def test_passing_None_to_add_boot_args(self): boot_args = 'extra-args' self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args(None) self.assertEqual(boot_args, self.board_conf.extra_boot_args_options) def test_passing_string_to_add_boot_args(self): boot_args = 'extra-args' extra_args = 'user-args' self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args(extra_args) self.assertEqual( "%s %s" % (boot_args, extra_args), self.board_conf.extra_boot_args_options) def test_passing_string_to_add_boot_args_with_no_default_extra_args(self): extra_args = 'user-args' self.board_conf.add_boot_args(extra_args) self.assertEqual(extra_args, self.board_conf.extra_boot_args_options) def test_add_boot_args_from_file(self): boot_args = 'extra-args' extra_args = 'user-args' boot_arg_path = self.createTempFileAsFixture() with open(boot_arg_path, 'w') as boot_arg_file: boot_arg_file.write(extra_args) self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args_from_file(boot_arg_path) self.assertEqual( "%s %s" % (boot_args, extra_args), self.board_conf.extra_boot_args_options) def test_passing_None_to_add_boot_args_from_file(self): boot_args = 'extra-args' self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args_from_file(None) self.assertEqual(boot_args, self.board_conf.extra_boot_args_options) def test_add_boot_args_from_file_strips_whitespace_from_file(self): boot_args = 'extra-args' extra_args = 'user-args' boot_arg_path = self.createTempFileAsFixture() with open(boot_arg_path, 'w') as boot_arg_file: boot_arg_file.write('\n\n \t ' + extra_args + ' \n\n') self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args_from_file(boot_arg_path) self.assertEqual( "%s %s" % (boot_args, extra_args), self.board_conf.extra_boot_args_options) class TestUnpackBinaryTarball(TestCaseWithFixtures): def setUp(self): super(TestUnpackBinaryTarball, self).setUp() self.tar_dir_fixture = CreateTempDirFixture() self.useFixture(self.tar_dir_fixture) self.tarball_fixture = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir()) self.useFixture(self.tarball_fixture) def test_unpack_binary_tarball(self): tmp_dir = self.useFixture(CreateTempDirFixture()).get_temp_dir() rc = unpack_binary_tarball( self.tarball_fixture.get_tarball(), tmp_dir, as_root=False) self.assertEqual(rc, 0) class TestGetUuid(TestCaseWithFixtures): def setUp(self): super(TestGetUuid, self).setUp() def test_get_uuid(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) get_uuid("/dev/rootfs") self.assertEquals( ["%s blkid -o udev -p -c /dev/null /dev/rootfs" % sudo_args], fixture.mock.commands_executed) def test_parse_blkid_output(self): output = ( "ID_FS_UUID=67d641db-ea7d-4acf-9f46-5f1f8275dce2\n" "ID_FS_UUID_ENC=67d641db-ea7d-4acf-9f46-5f1f8275dce2\n" "ID_FS_TYPE=ext4\n") uuid = _parse_blkid_output(output) self.assertEquals("67d641db-ea7d-4acf-9f46-5f1f8275dce2", uuid) class TestBoards(TestCaseWithFixtures): def _mock_get_file_matching(self): self.useFixture(MockSomethingFixture( boards, '_get_file_matching', lambda regex: regex)) def _mock_Popen(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) return fixture def test_make_uImage(self): self._mock_get_file_matching() fixture = self._mock_Popen() make_uImage('load_addr', 'parts_dir/vmlinuz-*-sub_arch', 'boot_disk') expected = [ '%s mkimage -A arm -O linux -T kernel -C none -a load_addr ' '-e load_addr -n Linux -d parts_dir/vmlinuz-*-sub_arch ' 'boot_disk/uImage' % sudo_args] self.assertEqual(expected, fixture.mock.commands_executed) def test_make_uInitrd(self): self._mock_get_file_matching() fixture = self._mock_Popen() make_uInitrd('parts_dir/initrd.img-*-sub_arch', 'boot_disk') expected = [ '%s mkimage -A arm -O linux -T ramdisk -C none -a 0 -e 0 ' '-n initramfs -d parts_dir/initrd.img-*-sub_arch ' 'boot_disk/uInitrd' % sudo_args] self.assertEqual(expected, fixture.mock.commands_executed) def test_make_dtb(self): self._mock_get_file_matching() fixture = self._mock_Popen() make_dtb('parts_dir/dt-*-sub_arch/board_name.dtb', 'boot_disk') expected = [ '%s cp parts_dir/dt-*-sub_arch/board_name.dtb ' 'boot_disk/board.dtb' % sudo_args] self.assertEqual(expected, fixture.mock.commands_executed) def test_make_flashable_env_too_small_env(self): env = {'verylong': 'evenlonger'} self.assertRaises(AssertionError, make_flashable_env, env, 8) def test_make_flashable_env(self): env_file = self.createTempFileAsFixture() self.useFixture(MockSomethingFixture( tempfile, "mkstemp", lambda: (None, env_file))) env = {'a': 'b', 'x': 'y'} make_flashable_env(env, 12) with open(env_file, "r") as fd: self.assertEqual("\x80\x29\x2E\x89a=b\x00x=y\x00", fd.read()) def test_install_mx5_boot_loader(self): fixture = self._mock_Popen() imx_file = self.createTempFileAsFixture() install_mx5_boot_loader(imx_file, "boot_device_or_file", BoardConfig.LOADER_MIN_SIZE_S) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 ' 'conv=notrunc seek=2' % (sudo_args, imx_file)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_mx5_boot_loader_too_large(self): self.useFixture(MockSomethingFixture( os.path, "getsize", lambda s: (BoardConfig.LOADER_MIN_SIZE_S - 1) * SECTOR_SIZE + 1)) self.assertRaises( AssertionError, install_mx5_boot_loader, "imx_file", "boot_device_or_file", BoardConfig.LOADER_MIN_SIZE_S) def test_install_omap_boot_loader(self): fixture = self._mock_Popen() self.useFixture(MockSomethingFixture( boards, '_get_mlo_file', lambda chroot_dir: "%s/MLO" % chroot_dir)) board_conf = BoardConfig() board_conf.set_metadata([]) install_omap_boot_loader("chroot_dir", "boot_disk", board_conf) expected = [ '%s cp -v chroot_dir/MLO boot_disk' % sudo_args, 'sync'] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_smdk_u_boot(self): fixture = self._mock_Popen() board_conf = boards.SMDKV310Config() bootloader_flavor = board_conf.bootloader_flavor self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) board_conf.install_samsung_boot_loader( "%s/%s/SPL" % ("chroot_dir", bootloader_flavor), "%s/%s/uboot" % ("chroot_dir", bootloader_flavor), "boot_disk") expected = [ '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl1_start), '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl2_start)] self.assertEqual(expected, fixture.mock.commands_executed) def _set_up_board_config(self, board_name): """Internal method to set-up correctly board config, with the appropriate mock objects. :param board_name: The name of the board to set-up. :return A 3-tuple: the config, the name of the bootloader, and the value of chroot_dir. """ board_conf = get_board_config(board_name) bootloader_flavor = board_conf.bootloader_flavor # Made-up value to be used as the chroot directory. chroot_dir_value = 'chroot_dir' board_conf._get_samsung_spl = MagicMock() board_conf._get_samsung_spl.return_value = ("%s/%s/SPL" % (chroot_dir_value, bootloader_flavor)) board_conf._get_samsung_bootloader = MagicMock() board_conf._get_samsung_bootloader.return_value = ("%s/%s/uboot" % (chroot_dir_value, bootloader_flavor)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) return board_conf, bootloader_flavor, chroot_dir_value def test_install_origen_u_boot(self): fixture = self._mock_Popen() board_conf, bootloader_flavor, chroot_dir_value = \ self._set_up_board_config('origen') self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) board_conf.install_samsung_boot_loader( board_conf._get_samsung_spl(chroot_dir_value), board_conf._get_samsung_bootloader(chroot_dir_value), "boot_disk") expected = [ '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl1_start), '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl2_start)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_origen_quad_u_boot(self): fixture = self._mock_Popen() board_conf, bootloader_flavor, chroot_dir_value = \ self._set_up_board_config('origen_quad') self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) board_conf.install_samsung_boot_loader( board_conf._get_samsung_spl(chroot_dir_value), board_conf._get_samsung_bootloader(chroot_dir_value), "boot_disk") expected = [ '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl1_start), '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl2_start)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_arndale_u_boot(self): fixture = self._mock_Popen() board_conf, bootloader_flavor, chroot_dir_value = \ self._set_up_board_config('arndale') self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) board_conf.install_samsung_boot_loader( board_conf._get_samsung_spl(chroot_dir_value), board_conf._get_samsung_bootloader(chroot_dir_value), "boot_disk") expected = [ '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl1_start), '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl2_start)] self.assertEqual(expected, fixture.mock.commands_executed) def test_get_plain_boot_script_contents(self): boot_env = {'bootargs': 'mybootargs', 'bootcmd': 'mybootcmd', 'initrd_high': '0xffffffff', 'fdt_high': '0xffffffff'} boot_script_data = get_plain_boot_script_contents(boot_env) self.assertEqual(textwrap.dedent("""\ setenv initrd_high "0xffffffff" setenv fdt_high "0xffffffff" setenv bootcmd "mybootcmd" setenv bootargs "mybootargs" boot"""), boot_script_data) def test_make_boot_script(self): self.useFixture(MockSomethingFixture( tempfile, 'mkstemp', lambda: (-1, '/tmp/random-abxzr'))) tempdir = self.useFixture(CreateTempDirFixture()).tempdir self._mock_get_file_matching() fixture = self._mock_Popen() boot_script_path = os.path.join(tempdir, 'boot.scr') plain_boot_script_path = os.path.join(tempdir, 'boot.txt') boot_env = {'bootargs': 'mybootargs', 'bootcmd': 'mybootcmd', 'initrd_high': '0xffffffff', 'fdt_high': '0xffffffff'} make_boot_script(boot_env, boot_script_path) expected = [ '%s cp /tmp/random-abxzr %s' % ( sudo_args, plain_boot_script_path), '%s mkimage -A arm -O linux -T script -C none -a 0 -e 0 ' '-n boot script -d %s %s' % ( sudo_args, plain_boot_script_path, boot_script_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_get_file_matching(self): prefix = ''.join( random.choice(string.ascii_lowercase) for x in range(5)) file1 = self.createTempFileAsFixture(prefix) directory = os.path.dirname(file1) self.assertEqual( file1, _get_file_matching('%s/%s*' % (directory, prefix))) def test_get_file_matching_too_many_files_found(self): prefix = ''.join( random.choice(string.ascii_lowercase) for x in range(5)) file1 = self.createTempFileAsFixture(prefix) file2 = self.createTempFileAsFixture(prefix) directory = os.path.dirname(file1) assert directory == os.path.dirname(file2), ( "file1 and file2 should be in the same directory") self.assertRaises( ValueError, _get_file_matching, '%s/%s*' % (directory, prefix)) def test_get_kflavor_files_more_specific(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavorx = 'flavorX' flavorxy = 'flavorXY' board_conf = BoardConfig() board_conf.kernel_flavors = [flavorx, flavorxy] for f in reversed(board_conf.kernel_flavors): kfile = os.path.join(tempdir, 'vmlinuz-1-%s' % f) ifile = os.path.join(tempdir, 'initrd.img-1-%s' % f) open(kfile, "w").close() open(ifile, "w").close() self.assertEqual( (kfile, ifile, None), board_conf._get_kflavor_files(tempdir)) def test_get_dt_kflavor_files_more_specific(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavorx = 'flavorX' flavorxy = 'flavorXY' board_conf = BoardConfig() board_conf.kernel_flavors = [flavorx, flavorxy] board_conf.dtb_name = 'board_name.dtb' for f in reversed(board_conf.kernel_flavors): kfile = os.path.join(tempdir, 'vmlinuz-1-%s' % f) ifile = os.path.join(tempdir, 'initrd.img-1-%s' % f) dt = os.path.join(tempdir, 'dt-1-%s' % f) os.mkdir(dt) dfile = os.path.join(dt, board_conf.dtb_name) open(kfile, "w").close() open(ifile, "w").close() open(dfile, "w").close() self.assertEqual( (kfile, ifile, dfile), board_conf._get_kflavor_files(tempdir)) def test_get_kflavor_files_later_in_flavors(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavor1 = 'flavorXY' flavor2 = 'flavorAA' board_conf = BoardConfig() board_conf.kernel_flavors = [flavor1, flavor2] kfile = os.path.join(tempdir, 'vmlinuz-1-%s' % flavor1) ifile = os.path.join(tempdir, 'initrd.img-1-%s' % flavor1) open(kfile, "w").close() open(ifile, "w").close() self.assertEqual( (kfile, ifile, None), board_conf._get_kflavor_files(tempdir)) def test_get_dt_kflavor_files_later_in_flavors(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavor1 = 'flavorXY' flavor2 = 'flavorAA' board_conf = BoardConfig() board_conf.kernel_flavors = [flavor1, flavor2] board_conf.dtb_name = 'board_name.dtb' kfile = os.path.join(tempdir, 'vmlinuz-1-%s' % flavor1) ifile = os.path.join(tempdir, 'initrd.img-1-%s' % flavor1) dt = os.path.join(tempdir, 'dt-1-%s' % flavor1) os.mkdir(dt) dfile = os.path.join(dt, board_conf.dtb_name) open(kfile, "w").close() open(ifile, "w").close() open(dfile, "w").close() self.assertEqual( (kfile, ifile, dfile), board_conf._get_kflavor_files(tempdir)) def test_get_kflavor_files_raises_when_no_match(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavor1 = 'flavorXY' flavor2 = 'flavorAA' board_conf = BoardConfig() board_conf.kernel_flavors = [flavor1, flavor2] self.assertRaises(ValueError, board_conf._get_kflavor_files, tempdir) def test_get_file_matching_no_files_found(self): self.assertEqual( None, _get_file_matching('/foo/bar/baz/*non-existent')) def test_run_mkimage(self): # Create a fake boot script. filename = self.createTempFileAsFixture() f = open(filename, 'w') f.write("setenv bootcmd 'fatload mmc 0:1 0x80000000 uImage;\nboot") f.close() img = self.createTempFileAsFixture() # Use that fake boot script to create a boot loader using mkimage. # Send stdout to /dev/null as mkimage will print to stdout and we # don't want that. retval = _run_mkimage( 'script', '0', '0', 'boot script', filename, img, stdout=open('/dev/null', 'w'), as_root=False) self.assertEqual(0, retval) class TestCreatePartitions(TestCaseWithFixtures): media = Media('/dev/xdz') def setUp(self): super(TestCreatePartitions, self).setUp() # Stub time.sleep() as create_partitions() use that. self.orig_sleep = time.sleep time.sleep = lambda s: None def tearDown(self): super(TestCreatePartitions, self).tearDown() time.sleep = self.orig_sleep def test_create_partitions_for_mx5(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = boards.Mx5Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, self.media, HEADS, SECTORS, '') self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), 'sync', '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', HEADS, SECTORS, '', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_smdkv310(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('smdkv310') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, self.media, HEADS, SECTORS, '') self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), 'sync', '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', HEADS, SECTORS, '', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_origen(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('origen') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, self.media, HEADS, SECTORS, '') self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), 'sync', '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', HEADS, SECTORS, '', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_origen_quad(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('origen_quad') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions( board_conf, self.media, HEADS, SECTORS, '') self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), 'sync', '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', HEADS, SECTORS, '', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_arndale(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('arndale') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions( board_conf, self.media, HEADS, SECTORS, '') self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), 'sync', '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', HEADS, SECTORS, '', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_beagle(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('beagle') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, self.media, HEADS, SECTORS, '') self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), 'sync', '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) self.assertEqual( [('63,106432,0x0C,*\n106496,,,-', HEADS, SECTORS, '', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_with_img_file(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) tmpfile = self.createTempFileAsFixture() board_conf = get_board_config('beagle') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, Media(tmpfile), HEADS, SECTORS, '') # Unlike the test for partitioning of a regular block device, in this # case parted was not called as there's no existing partition table # for us to overwrite on the image file. self.assertEqual( ['%s sfdisk -l %s' % (sudo_args, tmpfile), 'sync', '%s sfdisk -l %s' % (sudo_args, tmpfile)], popen_fixture.mock.commands_executed) self.assertEqual( [('63,106432,0x0C,*\n106496,,,-', HEADS, SECTORS, '', tmpfile)], sfdisk_fixture.mock.calls) def test_run_sfdisk_commands(self): tmpfile = self.createTempFileAsFixture() proc = cmd_runner.run( ['dd', 'of=%s' % tmpfile, 'bs=1', 'seek=10M', 'count=0'], stderr=open('/dev/null', 'w')) proc.communicate() stdout, stderr = run_sfdisk_commands( '2,16063,0xDA', HEADS, SECTORS, '', tmpfile, as_root=False, stderr=subprocess.PIPE) self.assertIn('Successfully wrote the new partition table', stdout) def test_run_sfdisk_commands_raises_on_non_zero_returncode(self): tmpfile = self.createTempFileAsFixture() self.assertRaises( cmd_runner.SubcommandNonZeroReturnValue, run_sfdisk_commands, ',1,0xDA', HEADS, SECTORS, '', tmpfile, as_root=False, stderr=subprocess.PIPE) def test_wait_partitions_to_settle(self): class Namespace: pass ns = Namespace() ns.count = 0 class MockCmdRunnerPopen(object): def __call__(self, cmd, *args, **kwargs): ns.count += 1 self.returncode = 0 if ns.count < 5: raise cmd_runner.SubcommandNonZeroReturnValue(args, 1) else: return self def communicate(self, input=None): self.wait() return '', '' def wait(self): return self.returncode self.useFixture(MockCmdRunnerPopenFixture()) tmpfile = self.createTempFileAsFixture() media = Media(tmpfile) media.is_block_device = True self.assertEqual(0, wait_partition_to_settle(media, 'mbr')) def test_wait_partitions_to_settle_raises_SubcommandNonZeroReturnValue( self): def mock_run(args, as_root=False, chroot=None, stdin=None, stdout=None, stderr=None, cwd=None): raise cmd_runner.SubcommandNonZeroReturnValue(args, 1) self.useFixture(MockSomethingFixture( cmd_runner, 'run', mock_run)) tmpfile = self.createTempFileAsFixture() media = Media(tmpfile) media.is_block_device = True self.assertRaises(cmd_runner.SubcommandNonZeroReturnValue, wait_partition_to_settle, media, 'mbr') class TestPartitionSetup(TestCaseWithFixtures): def setUp(self): super(TestPartitionSetup, self).setUp() # Stub time.sleep() as create_partitions() use that. self.orig_sleep = time.sleep time.sleep = lambda s: None self.linux_image_size = 30 * 1024 ** 2 self.linux_offsets_and_sizes = [ (16384 * SECTOR_SIZE, 15746 * SECTOR_SIZE), (32768 * SECTOR_SIZE, (self.linux_image_size - 32768 * SECTOR_SIZE)) ] self.android_image_size = 256 * 1024 ** 2 # Extended partition info takes 32 sectors from the first ext partition ext_part_size = 32 self.android_offsets_and_sizes = [ (63 * SECTOR_SIZE, 32768 * SECTOR_SIZE), (32831 * SECTOR_SIZE, 65536 * SECTOR_SIZE), (98367 * SECTOR_SIZE, 65536 * SECTOR_SIZE), ((294975 + ext_part_size) * SECTOR_SIZE, (131072 - ext_part_size) * SECTOR_SIZE), ((426047 + ext_part_size) * SECTOR_SIZE, self.android_image_size - (426047 + ext_part_size) * SECTOR_SIZE) ] self.android_snowball_offsets_and_sizes = [ (8192 * SECTOR_SIZE, 24639 * SECTOR_SIZE), (32831 * SECTOR_SIZE, 65536 * SECTOR_SIZE), ((98367 + ext_part_size) * SECTOR_SIZE, (65536 - ext_part_size) * SECTOR_SIZE), (294975 * SECTOR_SIZE, 131072 * SECTOR_SIZE), ((426047 + ext_part_size) * SECTOR_SIZE, self.android_image_size - (426047 + ext_part_size) * SECTOR_SIZE) ] def tearDown(self): super(TestPartitionSetup, self).tearDown() time.sleep = self.orig_sleep def _create_tmpfile(self): # boot part at +8 MiB, root part at +16 MiB return self._create_qemu_img_with_partitions( '16384,15746,0x0C,*\n32768,,,-', '%s' % self.linux_image_size) def _create_android_tmpfile(self): # boot, system, cache, (extended), userdata and sdcard partitions return self._create_qemu_img_with_partitions( '63,32768,0x0C,*\n32831,65536,L\n98367,65536,L\n294975,-,E\n' '294975,131072,L\n426047,,,-', '%s' % self.android_image_size) def _create_snowball_android_tmpfile(self): # raw, boot, system, cache, (extended), userdata and sdcard partitions return self._create_qemu_img_with_partitions( '256,7936,0xDA\n8192,24639,0x0C,*\n32831,65536,L\n' '98367,-,E\n98367,65536,L\n294975,131072,L\n' '426047,,,-', '%s' % self.android_image_size) def test_check_min_size_small(self): """Check that we get back the minimum size as expeceted.""" self.assertEqual(MIN_IMAGE_SIZE, _check_min_size(12345)) def test_check_min_size_big(self): """Check that we get back the same value we pass.""" self.assertEqual(MIN_IMAGE_SIZE * 3, _check_min_size(3145728)) def test_convert_size_wrong_suffix(self): self.assertRaises(ValueError, get_partition_size_in_bytes, "123456H") def test_convert_size_no_suffix(self): self.assertEqual(2 ** 20, get_partition_size_in_bytes('123456')) def test_convert_size_one_mbyte(self): self.assertEqual(2 ** 20, get_partition_size_in_bytes('1M')) def test_convert_size_in_kbytes_to_bytes(self): self.assertEqual(2 * 2 ** 20, get_partition_size_in_bytes('2048K')) def test_convert_size_in_mbytes_to_bytes(self): self.assertEqual(100 * 2 ** 20, get_partition_size_in_bytes('100M')) def test_convert_size_in_gbytes_to_bytes(self): self.assertEqual(12 * 2 ** 30, get_partition_size_in_bytes('12G')) def test_convert_size_float_no_suffix(self): self.assertEqual(3 * 2 ** 20, get_partition_size_in_bytes('2348576.91')) def test_convert_size_float_in_kbytes_to_bytes(self): self.assertEqual(3 * 2 ** 20, get_partition_size_in_bytes('2345.8K')) def test_convert_size_float_in_mbytes_to_bytes_double(self): self.assertEqual(2 * 2 ** 20, get_partition_size_in_bytes('1.0000001M')) def test_convert_size_float_in_mbytes_to_bytes(self): self.assertEqual(877 * 2 ** 20, get_partition_size_in_bytes('876.123M')) def test_convert_size_float_in_gbytes_to_bytes(self): self.assertEqual(1946 * 2 ** 20, get_partition_size_in_bytes('1.9G')) def test_calculate_partition_size_and_offset(self): tmpfile = self._create_tmpfile() vfat_size, vfat_offset, linux_size, linux_offset = ( calculate_partition_size_and_offset(tmpfile)) self.assertEqual( self.linux_offsets_and_sizes, [(vfat_offset, vfat_size), (linux_offset, linux_size)]) def test_calculate_android_partition_size_and_offset(self): tmpfile = self._create_android_tmpfile() device_info = calculate_android_partition_size_and_offset(tmpfile) # We use map(None, ...) since it would catch if the lists are not of # equal length and zip() would not in all cases. for device_pair, expected_pair in map(None, device_info, self.android_offsets_and_sizes): self.assertEqual(device_pair, expected_pair) def test_calculate_snowball_android_partition_size_and_offset(self): tmpfile = self._create_snowball_android_tmpfile() device_info = calculate_android_partition_size_and_offset(tmpfile) # We use map(None, ...) since it would catch if the lists are not of # equal length and zip() would not in all cases. snowball_info = map(None, device_info, self.android_snowball_offsets_and_sizes) for device_pair, expected_pair in snowball_info: self.assertEqual(device_pair, expected_pair) def test_partition_numbering(self): # another Linux partition at +24 MiB after the boot/root parts tmpfile = self._create_qemu_img_with_partitions( '16384,15746,0x0C,*\n32768,15427,,-\n49152,,,-', '%s' % self.linux_image_size) vfat_size, vfat_offset, linux_size, linux_offset = ( calculate_partition_size_and_offset(tmpfile)) # check that the linux partition offset starts at +16 MiB so that it's # the partition immediately following the vfat one self.assertEqual(linux_offset, 32768 * 512) def test_get_boot_and_root_partitions_for_media_beagle(self): self.useFixture(MockSomethingFixture( partitions, '_get_device_file_for_partition_number', lambda dev, partition: '%s%d' % (tmpfile, partition))) tmpfile = self.createTempFileAsFixture() media = Media(tmpfile) media.is_block_device = True self.assertEqual( ("%s%d" % (tmpfile, 1), "%s%d" % (tmpfile, 2)), get_boot_and_root_partitions_for_media( media, get_board_config('beagle'))) def test_get_boot_and_root_partitions_for_media_mx5(self): self.useFixture(MockSomethingFixture( partitions, '_get_device_file_for_partition_number', lambda dev, partition: '%s%d' % (tmpfile, partition))) tmpfile = self.createTempFileAsFixture() media = Media(tmpfile) media.is_block_device = True self.assertEqual( ("%s%d" % (tmpfile, 2), "%s%d" % (tmpfile, 3)), get_boot_and_root_partitions_for_media(media, boards.Mx5Config())) def _create_qemu_img_with_partitions(self, sfdisk_commands, tempfile_size): tmpfile = self.createTempFileAsFixture() proc = cmd_runner.run( ['dd', 'of=%s' % tmpfile, 'bs=1', 'seek=%s' % tempfile_size, 'count=0'], stderr=open('/dev/null', 'w')) proc.communicate() stdout, stderr = run_sfdisk_commands( sfdisk_commands, HEADS, SECTORS, '', tmpfile, as_root=False, # Throw away stderr as sfdisk complains a lot when operating on a # qemu image. stderr=subprocess.PIPE) self.assertIn('Successfully wrote the new partition table', stdout) return tmpfile def test_ensure_partition_is_not_mounted_for_mounted_partition(self): self.useFixture(MockSomethingFixture( partitions, 'is_partition_mounted', lambda part: True)) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) ensure_partition_is_not_mounted('/dev/whatever') self.assertEqual( ['%s umount /dev/whatever' % sudo_args], popen_fixture.mock.commands_executed) def test_ensure_partition_is_not_mounted_for_umounted_partition(self): self.useFixture(MockSomethingFixture( partitions, 'is_partition_mounted', lambda part: False)) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) ensure_partition_is_not_mounted('/dev/whatever') self.assertEqual(None, popen_fixture.mock.calls) def test_get_boot_and_root_loopback_devices(self): tmpfile = self._create_tmpfile() atexit_fixture = self.useFixture(MockSomethingFixture( atexit, 'register', AtExitRegister())) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) # We can't test the return value of get_boot_and_root_loopback_devices # because it'd require running losetup as root, so we just make sure # it calls losetup correctly. get_boot_and_root_loopback_devices(tmpfile) self.assertEqual( ['%s losetup -f --show %s --offset %s --sizelimit %s' % (sudo_args, tmpfile, offset, size) for (offset, size) in self.linux_offsets_and_sizes], popen_fixture.mock.commands_executed) # get_boot_and_root_loopback_devices will also setup two exit handlers # to de-register the loopback devices set up above. self.assertEqual(2, len(atexit_fixture.mock.funcs)) popen_fixture.mock.calls = [] atexit_fixture.mock.run_funcs() # We did not really run losetup above (as it requires root) so here we # don't have a device to pass to 'losetup -d', but when a device is # setup it is passed to the atexit handler. self.assertEquals( ['%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args], popen_fixture.mock.commands_executed) def test_get_android_loopback_devices(self): tmpfile = self._create_android_tmpfile() atexit_fixture = self.useFixture(MockSomethingFixture( atexit, 'register', AtExitRegister())) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) # We can't test the return value of get_boot_and_root_loopback_devices # because it'd require running losetup as root, so we just make sure # it calls losetup correctly. get_android_loopback_devices(tmpfile) self.assertEqual( ['%s losetup -f --show %s --offset %s --sizelimit %s' % (sudo_args, tmpfile, offset, size) for (offset, size) in self.android_offsets_and_sizes], popen_fixture.mock.commands_executed) # get_boot_and_root_loopback_devices will also setup two exit handlers # to de-register the loopback devices set up above. self.assertEqual(5, len(atexit_fixture.mock.funcs)) popen_fixture.mock.calls = [] atexit_fixture.mock.run_funcs() # We did not really run losetup above (as it requires root) so here we # don't have a device to pass to 'losetup -d', but when a device is # setup it is passed to the atexit handler. self.assertEquals( ['%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args], popen_fixture.mock.commands_executed) def test_setup_partitions_for_image_file(self): # In practice we could pass an empty image file to setup_partitions, # but here we mock Popen() and thanks to that the image is not setup # (via dd) inside setup_partitions. That's why we pass an # already setup image file. tmpfile = self._create_tmpfile() popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) def ensure_partition_not_mounted(part): raise AssertionError( "ensure_partition_is_not_mounted must not be called when " "generating image files. It makes no sense to do that and " "it depends on UDisks, thus making it hard to run on a " "chroot") self.useFixture(MockSomethingFixture( partitions, 'ensure_partition_is_not_mounted', ensure_partition_not_mounted)) self.useFixture(MockSomethingFixture( partitions, 'get_boot_and_root_loopback_devices', lambda image: ('/dev/loop99', '/dev/loop98'))) board_conf = get_board_config('beagle') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 bootfs_dev, rootfs_dev = setup_partitions( board_conf, Media(tmpfile), '2G', 'boot', 'root', 'ext3', True, True, True) self.assertEqual( # This is the call that would create a 2 GiB image file. ['dd of=%s bs=1 seek=2147483648 count=0' % tmpfile, '%s sfdisk -l %s' % (sudo_args, tmpfile), # This call would partition the image file. '%s sfdisk --force -D -uS -H %s -S %s -C 1024 %s' % ( sudo_args, HEADS, SECTORS, tmpfile), # Make sure changes are written to disk. 'sync', '%s sfdisk -l %s' % (sudo_args, tmpfile), '%s mkfs.vfat -F 32 %s -n boot' % (sudo_args, bootfs_dev), '%s mkfs.ext3 %s -L root' % (sudo_args, rootfs_dev)], popen_fixture.mock.commands_executed) def test_setup_partitions_for_block_device(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) # Pretend the partitions are mounted. self.useFixture(MockSomethingFixture( partitions, 'is_partition_mounted', lambda part: True)) tmpfile = self._create_tmpfile() self.useFixture(MockSomethingFixture( partitions, '_get_device_file_for_partition_number', lambda dev, partition: '%s%d' % (tmpfile, partition))) media = Media(tmpfile) # Pretend our tmpfile is a block device. media.is_block_device = True popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) board_conf = get_board_config('beagle') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 bootfs_dev, rootfs_dev = setup_partitions( board_conf, media, '2G', 'boot', 'root', 'ext3', True, True, True) self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, tmpfile), '%s sfdisk -l %s' % (sudo_args, tmpfile), '%s sfdisk --force -D -uS -H %s -S %s %s' % ( sudo_args, HEADS, SECTORS, tmpfile), 'sync', '%s sfdisk -l %s' % (sudo_args, tmpfile), # Since the partitions are mounted, setup_partitions will umount # them before running mkfs. '%s umount %s' % (sudo_args, bootfs_dev), '%s umount %s' % (sudo_args, rootfs_dev), '%s mkfs.vfat -F 32 %s -n boot' % (sudo_args, bootfs_dev), '%s mkfs.ext3 %s -L root' % (sudo_args, rootfs_dev)], popen_fixture.mock.commands_executed) def test_get_device_file_for_partition_number_raises_DBusException(self): def mock_get_udisks_device_path(d): raise dbus.exceptions.DBusException self.useFixture(MockSomethingFixture( partitions, '_get_udisks_device_path', mock_get_udisks_device_path)) tmpfile = self.createTempFileAsFixture() partition = get_board_config('beagle').mmc_part_offset self.useFixture(MockSomethingFixture( glob, 'glob', lambda pathname: ['%s%d' % (tmpfile, partition)])) self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) media = Media(tmpfile) media.is_block_device = True self.assertRaises(dbus.exceptions.DBusException, _get_device_file_for_partition_number, media.path, partition) def test_get_device_file_for_partition_number(self): class Namespace: pass ns = Namespace() ns.count = 0 def mock_get_udisks_device_path(dev): ns.count += 1 if ns.count < 5: raise dbus.exceptions.DBusException else: return '/abc/123' def mock_get_udisks_device_file(dev, part): if ns.count < 5: raise dbus.exceptions.DBusException else: return '/abc/123' self.useFixture(MockSomethingFixture( partitions, '_get_udisks_device_path', mock_get_udisks_device_path)) self.useFixture(MockSomethingFixture( partitions, '_get_udisks_device_file', mock_get_udisks_device_file)) tmpfile = self.createTempFileAsFixture() partition = get_board_config('beagle').mmc_part_offset self.useFixture(MockSomethingFixture( glob, 'glob', lambda pathname: ['%s%d' % (tmpfile, partition)])) self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) media = Media(tmpfile) media.is_block_device = True self.assertIsNotNone(_get_device_file_for_partition_number( media.path, partition)) class TestException(Exception): """Just a test exception.""" class TestMountedPartitionContextManager(TestCaseWithFixtures): def test_basic(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) def test_func(): with partition_mounted('foo', 'bar', '-t', 'proc'): pass test_func() expected = ['%s mount foo bar -t proc' % sudo_args, 'sync', '%s umount bar' % sudo_args] self.assertEqual(expected, popen_fixture.mock.commands_executed) def test_exception_raised_inside_with_block(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) def test_func(): with partition_mounted('foo', 'bar'): raise TestException('something') try: test_func() except TestException: pass expected = ['%s mount foo bar' % sudo_args, 'sync', '%s umount bar' % sudo_args] self.assertEqual(expected, popen_fixture.mock.commands_executed) def test_umount_failure(self): # We ignore a SubcommandNonZeroReturnValue from umount because # otherwise it could shadow an exception raised inside the 'with' # block. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) def failing_umount(path): raise cmd_runner.SubcommandNonZeroReturnValue('umount', 1) self.useFixture(MockSomethingFixture( partitions, 'umount', failing_umount)) def test_func(): with partition_mounted('foo', 'bar'): pass test_func() expected = ['sudo -E mount foo bar', 'sync'] self.assertEqual(expected, popen_fixture.mock.commands_executed) class TestPopulateBoot(TestCaseWithFixtures): expected_args = ( 'chroot_dir/boot', False, False, [], 'chroot_dir', 'rootfs_id', 'boot_disk', 'boot_device_or_file') expected_args_live = ( 'chroot_dir/casper', True, False, [], 'chroot_dir', 'rootfs_id', 'boot_disk', 'boot_device_or_file') expected_calls = [ 'mkdir -p boot_disk', '%s mount boot_partition boot_disk' % sudo_args, 'sync', '%s umount boot_disk' % sudo_args] def save_args(self, *args): self.saved_args = args def prepare_config(self, config): self.config = config self.config.boot_script = 'boot_script' self.config.hardwarepack_handler = \ TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz') self.config.hardwarepack_handler.get_format = \ lambda: HardwarepackHandler.FORMAT_1 self.popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( self.config, 'make_boot_files', self.save_args)) self.config.hardwarepack_handler.get_file_from_package = \ self.get_file_from_package self.config.bootloader_copy_files = None def get_file_from_package(self, source_path, source_package): if source_package in self.config.bootloader_copy_files: for file_info in self.config.bootloader_copy_files[source_package]: if source_path in file_info: return source_path return None def prepare_config_v3(self, config): self.config = config self.config.boot_script = 'boot_script' self.config.hardwarepack_handler = \ TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz') self.config.hardwarepack_handler.get_format = lambda: '3.0' self.config.hardwarepack_handler.get_file = \ lambda file_alias: ['file1', 'file2'] self.config.hardwarepack_handler.get_file_from_package = \ self.get_file_from_package self.config.bootloader_copy_files = { "package1": [{"file1": "/boot/"}, {"file2": "/boot/grub/renamed"}]} self.popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( self.config, 'make_boot_files', self.save_args)) def call_populate_boot(self, config, is_live=False): config.populate_boot( 'chroot_dir', 'rootfs_id', 'boot_partition', 'boot_disk', 'boot_device_or_file', is_live, False, []) def test_populate_boot_live(self): self.prepare_config(BoardConfig()) self.call_populate_boot(self.config, is_live=True) self.assertEquals( self.expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args_live, self.saved_args) def test_populate_boot_regular(self): self.prepare_config(BoardConfig()) self.call_populate_boot(self.config) self.assertEquals( self.expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_boot_bootloader_flavor(self): self.prepare_config(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" self.call_populate_boot(self.config) self.assertEquals( self.expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_boot_bootloader_file_in_boot_part(self): self.prepare_config(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" self.config.bootloader_file_in_boot_part = True self.config.bootloader = "u_boot" self.call_populate_boot(self.config) expected_calls = self.expected_calls[:] expected_calls.insert( 2, '%s cp -v chroot_dir/usr/lib/u-boot/bootloader_flavor/u-boot.bin ' 'boot_disk' % sudo_args) self.assertEquals( expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_boot_bootloader_file_in_boot_part_false(self): self.prepare_config(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" self.config.bootloader_file_in_boot_part = False self.call_populate_boot(self.config) expected_calls = self.expected_calls[:] self.assertEquals( expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_bootloader_copy_files(self): self.prepare_config_v3(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" # Test that copy_files works per spec (puts stuff in boot partition) # even if bootloader not in_boot_part. self.config.bootloader_file_in_boot_part = False self.call_populate_boot(self.config) expected_calls = self.expected_calls[:] expected_calls.insert(2, '%s mkdir -p boot_disk/boot' % sudo_args) expected_calls.insert(3, '%s cp -v file1 ' 'boot_disk/boot/' % sudo_args) expected_calls.insert(4, '%s mkdir -p boot_disk/boot/grub' % sudo_args) expected_calls.insert(5, '%s cp -v file2 ' 'boot_disk/boot/grub/renamed' % sudo_args) self.assertEquals( expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_bootloader_copy_files_bootloader_set(self): self.prepare_config_v3(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" # Test that copy_files works per spec (puts stuff in boot partition) # even if bootloader not in_boot_part. self.config.bootloader_file_in_boot_part = False self.config.bootloader = "u_boot" self.call_populate_boot(self.config) expected_calls = self.expected_calls[:] expected_calls.insert(2, '%s mkdir -p boot_disk/boot' % sudo_args) expected_calls.insert(3, '%s cp -v file1 ' 'boot_disk/boot/' % sudo_args) expected_calls.insert(4, '%s mkdir -p boot_disk/boot/grub' % sudo_args) expected_calls.insert(5, '%s cp -v file2 ' 'boot_disk/boot/grub/renamed' % sudo_args) self.assertEquals( expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_boot_no_bootloader_flavor(self): self.prepare_config(BoardConfig()) self.config.bootloader_file_in_boot_part = True self.assertRaises( AssertionError, self.call_populate_boot, self.config) class TestPopulateRootFS(TestCaseWithFixtures): lines_added_to_fstab = None create_flash_kernel_config_called = False def test_populate_rootfs(self): def fake_append_to_fstab(disk, additions): self.lines_added_to_fstab = additions def fake_create_flash_kernel_config(disk, mmc_device_id, partition_offset): self.create_flash_kernel_config_called = True # Mock stdout, cmd_runner.Popen(), append_to_fstab and # create_flash_kernel_config. self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) self.useFixture(MockSomethingFixture( rootfs, 'append_to_fstab', fake_append_to_fstab)) self.useFixture(MockSomethingFixture( rootfs, 'create_flash_kernel_config', fake_create_flash_kernel_config)) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) # Store a dummy rootdisk and contents_dir in a tempdir. tempdir = self.useFixture(CreateTempDirFixture()).tempdir root_disk = os.path.join(tempdir, 'rootdisk') contents_dir = os.path.join(tempdir, 'contents') contents_bin = os.path.join(contents_dir, 'bin') contents_etc = os.path.join(contents_dir, 'etc') os.makedirs(contents_bin) os.makedirs(contents_etc) # Must mock rootfs._list_files() because populate_rootfs() uses its # return value but since we mock cmd_runner.run() _list_files() would # return an invalid value. def mock_list_files(directory): return [contents_bin, contents_etc] self.useFixture(MockSomethingFixture( rootfs, '_list_files', mock_list_files)) populate_rootfs( contents_dir, root_disk, partition='/dev/rootfs', rootfs_type='ext3', rootfs_id='UUID=uuid', should_create_swap=True, swap_size=100, mmc_device_id=0, partition_offset=0, os_release_id='ubuntu', board_config=None) self.assertEqual( ['UUID=uuid / ext3 errors=remount-ro 0 1', '/SWAP.swap none swap sw 0 0'], self.lines_added_to_fstab) self.assertEqual(True, self.create_flash_kernel_config_called) swap_file = os.path.join(root_disk, 'SWAP.swap') expected = [ '%s mount /dev/rootfs %s' % (sudo_args, root_disk), '%s mv %s %s %s' % ( sudo_args, contents_bin, contents_etc, root_disk), '%s dd if=/dev/zero of=%s bs=1M count=100' % ( sudo_args, swap_file), '%s mkswap %s' % (sudo_args, swap_file), 'sync', '%s umount %s' % (sudo_args, root_disk)] self.assertEqual(expected, popen_fixture.mock.commands_executed) def test_create_flash_kernel_config(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) tempdir = self.useFixture(CreateTempDirFixture()).tempdir create_flash_kernel_config(tempdir, mmc_device_id=0, boot_partition_number=1) calls = fixture.mock.calls self.assertEqual(1, len(calls), calls) call = calls[0] # The call writes to a tmpfile and then moves it to the, # /etc/flash-kernel.conf, so the tmpfile is the next to last in the # list of arguments stored. tmpfile = call[-2] self.assertEqual( '%s mv -f %s %s/etc/flash-kernel.conf' % ( sudo_args, tmpfile, tempdir), fixture.mock.commands_executed[0]) self.assertEqual('UBOOT_PART=/dev/mmcblk0p1', open(tmpfile).read().rstrip()) def test_list_files(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir # We don't want to mock cmd_runner.run() because we're testing the # command that it runs, but we need to monkey-patch SUDO_ARGS because # we don't want to use 'sudo' in tests. orig_sudo_args = cmd_runner.SUDO_ARGS def restore_sudo_args(): cmd_runner.SUDO_ARGS = orig_sudo_args self.addCleanup(restore_sudo_args) cmd_runner.SUDO_ARGS = [] file1 = self.createTempFileAsFixture(dir=tempdir) self.assertEqual([file1], rootfs._list_files(tempdir)) def test_move_contents(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) file1 = self.createTempFileAsFixture(dir=tempdir) def mock_list_files(directory): return [file1] self.useFixture(MockSomethingFixture( rootfs, '_list_files', mock_list_files)) move_contents(tempdir, '/tmp/') self.assertEqual(['%s mv %s /tmp/' % (sudo_args, file1)], popen_fixture.mock.commands_executed) def test_has_space_left_for_swap(self): statvfs = os.statvfs('/') space_left = statvfs.f_bavail * statvfs.f_bsize swap_size_in_megs = space_left / (1024 ** 2) self.assertTrue( has_space_left_for_swap('/', swap_size_in_megs)) def test_has_no_space_left_for_swap(self): statvfs = os.statvfs('/') space_left = statvfs.f_bavail * statvfs.f_bsize swap_size_in_megs = (space_left / (1024 ** 2)) + 1 self.assertFalse( has_space_left_for_swap('/', swap_size_in_megs)) def mock_write_data_to_protected_file(self, path, data): # Duplicate of write_data_to_protected_file() but does not sudo. _, tmpfile = tempfile.mkstemp() with open(tmpfile, 'w') as fd: fd.write(data) cmd_runner.run(['mv', '-f', tmpfile, path], as_root=False).wait() def test_update_interfaces_no_interfaces_no_update(self): self.useFixture(MockSomethingFixture( rootfs, 'write_data_to_protected_file', self.mock_write_data_to_protected_file)) tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() os.makedirs(os.path.join(tempdir, 'etc', 'network')) if_path = os.path.join(tempdir, 'etc', 'network', 'interfaces') update_network_interfaces(tempdir, BoardConfig()) self.assertFalse(os.path.exists(if_path)) def test_update_interfaces_creates_entry(self): self.useFixture(MockSomethingFixture( rootfs, 'write_data_to_protected_file', self.mock_write_data_to_protected_file)) tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() os.makedirs(os.path.join(tempdir, 'etc', 'network')) if_path = os.path.join(tempdir, 'etc', 'network', 'interfaces') board_conf = BoardConfig() board_conf.wired_interfaces = ['eth0'] expected = 'auto eth0\n' \ 'iface eth0 inet dhcp\n' update_network_interfaces(tempdir, board_conf) self.assertEqual(expected, open(if_path).read()) def test_update_interfaces_creates_entries(self): self.useFixture(MockSomethingFixture( rootfs, 'write_data_to_protected_file', self.mock_write_data_to_protected_file)) tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() os.makedirs(os.path.join(tempdir, 'etc', 'network')) if_path = os.path.join(tempdir, 'etc', 'network', 'interfaces') board_conf = BoardConfig() board_conf.wired_interfaces = ['eth0', 'eth1'] board_conf.wireless_interfaces = ['wlan0'] expected = 'auto %(if)s\n' \ 'iface %(if)s inet dhcp\n' update_network_interfaces(tempdir, board_conf) self.assertIn(expected % {'if': 'eth1'}, open(if_path).read()) self.assertIn(expected % {'if': 'eth0'}, open(if_path).read()) self.assertIn(expected % {'if': 'wlan0'}, open(if_path).read()) def test_update_interfaces_leaves_original(self): self.useFixture(MockSomethingFixture( rootfs, 'write_data_to_protected_file', self.mock_write_data_to_protected_file)) tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() os.makedirs(os.path.join(tempdir, 'etc', 'network')) if_path = os.path.join(tempdir, 'etc', 'network', 'interfaces') with open(if_path, 'w') as interfaces: interfaces.write('Original contents of file.\n') board_conf = BoardConfig() board_conf.wired_interfaces = ['eth0'] expected = 'Original contents of file.\n' \ 'auto eth0\n' \ 'iface eth0 inet dhcp\n' update_network_interfaces(tempdir, board_conf) self.assertEqual(expected, open(if_path).read()) def test_write_data_to_protected_file(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) data = 'foo' path = '/etc/nonexistant' write_data_to_protected_file(path, data) calls = fixture.mock.calls self.assertEqual(1, len(calls), calls) call = calls[0] # The call moves tmpfile to the given path, so tmpfile is the next to # last in the list of arguments stored. tmpfile = call[-2] self.assertEqual(['%s mv -f %s %s' % (sudo_args, tmpfile, path)], fixture.mock.commands_executed) self.assertEqual(data, open(tmpfile).read()) def test_rootfs_mount_options_for_btrfs(self): self.assertEqual("defaults", rootfs_mount_options('btrfs')) def test_rootfs_mount_options_for_ext4(self): self.assertEqual("errors=remount-ro", rootfs_mount_options('ext4')) def test_rootfs_mount_options_for_unknown(self): self.assertRaises(ValueError, rootfs_mount_options, 'unknown') def test_append_to_fstab(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() # we don't really need root (sudo) as we're not writing to a real # root owned /etc self.useFixture(MockSomethingFixture(os, 'getuid', lambda: 0)) etc = os.path.join(tempdir, 'etc') os.mkdir(etc) fstab = os.path.join(etc, 'fstab') open(fstab, "w").close() append_to_fstab(tempdir, ['foo', 'bar']) f = open(fstab) contents = f.read() f.close() self.assertEquals("\nfoo\nbar\n", contents) class TestCheckDevice(TestCaseWithFixtures): def _mock_does_device_exist_true(self): self.useFixture(MockSomethingFixture( check_device, '_does_device_exist', lambda device: True)) def _mock_does_device_exist_false(self): self.useFixture(MockSomethingFixture( check_device, '_does_device_exist', lambda device: False)) def _mock_print_devices(self): self.useFixture(MockSomethingFixture( check_device, '_print_devices', lambda: None)) def _mock_select_device(self): self.useFixture(MockSomethingFixture( check_device, '_select_device', lambda device: True)) def _mock_deselect_device(self): self.useFixture(MockSomethingFixture( check_device, '_select_device', lambda device: False)) def _mock_sys_stdout(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open(os.devnull, 'w'))) def setUp(self): super(TestCheckDevice, self).setUp() self._mock_sys_stdout() self._mock_print_devices() def test_ensure_device_partitions_not_mounted(self): partitions_umounted = [] def ensure_partition_is_not_mounted_mock(part): partitions_umounted.append(part) self.useFixture(MockSomethingFixture( partitions, 'ensure_partition_is_not_mounted', ensure_partition_is_not_mounted_mock)) self.useFixture(MockSomethingFixture( glob, 'glob', lambda pattern: ['/dev/sdz1', '/dev/sdz2'])) check_device._ensure_device_partitions_not_mounted('/dev/sdz') self.assertEquals(['/dev/sdz1', '/dev/sdz2'], partitions_umounted) def test_check_device_and_select(self): self._mock_does_device_exist_true() self._mock_select_device() self.assertTrue( check_device.confirm_device_selection_and_ensure_it_is_ready( None)) def test_check_device_and_deselect(self): self._mock_does_device_exist_true() self._mock_deselect_device() self.assertFalse( check_device.confirm_device_selection_and_ensure_it_is_ready( None)) def test_check_device_not_found(self): self._mock_does_device_exist_false() self.assertFalse( check_device.confirm_device_selection_and_ensure_it_is_ready( None)) class AtExitRegister(object): funcs = None def __call__(self, func, *args, **kwargs): if self.funcs is None: self.funcs = [] self.funcs.append((func, args, kwargs)) def run_funcs(self): for func, args, kwargs in self.funcs: func(*args, **kwargs) class TestInstallHWPack(TestCaseWithFixtures): def create_minimal_v3_hwpack(self, location, name, version, architecture): metadata = "\n".join([ "name: " + name, "version: " + version, "architecture: " + architecture, "format: 3.0" ]) print metadata tar_file = tarfile.open(location, mode='w:gz') tarinfo = tarfile.TarInfo("metadata") tarinfo.size = len(metadata) tar_file.addfile(tarinfo, StringIO(metadata)) tar_file.close() def mock_prepare_chroot(self, chroot_dir, tmp_dir): def fake_prepare_chroot(chroot_dir, tmp_dir): cmd_runner.run(['prepare_chroot %s %s' % (chroot_dir, tmp_dir)], as_root=True).wait() self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.chroot_utils, 'prepare_chroot', fake_prepare_chroot)) def test_temporarily_overwrite_file_on_dir(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) temporarily_overwrite_file_on_dir('/path/to/file', '/dir', '/tmp/dir') self.assertEquals( ['%s cp /path/to/file /dir' % sudo_args], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s rm -f /dir/file' % sudo_args], fixture.mock.commands_executed) def test_copy_file(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) copy_file('/path/to/file', '/dir') self.assertEquals( ['%s cp /path/to/file /dir' % sudo_args], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s rm -f /dir/file' % sudo_args], fixture.mock.commands_executed) def test_mount_chroot_proc(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) mount_chroot_proc('chroot') self.assertEquals( ['%s mount proc chroot/proc -t proc' % sudo_args], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s umount -v chroot/proc' % sudo_args], fixture.mock.commands_executed) def test_install_hwpack(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) chroot_dir = 'chroot_dir' hwpack_dir = tempfile.mkdtemp() hwpack_file_name = 'hwpack.tgz' hwpack_tgz_location = os.path.join(hwpack_dir, hwpack_file_name) hwpack_name = "foo" hwpack_version = "4" hwpack_architecture = "armel" extract_kpkgs = False self.create_minimal_v3_hwpack(hwpack_tgz_location, hwpack_name, hwpack_version, hwpack_architecture) force_yes = False install_hwpack(chroot_dir, hwpack_tgz_location, extract_kpkgs, force_yes, 'linaro-hwpack-install') self.assertEquals( ['%s cp %s %s' % (sudo_args, hwpack_tgz_location, chroot_dir), '%s %s %s linaro-hwpack-install --hwpack-version %s ' '--hwpack-arch %s --hwpack-name %s /%s' % (sudo_args, chroot_args, chroot_dir, hwpack_version, hwpack_architecture, hwpack_name, hwpack_file_name)], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s rm -f %s/hwpack.tgz' % (sudo_args, chroot_dir)], fixture.mock.commands_executed) def test_install_hwpack_extract(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) chroot_dir = 'chroot_dir' hwpack_dir = tempfile.mkdtemp() hwpack_file_name = 'hwpack.tgz' hwpack_tgz_location = os.path.join(hwpack_dir, hwpack_file_name) hwpack_name = "foo" hwpack_version = "4" hwpack_architecture = "armel" extract_kpkgs = True self.create_minimal_v3_hwpack(hwpack_tgz_location, hwpack_name, hwpack_version, hwpack_architecture) force_yes = False install_hwpack(chroot_dir, hwpack_tgz_location, extract_kpkgs, force_yes, 'linaro-hwpack-install') self.assertEquals( ['%s cp %s %s' % (sudo_args, hwpack_tgz_location, chroot_dir), '%s linaro-hwpack-install --hwpack-version %s ' '--hwpack-arch %s --hwpack-name %s --extract-kernel-only %s/%s' % (sudo_args, hwpack_version, hwpack_architecture, hwpack_name, chroot_dir, hwpack_file_name)], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s rm -f %s/hwpack.tgz' % (sudo_args, chroot_dir)], fixture.mock.commands_executed) def test_install_hwpacks(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) chroot_dir = 'chroot_dir' tmp_dir = 'tmp_dir' self.mock_prepare_chroot(chroot_dir, tmp_dir) force_yes = True prefer_dir = preferred_tools_dir() hwpack_dir = tempfile.mkdtemp() hwpack_file_names = ['hwpack1.tgz', 'hwpack2.tgz'] hwpack_tgz_locations = [] hwpack_names = [] extract_kpkgs = False for hwpack_file_name in hwpack_file_names: hwpack_tgz_location = os.path.join(hwpack_dir, hwpack_file_name) hwpack_tgz_locations.append(hwpack_tgz_location) hwpack_names.append(hwpack_file_name) hwpack_version = "4" hwpack_architecture = "armel" self.create_minimal_v3_hwpack( hwpack_tgz_location, hwpack_file_name, hwpack_version, hwpack_architecture) install_hwpacks( chroot_dir, tmp_dir, prefer_dir, force_yes, [], extract_kpkgs, hwpack_tgz_locations[0], hwpack_tgz_locations[1]) linaro_hwpack_install = find_command( 'linaro-hwpack-install', prefer_dir=prefer_dir) expected = [ 'prepare_chroot %(chroot_dir)s %(tmp_dir)s', 'cp %(linaro_hwpack_install)s %(chroot_dir)s/usr/bin', 'mount proc %(chroot_dir)s/proc -t proc', 'chroot %(chroot_dir)s true', 'cp %(hwpack1)s %(chroot_dir)s', ('%(chroot_args)s %(chroot_dir)s linaro-hwpack-install ' '--hwpack-version %(hp_version)s ' '--hwpack-arch %(hp_arch)s --hwpack-name %(hp_name1)s' ' --force-yes /hwpack1.tgz'), 'cp %(hwpack2)s %(chroot_dir)s', ('%(chroot_args)s %(chroot_dir)s linaro-hwpack-install ' '--hwpack-version %(hp_version)s ' '--hwpack-arch %(hp_arch)s --hwpack-name %(hp_name2)s' ' --force-yes /hwpack2.tgz'), 'rm -f %(chroot_dir)s/hwpack2.tgz', 'rm -f %(chroot_dir)s/hwpack1.tgz', 'umount -v %(chroot_dir)s/proc', 'rm -f %(chroot_dir)s/usr/bin/linaro-hwpack-install'] keywords = dict( chroot_dir=chroot_dir, tmp_dir=tmp_dir, chroot_args=chroot_args, linaro_hwpack_install=linaro_hwpack_install, hwpack1=hwpack_tgz_locations[0], hwpack2=hwpack_tgz_locations[1], hp_version=hwpack_version, hp_name1=hwpack_names[0], hp_name2=hwpack_names[1], hp_arch=hwpack_architecture) expected = [ "%s %s" % (sudo_args, line % keywords) for line in expected] self.assertEquals(expected, fixture.mock.commands_executed) def test_install_packages(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) chroot_dir = 'chroot_dir' tmp_dir = 'tmp_dir' self.mock_prepare_chroot(chroot_dir, tmp_dir) install_packages(chroot_dir, tmp_dir, 'pkg1', 'pkg2') expected = [ 'prepare_chroot %(chroot_dir)s %(tmp_dir)s', 'mount proc %(chroot_dir)s/proc -t proc', '%(chroot_args)s %(chroot_dir)s apt-get --yes install pkg1 pkg2', '%(chroot_args)s %(chroot_dir)s apt-get clean', 'umount -v %(chroot_dir)s/proc'] keywords = dict( chroot_dir=chroot_dir, tmp_dir=tmp_dir, chroot_args=chroot_args) expected = [ "%s %s" % (sudo_args, line % keywords) for line in expected] self.assertEquals(expected, fixture.mock.commands_executed) def test_prepare_chroot(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) prepare_chroot('chroot', '/tmp/dir') run_local_atexit_funcs() expected = [ 'cp /etc/resolv.conf chroot/etc', 'cp /etc/hosts chroot/etc', 'cp /usr/bin/qemu-armeb-static chroot/usr/bin', 'cp /usr/bin/qemu-arm-static chroot/usr/bin', 'cp /usr/bin/qemu-aarch64-static chroot/usr/bin', 'rm -f chroot/usr/bin/qemu-aarch64-static', 'rm -f chroot/usr/bin/qemu-arm-static', 'rm -f chroot/usr/bin/qemu-armeb-static', 'rm -f chroot/etc/hosts', 'rm -f chroot/etc/resolv.conf'] expected = [ "%s %s" % (sudo_args, line) for line in expected] self.assertEquals(expected, fixture.mock.commands_executed) def test_run_local_atexit_funcs(self): self.useFixture(MockSomethingFixture( sys, 'stderr', open('/dev/null', 'w'))) self.call_order = [] class TestException(Exception): pass def raising_func(): self.call_order.append('raising_func') raise TestException() def behaving_func(): self.call_order.append('behaving_func') self.behaving_func_called = True # run_local_atexit_funcs() runs the atexit handlers in LIFO order, but # even though the first function called (raising_func) will raise # an exception, the second one will still be called after it. linaro_image_tools.media_create.chroot_utils.local_atexit = [ behaving_func, raising_func] # run_local_atexit_funcs() also propagates the last exception raised # by one of the functions. chroot_utils = linaro_image_tools.media_create.chroot_utils self.assertRaises(TestException, chroot_utils.run_local_atexit_funcs) self.assertEquals( ['raising_func', 'behaving_func'], self.call_order) def test_hwpack_atexit(self): self.run_local_atexit_functions_called = False def mock_run_local_atexit_functions(): self.run_local_atexit_functions_called = True def mock_install_hwpack(p1, p2, p3, p4): raise Exception('hwpack mock exception') self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.chroot_utils, 'install_hwpack', mock_install_hwpack)) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.chroot_utils, 'run_local_atexit_funcs', mock_run_local_atexit_functions)) force_yes = True exception_caught = False extract_kpkgs = False try: install_hwpacks( 'chroot', '/tmp/dir', preferred_tools_dir(), force_yes, [], extract_kpkgs, 'hwp.tgz', 'hwp2.tgz') except: exception_caught = True self.assertTrue(self.run_local_atexit_functions_called) self.assertTrue(exception_caught) def setUp(self): super(TestInstallHWPack, self).setUp() # Ensure the list of cleanup functions gets cleared to make sure tests # don't interfere with one another. def clear_atexits(): linaro_image_tools.media_create.chroot_utils.local_atexit = [] self.addCleanup(clear_atexits) linaro-image-tools-2014.11/linaro_image_tools/media_create/tests/fixtures.py0000664000175000017500000000462312400111666030313 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os import subprocess from linaro_image_tools.media_create import partitions from linaro_image_tools.tests.fixtures import MockSomethingFixture class CreateTarballFixture(object): def __init__(self, basedir, reldir='tarball', filename='tarball.tar.gz'): self.basedir = basedir self.reldir = reldir self.tarball = os.path.join(self.basedir, filename) def setUp(self): # Create gzipped tar archive. os.mkdir(os.path.join(self.basedir, self.reldir)) args = ['tar', '-C', self.basedir, '-czf', self.tarball, self.reldir] proc = subprocess.Popen(args) proc.wait() def tearDown(self): if os.path.exists(self.tarball): os.remove(self.tarball) dir = os.path.join(self.basedir, self.reldir) if os.path.exists(dir): os.rmdir(dir) def get_tarball(self): return self.tarball class MockCallableWithPositionalArgs(object): """A callable mock which just stores the positional args given to it. Every time an instance of this is "called", it will append a tuple containing the positional arguments given to it to self.calls. """ calls = None return_value = None def __call__(self, *args): if self.calls is None: self.calls = [] self.calls.append(args) return self.return_value class MockRunSfdiskCommandsFixture(MockSomethingFixture): def __init__(self): mock = MockCallableWithPositionalArgs() mock.return_value = ('', '') super(MockRunSfdiskCommandsFixture, self).__init__( partitions, 'run_sfdisk_commands', mock) linaro-image-tools-2014.11/linaro_image_tools/media_create/android_boards.py0000664000175000017500000006425212400111666030256 0ustar fboudrafboudra00000000000000# Copyright (C) 2011 Linaro # # Author: Jeremy Chang # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . """Configuration for boards supported by linaro-android-media-create. To add support for a new board, you need to create a subclass of AndroidBoardConfig, create an Android hwpack as explained here: https://wiki.linaro.org/AndroidHardwarePacksV3 and add the board to 'android_board_configs' at the end of this file. """ import os import yaml import logging from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.hwpack_fields import FORMAT_FIELD from linaro_image_tools.media_create.partitions import SECTOR_SIZE from linaro_image_tools.media_create.boards import ( ArndaleConfig, ArndaleOctaConfig, BeagleConfig, BoardConfig, BoardConfigException, Mx53LoCoConfig, OrigenConfig, OrigenQuadConfig, PART_ALIGN_S, PandaConfig, SMDKV310Config, SnowballEmmcConfig, SnowballSdConfig, VexpressConfig, align_partition, align_up, install_mx5_boot_loader, make_boot_script, _dd, BoardException, ) from linaro_image_tools.utils import DEFAULT_LOGGER_NAME logger = logging.getLogger(DEFAULT_LOGGER_NAME) BOOT_MIN_SIZE_S = align_up(128 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE SYSTEM_MIN_SIZE_S = align_up(768 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE CACHE_MIN_SIZE_S = align_up(256 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE USERDATA_MIN_SIZE_S = align_up(576 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE SDCARD_MIN_SIZE_S = align_up(256 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE LOADER_MIN_SIZE_S = align_up(1 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE class AndroidBoardConfig(BoardConfig): def __init__(self): super(AndroidBoardConfig, self).__init__() self.dtb_name = None self._extra_serial_options = [] self._android_specific_args = [] self._extra_boot_args_options = [] self._live_serial_options = [] def _get_android_specific_args(self): android_args = self._android_specific_args if isinstance(android_args, list): android_args = ' '.join(self._android_specific_args) return android_args def _set_android_specific_args(self, value): self._android_specific_args = value android_specific_args = property(_get_android_specific_args, _set_android_specific_args) def _get_extra_boot_args_options(self): extra_boot_args = self._extra_boot_args_options if isinstance(extra_boot_args, list): extra_boot_args = ' '.join(self._extra_boot_args_options) return extra_boot_args def _set_extra_boot_args_options(self, value): self._extra_boot_args_options = value extra_boot_args_options = property(_get_extra_boot_args_options, _set_extra_boot_args_options) def _get_extra_serial_options(self): extra_serial = self._extra_serial_options if isinstance(extra_serial, list): extra_serial = ' '.join(self._extra_serial_options) return extra_serial def _set_extra_serial_options(self, value): self._extra_serial_options = value extra_serial_options = property(_get_extra_serial_options, _set_extra_serial_options) def _get_live_serial_options(self): serial_options = self._live_serial_options if serial_options: if isinstance(serial_options, list): serial_options = ' '.join(self._live_serial_options) if self._check_placeholder_presence(serial_options, '%s'): serial_options = serial_options % self.serial_tty return serial_options def _set_live_serial_options(self, value): self._live_serial_options = value live_serial_options = property(_get_live_serial_options, _set_live_serial_options) def from_file(self, hwpack): """Loads the Android board configuration from an Android hardware pack configuration file and sets the config attributes with their values. :param hwpack: The Android hwpack configuration file. :return The configuration read from the file as a dictionary. """ try: with open(hwpack, 'r') as hw: config = yaml.safe_load(hw) self._set_attributes(config) return config except yaml.YAMLError, ex: logger.debug("Error loading YAML file %s" % hwpack, ex) raise BoardConfigException("Error reading Android hwpack %s" % hwpack) except IOError, ex: logger.debug("Error reading hwpack file %s" % hwpack, ex) raise BoardConfigException("Android hwpack %s could not be found" % hwpack) def _set_attributes(self, config): """Initialize the class attributes with the values read from the Android hardware pack configuration file. :param config: The config read from the Android hwpack. """ for name, value in config.iteritems(): if name == FORMAT_FIELD: setattr(self, 'hwpack_format', value) elif hasattr(self, name): setattr(self, name, value) else: logger.warning("Attribute '%s' does not belong to this " "instance of '%s'." % (name, self.__class__)) def _get_bootargs(self, consoles): """Get the bootargs for this board. In general subclasses should not have to override this. """ boot_args_options = 'rootwait ro' if self.extra_boot_args_options: boot_args_options += ' %s' % self.extra_boot_args_options boot_args_options += ' %s' % self.android_specific_args serial_options = self.extra_serial_options for console in consoles: serial_options += ' console=%s' % console replacements = dict( serial_options=serial_options, boot_args_options=boot_args_options) return ( "%(serial_options)s " "%(boot_args_options)s" % replacements).strip() def _get_boot_env(self, consoles): """Get the boot environment for this board. In general subclasses should not have to override this. """ boot_env = {} boot_env["bootargs"] = self._get_bootargs(consoles) initrd = False if self.initrd_addr: initrd = True # On Android, the DTB file is always built as part of the kernel it # comes from - and lives in the same directory in the boot tarball, so # here we don't need to pass the whole path to it. boot_env["bootcmd"] = self._get_bootcmd(initrd, self.dtb_name) boot_env["initrd_high"] = self.initrd_high boot_env["fdt_high"] = self.fdt_high return boot_env def populate_boot_script(self, boot_partition, boot_disk, consoles): cmd_runner.run(['mkdir', '-p', boot_disk]).wait() # TODO: Use partition_mounted() here to make sure the partition is # always umounted after we're done. cmd_runner.run(['mount', boot_partition, boot_disk], as_root=True).wait() boot_env = self._get_boot_env(consoles) cmdline_filepath = os.path.join(boot_disk, "cmdline") with open(cmdline_filepath, 'r') as cmdline_file: android_kernel_cmdline = cmdline_file.read().strip() boot_env['bootargs'] = boot_env['bootargs'] + ' ' + \ android_kernel_cmdline boot_dir = boot_disk boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) try: cmd_runner.run(['umount', boot_disk], as_root=True).wait() except cmd_runner.SubcommandNonZeroReturnValue: pass def get_sfdisk_cmd(self, should_align_boot_part=False, start_addr=0, extra_part=False): if self.fat_size == 32: partition_type = '0x0C' else: partition_type = '0x0E' # align on sector 63 for compatibility with broken versions of x-loader # unless align_boot_part is set boot_align = 63 if should_align_boot_part: boot_align = PART_ALIGN_S # can only start on sector 1 (sector 0 is MBR / partition table) boot_start, boot_end, boot_len = align_partition( start_addr + 1, BOOT_MIN_SIZE_S, boot_align, PART_ALIGN_S) # apparently OMAP3 ROMs require the vfat length to be an even number # of sectors (multiple of 1 KiB); decrease the length if it's odd, # there should still be enough room boot_len = boot_len - boot_len % 2 boot_end = boot_start + boot_len - 1 system_start, _system_end, _system_len = align_partition( boot_end + 1, SYSTEM_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) cache_start, _cache_end, _cache_len = align_partition( _system_end + 1, CACHE_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) userdata_start, _userdata_end, _userdata_len = align_partition( _cache_end + 1, USERDATA_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) sdcard_start, _sdcard_end, _sdcard_len = align_partition( _userdata_end + 1, SDCARD_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) # Snowball board needs a raw partition added to the beginning of image. # If extra_part is True an extra primary partition will be added. # Due to a maximum of 4 primary partitions cache data will be placed in # a extended partition if extra_part is True: assert start_addr > 0, ("Not possible to add extra partition" "when boot partition starts at '0'") return '%s,%s,%s,*\n%s,%s,L\n%s,-,E\n%s,%s,L\n%s,%s,L\n%s,,,-' % ( boot_start, boot_len, partition_type, system_start, _system_len, cache_start, cache_start, _cache_len, userdata_start, _userdata_len, sdcard_start) return '%s,%s,%s,*\n%s,%s,L\n%s,%s,L\n%s,-,E\n%s,%s,L\n%s,,,-' % ( boot_start, boot_len, partition_type, system_start, _system_len, cache_start, _cache_len, userdata_start, userdata_start, _userdata_len, sdcard_start) def populate_raw_partition(self, media, boot_dir): super(AndroidBoardConfig, self).populate_raw_partition(media, boot_dir) def install_boot_loader(self, boot_partition, boot_device_or_file): pass class AndroidOmapConfig(AndroidBoardConfig): """Placeholder class for OMAP configuration inheritance.""" class AndroidBeagleConfig(AndroidOmapConfig, BeagleConfig): """Placeholder class for Beagle configuration inheritance.""" def __init__(self): super(AndroidBeagleConfig, self).__init__() self._android_specific_args = 'init=/init androidboot.console=ttyO2' self._extra_serial_options = 'console=tty0 console=ttyO2,115200n8' class AndroidPandaConfig(AndroidBoardConfig, PandaConfig): """Placeholder class for Panda configuration inheritance.""" def __init__(self): super(AndroidPandaConfig, self).__init__() self.dtb_name = 'board.dtb' self._extra_serial_options = 'console=ttyO2,115200n8' self._extra_boot_args_options = ( 'earlyprintk fixrtc nocompcache vram=48M ' 'omapfb.vram=0:24M,1:24M mem=456M@0x80000000 mem=512M@0xA0000000') self._android_specific_args = 'init=/init androidboot.console=ttyO2' self.dtb_addr = '0x815f0000' self.bootloader_flavor = 'omap4_panda' class AndroidSnowballSdConfig(AndroidBoardConfig, SnowballSdConfig): """Placeholder class for Snowball Sd configuration inheritance.""" def __init__(self): super(AndroidSnowballSdConfig, self).__init__() self.dtb_name = 'board.dtb' self._android_specific_args = 'init=/init androidboot.console=ttyAMA2' self._extra_boot_args_options = ( 'earlyprintk mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M vmalloc=500M') self._extra_serial_options = 'console=ttyAMA2,115200n8' self.boot_script = 'boot.scr' self.fdt_high = '0x05000000' self.initrd_addr = '0x05000000' self.initrd_high = '0x06000000' self.dtb_addr = '0x8000000' class AndroidSnowballEmmcConfig(AndroidBoardConfig, SnowballEmmcConfig): """Class for Snowball Emmc configuration inheritance.""" def __init__(self): super(AndroidSnowballEmmcConfig, self).__init__() self.dtb_name = 'board.dtb' self._extra_boot_args_options = ( 'earlyprintk mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M vmalloc=500M') self._extra_serial_options = 'console=ttyAMA2,115200n8' self._android_specific_args = 'init=/init androidboot.console=ttyAMA2' self.boot_script = 'boot.scr' self.fdt_high = '0x05000000' self.initrd_addr = '0x05000000' self.initrd_high = '0x06000000' self.mmc_option = '0:2' self.dtb_addr = '0x8000000' def get_sfdisk_cmd(self, should_align_boot_part=False): loader_start, loader_end, loader_len = align_partition( SnowballEmmcConfig.SNOWBALL_LOADER_START_S, LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) command = super(AndroidSnowballEmmcConfig, self).get_sfdisk_cmd( should_align_boot_part=True, start_addr=loader_end, extra_part=True) return '%s,%s,0xDA\n%s' % ( loader_start, loader_len, command) def populate_raw_partition(self, media, boot_dir): # To avoid adding a Snowball specific command line option, we assume # that the user already has unpacked the startfiles to ./startupfiles config_files_dir = self.snowball_config(boot_dir) assert os.path.exists(config_files_dir), ( "You need to unpack the Snowball startupfiles to the directory " "'startupfiles' in your current working directory. See " "igloocommunity.org for more information.") # We copy the u-boot files from the unpacked boot.tar.bz2 # and put it with the startfiles. boot_files = ['u-boot.bin'] for boot_file in boot_files: cmd_runner.run(['cp', os.path.join(boot_dir, 'boot', boot_file), config_files_dir], as_root=True).wait() super(AndroidSnowballEmmcConfig, self).populate_raw_partition( media, boot_dir) def snowball_config(self, chroot_dir): # The user is expected to have unpacked the startupfiles to this subdir # of their working dir. return os.path.join('.', 'startupfiles') @property def delete_startupfiles(self): # The startupfiles will have been unpacked to the user's working dir # and should not be deleted after they have been installed. return False class AndroidMx53LoCoConfig(AndroidBoardConfig, Mx53LoCoConfig): """Class for Mx53LoCo configuration inheritance.""" def __init__(self): super(AndroidMx53LoCoConfig, self).__init__() self._extra_boot_args_options = ( 'earlyprintk rootdelay=1 fixrtc nocompcache di1_primary tve') self._extra_serial_options = 'console=%s,115200n8' self._android_specific_args = 'init=/init androidboot.console=%s' def _get_extra_serial_options(self): serial_options = self._extra_serial_options if serial_options: if isinstance(serial_options, list): serial_options = ' '.join(serial_options) if self._check_placeholder_presence(serial_options, '%s'): serial_options = serial_options % self.serial_tty return serial_options def _set_extra_serial_options(self, value): self._extra_serial_options = value extra_serial_options = property(_get_extra_serial_options, _set_extra_serial_options) def _get_android_specific_args(self): android_args = self._android_specific_args if android_args: if isinstance(android_args, list): android_args = ' '.join(android_args) if self._check_placeholder_presence(android_args, '%s'): android_args = android_args % self.serial_tty return android_args def _set_android_specific_args(self, value): self._android_specific_args = value android_specific_args = property(_get_android_specific_args, _set_android_specific_args) def get_sfdisk_cmd(self, should_align_boot_part=False): loader_start, loader_end, loader_len = align_partition( 1, self.LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) command = super(AndroidMx53LoCoConfig, self).get_sfdisk_cmd( should_align_boot_part=True, start_addr=loader_end, extra_part=True) return '%s,%s,0xDA\n%s' % ( loader_start, loader_len, command) def install_boot_loader(self, boot_partition, boot_device_or_file): install_mx5_boot_loader( os.path.join(boot_device_or_file, "u-boot.imx"), boot_partition, self.LOADER_MIN_SIZE_S) class AndroidMx6QSabreliteConfig(AndroidMx53LoCoConfig): """Placeholder class for Mx6Q Sabrelite configuration inheritance.""" def __init__(self): super(AndroidMx6QSabreliteConfig, self).__init__() self.dtb_name = 'board.dtb' self.bootloader_flavor = 'mx6qsabrelite' self.kernel_addr = '0x10000000' self.initrd_addr = '0x12000000' self.load_addr = '0x10008000' self.dtb_addr = '0x11ff0000' class AndroidSamsungConfig(AndroidBoardConfig): def get_sfdisk_cmd(self, should_align_boot_part=False): loaders_min_len = (self.samsung_bl1_start + self.samsung_bl1_len + self.samsung_bl2_len + self.samsung_env_len) loader_start, loader_end, loader_len = align_partition( 1, loaders_min_len, 1, PART_ALIGN_S) command = super(AndroidSamsungConfig, self).get_sfdisk_cmd( should_align_boot_part=False, start_addr=loader_end, extra_part=True) return '%s,%s,0xDA\n%s' % ( loader_start, loader_len, command) class AndroidSMDKV310Config(AndroidSamsungConfig, SMDKV310Config): """Placeholder class for SMDKV310 configuration inheritance.""" def __init__(self): super(AndroidSMDKV310Config, self).__init__() self._extra_serial_options = 'console=tty0 console=ttySAC1,115200n8' self._android_specific_args = 'init=/init androidboot.console=ttySAC1' class AndroidOrigenConfig(AndroidSamsungConfig, OrigenConfig): """Placeholder class for Origen configuration inheritance.""" def __init__(self): super(AndroidOrigenConfig, self).__init__() self._extra_serial_options = 'console=tty0 console=ttySAC2,115200n8' self._android_specific_args = 'init=/init androidboot.console=ttySAC2' class AndroidOrigenQuadConfig(AndroidSamsungConfig, OrigenQuadConfig): def __init__(self): super(AndroidOrigenQuadConfig, self).__init__() self._extra_serial_options = 'console=tty0 console=ttySAC2,115200n8' self._android_specific_args = 'init=/init androidboot.console=ttySAC2' class AndroidVexpressConfig(AndroidBoardConfig, VexpressConfig): """Placeholder class for Vexpress configuration inheritance.""" def __init__(self): super(AndroidVexpressConfig, self).__init__() self._extra_serial_options = 'console=tty0 console=ttyAMA0,38400n8' self._android_specific_args = 'init=/init androidboot.console=ttyAMA0' class AndroidArndaleConfig(AndroidSamsungConfig, ArndaleConfig): """Placeholder class for Arndale configuration inheritance.""" def __init__(self): super(AndroidArndaleConfig, self).__init__() self.mmc_option = '0:1' self.kernel_addr = '0x40007000' self.initrd_addr = '0x41000000' self.dtb_addr = '0x41f00000' self.dtb_name = 'exynos5250-arndale.dtb' self._android_specific_args = ( 'init=/init androidboot.console=ttySAC2 console=ttySAC2 initrd=%s' % self.initrd_addr) self._extra_serial_options = 'ttySAC2,115200n8' self._extra_boot_args_options = 'rootdelay=3' def _get_bootcmd(self, i_img_data, d_img_data): """Get the bootcmd for this board. In general subclasses should not have to override this. """ replacements = dict( fatload_command=self.fatload_command, uimage_path=self.uimage_path, mmc_option=self.mmc_option, kernel_addr=self.kernel_addr, initrd_addr=self.initrd_addr, dtb_addr=self.dtb_addr) boot_script = ( ("%(fatload_command)s mmc %(mmc_option)s %(kernel_addr)s " + "%(uimage_path)suImage; ")) % replacements if i_img_data is not None: boot_script += ( ("%(fatload_command)s mmc %(mmc_option)s %(initrd_addr)s " + "%(uimage_path)suInitrd; ")) % replacements if d_img_data is not None: assert self.dtb_addr is not None, ( "Need a dtb_addr when passing d_img_data") boot_script += ("%(fatload_command)s mmc %(mmc_option)s " "%(dtb_addr)s " % replacements) boot_script += "%s; " % d_img_data boot_script += (("bootm %(kernel_addr)s")) % replacements if i_img_data is not None: boot_script += ((" %(initrd_addr)s")) % replacements if d_img_data is not None: boot_script += ((" %(dtb_addr)s")) % replacements return boot_script def populate_raw_partition(self, boot_device_or_file, chroot_dir): boot_bin_0 = {'name': 'arndale-bl1.bin', 'seek': 1} boot_bin_1 = {'name': 'u-boot-mmc-spl.bin', 'seek': 17} boot_bin_2 = {'name': 'u-boot.bin', 'seek': 49} boot_bins = [boot_bin_0, boot_bin_1, boot_bin_2] boot_partition = 'boot' # Zero the env so that the boot_script will get loaded _dd("/dev/zero", boot_device_or_file, count=self.samsung_env_len, seek=self.samsung_env_start) for boot_bin in boot_bins: name = boot_bin['name'] file_path = os.path.join(chroot_dir, boot_partition, name) if not os.path.exists(file_path): raise BoardException( "File '%s' does not exists. Cannot proceed." % name) _dd(file_path, boot_device_or_file, seek=boot_bin['seek']) class AndroidArndaleOctaConfig(AndroidArndaleConfig, ArndaleOctaConfig): """Placeholder class for Arndale-Octa configuration inheritance.""" def __init__(self): super(AndroidArndaleOctaConfig, self).__init__() self.samsung_env_start = 1231 self.mmc_option = '0:2' self.kernel_addr = '0x20007000' self.initrd_addr = '0x21000000' self.dtb_addr = '0x21f00000' self.dtb_name = 'exynos5420-arndale-octa.dtb' self._android_specific_args = ( 'init=/init androidboot.console=ttySAC3 console=ttySAC3 initrd=%s' % self.initrd_addr) self._extra_serial_options = 'ttySAC3,115200n8' self._extra_boot_args_options = 'rootdelay=3' def populate_raw_partition(self, boot_device_or_file, chroot_dir): boot_bin_0 = {'name': 'arndale-octa.bl1.bin', 'seek': 1} boot_bin_1 = {'name': 'smdk5420-spl.signed.bin', 'seek': 31} boot_bin_2 = {'name': 'u-boot.bin', 'seek': 63} boot_bin_3 = {'name': 'arndale-octa.tzsw.bin', 'seek': 719} boot_bins = [boot_bin_0, boot_bin_1, boot_bin_2, boot_bin_3] boot_partition = 'boot' # Zero the env so that the boot_script will get loaded _dd("/dev/zero", boot_device_or_file, count=self.samsung_env_len, seek=self.samsung_env_start) for boot_bin in boot_bins: name = boot_bin['name'] file_path = os.path.join(chroot_dir, boot_partition, name) if not os.path.exists(file_path): raise BoardException( "File '%s' does not exists. Cannot proceed." % name) _dd(file_path, boot_device_or_file, seek=boot_bin['seek']) # This dictionary is composed as follows: # : # The is the command line argument passed to l-a-m-c, the # is the corresponding config class in this file (not the instance). # If a new device does not have special needs, it is possible to use the # general AndroidBoardConfig class. android_board_configs = { 'arndale': AndroidArndaleConfig, 'arndale_octa': AndroidArndaleOctaConfig, 'beagle': AndroidBeagleConfig, 'iMX53': AndroidMx53LoCoConfig, 'mx53loco': AndroidMx53LoCoConfig, 'mx6qsabrelite': AndroidMx6QSabreliteConfig, 'origen': AndroidOrigenConfig, 'origen_quad': AndroidOrigenQuadConfig, 'panda': AndroidPandaConfig, 'smdkv310': AndroidSMDKV310Config, 'snowball_emmc': AndroidSnowballEmmcConfig, 'snowball_sd': AndroidSnowballSdConfig, 'vexpress': AndroidVexpressConfig, 'vexpress-a9': AndroidVexpressConfig, } def get_board_config(board): """Get the board configuration for the specified board. :param board: The name of the board to get the configuration of. :type board: str """ clazz = android_board_configs.get(board, None) if clazz: return clazz() else: raise BoardConfigException("Board name '%s' has no configuration " "available." % board) linaro-image-tools-2014.11/linaro_image_tools/media_create/check_device.py0000664000175000017500000001077012400111666027674 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import glob import dbus from linaro_image_tools.media_create import partitions def _get_system_bus_and_udisks_iface(): """Return the system bus and the UDisks interface. :return: System bus and UDisks inteface tuple. """ bus = dbus.SystemBus() udisks = dbus.Interface( bus.get_object("org.freedesktop.UDisks", "/org/freedesktop/UDisks"), 'org.freedesktop.UDisks') return (bus, udisks) def _get_dbus_property(prop, device, path): """ Return a named property for a specific device. :param prop: Named property. :param device: Device object. :param path: Device path. :return: Device property. """ return device.Get( path, prop, dbus_interface='org.freedesktop.DBus.Properties') def _does_device_exist(path): """Checks if the provided path is an existing device. :param path: Disk device path. :return: True if the device exist, else False. """ bus, udisks = _get_system_bus_and_udisks_iface() try: udisks.get_dbus_method('FindDeviceByDeviceFile')(path) except dbus.exceptions.DBusException: # TODO: Check that this exception isn't hiding other errors. return False return True def _print_devices(): """Print disk devices found on the system.""" bus, udisks = _get_system_bus_and_udisks_iface() print '%-16s %-16s %s' % ('Device', 'Mount point', 'Size') devices = udisks.get_dbus_method('EnumerateDevices')() devices.sort() for path in devices: device = bus.get_object("org.freedesktop.UDisks", path) device_file = _get_dbus_property('DeviceFile', device, path) mount_paths = _get_dbus_property('device-mount-paths', device, path) mount_point = ''.join(b for b in mount_paths) if mount_point == '': mount_point = 'none' if _get_dbus_property('DeviceIsPartition', device, path): part_size = _get_dbus_property('partition-size', device, path) print '%-16s %-16s %dMB' % ( device_file, mount_point, part_size / 1024 ** 2) else: device_size = _get_dbus_property('device-size', device, path) print '%-16s %-16s %dMB' % ( device_file, mount_point, device_size / 1024 ** 2) def _select_device(device): """Ask the user to confirm the selected device. :param device: Device path. :return: True if the user confirms the selection, else False. """ resp = raw_input('Are you 100%% sure, on selecting [%s] (y/n)? ' % device) if resp.lower() != 'y': return False return True def _ensure_device_partitions_not_mounted(device): """Ensure all partitions of the given device are not mounted.""" # Use '%s?*' as we only want the device files representing # partitions and not the one representing the device itself. for part in glob.glob('%s?*' % device): partitions.ensure_partition_is_not_mounted(part) def confirm_device_selection_and_ensure_it_is_ready( device, yes_to_mmc_selection=False): """Confirm this is the device to use and ensure it's ready. If the device exists, the user is asked to confirm that this is the device to use. Upon confirmation we ensure all partitions of the device are umounted. :param device: The path to the device. :return: True if the device exist and is selected, else False. """ if _does_device_exist(device): print '\nI see...' _print_devices() if yes_to_mmc_selection or _select_device(device): _ensure_device_partitions_not_mounted(device) return True else: print '\nAre you sure? I do not see [%s].' % device print 'Here is what I see...' _print_devices() return False linaro-image-tools-2014.11/linaro_image_tools/media_create/rootfs.py0000664000175000017500000001566612400111666026625 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import os import subprocess import tempfile from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.partitions import partition_mounted def populate_partition(content_dir, root_disk, partition): os.makedirs(root_disk) with partition_mounted(partition, root_disk): move_contents(content_dir, root_disk) def rootfs_mount_options(rootfs_type): """Return mount options for the specific rootfs type.""" if rootfs_type == "btrfs": return "defaults" if rootfs_type in ('ext2', 'ext3', 'ext4'): return "errors=remount-ro" raise ValueError('Unsupported rootfs type') def populate_rootfs(content_dir, root_disk, partition, rootfs_type, rootfs_id, should_create_swap, swap_size, mmc_device_id, partition_offset, os_release_id, board_config=None): """Populate the rootfs and make the necessary tweaks to make it usable. This consists of: 1. Create a directory on the path specified by root_disk 2. Mount the given partition onto the created directory. 3. Setup an atexit handler to unmount the partition mounted above. 4. Move the contents of content_dir to that directory. 5. If should_create_swap, then create it with the given size. 6. Add fstab entries for the / filesystem and swap (if created). 7. Create a /etc/flash-kernel.conf containing the target's boot device. """ print "\nPopulating rootfs partition" print "Be patient, this may take a few minutes\n" # Create a directory to mount the rootfs partition. os.makedirs(root_disk) with partition_mounted(partition, root_disk): move_contents(content_dir, root_disk) mount_options = rootfs_mount_options(rootfs_type) fstab_additions = ["%s / %s %s 0 1" % ( rootfs_id, rootfs_type, mount_options)] if should_create_swap: print "\nCreating SWAP File\n" if has_space_left_for_swap(root_disk, swap_size): proc = cmd_runner.run([ 'dd', 'if=/dev/zero', 'of=%s/SWAP.swap' % root_disk, 'bs=1M', 'count=%s' % swap_size], as_root=True) proc.wait() proc = cmd_runner.run( ['mkswap', '%s/SWAP.swap' % root_disk], as_root=True) proc.wait() fstab_additions.append("/SWAP.swap none swap sw 0 0") else: print ("Swap file is bigger than space left on partition; " "continuing without swap.") append_to_fstab(root_disk, fstab_additions) if os_release_id == 'debian' or os_release_id == 'ubuntu' or \ os.path.exists('%s/etc/debian_version' % root_disk): print "\nCreating /etc/flash-kernel.conf\n" create_flash_kernel_config( root_disk, mmc_device_id, 1 + partition_offset) if board_config is not None: print "\nUpdating /etc/network/interfaces\n" update_network_interfaces(root_disk, board_config) def update_network_interfaces(root_disk, board_config): interfaces = [] if board_config.wired_interfaces is not None: interfaces.extend(board_config.wired_interfaces) if board_config.wireless_interfaces is not None: interfaces.extend(board_config.wireless_interfaces) if_path = os.path.join(root_disk, 'etc', 'network', 'interfaces') if os.path.exists(if_path): with open(if_path) as if_file: config = if_file.read() else: config = '' for interface in interfaces: if interface not in config: config += "auto %(if)s\niface %(if)s inet dhcp\n" % ( {'if': interface}) if config != '': write_data_to_protected_file(if_path, config) def create_flash_kernel_config(root_disk, mmc_device_id, boot_partition_number): """Create a flash-kernel.conf file under root_disk/etc. Uses the given partition number to figure out the boot partition. """ target_boot_dev = '/dev/mmcblk%dp%s' % ( mmc_device_id, boot_partition_number) flash_kernel = os.path.join(root_disk, 'etc', 'flash-kernel.conf') write_data_to_protected_file( flash_kernel, "UBOOT_PART=%s\n" % target_boot_dev) def _list_files(directory): """List the files and dirs under the given directory. Runs as root because we want to list everything, including stuff that may not be world-readable. """ p = cmd_runner.run( ['find', directory, '-maxdepth', '1', '-mindepth', '1', '!', '-name', 'lost+found'], stdout=subprocess.PIPE, as_root=True) stdout, _ = p.communicate() return stdout.split() def move_contents(from_, root_disk): """Move everything under from_ to the given root disk. Uses sudo for moving. """ assert os.path.isdir(from_), "%s is not a directory" % from_ files = _list_files(from_) mv_cmd = ['mv'] mv_cmd.extend(sorted(files)) mv_cmd.append(root_disk) cmd_runner.run(mv_cmd, as_root=True).wait() def has_space_left_for_swap(root_disk, swap_size_in_mega_bytes): """Is there enough space for a swap file in the given root disk?""" statvfs = os.statvfs(root_disk) free_space = statvfs.f_bavail * statvfs.f_bsize swap_size_in_bytes = int(swap_size_in_mega_bytes) * 1024 ** 2 if free_space >= swap_size_in_bytes: return True return False def append_to_fstab(root_disk, fstab_additions): fstab = os.path.join(root_disk, 'etc', 'fstab') data = open(fstab).read() + '\n' + '\n'.join(fstab_additions) + '\n' write_data_to_protected_file(fstab, data) def write_data_to_protected_file(path, data): """Write data to the file on the given path. This is meant to be used when the given file is only writable by root, and we overcome that by writing the data to a tempfile and then moving the tempfile on top of the given one using sudo. """ _, tmpfile = tempfile.mkstemp() with open(tmpfile, 'w') as fd: fd.write(data) cmd_runner.run(['mv', '-f', tmpfile, path], as_root=True).wait() linaro-image-tools-2014.11/linaro_image_tools/media_create/boards.py0000664000175000017500000026466412400111666026567 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . """Configuration for boards supported by linaro-media-create. To add support for a new board, you need to create a subclass of BoardConfig, set appropriate values for its variables and add it to board_configs at the bottom of this file. """ from binascii import crc32 from parted import Device import atexit import glob import logging import os import re import string import struct import tempfile from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.handler import HardwarepackHandler from linaro_image_tools.media_create.partitions import ( SECTOR_SIZE, partition_mounted, register_loopback, ) from linaro_image_tools.hwpack.hwpack_fields import ( BOOTFS, BOOTFS16, BOOT_MIN_SIZE_FIELD, BOOT_SCRIPT_FIELD, DTB_ADDR_FIELD, DTB_FILES_FIELD, DTB_FILE_FIELD, ENV_DD_FIELD, EXTRA_BOOT_OPTIONS_FIELD, EXTRA_SERIAL_OPTIONS_FIELD, INITRD_ADDR_FIELD, KERNEL_ADDR_FIELD, LOADER_MIN_SIZE_FIELD, LOADER_START_FIELD, LOAD_ADDR_FIELD, MMC_ID_FIELD, PARTITION_LAYOUT_FIELD, RESERVED_BOOTFS, ROOT_MIN_SIZE_FIELD, SAMSUNG_BL1_LEN_FIELD, SAMSUNG_BL1_START_FIELD, SAMSUNG_BL2_LEN_FIELD, SAMSUNG_BL2_START_FIELD, SAMSUNG_ENV_LEN_FIELD, SAMSUNG_ENV_START_FIELD, SERIAL_TTY_FIELD, SNOWBALL_STARTUP_FILES_CONFIG_FIELD, SPL_DD_FIELD, SPL_IN_BOOT_PART_FIELD, WIRED_INTERFACES_FIELD, WIRELESS_INTERFACES_FIELD, ) logger = logging.getLogger(__name__) KERNEL_GLOB = 'vmlinuz-*-%(kernel_flavor)s' INITRD_GLOB = 'initrd.img-*-%(kernel_flavor)s' DTB_GLOB = 'dt-*-%(kernel_flavor)s/%(dtb_name)s' # Notes: # * since we align partitions on 4 MiB by default, geometry is currently 128 # heads and 32 sectors (2 MiB) as to have CHS-aligned partition start/end # offsets most of the time and hence avoid some warnings with disk # partitioning tools # * apparently some OMAP3 ROMs don't tolerate vfat length of an odd number of # sectors (only sizes rounded to 1 KiB seem to boot) # * we want partitions aligned on 4 MiB as to get the best performance and # limit wear-leveling # * image_size is passed on the command-line and should preferably be a power # of 2; it should be used as a "don't go over this size" information for a # real device, and a "give me a file exactly this big" requirement for an # image file. Having exactly a power of 2 helps with QEMU; there seem to be # some truncating issues otherwise. XXX to be researched # align on 4 MiB PART_ALIGN_S = 4 * 1024 * 1024 / SECTOR_SIZE def align_up(value, align): """Round value to the next multiple of align.""" return (value + align - 1) / align * align def align_partition(min_start, min_length, start_alignment, end_alignment): """Compute partition start and end offsets based on specified constraints. :param min_start: Minimal start offset of partition :param min_lengh: Minimal length of partition :param start_alignment: Alignment of this partition :param end_alignment: Alignment of the data following this partition :return: start offset, end offset (inclusive), length """ start = align_up(min_start, start_alignment) # end offset is inclusive, so substact one end = align_up(start + min_length, end_alignment) - 1 # and add one to length length = end - start + 1 return start, end, length def copy_drop(src, dest_dir): """Copy a file from src to destdir, dropping root ownership on the way. """ cmd = ["cp", "-v", src, dest_dir] cmd_runner.run(cmd, as_root=True).wait() final = os.path.join(dest_dir, os.path.basename(src)) cmd = ["chown", "%s:%s" % (os.getuid(), os.getgid()), final] cmd_runner.run(cmd, as_root=True).wait() os.chmod(final, 0644) class BoardException(Exception): """Class for board related exceptions.""" class BoardConfig(object): """The configuration used when building an image for a board.""" LOADER_MIN_SIZE_S = align_up(1 * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE BOOT_MIN_SIZE_S = align_up(50 * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE ROOT_MIN_SIZE_S = align_up(50 * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE def __init__(self): super(BoardConfig, self).__init__() # XXX: when killing v1 hwpack, we might rename these two without the # leading underscore. It is done in this way since sublcasses use # placeholders in the string for dinamically change values. But this # is done only for hwpack v1. self._extra_serial_options = '' self._live_serial_options = '' self.board = None self.boot_script = None self.bootloader_dd = False self.bootloader_file_in_boot_part = False self.bootloader_flavor = None self.dtb_addr = None self.dtb_file = None self.dtb_files = None self.dtb_name = None self.env_dd = False self.extra_boot_args_options = None self.fat_size = 32 self.fatload_command = 'fatload' self.load_interface = 'mmc' self.bootfs_type = 'vfat' self.fdt_high = '0xffffffff' self.hardwarepack_handler = None self.hwpack_format = None self.initrd_addr = None self.initrd_high = '0xffffffff' self.kernel_addr = None self.kernel_flavors = None self.load_addr = None self.loader_start_s = 1 self.mmc_device_id = 0 self.mmc_id = None self.mmc_option = '0:1' self.mmc_part_offset = 0 self.partition_layout = None self.serial_tty = None self.spl_dd = False self.spl_in_boot_part = False self.supports_writing_to_mmc = True self.uimage_path = '' self.wired_interfaces = None self.wireless_interfaces = None # Samsung Boot-loader implementation notes and terminology # # BL0, BL1, BL2, BL3 are the boot loader stages in order of execution # # BL0 - embedded boot loader on the internal ROM # BL1 - chip-specific boot loader provided by Samsung # BL2 - tiny boot loader; SPL (Second Program Loader) # BL3 - customized boot loader; U-Boot # # In linaro-image-tools, variables have been named samsung_blN-1 # e.g BL1 is samsung_bl0, BL2 is samsung_bl1, BL3 is samsung_bl2 # # samsung_bl0_{start,len}: Offset and maximum size for BL1 # samsung_bl1_{start,len}: Offset and maximum size for BL2 # samsung_bl2_{start,len}: Offset and maximum size for BL3 # samsung_env_{start,len}: Offset and maximum size for settings # self.samsung_bl1_start = 1 self.samsung_bl1_len = 32 self.samsung_bl2_start = 65 self.samsung_bl2_len = 1024 self.samsung_env_start = 33 self.samsung_env_len = 32 # XXX: attributes that are not listed in hwpackV3, should be removed? self.vmlinuz = None self.initrd = None # XXX: can be removed when killing v1 hwpack. def _get_live_serial_options(self): live_serial = self._live_serial_options if live_serial: if isinstance(live_serial, list): live_serial = ' '.join(live_serial) if self._check_placeholder_presence(live_serial, r'%s'): live_serial = live_serial % self.serial_tty return live_serial def _set_live_serial_options(self, value): self._live_serial_options = value live_serial_options = property(_get_live_serial_options, _set_live_serial_options) # XXX: can be removed when killing v1 hwpack. def _get_extra_serial_options(self): extra_serial = self._extra_serial_options if extra_serial: if isinstance(extra_serial, list): extra_serial = ' '.join(extra_serial) if self._check_placeholder_presence(extra_serial, r'%s'): extra_serial = extra_serial % self.serial_tty return extra_serial def _set_extra_serial_options(self, value): self._extra_serial_options = value extra_serial_options = property(_get_extra_serial_options, _set_extra_serial_options) def get_metadata_field(self, field_name): """ Return the metadata value for field_name if it can be found. """ data, _ = self.hardwarepack_handler.get_field(field_name) return data def set_metadata(self, hwpacks, bootloader=None, board=None, dtb_file=None): self.hardwarepack_handler = HardwarepackHandler(hwpacks, bootloader, board) with self.hardwarepack_handler: self.hwpack_format = self.hardwarepack_handler.get_format() if (self.hwpack_format == self.hardwarepack_handler.FORMAT_1): self.bootloader_copy_files = None return if (self.hwpack_format != self.hardwarepack_handler.FORMAT_1): # Clear V1 defaults. # TODO When removing v1 support, remove also default values # in the constructor and avoid all this. self.kernel_addr = None self.initrd_addr = None self.load_addr = None self.serial_tty = None self.fat_size = None self.dtb_name = None self.dtb_addr = None self.extra_boot_args_options = None self.boot_script = None self.kernel_flavors = None self.mmc_option = None self.mmc_part_offset = None self.samsung_bl1_start = None self.samsung_bl1_len = None self.samsung_env_len = None self.samsung_bl2_len = None # self.samsung_bl2_start and self.samsung_env_start should # be initialized to default value for backward compatibility. self.board = board # Set new values from metadata. self.kernel_addr = self.get_metadata_field(KERNEL_ADDR_FIELD) self.initrd_addr = self.get_metadata_field(INITRD_ADDR_FIELD) self.load_addr = self.get_metadata_field(LOAD_ADDR_FIELD) self.dtb_addr = self.get_metadata_field(DTB_ADDR_FIELD) self.serial_tty = self.get_metadata_field(SERIAL_TTY_FIELD) wired_interfaces = self.get_metadata_field(WIRED_INTERFACES_FIELD) if wired_interfaces: self.wired_interfaces = wired_interfaces wireless_interfaces = self.get_metadata_field( WIRELESS_INTERFACES_FIELD) if wireless_interfaces: self.wireless_interfaces = wireless_interfaces self.dtb_file = self.get_metadata_field(DTB_FILE_FIELD) # XXX: need to deprecate dtb_file field and use only dtb_files # for multiple entries. if self.dtb_file: logger.warn("Deprecation warning: use the 'dtb_files' field " "instead of 'dtb_file'.") self.dtb_files = self.get_metadata_field(DTB_FILES_FIELD) if dtb_file: dtb_dict = self._find_dtb_dict(dtb_file) if dtb_dict: self.dtb_files = [] self.dtb_files.append(dtb_dict) self.extra_boot_args_options = self.get_metadata_field( EXTRA_BOOT_OPTIONS_FIELD) self.boot_script = self.get_metadata_field(BOOT_SCRIPT_FIELD) self.extra_serial_options = self.get_metadata_field( EXTRA_SERIAL_OPTIONS_FIELD) self.snowball_startup_files_config = self.get_metadata_field( SNOWBALL_STARTUP_FILES_CONFIG_FIELD) self.partition_layout = self.get_metadata_field( PARTITION_LAYOUT_FIELD) if self.partition_layout in [BOOTFS, RESERVED_BOOTFS, None]: self.fat_size = 32 elif self.partition_layout == BOOTFS16: self.fat_size = 16 else: raise AssertionError("Unknown partition layout '%s'." % self.partition_layout) self.mmc_option = self.get_metadata_field(MMC_ID_FIELD) if self.mmc_option: self.mmc_device_id = int(self.mmc_option.split(':')[0]) self.mmc_part_offset = int(self.mmc_option.split(':')[1]) - 1 # XXX: need to fix these values. boot_min_size = self.get_metadata_field(BOOT_MIN_SIZE_FIELD) if boot_min_size: self.BOOT_MIN_SIZE_S = align_up(int(boot_min_size) * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE root_min_size = self.get_metadata_field(ROOT_MIN_SIZE_FIELD) if root_min_size: self.ROOT_MIN_SIZE_S = align_up(int(root_min_size) * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE loader_min_size = self.get_metadata_field(LOADER_MIN_SIZE_FIELD) if loader_min_size: self.LOADER_MIN_SIZE_S = ( align_up(int(loader_min_size) * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE) spl_in_boot_part = self.get_metadata_field(SPL_IN_BOOT_PART_FIELD) if spl_in_boot_part is None: self.spl_in_boot_part = False elif string.lower(spl_in_boot_part) == 'yes': self.spl_in_boot_part = True elif string.lower(spl_in_boot_part) == 'no': self.spl_in_boot_part = False env_dd = self.get_metadata_field(ENV_DD_FIELD) if env_dd is None: self.env_dd = False elif string.lower(env_dd) == 'yes': self.env_dd = True elif string.lower(env_dd) == 'no': self.env_dd = False # XXX: in hwpack v3 this field is just called 'dd'. # Need to check its use. bootloader_dd = self.get_metadata_field('bootloader_dd') # Either bootloader_dd is not specified, or it contains the dd # offset. if bootloader_dd is None: self.bootloader_dd = False else: self.bootloader_dd = int(bootloader_dd) spl_dd = self.get_metadata_field(SPL_DD_FIELD) # Either spl_dd is not specified, or it contains the dd offset. if spl_dd is None: self.spl_dd = False else: self.spl_dd = int(spl_dd) loader_start = self.get_metadata_field(LOADER_START_FIELD) if loader_start: self.loader_start_s = int(loader_start) samsung_bl1_start = self.get_metadata_field( SAMSUNG_BL1_START_FIELD) if samsung_bl1_start: self.samsung_bl1_start = int(samsung_bl1_start) samsung_bl1_len = self.get_metadata_field( SAMSUNG_BL1_LEN_FIELD) if samsung_bl1_len: self.samsung_bl1_len = int(samsung_bl1_len) samsung_bl2_start = self.get_metadata_field( SAMSUNG_BL2_START_FIELD) if samsung_bl2_start: self.samsung_bl2_start = int(samsung_bl2_start) samsung_bl2_len = self.get_metadata_field( SAMSUNG_BL2_LEN_FIELD) if samsung_bl2_len: self.samsung_bl2_len = int(samsung_bl2_len) samsung_env_start = self.get_metadata_field( SAMSUNG_ENV_START_FIELD) if samsung_env_start is not None: self.samsung_env_start = int(samsung_env_start) samsung_env_len = self.get_metadata_field( SAMSUNG_ENV_LEN_FIELD) if samsung_env_len: self.samsung_env_len = int(samsung_env_len) self.bootloader_copy_files = self.hardwarepack_handler.get_field( "bootloader_copy_files")[0] # XXX: no reference in hwpackV3 format of these fields, double # check if they can be dropped when killing v1. self.bootloader = self.hardwarepack_handler.get_field( "bootloader") self.vmlinuz = self.get_metadata_field('vmlinuz') self.initrd = self.get_metadata_field('initrd') bootloader_file_in_boot_part = self.get_metadata_field( 'bootloader_file_in_boot_part') if bootloader_file_in_boot_part is None: self.bootloader_file_in_boot_part = False elif string.lower(bootloader_file_in_boot_part) == 'yes': self.bootloader_file_in_boot_part = True elif string.lower(bootloader_file_in_boot_part) == 'no': self.bootloader_file_in_boot_part = False def get_file(self, file_alias, default=None): # XXX remove the 'default' parameter when V1 support is removed! file_in_hwpack = self.hardwarepack_handler.get_file(file_alias) if file_in_hwpack is not None: return file_in_hwpack else: return default def get_v1_sfdisk_cmd(self, should_align_boot_part=False): # XXX: This default implementation and all overrides are left for V1 # compatibility only. They should be removed as part of the work to # kill off hwpacks V1. return self.get_normal_sfdisk_cmd(should_align_boot_part) def get_normal_params(self, should_align_boot_part=False): if self.bootfs_type == 'vfat': if self.fat_size == 32: partition_type = '0x0C' else: partition_type = '0x0E' else: partition_type = '0x83' # align on sector 63 for compatibility with broken versions of x-loader # unless align_boot_part is set # XXX OMAP specific, might break other boards? boot_align = 63 if should_align_boot_part: boot_align = PART_ALIGN_S # can only start on sector 1 (sector 0 is MBR / partition table) boot_start, boot_end, boot_len = align_partition( 1, self.BOOT_MIN_SIZE_S, boot_align, PART_ALIGN_S) # apparently OMAP3 ROMs require the vfat length to be an even number # of sectors (multiple of 1 KiB); decrease the length if it's odd, # there should still be enough room # XXX OMAP specific, might break other boards? boot_len = boot_len - boot_len % 2 boot_end = boot_start + boot_len - 1 # we ignore _root_end / _root_len and return a sfdisk command to # instruct the use of all remaining space; XXX we now have root size # config, so we can do something more sensible root_start, _root_end, _root_len = align_partition( boot_end + 1, self.ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) return (boot_start, boot_len, partition_type, root_start) def get_reserved_params(self, should_align_boot_part=None): loader_start, loader_end, loader_len = align_partition( self.loader_start_s, self.LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) boot_start, boot_end, boot_len = align_partition( loader_end + 1, self.BOOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) root_start, _root_end, _root_len = align_partition( boot_end + 1, self.ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) return (loader_start, loader_len, boot_start, boot_len, root_start) def get_normal_sfdisk_cmd(self, should_align_boot_part=False): """Return the sfdisk command to partition the media. :param should_align_boot_part: Whether to align the boot partition too. This returns a boot partition of type FAT16 or FAT32 or Linux, followed by a root partition. """ (boot_start, boot_len, partition_type, root_start) = self.get_normal_params(should_align_boot_part) return '%s,%s,%s,*\n%s,,,-' % ( boot_start, boot_len, partition_type, root_start) def get_reserved_sfdisk_cmd(self, should_align_boot_part=None): """Return the sfdisk command to partition the media. :param should_align_boot_part: Ignored. This returns a loader partition, then a boot vfat partition of type FAT16 or FAT32, followed by a root partition. """ (loader_start, loader_len, boot_start, boot_len, root_start) = self.get_reserved_params(should_align_boot_part) return '%s,%s,0xDA\n%s,%s,0x0C,*\n%s,,,-' % ( loader_start, loader_len, boot_start, boot_len, root_start) def get_normal_sgdisk_cmd(self, should_align_boot_part=None): (boot_start, boot_len, partition_type, root_start) = self.get_normal_params(should_align_boot_part) # Ignoring partition type because we need 0xEF for GPT return '-n 1:%s:%s -t 1:EF00 ' \ '-n 2:%s:- -t 2:8300' % (boot_start, boot_len, root_start) def get_reserved_sgdisk_cmd(self, should_align_boot_part=None): """Return the sgdisk command to partition the media. :param should_align_boot_part: Ignored. """ (loader_start, loader_len, boot_start, boot_len, root_start) = self.get_reserved_params(should_align_boot_part) return '-n 1:%s:%s -t 1:DA00 ' \ '-n 2:%s:%s -t 2:0C00 ' \ '-n 3:%s:- -t 3:8300' % ( loader_start, loader_len, boot_start, boot_len, root_start) def get_sfdisk_cmd(self, should_align_boot_part=False): if (self.partition_layout in ['bootfs_rootfs', 'bootfs16_rootfs'] or self.board == 'snowball_sd'): return self.get_normal_sfdisk_cmd(should_align_boot_part) elif self.partition_layout in ['reserved_bootfs_rootfs']: return self.get_reserved_sfdisk_cmd(should_align_boot_part) else: assert (self.hwpack_format == HardwarepackHandler.FORMAT_1), ( "Hwpack format is not 1.0 but " "partition_layout is unspecified.") return self.get_v1_sfdisk_cmd(should_align_boot_part) def get_sgdisk_cmd(self, should_align_boot_part=False): if (self.partition_layout in ['bootfs_rootfs', 'bootfs16_rootfs'] or self.board == 'snowball_sd'): return self.get_normal_sgdisk_cmd(should_align_boot_part) elif self.partition_layout in ['reserved_bootfs_rootfs']: return self.get_reserved_sgdisk_cmd(should_align_boot_part) else: assert (self.hwpack_format == HardwarepackHandler.FORMAT_1), ( "Hwpack format is not 1.0 but " "partition_layout is unspecified.") return self.get_normal_sgdisk_cmd(should_align_boot_part) def _get_bootcmd(self, i_img_data, d_img_data): """Get the bootcmd for this board. In general subclasses should not have to override this. """ replacements = dict( fatload_command=self.fatload_command, uimage_path=self.uimage_path, mmc_option=self.mmc_option, kernel_addr=self.kernel_addr, initrd_addr=self.initrd_addr, dtb_addr=self.dtb_addr, load_interface=self.load_interface) boot_script = ( ("%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(kernel_addr)s %(uimage_path)suImage; ")) % replacements boot_script_bootm = (("bootm %(kernel_addr)s")) % replacements if i_img_data is not None and d_img_data is not None: boot_script += ( ("%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(initrd_addr)s %(uimage_path)suInitrd; " "%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(dtb_addr)s board.dtb; ")) % replacements boot_script_bootm += ( (" %(initrd_addr)s %(dtb_addr)s")) % replacements elif i_img_data is None and d_img_data is not None: boot_script += ( ("%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(dtb_addr)s board.dtb; ")) % replacements boot_script_bootm += ((" - %(dtb_addr)s")) % replacements elif i_img_data is not None and d_img_data is None: boot_script += ( ("%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(initrd_addr)s %(uimage_path)suInitrd; ")) % replacements boot_script_bootm += ((" %(initrd_addr)s")) % replacements boot_script += boot_script_bootm return boot_script def add_boot_args(self, extra_args): if extra_args is not None: if self.extra_boot_args_options is None: self.extra_boot_args_options = extra_args else: self.extra_boot_args_options += ' %s' % extra_args def add_boot_args_from_file(self, path): if path is not None: with open(path, 'r') as boot_args_file: self.add_boot_args(boot_args_file.read().strip()) def _get_bootargs(self, is_live, is_lowmem, consoles, rootfs_id): """Get the bootargs for this board. In general subclasses should not have to override this. """ boot_args_options = 'rootwait ro' serial_options = '' if self.extra_boot_args_options: boot_args_options += ' %s' % self.extra_boot_args_options.strip() if self.extra_serial_options: serial_options = self.extra_serial_options.strip() for console in consoles: serial_options += ' console=%s' % console.strip() lowmem_opt = '' boot_snippet = 'root=%s' % rootfs_id.strip() if is_live: serial_options += ' %s' % self.live_serial_options boot_snippet = 'boot=casper' if is_lowmem: lowmem_opt = 'only-ubiquity' replacements = dict(serial_options=serial_options.strip(), lowmem_opt=lowmem_opt, boot_snippet=boot_snippet, boot_args_options=boot_args_options) boot_args = ("%(serial_options)s %(lowmem_opt)s %(boot_snippet)s" " %(boot_args_options)s" % replacements).strip() return boot_args def _get_boot_env(self, is_live, is_lowmem, consoles, rootfs_id, i_img_data, d_img_data): """Get the boot environment for this board. In general subclasses should not have to override this. """ boot_env = {} boot_env["bootargs"] = self._get_bootargs( is_live, is_lowmem, consoles, rootfs_id) boot_env["bootcmd"] = self._get_bootcmd(i_img_data, d_img_data) boot_env["initrd_high"] = self.initrd_high boot_env["fdt_high"] = self.fdt_high return boot_env def make_boot_files(self, bootloader_parts_dir, is_live, is_lowmem, consoles, chroot_dir, rootfs_id, boot_dir, boot_device_or_file): if self.hwpack_format == HardwarepackHandler.FORMAT_1: parts_dir = bootloader_parts_dir else: parts_dir = chroot_dir (k_img_data, i_img_data, d_img_data) = self._get_kflavor_files( parts_dir) boot_env = self._get_boot_env(is_live, is_lowmem, consoles, rootfs_id, i_img_data, d_img_data) if self.hwpack_format == HardwarepackHandler.FORMAT_1: self._make_boot_files( boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data) else: self._make_boot_files_v2( boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data) def _copy_dtb_files(self, dtb_files, dest_dir, search_dir): """Copy the files defined in dtb_files into the boot directory. :param dtb_files: The list of dtb files :param dest_dir: The directory where to copy each dtb file. :param search_dir: The directory where to search for the real file. """ logger.info("Copying dtb files") for dtb_file in dtb_files: if dtb_file: if isinstance(dtb_file, dict): for key, value in dtb_file.iteritems(): # The name of the dtb file in the new position. to_file = os.path.basename(key) # The directory where to copy the dtb file. to_dir = os.path.join(dest_dir, os.path.dirname(key)) from_file = value # User specified only the directory, without renaming # the file. if not to_file: to_file = os.path.basename(from_file) if not os.path.exists(to_dir): cmd_runner.run(["mkdir", "-p", to_dir], as_root=True).wait() dtb = _get_file_matching(os.path.join(search_dir, from_file)) if not dtb: logger.warn('Could not find a valid dtb file, ' 'skipping it.') continue else: dest = os.path.join(to_dir, to_file) logger.debug('Copying %s into %s' % (dtb, dest)) cmd_runner.run(['cp', dtb, dest], as_root=True).wait() else: # Hopefully we should never get here. # This should only happen if the hwpack config YAML file is # wrong. logger.warn('WARNING: Wrong syntax in metadata file. ' 'Check the hwpack configuration file used to ' 'generate the hwpack archive.') def _dd_file(self, from_file, to_file, seek, max_size=None): assert from_file is not None, "No source file name given." if max_size is not None: assert os.path.getsize(from_file) <= max_size, ( "'%s' is larger than %s" % (from_file, max_size)) logger.info("Writing '%s' to '%s' at %s." % (from_file, to_file, seek)) _dd(from_file, to_file, seek=seek) def install_samsung_boot_loader(self, samsung_spl_file, bootloader_file, boot_device_or_file): self._dd_file(samsung_spl_file, boot_device_or_file, self.samsung_bl1_start, self.samsung_bl1_len * SECTOR_SIZE) self._dd_file(bootloader_file, boot_device_or_file, self.samsung_bl2_start, self.samsung_bl2_len * SECTOR_SIZE) def _make_boot_files_v2(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): with self.hardwarepack_handler: spl_file = self.get_file('spl_file') if self.spl_in_boot_part: assert spl_file is not None, ( "SPL binary could not be found") logger.info( "Copying spl '%s' to boot partition." % spl_file) cmd_runner.run(["cp", "-v", spl_file, boot_dir], as_root=True).wait() if self.spl_dd: self._dd_file(spl_file, boot_device_or_file, self.spl_dd) bootloader_file = self.get_file('bootloader_file') if self.bootloader_dd: self._dd_file(bootloader_file, boot_device_or_file, self.bootloader_dd) make_uImage(self.load_addr, k_img_data, boot_dir) if i_img_data is not None: make_uInitrd(i_img_data, boot_dir) if d_img_data is not None: make_dtb(d_img_data, boot_dir) if self.boot_script is not None: boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) # Only used for Omap, will this be bad for the other boards? make_boot_ini(boot_script_path, boot_dir) if (self.snowball_startup_files_config is not None and self.board != 'snowball_sd'): self.populate_raw_partition(boot_device_or_file, chroot_dir) if self.env_dd: # Do we need to zero out the env before flashing it? _dd("/dev/zero", boot_device_or_file, count=self.samsung_env_len, seek=self.samsung_env_start) env_size = self.samsung_env_len * SECTOR_SIZE env_file = make_flashable_env(boot_env, env_size) self._dd_file(env_file, boot_device_or_file, self.samsung_env_start) def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): """Make the necessary boot files for this board. This is usually board-specific so ought to be defined in every subclass. """ raise NotImplementedError() def populate_boot(self, chroot_dir, rootfs_id, boot_partition, boot_disk, boot_device_or_file, is_live, is_lowmem, consoles): parts_dir = 'boot' if is_live: parts_dir = 'casper' bootloader_parts_dir = os.path.join(chroot_dir, parts_dir) cmd_runner.run(['mkdir', '-p', boot_disk]).wait() with partition_mounted(boot_partition, boot_disk): with self.hardwarepack_handler: if self.bootloader_file_in_boot_part: # if self.bootloader_flavor is not None: default = os.path.join( chroot_dir, 'usr', 'lib', 'u-boot', self.bootloader_flavor, 'u-boot.img') if not os.path.exists(default): default = os.path.join( chroot_dir, 'usr', 'lib', 'u-boot', self.bootloader_flavor, 'u-boot.bin') else: default = None # bootloader_bin = self.get_file('bootloader_file', default=default) assert bootloader_bin is not None, ( "bootloader binary could not be found") proc = cmd_runner.run( ['cp', '-v', bootloader_bin, boot_disk], as_root=True) proc.wait() # Handle copy_files field. self.copy_files(boot_disk) # Handle dtb_files field. if self.dtb_files: self._copy_dtb_files(self.dtb_files, boot_disk, chroot_dir) self.make_boot_files( bootloader_parts_dir, is_live, is_lowmem, consoles, chroot_dir, rootfs_id, boot_disk, boot_device_or_file) def copy_files(self, boot_disk): """Handle the copy_files metadata field.""" # Extract anything specified by copy_files sections # self.bootloader_copy_files is always of the form: # {'source_package': # [ # {'source_path': 'dest_path'} # ] # } if self.bootloader_copy_files is None: return for source_package, file_list in \ self.bootloader_copy_files.iteritems(): for file_info in file_list: for source_path, dest_path in file_info.iteritems(): source = self.hardwarepack_handler.get_file_from_package( source_path, source_package) dest_path = dest_path.lstrip("/\\") dirname = os.path.dirname(dest_path) dirname = os.path.join(boot_disk, dirname) if not os.path.exists(dirname): cmd_runner.run(["mkdir", "-p", dirname], as_root=True).wait() proc = cmd_runner.run( ['cp', '-v', source, os.path.join(boot_disk, dest_path)], as_root=True) proc.wait() def _get_kflavor_files(self, path): """Search for kernel, initrd and optional dtb in path.""" if self.kernel_flavors is None: # V2 metadata specifies each glob, not flavors. # XXX This duplication is temporary until V1 dies. return self._get_kflavor_files_v2(path) for flavor in self.kernel_flavors: kregex = KERNEL_GLOB % {'kernel_flavor': flavor} iregex = INITRD_GLOB % {'kernel_flavor': flavor} dregex = DTB_GLOB % {'kernel_flavor': flavor, 'dtb_name': self.dtb_name} kernel = _get_file_matching(os.path.join(path, kregex)) if kernel is not None: initrd = _get_file_matching(os.path.join(path, iregex)) if initrd is not None: dtb = None if self.dtb_name is not None: dtb = _get_file_matching(os.path.join(path, dregex)) return (kernel, initrd, dtb) raise ValueError( "Found kernel for flavor %s but no initrd matching %s" % ( flavor, iregex)) raise ValueError( "No kernel found matching %s for flavors %s" % ( KERNEL_GLOB, " ".join(self.kernel_flavors))) def _get_kflavor_files_v2(self, path): kernel = initrd = dtb = None if self.vmlinuz: kernel = _get_file_matching(os.path.join(path, self.vmlinuz)) if not self.vmlinuz or not kernel: raise ValueError("Unable to find a valid kernel image.") if self.initrd: initrd = _get_file_matching(os.path.join(path, self.initrd)) if not self.initrd or not initrd: logger.warn("Could not find a valid initrd, skipping uInitrd.") if self.dtb_file: dtb = _get_file_matching(os.path.join(path, self.dtb_file)) if not self.dtb_file or not dtb: logger.warn("Could not find a valid dtb file from dtb_file, " "trying dtb_files...") if self.dtb_files: # Use first file from list as a default dtb file. dtb_file = self.dtb_files[0] if dtb_file: if isinstance(dtb_file, dict): for key, value in dtb_file.iteritems(): # The name of the dtb file. to_file = os.path.basename(key) from_file = value # User specified only the directory, without renaming # the file. if not to_file: to_file = os.path.basename(from_file) dtb = _get_file_matching(os.path.join(path, from_file)) if not self.dtb_files or not dtb: logger.warn("Could not find a valid dtb file, skipping it.") logger.info("Will use kernel=%s, initrd=%s, dtb=%s." % (kernel, initrd, dtb)) return (kernel, initrd, dtb) def populate_raw_partition(self, media, boot_dir): # Override in subclass if needed pass def snowball_config(self, chroot_dir): # Override in subclasses where applicable raise NotImplementedError( "snowball_config() must only be called on BoardConfigs that " "use the Snowball startupfiles.") # XXX: can be removed when killing v1 hwpack and updating the attributes # that use it. @staticmethod def _check_placeholder_presence(string, placeholder): """Checks if the passed string contains the particular placeholder.""" # Very simple way of achieving that. presence = False if string and placeholder in string: presence = True return presence def _find_dtb_dict(self, dtb): """Returns dictionary entry from dt_files containing dtb file.""" for dtb_file in self.dtb_files: if isinstance(dtb_file, dict): for key, value in dtb_file.iteritems(): # The name of the dtb file. if dtb in key: return dtb_file return None class OmapConfig(BoardConfig): def __init__(self): super(OmapConfig, self).__init__() self.kernel_flavors = ['linaro-omap4', 'linaro-lt-omap', 'linaro-omap', 'omap4'] self.bootloader_file_in_boot_part = True # XXX: Here we define these things as dynamic properties because our # temporary hack to fix bug 697824 relies on changing the board's # serial_tty at run time. self._serial_tty = None # XXX: when killing v1 hwpack this should be safely removed. def _get_serial_tty(self): return self._serial_tty def _set_serial_tty(self, value): self._serial_tty = value serial_tty = property(_get_serial_tty, _set_serial_tty) def set_appropriate_serial_tty(self, chroot_dir): """Set the appropriate serial_tty depending on the kernel used. If the kernel found in the chroot dir is << 2.6.36 we use tyyS2, else we use the default value (_serial_tty). """ # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 # XXX: This is also part of our temporary hack to fix bug 697824. # cls.serial_tty = classproperty(lambda cls: cls._serial_tty) vmlinuz = _get_file_matching( os.path.join(chroot_dir, 'boot', 'vmlinuz*')) basename = os.path.basename(vmlinuz) match = re.match('.*2\.6\.([0-9]{2}).*', basename) # Assume if it doesn't match that it is 3.0 or later. if match is not None: minor_version = match.group(1) if int(minor_version) < 36: self.serial_tty = 'ttyS2' def make_boot_files(self, bootloader_parts_dir, is_live, is_lowmem, consoles, chroot_dir, rootfs_id, boot_dir, boot_device_or_file): # XXX: This is also part of our temporary hack to fix bug 697824; we # need to call set_appropriate_serial_tty() before doing anything that # may use self.serial_tty. if self.hwpack_format == HardwarepackHandler.FORMAT_1: self.set_appropriate_serial_tty(chroot_dir) super(OmapConfig, self).make_boot_files( bootloader_parts_dir, is_live, is_lowmem, consoles, chroot_dir, rootfs_id, boot_dir, boot_device_or_file) def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 install_omap_boot_loader(chroot_dir, boot_dir, self) make_uImage(self.load_addr, k_img_data, boot_dir) make_uInitrd(i_img_data, boot_dir) make_dtb(d_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) make_boot_ini(boot_script_path, boot_dir) class BeagleConfig(OmapConfig): def __init__(self): super(BeagleConfig, self).__init__() self.boot_script = 'boot.scr' self.bootloader_flavor = 'omap3_beagle' self.dtb_addr = '0x815f0000' self.dtb_name = 'omap3-beagle.dtb' self.extra_boot_args_options = ( 'earlyprintk fixrtc nocompcache vram=12M ' 'omapfb.mode=dvi:1280x720MR-16@60 mpurate=${mpurate}') self.initrd_addr = '0x81600000' self.kernel_addr = '0x80000000' self.load_addr = '0x80008000' self._serial_tty = 'ttyO2' self._extra_serial_options = 'console=tty0 console=%s,115200n8' self._live_serial_options = 'serialtty=%s' class OveroConfig(OmapConfig): def __init__(self): super(OveroConfig, self).__init__() self.boot_script = 'boot.scr' self.bootloader_flavor = 'omap3_overo' self.dtb_addr = '0x815f0000' self.dtb_name = 'omap3-overo.dtb' self.extra_boot_args_options = ( 'earlyprintk mpurate=${mpurate} vram=12M ' 'omapdss.def_disp=${defaultdisplay} omapfb.mode=dvi:${dvimode}') self.initrd_addr = '0x81600000' self.kernel_addr = '0x80000000' self.load_addr = '0x80008000' self._extra_serial_options = 'console=tty0 console=%s,115200n8' self._serial_tty = 'ttyO2' class PandaConfig(OmapConfig): def __init__(self): super(PandaConfig, self).__init__() self._serial_tty = 'ttyO2' self.boot_script = 'boot.scr' self.bootloader_flavor = 'omap4_panda' self.dtb_addr = '0x815f0000' self.dtb_name = 'omap4-panda.dtb' self.extra_boot_args_options = ( 'earlyprintk fixrtc nocompcache vram=48M ' 'omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000') self.initrd_addr = '0x81600000' self.kernel_addr = '0x80200000' self.load_addr = '0x80008000' self._extra_serial_options = 'console=tty0 console=%s,115200n8' self._live_serial_options = 'serialtty=%s' class BeagleBoneConfig(OmapConfig): def __init__(self): super(BeagleBoneConfig, self).__init__() self.boot_script = 'boot.scr' self.bootloader_flavor = 'am335x_evm' self.kernel_flavors = ['am335x'] self._serial_tty = 'ttyO0' self.dtb_addr = '0x815f0000' self.initrd_addr = '0x81600000' self.kernel_addr = '0x80200000' self.load_addr = '0x80008000' self.extra_boot_args_options = ('fixrtc') self._extra_serial_options = 'console=ttyO0,115200n8' class IgepConfig(BeagleConfig): def __init__(self): super(IgepConfig, self).__init__() self.bootloader_file_in_boot_part = False self.bootloader_flavor = None self.dtb_name = 'isee-igep-v2.dtb' def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 make_uImage(self.load_addr, k_img_data, boot_dir) make_uInitrd(i_img_data, boot_dir) make_dtb(d_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) make_boot_ini(boot_script_path, boot_dir) class Ux500Config(BoardConfig): def __init__(self): super(Ux500Config, self).__init__() self.boot_script = 'flash.scr' self.extra_boot_args_options = ( 'earlyprintk rootdelay=1 fixrtc nocompcache ' 'mem=96M@0 mem_modem=32M@96M mem=44M@128M pmem=22M@172M ' 'mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M ' 'hwmem=48M@302M mem=152M@360M') self.initrd_addr = '0x08000000' self.kernel_addr = '0x00100000' self.kernel_flavors = ['u8500', 'ux500'] self.load_addr = '0x00008000' self.mmc_option = '1:1' self.serial_tty = 'ttyAMA2' self._extra_serial_options = 'console=tty0 console=%s,115200n8' self._live_serial_options = 'serialtty=%s' def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 make_uImage(self.load_addr, k_img_data, boot_dir) make_uInitrd(i_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) class SnowballSdConfig(Ux500Config): '''Use only with --mmc option. Creates the standard vfat and ext2 partitions for kernel and rootfs on an SD card. Note that the Snowball board needs a loader partition on the internal eMMC flash to boot. That partition is created with the SnowballConfigImage configuration.''' def __init__(self): super(SnowballSdConfig, self).__init__() def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 make_uImage(self.load_addr, k_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) class SnowballEmmcConfig(SnowballSdConfig): '''Use only with --image option. Creates a raw image which contains an additional (raw) loader partition, containing some boot stages and u-boot.''' SNOWBALL_LOADER_START_S = (128 * 1024) / SECTOR_SIZE TOC_SIZE = 512 def __init__(self): super(SnowballEmmcConfig, self).__init__() # Boot ROM looks for a boot table of contents (TOC) at 0x20000 # Actually, it first looks at address 0, but that's where l-m-c # puts the MBR, so the boot loader skips that address. self.supports_writing_to_mmc = False self.snowball_startup_files_config = 'startfiles.cfg' self.mmc_option = '0:2' def get_v1_sfdisk_cmd(self, should_align_boot_part=None): """Return the sfdisk command to partition the media. :param should_align_boot_part: Ignored. The Snowball partitioning scheme depends on whether the target is a raw image or an SD card. Both targets have the normal FAT 32 boot partition and EXT? root partition. The raw image prepends these two partitions with a raw loader partition, containing HW-dependent boot stages up to and including u-boot. This is done since the boot rom always boots off the internal memory; there simply is no point to having a loader partition on SD card. """ # boot ROM expects bootloader at 0x20000, which is sector 0x100 # with the usual SECTOR_SIZE of 0x200. # (sector 0 is MBR / partition table) loader_start, loader_end, loader_len = align_partition( self.SNOWBALL_LOADER_START_S, self.LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) boot_start, boot_end, boot_len = align_partition( loader_end + 1, self.BOOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) # we ignore _root_end / _root_len and return an sfdisk command to # instruct the use of all remaining space; XXX if we had some root size # config, we could do something more sensible root_start, _root_end, _root_len = align_partition( boot_end + 1, self.ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) return '%s,%s,0xDA\n%s,%s,0x0C,*\n%s,,,-' % ( loader_start, loader_len, boot_start, boot_len, root_start) def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 make_uImage(self.load_addr, k_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) self.populate_raw_partition(boot_device_or_file, chroot_dir) def populate_raw_partition(self, boot_device_or_file, chroot_dir): # Populate created raw partition with TOC and startup files. _, toc_filename = tempfile.mkstemp() config_files_dir = self.snowball_config(chroot_dir) new_files = self.get_file_info(chroot_dir, config_files_dir) with open(toc_filename, 'wb') as toc: self.create_toc(toc, new_files) self.install_snowball_boot_loader(toc_filename, new_files, boot_device_or_file, self.SNOWBALL_LOADER_START_S, self.delete_startupfiles) self.delete_file(toc_filename) if self.delete_startupfiles: self.delete_file(os.path.join(config_files_dir, self.snowball_startup_files_config)) def snowball_config(self, chroot_dir): # We will find the startupfiles in the target boot partition. return os.path.join(chroot_dir, 'boot') @property def delete_startupfiles(self): # The startupfiles will have been installed to the target boot # partition by the hwpack, and should be deleted so we don't leave # them on the target system. return True def install_snowball_boot_loader(self, toc_file_name, files, boot_device_or_file, start_sector, delete_startupfiles=False): ''' Copies TOC and boot files into the boot partition. A sector size of 1 is used for some files, as they do not necessarily start on an even address. ''' assert os.path.getsize(toc_file_name) <= self.TOC_SIZE _dd(toc_file_name, boot_device_or_file, seek=start_sector) for file in files: # XXX We need checks that these files do not overwrite each # other. This code assumes that offset and file sizes are ok. filename = file['filename'] if (file['offset'] % SECTOR_SIZE) != 0: seek_bytes = start_sector * SECTOR_SIZE + file['offset'] _dd(filename, boot_device_or_file, block_size=1, seek=seek_bytes) else: seek_sectors = start_sector + file['offset'] / SECTOR_SIZE _dd(filename, boot_device_or_file, seek=seek_sectors) if delete_startupfiles: self.delete_file(filename) def delete_file(self, file_path): cmd = ["rm", "%s" % file_path] proc = cmd_runner.run(cmd, as_root=True) proc.wait() def create_toc(self, f, files): ''' Writes a table of contents of the boot binaries. Boot rom searches this table to find the binaries.''' # Format string means: < little endian, # I; unsigned int; offset, # I; unsigned int; size, # I; unsigned int; flags, # i; int; align, # i; int; load_address, # 12s; string of char; name # http://igloocommunity.org/support/index.php/ConfigPartitionOverview toc_format = '/MLO) files = glob.glob( os.path.join(chroot_dir, 'usr', 'lib', '*', '*', 'MLO')) if len(files) == 0: # This one matches the path of MLO files installed by older # x-loader-omap package (e.g. /usr/lib/x-loader-omap[34]/MLO) files = glob.glob( os.path.join(chroot_dir, 'usr', 'lib', '*', 'MLO')) if len(files) == 1: return files[0] elif len(files) > 1: raise AssertionError( "More than one MLO file found on %s" % chroot_dir) else: raise AssertionError("No MLO files found on %s" % chroot_dir) def install_omap_boot_loader(chroot_dir, boot_disk, cls): with cls.hardwarepack_handler: try: default = _get_mlo_file(chroot_dir) except AssertionError: default = None mlo_file = cls.get_file('spl_file', default=default) cmd_runner.run(["cp", "-v", mlo_file, boot_disk], as_root=True).wait() def make_boot_ini(boot_script_path, boot_disk): proc = cmd_runner.run( ["cp", "-v", boot_script_path, "%s/boot.ini" % boot_disk], as_root=True) proc.wait() linaro-image-tools-2014.11/linaro_image_tools/testing.py0000664000175000017500000000411112400111666024343 0ustar fboudrafboudra00000000000000# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import os import tempfile from testtools import TestCase class TestCaseWithFixtures(TestCase): """A TestCase with the ability to easily add 'fixtures'. A fixture is an object which can be created and cleaned up, and this test case knows how to manage them to ensure that they will always be cleaned up at the end of the test. """ def useFixture(self, fixture): """Make use of a fixture, ensuring that it will be cleaned up. Given a fixture, this method will run the `setUp` method of the fixture, and ensure that its `tearDown` method will be called at the end of the test, regardless of success or failure. :param fixture: the fixture to use. :type fixture: an object with setUp and tearDown methods. :return: the fixture that was passed in. """ self.addCleanup(fixture.tearDown) fixture.setUp() return fixture def createTempFileAsFixture(self, prefix='tmp', dir=None): """Create a temp file and make sure it is removed on tearDown. :return: The filename of the file created. """ _, filename = tempfile.mkstemp(prefix=prefix, dir=dir) self.addCleanup(os.unlink, filename) return filename linaro-image-tools-2014.11/linaro-hwpack-create0000775000175000017500000000412612400111666022402 0ustar fboudrafboudra00000000000000#!/usr/bin/python # Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import argparse import sys from linaro_image_tools.hwpack.builder import ( ConfigFileMissing, HardwarePackBuilder) from linaro_image_tools.utils import get_logger from linaro_image_tools.__version__ import __version__ if __name__ == '__main__': parser = argparse.ArgumentParser(version='%(prog)s ' + __version__) parser.add_argument( "CONFIG_FILE", help="The configuration file to take the hardware pack information " "from.") parser.add_argument( "VERSION", help="The version of the hardware pack to create.") parser.add_argument( "--local-deb", action="append", dest="local_debs", default=[], metavar="LOCAL_DEB", help=("Include LOCAL_DEB in the hardware pack, even if it's an older " "version than a package that would be otherwise installed. " "Can be used more than once.")) parser.add_argument("--debug", action="store_true") args = parser.parse_args() logger = get_logger(debug=args.debug) try: builder = HardwarePackBuilder(args.CONFIG_FILE, args.VERSION, args.local_debs) except ConfigFileMissing, e: logger.error(str(e)) sys.exit(1) builder.build() linaro-image-tools-2014.11/setup.py0000775000175000017500000000122312400111666020164 0ustar fboudrafboudra00000000000000#!/usr/bin/env python # https://launchpad.net/python-distutils-extra import DistUtilsExtra.auto from linaro_image_tools.__version__ import __version__ DistUtilsExtra.auto.setup( name="linaro-image-tools", version=__version__, description="Tools to create and write Linaro images", url="https://launchpad.net/linaro-image-tools", license="GPL v3 or later", author='Linaro Infrastructure team', author_email="linaro-dev@lists.linaro.org", scripts=[ "initrd-do", "linaro-hwpack-create", "linaro-hwpack-install", "linaro-media-create", "linaro-android-media-create", "linaro-hwpack-replace"], ) linaro-image-tools-2014.11/COPYING.GPL-30000664000175000017500000010451312400111666020271 0ustar fboudrafboudra00000000000000 GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Use with the GNU Affero General Public License. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an "about box". You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read . linaro-image-tools-2014.11/org.linaro.linaro-image-tools.policy0000664000175000017500000000143712400111666025452 0ustar fboudrafboudra00000000000000 Linaro http://www.linaro.org Create a disk image by combining an OS image and a hardware pack. Authentication is required to run linaro-media-create as $(user) no auth_admin /usr/bin/linaro-media-create