system-storage-manager-0.4/0000775000175000017500000000000012200423561017442 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/PKG-INFO0000664000175000017500000010567512200423561020555 0ustar lczernerlczerner00000000000000Metadata-Version: 1.1 Name: system-storage-manager Version: 0.4 Summary: System Storage Manager - A single tool to manage your storage Home-page: http://storagemanager.sf.net Author: Lukas Czerner Author-email: lczerner@redhat.com License: GNU General Public License version 2 or any later version Description: System Storage Manager ********************** A single tool to manage your storage. Description *********** System Storage Manager provides easy to use command line interface to manage your storage using various technologies like lvm, btrfs, encrypted volumes and more. In more sophisticated enterprise storage environments, management with Device Mapper (dm), Logical Volume Manager (LVM), or Multiple Devices (md) is becoming increasingly more difficult. With file systems added to the mix, the number of tools needed to configure and manage storage has grown so large that it is simply not user friendly. With so many options for a system administrator to consider, the opportunity for errors and problems is large. The btrfs administration tools have shown us that storage management can be simplified, and we are working to bring that ease of use to Linux filesystems in general. Licence ******* (C)2011 Red Hat, Inc., Lukas Czerner This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Commands ******** Introduction ************ System Storage Manager have several commands you can specify on the command line as a first argument to the ssm. They all have specific use and its own arguments, but global ssm arguments are propagated to all commands. Create command ************** This command creates a new volume with defined parameters. If **device** is provided it will be used to create a volume, hence it will be added into the **pool** prior the volume creation (See *Add command section*). More devices can be used to create a volume. If the **device** is already used in the different pool, then **ssm** will ask you whether you want to remove it from the original pool. If you decline, or the removal fails, then the **volume** creation fails if the *SIZE* was not provided. On the other hand, if the *SIZE* is provided and some devices can not be added to the **pool** the volume creation might succeed if there is enough space in the **pool**. *POOL* name can be specified as well. If the pool exists new volume will be created from that pool (optionally adding **device** into the pool). However if the *POOL* does not exist **ssm** will attempt to create a new pool with provided **device** and then create a new volume from this pool. If **--backend** argument is omitted, the default **ssm** backend will be used. Default backend is *lvm*. **ssm** also supports creating RAID configuration, however some back- ends might not support all the levels, or it might not support RAID at all. In this case, volume creation will fail. If **mount** point is provided **ssm** will attempt to mount the volume after it is created. However it will fail if mountable file system is not present on the volume. List command ************ List informations about all detected devices, pools, volumes and snapshots found in the system. **list** command can be used either alone to list all the information, or you can request specific section only. Following sections can be specified: {volumes | vol} List information about all **volumes** found in the system. {devices | dev} List information about all **devices** found in the system. Some devices are intentionally hidden, like for example cdrom, or DM/MD devices since those are actually listed as volumes. {pools | pool} List information about all **pools** found in the system. {filesystems | fs} List information about all volumes containing **filesystems** found in the system. {snapshots | snap} List information about all **snapshots** found in the system. Note that some back-ends does not support snapshotting and some can not distinguish between snapshot and regular volume. in this case **ssm** will try to recognize volume name in order to identify **snapshot**, but if the **ssm** regular expression does not match the snapshot pattern, this snapshot will not be recognized. Remove command ************** This command removes **item** from the system. Multiple items can be specified. If the **item** can not be removed for some reason, it will be skipped. **item** can represent: device Remove **device** from the pool. Note that this can not be done in some cases where the device is used by pool. You can use **-f** argument to *force* removal. If the device does not belong to any pool, it will be skipped. pool Remove the **pool** from the system. This will also remove all volumes created from that pool. volume Remove the **volume** from the system. Note that this will fail if the **volume** is mounted and it can not be *forced* with **-f**. Resize command ************** Change size of the **volume** and file system. If there is no file system only the **volume** itself will be resized. You can specify **device** to add into the **volume** pool prior the resize. Note that **device** will only be added into the pool if the **volume** size is going to grow. If the **device** is already used in the different pool, then **ssm** will ask you whether you want to remove it from the original pool. In some cases file system has to be mounted in order to resize. This will be handled by **ssm** automatically by mounting the **volume** temporarily. Check command ************* Check the file system consistency on the **volume**. You can specify multiple volumes to check. If there is no file system on the **volume**, this **volume** will be skipped. In some cases file system has to be mounted in order to check the file system This will be handled by **ssm** automatically by mounting the **volume** temporarily. Snapshot command **************** Take a snapshot of existing **volume**. This operation will fail if back-end which the **volume** belongs to does not support snapshotting. Note that you can not specify both *NAME* and *DESC* since those options are mutually exclusive. In some cases file system has to be mounted in order to take a snapshot of the **volume**. This will be handled by **ssm** automatically by mounting the **volume** temporarily. Add command *********** This command adds **device** into the pool. The **device** will not be added if it's already part of different pool by default, but user will be asked whether to remove the device from it's pool. When multiple devices are provided, all of them are added into the pool. If one of the devices can not be added into the pool for any reason, add command will fail. If no pool is specified, default pool will be chosen. In the case of non existing pool, it will be created using provided devices. Backends ******** Introduction ************ Ssm aims to create unified user interface for various technologies like Device Mapper (dm), Btrfs file system, Multiple Devices (md) and possibly more. In order to do so we have a core abstraction layer in "ssmlib/main.py". This abstraction layer should ideally know nothing about the underlying technology, but rather comply with **device**, **pool** and **volume** abstraction. Various backends can be registered in "ssmlib/main.py" in order to handle specific storage technology implementing methods like *create*, *snapshot*, or *remove* volumes and pools. The core will then call these methods to manage the storage without needing to know what lies underneath it. There are already several backends registered in ssm. Btrfs backend ************* Btrfs is the file system with many advanced features including volume management. This is the reason why btrfs is handled differently than other *conventional* file systems in **ssm**. It is used as a volume management back-end. Pools, volumes and snapshots can be created with btrfs backend and here is what it means from the btrfs point of view: pool Pool is actually a btrfs file system itself, because it can be extended by adding more devices, or shrink by removing devices from it. Subvolumes and snapshots can also be created. When the new btrfs pool should be created **ssm** simply creates a btrfs file system, which means that every new btrfs pool has one volume of the same name as the pool itself which can not be removed without removing the entire pool. Default btrfs pool name is **btrfs_pool**. When creating new btrfs pool, the name of the pool is used as the file system label. If there is already existing btrfs file system in the system without a label, btrfs pool name will be generated for internal use in the following format "btrfs_{device base name}". Btrfs pool is created when **create** or **add** command is used with devices specified and non existing pool name. volume Volume in btrfs back-end is actually just btrfs subvolume with the exception of the first volume created on btrfs pool creation, which is the file system itself. Subvolumes can only be created on btrfs file system when the it is mounted, but user does not have to worry about that, since **ssm** will automatically mount the file system temporarily in order to create a new subvolume. Volume name is used as subvolume path in the btrfs file system and every object in this path must exists in order to create a volume. Volume name for internal tracking and for representing to the user is generated in the format "{pool_name}:{volume name}", but volumes can be also referenced with its mount point. Btrfs volumes are only shown in the *list* output, when the file system is mounted, with the exception of the main btrfs volume - the file system itself. New btrfs volume can be created with **create** command. snapshot Btrfs file system support subvolume snapshotting, so you can take a snapshot of any btrfs volume in the system with **ssm**. However btrfs does not distinguish between subvolumes and snapshots, because snapshot actually is just a subvolume with some block shared with different subvolume. It means, that **ssm** is not able to recognize btrfs snapshot directly, but instead it is trying to recognize special name format of the btrfs volume. However, if the *NAME* is specified when creating snapshot which does not match the special pattern, snapshot will not be recognized by the **ssm** and it will be listed as regular btrfs volume. New btrfs snapshot can be created with **snapshot** command. device Btrfs does not require any special device to be created on. Lvm backend *********** Pools, volumes and snapshots can be created with lvm, which pretty much match the lvm abstraction. pool Lvm pool is just *volume group* in lvm language. It means that it is grouping devices and new logical volumes can be created out of the lvm pool. Default lvm pool name is **lvm_pool**. Lvm pool is created when **create** or **add** command is used with devices specified and non existing pool name. volume Lvm volume is just *logical volume* in lvm language. Lvm volume can be created wit **create** command. snapshot Lvm volumes can be snapshotted as well. When a snapshot is created from the lvm volume, new *snapshot* volume is created, which can be handled as any other lvm volume. Unlike *btrfs* lvm is able to distinguish snapshot from regular volume, so there is no need for a snapshot name to match special pattern. device Lvm requires *physical device* to be created on the device, but with **ssm** this is transparent for the user. Crypt backend ************* Crypt backend in **ssm** is currently limited to only gather the information about encrypted volumes in the system. You can not create or manage encrypted volumes or pools, but it will be extended in the future. Environment variables ********************* SSM_DEFAULT_BACKEND Specify which backend will be used by default. This can be overridden by specifying **-b** or **--backend** argument. Currently only *lvm* and *btrfs* is supported. SSM_LVM_DEFAULT_POOL Name of the default lvm pool to be used if **-p** or **--pool** argument is omitted. SSM_BTRFS_DEFAULT_POOL Name of the default btrfs pool to be used if **-p** or **--pool** argument is omitted. SSM_PREFIX_FILTER When this is set **ssm** will filter out all devices, volumes and pools which name does not start with this prefix. It is used mainly in **ssm** test suite to make sure that we do not scramble local system configuration. Quick examples ************** List system storage: # ssm list ---------------------------------- Device Total Mount point ---------------------------------- /dev/loop0 5.00 GB /dev/loop1 5.00 GB /dev/loop2 5.00 GB /dev/loop3 5.00 GB /dev/loop4 5.00 GB /dev/sda 149.05 GB PARTITIONED /dev/sda1 19.53 GB / /dev/sda2 78.12 GB /dev/sda3 1.95 GB SWAP /dev/sda4 1.00 KB /dev/sda5 49.44 GB /mnt/test ---------------------------------- ------------------------------------------------------------------------------ Volume Pool Volume size FS FS size Free Type Mount point ------------------------------------------------------------------------------ /dev/dm-0 dm-crypt 78.12 GB ext4 78.12 GB 45.01 GB crypt /home /dev/sda1 19.53 GB ext4 19.53 GB 12.67 GB part / /dev/sda5 49.44 GB ext4 49.44 GB 29.77 GB part /mnt/test ------------------------------------------------------------------------------ Creating a volume of defined size with the defined file system. The default back-end is set to lvm and lvm default pool name is lvm_pool: # ssm create --fs ext4 -s 15G /dev/loop0 /dev/loop1 The name of the new volume is '/dev/lvm_pool/lvol001'. Resize the volume to 10GB: # ssm resize -s-5G /dev/lvm_pool/lvol001 Resize the volume to 100G, but it would require to add more devices into the pool: # ssm resize -s 25G /dev/lvm_pool/lvol001 /dev/loop2 Now we can try to create new lvm volume named 'myvolume' from the remaining pool space with xfs file system and mount it to /mnt/test1: # ssm create --fs xfs --name myvolume /mnt/test1 List all volumes with file system: # ssm list filesystems ----------------------------------------------------------------------------------------------- Volume Pool Volume size FS FS size Free Type Mount point ----------------------------------------------------------------------------------------------- /dev/lvm_pool/lvol001 lvm_pool 25.00 GB ext4 25.00 GB 23.19 GB linear /dev/lvm_pool/myvolume lvm_pool 4.99 GB xfs 4.98 GB 4.98 GB linear /mnt/test1 /dev/dm-0 dm-crypt 78.12 GB ext4 78.12 GB 45.33 GB crypt /home /dev/sda1 19.53 GB ext4 19.53 GB 12.67 GB part / /dev/sda5 49.44 GB ext4 49.44 GB 29.77 GB part /mnt/test ----------------------------------------------------------------------------------------------- You can then easily remove the old volume by: # ssm remove /dev/lvm_pool/lvol001 Now lest try to create btrfs volume. Btrfs is separate backend, not just a file system. That is because btrfs itself have integrated volume manager. Defaul btrfs pool name is btrfs_pool.: # ssm -b btrfs create /dev/loop3 /dev/loop4 Now create we btrfs subvolumes. Note that btrfs file system has to be mounted in order to create subvolumes. However ssm will handle it for you.: # ssm create -p btrfs_pool # ssm create -n new_subvolume -p btrfs_pool # ssm list filesystems ----------------------------------------------------------------- Device Free Used Total Pool Mount point ----------------------------------------------------------------- /dev/loop0 0.00 KB 10.00 GB 10.00 GB lvm_pool /dev/loop1 0.00 KB 10.00 GB 10.00 GB lvm_pool /dev/loop2 0.00 KB 10.00 GB 10.00 GB lvm_pool /dev/loop3 8.05 GB 1.95 GB 10.00 GB btrfs_pool /dev/loop4 6.54 GB 1.93 GB 8.47 GB btrfs_pool /dev/sda 149.05 GB PARTITIONED /dev/sda1 19.53 GB / /dev/sda2 78.12 GB /dev/sda3 1.95 GB SWAP /dev/sda4 1.00 KB /dev/sda5 49.44 GB /mnt/test ----------------------------------------------------------------- ------------------------------------------------------- Pool Type Devices Free Used Total ------------------------------------------------------- lvm_pool lvm 3 0.00 KB 29.99 GB 29.99 GB btrfs_pool btrfs 2 3.84 MB 18.47 GB 18.47 GB ------------------------------------------------------- ----------------------------------------------------------------------------------------------- Volume Pool Volume size FS FS size Free Type Mount point ----------------------------------------------------------------------------------------------- /dev/lvm_pool/lvol001 lvm_pool 25.00 GB ext4 25.00 GB 23.19 GB linear /dev/lvm_pool/myvolume lvm_pool 4.99 GB xfs 4.98 GB 4.98 GB linear /mnt/test1 /dev/dm-0 dm-crypt 78.12 GB ext4 78.12 GB 45.33 GB crypt /home btrfs_pool btrfs_pool 18.47 GB btrfs 18.47 GB 18.47 GB btrfs /dev/sda1 19.53 GB ext4 19.53 GB 12.67 GB part / /dev/sda5 49.44 GB ext4 49.44 GB 29.77 GB part /mnt/test ----------------------------------------------------------------------------------------------- Now let's free up some of the loop devices so we cat try to add them into then btrfs_pool. So we'll simply remove lvm mvolume and resize lvol001 so we can remove /dev/loop2. Note that myvolume is mounted so we have to unmount it first.: # umount /mnt/test1 # ssm remove /dev/lvm_pool/myvolume # ssm resize -s-10G /dev/lvm_pool/lvol001 # ssm remove /dev/loop2 Add device to the btrfs file system: # ssm add /dev/loop2 -p btrfs_pool Set' see what happend. Note that to actually see btrfs subvolumes you have to mount the file system first: # mount -L btrfs_pool /mnt/test1/ # ssm list volumes ------------------------------------------------------------------------------------------------------------------------ Volume Pool Volume size FS FS size Free Type Mount point ------------------------------------------------------------------------------------------------------------------------ /dev/lvm_pool/lvol001 lvm_pool 15.00 GB ext4 15.00 GB 13.85 GB linear /dev/dm-0 dm-crypt 78.12 GB ext4 78.12 GB 45.33 GB crypt /home btrfs_pool btrfs_pool 28.47 GB btrfs 28.47 GB 28.47 GB btrfs /mnt/test1 btrfs_pool:2012-05-09-T113426 btrfs_pool 28.47 GB btrfs 28.47 GB 28.47 GB btrfs /mnt/test1/2012-05-09-T113426 btrfs_pool:new_subvolume btrfs_pool 28.47 GB btrfs 28.47 GB 28.47 GB btrfs /mnt/test1/new_subvolume /dev/sda1 19.53 GB ext4 19.53 GB 12.67 GB part / /dev/sda5 49.44 GB ext4 49.44 GB 29.77 GB part /mnt/test ------------------------------------------------------------------------------------------------------------------------ Remove the whole lvm pool and one of the btrfs subvolume, and one unused device from the btrfs pool btrfs_loop3. Note that with btrfs, pool have the same name as the volume: # ssm remove lvm_pool /dev/loop2 /mnt/test1/new_subvolume/ Snapshots can also be done with ssm: # ssm snapshot btrfs_pool # ssm snapshot -n btrfs_snapshot btrfs_pool With lvm, you can also create snapshots: root# ssm create -s 10G /dev/loop[01] # ssm snapshot /dev/lvm_pool/lvol001 Now list all snapshots. Note that btrfs snapshots are actually just subvolumes with some blocks shared with the original subvolume, so there currently no way to distinguish between those. ssm is using a little trick to search for name patters to recognize snapshots, so if you specify your own name for the snapshot ssm will not recognize it as snapshot, but rather as regular volume (subvolume). This problem does not exist with lvm.: # ssm list snapshots ------------------------------------------------------------------------------------------------------------- Snapshot Origin Volume size Size Type Mount point ------------------------------------------------------------------------------------------------------------- /dev/lvm_pool/snap20120509T121611 lvol001 2.00 GB 0.00 KB linear btrfs_pool:snap-2012-05-09-T121313 18.47 GB btrfs /mnt/test1/snap-2012-05-09-T121313 ------------------------------------------------------------------------------------------------------------- Installation ************ To install System Storage Manager into your system simply run: python setup.py install as root in the System Storage Manager directory. Make sure that your system configuration meet the *requirements* in order for ssm to work correctly. Note that you can run **ssm** even without installation from using the local sources with: bin/ssm.local Requirements ************ Python 2.6 or higher is required to run this tool. System Storage Manager can only be run as root since most of the commands requires root privileges. There are other requirements listed bellow, but note that you do not necessarily need all dependencies for all backends, however if some of the tools required by the backend is missing, the backend would not work. Python modules ============== * os * re * sys * stat * argparse * datetime * threading * subprocess System tools ============ * tune2fs * fsck.SUPPORTED_FS * resize2fs * xfs_db * xfs_check * xfs_growfs * mkfs.SUPPORTED_FS * which * mount * blkid * wipefs Lvm backend =========== * lvm2 binaries Btrfs backend ============= * btrfs progs Crypt backend ============= * dmsetup * cryptsetup For developers ************** We are accepting patches! If you're interested contributing to the System Storage Manager code, just checkout the git repository located on SourceForge. Please, base all of your work on the "devel" branch since it is more up-to-date and it will save us some work when merging your patches: git clone --branch devel git://git.code.sf.net/p/storagemanager/code storagemanager-code Any form of contribution - patches, documentation, reviews or rants are appreciated. See *Mailing list section* section. Tests ===== System Storage Manager contains regression testing suite to make sure that we do not break thing that should already work. And we recommend every developer to run tests before sending patches: python test.py Tests in System Storage Manager are divided into four levels. 1. First the doctest is executed. 2. Then we have unittests in "tests/unittests/test_ssm.py" which is testing the core of ssm "ssmlib/main.py". It is checking for basic things like required backend methods and variables, flag propagations, proper class initialization and finally whether commands actually result in the proper backend callbacks. It does not require root permissions and it does not touch your system configuration in any way. It actually should not invoke any shell command, and if it does it's a bug. 3. Second part of unittests is backend testing. We are mainly testing whether ssm commands result in proper backend operations. It does not require root permissions and it does not touch your system configuration in any way. It actually should not invoke any shell command and if it does it's a bug. 4. And finally there are real bash tests located in "tests/bashtests". Bash tests are divided into files. Each file tests one command for one backend and it containing series of test cases followed by checks whether the command created the expected result. In order to test real system commands we have to create system device to test on and not touch any of the existing system configuration. Before each test a number of devices are created using *dmsetup* in the test directory. These devices will be used in test cases instead of real devices. Real operation are performed in those devices as it would on the real system devices. It implies that this phase requires root privileges and it would not be run otherwise. In order to make sure that **ssm** does not touch any existing system configuration, each device, poor and volume name is include special prefix and SSM_PREFIX_FILTER environment variable is set to make **ssm** to exclude all items which does not match this filter. Even though we tried hard to make sure that the bash tests does not change any of your system configuration the recommendation is **not** to run tests as with root privileges on your work or production system, but rather run it on your testing machine. If you change or create new functionality, please make sure that it is covered by the System Storage Manager regression test suite to make sure that we do not break it unintentionally. Important: Please, make sure to run full tests before you send a patch to the mailing list. To do so, simply run "python test.py" as root on your test machine. Documentation ============= System Storage Manager documentation is stored in "doc/" directory. The documentation is build using **sphinx** software which help us not to duplicate texts for different type of documentation (man page, html pages, readme). If you are going to modify documentation, please make sure not to modify manual page, html pages or README directly, but rather modify "doc/*.rst" and "doc/src/*.rst" files accordingly so the change is propagated to all documents. Moreover, parts of the documentation such as *synopsis* or ssm command *options* are parsed directly from the ssm help output. It means that when you're going to add or change argument into **ssm** the only thing you have to do is to add or change it in the "ssmlib/main.py" source code and then run "make dist" in the "doc/" directory and all the documents should be updated automatically. Important: Please make sure you update the documentation when you add or change **ssm** functionality if the format of the change requires it. Then regenerate all the documents using "make dist" and include changes in the patch. Mailing list ============ System Storage Manager developers communicate via the mailing list. Address of our mailing list is storagemanager- devel@lists.sourceforge.net and you can subscribe on the SourceForge project page https://lists.sourceforge.net/lists/listinfo /storagemanager-devel. Mailing list archives can be found here http://sourceforge.net/mailarchive/forum.php?forum_name =storagemanager-devel. This is also the list where to send patches and where the review process is happening. We do not have separate *user* mailing list, so feel free to drop your questions there as well. Posting patches =============== As already mentioned, we are accepting patches! And we are very happy for every contribution. If you're going to send a path in, please make sure to follow some simple rules: 1. Before you're going to post a patch, please run our regression testing suite to make sure that your change does not break someone else work. See *Tests section* 2. If you're making a change that might require documentation update, please update the documentation as well. See *Documentation section* 3. Make sure your patch have all the requisites such as *short description* preferably 50 characters long at max describing the main idea of the change. *Long description* describing what was changed with and why and finally Signed-off-by tag. 4. If you're going to send a patch to the mailing list, please send the patch inlined in the email body. It is much better for review process. Hint: You can use **git** to do all the work for you. "git format-patch" and "git send-email" will help you with creating and sending the patch. Platform: Linux Requires: argparse Requires: itertools Requires: tempfile Requires: threading Requires: subprocess Requires: datetime Requires: re Requires: os Requires: sys Requires: stat system-storage-manager-0.4/setup.py0000664000175000017500000000222512171200774021163 0ustar lczernerlczerner00000000000000import os import sys from distutils.core import setup from ssmlib import main VERSION=main.VERSION DOC_BUILD='doc/_build/' NAME="system-storage-manager" if sys.version < '2.6': print "Python version 2.6 or higher is required " + \ "for System Storage Manager to run correctly!" sys.exit(1) setup( name=NAME, version=VERSION, author='Lukas Czerner', author_email='lczerner@redhat.com', maintainer='Lukas Czerner', maintainer_email='lczerner@redhat.com', url='http://storagemanager.sf.net', packages=['ssmlib', 'ssmlib.backends'], scripts=['bin/ssm'], description='System Storage Manager - A single tool to manage your storage', license='GNU General Public License version 2 or any later version', long_description=open('README').read(), requires=['argparse', 'itertools', 'tempfile', 'threading', 'subprocess', 'datetime', 're', 'os', 'sys', 'stat'], platforms=['Linux'], data_files=[('/usr/share/man/man8', ['doc/_build/man/ssm.8']), ('/usr/share/doc/{0}-{1}'.format(NAME, VERSION), ['README', 'CHANGES', 'COPYING', 'AUTHORS', 'INSTALL'])] ) system-storage-manager-0.4/Makefile0000664000175000017500000000356112200415215021104 0ustar lczernerlczerner00000000000000all: help help: @echo "Usage: make " @echo @echo "Available targets are:" @echo " help show this text" @echo " clean remove python bytecode and temp files" @echo " install install program on current system" @echo " spec prepare changelog for spec file" @echo " log prepare changelog" @echo " authors prepare list of authors" @echo " docs generate up-to-date documentation" @echo " source create source tarball" @echo " test run tests/run_tests.py" check_vars: ifndef VERSION @echo "VERSION variable not defined" @exit 1 endif ifndef PREVIOUS @echo "PREVIOUS variable not defined" @exit 1 endif clean: @python setup.py clean rm -f MANIFEST find . -\( -name "*.pyc" -o -name '*.pyo' -o -name "*~" -\) -delete git-clean: git clean -f install: @python setup.py install spec: check_vars @(LC_ALL=C date +"* %a %b %e %Y `git config --get user.name` <`git config --get user.email`> - $(VERSION)"; git log --pretty="format:- %s (%an)" $(PREVIOUS)..HEAD| cat; echo -e "\n\n"; cat CHANGES) > CHANGES.bck; mv CHANGES.bck CHANGES log: check_vars @(LC_ALL=C date +"[%a %b %e %Y] `git config --get user.name` <`git config --get user.email`> - $(VERSION)"; echo; git shortlog -e $(PREVIOUS)..HEAD | cat; git diff --stat $(PREVIOUS)..HEAD | cat) | sed -e 's/@/_O_/g' | less authors: @(echo -e "System Storage Manager was written by:\n\tLukáš Czerner "; echo -e "\nContributions (commits):"; git log --no-merges | grep '^Author:' | sort | uniq -c | sort -rn | sed -e 's/^\s*\([0-9]*\) Author: /\t(\1) /') | sed -e 's/@/_O_/g' > AUTHORS docs: @make dist -C doc source: test clean @python setup.py sdist test: @python test.py release: git-clean clean check_vars authors spec log docs source system-storage-manager-0.4/test.py0000664000175000017500000001030612200411405020765 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # Main testing script for the system storage manager import os import re import sys import time import doctest import unittest os.environ['SSM_NONINTERACTIVE'] = "1" from ssmlib import main from ssmlib import misc from ssmlib.backends import lvm, crypt, btrfs from tests.unittests import * def run_bash_tests(): cur = os.getcwd() os.chdir('./tests/bashtests') command = ['ls', '-m'] if os.access('.coverage', os.R_OK): os.remove('.coverage') failed = [] passed = [] count = 0 misc.run('./set.sh', stdout=False) output = misc.run(command, stdout=False)[1] t0 = time.time() for script in output.split(","): script = script.strip() if not re.match("^\d\d\d-.*\.sh$", script): continue count += 1 print "{0:<29}".format(script), sys.stdout.flush() bad_file = re.sub("\.sh$",".bad", script) if os.access(bad_file, os.R_OK): os.remove(bad_file) ret, out = misc.run(['./' + script], stdout=False, can_fail=True) if ret: print "\033[91m[FAILED]\033[0m" failed.append(script) with open(bad_file, 'w') as f: f.write(out) elif re.search("Traceback", out): # There should be no tracebacks in the output out += "\nWARNING: Traceback in the output!\n" print "\033[93m[WARNING]\033[0m" with open(bad_file, 'w') as f: f.write(out) else: print "\033[92m[PASSED]\033[0m" passed.append(script) t1 = time.time() - t0 print "Ran {0} tests in {1} seconds.".format(count, round(t1, 2)) print "{0} tests PASSED: {1}".format(len(passed), ", ".join(passed)) ret = 0 if len(failed) > 0: print "{0} tests FAILED: {1}".format(len(failed), ", ".join(failed)) print "See files with \"bad\" extension for output" ret = 1 # Show coverage report output if possible if misc.check_binary('coverage'): print "[+] Coverage" misc.run(['coverage', 'report'], stdout=True, can_fail=True) os.chdir(cur) return ret def quick_test(): print "[+] Running doctests" doctest_flags = doctest.IGNORE_EXCEPTION_DETAIL | doctest.ELLIPSIS result = doctest.testmod(main, exclude_empty=True, report=True, raise_on_error=True, optionflags=doctest_flags) result = doctest.testmod(lvm, exclude_empty=True, report=True, raise_on_error=True, optionflags=doctest_flags) result = doctest.testmod(crypt, exclude_empty=True, report=True, raise_on_error=True, optionflags=doctest_flags) result = doctest.testmod(btrfs, exclude_empty=True, report=True, raise_on_error=True, optionflags=doctest_flags) result = doctest.testmod(misc, exclude_empty=True, report=True, raise_on_error=True, optionflags=doctest_flags) print "[+] Running unittests" test_loader = unittest.TestLoader() tests_lvm = test_loader.loadTestsFromModule(test_lvm) tests_btrfs = test_loader.loadTestsFromModule(test_btrfs) tests_ssm = test_loader.loadTestsFromModule(test_ssm) tests = unittest.TestSuite([tests_lvm, tests_btrfs, tests_ssm]) test_runner = unittest.TextTestRunner(verbosity=2) test_runner.run(tests) if __name__ == '__main__': quick_test() if not os.geteuid() == 0: print "\nRoot privileges required to run more tests!\n" sys.exit(0) print "[+] Running bash tests" result = run_bash_tests() sys.exit(result) system-storage-manager-0.4/tests/0000775000175000017500000000000012200423561020604 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/tests/bashtests/0000775000175000017500000000000012200423561022604 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/tests/bashtests/011-lvm-list.sh0000775000175000017500000000726712200411405025217 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2012 Red Hat, Inc., Tom Marek # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='011-lvm-list' export test_description='Check whether list command prints correct values for lvm' . lib/test DEV_COUNT=10 DEV_SIZE=100 TEST_MAX_SIZE=$(($DEV_COUNT*$DEV_SIZE)) aux prepare_devs $DEV_COUNT $DEV_SIZE TEST_DEVS=$(cat DEVICES) export LVOL_PREFIX="lvol" export SSM_DEFAULT_BACKEND='lvm' export SSM_LVM_DEFAULT_POOL=$vg1 export SSM_NONINTERACTIVE='1' snap1="snap1" snap2="snap2" pool1=$vg2 pool2=$vg3 TEST_FS= which mkfs.ext2 && TEST_FS+="ext2 " which mkfs.ext3 && TEST_FS+="ext3 " which mkfs.ext4 && TEST_FS+="ext4 " which mkfs.xfs && TEST_FS+="xfs" TEST_MNT=$TESTDIR/mnt [ ! -d $TEST_MNT ] && mkdir $TEST_MNT &> /dev/null ##LVM # Check devices ssm add $TEST_DEVS ssm_output=$(ssm list dev) for device in ${TEST_DEVS}; do check list_table "$ssm_output" $device 96.00MB 0.00KB 96.00MB $SSM_LVM_DEFAULT_POOL done # Check pools check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL $SSM_DEFAULT_BACKEND $DEV_COUNT none none 960.00MB ssm -f remove --all # create multiple pools ssm create --pool $pool1 $dev1 $dev2 $dev3 $dev4 ssm create --pool $pool2 $dev5 $dev6 $dev7 $dev8 ssm_output=$(ssm list pool) check list_table "$ssm_output" $pool1 lvm 4 none none 384.00MB check list_table "$ssm_output" $pool2 lvm 4 none none 384.00MB ssm -f remove --all ssm add $TEST_DEVS # Check LVM volume listings with various fs for fs in $TEST_FS; do name="${fs}vol" ssm create --fs=$fs -r 0 -I 8 -i $((DEV_COUNT/5)) -s $(($DEV_SIZE*2))M -n "${fs}vol" done # Mounted for fs in $TEST_FS; do mount ${DM_DEV_DIR}/${SSM_LVM_DEFAULT_POOL}/${fs}vol $TEST_MNT check list_table "$(ssm list vol)" ${fs}vol $SSM_LVM_DEFAULT_POOL $(($DEV_SIZE*2)).00MB ${fs} none none striped $TEST_MNT umount $TEST_MNT done # Unmounted ssm_output=$(ssm list vol) for fs in $TEST_FS; do check list_table "$ssm_output" ${fs}vol $SSM_LVM_DEFAULT_POOL $(($DEV_SIZE*2)).00MB ${fs} none none striped done ssm -f remove $SSM_LVM_DEFAULT_POOL # Check lvm snapshot lvol1=${LVOL_PREFIX}001 # Create volume with all devices at once size=$(($DEV_SIZE*6)) snap_size1=$(($DEV_SIZE)) snap_size2=$(($size/5)) ssm create --size ${size}M $TEST_DEVS ssm snapshot --name $snap1 --size ${snap_size1}M $SSM_LVM_DEFAULT_POOL/$lvol1 ssm snapshot --name $snap2 --size ${snap_size2}M $SSM_LVM_DEFAULT_POOL/$lvol1 ssm_output=$(ssm list snap) check list_table "$ssm_output" $snap1 $lvol1 $SSM_LVM_DEFAULT_POOL ${snap_size1}.00MB 0.00KB linear check list_table "$ssm_output" $snap2 $lvol1 $SSM_LVM_DEFAULT_POOL ${snap_size2}.00MB 0.00KB linear ssm -f remove --all # Snapshot of the volumes in defferent pools ssm create --pool $pool1 $dev1 $dev2 ssm add $dev3 $dev4 --pool $pool1 ssm create --pool $pool2 $dev5 $dev6 ssm add $dev7 $dev8 --pool $pool2 ssm snapshot --name $snap1 $pool1/$lvol1 ssm snapshot --name $snap1 $pool2/$lvol1 ssm_output=$(ssm list snap) check list_table "$ssm_output" "$pool1/$snap1" $lvol1 $pool1 40.00MB 0.00KB linear check list_table "$ssm_output" "$pool2/$snap1" $lvol1 $pool2 40.00MB 0.00KB linear ssm -f remove --all # all_done!!! system-storage-manager-0.4/tests/bashtests/lib/0000775000175000017500000000000012200423561023352 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/tests/bashtests/lib/utils.sh0000664000175000017500000001111112171200774025047 0ustar lczernerlczerner00000000000000#!/bin/bash # Copyright (C) 2011 Red Hat, Inc. All rights reserved. # # This copyrighted material is made available to anyone wishing to use, # modify, copy, or redistribute it subject to the terms and conditions # of the GNU General Public License v.2. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software Foundation, # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA set -e die() { echo >&2 "$@"; exit 1; } MAX_TRIES=4 rand_bytes() { n=$1 chars=abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 dev_rand=/dev/urandom if test -r "$dev_rand"; then # Note: 256-length($chars) == 194; 3 copies of $chars is 186 + 8 = 194. head -c$n "$dev_rand" | tr -c $chars 01234567$chars$chars$chars return fi cmds='date; date +%N; free; who -a; w; ps auxww; ps ef; netstat -n' data=$( (eval "$cmds") 2>&1 | gzip ) n_plus_50=$(expr $n + 50) # Ensure that $data has length at least 50+$n while :; do len=$(echo "$data"|wc -c) test $n_plus_50 -le $len && break; data=$( (echo "$data"; eval "$cmds") 2>&1 | gzip ) done echo "$data" \ | dd bs=1 skip=50 count=$n 2>/dev/null \ | tr -c $chars 01234567$chars$chars$chars } mkdtemp() { case $# in 2);; *) die "Usage: mkdtemp DIR TEMPLATE";; esac destdir=$1 template=$2 case $template in *XXXX) ;; *) die "invalid template: $template (must have a suffix of at least 4 X's)";; esac fail=0 # First, try to use mktemp. d=$(env -u TMPDIR mktemp -d -t -p "$destdir" "$template" 2>/dev/null) \ || fail=1 # The resulting name must be in the specified directory. case $d in "$destdir"*);; *) fail=1;; esac # It must have created the directory. test -d "$d" || fail=1 # It must have 0700 permissions. perms=$(ls -dgo "$d" 2>/dev/null) || fail=1 case $perms in drwx------*) ;; *) fail=1;; esac test $fail = 0 && { echo "$d" return } # If we reach this point, we'll have to create a directory manually. # Get a copy of the template without its suffix of X's. base_template=$(echo "$template"|sed 's/XX*$//') # Calculate how many X's we've just removed. nx=$(expr length "$template" - length "$base_template") err= i=1 while :; do X=$(rand_bytes $nx) candidate_dir="$destdir/$base_template$X" err=$(mkdir -m 0700 "$candidate_dir" 2>&1) \ && { echo "$candidate_dir"; return; } test $MAX_TRIES -le $i && break; i=$(expr $i + 1) done die "$err" } init_udev_transaction() { if test "$DM_UDEV_SYNCHRONISATION" = 1; then COOKIE=$(dmsetup udevcreatecookie) # Cookie is not generated if udev is not running! if test -n "$COOKIE"; then export DM_UDEV_COOKIE=$COOKIE fi fi } finish_udev_transaction() { if test "$DM_UDEV_SYNCHRONISATION" = 1 -a -n "$DM_UDEV_COOKIE"; then dmsetup udevreleasecookie unset DM_UDEV_COOKIE fi } teardown_udev_cookies() { if test "$DM_UDEV_SYNCHRONISATION" = 1; then # Delete any cookies created more than 10 minutes ago # and not used in the last 10 minutes. dmsetup udevcomplete_all -y 10 fi } skip() { touch SKIP_THIS_TEST exit 200 } kernel_at_least() { major=$(uname -r |cut -d. -f1) minor=$(uname -r |cut -d. -f2 | cut -d- -f1) test $major -gt $1 && return 0 test $major -lt $1 && return 1 test $minor -gt $2 && return 0 test $minor -lt $2 && return 1 test -z "$3" && return 0 minor2=$(uname -r | cut -d. -f3 | cut -d- -f1) test -z "$minor2" -a $3 -ne 0 && return 1 test $minor2 -ge $3 2>/dev/null && return 0 return 1 } align_size_up() { size=$1 [ -z $2 ] && stripes=0 [ -z $3 ] && extent=4 [ -z $size ] || [ -z $stripes ] || [ -z $extent ] && exit 1 tmp=$((size%extent)) if [ $tmp -ne 0 ]; then size=$(($size+($extent-$tmp))) fi if [ $stripes -eq 0 ]; then echo "$size" return 0 fi extents=$(($size/$extent)) tmp=$(($extents%$stripes)) if [ $tmp -ne 0 ]; then extents=$(($extents-$tmp+$stripes)) fi echo "$(($extents*$extent))" } umount_all() { [ ! -d $TEST_MNT ] && return for mp in $(ls $TEST_MNT); do mount | grep " $TEST_MNT/$mp " 2>&1> /dev/null && { while umount $TEST_MNT/$mp 2> /dev/null; do continue; done } || true done } if test -n "$PREFIX"; then vg=${PREFIX}vg lv=LV for i in `seq 1 16`; do name="${PREFIX}pv$i" dev="$DM_DEV_DIR/mapper/$name" mnt="$TEST_MNT/test$i" eval "dev$i=$dev" eval "lv$i=LV$i" eval "vg$i=${PREFIX}vg$i" eval "mnt$i=$mnt" done fi system-storage-manager-0.4/tests/bashtests/lib/check.sh0000664000175000017500000001707412200413616024774 0ustar lczernerlczerner00000000000000#!/bin/bash # Copyright (C) 2010 Red Hat, Inc. All rights reserved. # # This copyrighted material is made available to anyone wishing to use, # modify, copy, or redistribute it subject to the terms and conditions # of the GNU General Public License v.2. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software Foundation, # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # check.sh: assert various things about volumes # USAGE # check linear VG LV # check lv_on VG LV PV # check mirror VG LV [LOGDEV|core] # check mirror_nonredundant VG LV # check mirror_legs VG LV N # check mirror_images_on VG LV DEV [DEV...] # ... set -e -o pipefail trim() { trimmed=${1%% } trimmed=${trimmed## } echo "$trimmed" } lvl() { lvs -a --noheadings "$@" } lvdevices() { lvl -odevices "$@" | sed 's/([^)]*)//g; s/,/ /g' } mirror_images_redundant() { vg=$1 lv=$vg/$2 lvs -a $vg -o+devices for i in `lvdevices $lv`; do echo "# $i:" lvdevices $vg/$i | sort | uniq done > check.tmp.all (grep -v ^# check.tmp.all || true) | sort | uniq -d > check.tmp test "`cat check.tmp | wc -l`" -eq 0 || { echo "mirror images of $lv expected redundant, but are not:" cat check.tmp.all exit 1 } } mirror_images_on() { vg=$1 lv=$2 shift 2 for i in `lvdevices $lv`; do lv_on $vg $lv $1 shift done } lv_on() { lv="$1/$2" lvdevices $lv | grep -F "$3" || { echo "LV $lv expected on $3 but is not:" >&2 lvdevices $lv >&2 exit 1 } test `lvdevices $lv | grep -vF "$3" | wc -l` -eq 0 || { echo "LV $lv contains unexpected devices:" >&2 lvdevices $lv >&2 exit 1 } } mirror_log_on() { vg="$1" lv="$2" where="$3" if test "$where" = "core"; then lvl -omirror_log "$vg/$lv" | not grep mlog else lv_on $vg "${lv}_mlog" "$where" fi } lv_is_contiguous() { test `lvl --segments $1 | wc -l` -eq 1 || { echo "LV $1 expected to be contiguous, but is not:" lvl --segments $1 exit 1 } } lv_is_clung() { test `lvdevices $1 | sort | uniq | wc -l` -eq 1 || { echo "LV $1 expected to be clung, but is not:" lvdevices $! | sort | uniq exit 1 } } mirror_images_contiguous() { for i in `lvdevices $1/$2`; do lv_is_contiguous $1/$i done } mirror_images_clung() { for i in `lvdevices $1/$2`; do lv_is_clung $1/$i done } mirror() { mirror_nonredundant "$@" mirror_images_redundant "$1" "$2" } mirror_nonredundant() { lv="$1/$2" lvs -oattr "$lv" | grep "^ *m.......$" >/dev/null || { if lvs -oattr "$lv" | grep "^ *o.......$" >/dev/null && lvs -a | fgrep "[${2}_mimage" >/dev/null; then echo "TEST WARNING: $lv is a snapshot origin and looks like a mirror," echo "assuming it is actually a mirror" else echo "$lv expected a mirror, but is not:" lvs -a $lv exit 1 fi } if test -n "$3"; then mirror_log_on "$1" "$2" "$3"; fi } mirror_legs() { lv="$1/$2" expect="$3" lvdevices "$lv" real=`lvdevices "$lv" | wc -w` test "$expect" = "$real" } mirror_no_temporaries() { vg=$1 lv=$2 lvl -oname $vg | grep $lv | not grep "tmp" || { echo "$lv has temporary mirror images unexpectedly:" lvl $vg | grep $lv exit 1 } } linear() { lv="$1/$2" lvl -ostripes "$lv" | grep "1" >/dev/null || { echo "$lv expected linear, but is not:" lvl "$lv" -o+devices exit 1 } } active() { lv="$1/$2" lvl -oattr "$lv" 2> /dev/null | grep "^ *....a...$" >/dev/null || { echo "$lv expected active, but lvs says it's not:" lvl "$lv" -o+devices 2>/dev/null exit 1 } dmsetup table | egrep "$1-$2: *[^ ]+" >/dev/null || { echo "$lv expected active, lvs thinks it is but there are no mappings!" dmsetup table | grep $1-$2: exit 1 } } inactive() { lv="$1/$2" lvl -oattr "$lv" 2> /dev/null | grep '^ *....[-isd]...$' >/dev/null || { echo "$lv expected inactive, but lvs says it's not:" lvl "$lv" -o+devices 2>/dev/null exit 1 } dmsetup table | not egrep "$1-$2: *[^ ]+" >/dev/null || { echo "$lv expected inactive, lvs thinks it is but there are mappings!" dmsetup table | grep $1-$2: exit 1 } } lv_exists() { lv="$1/$2" lvl "$lv" >& /dev/null || { echo "$lv expected to exist but does not" exit 1 } } pv_field() { actual=$(trim $(pvs --noheadings $4 -o $2 $1)) test "$actual" = "$3" || { echo "pv_field: PV=$1, field=$2, actual=$actual, expected=$3" exit 1 } } vg_field() { actual=$(trim $(vgs --noheadings $4 -o $2 $1)) test "$actual" = "$3" || { echo "vg_field: vg=$1, field=$2, actual=$actual, expected=$3" exit 1 } } lv_field() { actual=$(trim $(lvs --noheadings $4 -o $2 $1)) test "$actual" = "$3" || { echo "lv_field: lv=$1, field=$2, actual=$actual, expected=$3" exit 1 } } compare_fields() { local cmd1=$1; local obj1=$2; local field1=$3; local cmd2=$4; local obj2=$5; local field2=$6; local val1; local val2; val1=$($cmd1 --noheadings -o $field1 $obj1) val2=$($cmd2 --noheadings -o $field2 $obj2) test "$val1" = "$val2" || { echo "compare_fields $obj1($field1): $val1 $obj2($field2): $val2" exit 1 } } compare_vg_field() { local vg1=$1; local vg2=$2; local field=$3; val1=$(vgs --noheadings -o $field $vg1) val2=$(vgs --noheadings -o $field $vg2) test "$val1" = "$val2" || { echo "compare_vg_field: $vg1: $val1, $vg2: $val2" exit 1 } } pvlv_counts() { local local_vg=$1 local num_pvs=$2 local num_lvs=$3 local num_snaps=$4 lvs -a -o+devices $local_vg vg_field $local_vg pv_count $num_pvs vg_field $local_vg lv_count $num_lvs vg_field $local_vg snap_count $num_snaps } btrfs_fs_field() { lines=$(btrfs filesystem show 2> /dev/null | grep -A 1 "'$1'" || true) case $2 in "label") actual=$(echo $lines | cut -f2 -d' ' | sed -e "s/'//g") ;; "dev_count") actual=$(echo $lines | cut -f7 -d' ') ;; "uuid") actual=$(echo $lines | cut -f4 -d' ') ;; *) echo "Unknown filed $2" exit 1 ;; esac test "$actual" = "$3" || { echo "btrfs_fs_field: label=$1, field=$2, actual=$actual, expected=$3" exit 1 } } btrfs_vol_field() { case $2 in "vol_count") output=$(btrfs subvolume list -a $1 || btrfs subvolume list $1) actual=$(echo "$output" | wc -l);; "subvolume") output=$(btrfs subvolume list -a $1 || btrfs subvolume list $1) actual=$(echo "$output" | sed "s/\/*//" | \ grep -E "$3$" || true) actual=${actual##*path } ;; *) echo "Unknown filed $2" exit 1 ;; esac test "$actual" = "$3" || { btrfs subvolume list $1 echo "btrfs_fs_field: mount=$1, field=$2, actual=$actual, expected=$3" exit 1 } } list_table() { # $1=ssm_list_output, $2=unique_value_in_desired_row # if some argument is defined as none = it is not checked # arguments might be regular expressions # size from ssm output is converted from 200.00 MB to 200.00MB row=($(echo "$1" | sed 's/\(.[0-9][0-9]\).\(.B\)/\1\2/g' | \ sed 's/\( \+\)/ /g' | grep "$2 ")) || { echo "table_list_failed: pattern \"$2\" not found" exit 1 } counter=1 for arg in "${@:3}"; do if [ $arg != "none" ] ; then if [[ ! ${row[$counter]} =~ $arg ]] ; then echo "table_list_failed: field=$(($counter + 1)), \ actual=${row[$counter]}, expected=$arg" exit 1 fi fi counter=$(($counter + 1)) done } mountpoint() { if [ $# -lt 2 ] ; then echo usage: mountpoint dev mountpoint exit 1 fi if ! grep "$1.*$2" /proc/mounts ; then echo error creating volume $1 with mountpoint at $2 exit 1 fi exit 0 } "$@" system-storage-manager-0.4/tests/bashtests/lib/ssm.sh0000664000175000017500000000076712171200774024530 0ustar lczernerlczerner00000000000000#!/bin/bash # Copyright 2011 (C) Red Hat, Inc., Lukas Czerner # # This copyrighted material is made available to anyone wishing to use, # modify, copy, or redistribute it subject to the terms and conditions # of the GNU General Public License v.2. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software Foundation, # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA $run_coverage $SSM "$@" system-storage-manager-0.4/tests/bashtests/lib/not.sh0000664000175000017500000000076512171200774024524 0ustar lczernerlczerner00000000000000#!/bin/bash # Copyright 2011 (C) Red Hat, Inc., Lukas Czerner # # This copyrighted material is made available to anyone wishing to use, # modify, copy, or redistribute it subject to the terms and conditions # of the GNU General Public License v.2. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software Foundation, # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA "$@" && exit 1 exit 0 system-storage-manager-0.4/tests/bashtests/lib/get.sh0000664000175000017500000000116612171200774024477 0ustar lczernerlczerner00000000000000# Copyright (C) 2011 Red Hat, Inc. All rights reserved. # # This copyrighted material is made available to anyone wishing to use, # modify, copy, or redistribute it subject to the terms and conditions # of the GNU General Public License v.2. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software Foundation, # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA pv_field() { pvs --noheading -o $2 $1 | sed 's/^ *//' } vg_field() { vgs --noheading -o $2 $1 | sed 's/^ *//' } lv_field() { lvs --noheading -o $2 $1 | sed 's/^ *//' } "$@" system-storage-manager-0.4/tests/bashtests/lib/test.sh0000664000175000017500000000364612200411405024670 0ustar lczernerlczerner00000000000000# Copyright (C) 2011 Red Hat, Inc. All rights reserved. # # This copyrighted material is made available to anyone wishing to use, # modify, copy, or redistribute it subject to the terms and conditions # of the GNU General Public License v.2. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software Foundation, # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # sanitize the environment LANG=C LC_ALL=C TZ=UTC unset CDPATH export HERE=$(pwd) export PATH=$HERE/lib:$PATH export SSM="$HERE/../../bin/ssm.local" chmod +x $SSM # grab some common utilities . lib/utils # set terminal width for testing ssm output export COLUMNS=1024 export LINES=25 export OLDDIR=$HERE export COMMON_PREFIX="SSMTEST" export PREFIX="${COMMON_PREFIX}$$" export SSM_PREFIX_FILTER=$PREFIX export TESTDIR=$(mkdtemp ${LVM_TEST_DIR-$(pwd)} $PREFIX.XXXXXXXXXX) \ || { echo "failed to create temporary directory in ${LVM_TEST_DIR-$(pwd)}"; exit 1; } export TEST_MNT=$TESTDIR/mnt # check if coverage exists export COVERAGE=$(which coverage) || unset COVERAGE if test -n "$COVERAGE"; then export run_coverage="$COVERAGE run -a" export COVERAGE_FILE=$OLDDIR/.coverage $COVERAGE erase fi trap 'aux teardown' EXIT # don't forget to clean up export LVM_SYSTEM_DIR=$TESTDIR/etc DM_DEV_DIR=$TESTDIR/dev mkdir $LVM_SYSTEM_DIR $TESTDIR/lib $DM_DEV_DIR if test -n "$LVM_TEST_DEVDIR" ; then DM_DEV_DIR="$LVM_TEST_DEVDIR" else mknod $DM_DEV_DIR/testnull c 1 3 || exit 1; echo >$DM_DEV_DIR/testnull || { echo "Filesystem does support devices in $DM_DEV_DIR (mounted with nodev?)"; exit 1; } mkdir -p $DM_DEV_DIR/mapper fi export DM_DEV_DIR cd $TESTDIR ln -s $HERE/lib/* $TESTDIR/lib # re-do the utils now that we have TESTDIR/PREFIX/... . lib/utils set -eE -o pipefail aux lvmconf echo "@TESTDIR=$TESTDIR" echo "@PREFIX=$PREFIX" echo "@SSM_PREFIX_FILTER=$SSM_PREFIX_FILTER" set -vx system-storage-manager-0.4/tests/bashtests/lib/aux.sh0000664000175000017500000002310112171200774024506 0ustar lczernerlczerner00000000000000#!/bin/bash # Copyright (C) 2011 Red Hat, Inc. All rights reserved. # # This copyrighted material is made available to anyone wishing to use, # modify, copy, or redistribute it subject to the terms and conditions # of the GNU General Public License v.2. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software Foundation, # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA . lib/utils teardown_devs() { # Delete any remaining dm/udev semaphores teardown_udev_cookies test -n "$PREFIX" && { rm -rf $TESTDIR/dev/$PREFIX* init_udev_transaction while dmsetup table | grep -q ^$PREFIX; do for s in `dmsetup info -c -o name --noheading | grep ^$PREFIX`; do umount -fl $DM_DEV_DIR/mapper/$s >& /dev/null || true dmsetup remove -f $s >& /dev/null || true done done finish_udev_transaction } udev_wait # NOTE: SCSI_DEBUG_DEV test must come before the LOOP test because # prepare_scsi_debug_dev() also sets LOOP to short-circuit prepare_loop() if test -f SCSI_DEBUG_DEV; then modprobe -r scsi_debug else test -f LOOP && losetup -d $(cat LOOP) test -f LOOPFILE && rm -f $(cat LOOPFILE) fi rm -f DEVICES # devs is set in prepare_devs() rm -f LOOP # Attempt to remove any loop devices that failed to get torn down if earlier tests aborted test -n "$COMMON_PREFIX" && { # Resume any linears to be sure we do not deadlock STRAY_DEVS=$(dmsetup table | sed 's/:.*//' | grep $COMMON_PREFIX | cut -d' ' -f 1) for dm in $STRAY_DEVS ; do # FIXME: only those really suspended echo dmsetup resume $dm dmsetup resume $dm || true done STRAY_MOUNTS=`mount | grep $COMMON_PREFIX | cut -d\ -f1` if test -n "$STRAY_MOUNTS"; then echo "Removing stray mounted devices containing $COMMON_PREFIX:" mount | grep $COMMON_PREFIX umount -fl $STRAY_MOUNTS || true sleep 2 fi init_udev_transaction NUM_REMAINING_DEVS=999 while NUM_DEVS=`dmsetup table | grep ^$COMMON_PREFIX | wc -l` && \ test $NUM_DEVS -lt $NUM_REMAINING_DEVS -a $NUM_DEVS -ne 0; do echo "Removing $NUM_DEVS stray mapped devices with names beginning with $COMMON_PREFIX:" STRAY_DEVS=$(dmsetup table | sed 's/:.*//' | grep $COMMON_PREFIX | cut -d' ' -f 1) dmsetup info -c | grep ^$COMMON_PREFIX for dm in $STRAY_DEVS ; do echo dmsetup remove $dm dmsetup remove -f $dm || true done NUM_REMAINING_DEVS=$NUM_DEVS done finish_udev_transaction udev_wait STRAY_LOOPS=`losetup -a | grep $COMMON_PREFIX | cut -d: -f1` if test -n "$STRAY_LOOPS"; then echo "Removing stray loop devices containing $COMMON_PREFIX:" losetup -a | grep $COMMON_PREFIX losetup -d $STRAY_LOOPS || true fi } } teardown() { # Print out coverage if test -n "$COVERAGE"; then $COVERAGE report fi echo -n "## teardown..." # Umount devices umount_all echo -n . test -d $DM_DEV_DIR/mapper && teardown_devs echo -n . test -n "$TESTDIR" && { cd $OLDDIR rm -rf $TESTDIR || echo BLA } echo "ok" } make_ioerror() { echo 0 10000000 error | dmsetup create -u TEST-ioerror ioerror ln -s $DM_DEV_DIR/mapper/ioerror $DM_DEV_DIR/ioerror } prepare_mnts() { count=$1 [ ! -d $TEST_MNT ] && mkdir $TEST_MNT &> /dev/null for n in $(seq $count); do mnt="$TEST_MNT/test$n" mkdir $mnt done } prepare_loop() { size=$1 test -n "$size" || size=32 echo -n "## preparing loop device..." # skip if prepare_scsi_debug_dev() was used if [ -f "SCSI_DEBUG_DEV" -a -f "LOOP" ]; then echo "(skipped)" return 0 fi test ! -e LOOP test -n "$DM_DEV_DIR" for i in 0 1 2 3 4 5 6 7; do test -e $DM_DEV_DIR/loop$i || mknod $DM_DEV_DIR/loop$i b 7 $i done echo -n . LOOPFILE="$PWD/test.img" dd if=/dev/zero of="$LOOPFILE" bs=$((1024*1024)) count=0 seek=$(($size-1)) 2> /dev/null if LOOP=`losetup -s -f "$LOOPFILE" 2>/dev/null`; then : elif LOOP=`losetup -f` && losetup $LOOP "$LOOPFILE"; then # no -s support : else # no -f support # Iterate through $DM_DEV_DIR/loop{,/}{0,1,2,3,4,5,6,7} for slash in '' /; do for i in 0 1 2 3 4 5 6 7; do local dev=$DM_DEV_DIR/loop$slash$i ! losetup $dev >/dev/null 2>&1 || continue # got a free losetup "$dev" "$LOOPFILE" LOOP=$dev break done if [ -n "$LOOP" ]; then break fi done fi test -n "$LOOP" # confirm or fail echo "$LOOP" > LOOP echo "ok ($LOOP)" } # A drop-in replacement for prepare_loop() that uses scsi_debug to create # a ramdisk-based SCSI device upon which all LVM devices will be created # - scripts must take care not to use a DEV_SIZE that will enduce OOM-killer prepare_scsi_debug_dev() { local DEV_SIZE="$1" shift local SCSI_DEBUG_PARAMS="$@" test -f "SCSI_DEBUG_DEV" && return 0 test -z "$LOOP" test -n "$DM_DEV_DIR" # Skip test if awk isn't available (required for get_sd_devs_) which awk || skip # Skip test if scsi_debug module is unavailable or is already in use modprobe --dry-run scsi_debug || skip lsmod | grep -q scsi_debug && skip # Create the scsi_debug device and determine the new scsi device's name # NOTE: it will _never_ make sense to pass num_tgts param; # last param wins.. so num_tgts=1 is imposed modprobe scsi_debug dev_size_mb=$DEV_SIZE $SCSI_DEBUG_PARAMS num_tgts=1 || skip sleep 2 # allow for async Linux SCSI device registration local DEBUG_DEV=/dev/$(grep -H scsi_debug /sys/block/*/device/model | cut -f4 -d /) [ -b $DEBUG_DEV ] || exit 1 # should not happen # Create symlink to scsi_debug device in $DM_DEV_DIR SCSI_DEBUG_DEV="$DM_DEV_DIR/$(basename $DEBUG_DEV)" echo "$SCSI_DEBUG_DEV" > SCSI_DEBUG_DEV echo "$SCSI_DEBUG_DEV" > LOOP # Setting $LOOP provides means for prepare_devs() override ln -snf $DEBUG_DEV $SCSI_DEBUG_DEV return 0 } cleanup_scsi_debug_dev() { aux teardown_devs rm -f SCSI_DEBUG_DEV rm -f LOOP } prepare_devs() { local n="$1" test -z "$n" && n=3 local devsize="$2" test -z "$devsize" && devsize=34 local pvname="$3" test -z "$pvname" && pvname="pv" prepare_loop $(($n*$devsize)) echo -n "## preparing $n devices..." if ! loopsz=`blockdev --getsz $LOOP 2>/dev/null`; then loopsz=`blockdev --getsize $LOOP 2>/dev/null` fi local size=$(($loopsz/$n)) devs= init_udev_transaction for i in `seq 1 $n`; do local name="${PREFIX}$pvname$i" local dev="$DM_DEV_DIR/mapper/$name" devs="$devs $dev" echo 0 $size linear $LOOP $((($i-1)*$size)) > $name.table dmsetup create -u TEST-$name $name $name.table done finish_udev_transaction #for i in `seq 1 $n`; do # local name="${PREFIX}$pvname$i" # dmsetup info -c $name #done #for i in `seq 1 $n`; do # local name="${PREFIX}$pvname$i" # dmsetup table $name #done echo $devs > DEVICES echo "ok" } disable_dev() { init_udev_transaction for dev in "$@"; do dmsetup remove -f $dev || true done finish_udev_transaction } enable_dev() { init_udev_transaction for dev in "$@"; do local name=`echo "$dev" | sed -e 's,.*/,,'` dmsetup create -u TEST-$name $name $name.table || dmsetup load $name $name.table dmsetup resume $dev done finish_udev_transaction } backup_dev() { for dev in "$@"; do dd if=$dev of=$dev.backup bs=1024 done } restore_dev() { for dev in "$@"; do test -e $dev.backup || { echo "Internal error: $dev not backed up, can't restore!" exit 1 } dd of=$dev if=$dev.backup bs=1024 done } prepare_pvs() { prepare_devs "$@" pvcreate -ff $devs } prepare_vg() { vgremove -ff $vg >& /dev/null || true teardown_devs prepare_pvs "$@" vgcreate -c n $vg $devs #pvs -v } lvmconf() { if test -z "$LVM_TEST_LOCKING"; then LVM_TEST_LOCKING=1; fi if test "$DM_DEV_DIR" = "/dev"; then LVM_VERIFY_UDEV=${LVM_VERIFY_UDEV:-0}; else LVM_VERIFY_UDEV=${LVM_VERIFY_UDEV:-1}; fi test -f CONFIG_VALUES || { cat > CONFIG_VALUES <<-EOF devices/dir = "$DM_DEV_DIR" devices/scan = "$DM_DEV_DIR" devices/filter = [ "a/dev\/mirror/", "a/dev\/mapper\/.*pv[0-9_]*$/", "r/.*/" ] devices/cache_dir = "$TESTDIR/etc" devices/sysfs_scan = 0 devices/default_data_alignment = 1 log/syslog = 0 log/indent = 1 log/level = 9 log/file = "$TESTDIR/debug.log" log/overwrite = 1 log/activation = 1 backup/backup = 0 backup/archive = 0 global/abort_on_internal_errors = 1 global/detect_internal_vg_cache_corruption = 1 global/library_dir = "$TESTDIR/lib" global/locking_dir = "$TESTDIR/var/lock/lvm" global/locking_type=$LVM_TEST_LOCKING global/si_unit_consistency = 1 global/fallback_to_local_locking = 0 activation/checks = 1 activation/udev_sync = 1 activation/udev_rules = 1 activation/verify_udev_operations = $LVM_VERIFY_UDEV activation/polling_interval = 0 activation/snapshot_autoextend_percent = 50 activation/snapshot_autoextend_threshold = 50 activation/monitoring = 0 EOF } for v in "$@"; do echo "$v" >> CONFIG_VALUES done rm -f CONFIG for s in `cat CONFIG_VALUES | cut -f1 -d/ | sort | uniq`; do echo "$s {" >> CONFIG for k in `grep ^$s/ CONFIG_VALUES | cut -f1 -d= | sed -e 's, *$,,' | sort | uniq`; do grep "^$k" CONFIG_VALUES | tail -n 1 | sed -e "s,^$s/, ," >> CONFIG done echo "}" >> CONFIG echo >> CONFIG done mv -f CONFIG $TESTDIR/etc/lvm.conf } udev_wait() { pgrep udev >/dev/null || return 0 which udevadm >/dev/null || return 0 if test -n "$1" ; then udevadm settle --exit-if-exists=$1 else udevadm settle --timeout=15 fi } test -f DEVICES && devs=$(cat DEVICES) test -f LOOP && LOOP=$(cat LOOP) "$@" system-storage-manager-0.4/tests/bashtests/006-btrfs-add.sh0000775000175000017500000000664612200411405025322 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2012 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='006-btrfs-add' test_description='Exercise ssm add with btrfs backend' . lib/test DEV_COUNT=10 DEV_SIZE=300 TEST_MAX_SIZE=$(($DEV_COUNT*$DEV_SIZE)) aux prepare_devs $DEV_COUNT $DEV_SIZE aux prepare_mnts 10 TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='btrfs' export SSM_BTRFS_DEFAULT_POOL=$vg1 export VOL_PREFIX="vol" export SSM_NONINTERACTIVE='1' vol1=${VOL_PREFIX}001 vol2=${VOL_PREFIX}002 vol3=${VOL_PREFIX}003 pool1=$vg2 pool2=$vg3 # Create default pool with all devices at once ssm add $TEST_DEVS check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT check list_table "$(ssm list pool)" $SSM_BTRFS_DEFAULT_POOL btrfs 10 none none none ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Specify backend ssm -b btrfs add $TEST_DEVS check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT ssm -f remove $SSM_BTRFS_DEFAULT_POOL export SSM_DEFAULT_BACKEND='lvm' ssm --backend btrfs add $TEST_DEVS check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT ssm -f remove $SSM_BTRFS_DEFAULT_POOL export SSM_DEFAULT_BACKEND='btrfs' ssm add $TEST_DEVS check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT check list_table "$(ssm list pool)" $SSM_BTRFS_DEFAULT_POOL btrfs $DEV_COUNT none none none ssm remove $dev1 $dev2 $dev3 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $(($DEV_COUNT-3)) ssm remove $dev4 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $(($DEV_COUNT-4)) check list_table "$(ssm list pool)" $SSM_BTRFS_DEFAULT_POOL btrfs $(($DEV_COUNT-4)) none none none ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Create default pool by adding devices one per a call for i in $TEST_DEVS; do ssm add $i done check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Create different groups from different devices ssm add $dev4 ssm add $dev1 $dev2 $dev3 -p $pool1 ssm add --pool $pool2 $dev7 $dev8 ssm add $dev5 $dev6 not ssm add $dev5 $dev6 $dev1 -p $pool1 ssm add $dev9 $dev1 -p $pool1 ssm add $dev10 -p $pool2 not ssm add $dev10 -p $pool1 not ssm add $dev10 $pool2 -p $pool1 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count 3 check btrfs_fs_field $pool1 dev_count 4 check btrfs_fs_field $pool2 dev_count 3 ssm_output=$(ssm list pool) check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL btrfs 3 none none none check list_table "$ssm_output" $pool1 btrfs 4 none none none check list_table "$ssm_output" $pool2 btrfs 3 none none none ssm -f remove --all ssm add --help # Some cases which should fail not ssm _garbage_ not ssm add not ssm add _garbage_ not ssm add $dev1 ${dev1}not_exist not check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL label $SSM_BTRFS_DEFAULT_POOL not ssm add _somepool not ssm add $dev1 $dev2 $dev3 -p $pool1 _otherpool system-storage-manager-0.4/tests/bashtests/003-lvm-remove.sh0000775000175000017500000001263612200411405025536 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='003-remove' test_description='Exercise ssm remove' . lib/test DEV_COUNT=10 DEV_SIZE=10 TEST_MAX_SIZE=$(($DEV_COUNT*$DEV_SIZE)) aux prepare_devs $DEV_COUNT $DEV_SIZE aux prepare_mnts 1 TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='lvm' export SSM_LVM_DEFAULT_POOL=$vg1 export LVOL_PREFIX="lvol" export SSM_NONINTERACTIVE='1' lvol1=${LVOL_PREFIX}001 lvol2=${LVOL_PREFIX}002 lvol3=${LVOL_PREFIX}003 pool1=$vg2 pool2=$vg3 pool3=$vg4 DEFAULT_VOLUME=${SSM_LVM_DEFAULT_POOL}/$lvol1 _FS= which mkfs.ext2 && _FS="ext2" which mkfs.ext3 && _FS="ext3" which mkfs.ext4 && _FS="ext4" which mkfs.xfs && _FS="xfs" TEST_MNT=$TESTDIR/mnt # Remove logical volume ssm create $TEST_DEVS check lv_field $DEFAULT_VOLUME lv_name $lvol1 ssm -f remove $DEFAULT_VOLUME not check lv_field $DEFAULT_VOLUME lv_name $lvol1 # Remove volume group ssm create $TEST_DEVS check vg_field $SSM_LVM_DEFAULT_POOL vg_name $SSM_LVM_DEFAULT_POOL ssm -f remove $SSM_LVM_DEFAULT_POOL not check vg_field $SSM_LVM_DEFAULT_POOL vg_name $SSM_LVM_DEFAULT_POOL # Remove unused devices from the pool ssm create $dev1 $dev2 $dev3 ssm add $TEST_DEVS check vg_field $SSM_LVM_DEFAULT_POOL pv_count $DEV_COUNT ssm -f remove $TEST_DEVS check vg_field $SSM_LVM_DEFAULT_POOL pv_count 3 not ssm remove $dev1 $dev2 $dev3 ssm -f remove $DEFAULT_VOLUME ssm remove $dev1 $dev2 $dev3 not ssm remove $dev3 ssm -f remove --all # Remove multiple things ssm add $dev1 $dev2 -p $pool1 ssm add $dev3 $dev4 --pool $pool2 ssm create -p $pool2 ssm create $dev5 $dev6 ssm create $dev7 $dev8 ssm add $dev9 check vg_field $pool1 pv_count 2 check vg_field $pool2 pv_count 2 check vg_field $pool2 lv_count 1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 5 check vg_field $SSM_LVM_DEFAULT_POOL lv_count 2 check vg_field $pool1 vg_name $pool1 check lv_field ${pool2}/$lvol1 lv_name $lvol1 ssm -f remove $pool1 ${pool2}/$lvol1 $DEFAULT_VOLUME $dev9 not check vg_field $pool1 vg_name $pool1 not check lv_field ${pool2}/$lvol1 lv_name $lvol1 not check lv_field $DEFAULT_VOLUME lv_name $lvol1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 4 ssm -f remove --all # Remove multiple devices ssm add $dev1 $dev2 $dev3 -p $pool1 ssm add $dev4 $dev5 --pool $pool2 ssm add $dev6 -p $pool3 ssm remove $dev1 $dev2 check vg_field $pool1 pv_count 1 ssm add $dev1 $dev2 $dev3 -p $pool1 check vg_field $pool1 pv_count 3 ssm remove $dev1 $dev2 $dev4 $dev6 check vg_field $pool1 pv_count 1 check vg_field $pool2 pv_count 1 check vg_field $pool3 pv_count 1 ssm -f remove -a # Remove multiple volumes ssm create $dev1 $dev2 ssm add -p $pool1 $TEST_DEVS check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 2 check vg_field $pool1 pv_count 8 ssm create -s ${DEV_SIZE}M -p $pool1 -n $lvol1 ssm create -s ${DEV_SIZE}M -p $pool1 -n $lvol2 ssm create -s ${DEV_SIZE}M -p $pool1 -n $lvol3 check vg_field $pool1 lv_count 3 ssm -f remove $pool1/$lvol1 $pool1/$lvol2 check vg_field $pool1 lv_count 1 ssm create -s ${DEV_SIZE}M -p $pool1 -n $lvol1 ssm create -s ${DEV_SIZE}M -p $pool1 -n $lvol2 ssm -f remove $SSM_LVM_DEFAULT_POOL/$lvol1 $pool1/$lvol1 $pool1/$lvol2 check vg_field $SSM_LVM_DEFAULT_POOL lv_count 0 check vg_field $pool1 lv_count 1 ssm -f remove -a # Remove multiple pools ssm create $dev1 $dev2 ssm create -p $pool1 $dev3 $dev4 ssm add -p $pool2 $dev5 $dev6 $dev7 ssm -f remove $SSM_LVM_DEFAULT_POOL $pool1 $pool2 not check vg_field $SSM_LVM_DEFAULT_POOL vg_name $SSM_LVM_DEFAULT_POOL not check vg_field $pool1 vg_name $pool1 not check vg_field $pool1 vg_name $pool1 # Remove mounted volumes ssm create --fs ext4 $dev1 $dev2 $mnt1 # Check mounted fs not ssm check $SSM_LVM_DEFAULT_POOL/$lvol1 # Force the removal ssm -f remove $SSM_LVM_DEFAULT_POOL/$lvol1 not check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_name $lvol1 ssm create --fs ext4 $dev1 $dev2 $mnt1 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 2 ssm list not ssm -f remove $SSM_LVM_DEFAULT_POOL check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_name $lvol1 umount $mnt1 ssm check $SSM_LVM_DEFAULT_POOL/$lvol1 ssm -f remove --all # Remove all ssm add $dev1 $dev2 -p $pool1 ssm add $dev3 $dev4 --pool $pool2 ssm create --pool $pool2 ssm create $dev5 $dev6 ssm create $dev7 $dev8 ssm add $dev9 check vg_field $pool1 pv_count 2 check vg_field $pool2 pv_count 2 check vg_field $pool2 lv_count 1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 5 check vg_field $SSM_LVM_DEFAULT_POOL lv_count 2 check vg_field $pool1 vg_name $pool1 check vg_field $pool2 vg_name $pool2 check vg_field $SSM_LVM_DEFAULT_POOL vg_name $SSM_LVM_DEFAULT_POOL ssm -f remove --all not ssm -f remove --all not check vg_field $pool1 vg_name $pool1 not check vg_field $pool2 vg_name $pool2 not check vg_field $SSM_LVM_DEFAULT_POOL vg_name $SSM_LVM_DEFAULT_POOL ssm remove --help # Some cases which should fail not ssm remove not ssm remove non_exist system-storage-manager-0.4/tests/bashtests/runbashtests.sh0000775000175000017500000000340112200411405025660 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2013 Red Hat, Inc., Jimmy Pan # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . if [ "$1" == "-h" -o "$1" == "--help" -o "$#" == 0 ] ; then echo "usage: runbashtests.sh [all|*.sh ...]" exit 1 fi set -e set -o pipefail set +xv . set.sh rm -f lib/ssm PS4='+$BASH_SOURCE $LINENO: ' export PS4 function runtest() { echo "**************************************" echo " RUNNING $1" echo "**************************************" if [ "$2" == "suppress_output" ] ; then ./$1 >${i}.out 2>&1 else ./$1 2>&1 | tee ${i}.out fi if [ "$?" == "0" ] ; then echo "**************************************" echo " TEST PASSED" echo "**************************************" return 0 else echo "**************************************" echo " TEST FAILED" echo "**************************************" return 1 fi } set -i set +e if [ "$@" == "all" ] ; then for i in 0*.sh ; do runtest $i suppress_output done else for i in "$@" ; do if ! [ -e $i ] ; then echo file doesn\'t exist continue; fi runtest $i done fi system-storage-manager-0.4/tests/bashtests/002-lvm-create.sh0000775000175000017500000002714112200413616025506 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='002-create' test_description='Exercise ssm create' . lib/test DEV_COUNT=10 DEV_SIZE=100 TEST_MAX_SIZE=$(($DEV_COUNT*$DEV_SIZE)) aux prepare_devs $DEV_COUNT $DEV_SIZE aux prepare_mnts 4 TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='lvm' export SSM_LVM_DEFAULT_POOL=$vg1 export LVOL_PREFIX="lvol" export SSM_NONINTERACTIVE='1' lvol1=${LVOL_PREFIX}001 lvol2=${LVOL_PREFIX}002 lvol3=${LVOL_PREFIX}003 pool1=$vg2 pool2=$vg3 TEST_FS= which mkfs.ext2 && TEST_FS+="ext2 " which mkfs.ext3 && TEST_FS+="ext3 " which mkfs.ext4 && TEST_FS+="ext4 " which mkfs.xfs && TEST_FS+="xfs" # Create with single device ssm create $dev1 ssm create -p $pool1 $dev2 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 1 check lv_field $pool1/$lvol1 pv_count 1 ssm -f remove -a # Create volume with all devices at once ssm create $TEST_DEVS not ssm create $TEST_DEVS not ssm create $TEST_DEVS -p $pool1 not ssm create -s ${DEV_SIZE}M $TEST_DEVS check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count $DEV_COUNT check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL lvm 10 none none 960.00MB check list_table "$(ssm list vol)" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL 960.00MB linear ssm -f remove $SSM_LVM_DEFAULT_POOL # Specify backend ssm -b lvm create $TEST_DEVS not ssm create $TEST_DEVS not ssm create $TEST_DEVS -p $pool1 not ssm create -s ${DEV_SIZE}M $TEST_DEVS check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count $DEV_COUNT ssm -f remove $SSM_LVM_DEFAULT_POOL export SSM_DEFAULT_BACKEND='btrfs' ssm --backend lvm create $TEST_DEVS not ssm create $TEST_DEVS not ssm create $TEST_DEVS -p $pool1 not ssm create -s ${DEV_SIZE}M $TEST_DEVS check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count $DEV_COUNT ssm -f remove $SSM_LVM_DEFAULT_POOL export SSM_DEFAULT_BACKEND='lvm' # Create the group first and then create volume using the whole group ssm add $TEST_DEVS ssm create check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count $DEV_COUNT ssm -f remove $SSM_LVM_DEFAULT_POOL/$lvol1 ssm -f create -s ${DEV_SIZE}m check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size $DEV_SIZE.00m ssm -f remove $SSM_LVM_DEFAULT_POOL # Create a logical volume of fixed size size=$(($DEV_SIZE*6)) ssm create -s ${size}M $TEST_DEVS not ssm create -s ${TEST_MAX_SIZE}M size=$(align_size_up $size) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size ${size}.00m check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL lvm 10 360.00MB 600.00MB 960.00MB check list_table "$(ssm list vol)" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL 600.00MB linear ssm -f remove $SSM_LVM_DEFAULT_POOL # Create a striped logical volume not ssm create -I 32 $TEST_DEVS ssm create -r 0 -I 32 $TEST_DEVS not ssm create -I 32 -s ${TEST_MAX_SIZE}M check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripesize 32.00k check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripes $DEV_COUNT check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL lvm 10 none none 960.00MB check list_table "$(ssm list vol)" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL 960.00MB striped ssm -f remove $SSM_LVM_DEFAULT_POOL # Create a default raid 0 logical volume ssm create -r 0 $TEST_DEVS not ssm create -r 0 -s ${TEST_MAX_SIZE}M check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripesize 64.00k check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripes $DEV_COUNT check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL lvm 10 none none 960.00MB check list_table "$(ssm list vol)" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL 960.00MB striped ssm -f remove $SSM_LVM_DEFAULT_POOL # Create several volumes with different parameters ssm add $TEST_DEVS not ssm create -I 8 -i $(($DEV_COUNT/2)) -s $(($DEV_SIZE*2))M ssm create -r 0 -I 8 -i $(($DEV_COUNT/2)) -s $(($DEV_SIZE*2))M not ssm create -i $(($DEV_COUNT)) -s $(($DEV_SIZE))M ssm create -r 0 -i $(($DEV_COUNT)) -s $(($DEV_SIZE))M not ssm create -r 0 -I 32 -s $(($DEV_SIZE*2))M ssm create check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripesize 8.00k check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripes $(($DEV_COUNT/2)) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol2 stripes $DEV_COUNT check lv_field $SSM_LVM_DEFAULT_POOL/$lvol3 segtype linear check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL lvm 10 none none 960.00MB ssm_output=$(ssm list vol) check list_table "$ssm_output" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL 200.00MB striped check list_table "$ssm_output" $SSM_LVM_DEFAULT_POOL/$lvol2 $SSM_LVM_DEFAULT_POOL 120.00MB striped check list_table "$ssm_output" $SSM_LVM_DEFAULT_POOL/$lvol3 $SSM_LVM_DEFAULT_POOL 640.00MB linear ssm -f remove $SSM_LVM_DEFAULT_POOL # Create several volumes with different parameters from different groups ssm add $dev1 $dev2 $dev3 -p $pool1 not ssm create $dev1 $dev2 ssm add $dev4 $dev5 $dev6 -p $pool2 not ssm create --stripesize 32 --stripes 3 --size $(($DEV_SIZE*2))M -p $pool2 ssm create -r 0 --stripesize 32 --stripes 3 --size $(($DEV_SIZE*2))M -p $pool2 ssm create -r 0 --stripesize 32 --stripes 3 --size $((DEV_SIZE/2))M -p $pool2 ssm create -r 0 $dev7 $dev8 $dev9 --stripesize 8 not ssm create -p $pool1 --stripes 3 ssm create -r 0 -p $pool1 --stripes 3 not ssm create -s ${DEV_SIZE}M -p $pool1 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripesize 8.00k check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripes 3 check lv_field $pool1/$lvol1 stripes 3 check lv_field $pool2/$lvol1 stripesize 32.00k check lv_field $pool2/$lvol1 stripes 3 check lv_field $pool2/$lvol2 stripesize 32.00k check lv_field $pool2/$lvol2 stripes 3 ssm_output=$(ssm list pool) check list_table "$ssm_output" $SSM_LVM_DEFAULT_POOL lvm 3 none none 288.00MB check list_table "$ssm_output" $vg2 lvm 3 none none 288.00MB check list_table "$ssm_output" $vg3 lvm 3 none none 288.00MB ssm_output=$(ssm list vol) check list_table "$ssm_output" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL 288.00MB striped check list_table "$ssm_output" $vg2/$lvol1 $vg2 288.00MB striped check list_table "$ssm_output" $vg3/$lvol1 $vg3 204.00MB striped check list_table "$ssm_output" $vg3/$lvol2 $vg3 60.00MB striped ssm -f remove --all # Create logical volumes with file system for fs in $TEST_FS; do # check addition with different names and sizes from existent and nonexistent pool # pool already exists ssm add $dev1 $dev2 $dev3 ssm create --fstype $fs -s ${DEV_SIZE}m -n $lvol1 $mnt1 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size $((DEV_SIZE)).00m check mountpoint $SSM_LVM_DEFAULT_POOL-$lvol1 $mnt1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 3 # pool doesn't exist ssm create --fstype $fs -s $((DEV_SIZE/2))m -n randvol $dev4 $mnt2 size=`align_size_up $((DEV_SIZE/2))` check lv_field $SSM_LVM_DEFAULT_POOL/randvol lv_size $size.00m check mountpoint $SSM_LVM_DEFAULT_POOL-randvol $mnt2 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 4 umount $mnt1 $mnt2 ssm check $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL/randvol ssm -f remove $SSM_LVM_DEFAULT_POOL # When we want to get unsed device from other pull, should succeed ssm add -p $pool1 $dev1 $dev2 $dev3 ssm -f create -p $pool2 --fstype $fs -s ${DEV_SIZE}m $dev1 $dev2 $mnt1 -n randvol check lv_field $pool2/randvol lv_size $((DEV_SIZE)).00m check mountpoint $pool2-randvol $mnt1 umount $mnt1 ssm check $pool2/randvol check vg_field $pool2 pv_count 2 ssm -f remove $pool1 ssm -f remove $pool2 ssm create --fs=$fs --name $lvol3 -s $(($DEV_SIZE*6))M $TEST_DEVS check lv_field $SSM_LVM_DEFAULT_POOL/$lvol3 pv_count $DEV_COUNT ssm -f check ${SSM_LVM_DEFAULT_POOL}/$lvol3 ssm check ${SSM_LVM_DEFAULT_POOL}/$lvol3 check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL lvm 10 none none 960.00MB check list_table "$(ssm list vol)" $SSM_LVM_DEFAULT_POOL/$lvol3 $SSM_LVM_DEFAULT_POOL $(($DEV_SIZE*6)).00MB $fs none none linear check list_table "$(ssm list fs)" $SSM_LVM_DEFAULT_POOL/$lvol3 $SSM_LVM_DEFAULT_POOL $(($DEV_SIZE*6)).00MB $fs none none linear ssm -f remove $SSM_LVM_DEFAULT_POOL ssm create --fs=$fs -r 0 -I 32 -s $(($DEV_SIZE*6))M $TEST_DEVS check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count $DEV_COUNT check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripesize 32.00k ssm check ${SSM_LVM_DEFAULT_POOL}/$lvol1 check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL lvm 10 none none 960.00MB check list_table "$(ssm list vol)" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL $(($DEV_SIZE*6)).00MB $fs none none striped check list_table "$(ssm list fs)" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL $(($DEV_SIZE*6)).00MB $fs none none striped ssm -f remove $SSM_LVM_DEFAULT_POOL ssm add $TEST_DEVS ssm create --fs=$fs -r 0 -I 8 -i $((DEV_COUNT/5)) -s $(($DEV_SIZE*2))M check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripes $(($DEV_COUNT/5)) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 stripesize 8.00k ssm check ${SSM_LVM_DEFAULT_POOL}/$lvol1 check list_table "$(ssm list pool)" $SSM_LVM_DEFAULT_POOL lvm 10 none none 960.00MB check list_table "$(ssm list vol)" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL $(($DEV_SIZE*2)).00MB $fs none none striped check list_table "$(ssm list fs)" $SSM_LVM_DEFAULT_POOL/$lvol1 $SSM_LVM_DEFAULT_POOL $(($DEV_SIZE*2)).00MB $fs none none striped ssm -f remove $SSM_LVM_DEFAULT_POOL done # Create volume without enough space in the pool ssm create $dev1 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 1 ssm add $dev2 $dev3 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 3 not ssm create -s $(($DEV_SIZE*3))M not check lv_field $SSM_LVM_DEFAULT_POOL/$lvol2 lv_name $lvol2 # Force to adjust the size ssm -f create -s $(($DEV_SIZE*3))M check lv_field $SSM_LVM_DEFAULT_POOL/$lvol2 lv_name $lvol2 check list_table "$(ssm list vol)" $SSM_LVM_DEFAULT_POOL/$lvol2 $SSM_LVM_DEFAULT_POOL 192.00MB linear ssm -f remove --all ssm create $dev1 $dev2 ssm -f remove $SSM_LVM_DEFAULT_POOL # Create volume on device with existing file system mkfs.ext3 $dev1 not ssm create $dev1 ssm create $dev1 $dev2 ssm -f remove --all # Create volume on device with existing file system with force mkfs.ext3 $dev1 ssm -f create $dev1 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 1 ssm -f remove $SSM_LVM_DEFAULT_POOL mkfs.ext3 $dev1 ssm -f create $dev1 $dev2 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 2 ssm -f remove --all # Create volume with device already used in different pool ssm add $dev1 $dev2 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 # Fail because $dev1 is already used not ssm create -p $pool1 $dev1 not ssm create -p $pool1 $dev1 $dev2 # Succeed because we have enough space to create volume just with $dev2 ssm create -s $(($DEV_SIZE/2))M -p $pool1 $dev1 $dev3 check lv_field $pool1/$lvol1 pv_count 1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 ssm -f remove --all # Create volume with device already used in different pool with force ssm add $dev1 $dev2 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 ssm -f create -p $pool1 $dev1 check lv_field $pool1/$lvol1 pv_count 1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 1 ssm -f remove --all ssm create --help # Some cases which should fail not ssm create ssm add $TEST_DEVS not ssm create -p $pool1 not ssm create -r 0 -I 16 -i 3 $dev1 $dev2 ssm -f remove --all system-storage-manager-0.4/tests/bashtests/010-btrfs-snapshot.sh0000775000175000017500000001226612200411405026417 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2012 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='010-btrfs-snapshot' test_description='Exercise ssm snapshot command with btrfs' . lib/test DEV_COUNT=10 DEV_SIZE=300 TEST_MAX_SIZE=$(($DEV_COUNT*$DEV_SIZE)) aux prepare_devs $DEV_COUNT $DEV_SIZE aux prepare_mnts 10 TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='btrfs' export SSM_BTRFS_DEFAULT_POOL=$vg1 export VOL_PREFIX="vol" export SSM_NONINTERACTIVE='1' vol1=${VOL_PREFIX}001 vol2=${VOL_PREFIX}002 vol3=${VOL_PREFIX}003 pool1=$vg2 pool2=$vg3 snap1="snap1" snap2="snap2" snap3="snap3" snap4="snap4" snap5="snap5" snap6="snap6" # Create volume with all devices at once ssm create $TEST_DEVS $mnt1 # Take a snapshot with the default params export SSM_DEFAULT_BACKEND='lvm' ssm snapshot $mnt1 check btrfs_vol_field $mnt1 vol_count 1 check list_table "$(ssm list snap)" $SSM_BTRFS_DEFAULT_POOL:snap-....-..-..-....... $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/snap-....-..-..-....... export SSM_DEFAULT_BACKEND='btrfs' umount $mnt1 # Remove entire pool ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Create volume with all devices at once ssm create $TEST_DEVS # Take a snapshot with the default params ssm snapshot $SSM_BTRFS_DEFAULT_POOL mount LABEL=$SSM_BTRFS_DEFAULT_POOL $mnt1 check btrfs_vol_field $mnt1 vol_count 1 umount $mnt1 # Remove entire pool ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Create volume with all devices at once ssm create $TEST_DEVS # Take a snapshot with defined name ssm snapshot --name $snap1 $SSM_BTRFS_DEFAULT_POOL ssm snapshot --name $snap2 $SSM_BTRFS_DEFAULT_POOL ssm snapshot --name $snap3 $SSM_BTRFS_DEFAULT_POOL mount LABEL=$SSM_BTRFS_DEFAULT_POOL $mnt1 check btrfs_vol_field $mnt1 vol_count 3 check btrfs_vol_field $mnt1 subvolume $snap1 check btrfs_vol_field $mnt1 subvolume $snap2 check btrfs_vol_field $mnt1 subvolume $snap3 ssm_output=$(ssm list vol) check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1 not check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap1 $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1/$snap1 not check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap2 $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1/$snap2 not check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap3 $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1/$snap3 ssm_output=$(ssm list snap) not check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL $SSM_BTRFS_DEFAULT_POOL none btrfs btrfs $mnt1 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap1 $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/$snap1 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap2 $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/$snap2 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap3 $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/$snap3 # Remove the snapshot volumes ssm -f remove $SSM_BTRFS_DEFAULT_POOL:$snap1 $SSM_BTRFS_DEFAULT_POOL:$snap2 $SSM_BTRFS_DEFAULT_POOL:$snap3 not check btrfs_vol_field $mnt1 subvolume $snap1 not check btrfs_vol_field $mnt1 subvolume $snap2 not check btrfs_vol_field $mnt1 subvolume $snap3 # Take a snapshot with defined name when volume is mounted ssm snapshot --name $snap1 $SSM_BTRFS_DEFAULT_POOL ssm snapshot --name $snap2 $mnt1 ssm snapshot --name $snap3 $SSM_BTRFS_DEFAULT_POOL ssm snapshot --name $snap4 $mnt1/$snap3 ssm snapshot --name $snap3/$snap4/$snap5 $mnt1 check btrfs_vol_field $mnt1 vol_count 5 check btrfs_vol_field $mnt1 subvolume $snap1 check btrfs_vol_field $mnt1 subvolume $snap2 check btrfs_vol_field $mnt1 subvolume $snap3 check btrfs_vol_field $mnt1 subvolume $snap3/$snap4 check btrfs_vol_field $mnt1 subvolume $snap3/$snap4/$snap5 ssm_output=$(ssm list vol) check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1 ssm_output=$(ssm list snap) check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap1 $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/$snap1 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap2 $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/$snap2 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap3 $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/$snap3 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap3/$snap4 $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/$snap3/$snap4 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$snap3/$snap4/$snap5 $SSM_BTRFS_DEFAULT_POOL none btrfs $mnt1/$snap3/$snap4/$snap5 umount_all ssm -f remove --all ssm snapshot --help # Some cases which should fail not ssm snapshot ssm create $TEST_DEVS not ssm snapshot $SSM_BTRFS_DEFAULT_POOL/$lvol1 ssm -f remove --all system-storage-manager-0.4/tests/bashtests/008-btrfs-remove.sh0000775000175000017500000001114712200411405026061 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2012 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='008-btrfs-remove' test_description='Exercise ssm remove command with btrfs backend' . lib/test DEV_COUNT=10 DEV_SIZE=300 TEST_MAX_SIZE=$(($DEV_COUNT*$DEV_SIZE)) aux prepare_devs $DEV_COUNT $DEV_SIZE aux prepare_mnts 10 TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='btrfs' export SSM_BTRFS_DEFAULT_POOL=$vg1 export VOL_PREFIX="vol" export SSM_NONINTERACTIVE='1' vol1=${VOL_PREFIX}001 vol2=${VOL_PREFIX}002 vol3=${VOL_PREFIX}003 pool1=$vg2 pool2=$vg3 # Remove subvolume ssm create $TEST_DEVS ssm create --name $vol1 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL label $SSM_BTRFS_DEFAULT_POOL ssm -f remove $SSM_BTRFS_DEFAULT_POOL not check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL label $SSM_BTRFS_DEFAULT_POOL # Remove volume group ssm create $TEST_DEVS $mnt1 ssm create --name $vol1 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT check btrfs_vol_field $mnt1 subvolume $vol1 ssm list ssm -f remove $mnt1/$vol1 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT not check btrfs_vol_field $mnt1 subvolume $vol1 umount $mnt1 ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Remove unused devices from the pool ssm create $dev1 btrfs filesystem show ssm add $TEST_DEVS btrfs filesystem show check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT ssm -f remove $TEST_DEVS btrfs filesystem show ssm list check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count 1 ssm -f remove --all # Remove multiple things ssm add $dev1 $dev2 -p $pool1 ssm create --pool $pool2 $dev3 $dev4 $mnt1 ssm create --name $vol1 -p $pool2 ssm create --name $vol1 $dev5 $dev6 $mnt3 ssm create --name $vol2 $dev7 $dev8 ssm add $dev9 check btrfs_fs_field $pool1 dev_count 2 check btrfs_fs_field $pool2 dev_count 2 check btrfs_vol_field $mnt1 vol_count 1 check btrfs_vol_field $mnt1 subvolume $vol1 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count 5 check btrfs_vol_field $mnt3 vol_count 1 check btrfs_vol_field $mnt3 subvolume $vol2 ssm list export SSM_DEFAULT_BACKEND='lvm' ssm -f remove $pool1 ${pool2}:$vol1 $mnt3/$vol2 $dev9 export SSM_DEFAULT_BACKEND='btrfs' not check btrfs_fs_field $pool1 label $pool1 not check btrfs_vol_field $mnt1 subvolume $vol1 not check btrfs_vol_field $mnt2 subvolume $vol2 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count 5 umount_all ssm -f remove --all # Remove all ssm add $dev1 $dev2 -p $pool1 ssm create --pool $pool2 $dev3 $dev4 $mnt1 ssm create --name $vol1 -p $pool2 ssm create --name $vol1 $dev5 $dev6 $mnt3 ssm create --name $vol2 $dev7 $dev8 ssm create --name $vol3 $mnt2 ssm add $dev9 # We can not remove mounted fs not ssm remove $pool2 # We can not remove mounted subvolume not ssm remove ${SSM_BTRFS_DEFAULT_POOL}:${vol3} check btrfs_fs_field $pool1 dev_count 2 check btrfs_fs_field $pool2 dev_count 2 check btrfs_vol_field $mnt1 vol_count 1 check btrfs_vol_field $mnt1 subvolume $vol1 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count 5 check btrfs_vol_field $mnt3 vol_count 2 check btrfs_vol_field $mnt3 subvolume $vol2 check btrfs_vol_field $mnt2 subvolume $vol3 # but we can force it ssm -f remove ${SSM_BTRFS_DEFAULT_POOL}:${vol3} not check btrfs_vol_field $mnt2 subvolume $vol3 umount_all ssm -f remove --all #Remove subvolume which is not mounted ssm create $dev1 $dev2 ssm create --name $vol1 ssm create --name $vol2 ssm create --name ${vol1}/${vol3} $mnt1 check btrfs_vol_field $mnt1 vol_count 3 ssm remove ${SSM_BTRFS_DEFAULT_POOL}:$vol2 ssm list check btrfs_vol_field $mnt1 vol_count 2 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count 2 check btrfs_vol_field $mnt1 subvolume $vol1 not check btrfs_vol_field $mnt1 subvolume $vol2 check btrfs_vol_field $mnt1 subvolume ${vol1}/${vol3} umount_all ssm -f remove --all not check btrfs_fs_field $pool1 label $pool1 not check btrfs_fs_field $pool2 label $pool2 not check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL label $SSM_BTRFS_DEFAULT_POOL ssm remove --help # Some cases which should fail not ssm remove not ssm -f remove --all system-storage-manager-0.4/tests/bashtests/005-lvm-snapshot.sh0000775000175000017500000001362712200415571026113 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='005-snapshot' export test_description='Exercise ssm snapshot' . lib/test DEV_COUNT=10 DEV_SIZE=100 TEST_MAX_SIZE=$(($DEV_COUNT*$DEV_SIZE)) aux prepare_devs $DEV_COUNT $DEV_SIZE TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='lvm' export SSM_LVM_DEFAULT_POOL=$vg1 export LVOL_PREFIX="lvol" export SSM_NONINTERACTIVE='1' lvol1=${LVOL_PREFIX}001 lvol2=${LVOL_PREFIX}002 lvol3=${LVOL_PREFIX}003 snap1="snap1" snap2="snap2" snap3="snap3" pool1=$vg2 pool2=$vg3 TEST_MNT=$TESTDIR/mnt # Create volume with all devices at once size=$(($DEV_SIZE*6)) ssm create --size ${size}M $TEST_DEVS # Take a snapshot with the default params export SSM_DEFAULT_BACKEND='btrfs' ssm snapshot --name $snap1 $SSM_LVM_DEFAULT_POOL/$lvol1 check vg_field $SSM_LVM_DEFAULT_POOL lv_count 2 check list_table "$(ssm list snap)" $snap1 $lvol1 none none none linear export SSM_DEFAULT_BACKEND='lvm' # Remove entire pool ssm -f remove $SSM_LVM_DEFAULT_POOL # Create volume with all devices at once size=$(($DEV_SIZE*6)) ssm create --size ${size}M $TEST_DEVS # Take a snapshot with defined name snap_size=$(($size/5)) ssm snapshot --name $snap1 $SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap1 lv_size ${snap_size}.00m ssm snapshot --name $snap2 $SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap2 lv_size ${snap_size}.00m ssm snapshot --name $snap3 $SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap3 lv_size ${snap_size}.00m check vg_field $SSM_LVM_DEFAULT_POOL lv_count 4 ssm_output=$(ssm list snap) check list_table "$ssm_output" $snap1 $lvol1 $SSM_LVM_DEFAULT_POOL ${snap_size}.00MB none linear check list_table "$ssm_output" $snap2 $lvol1 $SSM_LVM_DEFAULT_POOL ${snap_size}.00MB none linear check list_table "$ssm_output" $snap3 $lvol1 $SSM_LVM_DEFAULT_POOL ${snap_size}.00MB none linear # Remove the snapshot volumes ssm -f remove $SSM_LVM_DEFAULT_POOL/$snap1 $SSM_LVM_DEFAULT_POOL/$snap2 $SSM_LVM_DEFAULT_POOL/$snap3 # Take a snapshot with defined name and size snap_size=$(($DEV_SIZE)) ssm snapshot --size ${snap_size}M --name $snap1 $SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap1 lv_size ${snap_size}.00m ssm snapshot --size ${snap_size}M --name $snap2 $SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap2 lv_size ${snap_size}.00m ssm snapshot --size ${snap_size}M --name $snap3 $SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap3 lv_size ${snap_size}.00m check vg_field $SSM_LVM_DEFAULT_POOL lv_count 4 ssm -f remove --all # Create a logical volume with file system and mount it [ ! -d $TEST_MNT ] && mkdir $TEST_MNT &> /dev/null size=$(($DEV_SIZE*6)) ssm create --size ${size}M --fs ext4 $TEST_DEVS $TEST_MNT # Take a snapshot with defined name of volume referenced by the mountpoint snap_size=$(($size/5)) ssm snapshot --name $snap1 $TEST_MNT check lv_field $SSM_LVM_DEFAULT_POOL/$snap1 lv_size ${snap_size}.00m ssm snapshot --name $snap2 $TEST_MNT check lv_field $SSM_LVM_DEFAULT_POOL/$snap2 lv_size ${snap_size}.00m ssm snapshot --name $snap3 $TEST_MNT check lv_field $SSM_LVM_DEFAULT_POOL/$snap3 lv_size ${snap_size}.00m check vg_field $SSM_LVM_DEFAULT_POOL lv_count 4 ssm_output=$(ssm list snap) check list_table "$ssm_output" $snap1 $lvol1 $SSM_LVM_DEFAULT_POOL ${snap_size}.00MB none linear check list_table "$ssm_output" $snap2 $lvol1 $SSM_LVM_DEFAULT_POOL ${snap_size}.00MB none linear check list_table "$ssm_output" $snap3 $lvol1 $SSM_LVM_DEFAULT_POOL ${snap_size}.00MB none linear check list_table "$(ssm list fs)" $lvol1 $SSM_LVM_DEFAULT_POOL ${size}.00MB ext4 ${size}.00MB none linear $TEST_MNT # Remove the snapshot volumes ssm -f remove $SSM_LVM_DEFAULT_POOL/$snap1 $SSM_LVM_DEFAULT_POOL/$snap2 $SSM_LVM_DEFAULT_POOL/$snap3 # Take a snapshot with defined name of volume referenced by the full volume name snap_size=$(($DEV_SIZE)) ssm snapshot --size ${snap_size}M --name $snap1 $DM_DEV_DIR/$SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap1 lv_size ${snap_size}.00m ssm snapshot --size ${snap_size}M --name $snap2 $DM_DEV_DIR/$SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap2 lv_size ${snap_size}.00m ssm snapshot --size ${snap_size}M --name $snap3 $DM_DEV_DIR/$SSM_LVM_DEFAULT_POOL/$lvol1 check lv_field $SSM_LVM_DEFAULT_POOL/$snap3 lv_size ${snap_size}.00m check vg_field $SSM_LVM_DEFAULT_POOL lv_count 4 umount $TEST_MNT ssm -f remove --all # Snapshot of the volumes in defferent pools ssm create --pool $pool1 $dev1 $dev2 ssm add $dev3 $dev4 --pool $pool1 ssm create --pool $pool2 $dev5 $dev6 ssm add $dev7 $dev8 --pool $pool2 ssm snapshot --name $snap1 $pool1/$lvol1 ssm snapshot --name $snap1 $pool2/$lvol1 check lv_field $pool1/$snap1 lv_name $snap1 check lv_field $pool2/$snap1 lv_name $snap1 ssm -f remove --all ssm snapshot --help # Some cases which should fail not ssm snapshot ssm create $TEST_DEVS not ssm snapshot $SSM_LVM_DEFAULT_POOL/$lvol1 ssm -f resize -s $((DEV_SIZE*2)) $SSM_LVM_DEFAULT_POOL/$lvol1 not ssm snapshot $SSM_LVM_DEFAULT_POOL/${lvol1}notexist ssm -f remove --all # Snapshot size overflow should fail ssm create $dev1 $dev2 ssm add $dev3 not ssm snapthot -s $((DEV_SIZE*2)) -n $snap1 $SSM_LVM_DEFAULT_POOL/$lvol1 check vg_field $SSM_LVM_DEFAULT_POOL lv_count 1 ssm -f remove --all system-storage-manager-0.4/tests/bashtests/007-btrfs-create.sh0000775000175000017500000001375012200411405026030 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2012 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='007-btrfs-create' test_description='Exercise ssm create command with btrfs backend' . lib/test DEV_COUNT=10 DEV_SIZE=300 TEST_MAX_SIZE=$(($DEV_COUNT*$DEV_SIZE)) aux prepare_devs $DEV_COUNT $DEV_SIZE aux prepare_mnts 10 TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='btrfs' export SSM_BTRFS_DEFAULT_POOL=$vg1 export VOL_PREFIX="vol" export SSM_NONINTERACTIVE='1' vol1=${VOL_PREFIX}001 vol2=${VOL_PREFIX}002 vol3=${VOL_PREFIX}003 pool1=$vg2 pool2=$vg3 # Create volume with all devices at once ssm create $TEST_DEVS $mnt1 not ssm create $TEST_DEVS -p $pool1 #Create subvolume with nonexisting path not ssm create -n $vol1/$vol2 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count $DEV_COUNT check list_table "$(ssm list vol)" $SSM_BTRFS_DEFAULT_POOL $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1 ssm create ssm create --name $vol1 ssm create --name $vol1/$vol2 check btrfs_vol_field $mnt1 vol_count 3 check btrfs_vol_field $mnt1 subvolume $vol1 check btrfs_vol_field $mnt1 subvolume $vol1/$vol2 ssm_output=$(ssm list vol) check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:....-..-..-....... $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1/....-..-..-....... check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$vol1 $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1/$vol1 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$vol1/$vol2 $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1/$vol1/$vol2 umount $mnt1 ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Create volume with just one device ssm create $dev1 not ssm create $dev1 -p $pool1 ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Specify backend ssm --backend btrfs create $dev1 not ssm create $dev1 -p $pool1 ssm -f remove $SSM_BTRFS_DEFAULT_POOL export SSM_DEFAULT_BACKEND='lvm' ssm -b btrfs create $dev1 not ssm create $dev1 -p $pool1 ssm -f remove $SSM_BTRFS_DEFAULT_POOL export SSM_DEFAULT_BACKEND='btrfs' # Create raid 0 volume with just one device ssm create -r 0 $dev1 $dev2 $dev3 $dev4 not ssm create $dev1 -p $pool1 ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Create raid 1 volume with just one device ssm create -r 1 $dev1 $dev2 $dev3 $dev4 not ssm create $dev1 -p $pool1 ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Create raid 10 volume with just one device ssm create -r 10 $dev1 $dev2 $dev3 $dev4 not ssm create $dev1 -p $pool1 ssm -f remove $SSM_BTRFS_DEFAULT_POOL # Create several volumes with several pools ssm create $dev1 $mnt1 ssm create --name $vol1 check btrfs_fs_field $SSM_BTRFS_DEFAULT_POOL dev_count 1 check btrfs_vol_field $mnt1 subvolume $vol1 ssm create --pool $pool1 $dev2 $dev3 $mnt2 ssm create --name $vol2 --pool $pool1 # Also try to mount the subvolume somewhere else ssm create --name $vol3 --pool $pool1 $mnt3 check btrfs_fs_field $pool1 dev_count 2 check btrfs_vol_field $mnt2 subvolume $vol2 check btrfs_vol_field $mnt1 vol_count 1 check btrfs_vol_field $mnt2 vol_count 2 not check btrfs_vol_field $mnt2 subvolume $vol1 ssm create --name $vol1 --pool $pool2 $dev4 $dev5 $dev6 check btrfs_fs_field $pool2 dev_count 3 ssm create --name $vol2 --pool $pool2 $dev7 $dev8 ssm create --name $vol1 --pool $pool2 $dev9 $mnt4 check btrfs_fs_field $pool2 dev_count 6 check btrfs_vol_field $mnt2 subvolume $vol2 check btrfs_vol_field $mnt4 subvolume $vol1 check btrfs_vol_field $mnt2 vol_count 2 check btrfs_vol_field $mnt4 vol_count 2 not check btrfs_vol_field $mnt2 subvolume $vol1 ssm_output=$(ssm list vol) check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$vol1 $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1/$vol1 check list_table "$ssm_output" $pool1 $pool1 none btrfs none none btrfs $mnt2 check list_table "$ssm_output" $pool1:$vol2 $pool1 none btrfs none none btrfs $mnt2/$vol2 check list_table "$ssm_output" $pool1:$vol3 $pool1 none btrfs none none btrfs $mnt3 check list_table "$ssm_output" $pool2 $pool2 none btrfs none none btrfs check list_table "$ssm_output" $pool2:$vol1 $pool2 none btrfs none none btrfs $mnt4 check list_table "$ssm_output" $pool2:$vol2 $pool2 none btrfs none none btrfs ssm_output=$(ssm list pool) check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL btrfs 1 none none none check list_table "$ssm_output" $pool1 btrfs 2 none none none check list_table "$ssm_output" $pool2 btrfs 6 none none none umount_all ssm -f remove $SSM_BTRFS_DEFAULT_POOL $pool1 $pool2 # Create root mounted soubvolume and then another subvolume mounted at different mount point ssm create $TEST_DEVS $mnt1 ssm create --name $vol1 $mnt2 ssm create --name $vol1/$vol2 $mnt3 # Create subvolume with already existing path not ssm create --name $vol1/$vol2 not ssm create --name $vol1 $mnt3 ssm_output=$(ssm list vol) check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt1 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$vol1 $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt2 check list_table "$ssm_output" $SSM_BTRFS_DEFAULT_POOL:$vol1/$vol2 $SSM_BTRFS_DEFAULT_POOL none btrfs none none btrfs $mnt3 umount_all ssm -f remove $SSM_BTRFS_DEFAULT_POOL ssm create --help # Some cases which should fail not ssm create not ssm -f remove --all system-storage-manager-0.4/tests/bashtests/set.sh0000775000175000017500000000144612171200774023751 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . for i in $(ls lib/*.sh); do echo "$i -> ${i%%.sh}" cp $i ${i%%.sh} chmod +x ${i%%.sh} done system-storage-manager-0.4/tests/bashtests/004-lvm-resize.sh0000775000175000017500000001517312200411405025542 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='004-resize' test_description='Exercise ssm resize' . lib/test DEV_COUNT=10 DEV_SIZE=100 # The real size of the device which lvm will use is smaller TEST_MAX_SIZE=$(($DEV_COUNT*($DEV_SIZE-4))) aux prepare_devs $DEV_COUNT $DEV_SIZE TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='lvm' export SSM_LVM_DEFAULT_POOL=$vg1 export LVOL_PREFIX="lvol" export SSM_NONINTERACTIVE='1' lvol1=${LVOL_PREFIX}001 lvol2=${LVOL_PREFIX}002 lvol3=${LVOL_PREFIX}003 pool1=$vg2 pool2=$vg3 DEFAULT_VOLUME=${SSM_LVM_DEFAULT_POOL}/$lvol1 TEST_FS= which mkfs.ext2 && grep -E "^\sext[234]$" /proc/filesystems && TEST_FS+="ext2 " which mkfs.ext3 && grep -E "^\sext[34]$" /proc/filesystems && TEST_FS+="ext3 " which mkfs.ext4 && grep -E "^\sext4$" /proc/filesystems && TEST_FS+="ext4 " which mkfs.xfs && grep -E "^\sxfs$" /proc/filesystems && TEST_FS+="xfs" TEST_MNT=$TESTDIR/mnt _test_resize() { size=$((TEST_MAX_SIZE/2)) echo 'y' | ssm -f resize --size ${size}M ${DM_DEV_DIR}/$DEFAULT_VOLUME size=$(align_size_up $size) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size ${size}.00m # xfs does not support shrinking (xfs only grows big!! :)) if [ "$fs" != "xfs" ]; then ssm -f -v resize -s-$(($TEST_MAX_SIZE/4))M $DEFAULT_VOLUME size=$(align_size_up $(($size-($TEST_MAX_SIZE/4)))) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size ${size}.00m fi echo 'y' | ssm -f resize --size +$(($TEST_MAX_SIZE/5))M $DEFAULT_VOLUME size=$(align_size_up $(($size+($TEST_MAX_SIZE/5)))) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size ${size}.00m } ssm add $TEST_DEVS size=$((TEST_MAX_SIZE/3)) ssm create --size ${size}M $TEST_DEVS size=$(align_size_up $size) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size ${size}.00m export SSM_DEFAULT_BACKEND='btrfs' ssm -f resize --size +$(($TEST_MAX_SIZE/3))M ${DM_DEV_DIR}/$DEFAULT_VOLUME size=$(align_size_up $(($size+($TEST_MAX_SIZE/3)))) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size ${size}.00m export SSM_DEFAULT_BACKEND='lvm' ssm -f resize -s-$(($TEST_MAX_SIZE/2))M $DEFAULT_VOLUME size=$(align_size_up $(($size-($TEST_MAX_SIZE/2)))) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size ${size}.00m ssm -f resize --size $(($TEST_MAX_SIZE/2))M $DEFAULT_VOLUME size=$(align_size_up $(($TEST_MAX_SIZE/2))) check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 lv_size ${size}.00m ssm -f remove $SSM_LVM_DEFAULT_POOL ssm create $dev1 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 1 ssm resize --size +$((TEST_MAX_SIZE/2))M $DEFAULT_VOLUME $TEST_DEVS check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count $DEV_COUNT ssm -f remove $SSM_LVM_DEFAULT_POOL ssm create $dev1 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 1 ssm resize --size $((TEST_MAX_SIZE/2))M ${DM_DEV_DIR}/$DEFAULT_VOLUME $TEST_DEVS check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count $DEV_COUNT ssm -f remove $SSM_LVM_DEFAULT_POOL ssm create --size $((DEV_SIZE/2))M $dev1 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 1 ssm resize --size +$((DEV_SIZE/3))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 2 ssm -f resize -s-$((DEV_SIZE/3))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 $dev3 ssm resize --size +$((DEV_SIZE/3))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 $dev3 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 3 ssm -f resize -s-$((DEV_SIZE/3))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 $dev3 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 3 ssm resize --size +${DEV_SIZE}M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 $dev3 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 3 ssm -f remove $SSM_LVM_DEFAULT_POOL [ ! -d $TEST_MNT ] && mkdir $TEST_MNT &> /dev/null for fs in $TEST_FS; do # umounted test ssm add $TEST_DEVS size=$((TEST_MAX_SIZE/4)) ssm create --fs $fs --size ${size}M $TEST_DEVS _test_resize ssm -f check $DEFAULT_VOLUME ssm -f remove $SSM_LVM_DEFAULT_POOL echo $fs if [ $fs == 'ext2' ]; then continue fi # Disable this for now, since fsadm does not handle -f and -y correctly # mounted test #ssm add $TEST_DEVS size=$((TEST_MAX_SIZE/4)) #ssm create --fs $fs --size ${size}M $TEST_DEVS #mount ${DM_DEV_DIR}/$DEFAULT_VOLUME $TEST_MNT #_test_resize #umount $TEST_MNT #ssm -f check $DEFAULT_VOLUME #ssm -f remove $SSM_LVM_DEFAULT_POOL done not ssm -f remove --all ssm create $dev1 ssm resize -s +$((DEV_SIZE/2))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 ssm -f remove $SSM_LVM_DEFAULT_POOL # Use device with existing file system mkfs.ext3 $dev2 ssm create $dev1 not ssm resize -s +$((DEV_SIZE/2))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 ssm resize -s +$((DEV_SIZE/2))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 $dev3 ssm -f remove --all # Use device with existing file system mkfs.ext3 $dev2 ssm create $dev1 ssm -f resize -s +$((DEV_SIZE/2))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 2 ssm -f remove $SSM_LVM_DEFAULT_POOL mkfs.ext3 $dev2 ssm create $dev1 ssm -f resize -s +$((DEV_SIZE/2))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 $dev3 check lv_field $SSM_LVM_DEFAULT_POOL/$lvol1 pv_count 3 ssm -f remove --all # Use device already used in different pool ssm create $dev1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 1 ssm add -p $pool1 $dev2 $dev3 check vg_field $pool1 pv_count 2 not ssm resize -s +$((DEV_SIZE/2))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 ssm resize -s +$((DEV_SIZE/2))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev3 $dev4 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 ssm -f remove --all # Use device already used in different pool with force ssm create $dev1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 1 ssm add -p $pool1 $dev2 $dev3 check vg_field $pool1 pv_count 2 ssm -f resize -s +$((DEV_SIZE/2))M $SSM_LVM_DEFAULT_POOL/$lvol1 $dev2 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 ssm -f remove --all ssm resize --help # Some cases which should fail not ssm resize not ssm resize _garbage_ not ssm resize $dev1 ssm create $TEST_DEVS not ssm resize $DEFAULT_VOLUME not ssm -f resize --size +10G $DEFAULT_VOLUME not ssm -f resize -s-10G $DEFAULT_VOLUME system-storage-manager-0.4/tests/bashtests/001-lvm-add.sh0000775000175000017500000001423512200411405024764 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . export test_name='001-add' export test_description='Exercise ssm add' . lib/test export COLUMNS=1024 DEV_COUNT=10 aux prepare_devs $DEV_COUNT 10 TEST_DEVS=$(cat DEVICES) export SSM_DEFAULT_BACKEND='lvm' export SSM_LVM_DEFAULT_POOL=$vg1 export SSM_NONINTERACTIVE='1' pool1=$vg2 pool2=$vg3 ssm list dev # Create default pool with all devices at once ssm add $TEST_DEVS check vg_field $SSM_LVM_DEFAULT_POOL pv_count $DEV_COUNT ssm_output=$(ssm list dev) check list_table "$ssm_output" $dev1 none none 8.00MB $vg1 check list_table "$ssm_output" $dev2 none none 8.00MB $vg1 check list_table "$ssm_output" $dev3 none none 8.00MB $vg1 check list_table "$ssm_output" $dev4 none none 8.00MB $vg1 check list_table "$ssm_output" $dev5 none none 8.00MB $vg1 check list_table "$ssm_output" $dev6 none none 8.00MB $vg1 check list_table "$ssm_output" $dev7 none none 8.00MB $vg1 check list_table "$ssm_output" $dev8 none none 8.00MB $vg1 check list_table "$ssm_output" $dev9 none none 8.00MB $vg1 check list_table "$ssm_output" $dev10 none none 8.00MB $vg1 ssm -f remove $SSM_LVM_DEFAULT_POOL # Specify backend ssm -b lvm add $TEST_DEVS check vg_field $SSM_LVM_DEFAULT_POOL pv_count $DEV_COUNT ssm -f remove $SSM_LVM_DEFAULT_POOL export SSM_DEFAULT_BACKEND='btrfs' ssm --backend lvm add $TEST_DEVS check vg_field $SSM_LVM_DEFAULT_POOL pv_count $DEV_COUNT ssm -f remove $SSM_LVM_DEFAULT_POOL export SSM_DEFAULT_BACKEND='lvm' ssm add $TEST_DEVS check vg_field $SSM_LVM_DEFAULT_POOL pv_count $DEV_COUNT ssm remove $dev1 $dev2 $dev3 check vg_field $SSM_LVM_DEFAULT_POOL pv_count $(($DEV_COUNT-3)) ssm_output=$(ssm list dev) check list_table "$ssm_output" $dev1 9.90MB check list_table "$ssm_output" $dev2 9.90MB check list_table "$ssm_output" $dev3 9.90MB check list_table "$ssm_output" $dev4 none none 8.00MB $vg1 check list_table "$ssm_output" $dev5 none none 8.00MB $vg1 check list_table "$ssm_output" $dev6 none none 8.00MB $vg1 check list_table "$ssm_output" $dev7 none none 8.00MB $vg1 check list_table "$ssm_output" $dev8 none none 8.00MB $vg1 check list_table "$ssm_output" $dev9 none none 8.00MB $vg1 check list_table "$ssm_output" $dev10 none none 8.00MB $vg1 ssm remove $dev4 check vg_field $SSM_LVM_DEFAULT_POOL pv_count $(($DEV_COUNT-4)) ssm_output=$(ssm list dev) check list_table "$ssm_output" $dev1 9.90MB check list_table "$ssm_output" $dev2 9.90MB check list_table "$ssm_output" $dev3 9.90MB check list_table "$ssm_output" $dev4 9.90MB check list_table "$ssm_output" $dev5 none none 8.00MB $vg1 check list_table "$ssm_output" $dev6 none none 8.00MB $vg1 check list_table "$ssm_output" $dev7 none none 8.00MB $vg1 check list_table "$ssm_output" $dev8 none none 8.00MB $vg1 check list_table "$ssm_output" $dev9 none none 8.00MB $vg1 check list_table "$ssm_output" $dev10 none none 8.00MB $vg1 ssm -f remove $SSM_LVM_DEFAULT_POOL # Create default pool by adding devices one per a call for i in $TEST_DEVS; do ssm add $i done check vg_field $SSM_LVM_DEFAULT_POOL pv_count $DEV_COUNT ssm -f remove $SSM_LVM_DEFAULT_POOL # Create different groups from different devices ssm add $dev4 ssm add $dev1 $dev2 $dev3 -p $pool1 ssm add --pool $pool2 $dev7 $dev8 ssm add $dev5 $dev6 not ssm add $dev5 $dev6 $dev1 -p $pool1 ssm add $dev9 $dev1 -p $pool1 ssm add $dev10 -p $pool2 not ssm add $dev10 -p $pool1 not ssm add $dev10 $pool2 -p $pool1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 3 check vg_field $pool1 pv_count 4 check vg_field $pool2 pv_count 3 ssm_output=$(ssm list dev) check list_table "$ssm_output" $dev1 none none 8.00MB $vg2 check list_table "$ssm_output" $dev2 none none 8.00MB $vg2 check list_table "$ssm_output" $dev3 none none 8.00MB $vg2 check list_table "$ssm_output" $dev4 none none 8.00MB $vg1 check list_table "$ssm_output" $dev5 none none 8.00MB $vg1 check list_table "$ssm_output" $dev6 none none 8.00MB $vg1 check list_table "$ssm_output" $dev7 none none 8.00MB $vg3 check list_table "$ssm_output" $dev8 none none 8.00MB $vg3 check list_table "$ssm_output" $dev9 none none 8.00MB $vg2 check list_table "$ssm_output" $dev10 none none 8.00MB $vg3 ssm -f remove --all ssm add $dev1 $dev2 ssm -f remove $SSM_LVM_DEFAULT_POOL # Try to use device with existing file system mkfs.ext3 $dev1 # Default answer is No not ssm add $dev1 not check vg_field $SSM_LVM_DEFAULT_POOL pv_count 1 ssm add $dev1 $dev2 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 1 ssm -f remove --all # Try to use device with existing file system with force mkfs.ext3 $dev1 # Default answer is No ssm -f add $dev1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 1 ssm -f remove $SSM_LVM_DEFAULT_POOL mkfs.ext3 $dev1 ssm -f add $dev1 $dev2 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 ssm -f remove --all # Create pool with device already used in different pool ssm add $dev1 $dev2 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 # Fail because $dev1 is already used not ssm add -p $pool1 $dev1 not ssm add -p $pool1 $dev1 $dev2 ssm add -p $pool1 $dev1 $dev2 $dev3 check vg_field $pool1 pv_count 1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 ssm -f remove --all # Create pool with device already used in different pool with force ssm add $dev1 $dev2 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 2 ssm -f add -p $pool1 $dev1 check vg_field $pool1 pv_count 1 check vg_field $SSM_LVM_DEFAULT_POOL pv_count 1 ssm -f remove --all ssm add --help # Some cases which should fail not ssm _garbage_ not ssm add not ssm add $dev1 ${dev1}not_exist not check vg_field $SSM_LVM_DEFAULT_POOL vg_name $SSM_LVM_DEFAULT_POOL not ssm add _somepool not ssm add $dev1 $dev2 $dev3 -p $pool1 _otherpool system-storage-manager-0.4/tests/__init__.py0000664000175000017500000000136712171200774022732 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . __all__ = ["bashtests", "unittests"] system-storage-manager-0.4/tests/unittests/0000775000175000017500000000000012200423561022646 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/tests/unittests/test_btrfs.py0000664000175000017500000005021312200411405025372 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2012 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # Unittests for the system storage manager btrfs backend import unittest from ssmlib import main from ssmlib.backends import btrfs from tests.unittests.common import * class BtrfsFunctionCheck(MockSystemDataSource): def setUp(self): super(BtrfsFunctionCheck, self).setUp() self._addDevice('/dev/sda', 11489037516) self._addDevice('/dev/sdb', 234566451) self._addDevice('/dev/sdc', 2684354560) self._addDevice('/dev/sdc1', 894784853, 1) self._addDevice('/dev/sdc2', 29826161, 2) self._addDevice('/dev/sdc3', 1042177280, 3) self._addDevice('/dev/sdd', 11673) self._addDevice('/dev/sde', 1042177280) main.SSM_DEFAULT_BACKEND = 'btrfs' self.check_new_path_orig = btrfs.BtrfsPool._check_new_path btrfs.BtrfsPool._check_new_path = self.mock_check_new_path def tearDown(self): super(BtrfsFunctionCheck, self).tearDown() btrfs.BtrfsPool._check_new_path = self.check_new_path_orig def mock_check_new_path(self, path, name): pass def mock_run(self, cmd, *args, **kwargs): # Convert all parts of cmd into string for i, item in enumerate(cmd): if type(item) is not str: cmd[i] = str(item) self.run_data.append(" ".join(cmd)) output = "" if cmd[:3] == ['btrfs', 'filesystem', 'show']: for pool, p_data in self.pool_data.iteritems(): output += "Label: {0} uuid: some_random_uuid\n".format(pool) count = 0 d_output = "" for dev, d_data in sorted(self.dev_data.iteritems()): if 'pool_name' not in d_data or \ d_data['pool_name'] != pool: continue count += 1 d_output += " devid {0} size {1}.00K used {2} path {3}\n".format( count, d_data['dev_size'], d_data['dev_used'], dev) output += " Total devices {0} FS bytes used 44.00KB\n".format(count) output += d_output elif cmd[:3] == ['btrfs', 'subvolume', 'list']: mpoint = cmd[-1] for pool, p_data in self.pool_data.iteritems(): if 'mount' not in p_data or p_data['mount'] != mpoint: continue count = 0 for vol, v_data in iter(sorted(self.vol_data.iteritems())): if v_data['pool_name'] != pool: continue count += 1 output += "ID {0} top level 5 path {1}\n".format(count, v_data['dev_name']) if 'return_stdout' in kwargs and not kwargs['return_stdout']: output = None return (0, output) def test_btrfs_create(self): default_pool = btrfs.SSM_BTRFS_DEFAULT_POOL # Create volume using single device from non existent default pool self._checkCmd("ssm create", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) # Specify default backend self._checkCmd("ssm -b btrfs create", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) main.SSM_DEFAULT_BACKEND = 'lvm' self._checkCmd("ssm --backend btrfs create", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) main.SSM_DEFAULT_BACKEND = 'btrfs' self._checkCmd("ssm -f create", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) self._checkCmd("ssm -v create", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) self._checkCmd("ssm -f -v create", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) self._checkCmd("ssm create", ['-s 2.6T', '/dev/sda'], "mkfs.btrfs -L {0} -b 2858730232217 -f /dev/sda".format(default_pool)) self._checkCmd("ssm create", ['-r 0', '-s 2.6T', '/dev/sda'], "mkfs.btrfs -L btrfs_pool -m raid0 -d raid0 -b 2858730232217 -f /dev/sda".format(default_pool)) self._checkCmd("ssm create", ['-r 0', '-s 2.6T', '/dev/sda'], "mkfs.btrfs -L btrfs_pool -m raid0 -d raid0 -b 2858730232217 -f /dev/sda".format(default_pool)) self._checkCmd("ssm create", ['-r 1', '-s 512k', '/dev/sda'], "mkfs.btrfs -L btrfs_pool -m raid1 -d raid1 -b 524288 -f /dev/sda".format(default_pool)) self._checkCmd("ssm create", ['-r 10', '-s 10M', '/dev/sda'], "mkfs.btrfs -L btrfs_pool -m raid10 -d raid10 -b 10485760 -f /dev/sda".format(default_pool)) # Create volume using single device from non existent my_pool self._checkCmd("ssm create", ['--pool my_pool', '/dev/sda'], "mkfs.btrfs -L my_pool -f /dev/sda") self._checkCmd("ssm create", ['-p my_pool', '-r 0', '-s 2.6T', '/dev/sda'], "mkfs.btrfs -L my_pool -m raid0 -d raid0 -b 2858730232217 -f /dev/sda") # Create volume using multiple devices self._checkCmd("ssm create /dev/sda /dev/sdb", [], "mkfs.btrfs -L {0} -f /dev/sda /dev/sdb".format(default_pool)) # Create volume using single device from existing pool self._addPool(default_pool, ['/dev/sdb', '/dev/sdd']) self._addPool("my_pool", ['/dev/sdc2', '/dev/sdc3']) self._checkCmd("ssm create", ['-n myvolume'], "btrfs subvolume create /tmp/mount/myvolume") self._checkCmd("ssm create", ['-p my_pool', '-n myvolume'], "btrfs subvolume create /tmp/mount/myvolume") self._addVol('vol002', 1172832, 1, 'my_pool', ['/dev/sdc2'], '/mnt/test') self._checkCmd("ssm create", ['-p my_pool', '-n myvolume'], "btrfs subvolume create /tmp/mount/myvolume") # Create volume using multiple devices which one of the is in already # in the pool self._checkCmd("ssm create", ['-n myvolume', '/dev/sda /dev/sdb'], "btrfs subvolume create /tmp/mount/myvolume") self._cmdEq("btrfs device add /dev/sda /tmp/mount", -2) self._checkCmd("ssm create", ['-p my_pool', '-n myvolume', '/dev/sdc2 /dev/sda'], "btrfs subvolume create /tmp/mount/myvolume") self._cmdEq("btrfs device add /dev/sda /mnt/test", -2) self._checkCmd("ssm create", ['-n myvolume', '/dev/sda /dev/sdb /dev/sde'], "btrfs subvolume create /tmp/mount/myvolume") self._cmdEq("btrfs device add /dev/sda /dev/sde /tmp/mount", -2) def test_btrfs_remove(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) # remove volume self._checkCmd("ssm remove default_pool", [], "wipefs -p /dev/sda") self._cmdEq("wipefs -p /dev/sdb", -2) self._checkCmd("ssm remove my_pool", [], "wipefs -p /dev/sdc3") self._cmdEq("wipefs -p /dev/sdc1", -2) self._cmdEq("wipefs -p /dev/sdc2", -3) # remove subvolume self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda'], '/mnt/test') self._checkCmd("ssm remove default_pool:/dev/default_pool/vol001", [], "btrfs subvolume delete /mnt/test") # remove device self._checkCmd("ssm remove /dev/sdc1", [], "btrfs device delete /dev/sdc1 /tmp/mount") # remove multiple devices self._checkCmd("ssm remove /dev/sdc1 /dev/sdb", [], "btrfs device delete /dev/sdb /mnt/test") self._cmdEq("btrfs device delete /dev/sdc1 /tmp/mount", -2) # remove combination self._addPool('other_pool', ['/dev/sdd', '/dev/sde']) self._checkCmd("ssm remove /dev/sdd /dev/sdb other_pool my_pool default_pool:/dev/default_pool/vol001", [], "btrfs subvolume delete /mnt/test") self._cmdEq("wipefs -p /dev/sdc1", -2) self._cmdEq("wipefs -p /dev/sdc3", -3) self._cmdEq("wipefs -p /dev/sdc2", -4) self._cmdEq("wipefs -p /dev/sde", -5) self._cmdEq("wipefs -p /dev/sdd", -6) self._cmdEq("btrfs device delete /dev/sdb /mnt/test", -7) self._cmdEq("btrfs device delete /dev/sdd /tmp/mount", -8) self._removeMount("/dev/sda") # remove all self._checkCmd("ssm remove --all", [], "wipefs -p /dev/sde") self._cmdEq("wipefs -p /dev/sdd", -2) self._cmdEq("wipefs -p /dev/sdc1", -3) self._cmdEq("wipefs -p /dev/sdc3", -4) self._cmdEq("wipefs -p /dev/sdc2", -5) self._cmdEq("wipefs -p /dev/sda", -6) self._cmdEq("wipefs -p /dev/sdb", -7) # TODO # remove force # remove verbose # remove verbose + force def test_btrfs_snapshot(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol002', 237284225, 1, 'default_pool', ['/dev/sda'], '/mnt/mount1') self._addVol('vol003', 1024, 1, 'default_pool', ['/dev/sdd']) self._addVol('vol004', 209715200, 2, 'default_pool', ['/dev/sda', '/dev/sdb'], '/mnt/mount') # Create snapshot self._checkCmd("ssm snapshot --name new_snap", ['default_pool'], "btrfs subvolume snapshot /mnt/mount /mnt/mount/new_snap") main.SSM_DEFAULT_BACKEND = 'lvm' self._checkCmd("ssm snapshot --name new_snap", ['default_pool'], "btrfs subvolume snapshot /mnt/mount /mnt/mount/new_snap") main.SSM_DEFAULT_BACKEND = 'btrfs' self._checkCmd("ssm snapshot --name new_snap", ['default_pool:/dev/default_pool/vol001'], "btrfs subvolume snapshot /mnt/mount /mnt/mount/new_snap") self._checkCmd("ssm snapshot --name new_snap", ['my_pool'], "btrfs subvolume snapshot /tmp/mount /tmp/mount/new_snap") # Create snapshot verbose self._checkCmd("ssm -v snapshot --name new_snap", ['default_pool'], "btrfs subvolume snapshot /mnt/mount /mnt/mount/new_snap") def test_btrfs_resize(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3']) self._addVol('vol001', 2982616, 1, 'my_pool', ['/dev/sdc2'], '/mnt/test1') # Extend Volume self._checkCmd("ssm resize --size +4m", ['default_pool /dev/sde'], "btrfs filesystem resize 11723608063K /tmp/mount") # Specify backend self._checkCmd("ssm -b btrfs resize --size +4m", ['default_pool /dev/sde'], "btrfs filesystem resize 11723608063K /tmp/mount") main.SSM_DEFAULT_BACKEND = 'lvm' self._checkCmd("ssm resize --size +4m", ['default_pool /dev/sde'], "btrfs filesystem resize 11723608063K /tmp/mount") main.SSM_DEFAULT_BACKEND = 'btrfs' self._cmdEq("btrfs device add /dev/sde /tmp/mount", -2) self._checkCmd("ssm resize --size +1g", ['my_pool /dev/sde'], "btrfs filesystem resize 1073052017K /mnt/test1") self._cmdEq("btrfs device add /dev/sde /mnt/test1", -2) # Shrink volume self._checkCmd("ssm resize", ['-s-100G', 'default_pool'], "btrfs filesystem resize 11618746367K /tmp/mount") self._checkCmd("ssm resize -s-500G", ['my_pool /dev/sde'], "btrfs filesystem resize 547715441K /mnt/test1") self.assertNotEqual(self.run_data[-2], "btrfs device add /dev/sde /mnt/test1") # Set volume size self._checkCmd("ssm resize", ['-s 10M', 'default_pool'], "btrfs filesystem resize 10240K /tmp/mount") self._checkCmd("ssm resize", ['-s 10M', 'my_pool'], "btrfs filesystem resize 10240K /mnt/test1") # Set volume and add devices self._checkCmd("ssm resize -s 12T default_pool /dev/sdc1 /dev/sde", [], "btrfs filesystem resize 12884901888K /tmp/mount") self.assertEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /tmp/mount") self._checkCmd("ssm resize -s 1T my_pool /dev/sdc1 /dev/sde", [], "btrfs filesystem resize 1073741824K /mnt/test1") self.assertEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /mnt/test1") self._checkCmd("ssm resize -s 1T my_pool /dev/sde /dev/sdc2", [], "btrfs filesystem resize 1073741824K /mnt/test1") self.assertEqual(self.run_data[-2], "btrfs device add /dev/sde /mnt/test1") # Set volume in without the need adding more devices self._checkCmd("ssm resize -s 10G default_pool /dev/sdc1 /dev/sde", [], "btrfs filesystem resize 10485760K /tmp/mount") self.assertNotEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /tmp/mount") self._checkCmd("ssm resize -s 10G my_pool /dev/sdd /dev/sde", [], "btrfs filesystem resize 10485760K /mnt/test1") self.assertNotEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /mnt/test1") # Extend volume and add devices self._checkCmd("ssm resize -s +500G default_pool /dev/sdc1 /dev/sde", [], "btrfs filesystem resize 12247891967K /tmp/mount") self.assertEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /tmp/mount") # Extend volume in without the need adding more devices self._checkCmd("ssm resize -s 1k default_pool /dev/sdc1 /dev/sde", [], "btrfs filesystem resize 1K /tmp/mount") self.assertNotEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /tmp/mount") self.assertNotEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /tmp/mount") # Shrink volume with devices provided self._checkCmd("ssm resize -s-10G default_pool /dev/sdc1 /dev/sde", [], "btrfs filesystem resize 11713118207K /tmp/mount") self.assertNotEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /tmp/mount") self.assertNotEqual(self.run_data[-2], "btrfs device add /dev/sdc1 /dev/sde /tmp/mount") def test_btrfs_add(self): default_pool = btrfs.SSM_BTRFS_DEFAULT_POOL # Adding to non existent pool # Add device into default pool self._checkCmd("ssm add", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) # Specify backend self._checkCmd("ssm --backend btrfs add", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) main.SSM_DEFAULT_BACKEND = 'lvm' self._checkCmd("ssm -b btrfs add", ['/dev/sda'], "mkfs.btrfs -L {0} -f /dev/sda".format(default_pool)) main.SSM_DEFAULT_BACKEND = 'btrfs' # Add more devices into default pool self._checkCmd("ssm add", ['/dev/sda /dev/sdc1'], "mkfs.btrfs -L {0} -f /dev/sda /dev/sdc1".format(default_pool)) # Add device into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda'], "mkfs.btrfs -L my_pool -f /dev/sda") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda'], "mkfs.btrfs -L my_pool -f /dev/sda") # Add more devices into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda /dev/sdc1'], "mkfs.btrfs -L my_pool -f /dev/sda /dev/sdc1") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda /dev/sdc1'], "mkfs.btrfs -L my_pool -f /dev/sda /dev/sdc1") # Adding to existing default pool self._addPool(default_pool, ['/dev/sdb', '/dev/sdd']) # Add device into default pool self._checkCmd("ssm add", ['/dev/sda'], "btrfs device add /dev/sda /tmp/mount") # Add more devices into default pool self._checkCmd("ssm add", ['/dev/sda /dev/sdc1'], "btrfs device add /dev/sda /dev/sdc1 /tmp/mount") # Adding to existing defined pool self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3']) self._addVol('vol001', 2982616, 1, 'my_pool', ['/dev/sdc2'], '/mnt/test1') # Add device into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda'], "btrfs device add /dev/sda /mnt/test1") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda'], "btrfs device add /dev/sda /mnt/test1") # Add more devices into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda /dev/sdc1'], "btrfs device add /dev/sda /dev/sdc1 /mnt/test1") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda /dev/sdc1'], "btrfs device add /dev/sda /dev/sdc1 /mnt/test1") # Add verbose self._checkCmd("ssm -v add", ['--pool {0}'.format(default_pool), '/dev/sda /dev/sdc1'], "btrfs device add /dev/sda /dev/sdc1 /tmp/mount") # Add two devices into existing pool (one of the devices already is in # the pool self._checkCmd("ssm add", ['--pool my_pool', '/dev/sdc2 /dev/sda'], "btrfs device add /dev/sda /mnt/test1") self._checkCmd("ssm add", ['/dev/sda /dev/sdb'], "btrfs device add /dev/sda /tmp/mount") def test_btrfs_mount(self): self._addDir("/mnt/test") self._addDir("/mnt/test1") self._addDir("/mnt/test2") # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol002', 237284225, 1, 'default_pool', ['/dev/sda'], '/mnt/mount1') self._addVol('vol003/vol006', 1024, 1, 'default_pool', ['/dev/sdd']) self._addVol('vol004', 209715200, 2, 'default_pool', ['/dev/sda', '/dev/sdb'], '/mnt/mount') # Mount subvolume main.main("ssm mount default_pool:/dev/default_pool/vol002 /mnt/test") self._cmdEq("mount -o subvolid=2 /dev/sda /mnt/test") main.main("ssm mount -o discard default_pool:/dev/default_pool/vol002 /mnt/test") self._cmdEq("mount -o discard,subvolid=2 /dev/sda /mnt/test") main.main("ssm mount --options rw,discard,neco=44 default_pool:/dev/default_pool/vol004 /mnt/test1") self._cmdEq("mount -o rw,discard,neco=44,subvolid=4 /dev/sda /mnt/test1") main.main("ssm mount -o discard default_pool:/dev/default_pool/vol003/vol006 /mnt/test2") self._cmdEq("mount -o discard,subvolid=3 /dev/sda /mnt/test2") # Mount the whole file system main.main("ssm mount default_pool /mnt/test2") self._cmdEq("mount -o subvolid=0 /dev/sda /mnt/test2") main.main("ssm mount --options rw,discard,neco=44 default_pool /mnt/test1") self._cmdEq("mount -o rw,discard,neco=44,subvolid=0 /dev/sda /mnt/test1") main.main("ssm mount /dev/sdb /mnt/test1") self._cmdEq("mount /dev/sdb /mnt/test1") main.main("ssm mount -o rw,discard,neco=44 /dev/sdb /mnt/test1") self._cmdEq("mount -o rw,discard,neco=44 /dev/sdb /mnt/test1") # Non existing volume main.main("ssm mount nonexisting /mnt/test1") self._cmdEq("mount nonexisting /mnt/test1") main.main("ssm mount -o discard,rw nonexisting /mnt/test1") self._cmdEq("mount -o discard,rw nonexisting /mnt/test1") system-storage-manager-0.4/tests/unittests/test_lvm.py0000664000175000017500000005137712200411405025064 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # Unittests for the system storage manager lvm backend import unittest from ssmlib import main from ssmlib import problem from ssmlib.backends import lvm from tests.unittests.common import * class LvmFunctionCheck(MockSystemDataSource): def setUp(self): super(LvmFunctionCheck, self).setUp() self._addDevice('/dev/sda', 11489037516) self._addDevice('/dev/sdb', 234566451) self._addDevice('/dev/sdc', 2684354560) self._addDevice('/dev/sdc1', 894784853, 1) self._addDevice('/dev/sdc2', 29826161, 2) self._addDevice('/dev/sdc3', 1042177280, 3) self._addDevice('/dev/sdd', 11673) self._addDevice('/dev/sde', 1042177280) main.SSM_DEFAULT_BACKEND = 'lvm' def mock_run(self, cmd, *args, **kwargs): # Convert all parts of cmd into string for i, item in enumerate(cmd): if type(item) is not str: cmd[i] = str(item) self.run_data.append(" ".join(cmd)) output = "" if cmd[1] == 'pvs': for dev, data in self.dev_data.iteritems(): if 'pool_name' in data: output += "{0}|{1}|{2}|{3}\n".format(dev, data['pool_name'], data['dev_free'], data['dev_used']) elif cmd[1] == 'vgs': for pool, data in self.pool_data.iteritems(): output += "{0}|{1}|{2}|{3}|{4}\n".format(pool, data['dev_count'], data['pool_size'], data['pool_free'], data['vol_count']) elif cmd[1] == 'lvs': for vol, data in self.vol_data.iteritems(): output += "{0}|{1}|{2}|{3}|{4}|{5}|{6}\n".format(data['pool_name'], data['vol_size'], data['stripes'], data['stripesize'], data['type'], data['dev_name'].split("/")[-1], data['origin']) if 'return_stdout' in kwargs and not kwargs['return_stdout']: output = None return (0, output) def test_lvm_create(self): default_pool = lvm.SSM_LVM_DEFAULT_POOL # Create volume using single device from non existent default pool self._checkCmd("ssm create", ['/dev/sda'], "lvm lvcreate {0} -l 100%PVS -n lvol001 /dev/sda".format(default_pool)) self._cmdEq("lvm vgcreate {0} /dev/sda".format(default_pool), -2) # Specify backnend self._checkCmd("ssm -b lvm create", ['/dev/sda'], "lvm lvcreate {0} -l 100%PVS -n lvol001 /dev/sda".format(default_pool)) self._cmdEq("lvm vgcreate {0} /dev/sda".format(default_pool), -2) main.SSM_DEFAULT_BACKEND = "btrfs" self._checkCmd("ssm --backend lvm create", ['/dev/sda'], "lvm lvcreate {0} -l 100%PVS -n lvol001 /dev/sda".format(default_pool)) self._cmdEq("lvm vgcreate {0} /dev/sda".format(default_pool), -2) main.SSM_DEFAULT_BACKEND = "lvm" self._checkCmd("ssm create", ['--name myvolume', '--fstype ext4', '/dev/sda']) self._cmdEq("mkfs.ext4 /dev/{0}/myvolume".format(default_pool)) self._cmdEq("lvm lvcreate {0} -l 100%PVS -n myvolume /dev/sda".format(default_pool), -2) self._cmdEq("lvm vgcreate {0} /dev/sda".format(default_pool), -3) self._checkCmd("ssm -f create", ['--fstype ext4', '/dev/sda']) self._cmdEq("mkfs.ext4 -F /dev/{0}/lvol001".format(default_pool)) self._cmdEq("lvm lvcreate {0} -l 100%PVS -n lvol001 /dev/sda".format(default_pool), -2) self._cmdEq("lvm vgcreate -f {0} /dev/sda".format(default_pool), -3) self._checkCmd("ssm -v create", ['--name myvolume', '--fstype xfs', '/dev/sda']) self._cmdEq("mkfs.xfs /dev/{0}/myvolume".format(default_pool)) self._cmdEq("lvm lvcreate -v {0} -l 100%PVS -n myvolume /dev/sda".format(default_pool), -2) self._cmdEq("lvm vgcreate -v {0} /dev/sda".format(default_pool), -3) self._checkCmd("ssm -v -f create", ['--name myvolume', '--fstype xfs', '/dev/sda']) self._cmdEq("mkfs.xfs -f /dev/{0}/myvolume".format(default_pool)) self._cmdEq("lvm lvcreate -v {0} -l 100%PVS -n myvolume /dev/sda".format(default_pool), -2) self._cmdEq("lvm vgcreate -v -f {0} /dev/sda".format(default_pool), -3) self._checkCmd("ssm create", ['-s 2.6T', '/dev/sda'], "lvm lvcreate {0} -L 2791728742.40K -n lvol001 /dev/sda".format(default_pool)) self._cmdEq("lvm vgcreate {0} /dev/sda".format(default_pool), -2) self._checkCmd("ssm create", ['-r 0', '-s 2.6T', '-I 16', '/dev/sda'], "lvm lvcreate {0} -L 2791728742.40K -n lvol001 -I 16 -i 1 /dev/sda".format(default_pool)) self._cmdEq("lvm vgcreate {0} /dev/sda".format(default_pool), -2) self._checkCmd("ssm create", ['-r 0', '-s 2.6T', '-I 16', '/dev/sda'], "lvm lvcreate {0} -L 2791728742.40K -n lvol001 -I 16 -i 1 /dev/sda".format(default_pool)) self._cmdEq("lvm vgcreate {0} /dev/sda".format(default_pool), -2) # Number of stripes must not exceed number of devices self.assertRaises(problem.GeneralError, main.main, "ssm create -r 0 -s 2.6T -I 16 -i 4 /dev/sda") # Create volume using single device from non existent my_pool self._checkCmd("ssm create", ['--pool my_pool', '/dev/sda'], "lvm lvcreate my_pool -l 100%PVS -n lvol001 /dev/sda") self._cmdEq("lvm vgcreate my_pool /dev/sda", -2) self._checkCmd("ssm create", ['-r 0', '-p my_pool', '-s 2.6T', '-I 16', '-i 2', '/dev/sda /dev/sdb'], "lvm lvcreate my_pool -L 2791728742.40K -n lvol001 -I 16 -i 2 /dev/sda /dev/sdb") self._cmdEq("lvm vgcreate my_pool /dev/sda /dev/sdb", -2) # Create volume using multiple devices self._checkCmd("ssm create", ['/dev/sda /dev/sdc1'], "lvm lvcreate {0} -l 100%PVS -n lvol001 /dev/sda /dev/sdc1".format(default_pool)) self._cmdEq("lvm vgcreate {0} /dev/sda /dev/sdc1".format(default_pool), -2) # Create volume using single devices from existing pool self._addPool(default_pool, ['/dev/sdb', '/dev/sdd']) self._checkCmd("ssm create", ['-r 0', '-s 2.6T', '-I 16', '-n myvolume', '/dev/sda'], "lvm lvcreate {0} -L 2791728742.40K -n myvolume -I 16 -i 1 /dev/sda". format(default_pool)) self._cmdEq("lvm vgextend {0} /dev/sda".format(default_pool), -2) self._addPool("my_pool", ['/dev/sdc2', '/dev/sdc3']) self._checkCmd("ssm create", ['-r 0', '-p my_pool', '-s 2.6T', '-I 16', '-n myvolume', '/dev/sda'], "lvm lvcreate my_pool -L 2791728742.40K -n myvolume -I 16 -i 1 /dev/sda") self._cmdEq("lvm vgextend my_pool /dev/sda", -2) # Create volume using multiple devices which one of the is in already # in the pool self._checkCmd("ssm create", ['-n myvolume', '/dev/sda /dev/sdb'], "lvm lvcreate {0} -l 100%PVS -n myvolume /dev/sda /dev/sdb". format(default_pool)) self._cmdEq("lvm vgextend {0} /dev/sda".format(default_pool), -2) self._addPool("my_pool", ['/dev/sdc2', '/dev/sdc3']) self._checkCmd("ssm create", ['-p my_pool', '-n myvolume', '/dev/sdc2 /dev/sda'], "lvm lvcreate my_pool -l 100%PVS -n myvolume /dev/sdc2 /dev/sda") self._cmdEq("lvm vgextend my_pool /dev/sda", -2) self._checkCmd("ssm create", ['-n myvolume', '/dev/sda /dev/sdb /dev/sde'], "lvm lvcreate {0} -l 100%PVS -n myvolume /dev/sda /dev/sdb /dev/sde". format(default_pool)) self._cmdEq("lvm vgextend {0} /dev/sda /dev/sde".format(default_pool), -2) def test_lvm_remove(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol002', 237284225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol003', 1024, 1, 'default_pool', ['/dev/sdd']) self._addVol('vol004', 209715200, 2, 'default_pool', ['/dev/sda', '/dev/sdb']) # remove volume main.main("ssm remove /dev/default_pool/vol002") self._cmdEq("lvm lvremove /dev/default_pool/vol002") # remove multiple volumes main.main("ssm remove /dev/default_pool/vol002 /dev/default_pool/vol003") self.assertEqual(self.run_data[-2], "lvm lvremove /dev/default_pool/vol002") self._cmdEq("lvm lvremove /dev/default_pool/vol003") # remove pool main.main("ssm remove my_pool") self._cmdEq("lvm vgremove my_pool") # remove multiple pools main.main("ssm remove my_pool default_pool") self.assertEqual(self.run_data[-2], "lvm vgremove my_pool") self._cmdEq("lvm vgremove default_pool") # remove device main.main("ssm remove /dev/sdc1") self._cmdEq("lvm vgreduce my_pool /dev/sdc1") # remove multiple devices main.main("ssm remove /dev/sdc1 /dev/sdb") self.assertEqual(self.run_data[-2], "lvm vgreduce my_pool /dev/sdc1") self._cmdEq("lvm vgreduce default_pool /dev/sdb") # remove combination main.main("ssm remove /dev/sdb my_pool /dev/default_pool/vol001") self.assertEqual(self.run_data[-3], "lvm vgreduce default_pool /dev/sdb") self.assertEqual(self.run_data[-2], "lvm vgremove my_pool") self._cmdEq("lvm lvremove /dev/default_pool/vol001") # remove all main.main("ssm remove --all") self.assertEqual(self.run_data[-2], "lvm vgremove default_pool") self._cmdEq("lvm vgremove my_pool") # remove force main.main("ssm -f remove /dev/default_pool/vol002") self._cmdEq("lvm lvremove -f /dev/default_pool/vol002") # remove verbose main.main("ssm -v remove /dev/default_pool/vol002") self._cmdEq("lvm lvremove -v /dev/default_pool/vol002") # remove verbose + force main.main("ssm -v -f remove /dev/default_pool/vol002") self._cmdEq("lvm lvremove -v -f /dev/default_pool/vol002") def test_lvm_snapshot(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol002', 237284225, 1, 'default_pool', ['/dev/sda'], '/mnt/mount1') self._addVol('vol003', 1024, 1, 'default_pool', ['/dev/sdd']) self._addVol('vol004', 209715200, 2, 'default_pool', ['/dev/sda', '/dev/sdb'], '/mnt/mount') # Create snapshot self._checkCmd("ssm snapshot --name new_snap", ['/dev/default_pool/vol001'], "lvm lvcreate --size 23456645.0K --snapshot --name new_snap /dev/default_pool/vol001") main.SSM_DEFAULT_BACKEND = "btrfs" self._checkCmd("ssm snapshot --name new_snap", ['/dev/default_pool/vol001'], "lvm lvcreate --size 23456645.0K --snapshot --name new_snap /dev/default_pool/vol001") main.SSM_DEFAULT_BACKEND = "lvm" # Create snapshot verbose self._checkCmd("ssm -v snapshot --name new_snap", ['/dev/default_pool/vol001'], "lvm lvcreate -v --size 23456645.0K --snapshot --name new_snap /dev/default_pool/vol001") # Create snapshot force self._checkCmd("ssm -f snapshot --name new_snap", ['/dev/default_pool/vol001'], "lvm lvcreate -f --size 23456645.0K --snapshot --name new_snap /dev/default_pool/vol001") # Create snapshot force verbose self._checkCmd("ssm -f -v snapshot --name new_snap", ['/dev/default_pool/vol001'], "lvm lvcreate -v -f --size 23456645.0K --snapshot --name new_snap /dev/default_pool/vol001") # Create snapshot with size and name specified self._checkCmd("ssm snapshot", ['--size 1G', '--name new_snap', '/dev/default_pool/vol001'], "lvm lvcreate --size 1048576.0K --snapshot --name new_snap /dev/default_pool/vol001") def test_lvm_resize(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3']) self._addVol('vol001', 2982616, 1, 'my_pool', ['/dev/sdc2'], '/mnt/test1') self._addVol('vol002', 237284225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol003', 1024, 1, 'default_pool', ['/dev/sdd']) self._addDevice('/dev/sde', 11489037516) # Extend Volume self._checkCmd("ssm resize", ['--size +4m', '/dev/default_pool/vol003'], "lvm lvresize -L 5120.0k /dev/default_pool/vol003") # Specify backend self._checkCmd("ssm --backend lvm resize", ['--size +4m', '/dev/default_pool/vol003'], "lvm lvresize -L 5120.0k /dev/default_pool/vol003") main.SSM_DEFAULT_BACKEND = "btrfs" self._checkCmd("ssm resize", ['--size +4m', '/dev/default_pool/vol003'], "lvm lvresize -L 5120.0k /dev/default_pool/vol003") main.SSM_DEFAULT_BACKEND = "lvm" # Shrink volume self._checkCmd("ssm resize", ['-s-100G', '/dev/default_pool/vol002'], "lvm lvresize -L 132426625.0k /dev/default_pool/vol002") # Set volume size self._checkCmd("ssm resize", ['-s 10M', '/dev/my_pool/vol001'], "lvm lvresize -L 10240.0k /dev/my_pool/vol001") # Set volume and add devices self._checkCmd("ssm resize -s 12T /dev/default_pool/vol003 /dev/sdc1 /dev/sde", [], "lvm lvresize -L 12884901888.0k /dev/default_pool/vol003") self.assertEqual(self.run_data[-2], "lvm vgextend default_pool /dev/sdc1 /dev/sde") # Set volume size self._checkCmd("ssm resize -s 10G /dev/default_pool/vol003 /dev/sdc1 /dev/sde", [], "lvm lvresize -L 10485760.0k /dev/default_pool/vol003") self.assertEqual(self.run_data[-2], "lvm vgextend default_pool /dev/sdc1 /dev/sde") # Extend volume and add devices self._checkCmd("ssm resize -s +11T /dev/default_pool/vol003 /dev/sdc1 /dev/sde", [], "lvm lvresize -L 11811161088.0k /dev/default_pool/vol003") self.assertEqual(self.run_data[-2], "lvm vgextend default_pool /dev/sdc1 /dev/sde") # Extend volume self._checkCmd("ssm resize -s +10G /dev/default_pool/vol003 /dev/sdc1 /dev/sde", [], "lvm lvresize -L 10486784.0k /dev/default_pool/vol003") self.assertEqual(self.run_data[-2], "lvm vgextend default_pool /dev/sdc1 /dev/sde") # Shrink volume with devices provided self._checkCmd("ssm resize -s-10G /dev/default_pool/vol002 /dev/sdc1 /dev/sde", [], "lvm lvresize -L 226798465.0k /dev/default_pool/vol002") self.assertNotEqual(self.run_data[-2], "lvm vgextend default_pool /dev/sdc1 /dev/sde") # Test that we do not use devices which are already used in different # pool self.assertRaises(Exception, main.main, "ssm resize -s +1.5T /dev/my_pool/vol001 /dev/sdb /dev/sda") # If the device we are able to use can cover the size, then # it will be resized successfully self._checkCmd("ssm resize -s +1.5T /dev/my_pool/vol001 /dev/sdb /dev/sda /dev/sdc1", [], "lvm lvresize -L 1613595352.0k /dev/my_pool/vol001") def test_lvm_add(self): default_pool = lvm.SSM_LVM_DEFAULT_POOL # Adding to non existent pool # Add device into default pool self._checkCmd("ssm add", ['/dev/sda'], "lvm vgcreate {0} /dev/sda".format(default_pool)) # Specify backend self._checkCmd("ssm --backend lvm add", ['/dev/sda'], "lvm vgcreate {0} /dev/sda".format(default_pool)) main.SSM_DEFAULT_BACKEND = "btrfs" self._checkCmd("ssm --backend lvm add", ['/dev/sda'], "lvm vgcreate {0} /dev/sda".format(default_pool)) main.SSM_DEFAULT_BACKEND = "lvm" # Add more devices into default pool self._checkCmd("ssm add", ['/dev/sda /dev/sdc1'], "lvm vgcreate {0} /dev/sda /dev/sdc1".format(default_pool)) # Add device into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda'], "lvm vgcreate my_pool /dev/sda") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda'], "lvm vgcreate my_pool /dev/sda") # Add more devices into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda /dev/sdc1'], "lvm vgcreate my_pool /dev/sda /dev/sdc1") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda /dev/sdc1'], "lvm vgcreate my_pool /dev/sda /dev/sdc1") # Add force self._checkCmd("ssm -f add", ['--pool my_pool', '/dev/sda'], "lvm vgcreate -f my_pool /dev/sda") # Add verbose self._checkCmd("ssm -v add", ['--pool my_pool', '/dev/sda'], "lvm vgcreate -v my_pool /dev/sda") # Add force and verbose self._checkCmd("ssm -v -f add", ['--pool my_pool', '/dev/sda'], "lvm vgcreate -v -f my_pool /dev/sda") # Adding to existing default pool self._addPool(default_pool, ['/dev/sdb', '/dev/sdd']) # Add device into default pool self._checkCmd("ssm add", ['/dev/sda'], "lvm vgextend {0} /dev/sda".format(default_pool)) # Add more devices into default pool self._checkCmd("ssm add", ['/dev/sda /dev/sdc1'], "lvm vgextend {0} /dev/sda /dev/sdc1".format(default_pool)) # Adding to existing defined pool self._addPool("my_pool", ['/dev/sdc2', '/dev/sdc3']) # Add device into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda'], "lvm vgextend my_pool /dev/sda") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda'], "lvm vgextend my_pool /dev/sda") # Add more devices into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda /dev/sdc1'], "lvm vgextend my_pool /dev/sda /dev/sdc1") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda /dev/sdc1'], "lvm vgextend my_pool /dev/sda /dev/sdc1") # Add force self._checkCmd("ssm -f add", ['--pool my_pool', '/dev/sda'], "lvm vgextend -f my_pool /dev/sda") # Add verbose self._checkCmd("ssm -v add", ['--pool my_pool', '/dev/sda'], "lvm vgextend -v my_pool /dev/sda") # Add force and verbose self._checkCmd("ssm -v -f add", ['--pool my_pool', '/dev/sda'], "lvm vgextend -v -f my_pool /dev/sda") # Add two devices into existing pool (one of the devices already is in # the pool self._checkCmd("ssm add", ['--pool my_pool', '/dev/sdc2 /dev/sda'], "lvm vgextend my_pool /dev/sda") self._checkCmd("ssm add", ['/dev/sda /dev/sdb'], "lvm vgextend {0} /dev/sda".format(default_pool)) def test_lvm_mount(self): self._addDir("/mnt/test") self._addDir("/mnt/test1") self._addDir("/mnt/test2") # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda'], '/mnt/test1') self._addVol('vol002', 237284225, 1, 'my_pool', ['/dev/sda']) # Simple mount test main.main("ssm mount /dev/default_pool/vol001 /mnt/test") self._cmdEq("mount /dev/default_pool/vol001 /mnt/test") main.main("ssm mount -o discard /dev/default_pool/vol001 /mnt/test") self._cmdEq("mount -o discard /dev/default_pool/vol001 /mnt/test") main.main("ssm mount --options rw,discard,neco=44 /dev/my_pool/vol002 /mnt/test1") self._cmdEq("mount -o rw,discard,neco=44 /dev/my_pool/vol002 /mnt/test1") # Non existing volume main.main("ssm mount nonexisting /mnt/test1") self._cmdEq("mount nonexisting /mnt/test1") main.main("ssm mount -o discard,rw nonexisting /mnt/test1") self._cmdEq("mount -o discard,rw nonexisting /mnt/test1") system-storage-manager-0.4/tests/unittests/test_ssm.py0000664000175000017500000011062712200411405025062 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # Unittests for the system storage manager import os import re import sys import stat import time import doctest import unittest import argparse from ssmlib import main from ssmlib import misc from ssmlib import problem from tests.unittests.common import * class SimpleStorageHandleSanityCheck(BaseStorageHandleInit): """ Simple sanity ckecks for StorageHandle class and some of its methods. """ def test_constructor(self): # Check initial variables self.assertFalse(self.storage.options.force) self.assertFalse(self.storage.options.verbose) self.assertFalse(self.storage.options.yes) self.assertFalse(self.storage.options.interactive) self.assertFalse(self.storage.options.debug) self.assert_(self.storage.options.config is None) self.assert_(self.storage._mpoint is None) self.assert_(self.storage._dev is None) self.assert_(self.storage._pool is None) self.assert_(self.storage._volumes is None) def test_create_fs(self): options = main.Options() for fs in main.EXTN: options.force = True options.verbose = True options.yes = False options.config = "my_new_config" options.interactive = False self.storage.set_globals(options) self.storage._create_fs(fs, "/dev/foo/bar") self.assertEqual('mkfs.{0} -v -F /dev/foo/bar'.format(fs), self.run_data[-1]) options.force = False options.verbose = False self.storage.set_globals(options) self.storage._create_fs(fs, "/dev/foo/bar") self.assertEqual('mkfs.{0} /dev/foo/bar'.format(fs), self.run_data[-1]) options.force = True self.storage.set_globals(options) self.storage._create_fs(fs, "/dev/foo/bar") self.assertEqual('mkfs.{0} -F /dev/foo/bar'.format(fs), self.run_data[-1]) options.force = False options.verbose = True self.storage.set_globals(options) self.storage._create_fs(fs, "/dev/foo/bar") self.assertEqual('mkfs.{0} -v /dev/foo/bar'.format(fs), self.run_data[-1]) options.force = True options.verbose = True self.storage.set_globals(options) self.storage._create_fs("xfs", "/dev/foo/bar") self.assertEqual('mkfs.xfs -f /dev/foo/bar', self.run_data[-1]) options.force = False options.verbose = False self.storage.set_globals(options) self.storage._create_fs("xfs", "/dev/foo/bar") self.assertEqual('mkfs.xfs /dev/foo/bar', self.run_data[-1]) options.force = True self.storage.set_globals(options) self.storage._create_fs("xfs", "/dev/foo/bar") self.assertEqual('mkfs.xfs -f /dev/foo/bar', self.run_data[-1]) options.force = False options.verbose = True self.storage.set_globals(options) self.storage._create_fs("xfs", "/dev/foo/bar") self.assertEqual('mkfs.xfs /dev/foo/bar', self.run_data[-1]) class StorageHandleSanityCheck(BaseStorageHandleInit): """ Sanity checks for StorageHandle class and other classes used via StorageHandle class. """ def setUp(self): super(StorageHandleSanityCheck, self).setUp() options = main.Options() options.force = True options.verbose = True options.yes = True options.config = "my_config" options.interactive = True options.debug = True self.storage.set_globals(options) self.dev = self.storage.dev self.vol = self.storage.vol self.pool = self.storage.pool def tearDown(self): super(StorageHandleSanityCheck, self).tearDown() self.dev = None self.vol = None self.pool = None def test_storage_constructor(self): # Check initial variables self.assertTrue(self.storage.options.force) self.assertTrue(self.storage.options.verbose) self.assertTrue(self.storage.options.yes) self.assertTrue(self.storage.options.interactive) self.assertTrue(self.storage.options.debug) self.assertEqual(self.storage.options.config, "my_config") # Check if we have right instances self.assert_(isinstance(self.dev, main.Devices)) self.assert_(isinstance(self.vol, main.Volumes)) self.assert_(isinstance(self.pool, main.Pool)) # Check initial variables for source in self.dev, self.vol, self.pool: self.assertTrue(source.options.force) self.assertTrue(source.options.verbose) self.assertTrue(source.options.yes) self.assertTrue(source.options.interactive) self.assertTrue(source.options.debug) self.assertEqual(source.options.config, "my_config") self.assert_(isinstance(source._data, dict)) self.assert_(len(source._data) > 0) self.assert_(isinstance(source.header, list)) self.assert_(len(source.header) > 0) self.assert_(isinstance(source.attrs, list)) self.assert_(len(source.attrs) > 0) self.assert_(isinstance(source.types, list)) self.assert_(len(source.types) > 0) for item in source._data.itervalues(): self.assertTrue(item.options.force) self.assertTrue(item.options.verbose) self.assertTrue(item.options.yes) self.assertTrue(item.options.interactive) self.assertTrue(item.options.debug) self.assertEqual(item.options.config, "my_config") def test_set_globals_propagation(self): options = main.Options() options.force = False options.verbose = False options.yes = False options.config = "my_config" options.interactive = False options.debug = False self.storage.set_globals(options) # Check initial variables self.assertFalse(self.storage.options.force) self.assertFalse(self.storage.options.verbose) self.assertFalse(self.storage.options.yes) self.assertEqual(self.storage.options.config, "my_config") self.assertFalse(self.storage.options.interactive) self.assertFalse(self.storage.options.debug) # Check initial variables for source in self.dev, self.vol, self.pool: self.assertFalse(source.options.force) self.assertFalse(source.options.verbose) self.assertFalse(source.options.yes) self.assertFalse(source.options.interactive) self.assertFalse(source.options.debug) self.assertEqual(source.options.config, "my_config") self.assert_(isinstance(source._data, dict)) self.assert_(len(source._data) > 0) self.assert_(isinstance(source.header, list)) self.assert_(len(source.header) > 0) self.assert_(isinstance(source.attrs, list)) self.assert_(len(source.attrs) > 0) self.assert_(isinstance(source.types, list)) self.assert_(len(source.types) > 0) for item in source._data.itervalues(): self.assertFalse(item.options.force) self.assertFalse(item.options.verbose) self.assertFalse(item.options.yes) self.assertFalse(item.options.interactive) self.assertFalse(item.options.debug) self.assertEqual(item.options.config, "my_config") def test_backend_generic_methods(self): for source in self.dev, self.vol, self.pool: obj = dir(source) self.assert_("__iter__" in obj) self.assert_("__contains__" in obj) self.assert_("__getitem__" in obj) self.assert_("filesystems" in obj) self.assert_("ptable" in obj) self.assert_("set_globals" in obj) # Variables self.assert_("_data" in obj) self.assert_("header" in obj) self.assert_("attrs" in obj) self.assert_("types" in obj) for bknd in source._data.itervalues(): obj = dir(bknd) self.assert_("__iter__" in obj) self.assert_("__getitem__" in obj) self.assert_("__init__" in obj) # Variables self.assert_("type" in obj) self.assert_("options" in obj) # DeviceInfo does not need default_pool_name if bknd.type != "device": self.assert_("default_pool_name" in obj) def test_volumes_specific_methods(self): for bknd in self.vol._data.itervalues(): obj = dir(bknd) self.assert_("remove" in obj) def test_pool_specific_methods(self): for bknd in self.pool._data.itervalues(): obj = dir(bknd) self.assert_("reduce" in obj) self.assert_("remove" in obj) self.assert_("create" in obj) self.assert_("new" in obj) self.assert_("extend" in obj) class SimpleSsmSanityCheck(unittest.TestCase): """ Simple sanity check of ssm module. """ def test_existing_objects(self): obj = dir(main) self.assert_("StorageHandle" in obj) self.assert_("FsInfo" in obj) self.assert_("DeviceInfo" in obj) self.assert_("SUPPORTED_FS" in obj) self.assert_("DEFAULT_DEVICE_POOL" in obj) self.assert_("Storage" in obj) self.assert_("Pool" in obj) self.assert_("Devices" in obj) self.assert_("Volumes" in obj) def test_fsinfo_methods(self): obj = dir(main.FsInfo) self.assert_("extN_get_info" in obj) self.assert_("extN_fsck" in obj) for fs in set(main.SUPPORTED_FS) - set(main.EXTN): if fs == 'btrfs': continue self.assert_("{0}_get_info".format(fs) in obj) self.assert_("{0}_fsck".format(fs) in obj) def test_storage_handle_methods(self): obj = dir(main.StorageHandle) self.assert_("dev" in obj) self.assert_("pool" in obj) self.assert_("vol" in obj) self.assert_("check" in obj) self.assert_("resize" in obj) self.assert_("create" in obj) self.assert_("list" in obj) self.assert_("add" in obj) self.assert_("remove" in obj) self.assert_("snapshot" in obj) self.assert_("mount" in obj) self.assert_("set_globals" in obj) class SsmFunctionCheck(MockSystemDataSource): """ Prepare data for ssm function check. """ def setUp(self): super(SsmFunctionCheck, self).setUp() self.volumes_orig = main.Volumes main.Volumes = Volumes self.pool_orig = main.Pool main.Pool = Pool self.devices_orig = main.Devices main.Devices = Devices self._addDevice('/dev/sda', 11489037516) self._addDevice('/dev/sdb', 234566451) self._addDevice('/dev/sdc', 2684354560) self._addDevice('/dev/sdc1', 894784853, 1) self._addDevice('/dev/sdc2', 29826161, 2) self._addDevice('/dev/sdc3', 1042177280, 3) self._addDevice('/dev/sdd', 11673) def tearDown(self): super(SsmFunctionCheck, self).tearDown() main.Volumes = self.volumes_orig main.Pool = self.pool_orig main.Devices = self.devices_orig def mock_run(self, cmd, *args, **kwargs): for i,arg in enumerate(cmd): if arg is None: cmd[i] = '' elif type(arg) is not str: cmd[i] = str(arg) self.run_data.append(re.sub('\s\s+',' '," ".join(cmd).strip())) output = "" if cmd[0] == 'pooldata': return self.pool_data elif cmd[0] == 'volumedata': return self.vol_data elif cmd[0] == 'devdata': return self.dev_data if 'return_stdout' in kwargs and not kwargs['return_stdout']: output = None return (0, output) def test_resize(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3']) self._addVol('vol001', 2982616, 1, 'my_pool', ['/dev/sdc2'], '/mnt/test1') self._addVol('vol002', 237284225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol003', 1024, 1, 'default_pool', ['/dev/sdd']) self._addDevice('/dev/sde', 11489037516) # Extend Volume self._checkCmd("ssm resize", ['--size +4m', '/dev/default_pool/vol003'], "vol resize /dev/default_pool/vol003 5120.0 False") # Shrink volume self._checkCmd("ssm resize", ['-s-100G', '/dev/default_pool/vol002'], "vol resize /dev/default_pool/vol002 132426625.0 False") # Set volume size self._checkCmd("ssm resize", ['-s 10M', '/dev/my_pool/vol001'], "vol resize /dev/my_pool/vol001 10240.0 False") # Set volume and add devices self._checkCmd("ssm resize -s 12T /dev/default_pool/vol003 /dev/sdc1 /dev/sde", [], "vol resize /dev/default_pool/vol003 12884901888.0 False") self.assertEqual(self.run_data[-2], "pool extend default_pool /dev/sdc1 /dev/sde") # Set volume size self._checkCmd("ssm resize -s 10G /dev/default_pool/vol003 /dev/sdc1 /dev/sde", [], "vol resize /dev/default_pool/vol003 10485760.0 False") self.assertEqual(self.run_data[-2], "pool extend default_pool /dev/sdc1 /dev/sde") # Extend volume size with adding more devices self._checkCmd("ssm resize -s +10G /dev/default_pool/vol003 /dev/sdc1 /dev/sde", [], "vol resize /dev/default_pool/vol003 10486784.0 False") self.assertEqual(self.run_data[-2], "pool extend default_pool /dev/sdc1 /dev/sde") # Shrink volume with devices provided self._checkCmd("ssm resize -s-10G /dev/default_pool/vol002 /dev/sdc1 /dev/sde", [], "vol resize /dev/default_pool/vol002 226798465.0 False") self.assertNotEqual(self.run_data[-2], "pool extend default_pool /dev/sdc1 /dev/sde") self.assertNotEqual(self.run_data[-2], "pool extend default_pool /dev/sdc1") # Test that we do not use devices which are already used in different # pool self.assertRaises(Exception, main.main, "ssm resize -s +1.5T /dev/my_pool/vol001 /dev/sdb /dev/sda") # If the device we are able to use can cover the size, then # it will be resized successfully self._checkCmd("ssm resize -s +1.5T /dev/my_pool/vol001 /dev/sdb /dev/sda /dev/sdc1", [], "vol resize /dev/my_pool/vol001 1613595352.0 False") def test_create(self): # Create volume using single device from non existent default pool self._checkCmd("ssm create", ['/dev/sda'], "pool create {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) self._checkCmd("ssm create", ['--name myvolume', '--fstype ext4', '/dev/sda']) self._cmdEq("mkfs.ext4 /dev/{0}/myvolume".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("pool create {0} myvolume /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) self._cmdEq("pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -3) self._checkCmd("ssm -f create", ['--fstype ext4', '/dev/sda']) self._cmdEq("mkfs.ext4 -F /dev/{0}/lvol001".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("force pool create {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) self._cmdEq("force pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -3) self._checkCmd("ssm -v create", ['--name myvolume', '--fstype xfs', '/dev/sda']) self._cmdEq("mkfs.xfs /dev/{0}/myvolume".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("verbose pool create {0} myvolume /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) self._cmdEq("verbose pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -3) self._checkCmd("ssm -v -f create", ['--name myvolume', '--fstype xfs', '/dev/sda']) self._cmdEq("mkfs.xfs -f /dev/{0}/myvolume".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("force verbose pool create {0} myvolume /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) self._cmdEq("force verbose pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -3) self._checkCmd("ssm create", ['-s 2.6T', '/dev/sda'], "pool create {0} 2791728742.40 /dev/sda".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) self._checkCmd("ssm create", ['-r 0', '-s 2.6T', '-I 16', '/dev/sda'], "pool create {0} 2791728742.40 0 16 /dev/sda".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) # Number of stripes must not exceed number of devices self.assertRaises(problem.GeneralError, main.main, "ssm create -r 1 -s 2.6T -I 16 -i 4 /dev/sda") self._checkCmd("ssm create", ['-r 1', '-s 2.6T', '-I 16', '/dev/sda'], "pool create {0} 2791728742.40 1 16 /dev/sda".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) # Create volume using single device from non existent my_pool self._checkCmd("ssm create", ['--pool my_pool', '/dev/sda'], "pool create my_pool /dev/sda") self._cmdEq("pool new my_pool /dev/sda", -2) self._checkCmd("ssm create", ['--pool my_pool', '-s 2.6T', '/dev/sda'], "pool create my_pool 2791728742.40 /dev/sda") self._cmdEq("pool new my_pool /dev/sda", -2) self._checkCmd("ssm create", ['-r 10', '-p my_pool', '-s 2.6T', '-I 16', '/dev/sda'], "pool create my_pool 2791728742.40 10 16 /dev/sda") self._cmdEq("pool new my_pool /dev/sda", -2) self._checkCmd("ssm create", ['-r 0', '-p my_pool', '-s 2.6T', '-I 16', '/dev/sda'], "pool create my_pool 2791728742.40 0 16 /dev/sda") self._cmdEq("pool new my_pool /dev/sda", -2) # Create volume using multiple devices self._checkCmd("ssm create", ['/dev/sda /dev/sdc1'], "pool create {0} /dev/sda /dev/sdc1".format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("pool new {0} /dev/sda /dev/sdc1".format(main.DEFAULT_DEVICE_POOL), -2) self._checkCmd("ssm create", ['--pool my_pool', '/dev/sda /dev/sdc1'], "pool create my_pool /dev/sda /dev/sdc1") self._cmdEq("pool new my_pool /dev/sda /dev/sdc1", -2) # Create volume using single device from existing pool self._addPool(main.DEFAULT_DEVICE_POOL, ['/dev/sdb', '/dev/sdd']) self._checkCmd("ssm create", ['-r 10', '-s 2.6T', '-I 16', '-n myvolume', '/dev/sda'], "pool create {0} 2791728742.40 myvolume 10 16 /dev/sda". format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("pool extend {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) self._addPool("my_pool", ['/dev/sdc2', '/dev/sdc3']) self._checkCmd("ssm create", ['-r 1', '-p my_pool', '-s 2.6T', '-I 16', '-n myvolume', '/dev/sda'], "pool create my_pool 2791728742.40 myvolume 1 16 /dev/sda") self._cmdEq("pool extend my_pool /dev/sda", -2) # Create volume using multiple devices which one of the is in already # in the pool self._checkCmd("ssm create", ['-r 0', '-s 2.6T', '-I 16', '-i 2', '-n myvolume', '/dev/sda /dev/sdb'], "pool create {0} 2791728742.40 myvolume 0 2 16 /dev/sda /dev/sdb". format(main.DEFAULT_DEVICE_POOL)) self._cmdEq("pool extend {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL), -2) self._addPool("my_pool", ['/dev/sdc2', '/dev/sdc3']) self._checkCmd("ssm create", ['-r 10', '-p my_pool', '-s 2.6T', '-I 16', '-i 2', '-n myvolume', '/dev/sdc2 /dev/sda'], "pool create my_pool 2791728742.40 myvolume 10 2 16 /dev/sdc2 /dev/sda") self._cmdEq("pool extend my_pool /dev/sda", -2) # Test that we do not use devices which are already used in different # pool self.assertRaises(Exception, main.main, "ssm create -p new_pool /dev/sdc2 /dev/sdc3") self.assertRaises(Exception, main.main, "ssm create -p new_pool /dev/sdc2 /dev/sdc3 /dev/sda") # If the device we are able to use can cover the size, then # it will be created self._checkCmd("ssm create", ['-s 100M', '-p new_pool', '/dev/sdc2 /dev/sdc3 /dev/sda'], "pool create new_pool 102400.00 /dev/sda") #sys.stdout = out.stdout def test_add(self): # Adding to non existent pool # Add device into default pool self._checkCmd("ssm add", ['/dev/sda'], "pool new {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL)) # Add more devices into default pool self._checkCmd("ssm add", ['/dev/sda /dev/sdc1'], "pool new {0} /dev/sda /dev/sdc1".format(main.DEFAULT_DEVICE_POOL)) # Add device into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda'], "pool new my_pool /dev/sda") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda'], "pool new my_pool /dev/sda") # Add more devices into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda /dev/sdc1'], "pool new my_pool /dev/sda /dev/sdc1") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda /dev/sdc1'], "pool new my_pool /dev/sda /dev/sdc1") # Add force self._checkCmd("ssm -f add", ['--pool my_pool', '/dev/sda'], "force pool new my_pool /dev/sda") # Add verbose self._checkCmd("ssm -v add", ['--pool my_pool', '/dev/sda'], "verbose pool new my_pool /dev/sda") # Add force and verbose self._checkCmd("ssm -v -f add", ['--pool my_pool', '/dev/sda'], "force verbose pool new my_pool /dev/sda") # Adding to existing default pool self._addPool(main.DEFAULT_DEVICE_POOL, ['/dev/sdb', '/dev/sdd']) # Add device into default pool self._checkCmd("ssm add", ['/dev/sda'], "pool extend {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL)) # Add more devices into default pool self._checkCmd("ssm add", ['/dev/sda /dev/sdc1'], "pool extend {0} /dev/sda /dev/sdc1".format(main.DEFAULT_DEVICE_POOL)) # Adding to existing defined pool self._addPool("my_pool", ['/dev/sdc2', '/dev/sdc3']) # Add device into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda'], "pool extend my_pool /dev/sda") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda'], "pool extend my_pool /dev/sda") # Add more devices into defined pool self._checkCmd("ssm add", ['-p my_pool', '/dev/sda /dev/sdc1'], "pool extend my_pool /dev/sda /dev/sdc1") self._checkCmd("ssm add", ['--pool my_pool', '/dev/sda /dev/sdc1'], "pool extend my_pool /dev/sda /dev/sdc1") # Add force self._checkCmd("ssm -f add", ['--pool my_pool', '/dev/sda'], "force pool extend my_pool /dev/sda") # Add verbose self._checkCmd("ssm -v add", ['--pool my_pool', '/dev/sda'], "verbose pool extend my_pool /dev/sda") # Add force and verbose self._checkCmd("ssm -v -f add", ['--pool my_pool', '/dev/sda'], "force verbose pool extend my_pool /dev/sda") # Add two devices into existing pool (one of the devices already is in # the pool self._checkCmd("ssm add", ['--pool my_pool', '/dev/sdc2 /dev/sda'], "pool extend my_pool /dev/sda") self._checkCmd("ssm add", ['/dev/sda /dev/sdb'], "pool extend {0} /dev/sda".format(main.DEFAULT_DEVICE_POOL)) def test_remove(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol002', 237284225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol003', 1024, 1, 'default_pool', ['/dev/sdd']) self._addVol('vol004', 209715200, 2, 'default_pool', ['/dev/sda', '/dev/sdb']) # remove volume main.main("ssm remove /dev/default_pool/vol002") self._cmdEq("vol remove /dev/default_pool/vol002") # remove multiple volumes main.main("ssm remove /dev/default_pool/vol002 /dev/default_pool/vol003") self._cmdEq("vol remove /dev/default_pool/vol002", -2) self._cmdEq("vol remove /dev/default_pool/vol003") # remove pool main.main("ssm remove my_pool") self._cmdEq("pool remove my_pool") # remove multiple pools main.main("ssm remove my_pool default_pool") self._cmdEq("pool remove my_pool", -2) self._cmdEq("pool remove default_pool") # remove device main.main("ssm remove /dev/sdc1") self._cmdEq("pool reduce my_pool /dev/sdc1") # remove multiple devices main.main("ssm remove /dev/sdc1 /dev/sdb") self._cmdEq("pool reduce my_pool /dev/sdc1", -2) self._cmdEq("pool reduce default_pool /dev/sdb") # remove combination main.main("ssm remove /dev/sdb my_pool /dev/default_pool/vol001") self._cmdEq("pool reduce default_pool /dev/sdb", -3) self._cmdEq("pool remove my_pool", -2) self._cmdEq("vol remove /dev/default_pool/vol001") # remove all main.main("ssm remove --all") self._cmdEq("pool remove default_pool", -2) self._cmdEq("pool remove my_pool") # remove force main.main("ssm -f remove /dev/default_pool/vol002") self._cmdEq("force vol remove /dev/default_pool/vol002") # remove verbose main.main("ssm -v remove /dev/default_pool/vol002") self._cmdEq("verbose vol remove /dev/default_pool/vol002") # remove verbose + force main.main("ssm -v -f remove /dev/default_pool/vol002") self._cmdEq("force verbose vol remove /dev/default_pool/vol002") def test_snapshot(self): # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda'], '/mnt/test1') self._addVol('vol002', 237284225, 1, 'default_pool', ['/dev/sda']) self._addVol('vol003', 1024, 1, 'default_pool', ['/dev/sdd']) self._addVol('vol004', 209715200, 2, 'default_pool', ['/dev/sda', '/dev/sdb'], '/mnt/test') # Create snapshot self._checkCmd("ssm snapshot", ['/dev/default_pool/vol004'], "vol snapshot /dev/default_pool/vol004 41943040.0 False") # Create snapshot verbose self._checkCmd("ssm -v snapshot", ['/dev/default_pool/vol004'], "verbose vol snapshot /dev/default_pool/vol004 41943040.0 False") # Create snapshot force self._checkCmd("ssm -f snapshot", ['/dev/default_pool/vol004'], "force vol snapshot /dev/default_pool/vol004 41943040.0 False") # Create snapshot force verbose self._checkCmd("ssm -f -v snapshot", ['/dev/default_pool/vol004'], "force verbose vol snapshot /dev/default_pool/vol004 41943040.0 False") # Create snapshot with size specified self._checkCmd("ssm snapshot --size 1G", ['/dev/default_pool/vol004'], "vol snapshot /dev/default_pool/vol004 1048576.0 True") # Create snapshot with destination specified self._checkCmd("ssm snapshot --dest /mnt/test", ['/dev/default_pool/vol004'], "vol snapshot /dev/default_pool/vol004 /mnt/test 41943040.0 False") # Create snapshot with the name specified self._checkCmd("ssm snapshot --name test", ['/dev/default_pool/vol004'], "vol snapshot /dev/default_pool/vol004 test 41943040.0 False") # Create snapshot with both destination and size specified self._checkCmd("ssm snapshot", ['--size 1G', '--dest /mnt/test' ,'/dev/default_pool/vol004'], "vol snapshot /dev/default_pool/vol004 /mnt/test 1048576.0 True") # Create snapshot with both name and size specified self._checkCmd("ssm snapshot", ['--size 1G', '--name test' ,'/dev/default_pool/vol004'], "vol snapshot /dev/default_pool/vol004 test 1048576.0 True") # Repeat the test with specifying mount point instead of volume # Create snapshot self._checkCmd("ssm snapshot", ['/mnt/test'], "vol snapshot /dev/default_pool/vol004 41943040.0 False") # Create snapshot with size specified self._checkCmd("ssm snapshot --size 1G", ['/mnt/test'], "vol snapshot /dev/default_pool/vol004 1048576.0 True") # Create snapshot with destination specified self._checkCmd("ssm snapshot --dest /mnt/test", ['/mnt/test'], "vol snapshot /dev/default_pool/vol004 /mnt/test 41943040.0 False") # Create snapshot with the name specified self._checkCmd("ssm snapshot --name test", ['/mnt/test'], "vol snapshot /dev/default_pool/vol004 test 41943040.0 False") # Create snapshot with both destination and size specified self._checkCmd("ssm snapshot", ['--size 1G', '--dest /mnt/test' ,'/mnt/test'], "vol snapshot /dev/default_pool/vol004 /mnt/test 1048576.0 True") # Create snapshot with both name and size specified self._checkCmd("ssm snapshot", ['--size 1G', '--name test' ,'/mnt/test'], "vol snapshot /dev/default_pool/vol004 test 1048576.0 True") def test_mount(self): self._addDir("/mnt/test") self._addDir("/mnt/test1") self._addDir("/mnt/test2") # Generate some storage data self._addPool('default_pool', ['/dev/sda', '/dev/sdb']) self._addPool('my_pool', ['/dev/sdc2', '/dev/sdc3', '/dev/sdc1']) self._addVol('vol001', 117283225, 1, 'default_pool', ['/dev/sda'], '/mnt/test1') self._addVol('vol002', 237284225, 1, 'my_pool', ['/dev/sda']) # Simple mount test main.main("ssm mount /dev/default_pool/vol001 /mnt/test") self._cmdEq("mount /dev/default_pool/vol001 /mnt/test") main.main("ssm mount -o discard /dev/default_pool/vol001 /mnt/test") self._cmdEq("mount -o discard /dev/default_pool/vol001 /mnt/test") main.main("ssm mount --options rw,discard,neco=44 /dev/my_pool/vol002 /mnt/test1") self._cmdEq("mount -o rw,discard,neco=44 /dev/my_pool/vol002 /mnt/test1") # Non existing volume main.main("ssm mount nonexisting /mnt/test1") self._cmdEq("mount nonexisting /mnt/test1") main.main("ssm mount -o discard,rw nonexisting /mnt/test1") self._cmdEq("mount -o discard,rw nonexisting /mnt/test1") class MyInfo(object): def __init__(self, options, data=None): self.data = data or {} self.options = options self.type = 'backend' @property def y(self): return 'yes' if self.options.yes else '' @property def f(self): return 'force' if self.options.force else '' @property def v(self): return 'verbose' if self.options.verbose else '' def __iter__(self): for item in sorted(self.data.iterkeys()): yield item def __getitem__(self, key): if key in self.data.iterkeys(): return self.data[key] class PoolInfo(MyInfo): def __init__(self, *args, **kwargs): super(PoolInfo, self).__init__(*args, **kwargs) self.exist = False self.data.update(misc.run(['pooldata'])) def reduce(self, pool, devices): if type(devices) is not list: devices = [devices] cmd = [self.f, self.v, self.y, 'pool reduce', pool] cmd.extend(devices) misc.run(cmd) def remove(self, pool): misc.run([self.f, self.v, self.y, 'pool remove', pool]) def new(self, pool, devices): if type(devices) is not list: devices = [devices] cmd = [self.f, self.v, self.y, 'pool new', pool] cmd.extend(devices) misc.run(cmd) def extend(self, pool, devices): if type(devices) is not list: devices = [devices] cmd = [self.f, self.v, self.y, 'pool extend', pool] cmd.extend(devices) misc.run(cmd) def create(self, pool, size='', name='', devs='', raid=None): if type(devs) is not list: devices = [devs] if raid: stripes = raid['stripes'] stripesize = raid['stripesize'] level = raid['level'] else: stripes = stripesize = level = "" misc.run([self.f, self.v, self.y, 'pool create', pool, size, name, level, stripes, stripesize, " ".join(devs)]) if not name: name = "lvol001" return "/dev/{0}/{1}".format(pool, name) class VolumeInfo(MyInfo): def __init__(self, *args, **kwargs): super(VolumeInfo, self).__init__(*args, **kwargs) self.data.update(misc.run(['volumedata'])) def remove(self, volume): misc.run([self.f, self.v, self.y, 'vol remove', volume]) def resize(self, lv, size, resize_fs=True): misc.run([self.f, self.v, self.y, 'vol resize', lv, str(size), str(resize_fs)]) def snapshot(self, volume, destination, name, size, user_set_size): misc.run([self.f, self.v, self.y, 'vol snapshot', volume, destination, name, str(size), str(user_set_size)]) class DevInfo(MyInfo): def __init__(self, *args, **kwargs): super(DevInfo, self).__init__(*args, **kwargs) #if len(misc.run(['devdata']) > 0): self.data.update(misc.run(['devdata'])) def remove(self, devices): if type(devices) is not list: devices = [devices] misc.run([self.f, self.v, self.y, 'dev remove', devices]) cmd.extend(devices) misc.run(cmd) class Pool(main.Storage): def __init__(self, *args, **kwargs): super(Pool, self).__init__(*args, **kwargs) _default_backend = PoolInfo(options=self.options) self._data = {'test': _default_backend} self.default = main.Item(_default_backend, main.DEFAULT_DEVICE_POOL) self.header = ['Pool', 'Devices', 'Free', 'Used', 'Total'] self.attrs = ['pool_name', 'dev_count', 'pool_free', 'pool_used', 'pool_size'] self.types = [str, str, float, float, float] class Volumes(main.Storage): def __init__(self, *args, **kwargs): super(Volumes, self).__init__(*args, **kwargs) self._data = {'test': VolumeInfo(options=self.options)} self.header = ['Volume', 'Volume size', 'FS', 'Free', 'Used', 'FS size', 'Type', 'Mount point'] self.attrs = ['dev_name', 'dev_size', 'fs_type', 'fs_free', 'fs_used', 'fs_size', 'type', 'mount'] self.types = [str, float, str, float, float, float, str, str] class Devices(main.Storage): def __init__(self, *args, **kwargs): super(Devices, self).__init__(*args, **kwargs) self._data = {'dev': main.DeviceInfo(options=self.options, data=DevInfo(options=self.options).data)} self.header = ['Device', 'Free', 'Used', 'Total', 'Pool', 'Mount point'] self.attrs = ['dev_name', 'dev_free', 'dev_used', 'dev_size', 'pool_name', 'mount'] self.types = [str, float, float, float, str, str] system-storage-manager-0.4/tests/unittests/__init__.py0000664000175000017500000000140312171200774024763 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . __all__ = ["test_ssm", "test_lvm", "test_btrfs"] system-storage-manager-0.4/tests/unittests/common.py0000664000175000017500000002211012200411405024476 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # Common classes for unit testing import os import sys import unittest import argparse from ssmlib import main from ssmlib import misc class MyStdout(object): def __init__(self): self.output = "" self.stdout = sys.stdout def write(self, s): self.output += s class BaseStorageHandleInit(unittest.TestCase): """ Initialize StorageHandle class and some mock functions. """ def setUp(self): self.storage = main.StorageHandle() self.run_data = [] self.run_orig = misc.run misc.run = self.mock_run main.SSM_NONINTERACTIVE = True def mock_run(self, cmd, *args, **kwargs): # Convert all parts of cmd into string for i, item in enumerate(cmd): if type(item) is not str: cmd[i] = str(item) self.run_data.append(" ".join(cmd)) output = "" if 'return_stdout' in kwargs and not kwargs['return_stdout']: output = None return (0, output) def tearDown(self): self.storage = None self.run_data = [] misc.run = self.run_orig main.SSM_NONINTERACTIVE = False class MockSystemDataSource(unittest.TestCase): def setUp(self): self.directories = [] self.run_data = [] self.run_orig = misc.run misc.run = self.mock_run self.get_partitions_orig = misc.get_partitions misc.get_partitions = self.mock_get_partitions self.is_bdevice_orig = main.is_bdevice main.is_bdevice = self.mock_is_bdevice self.is_directory_orig = main.is_directory main.is_directory = self.mock_is_directory self.check_create_item_orig = main.StorageHandle.check_create_item main.StorageHandle.check_create_item = self.mock_check_create_item self.get_mounts_orig = misc.get_mounts misc.get_mounts = self.mock_get_mounts self.temp_mount_orig = misc.temp_mount misc.temp_mount = self.mock_temp_mount self.check_binary_orig = misc.check_binary misc.check_binary = self.mock_check_binary self.send_udev_event_orig = misc.send_udev_event misc.send_udev_event = self.mock_send_udev_event self.dev_data = {} self.vol_data = {} self.pool_data = {} self.mount_data = {} self._mpoint = False main.SSM_NONINTERACTIVE = True def tearDown(self): self.directories = [] self.run_data = [] self.dev_data = {} self.pool_data = {} self.vol_data = {} self.mount_data = {} misc.run = self.run_orig misc.get_partitions = self.get_partitions_orig main.is_bdevice = self.is_bdevice_orig main.is_directory = self.is_directory_orig misc.get_mounts = self.get_mounts_orig main.StorageHandle.check_create_item = self.check_create_item_orig misc.temp_mount = self.temp_mount_orig misc.check_binary = self.check_binary_orig misc.send_udev_event = self.send_udev_event_orig main.SSM_NONINTERACTIVE = False def _cmdEq(self, out, index=-1): self.assertEqual(self.run_data[index], out) def _checkCmd(self, command, args, expected=None): self.run_data = [] for case in misc.permutations(args): cmd = command + " " + " ".join(case) main.main(cmd) if expected: self._cmdEq(expected) def mock_run(self, cmd, *args, **kwargs): self.run_data.append(" ".join(cmd)) output = "" if 'return_stdout' in kwargs and not kwargs['return_stdout']: output = None return (0, output) def mock_check_binary(self, name): return True def mock_temp_mount(self, device, options=None): return "/tmp/mount" def mock_get_partitions(self): partitions = [] for name, data in self.dev_data.iteritems(): partitions.append([data['major'], data['minor'], data['dev_size'], data['dev_name'].rpartition("/")[2]]) return partitions def mock_is_directory(self, string): if string in self.directories: return string else: err = "'{0}' does not exist.".format(string) raise argparse.ArgumentTypeError(err) def mock_get_mounts(self, regex=None): return self.mount_data def mock_is_bdevice(self, path): if path in self.dev_data: return path else: err = "'{0}' is not valid block device".format(path) raise argparse.ArgumentTypeError(err) def mock_check_create_item(self, path): if not self._mpoint: if path in self.directories: self._mpoint = path return return main.is_bdevice(path) def mock_send_udev_event(self, device, event): pass def _removeMount(self, device): del self.mount_data[device] def _addDir(self, dirname): self.directories.append(dirname) def _addDevice(self, dev_name, dev_size, minor=0): self.dev_data[dev_name] = {'dev_name': dev_name, 'dev_size': dev_size, 'major': '8', 'minor': str(minor)} def _addPool(self, pool_name, devices): if pool_name in self.pool_data: pool_data = self.pool_data[pool_name] pool_size = float(pool_data['pool_size']) pool_free = float(pool_data['pool_free']) pool_used = float(pool_data['pool_used']) dev_count = int(pool_data['dev_count']) else: pool_size = pool_free = pool_used = 0.0 dev_count = 0 for dev in devices: dev_data = self.dev_data[dev] pool_size += float(dev_data['dev_size']) pool_free += float(dev_data['dev_size']) dev_data['pool_name'] = str(pool_name) dev_data['dev_free'] = dev_data['dev_size'] dev_data['dev_used'] = '0.0' dev_count += len(devices) self.pool_data[pool_name] = {'pool_name': pool_name, 'pool_size': str(pool_size), 'pool_used': str(pool_used), 'dev_count': str(dev_count), 'pool_free': str(pool_size), 'vol_count': '0'} def _addVol(self, vol_name, vol_size, stripes, pool_name, devices, mount=None): pool_data = self.pool_data[pool_name] pool_free = float(pool_data['pool_free']) - vol_size pool_used = float(pool_data['pool_used']) + vol_size if mount: self.pool_data[pool_name]['mount'] = mount self._addDir(mount) self.mount_data[devices[0]] = {'dev': devices[0], 'mp': mount, 'root': "/"} space_per_dev = vol_size / stripes size = vol_size for dev in devices: dev_data = self.dev_data[dev] if 'dev_free' not in dev_data: self._addPool(pool_name, [dev]) if stripes > 1: dev_data['pool_name'] = pool_name dev_data['dev_used'] = str(float(dev_data['dev_used']) + space_per_dev) dev_data['dev_free'] = str(float(dev_data['dev_free']) - space_per_dev) size -= space_per_dev else: dev_data['pool_name'] = pool_name if size > float(dev_data['dev_size']): use = float(dev_data['dev_size']) else: use = size dev_data['dev_used'] = str(float(dev_data['dev_used']) + space_per_dev) dev_data['dev_free'] = str(float(dev_data['dev_free']) - space_per_dev) size -= use if size > 1: raise Exception("Error in the test in _addVol") if stripes > 1: vol_type = "striped" stripesize = 32 else: vol_type = "linear" stripesize = 0 vol_name = "/dev/{0}/{1}".format(pool_name, vol_name) self.vol_data[vol_name] = {'dm_name': vol_name, 'real_dev': vol_name, 'stripes': stripes, 'dev_name': vol_name, 'stripesize': 0, 'pool_name': pool_name, 'vol_size': vol_size, 'dev_size': vol_size, 'type': vol_type, 'origin': "", 'mount': mount} system-storage-manager-0.4/doc/0000775000175000017500000000000012200423561020207 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/doc/_build/0000775000175000017500000000000012200423561021445 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/doc/_build/man/0000775000175000017500000000000012200423561022220 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/doc/_build/man/ssm.80000664000175000017500000006117712200423505023125 0ustar lczernerlczerner00000000000000.\" Man page generated from reStructuredText. . .TH "SSM" "8" "August 07, 2013" "0.4" "System Storage Manager" .SH NAME ssm \- System Storage Manager: a single tool to manage your storage . .nr rst2man-indent-level 0 . .de1 rstReportMargin \\$1 \\n[an-margin] level \\n[rst2man-indent-level] level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] - \\n[rst2man-indent0] \\n[rst2man-indent1] \\n[rst2man-indent2] .. .de1 INDENT .\" .rstReportMargin pre: . RS \\$1 . nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] . nr rst2man-indent-level +1 .\" .rstReportMargin post: .. .de UNINDENT . RE .\" indent \\n[an-margin] .\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] .nr rst2man-indent-level -1 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] .in \\n[rst2man-indent\\n[rst2man-indent-level]]u .. . .nr rst2man-indent-level 0 . .de1 rstReportMargin \\$1 \\n[an-margin] level \\n[rst2man-indent-level] level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] - \\n[rst2man-indent0] \\n[rst2man-indent1] \\n[rst2man-indent2] .. .de1 INDENT .\" .rstReportMargin pre: . RS \\$1 . nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] . nr rst2man-indent-level +1 .\" .rstReportMargin post: .. .de UNINDENT . RE .\" indent \\n[an-margin] .\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] .nr rst2man-indent-level -1 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] .in \\n[rst2man-indent\\n[rst2man-indent-level]]u .. .SH SYNOPSIS .sp \fBssm\fP [\fB\-h\fP] [\fB\-\-version\fP] [\fB\-v\fP] [\fB\-f\fP] [\fB\-b\fP BACKEND] {check,resize,create,list,add,remove,snapshot,mount} ... .sp \fBssm\fP \fBcreate\fP [\fB\-h\fP] [\fB\-s\fP SIZE] [\fB\-n\fP NAME] [\fB\-\-fstype\fP FSTYPE] [\fB\-r\fP LEVEL] [\fB\-I\fP STRIPESIZE] [\fB\-i\fP STRIPES] [\fB\-p\fP POOL] [\fBdevice\fP [\fBdevice\fP ...]] [mount] .sp \fBssm\fP \fBlist\fP [\fB\-h\fP] [{volumes,vol,dev,devices,pool,pools,fs,filesystems,snap,snapshots}] .sp \fBssm\fP \fBremove\fP [\fB\-h\fP] [\fB\-a\fP] [\fBitems\fP [\fBitems\fP ...]] .sp \fBssm\fP \fBresize\fP [\fB\-h\fP] [\fB\-s\fP SIZE] \fBvolume\fP [\fBdevice\fP [\fBdevice\fP ...]] .sp \fBssm\fP \fBcheck\fP [\fB\-h\fP] \fBdevice\fP [\fBdevice\fP ...] .sp \fBssm\fP \fBsnapshot\fP [\fB\-h\fP] [\fB\-s\fP SIZE] [\fB\-d\fP DEST | \fB\-n\fP NAME] volume .sp \fBssm\fP \fBadd\fP [\fB\-h\fP] [\fB\-p\fP POOL] \fBdevice\fP [\fBdevice\fP ...] .sp \fBssm\fP \fBmount\fP [\fB\-h\fP] [\fB\-o\fP OPTIONS] \fBvolume\fP directory .SH DESCRIPTION .sp System Storage Manager provides easy to use command line interface to manage your storage using various technologies like lvm, btrfs, encrypted volumes and more. .sp In more sophisticated enterprise storage environments, management with Device Mapper (dm), Logical Volume Manager (LVM), or Multiple Devices (md) is becoming increasingly more difficult. With file systems added to the mix, the number of tools needed to configure and manage storage has grown so large that it is simply not user friendly. With so many options for a system administrator to consider, the opportunity for errors and problems is large. .sp The btrfs administration tools have shown us that storage management can be simplified, and we are working to bring that ease of use to Linux filesystems in general. .SH OPTIONS .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .TP .B \-\-version show program\(aqs version number and exit .TP .B \-v\fP,\fB \-\-verbose Show aditional information while executing. .TP .B \-f\fP,\fB \-\-force Force execution in the case where ssm has some doubts or questions. .TP .BI \-b \ BACKEND\fP,\fB \ \-\-backend \ BACKEND Choose backend to use. Currently you can choose from (lvm,btrfs). .UNINDENT .SH SYSTEM STORAGE MANAGER COMMANDS .SS Introduction .sp System Storage Manager have several commands you can specify on the command line as a first argument to the ssm. They all have specific use and its own arguments, but global ssm arguments are propagated to all commands. .SS Create command .sp \fBssm\fP \fBcreate\fP [\fB\-h\fP] [\fB\-s\fP SIZE] [\fB\-n\fP NAME] [\fB\-\-fstype\fP FSTYPE] [\fB\-r\fP LEVEL] [\fB\-I\fP STRIPESIZE] [\fB\-i\fP STRIPES] [\fB\-p\fP POOL] [\fBdevice\fP [\fBdevice\fP ...]] [mount] .sp This command creates a new volume with defined parameters. If \fBdevice\fP is provided it will be used to create a volume, hence it will be added into the \fBpool\fP prior the volume creation (See \fIAdd command section\fP). More devices can be used to create a volume. .sp If the \fBdevice\fP is already used in the different pool, then \fBssm\fP will ask you whether you want to remove it from the original pool. If you decline, or the removal fails, then the \fBvolume\fP creation fails if the \fISIZE\fP was not provided. On the other hand, if the \fISIZE\fP is provided and some devices can not be added to the \fBpool\fP the volume creation might succeed if there is enough space in the \fBpool\fP\&. .sp \fIPOOL\fP name can be specified as well. If the pool exists new volume will be created from that pool (optionally adding \fBdevice\fP into the pool). However if the \fIPOOL\fP does not exist \fBssm\fP will attempt to create a new pool with provided \fBdevice\fP and then create a new volume from this pool. If \fB\-\-backend\fP argument is omitted, the default \fBssm\fP backend will be used. Default backend is \fIlvm\fP\&. .sp \fBssm\fP also supports creating RAID configuration, however some back\-ends might not support all the levels, or it might not support RAID at all. In this case, volume creation will fail. .sp If \fBmount\fP point is provided \fBssm\fP will attempt to mount the volume after it is created. However it will fail if mountable file system is not present on the volume. .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .TP .BI \-s \ SIZE\fP,\fB \ \-\-size \ SIZE Gives the size to allocate for the new logical volume A size suffix K|k, M|m, G|g, T|t, P|p, E|e can be used to define \(aqpower of two\(aq units. If no unit is provided, it defaults to kilobytes. This is optional if if not given maximum possible size will be used. .TP .BI \-n \ NAME\fP,\fB \ \-\-name \ NAME The name for the new logical volume. This is optional and if omitted, name will be generated by the corresponding backend. .TP .BI \-\-fstype \ FSTYPE Gives the file system type to create on the new logical volume. Supported file systems are (ext3, ext4, xfs, btrfs). This is optional and if not given file system will not be created. .TP .BI \-r \ LEVEL\fP,\fB \ \-\-raid \ LEVEL Specify a RAID level you want to use when creating a new volume. Note that some backends might not implement all supported RAID levels. This is optional and if no specified, linear volume will be created. You can choose from the following list of supported levels (0,1,10). .TP .BI \-I \ STRIPESIZE\fP,\fB \ \-\-stripesize \ STRIPESIZE Gives the number of kilobytes for the granularity of stripes. This is optional and if not given, backend default will be used. Note that you have to specify RAID level as well. .TP .BI \-i \ STRIPES\fP,\fB \ \-\-stripes \ STRIPES Gives the number of stripes. This is equal to the number of physical volumes to scatter the logical volume. This is optional and if stripesize is set and multiple devices are provided stripes is determined automatically from the number of devices. Note that you have to specify RAID level as well. .TP .BI \-p \ POOL\fP,\fB \ \-\-pool \ POOL Pool to use to create the new volume. .UNINDENT .SS List command .sp \fBssm\fP \fBlist\fP [\fB\-h\fP] [{volumes,vol,dev,devices,pool,pools,fs,filesystems,snap,snapshots}] .sp List informations about all detected devices, pools, volumes and snapshots found in the system. \fBlist\fP command can be used either alone to list all the information, or you can request specific section only. .sp Following sections can be specified: .INDENT 0.0 .TP .B {volumes | vol} List information about all \fBvolumes\fP found in the system. .TP .B {devices | dev} List information about all \fBdevices\fP found in the system. Some devices are intentionally hidden, like for example cdrom, or DM/MD devices since those are actually listed as volumes. .TP .B {pools | pool} List information about all \fBpools\fP found in the system. .TP .B {filesystems | fs} List information about all volumes containing \fBfilesystems\fP found in the system. .TP .B {snapshots | snap} List information about all \fBsnapshots\fP found in the system. Note that some back\-ends does not support snapshotting and some can not distinguish between snapshot and regular volume. in this case \fBssm\fP will try to recognize volume name in order to identify \fBsnapshot\fP, but if the \fBssm\fP regular expression does not match the snapshot pattern, this snapshot will not be recognized. .UNINDENT .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .UNINDENT .SS Remove command .sp \fBssm\fP \fBremove\fP [\fB\-h\fP] [\fB\-a\fP] [\fBitems\fP [\fBitems\fP ...]] .sp This command removes \fBitem\fP from the system. Multiple items can be specified. If the \fBitem\fP can not be removed for some reason, it will be skipped. .sp \fBitem\fP can represent: .INDENT 0.0 .TP .B device Remove \fBdevice\fP from the pool. Note that this can not be done in some cases where the device is used by pool. You can use \fB\-f\fP argument to \fIforce\fP removal. If the device does not belong to any pool, it will be skipped. .TP .B pool Remove the \fBpool\fP from the system. This will also remove all volumes created from that pool. .TP .B volume Remove the \fBvolume\fP from the system. Note that this will fail if the \fBvolume\fP is mounted and it can not be \fIforced\fP with \fB\-f\fP\&. .UNINDENT .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .TP .B \-a\fP,\fB \-\-all Remove all pools in the system. .UNINDENT .SS Resize command .sp \fBssm\fP \fBresize\fP [\fB\-h\fP] [\fB\-s\fP SIZE] \fBvolume\fP [\fBdevice\fP [\fBdevice\fP ...]] .sp Change size of the \fBvolume\fP and file system. If there is no file system only the \fBvolume\fP itself will be resized. You can specify \fBdevice\fP to add into the \fBvolume\fP pool prior the resize. Note that \fBdevice\fP will only be added into the pool if the \fBvolume\fP size is going to grow. .sp If the \fBdevice\fP is already used in the different pool, then \fBssm\fP will ask you whether you want to remove it from the original pool. .sp In some cases file system has to be mounted in order to resize. This will be handled by \fBssm\fP automatically by mounting the \fBvolume\fP temporarily. .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .TP .BI \-s \ SIZE\fP,\fB \ \-\-size \ SIZE New size of the volume. With the + or \- sign the value is added to or subtracted from the actual size of the volume and without it, the value will be set as the new volume size. A size suffix of [k|K] for kilobytes, [m|M] for megabytes, [g|G] for gigabytes, [t|T] for terabytes or [p|P] for petabytes is optional. If no unit is provided the default is kilobytes. .UNINDENT .SS Check command .sp \fBssm\fP \fBcheck\fP [\fB\-h\fP] \fBdevice\fP [\fBdevice\fP ...] .sp Check the file system consistency on the \fBvolume\fP\&. You can specify multiple volumes to check. If there is no file system on the \fBvolume\fP, this \fBvolume\fP will be skipped. .sp In some cases file system has to be mounted in order to check the file system This will be handled by \fBssm\fP automatically by mounting the \fBvolume\fP temporarily. .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .UNINDENT .SS Snapshot command .sp \fBssm\fP \fBsnapshot\fP [\fB\-h\fP] [\fB\-s\fP SIZE] [\fB\-d\fP DEST | \fB\-n\fP NAME] volume .sp Take a snapshot of existing \fBvolume\fP\&. This operation will fail if back\-end which the \fBvolume\fP belongs to does not support snapshotting. Note that you can not specify both \fINAME\fP and \fIDESC\fP since those options are mutually exclusive. .sp In some cases file system has to be mounted in order to take a snapshot of the \fBvolume\fP\&. This will be handled by \fBssm\fP automatically by mounting the \fBvolume\fP temporarily. .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .TP .BI \-s \ SIZE\fP,\fB \ \-\-size \ SIZE Gives the size to allocate for the new snapshot volume A size suffix K|k, M|m, G|g, T|t, P|p, E|e can be used to define \(aqpower of two\(aq units. If no unit is provided, it defaults to kilobytes. This is option and if not give, the size will be determined automatically. .TP .BI \-d \ DEST\fP,\fB \ \-\-dest \ DEST Destination of the snapshot specified with absolute path to be used for the new snapshot. This is optional and if not specified default backend policy will be performed. .TP .BI \-n \ NAME\fP,\fB \ \-\-name \ NAME Name of the new snapshot. This is optional and if not specified default backend policy will be performed. .UNINDENT .SS Add command .sp \fBssm\fP \fBadd\fP [\fB\-h\fP] [\fB\-p\fP POOL] \fBdevice\fP [\fBdevice\fP ...] .sp This command adds \fBdevice\fP into the pool. The \fBdevice\fP will not be added if it\(aqs already part of different pool by default, but user will be asked whether to remove the device from it\(aqs pool. When multiple devices are provided, all of them are added into the pool. If one of the devices can not be added into the pool for any reason, add command will fail. If no pool is specified, default pool will be chosen. In the case of non existing pool, it will be created using provided devices. .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .TP .BI \-p \ POOL\fP,\fB \ \-\-pool \ POOL Pool to add device into. If not specified the default pool is used. .UNINDENT .SS Mount command .sp \fBssm\fP \fBmount\fP [\fB\-h\fP] [\fB\-o\fP OPTIONS] \fBvolume\fP directory .sp This command will mount the \fBvolume\fP at specified \fBdirectory\fP\&. The \fBvolume\fP can be specified in the same way as with \fBmount(8)\fP, however in addition one can also specify \fBvolume\fP in the format as it appear in the \fBssm list\fP table. .sp For example, instead of finding out what the device and subvolume id of the btrfs subvolume "btrfs_pool:vol001" is in order to mount it, on can simply call \fBssm mount btrfs_pool:vol001 /mnt/test\fP\&. .sp One can also specify \fIOPTIONS\fP in the same way as with \fBmount(8)\fP\&. .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help show this help message and exit .TP .BI \-o \ OPTIONS\fP,\fB \ \-\-options \ OPTIONS Options are specified with a \-o flag followed by a comma separated string of options. This option is equivalent to the same mount(8) option. .UNINDENT .SH BACK-ENDS .SS Introduction .sp Ssm aims to create unified user interface for various technologies like Device Mapper (dm), Btrfs file system, Multiple Devices (md) and possibly more. In order to do so we have a core abstraction layer in \fBssmlib/main.py\fP\&. This abstraction layer should ideally know nothing about the underlying technology, but rather comply with \fBdevice\fP, \fBpool\fP and \fBvolume\fP abstraction. .sp Various backends can be registered in \fBssmlib/main.py\fP in order to handle specific storage technology implementing methods like \fIcreate\fP, \fIsnapshot\fP, or \fIremove\fP volumes and pools. The core will then call these methods to manage the storage without needing to know what lies underneath it. There are already several backends registered in ssm. .SS Btrfs backend .sp Btrfs is the file system with many advanced features including volume management. This is the reason why btrfs is handled differently than other \fIconventional\fP file systems in \fBssm\fP\&. It is used as a volume management back\-end. .sp Pools, volumes and snapshots can be created with btrfs backend and here is what it means from the btrfs point of view: .INDENT 0.0 .TP .B pool Pool is actually a btrfs file system itself, because it can be extended by adding more devices, or shrink by removing devices from it. Subvolumes and snapshots can also be created. When the new btrfs pool should be created \fBssm\fP simply creates a btrfs file system, which means that every new btrfs pool has one volume of the same name as the pool itself which can not be removed without removing the entire pool. Default btrfs pool name is \fBbtrfs_pool\fP\&. .sp When creating new btrfs pool, the name of the pool is used as the file system label. If there is already existing btrfs file system in the system without a label, btrfs pool name will be generated for internal use in the following format "btrfs_{device base name}". .sp Btrfs pool is created when \fBcreate\fP or \fBadd\fP command is used with devices specified and non existing pool name. .TP .B volume Volume in btrfs back\-end is actually just btrfs subvolume with the exception of the first volume created on btrfs pool creation, which is the file system itself. Subvolumes can only be created on btrfs file system when the it is mounted, but user does not have to worry about that, since \fBssm\fP will automatically mount the file system temporarily in order to create a new subvolume. .sp Volume name is used as subvolume path in the btrfs file system and every object in this path must exists in order to create a volume. Volume name for internal tracking and for representing to the user is generated in the format "{pool_name}:{volume name}", but volumes can be also referenced with its mount point. .sp Btrfs volumes are only shown in the \fIlist\fP output, when the file system is mounted, with the exception of the main btrfs volume \- the file system itself. .sp New btrfs volume can be created with \fBcreate\fP command. .TP .B snapshot Btrfs file system support subvolume snapshotting, so you can take a snapshot of any btrfs volume in the system with \fBssm\fP\&. However btrfs does not distinguish between subvolumes and snapshots, because snapshot actually is just a subvolume with some block shared with different subvolume. It means, that \fBssm\fP is not able to recognize btrfs snapshot directly, but instead it is trying to recognize special name format of the btrfs volume. However, if the \fINAME\fP is specified when creating snapshot which does not match the special pattern, snapshot will not be recognized by the \fBssm\fP and it will be listed as regular btrfs volume. .sp New btrfs snapshot can be created with \fBsnapshot\fP command. .TP .B device Btrfs does not require any special device to be created on. .UNINDENT .SS Lvm backend .sp Pools, volumes and snapshots can be created with lvm, which pretty much match the lvm abstraction. .INDENT 0.0 .TP .B pool Lvm pool is just \fIvolume group\fP in lvm language. It means that it is grouping devices and new logical volumes can be created out of the lvm pool. Default lvm pool name is \fBlvm_pool\fP\&. .sp Lvm pool is created when \fBcreate\fP or \fBadd\fP command is used with devices specified and non existing pool name. .TP .B volume Lvm volume is just \fIlogical volume\fP in lvm language. Lvm volume can be created wit \fBcreate\fP command. .TP .B snapshot Lvm volumes can be snapshotted as well. When a snapshot is created from the lvm volume, new \fIsnapshot\fP volume is created, which can be handled as any other lvm volume. Unlike \fIbtrfs\fP lvm is able to distinguish snapshot from regular volume, so there is no need for a snapshot name to match special pattern. .TP .B device Lvm requires \fIphysical device\fP to be created on the device, but with \fBssm\fP this is transparent for the user. .UNINDENT .SS Crypt backend .sp Crypt backend in \fBssm\fP is currently limited to only gather the information about encrypted volumes in the system. You can not create or manage encrypted volumes or pools, but it will be extended in the future. .SS MD backend .sp MD backend in \fBssm\fP is currently limited to only gather the information about MD volumes in the system. You can not create or manage MD volumes or pools, but it will be extended in the future. .SH EXAMPLES .sp \fBList\fP system storage information: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm list .ft P .fi .UNINDENT .UNINDENT .sp \fBList\fP all pools in the system: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm list pools .ft P .fi .UNINDENT .UNINDENT .sp \fBCreate\fP a new 100GB \fBvolume\fP with default lvm backend using \fI/dev/sda\fP and \fI/dev/sdb\fP with xfs file system: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm create \-\-size 100G \-\-fs xfs /dev/sda /dev/sdb .ft P .fi .UNINDENT .UNINDENT .sp \fBCreate\fP a new \fBvolume\fP with btrfs backend using \fI/dev/sda\fP and \fI/dev/sdb\fP and let the volume to be RAID 1: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm \-b btrfs create \-\-raid 1 /dev/sda /dev/sdb .ft P .fi .UNINDENT .UNINDENT .sp Using lvm backend \fBcreate\fP a RAID 0 \fBvolume\fP with devices \fI/dev/sda\fP and \fI/dev/sdb\fP with 128kB stripe size, ext4 file system and mount it on \fI/home\fP: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm create \-\-raid 0 \-\-stripesize 128k /dev/sda /dev/sdb /home .ft P .fi .UNINDENT .UNINDENT .sp \fBExtend\fP btrfs \fBvolume\fP \fIbtrfs_pool\fP by 500GB and use \fI/dev/sdc\fP and \fI/dev/sde\fP to cover the resize: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm resize \-s +500G btrfs_pool /dev/sdc /dev/sde .ft P .fi .UNINDENT .UNINDENT .sp \fBShrink volume\fP \fI/dev/lvm_pool/lvol001\fP by 1TB: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm resize \-s\-1t /dev/lvm_pool/lvol001 .ft P .fi .UNINDENT .UNINDENT .sp \fBRemove\fP \fI/dev/sda\fP \fBdevice\fP from the pool, remove the \fIbtrfs_pool\fP \fBpool\fP and also remove the \fBvolume\fP \fI/dev/lvm_pool/lvol001\fP: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm remove /dev/sda btrfs_pool /dev/lvm_pool/lvol001 .ft P .fi .UNINDENT .UNINDENT .sp \fBTake a snapshot\fP of the btrfs volume \fIbtrfs_pool:my_volume\fP: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm snapshot btrfs_pool:my_volume .ft P .fi .UNINDENT .UNINDENT .sp \fBAdd devices\fP \fI/dev/sda\fP and \fI/dev/sdb\fP into the \fIbtrfs_pool\fP pool: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm add \-p btrfs_pool /dev/sda /dev/sdb .ft P .fi .UNINDENT .UNINDENT .sp \fBMount btrfs subvolume\fP \fIbtrfs_pool:vol001\fP on \fI/mnt/test\fP: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C # ssm mount btrfs_pool:vol001 /mnt/test .ft P .fi .UNINDENT .UNINDENT .SH ENVIRONMENT VARIABLES .INDENT 0.0 .TP .B SSM_DEFAULT_BACKEND Specify which backend will be used by default. This can be overridden by specifying \fB\-b\fP or \fB\-\-backend\fP argument. Currently only \fIlvm\fP and \fIbtrfs\fP is supported. .TP .B SSM_LVM_DEFAULT_POOL Name of the default lvm pool to be used if \fB\-p\fP or \fB\-\-pool\fP argument is omitted. .TP .B SSM_BTRFS_DEFAULT_POOL Name of the default btrfs pool to be used if \fB\-p\fP or \fB\-\-pool\fP argument is omitted. .TP .B SSM_PREFIX_FILTER When this is set \fBssm\fP will filter out all devices, volumes and pools which name does not start with this prefix. It is used mainly in \fBssm\fP test suite to make sure that we do not scramble local system configuration. .UNINDENT .SH LICENCE .sp (C)2011 Red Hat, Inc., Lukas Czerner <\fI\%lczerner@redhat.com\fP> .sp This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version. .sp This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. .sp You should have received a copy of the GNU General Public License along with this program. If not, see <\fI\%http://www.gnu.org/licenses/\fP>. .SH REQUIREMENTS .sp Python 2.6 or higher is required to run this tool. System Storage Manager can only be run as root since most of the commands requires root privileges. .sp There are other requirements listed bellow, but note that you do not necessarily need all dependencies for all backends, however if some of the tools required by the backend is missing, the backend would not work. .SS Python modules .INDENT 0.0 .IP \(bu 2 os .IP \(bu 2 re .IP \(bu 2 sys .IP \(bu 2 stat .IP \(bu 2 argparse .IP \(bu 2 datetime .IP \(bu 2 threading .IP \(bu 2 subprocess .UNINDENT .SS System tools .INDENT 0.0 .IP \(bu 2 tune2fs .IP \(bu 2 fsck.SUPPORTED_FS .IP \(bu 2 resize2fs .IP \(bu 2 xfs_db .IP \(bu 2 xfs_check .IP \(bu 2 xfs_growfs .IP \(bu 2 mkfs.SUPPORTED_FS .IP \(bu 2 which .IP \(bu 2 mount .IP \(bu 2 blkid .IP \(bu 2 wipefs .UNINDENT .SS Lvm backend .INDENT 0.0 .IP \(bu 2 lvm2 binaries .UNINDENT .SS Btrfs backend .INDENT 0.0 .IP \(bu 2 btrfs progs .UNINDENT .SS Crypt backend .INDENT 0.0 .IP \(bu 2 dmsetup .IP \(bu 2 cryptsetup .UNINDENT .SH AVAILABILITY .sp \fBSystem storage manager\fP is available from \fI\%http://storagemanager.sourceforge.net\fP\&. You can subscribe to \fI\%storagemanager-devel@lists.sourceforge.net\fP to follow the current development. .SH AUTHOR Lukáš Czerner .SH COPYRIGHT 2012, Red Hat, Inc., Lukáš Czerner .\" Generated by docutils manpage writer. . system-storage-manager-0.4/COPYING0000664000175000017500000004325412171200774020513 0ustar lczernerlczerner00000000000000 GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. system-storage-manager-0.4/AUTHORS0000664000175000017500000000046312200423444020515 0ustar lczernerlczerner00000000000000System Storage Manager was written by: Lukáš Czerner Contributions (commits): (171) Lukas Czerner (16) Tom Marek (13) Jimmy Pan (4) Tony Asleson (1) Tomas Racek system-storage-manager-0.4/ssmlib/0000775000175000017500000000000012200423561020733 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/ssmlib/problem.py0000664000175000017500000002312012200417227022745 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2012 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # # problem.py - dealing with problems and errors in ssm import sys __all__ = ["ProblemSet", "SsmError", "GeneralError", "ProgrammingError", "BadEnvVariable", "NotEnoughSpace", "ResizeMatch", "FsNotSpecified", "DeviceUsed", "ExistingFilesystem", "NoDevices", "ToolMissing", "CanNotRun", "CommandFailed", "UserInterrupted", "NotSupported"] # Define prompt codes PROMPT_NONE = 0 PROMPT_UNMOUNT = 1 PROMPT_SET_DEFAULT = 2 PROMPT_IGNORE = 3 PROMPT_REMOVE = 4 PROMPT_ADJUST = 5 PROMPT_USE = 6 PROMPT_MSG = [ None, 'Unmount', 'Set default', 'Ignore', 'Remove', 'Adjust', 'Use anyway', ] # Define problem flags FL_NONE = 0 FL_MSG_ONLY = 2 FL_VERBOSE_ONLY = (4 | FL_MSG_ONLY) FL_DEBUG_ONLY = (8 | FL_MSG_ONLY) FL_DEFAULT_NO = 16 FL_SILENT = 32 FL_EXIT_ON_NO = 64 FL_EXIT_ON_YES = 128 FL_FATAL = 256 FL_FORCE_YES = 512 FL_FORCE_NO = 1024 class SsmError(Exception): """Base exception class for the ssm.""" def __init__(self, msg, errcode=None): super(SsmError, self).__init__() self.msg = msg self.errcode = errcode def __str__(self): return repr("Error ({0}): {1}".format(self.errcode, self.msg)) class GeneralError(SsmError): def __init__(self, msg, errcode=2001): super(GeneralError, self).__init__(msg, errcode) class ProgrammingError(SsmError): def __init__(self, msg, errcode=2002): super(ProgrammingError, self).__init__(msg, errcode) class FsMounted(SsmError): def __init__(self, msg, errcode=2003): super(FsMounted, self).__init__(msg, errcode) class BadEnvVariable(SsmError): def __init__(self, msg, errcode=2004): super(BadEnvVariable, self).__init__(msg, errcode) class NotEnoughSpace(SsmError): def __init__(self, msg, errcode=2005): super(NotEnoughSpace, self).__init__(msg, errcode) class ResizeMatch(SsmError): def __init__(self, msg, errcode=2006): super(ResizeMatch, self).__init__(msg, errcode) class FsNotSpecified(SsmError): def __init__(self, msg, errcode=2007): super(FsNotSpecified, self).__init__(msg, errcode) class DeviceUsed(SsmError): def __init__(self, msg, errcode=2008): super(DeviceUsed, self).__init__(msg, errcode) class NoDevices(SsmError): def __init__(self, msg, errcode=2009): super(NoDevices, self).__init__(msg, errcode) class ToolMissing(SsmError): def __init__(self, msg, errcode=2010): super(ToolMissing, self).__init__(msg, errcode) class CanNotRun(SsmError): def __init__(self, msg, errcode=2011): super(CanNotRun, self).__init__(msg, errcode) class CommandFailed(SsmError): def __init__(self, msg, errcode=2012): super(CommandFailed, self).__init__(msg, errcode) class UserInterrupted(SsmError): def __init__(self, msg, errcode=2013): super(UserInterrupted, self).__init__(msg, errcode) class NotSupported(SsmError): def __init__(self, msg, errcode=2014): super(NotSupported, self).__init__(msg, errcode) class ExistingFilesystem(SsmError): def __init__(self, msg, errcode=2015): super(ExistingFilesystem, self).__init__(msg, errcode) class ProblemSet(object): def __init__(self, options): self.set_options(options) self.init_problem_set() def set_options(self, options): self.options = options def init_problem_set(self): self.PROGRAMMING_ERROR = \ ['Programming error detected! {0}', PROMPT_NONE, FL_FATAL, ProgrammingError] self.GENERAL_ERROR = \ ['SSM Error: {0}!', PROMPT_NONE, FL_FATAL, GeneralError] self.GENERAL_INFO = \ ['SSM Info: {0}', PROMPT_NONE, FL_NONE, None] self.GENERAL_WARNING = \ ['SSM Warning: {0}!', PROMPT_NONE, FL_NONE, None] self.FS_MOUNTED = \ ['Device \'{0}\' is mounted on \'{1}\'', PROMPT_UNMOUNT, FL_DEFAULT_NO | FL_EXIT_ON_NO | FL_FORCE_YES, FsMounted] self.BAD_ENV_VARIABLE = \ ['Environment variable \'{0}\' contains unsupported value \'{1}\'!', PROMPT_SET_DEFAULT, FL_EXIT_ON_NO, BadEnvVariable] self.RESIZE_NOT_ENOUGH_SPACE = \ ['There is not enough space in the pool \'{0}\' to grow volume' + ' \'{1}\' to size {2} KB!', PROMPT_NONE, FL_FATAL, NotEnoughSpace] self.CREATE_NOT_ENOUGH_SPACE = \ ['Not enough space ({0} KB) in the pool \'{1}\' to create ' + 'volume!', PROMPT_ADJUST, FL_DEFAULT_NO | FL_EXIT_ON_NO | FL_FORCE_YES, NotEnoughSpace] self.RESIZE_ALREADY_MATCH = \ ['\'{0}\' is already {1} KB long, there is nothing ' + 'to resize!', PROMPT_NONE, FL_FATAL, ResizeMatch] self.CREATE_MOUNT_NOFS = \ ['Mount point \'{0}\' specified, but no file system provided!', PROMPT_IGNORE, FL_EXIT_ON_NO | FL_FORCE_YES, FsNotSpecified] self.DEVICE_USED = \ ['Device \'{0}\' is already used in the \'{1}\'!', PROMPT_REMOVE, FL_DEFAULT_NO | FL_FORCE_YES, DeviceUsed] self.EXISTING_FILESYSTEM = \ ['Filesystem \'{0}\' detected on the device \'{1}\'!', PROMPT_USE, FL_DEFAULT_NO | FL_FORCE_YES, ExistingFilesystem] self.NO_DEVICES = \ ['No devices available to use for the \'{0}\' pool!', PROMPT_NONE, FL_FATAL, NoDevices] self.TOOL_MISSING = \ ['\'{0}\' is not installed on the system!', PROMPT_NONE, FL_FATAL, ToolMissing] self.CAN_NOT_RUN = \ ['Can not run command \'{0}\'', PROMPT_NONE, FL_FATAL, CanNotRun] self.COMMAND_FAILED = \ ['Error while running command \'{0}\'', PROMPT_NONE, FL_FATAL, CommandFailed] self.NOT_SUPPORTED = \ ['{0} is not supported!', PROMPT_NONE, FL_FATAL, NotSupported] def _can_print_message(self, flags): if (flags & FL_DEBUG_ONLY): return self.options.debug elif (flags & FL_VERBOSE_ONLY): return self.options.verbose else: return True def _read_char(self): import tty import termios fd = sys.stdin.fileno() old_settings = termios.tcgetattr(fd) try: tty.setraw(sys.stdin.fileno()) ch = sys.stdin.read(1) finally: termios.tcsetattr(fd, termios.TCSADRAIN, old_settings) return ch def _ask_question(self, flags): if flags & FL_DEFAULT_NO: print "(N/y/q) ?", else: print "(Y/n/q) ?", ch = '' if self.options.force and flags & FL_FORCE_NO: ch = 'N' elif self.options.force and flags & FL_FORCE_YES: ch = 'Y' elif self.options.interactive: while ch not in ['Y', 'N', 'Q', chr(13)]: ch = self._read_char().upper() elif flags & FL_DEFAULT_NO: ch = 'N' else: ch = 'Y' if ch == chr(13): if flags & FL_DEFAULT_NO: ch = 'N' else: ch = 'Y' print ch if ch == 'Y': return True elif ch == 'N': return False elif ch == 'Q': err = "Terminated by user!" raise UserInterrupted(err) def check(self, problem, args): if type(args) is not list: args = [args] message = problem[0].format(*args) prompt_msg = PROMPT_MSG[problem[1]] flags = problem[2] exc = problem[3] if (flags & FL_DEFAULT_NO): res = False else: res = True if self._can_print_message(flags) and \ (flags & FL_MSG_ONLY or prompt_msg is None): print >> sys.stderr, message, else: print message, if not flags & FL_MSG_ONLY and prompt_msg is not None: print '{0}'.format(prompt_msg), res = self._ask_question(flags) else: print >> sys.stderr if (flags & FL_FATAL): if exc: raise exc(message) else: raise Exception(message) if ((flags & FL_EXIT_ON_NO) and (not res)) or \ ((flags & FL_EXIT_ON_YES) and res): if exc: raise exc(message) else: raise Exception(message) return res def error(self, args): self.check(self.GENERAL_ERROR, args) def info(self, args): self.check(self.GENERAL_INFO, args) def warn(self, args): self.check(self.GENERAL_WARNING, args) def not_supported(self, args): self.check(self.NOT_SUPPORTED, args) system-storage-manager-0.4/ssmlib/backends/0000775000175000017500000000000012200423561022505 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/ssmlib/backends/crypt.py0000664000175000017500000000764712200420722024232 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # crypt module for System Storage Manager import re import os from ssmlib import misc from ssmlib import problem __all__ = ["DmCryptVolume"] try: SSM_CRYPT_DEFAULT_POOL = os.environ['SSM_CRYPT_DEFAULT_POOL'] except KeyError: SSM_CRYPT_DEFAULT_POOL = "crypt_pool" try: DM_DEV_DIR = os.environ['DM_DEV_DIR'] except KeyError: DM_DEV_DIR = "/dev" class DmCryptVolume(object): def __init__(self, options, data=None): self.type = 'crypt' self.data = data or {} self.output = None self.options = options self.mounts = misc.get_mounts('{0}/mapper'.format(DM_DEV_DIR)) self.default_pool_name = SSM_CRYPT_DEFAULT_POOL self.problem = problem.ProblemSet(options) if not misc.check_binary('dmsetup') or \ not misc.check_binary('cryptsetup'): return command = ['dmsetup', 'table'] self.output = misc.run(command, stderr=False)[1] for line in self.output.split("\n"): if not line or line == "No devices found": break dm = {} array = line.split() dm['type'] = array[3] if dm['type'] != 'crypt': continue dm['vol_size'] = str(int(array[2]) / 2.0) devname = re.sub(":$", "", "{0}/mapper/{1}".format(DM_DEV_DIR, array[0])) dm['dm_name'] = devname dm['pool_name'] = 'dm-crypt' dm['dev_name'] = misc.get_real_device(devname) dm['real_dev'] = dm['dev_name'] if dm['real_dev'] in self.mounts: dm['mount'] = self.mounts[dm['real_dev']]['mp'] # Check if the device really exists in the system. In some cases # (tests) DM_DEV_DIR can lie to us, if that is the case, simple # ignore the device. if not os.path.exists(devname): continue command = ['cryptsetup', 'status', devname] self._parse_cryptsetup(command, dm) self.data[dm['dev_name']] = dm def run_cryptsetup(self, command, stdout=True): if not self._binary: self.problem.check(self.problem.TOOL_MISSING, 'cryptsetup') command.insert(0, "cryptsetup") return misc.run(command, stdout=stdout) def _parse_cryptsetup(self, cmd, dm): self.output = misc.run(cmd, stderr=False)[1] for line in self.output.split("\n"): if not line: break array = line.split() if array[0].strip() == 'cipher:': dm['cipher'] = array[1] elif array[0].strip() == 'keysize:': dm['keysize'] = array[1] elif array[0].strip() == 'device:': dm['crypt_device'] = array[1] def remove(self, dm): command = ['remove', dm] self.run_cryptsetup(command) def resize(self, dm, size, resize_fs=True): size = str(int(size) * 2) command = ['resize', '--size', size, dm] self.run_cryptsetup(command) def __iter__(self): for item in sorted(self.data.iterkeys()): yield item def __getitem__(self, key): if key in self.data.iterkeys(): return self.data[key] system-storage-manager-0.4/ssmlib/backends/__init__.py0000664000175000017500000000136612171200774024632 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . __all__ = ["lvm", "btrfs", "crypt"] system-storage-manager-0.4/ssmlib/backends/md.py0000664000175000017500000001111312200421166023453 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2012 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # md module for System Storage Manager import os import stat import socket import datetime from ssmlib import misc from ssmlib import problem try: SSM_DM_DEFAULT_POOL = os.environ['SSM_DM_DEFAULT_POOL'] except KeyError: SSM_DM_DEFAULT_POOL = "md" MDADM = "mdadm" class MdRaid(object): def __init__(self, options, data=None): self.type = 'dm' self.data = data or {} self._vol = {} self._pool = {} self._dev = {} self.options = options self.hostname = socket.gethostname() self._binary = misc.check_binary(MDADM) self.default_pool_name = SSM_DM_DEFAULT_POOL self.attrs = ['dev_name', 'pool_name', 'dev_free', 'dev_used', 'dev_size'] if not self._binary: return self.problem = problem.ProblemSet(options) self.mounts = misc.get_mounts('/dev/md') mdnumber = misc.get_dmnumber("md") for line in misc.get_partitions(): devname = '/dev/' + line[3] devsize = int(line[2]) if line[0] == mdnumber: self._vol[devname] = self.get_volume_data(devname) for dev in misc.get_slaves(os.path.basename(devname)): self._dev[dev] = self.get_device_data(dev, devsize) def get_device_data(self, devname, devsize): data = {} data['dev_name'] = devname data['hide'] = False command = [MDADM, '--examine', devname] output = misc.run(command, stderr=False)[1].split("\n") for line in output: array = line.split(":") if len(array) < 2: continue item = array[0].strip() if item == "Name": data['pool_name'] = SSM_DM_DEFAULT_POOL data['dev_used'] = data['dev_size'] = devsize data['dev_free'] = 0 return data def get_volume_data(self, devname): data = {} data['dev_name'] = devname data['real_dev'] = devname data['pool_name'] = SSM_DM_DEFAULT_POOL if data['dev_name'] in self.mounts: data['mount'] = self.mounts[data['dev_name']]['mp'] command = [MDADM, '--detail', devname] for line in misc.run(command, stderr=False)[1].split("\n"): array = line.split(":") if len(array) < 2: continue item = array[0].strip() value = array[1].strip() if item == 'Raid Level': data['type'] = value elif item == 'Array Size': data['vol_size'] = value.split()[0] elif item == 'Total Devices': data['total_devices'] = value return data def run_mdadm(self, command): if not self._binary: self.problem.check(self.problem.TOOL_MISSING, MDADM) command.insert(0, MDADM) return misc.run(command, stdout=True) def __iter__(self): for item in sorted(self.data.iterkeys()): yield item def __getitem__(self, key): if key in self.data.iterkeys(): return self.data[key] class MdRaidVolume(MdRaid): def __init__(self, *args, **kwargs): super(MdRaidVolume, self).__init__(*args, **kwargs) if self.data: self.data.update(self._vol) else: self.data = self._vol def remove(self, vol): command = ['--stop', vol] self.run_mdadm(command) def resize(self, vol, size, resize_fs=True): self.problem.not_supported("Resizing with \"md\" backend") class MdRaidDevice(MdRaid): def __init__(self, *args, **kwargs): super(MdRaidDevice, self).__init__(*args, **kwargs) if self.data: self.data.update(self._dev) else: self.data = self._dev def remove(self, devices): self.problem.not_supported("Removing device from \"md\" backend") system-storage-manager-0.4/ssmlib/backends/btrfs.py0000664000175000017500000005160112200420677024207 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # btrfs module for System Storage Manager import re import os import sys import datetime from ssmlib import misc from ssmlib import problem __all__ = ["BtrfsVolume", "BtrfsPool", "BtrfsDev"] try: SSM_BTRFS_DEFAULT_POOL = os.environ['SSM_BTRFS_DEFAULT_POOL'] except KeyError: SSM_BTRFS_DEFAULT_POOL = "btrfs_pool" def get_real_number(string): number = float(string[0:-2]) unit = string[-2:-1] # The result will be in kilobytes units = ["K", "M", "G", "T", "P", "E", "Z", "Y"] for i, u in enumerate(units): if u == unit: number *= (2 ** (i * 10)) break return number def get_btrfs_version(): try: output = misc.run(['btrfs', '--version'], can_fail=True)[1] output = output.strip().split("\n")[-1] version = re.search('(?<=v)\d+\.\d+', output).group(0) except (OSError, AttributeError): version = "0.0" return float(version) BTRFS_VERSION = get_btrfs_version() class Btrfs(object): def __init__(self, options, data=None): self.type = 'btrfs' self.data = data or {} self.options = options self.default_pool_name = SSM_BTRFS_DEFAULT_POOL self._vol = {} self._pool = {} self._dev = {} self._snap = {} self._subvolumes = {} self._binary = misc.check_binary('btrfs') self.problem = problem.ProblemSet(options) self.modified_list_version = True if not self._binary: return self.mounts = misc.get_mounts('/dev/') command = ['btrfs', 'filesystem', 'show'] self.output = misc.run(command, stderr=False)[1] vol = {} pool = {} dev = {} partitions = {} fs_size = pool_size = fs_used = 0 pool_name = '' for line in misc.get_partitions(): partitions[line[3]] = line for line in self.output.strip().split("\n"): if not line: continue array = line.split() if array[0] == 'Label:': if len(vol) > 0: self._store_data(vol, pool, fs_used, fs_size, pool_size, pool_name) vol = {} pool = {} fs_size = pool_size = 0 pool_name = '' label = array[1].strip("'") uuid = array[3] pool['uuid'] = vol['uuid'] = uuid try: fallback = False vol['real_dev'] = misc.get_device_by_uuid(uuid) if vol['real_dev'] in self.mounts: pool['mount'] = self.mounts[vol['real_dev']]['mp'] vol['mount'] = self.mounts[vol['real_dev']]['mp'] else: for dev_i in self.mounts: found = re.findall(r'{0}:/.*'.format(vol['real_dev']), dev_i) if found: pool['mount'] = self.mounts[found[0]]['mp'] break except OSError: # udev is "hard-to-work-with" sometimes so this is fallback fallback = True vol['real_dev'] = "" if label != 'none': vol['label'] = label vol['ID'] = 0 elif array[0] == 'Total': pool['dev_count'] = array[2] fs_used = get_real_number(array[6]) elif array[0] == 'devid': dev['dev_name'] = array[7] if not pool_name: pool_name = self._find_uniq_pool_name(label, array[7]) dev['pool_name'] = pool_name # Fallback in case we could not find real_dev by uuid if fallback and 'mount' not in pool: if dev['dev_name'] in self.mounts: pool['mount'] = self.mounts[dev['dev_name']]['mp'] vol['real_dev'] = dev['dev_name'] if 'root' in self.mounts[dev['dev_name']]: if self.mounts[dev['dev_name']]['root'] == '/': vol['mount'] = self.mounts[dev['dev_name']]['mp'] else: for dev_i in self.mounts: found = re.findall(r'{0}:/.*'.format(dev['dev_name']), dev_i) if found: pool['mount'] = self.mounts[found[0]]['mp'] break dev_used = get_real_number(array[5]) dev['dev_used'] = str(dev_used) fs_size += get_real_number(array[3]) dev_size = \ int(partitions[dev['dev_name'].rpartition("/")[-1]][2]) pool_size += dev_size dev['dev_free'] = dev_size - dev_used self._dev[dev['dev_name']] = dev dev = {} if len(vol) > 0: self._store_data(vol, pool, fs_used, fs_size, pool_size, pool_name) def run_btrfs(self, command): if not self._binary: self.problem.check(self.problem.TOOL_MISSING, 'btrfs') command.insert(0, "btrfs") return misc.run(command, stdout=True) def _list_subvolumes(self, mount, list_snapshots=False): command = ['btrfs', 'subvolume', 'list'] if self.modified_list_version: command.append('-a') if list_snapshots: command.append('-s') ret, output = misc.run(command + [mount], stdout=False, can_fail=True) if ret: command = ['btrfs', 'subvolume', 'list'] if list_snapshots: command.append('-s') output = misc.run(command + [mount], stdout=False)[1] self.modified_list_version = False return output # There is no way in btrfs to list subvolumes which are not snapshots # so we have to get the list of snapshots to filter it out from # regular subvolume list so we do not have it in the output twice. # Once in volume list and once in snapshot list. def _get_snap_name_list(self, mount): snap = [] if BTRFS_VERSION < 0.20: return snap command = ['btrfs', 'subvolume', 'list', '-s', mount] output = misc.run(command, stdout=False)[1] for line in output.strip().split("\n"): if not line: continue path = re.search('(?<=path ).*$', line).group(0) snap.append(path) return snap def _fill_subvolumes(self, list_snapshots=False): if not self._binary: return if self._subvolumes: return for name, vol in self._vol.iteritems(): pool_name = vol['pool_name'] real_dev = vol['real_dev'] pool = self._pool[pool_name] if 'mount' in self._pool[pool_name]: mount = pool['mount'] else: # If btrfs is not mounted we will not process subvolumes continue snapshots = [] if not list_snapshots: snapshots = self._get_snap_name_list(mount) output = self._list_subvolumes(mount, list_snapshots) for volume in self._parse_subvolumes(output): new = vol.copy() new.update(volume) new['dev_name'] = "{0}:{1}".format(name, new['path']) item = "{0}:/{1}".format(real_dev, new['path']) # If the subvolume is mounted we should find it here if item in self.mounts: new['mount'] = self.mounts[item]['mp'] # Subvolume is mounted directly new['direct_mount'] = True else: # If subvolume is not mounted try to find whether parent # subvolume is mounted found = re.findall(r'^(.*)/([^/]*)$', new['path']) if found: parent_path, path = found[0] # try previously loaded subvolumes for prev_sv in self._subvolumes: # if subvolumes are mounted, use that mp if self._subvolumes[prev_sv]['path'] == parent_path: # if parent subvolume is not mounted this # subvolume is not mounted as well if self._subvolumes[prev_sv]['mount'] == '': new['mount'] = '' else: new['mount'] = "{0}/{1}".format( self._subvolumes[prev_sv]['mount'], path) break # if parent volume is not mounted, use root subvolume # if mounted else: if 'mount' in vol: new['mount'] = "{0}/{1}".format(vol['mount'], new['path']) new['hide'] = False # Store snapshot info if 'mount' in new and \ re.match("snap-\d{4}-\d{2}-\d{2}-T\d{6}", os.path.basename(new['mount'])): new['snap_name'] = "{0}:{1}".format(name, os.path.basename(new['path'])) new['snap_path'] = new['mount'] if volume['path'] in snapshots: new['hide'] = True self._subvolumes[new['dev_name']] = new def _parse_subvolumes(self, output): volume = {} for line in output.strip().split("\n"): if not line: continue # For the version with screwed 'subvolume list' command line = re.sub("/*", "", line) volume['ID'] = re.search('(?<=ID )\d+', line).group(0) volume['top_level'] = re.search('(?<=top level )\d+', line).group(0) volume['path'] = re.search('(?<=path ).*$', line).group(0) volume['subvolume'] = True yield volume def _find_uniq_pool_name(self, label, dev): if len(label) < 3 or label == "none": label = "btrfs_{0}".format(os.path.basename(dev)) if label not in self._pool: return label return os.path.basename(dev) def _store_data(self, vol, pool, fs_used, fs_size, pool_size, pool_name): vol['fs_type'] = 'btrfs' vol['fs_used'] = str(fs_used) vol['fs_free'] = str(fs_size - fs_used) vol['fs_size'] = vol['vol_size'] = pool['pool_used'] = \ str(fs_size) pool['pool_free'] = str(pool_size - fs_used) pool['pool_size'] = pool_size pool['pool_name'] = vol['pool_name'] = vol['dev_name'] = pool_name pool['type'] = 'btrfs' vol['type'] = 'btrfs' self._pool[pool['pool_name']] = pool self._vol[vol['dev_name']] = vol def __iter__(self): for item in sorted(self.data.iterkeys()): yield item def __getitem__(self, key): if key in self.data.iterkeys(): return self.data[key] def _remove_filesystem(self, name): if 'mount' in self._vol[name]: if self.problem.check(self.problem.FS_MOUNTED, [name, self._vol[name]['mount']]): misc.do_umount(self._vol[name]['mount']) for dev in self._dev.itervalues(): if dev['pool_name'] != name: continue misc.wipefs(dev['dev_name'], 'btrfs') class BtrfsVolume(Btrfs): def __init__(self, *args, **kwargs): super(BtrfsVolume, self).__init__(*args, **kwargs) self._fill_subvolumes() if self.data: self.data.update(self._vol) self.data.update(self._subvolumes) else: self.data = self._vol self.data.update(self._subvolumes) def mount(self, vol, mpoint, options=None): vol = self.data[vol] if options: options += "," else: options = "" options += "subvolid={0}".format(vol['ID']) misc.do_mount(vol['real_dev'], mpoint, options) def remove(self, vol): volume = self._vol[vol] if 'subvolume' in volume: # If subvolume is mounted directly we can not remove it. So ask # user whether he wants to umount it. The we'll have to mount the # root subvolume and remove this subvolume. if 'direct_mount' in volume and volume['direct_mount']: if self.problem.check(self.problem.FS_MOUNTED, [vol, volume['mount']]): misc.do_umount(volume['mount']) del volume['mount'] del volume['direct_mount'] if 'mount' not in volume: mount = misc.temp_mount("UUID={0}".format(volume['uuid'])) path = "{0}/{1}".format(mount, volume['path']) else: path = volume['mount'] self.run_btrfs(['subvolume', 'delete', path]) else: self._remove_filesystem(vol) def resize(self, vol, size, resize_fs=True): vol = self.data[vol] if 'mount' not in vol: tmp = misc.temp_mount("UUID={0}".format(vol['uuid'])) vol['mount'] = tmp command = ['filesystem', 'resize', str(int(size)) + "K", vol['mount']] self.run_btrfs(command) def snapshot(self, vol, destination, name, size, user_set_size): vol = self.data[vol] if 'mount' not in vol: tmp = misc.temp_mount("UUID={0}".format(vol['uuid'])) vol['mount'] = tmp if not destination and not name: now = datetime.datetime.now() destination = vol['mount'] + now.strftime("/snap-%Y-%m-%d-T%H%M%S") if name: destination = vol['mount'] + "/" + name if user_set_size: self.problem.warn("Btrfs doesn't allow setting a size of " + "subvolumes") command = ['subvolume', 'snapshot', vol['mount'], destination] self.run_btrfs(command) class BtrfsDev(Btrfs): def __init__(self, *args, **kwargs): super(BtrfsDev, self).__init__(*args, **kwargs) if self.data: self.data.update(self._dev) else: self.data = self._dev def remove(self, devices): raise Exception("Not sure what you want to" + "achieve by removing {0}".format(devices)) class BtrfsPool(Btrfs): def __init__(self, *args, **kwargs): super(BtrfsPool, self).__init__(*args, **kwargs) if self.data: self.data.update(self._pool) else: self.data = self._pool def _create_filesystem(self, pool, name, devs, size=None, raid=None): if not devs: raise Exception("To create btrfs volume, some devices must be " + "provided") self._binary = misc.check_binary('mkfs.btrfs') if not self._binary: self.problem.check(self.problem.TOOL_MISSING, 'mkfs.btrfs') command = ['mkfs.btrfs', '-L', name] if raid: if raid['level'] == '0': command.extend(['-m', 'raid0', '-d', 'raid0']) elif raid['level'] == '1': command.extend(['-m', 'raid1', '-d', 'raid1']) elif raid['level'] == '10': command.extend(['-m', 'raid10', '-d', 'raid10']) else: raise Exception("Btrfs backed currently does not support " + "RAID level {0}".format(raid['level'])) if size: command.extend(['-b', "{0}".format(int(float(size) * 1024))]) # This might seem weird, but btrfs is mostly broken when it comes to # checking existing signatures because it will for example check for # backup superblocks as well, which is wrong. Also we have check for # existing file system signatures in the ssm itself. Other things # than file system should be covered by the backend and we should # have tried to remove the device from the respective pool already. # So at this point there should not be any useful signatures to # speak of. However as I mentioned btrfs is broken, so force it. command.extend(['-f']) command.extend(devs) misc.run(command, stdout=True) misc.send_udev_event(devs[0], "change") return name def _check_new_path(self, path, name): msg = 0 parent = os.path.split(path)[0] if os.path.exists(path): msg = "Directory \'{0}\' already exist. ".format(path) + \ "Subvolume \'{0}\' can not be ".format(name) + \ "created" elif not os.path.exists(parent): msg = "Parent directory \'{0}\' ".format(parent) + \ "does not exist. Subvolume " + \ "\'{0}\' can not be created".format(name) if msg: self.problem.error(msg) def reduce(self, pool, device): pool = self.data[pool] if 'mount' not in pool: tmp = misc.temp_mount("UUID={0}".format(pool['uuid'])) pool['mount'] = tmp command = ['device', 'delete', device, pool['mount']] self.run_btrfs(command) misc.send_udev_event(device, "change") def new(self, pool, devices): if type(devices) is not list: devices = [devices] self.create(pool, devs=devices) def extend(self, pool, devices): pool = self.data[pool] if 'mount' not in pool: tmp = misc.temp_mount("UUID={0}".format(pool['uuid'])) pool['mount'] = tmp if type(devices) is not list: devices = [devices] command = ['device', 'add'] command.extend(devices) command.append(pool['mount']) self.run_btrfs(command) def remove(self, pool): # Volume and pool name should be the same, since it actually is the # same file system self._remove_filesystem(pool) def create(self, pool, size=None, name=None, devs=None, raid=None): if pool in self._pool: vol = None if size or raid: self.problem.warn("Only name, volume name and pool name " + "can be specified when creating btrfs " + "subvolume, the rest will be ignored") tmp = misc.temp_mount("UUID={0}".format(self._pool[pool]['uuid'])) self._pool[pool]['mount'] = tmp if not name: now = datetime.datetime.now() name = now.strftime("%Y-%m-%d-T%H%M%S") vol = "{0}/{1}".format(self._pool[pool]['mount'], name) elif os.path.isabs(name): vol = name else: vol = "{0}/{1}".format(self._pool[pool]['mount'], name) self._check_new_path(vol, name) self.run_btrfs(['subvolume', 'create', vol]) vol = "{0}:{1}".format(pool, name) else: if len(devs) == 0: self.problem.check(self.problem.NO_DEVICES, pool) if name: self.problem.warn("Creating new pool. Argument (--name " + "{0}) will be ignored!".format(name)) vol = self._create_filesystem(pool, pool, devs, size, raid) return vol class BtrfsSnap(Btrfs): def __init__(self, *args, **kwargs): super(BtrfsSnap, self).__init__(*args, **kwargs) self._fill_subvolumes(list_snapshots=True) for name, vol in self._subvolumes.iteritems(): if BTRFS_VERSION < 0.20: if 'snap_name' in vol: self._snap[vol['snap_name']] = vol.copy() self._snap[vol['snap_name']]['hide'] = False else: self._snap[vol['dev_name']] = vol.copy() if self.data: self.data.update(self._snap) else: self.data = self._snap system-storage-manager-0.4/ssmlib/backends/lvm.py0000664000175000017500000002763612200421147023671 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # lvm module for System Storage Manager import os import stat import datetime from ssmlib import misc from ssmlib import problem __all__ = ["PvsInfo", "VgsInfo", "LvsInfo"] try: SSM_LVM_DEFAULT_POOL = os.environ['SSM_LVM_DEFAULT_POOL'] except KeyError: SSM_LVM_DEFAULT_POOL = "lvm_pool" try: DM_DEV_DIR = os.environ['DM_DEV_DIR'] except KeyError: DM_DEV_DIR = "/dev" MAX_LVS = 999 class LvmInfo(object): def __init__(self, options, data=None): self.type = 'lvm' self.data = data or {} self.attrs = [] self.output = None self.options = options self.binary = misc.check_binary('lvm') self.default_pool_name = SSM_LVM_DEFAULT_POOL self.problem = problem.ProblemSet(options) def run_lvm(self, command, noforce=False): if not self.binary: self.problem.check(self.problem.TOOL_MISSING, 'lvm') if self.options.force and not noforce: command.insert(1, "-f") if self.options.verbose: command.insert(1, "-v") command.insert(0, "lvm") misc.run(command, stdout=True) def __str__(self): return self.output def _data_index(self, row): return row.values()[len(row.values()) - 1] def _skip_data(self, row): return False def _parse_data(self, command): if not self.binary: return self.output = misc.run(command, stderr=False)[1] for line in self.output.split("\n"): if not line: break array = line.split("|") row = dict([(self.attrs[index], array[index].lstrip()) for index in range(len(array))]) if self._skip_data(row): continue self._fill_aditional_info(row) self.data[self._data_index(row)] = row def _fill_aditional_info(self, row): pass def __iter__(self): for item in sorted(self.data.iterkeys()): yield item def __getitem__(self, key): if key in self.data.iterkeys(): return self.data[key] class VgsInfo(LvmInfo): def __init__(self, *args, **kwargs): super(VgsInfo, self).__init__(*args, **kwargs) command = ["lvm", "vgs", "--separator", "|", "--noheadings", "--nosuffix", "--units", "k", "-o", "vg_name,pv_count,vg_size,vg_free,lv_count"] self.attrs = ['pool_name', 'dev_count', 'pool_size', 'pool_free', 'vol_count'] self._parse_data(command) def _fill_aditional_info(self, vg): vg['type'] = 'lvm' vg['pool_used'] = float(vg['pool_size']) - float(vg['pool_free']) def _data_index(self, row): return row['pool_name'] def _generate_lvname(self, vg): for i in range(1, MAX_LVS): name = "lvol{0:0>{align}}".format(i, align=len(str(MAX_LVS))) path = "{0}/{1}/{2}".format(DM_DEV_DIR, vg, name) try: if stat.S_ISBLK(os.stat(path).st_mode): continue except OSError: pass return name self.problem.error("Can not find proper lvname!") def reduce(self, vg, device): command = ['vgreduce', vg, device] self.run_lvm(command) def new(self, vg, devices): if type(devices) is not list: devices = [devices] command = ['vgcreate', vg] command.extend(devices) self.run_lvm(command) def extend(self, vg, devices): if type(devices) is not list: devices = [devices] command = ['vgextend', vg] command.extend(devices) self.run_lvm(command) def remove(self, vg): command = ['vgremove', vg] self.run_lvm(command) def create(self, vg, size=None, name=None, devs=None, raid=None): devices = devs or [] command = ['lvcreate', vg] if size: command.extend(['-L', size + 'K']) else: if len(devices) > 0: size = "100%PVS" else: size = "100%FREE" command.extend(['-l', size]) if name: lvname = name else: lvname = self._generate_lvname(vg) command.extend(['-n', lvname.rpartition("/")[-1]]) if raid: if raid['level'] == '0': if not raid['stripesize']: raid['stripesize'] = "64" if not raid['stripes'] and len(devices) > 0: raid['stripes'] = str(len(devices)) if not raid['stripes']: self.problem.error("Devices or number of " + "stripes should be defined!") if raid['stripesize']: command.extend(['-I', raid['stripesize']]) if raid['stripes']: command.extend(['-i', raid['stripes']]) else: self.problem.not_supported("RAID level {0}".format(raid['level']) + " with \"lvm\" backend") command.extend(devices) self.run_lvm(command, noforce=True) return "{0}/{1}/{2}".format(DM_DEV_DIR, vg, lvname) class PvsInfo(LvmInfo): def __init__(self, *args, **kwargs): super(PvsInfo, self).__init__(*args, **kwargs) command = ["lvm", "pvs", "--separator", "|", "--noheadings", "--nosuffix", "--units", "k", "-o", "pv_name,vg_name,pv_free,pv_used,pv_size"] self.attrs = ['dev_name', 'pool_name', 'dev_free', 'dev_used', 'dev_size'] self._parse_data(command) def _data_index(self, row): return misc.get_real_device(row['dev_name']) def _fill_aditional_info(self, pv): pv['hide'] = False # If the device is not in any group we do not need this info # and we do not want it to show up in the device listing if not pv['pool_name']: pv['dev_used'] = '' pv['dev_free'] = '' def remove(self, devices): if len(devices) == 0: return command = ['pvremove'] command.extend(devices) self.run_lvm(command) class LvsInfo(LvmInfo): def __init__(self, *args, **kwargs): super(LvsInfo, self).__init__(*args, **kwargs) command = ["lvm", "lvs", "--separator", "|", "--noheadings", "--nosuffix", "--units", "k", "-o", "vg_name,lv_size,stripes,stripesize,segtype,lv_name,origin"] self.attrs = ['pool_name', 'vol_size', 'stripes', 'stripesize', 'type', 'lv_name', 'origin'] self.handle_fs = True self.mounts = misc.get_mounts('{0}/mapper'.format(DM_DEV_DIR)) self._parse_data(command) def _fill_aditional_info(self, lv): lv['dev_name'] = "{0}/{1}/{2}".format(DM_DEV_DIR, lv['pool_name'], lv['lv_name']) if lv['origin']: lv['hide'] = True lv['real_dev'] = misc.get_real_device(lv['dev_name']) sysfile = "/sys/block/{0}/dm/name".format( os.path.basename(lv['real_dev'])) # In some weird cases the "real" device might not be in /dev/dm-* # form (see tests). In this case constructed sysfile will not exist # so we just use real device name to search mounts. try: with open(sysfile, 'r') as f: lvname = f.readline()[:-1] lv['dm_name'] = "{0}/mapper/{1}".format(DM_DEV_DIR, lvname) except IOError: lv['dm_name'] = lv['real_dev'] if lv['real_dev'] in self.mounts: lv['mount'] = self.mounts[lv['real_dev']]['mp'] def __getitem__(self, name): if name in self.data.iterkeys(): return self.data[name] device = name if not os.path.exists(name): device = DM_DEV_DIR + "/" + name if not os.path.exists(device): return None device = misc.get_real_device(device) if device in self.data.iterkeys(): return self.data[device] return None def _data_index(self, row): return row['real_dev'] def _get_dev_name(self, lv): real = misc.get_real_device(lv) if real in self.data: return self.data[real]['dev_name'] else: return lv def remove(self, lv): vol = self[lv] if 'mount' in vol: if self.problem.check(self.problem.FS_MOUNTED, [vol['dev_name'], vol['mount']]): misc.do_umount(vol['mount']) lv = self._get_dev_name(lv) command = ['lvremove', lv] self.run_lvm(command) def resize(self, lv, size, resize_fs=True): lv = self._get_dev_name(lv) command = ['lvresize', '-L', str(size) + 'k', lv] if resize_fs: command.insert(1, '-r') self.run_lvm(command) def snapshot(self, lv, destination, name, size, user_set_size): lv = self._get_dev_name(lv) if not name: now = datetime.datetime.now() name = now.strftime("snap%Y%m%dT%H%M%S") command = ['lvcreate', '--size', str(size) + 'K', '--snapshot', '--name', name, lv] self.run_lvm(command) class SnapInfo(LvmInfo): def __init__(self, *args, **kwargs): super(SnapInfo, self).__init__(*args, **kwargs) command = ["lvm", "lvs", "--separator", "|", "--noheadings", "--nosuffix", "--units", "k", "-o", "vg_name,lv_size,stripes,stripesize,segtype," + "lv_name,origin,snap_percent"] self.attrs = ['pool_name', 'vol_size', 'stripes', 'stripesize', 'type', 'lv_name', 'origin', 'snap_size'] self.handle_fs = True self.mounts = misc.get_mounts('{0}/mapper'.format(DM_DEV_DIR)) self._parse_data(command) def _skip_data(self, row): if not row['origin']: return True else: return False def _data_index(self, row): return misc.get_real_device(row['dev_name']) def _fill_aditional_info(self, snap): snap['dev_name'] = "{0}/{1}/{2}".format(DM_DEV_DIR, snap['pool_name'], snap['lv_name']) snap['hide'] = False snap['snap_path'] = snap['dev_name'] size = float(snap['vol_size']) * float(snap['snap_size']) snap['snap_size'] = str(size / 100.00) snap['real_dev'] = misc.get_real_device(snap['dev_name']) sysfile = "/sys/block/{0}/dm/name".format( os.path.basename(snap['real_dev'])) # In some weird cases the "real" device might not be in /dev/dm-* # form (see tests). In this case constructed sysfile will not exist # so we just use real device name to search mounts. try: with open(sysfile, 'r') as f: lvname = f.readline()[:-1] snap['dm_name'] = "{0}/mapper/{1}".format(DM_DEV_DIR, lvname) except IOError: snap['dm_name'] = snap['real_dev'] if snap['dm_name'] in self.mounts: snap['mount'] = self.mounts[snap['dm_name']]['mp'] system-storage-manager-0.4/ssmlib/__init__.py0000664000175000017500000000140412171200774023051 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . __all__ = ["backends", "misc", "main", "problem"] system-storage-manager-0.4/ssmlib/main.py0000664000175000017500000017426012200423441022240 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # # System Storage Manager - ssm import re import os import sys import stat import argparse from ssmlib import misc from ssmlib import problem # Import backends from ssmlib.backends import lvm, crypt, btrfs, md EXTN = ['ext2', 'ext3', 'ext4'] SUPPORTED_FS = ['xfs', 'btrfs'] + EXTN SUPPORTED_BACKENDS = ['lvm', 'btrfs'] SUPPORTED_RAID = ['0', '1', '10'] os.environ['LC_NUMERIC'] = "C" # If you change this please change doc/conf.py as well VERSION = '0.4' # Should the script be run in interactive or non interactive mode ? try: SSM_NONINTERACTIVE = os.environ['SSM_NONINTERACTIVE'] if SSM_NONINTERACTIVE.upper() in ['YES', 'TRUE', '1']: SSM_NONINTERACTIVE = True elif SSM_NONINTERACTIVE.upper() in ['NO', 'FALSE', '0']: SSM_NONINTERACTIVE = False except KeyError: SSM_NONINTERACTIVE = not os.isatty(sys.stdout.fileno()) class Options(object): """ Structure that contains option setting allowing it to be passed between parts of ssm. """ def __init__(self): self.interactive = not SSM_NONINTERACTIVE self.verbose = False self.debug = False self.force = False self.yes = False self.config = None # Initialize problem set PR = problem.ProblemSet(Options()) # Name of the default pool try: DEFAULT_DEVICE_POOL = os.environ['DEFAULT_DEVICE_POOL'] except KeyError: DEFAULT_DEVICE_POOL = "device_pool" # Default back-end try: SSM_DEFAULT_BACKEND = os.environ['SSM_DEFAULT_BACKEND'] if SSM_DEFAULT_BACKEND not in SUPPORTED_BACKENDS: if PR.check(PR.BAD_ENV_VARIABLE, ['SSM_DEFAULT_BACKEND', SSM_DEFAULT_BACKEND]): SSM_DEFAULT_BACKEND = 'lvm' except KeyError: SSM_DEFAULT_BACKEND = 'lvm' # If this environment variable is set, ssm will only consider such devices, # pools and volumes which names start with this prefix. This is especially # useful for testing. try: SSM_PREFIX_FILTER = os.environ['SSM_PREFIX_FILTER'] PR.warn("SSM_PREFIX_FILTER is set to \'{0}\'".format(SSM_PREFIX_FILTER)) except KeyError: SSM_PREFIX_FILTER = None class Struct(object): def __init__(self): pass class StoreAll(argparse._StoreAction): """ Argparse class used to store all valid values. Valid values should not be empty or None """ def __call__(self, parser, namespace, values, option_string=None): for val in values[:]: if not val: values.remove(val) setattr(namespace, self.dest, values) class SetBackend(argparse._StoreAction): """ Action for the backend parameter, where we want to store provided in SSM_DEFAULT_BACKEND. """ def __call__(self, parser, namespace, values, option_string=None): # Set default backend to the provided value. All check should be # already done by argparse. global SSM_DEFAULT_BACKEND SSM_DEFAULT_BACKEND = values[0] setattr(namespace, self.dest, values) class FsInfo(object): """ Parse and store information about the file system. Methods specific for each file system should be part of this class """ def __init__(self, dev, options): self.data = {} self.options = options fstype = misc.get_fs_type(dev) if fstype not in [None, 'btrfs']: self.data['fs_type'] = fstype else: return self.fs_info = {} if fstype in EXTN: self.extN_get_info(dev) elif fstype == "xfs": self.xfs_get_info(dev) self.fstype = fstype self.device = dev self.mounted = False def _get_fs_func(self, func, *args, **kwargs): fstype = self.fstype if re.match("ext[2|3|4]", self.fstype): fstype = "extN" func = getattr(self, "{0}_{1}".format(fstype, func)) return func(*args, **kwargs) def fsck(self): return self._get_fs_func("fsck") def resize(self, *args, **kwargs): return self._get_fs_func("resize", *args, **kwargs) def get_info(self, *args, **kwargs): return self._get_fs_func("get_info", *args, **kwargs) def extN_get_info(self, dev): command = ["tune2fs", "-l", dev] if not misc.check_binary(command[0]): return output = misc.run(command)[1] for line in output.split("\n")[1:]: array = line.split(":") if len(array) == 2: self.fs_info[array[0]] = array[1].lstrip() bsize = int(self.fs_info['Block size']) bcount = int(self.fs_info['Block count']) rbcount = int(self.fs_info['Reserved block count']) fbcount = int(self.fs_info['Free blocks']) self.data['fs_size'] = bcount * bsize / 1024 self.data['fs_free'] = (fbcount - rbcount) * bsize / 1024 self.data['fs_used'] = (bcount - fbcount) * bsize / 1024 def extN_fsck(self): command = ['fsck.{0}'.format(self.fstype), '-f', '-n'] if not misc.check_binary(command[0]): PR.warn("\'{0}\' tool does not exist. ".format(command[0]) + "File system will not be checked") return 1 if self.options.force: command.append('-f') if self.options.verbose: command.append('-v') command.append(self.device) return misc.run(command, stdout=True, can_fail=True)[0] def extN_resize(self, new_size=None): command = ['resize2fs', self.device] if not misc.check_binary(command[0]): PR.warn("\'{0}\' tool does not exist. ".format(command[0]) + "File system will not be resized") return 1 if self.options.force: command.insert(1, "-f") if self.options.verbose: command.insert(1, "-p") if new_size: command.append(new_size) # Ext3/4 can resize offline in both directions, but It can not shrink # the file system while online. In addition ext2 can only resize # offline. if self.mounted and (self.fstype == "ext2" or new_size < self.data['fs_size']): raise Exception( "{0} is mounted on {1}".format(self.device, self.mounted) + " In this case, mounted file system can not be resized.") ret = self.fsck() if ret: raise Exception("File system on {0} is not ".format(self.device) + "clean, I will not attempt to resize it. Please," + "fix the problem first.") misc.run(command, stdout=True) def xfs_get_info(self, dev): command = ["xfs_db", "-r", "-c", "sb", "-c", "print", dev] if not misc.check_binary(command[0]): return output = misc.run(command)[1] for line in output.split("\n")[1:]: array = line.split("=") if len(array) == 2: self.fs_info[array[0].rstrip()] = array[1].lstrip() bsize = int(self.fs_info['blocksize']) bcount = int(self.fs_info['dblocks']) lbcount = int(self.fs_info['logblocks']) bcount -= lbcount agcount = int(self.fs_info['agcount']) fbcount = int(self.fs_info['fdblocks']) fbcount -= 4 + (4 + agcount) self.data['fs_size'] = bcount * bsize / 1024 self.data['fs_free'] = fbcount * bsize / 1024 self.data['fs_used'] = (bcount - fbcount) * bsize / 1024 def xfs_fsck(self): command = ['xfs_check'] if not misc.check_binary(command[0]): PR.warn("\'{0}\' tool does not exist. ".format(command[0]) + "File system will not be checked") return 1 if self.options.verbose: command.append('-v') command.append(self.device) return misc.run(command, stdout=True, can_fail=True)[0] def xfs_resize(self, new_size=None): command = ['xfs_growfs', self.device] if not misc.check_binary(command[0]): PR.warn("\'{0}\' tool does not exist. ".format(command[0]) + "File system will not be resized") return 1 if new_size: command.insert(1, ['-D', new_size + 'K']) if not self.mounted: raise Exception("Xfs file system on {0}".format(self.device) + " has to be mounted to perform an resize.") elif new_size and new_size < self.data['fs_size']: raise Exception("Xfs file system can not shrink.") else: misc.run(command, stdout=True) class DeviceInfo(object): """ Parse and store information about the devices present in the system. The main source of information are /proc/partitions, /proc/mounts and /proc/swaps. self.data should be appended to since there might be other data present which will add more information about devices, usually provided from backends. Important thing is that we hide all dm devices here, since they might really be a volumes. We let backend decide whether the device should be listed as device or not simply by setting 'hide' to True/False. """ def __init__(self, options, data=None): self.type = 'device' self.data = data or {} self.attrs = ['major', 'minor', 'dev_size', 'dev_name'] self.options = options hide_dmnumbers = [] for name in ['device-mapper', 'sr', 'md']: hide_dmnumbers.append(misc.get_dmnumber(name)) mounts = misc.get_mounts('/dev/') swaps = misc.get_swaps() for items in misc.get_partitions(): devices = dict(zip(self.attrs, items)) devices['vol_size'] = devices['dev_size'] devices['dev_name'] = "/dev/" + devices['dev_name'] if devices['major'] in hide_dmnumbers: devices['hide'] = True if devices['dev_name'] in self.data: if 'hide' in self.data[devices['dev_name']] and \ not self.data[devices['dev_name']]['hide']: devices['hide'] = False self.data[devices['dev_name']].update(devices) else: self.data[devices['dev_name']] = devices if devices['dev_name'] in mounts: devices['mount'] = mounts[devices['dev_name']]['mp'] for item in swaps: if item[0] in self.data: self.data[item[0]]['mount'] = "SWAP" for i, dev in enumerate(self.data.itervalues()): if 'minor' in dev and dev['minor'] != '0': continue part = 0 for a, d in enumerate(self.data.values()): if a == i: continue try: if dev['major'] != d['major']: continue except KeyError: continue if re.search(dev['dev_name'], d['dev_name']): d['partition'] = True d['type'] = 'part' part += 1 dev['partitioned'] = part if part > 0: dev['mount'] = "PARTITIONED" dev['type'] = 'disk' def set_globals(self, options): self.options = options def __iter__(self): for item in sorted(self.data.iterkeys()): yield item def __getitem__(self, name): device = misc.get_real_device(name) if device in self.data.iterkeys(): return self.data[device] return None class Item(object): """ Meta object which provides encapsulation for all devices, pools and volumes, so we can work with them as with the usual objects without the need to call Dev, Pool or Vol methods directly. """ def __init__(self, obj, name): self.obj = obj self.name = name self.type = obj.type @property def data(self): return self.obj[self.name] def __getattr__(self, func_name): func = getattr(self.obj, func_name) def _new_func(*args, **kwargs): if args and kwargs: return func(self.name, *args, **kwargs) elif kwargs: return func(self.name, **kwargs) elif args: return func(self.name, *args) else: return func(self.name) return _new_func def __getitem__(self, key): if key not in self.data and \ re.match(r"fs_.*", key): self._fill_fs_info() try: ret = self.data[key] except KeyError: ret = "" return ret def __contains__(self, item): if self[item]: return True else: return False def _fill_fs_info(self): if 'dm_name' in self.data: name = self.data['dm_name'] elif 'real_dev' in self.data: name = self.data['real_dev'] else: name = self.data['dev_name'] fs = FsInfo(name, self.obj.options) try: fs.mounted = self.data['mount'] except KeyError: fs.mounted = "" self.data.update(fs.data) self.data['fs_info'] = fs def exists(self): if self.name in self.obj: return True else: return False class Storage(object): """ Template class to use for storing information about Pools, Volumes and Devices from different backends. This simplify things a lot since we do not have to manually walk through all the backends, but this class will do this for us. """ def __init__(self, options): self._data = {} self.header = None self.attrs = None self.types = None self.set_globals(options) def __iter__(self): for source in self._data.itervalues(): for item in source: yield Item(source, item) def __contains__(self, item): if self[item]: return True else: return False def __getitem__(self, name): for source in self._data.itervalues(): item = source[name] if item: return Item(source, name) return None def reinitialize(self): self.__init__(self.options) def _apply_prefix_filter(self): """ If SSM_PREFIX FILTER is set, remove all items which basenames does not start with SSM_PREFIX_FILTER prefix. This is useful especially for testing so that ssm see only relevant devices and does not screw real system storage configuration. """ if not SSM_PREFIX_FILTER: return reg = re.compile("^{0}".format(SSM_PREFIX_FILTER)) for source in self._data.itervalues(): for item in source: if reg.search(os.path.basename(item)): continue if 'pool_name' in source.data[item] and \ reg.search(source.data[item]['pool_name']): continue if 'dm_name' in source.data[item] and \ reg.search(os.path.basename(source.data[item]['dm_name'])): continue del source.data[item] def get_backend(self, name): return self._data[name] def set_globals(self, options): self.options = options if self._data is None: return for source in self._data.itervalues(): source.options = options def filesystems(self): for item in self: if 'fs_type' in item: yield item def ptable(self, cond=None, more_data=None, cond_func=None): """ Print information table about the source (devices, pools, volumes) using the predefined variables (below). cond, or cond_func can be provided to decide which items not to print out. self.header - list of headers for the table self.attrs - list of attribute keys to print out self.types - types of the attributes to print out (str, or float/int) """ lines = [] fmt = "" if cond == "fs_only": iterator = self.filesystems() else: iterator = self # Keep track of used columns. Then we only print out columns with # values. columns = [False] * len(self.attrs) len_matrix = [] index = 0 # Gather all lines which are going to be printed into the list # and create matrix of attribute lengths. # Iterate through all items in each data source first. for data in misc.chain(iterator, more_data or []): if (cond_func and not cond_func(data)) or 'hide' in data: continue len_matrix.append([len(self.header[i]) for i in range(len(self.header))]) line = () # Iterate through all attributes in each item for i, attr in enumerate(self.attrs): if self.types[i] in (float, int): item = misc.humanize_size(data[attr]) elif attr + "_print" in data: item = data[attr + "_print"] else: item = data[attr] len_matrix[index][i] = len(item) line += item, if len(item) > 0: columns[i] = True lines.append(line) index += 1 if len(lines) == 0: return header = [item for item in misc.compress(self.header, columns)] alignment = list([(len(self.header[i])) for i in range(len(self.header))]) term_width = misc.terminal_size()[0] # Update matrix of attribute lengths and construct the final list # of alignment for each column in the table. for index in range(len(len_matrix)): line = None # Find maximum length for each column for a, array in enumerate(len_matrix): for i, item in enumerate(array): if not columns[i]: alignment[i] = 0 continue if item > alignment[i]: alignment[i] = item line = a # Check the overall line length and if it is longer then the # actual terminal width we can wrap the line right after the # first attribute. Simply set the alignment to the smaller # possible and let recalculate the list of column alignments. # Note that when even with the line wrap we would still exceed # the terminal width, then there is nothing we can do about it # so do not bother with line wrapping at all since it would # only screw the formatting even more. length = sum(alignment) + 2 * len(header) - 2 if length > term_width and \ (length - term_width) < (alignment[0] - len(header[0])) and \ line is not None: alignment[0] = len(header[0]) len_matrix[line][0] = len(header[0]) else: break # Get the actual line width width = sum(misc.compress(alignment, columns)) + 2 * len(header) - 2 pos = 0 # Use column alignments list to construct formatting string for each # line in the table. Note that some lines might be wrapped later on. for i, t in enumerate(self.types): if not columns[i]: continue if t in (float, int): fmt += "{{{0}:>{1}}} ".format(pos, alignment[i]) else: # Do not append additional spaces if this is the last item if i == len(header) - 1: fmt += "{{{0}:{1}}}".format(pos, alignment[i]) else: fmt += "{{{0}:{1}}} ".format(pos, alignment[i]) pos += 1 print "-" * width print fmt.format(*tuple(header)) print "-" * width # Now print each line of the table. When the first attribute of the # line is longer than it should be we know that we have to wrap the # line. for i, line in enumerate(lines): line = misc.compress(line, columns) tmp1 = line.next() if len(tmp1) > alignment[0]: print tmp1 print fmt.format('', *line) else: print fmt.format(tmp1, *line) print "-" * width class Pool(Storage): """ Store Pools from all the backends. When new backend is added into the ssm it should be registered withing this class with appropriate name. """ def __init__(self, *args, **kwargs): super(Pool, self).__init__(*args, **kwargs) try: self._data['lvm'] = lvm.VgsInfo(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about LVM pools") try: self._data['btrfs'] = btrfs.BtrfsPool(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about btrfs pools") backend = self.get_backend(SSM_DEFAULT_BACKEND) self.default = Item(backend, backend.default_pool_name) self.header = ['Pool', 'Type', 'Devices', 'Free', 'Used', 'Total'] self.attrs = ['pool_name', 'type', 'dev_count', 'pool_free', 'pool_used', 'pool_size'] self.types = [str, str, str, float, float, float] self._apply_prefix_filter() class Devices(Storage): """ Store Devices from all the backends. When new backend is added into the ssm it should be registered withing this class with appropriate name. If the backend only have new information about the device which is already discovered by the DeviceInfo() class then it should just add the information into the existing devices by passing the data. But if the backed discovers new devices, it should add them as a new entry. """ def __init__(self, *args, **kwargs): super(Devices, self).__init__(*args, **kwargs) try: my_lvm = lvm.PvsInfo(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about LVM physical volumes") try: my_btrfs = btrfs.BtrfsDev(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about btrfs devices") my_btrfs = Struct() my_btrfs.data = {} try: my_md = md.MdRaidDevice(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about MD devices") my_md = Struct() my_md.data = {} self._data['dev'] = DeviceInfo(data=dict(my_lvm.data.items() + my_btrfs.data.items() + my_md.data.items()), options=self.options) self.header = ['Device', 'Free', 'Used', 'Total', 'Pool', 'Mount point'] self.attrs = ['dev_name', 'dev_free', 'dev_used', 'dev_size', 'pool_name', 'mount'] self.types = [str, float, float, float, str, str] self._apply_prefix_filter() class Volumes(Storage): """ Store Volumes from all the backends. When new backend is added into the ssm it should be registered withing this class with appropriate name. """ def __init__(self, *args, **kwargs): super(Volumes, self).__init__(*args, **kwargs) try: self._data['lvm'] = lvm.LvsInfo(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about LVM volumes") try: self._data['crypt'] = crypt.DmCryptVolume(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about crypt volumes") try: self._data['btrfs'] = btrfs.BtrfsVolume(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about btrfs volumes") try: self._data['md'] = md.MdRaidVolume(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about md raid volumes") self.header = ['Volume', 'Pool', 'Volume size', 'FS', 'FS size', 'Free', 'Type', 'Mount point'] self.attrs = ['dev_name', 'pool_name', 'vol_size', 'fs_type', 'fs_size', 'fs_free', 'type', 'mount'] self.types = [str, str, float, str, float, float, str, str] self._apply_prefix_filter() class Snapshots(Storage): """ Store Snapshots from all the backends that supports snapshotting. When the snapshotting support is added into the backed it should be registered within this class with appropriate name. """ def __init__(self, *args, **kwargs): super(Snapshots, self).__init__(*args, **kwargs) try: self._data['lvm'] = lvm.SnapInfo(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about LVM snapshots") try: self._data['btrfs'] = btrfs.BtrfsSnap(options=self.options) except RuntimeError, err: PR.warn(err) PR.warn("Can not get information about btrfs snapshots") self.header = ['Snapshot', 'Origin', 'Pool', 'Volume size', 'Size', 'Type', 'Mount point'] self.attrs = ['dev_name', 'origin', 'pool_name', 'vol_size', 'snap_size', 'type', 'mount'] self.types = [str, str, str, float, float, str, str] self._apply_prefix_filter() class StorageHandle(object): """ The main class where all the magic is done. All the commands provided by ssm have its appropriate functions here which are then called by argparse. """ def __init__(self, options=Options()): self._mpoint = None self._dev = None self._pool = None self._volumes = None self._snapshots = None self.set_globals(options) self.options = options def set_globals(self, options): if self._dev: self.dev.set_globals(options) if self._volumes: self.vol.set_globals(options) if self._pool: self.pool.set_globals(options) if self._snapshots: self.snap.set_globals(options) self.options = options @property def dev(self): if self._dev: return self._dev self._dev = Devices(options=self.options) return self._dev def reinit_dev(self): if self._dev: self._dev.reinitialize() @property def pool(self): if self._pool: return self._pool self._pool = Pool(options=self.options) return self._pool def reinit_pool(self): if self._pool: self._pool.reinitialize() @property def vol(self): if self._volumes: return self._volumes self._volumes = Volumes(options=self.options) return self._volumes def reinit_vol(self): if self._volumes: self._volumes.reinitialize() @property def snap(self): if self._snapshots: return self._snapshots self._snapshots = Snapshots(options=self.options) return self._snapshots def reinit_snap(self): if self._snapshots: self._snapshots.reinitialize() def _create_fs(self, fstype, volume): """ Create a file system 'fstype' on the 'volume'. """ command = ["mkfs.{0}".format(fstype), volume] if not misc.check_binary(command[0]): PR.warn("\'{0}\' tool does not exist. ".format(command[0]) + "File system will not be created") return 1 if self.options.force: if fstype == 'xfs': command.insert(1, '-f') if fstype in EXTN: command.insert(1, '-F') if self.options.verbose: if fstype in EXTN: command.insert(1, '-v') return misc.run(command, stdout=True)[0] def _do_mount(self, volume, options=None, directory=None): if directory is None: directory = self._mpoint try: volume.mount(directory, options) except AttributeError: misc.do_mount(volume['real_dev'], directory, options) def check(self, args): """ Check the file system on the volume. FsInfo is used for that purpose, except for btrfs. """ err = 0 checked = 0 for fs in args.device: print "Checking {0} file system on \'{1}\':".format(fs.fstype, fs.device), if fs.mounted: print try: if PR.check(PR.FS_MOUNTED, [fs.device, fs.mounted]): misc.do_umount(fs.device) except problem.FsMounted: PR.warn("Unable to check file system " + "\'{0}\' on volume \'{1}\'".format(fs.fstype, fs.device)) continue ret = fs.fsck() checked += 1 err += ret if ret: print "FAIL" else: print "OK" if checked == 0: PR.error("Nothing was checked") if err > 0: PR.warn("Some file system(s) contains errors. Please run " + "the appropriate fsck utility") def _filter_device_list(self, args, have_size=None, new_size=None): """ Filter the args.device list. Only items which have to be added to pool are left in the args.device list. Function returns touple (have_size, devices) where have_size is the size of the devices which will be added to the pool (args.device) plus optional have_size argument. Devices is the list of devices which can be used for volume creation, it means that it does not contain devices which are used in other pools and are not removed from it in this function. """ if have_size is None: have_size = 0.0 else: have_size = float(have_size) changed = False devices = args.device args.device = [] for dev in devices[:]: if self.dev[dev] and 'pool_name' in self.dev[dev] and \ self.dev[dev]['pool_name'] != args.pool.name: if PR.check(PR.DEVICE_USED, [dev, self.dev[dev]['pool_name']]): remove_args = Struct() remove_args.all = False remove_args.items = [self.dev[dev]] if self.remove(remove_args): args.device.append(dev) changed = True elif new_size is None: PR.error("Device \'{0}\' can not be used!".format(dev)) else: devices.remove(dev) continue # This is tricky. We are going to create or resize a device # so we might actually need the device for create (or resize) # to finish successfully. Create and resize should check # whether is has enough space and fail if it does not. The # problem is, when the size was not specified, then the result # would be different than what user expected, so we should fail # right away. elif new_size is None: PR.error("Device \'{0}\' can not be used!".format(dev)) else: devices.remove(dev) continue if not self.dev[dev] or 'pool_name' not in self.dev[dev]: # Check signature of existing file system on the device # and ask user whether to use it or not. signature = misc.get_fs_type(dev) if signature and \ not PR.check(PR.EXISTING_FILESYSTEM, [signature, dev]): devices.remove(dev) continue else: args.device.append(dev) if changed: self.reinit_dev() for dev in devices: if not self.dev[dev]: have_size += misc.get_file_size(dev) else: try: have_size += float(self.dev[dev]['dev_free']) except ValueError: have_size += float(self.dev[dev]['dev_size']) return have_size, devices def resize(self, args): """ Resize the volume to the given size. If more devices are provided as arguments, it will be added into the pool prior to the volume resize only if the space in the pool is not sufficient. That said, only the number of devices are added into the pool to be able to cover the resize. """ args.pool = self.pool[args.volume['pool_name']] vol_size = float(args.volume['vol_size']) if not args.size: new_size = vol_size elif args.size[0] == '+': new_size = vol_size + float(args.size[1:]) elif args.size[0] == '-': new_size = vol_size + float(args.size) else: new_size = float(args.size) size_change = new_size - vol_size fs = True if 'fs_type' in args.volume else False if new_size <= 0: PR.error("New volume size \'{0} KB\' is too small".format(new_size)) if vol_size == new_size: # Try to grow the file system, since there is nothing to # do with the volume itself. if fs: args.volume['fs_info'].resize() else: PR.check(PR.RESIZE_ALREADY_MATCH, [args.volume.name, new_size]) return # Backend might not support pooling if args.pool is None: pool_free = None pool_name = "none" else: pool_free = float(args.pool['pool_free']) pool_name = args.pool.name have_size, devices = self._filter_device_list(args, pool_free, new_size) if have_size < size_change: PR.check(PR.RESIZE_NOT_ENOUGH_SPACE, [pool_name, args.volume.name, new_size]) elif len(args.device) > 0 and new_size > vol_size: self.add(args, True) if new_size != vol_size: args.volume.resize(new_size, fs) def create(self, args): """ Create new volume (or subvolume in case of btrfs) using the devices provided as arguments. If the device is not in the selected pool, then add() is called on the pool prior to create(). """ # Get the size in kilobytes if args.size: args.size = misc.get_real_size(args.size) if self._mpoint and not (args.fstype or args.pool.type == 'btrfs'): if PR.check(PR.CREATE_MOUNT_NOFS, self._mpoint): self._mpoint = None devices = args.device if args.pool.exists(): pool_free = float(args.pool['pool_free']) else: pool_free = 0.0 # If devices were provided we should only use those if len(devices) > 0: pool_free = None have_size, devices = self._filter_device_list(args, pool_free, args.size) # Currently we do not allow setting subvolume size with btrfs. This # should change in the future (quotas maybe) so the check should # be removed or pushed to the backend itself. if args.size and have_size < float(args.size) and \ not (args.pool.exists() and args.pool.type == 'btrfs'): if PR.check(PR.CREATE_NOT_ENOUGH_SPACE, [have_size, args.pool.name]): args.size = None # When the pool does not exist and there is no device usable # for creating the new pool, then there is no point of trying to # create a volume, since it would fail in the backend anyway. if not args.pool.exists() and len(devices) == 0: PR.check(PR.NO_DEVICES, args.pool.name) if have_size == 0: PR.error("Not enough space ({0} KB) to".format(have_size) + "to create volume") # Number of stripes must not exceed number of devices within the pool if args.stripes and len(devices) > 0 and args.stripes > len(devices): PR.error("Number of stripes ({0}) ".format(args.stripes) + "must not exceed number of devices " + "({0})".format(len(devices))) elif args.stripes and len(devices) == 0 and args.pool.exists(): tmp = int(args.pool['dev_count']) if args.stripes > tmp: PR.error("Number of stripes ({0}) ".format(args.stripes) + "must not exceed number of devices " + "({0})".format(tmp)) # If we want btrfs pool and it does not exist yet, we do not # want to call add since it would create it. Note that when # btrfs pool is created the new btrfs volume is created as well # because it is actually the same thing if len(args.device) > 0 and \ not (not args.pool.exists() and args.pool.type == 'btrfs'): self.add(args, True) if args.raid: raid = {'level': args.raid, 'stripesize': args.stripesize, 'stripes': args.stripes} else: raid = None lvname = args.pool.create(devs=devices, size=args.size, raid=raid, name=args.name) if args.fstype and args.pool.type != 'btrfs': if self._create_fs(args.fstype, lvname) != 0: self._mpoint = None if self._mpoint: self.reinit_vol() self._do_mount(self.vol[lvname]) def list(self, args): """ List devices, pools, volumes """ if not args.type: self.dev.ptable() self.pool.ptable() self.vol.ptable(more_data=self.dev.filesystems()) self.snap.ptable() elif args.type in ['fs', 'filesystems']: self.vol.ptable(more_data=self.dev.filesystems(), cond="fs_only") elif args.type in ['dev', 'devices']: self.dev.ptable() elif args.type in ["volumes", "vol"]: self.vol.ptable(more_data=self.dev.filesystems()) elif args.type in ["pool", "pools"]: self.pool.ptable() elif args.type in ['snap', 'snapshots']: self.snap.ptable() def add(self, args, skip_check=False): """ Add devices into the pool """ if not skip_check: for dev in args.device[:]: item = self.dev[dev] if item and 'pool_name' in item: if item['pool_name'] == args.pool.name: args.device.remove(dev) continue if PR.check(PR.DEVICE_USED, [item.name, item['pool_name']]): remove_args = Struct() remove_args.all = False remove_args.items = [item] if not self.remove(remove_args): args.device.remove(item.name) else: args.device.remove(dev) else: # Check signature of existing file system on the device # and as user whether to use it or not. signature = misc.get_fs_type(dev) if signature and \ not PR.check(PR.EXISTING_FILESYSTEM, [signature, dev]): args.device.remove(dev) continue if args.pool.exists(): if len(args.device) > 0: args.pool.extend(args.device) else: PR.check(PR.NO_DEVICES, args.pool.name) else: if len(args.device) > 0: args.pool.new(args.device) else: PR.check(PR.NO_DEVICES, args.pool.name) def remove(self, args): """ Remove the all the items, or all pools if all argument is specified. Items could be the devices, pools or volumes. """ ret = True removed = 0 if args.all: for pool in self.pool: try: pool.remove() removed += 1 except (RuntimeError, problem.SsmError), ex: PR.info("Unable to remove '{0}'".format(pool['pool_name'])) ret = False if removed == 0: PR.error("Nothing was removed!") return ret if len(args.items) == 0: err = "too few arguments" raise argparse.ArgumentTypeError(err) for item in args.items: try: if isinstance(item.obj, DeviceInfo): pool = self.pool[item['pool_name']] if pool: pool.reduce(item.name) removed += 1 continue else: PR.error("It is not clear what do you want " + "to achieve by removing " + "{0}!".format(item.name)) item.remove() removed += 1 except (RuntimeError, problem.SsmError), ex: PR.info("Unable to remove '{0}'".format(item.name)) ret = False if removed == 0: PR.error("Nothing was removed") return ret def snapshot(self, args): """ Create a new snapshot of the volume. """ pool = self.pool[args.volume['pool_name']] vol_size = float(args.volume['vol_size']) pool_free = float(pool['pool_free']) if not args.size: # We'll ceate snapshot of the size of 20% of the original volume snap_size = vol_size * 0.20 user_set_size = False else: snap_size = float(misc.get_real_size(args.size)) user_set_size = True if pool_free < snap_size: snap_size = pool_free if snap_size <= 0 and pool.type != 'btrfs': PR.error("Not enough space ({0} KB) to".format(pool_free) + "to create snapshot") args.volume.snapshot(args.dest, args.name, snap_size, user_set_size) def mount(self, args): """ Mount a volume at given directory. """ vol = self.vol[args.volume] try: if vol: self._do_mount(vol, args.options, args.directory) else: misc.do_mount(args.volume, args.directory, args.options) except RuntimeError: PR.error("Could not mount {0} to ".format(args.volume) + "{0} with options \'{1}\'".format(args.directory, args.options)) def is_fs(self, device): real = misc.get_real_device(device) vol = self.vol[real] if vol and 'fs_type' in vol: return vol['fs_info'] dev = self.dev[real] if dev and 'fs_type' in dev: return dev['fs_info'] err = "'{0}' does not contain valid file system".format(real) raise argparse.ArgumentTypeError(err) def _find_device_record(self, path): """ Try to find device name for path, which is used as an key in self.dev - this is usually the real block device, but in some rare cases (dmsetup) we can have real block device which name does not correspond with what we have in /proc/partitions """ if self.dev[path]: return path minor = os.minor(os.lstat(path).st_rdev) dm_dev = "/dev/dm-{0}".format(minor) if self.dev[dm_dev]: return dm_dev else: return path def check_create_item(self, path): """ Check the create argument for block device or directory. """ if not self._mpoint: try: mode = os.stat(path).st_mode except OSError: err = "'{0}' does not exist.".format(path) raise argparse.ArgumentTypeError(err) if stat.S_ISDIR(mode): self._mpoint = path return return self.get_bdevice(path) def get_bdevice(self, path): path = is_bdevice(path) return self._find_device_record(path) def is_pool(self, string): pool = self.pool[string] if not pool: if string: self.pool.default.name = string pool = self.pool.default return pool def is_volume(self, string): vol = self.vol[string] if vol: return vol dev = self.dev[string] if dev and 'fs_type' in dev: return dev err = "'{0}' is not a valid volume to resize".format(string) raise argparse.ArgumentTypeError(err) def can_snapshot(self, string): vol = self.vol[string] have = False if not vol: for vol in self.vol: if 'mount' in vol and (vol['mount'] == string.rstrip("/")): have = True break else: have = True if not have: err = "'{0}' is not valid volume nor mount point.".format(string) raise argparse.ArgumentTypeError(err) else: err = "Backend for '{0}' ".format(string) + \ "does not support snapshotting." try: if not getattr(vol, "snapshot"): raise argparse.ArgumentTypeError(err) else: return vol except AttributeError: raise argparse.ArgumentTypeError(err) def check_remove_item(self, string): """ Check the remove argument for volume, pool or device. """ volume = self.vol[string] if volume: return volume pool = self.pool[string] if pool: return pool device = self.dev[string] if device: return device else: try: path = self.get_bdevice(string) device = self.dev[path] if device: return device except argparse.ArgumentTypeError: pass for vol in self.vol: if 'mount' in vol and (vol['mount'] == string.rstrip("/")): return vol err = "'{0}' is not valid pool nor volume".format(string) raise argparse.ArgumentTypeError(err) def valid_resize_size(size): """ Validate that the 'size' is usable as resize argument. It means that the 'size' argument should be in this format: [+|-]number[unit]. It returns the number with the provided sign (even with the plus sign) converted to the kilobytes. Is no unit is specified, default is kilobytes. >>> valid_resize_size("3.14") '3.14' >>> valid_resize_size("+3.14") '+3.14' >>> valid_resize_size("-3.14") '-3.14' >>> valid_resize_size("3.14k") '3.14' >>> valid_resize_size("+3.14K") '+3.14' >>> valid_resize_size("-3.14k") '-3.14' >>> valid_resize_size("3.14G") '3292528.64' >>> valid_resize_size("+3.14g") '+3292528.64' >>> valid_resize_size("-3.14G") '-3292528.64' >>> valid_resize_size("G") Traceback (most recent call last): ... ArgumentTypeError: 'G' is not valid number for the resize. """ try: return misc.get_real_size(size) except Exception: err = "'{0}' is not valid number for the resize.".format(size) raise argparse.ArgumentTypeError(err) def is_directory(self, string): try: mode = os.stat(string).st_mode except OSError: err = "'{0}' does not exist.".format(string) raise argparse.ArgumentTypeError(err) if stat.S_ISDIR(mode): return string else: err = "'{0}' is not directory.".format(string) raise argparse.ArgumentTypeError(err) def is_bdevice(path): path = misc.get_real_device(path) try: mode = os.lstat(path).st_mode except OSError: err = "'{0}' is not valid block device".format(path) raise argparse.ArgumentTypeError(err) if not stat.S_ISBLK(mode): err = "'{0}' is not valid block device".format(path) raise argparse.ArgumentTypeError(err) return path def is_supported_fs(fs): if fs in SUPPORTED_FS: return fs err = "'{0}' is not supported file system".format(fs) raise argparse.ArgumentTypeError(err) class SsmParser(object): """ This class is used to generate argparse parser and run the actual parsing. """ def __init__(self, storage, prog=None): self.storage = storage self.parser = self._get_parser_global(prog) self.subcommands = self.parser.add_subparsers(title="Commands") self.parser_check = self._get_parser_check() self.parser_resize = self._get_parser_resize() self.parser_create = self._get_parser_create() self.parser_list = self._get_parser_list() self.parser_add = self._get_parser_add() self.parser_remove = self._get_parser_remove() self.parser_snapshot = self._get_parser_snapshot() self.parser_mount = self._get_parser_mount() self.args = None def parse(self): self.args = self.parser.parse_args() return self.args def _get_parser_global(self, prog): """ General ssm options """ parser = argparse.ArgumentParser( description="System Storage Manager", prog=prog, epilog='''To get help for particular command please specify \'%(prog)s [command] -h\'.''') parser.add_argument('--version', action='version', version='%(prog)s {0}'.format(VERSION)) parser.add_argument('-v', '--verbose', help="Show aditional information while executing.", action="store_true") parser.add_argument('-f', '--force', help="Force execution in the case where ssm has some " + "doubts or questions.", action="store_true") parser.add_argument('-b', '--backend', nargs=1, metavar='BACKEND', help="Choose backend to use. Currently you can choose from " + "({0}).".format(",".join(SUPPORTED_BACKENDS)), choices=SUPPORTED_BACKENDS, action=SetBackend) return parser def _get_parser_check(self): """ Check command """ parser_check = self.subcommands.add_parser("check", help="Check consistency of the file system on the device.") parser_check.add_argument('device', nargs='+', help="Device with file system to check.", type=self.storage.is_fs) parser_check.set_defaults(func=self.storage.check) return parser_check def _get_parser_resize(self): """ Resize command """ parser_resize = self.subcommands.add_parser("resize", help="Change or set the volume and file system size.") parser_resize.add_argument("volume", help="Volume to resize.", type=self.storage.is_volume) parser_resize.add_argument('-s', '--size', help='''New size of the volume. With the + or - sign the value is added to or subtracted from the actual size of the volume and without it, the value will be set as the new volume size. A size suffix of [k|K] for kilobytes, [m|M] for megabytes, [g|G] for gigabytes, [t|T] for terabytes or [p|P] for petabytes is optional. If no unit is provided the default is kilobytes.''', type=valid_resize_size) parser_resize.add_argument("device", nargs='*', help='''Devices to use for extending the volume. If the device is not in any pool, it is added into the volume's pool prior to the extension. Note that only really needed number of devices are added into the pool prior the resize.''') parser_resize.set_defaults(func=self.storage.resize) return parser_resize def _get_parser_create(self): """ Create command """ parser_create = self.subcommands.add_parser("create", help="Create a new volume with defined parameters.") parser_create.add_argument('-s', '--size', help='''Gives the size to allocate for the new logical volume A size suffix K|k, M|m, G|g, T|t, P|p, E|e can be used to define 'power of two' units. If no unit is provided, it defaults to kilobytes. This is optional if if not given maximum possible size will be used.''') parser_create.add_argument('-n', '--name', help='''The name for the new logical volume. This is optional and if omitted, name will be generated by the corresponding backend.''') parser_create.add_argument('--fstype', help='''Gives the file system type to create on the new logical volume. Supported file systems are (ext3, ext4, xfs, btrfs). This is optional and if not given file system will not be created.''', type=is_supported_fs) parser_create.add_argument('-r', '--raid', choices=SUPPORTED_RAID, metavar="LEVEL", help='''Specify a RAID level you want to use when creating a new volume. Note that some backends might not implement all supported RAID levels. This is optional and if no specified, linear volume will be created. You can choose from the following list of supported levels ({0}).'''.format(",".join(SUPPORTED_RAID))) parser_create.add_argument('-I', '--stripesize', type=int, help='''Gives the number of kilobytes for the granularity of stripes. This is optional and if not given, backend default will be used. Note that you have to specify RAID level as well.''') parser_create.add_argument('-i', '--stripes', type=int, help='''Gives the number of stripes. This is equal to the number of physical volumes to scatter the logical volume. This is optional and if stripesize is set and multiple devices are provided stripes is determined automatically from the number of devices. Note that you have to specify RAID level as well.''') parser_create.add_argument('-p', '--pool', default="", help="Pool to use to create the new volume.", type=self.storage.is_pool) parser_create.add_argument('device', nargs='*', help='''Devices to use for creating the volume. If the device is not in any pool, it is added into the pool prior the volume creation.''', type=self.storage.check_create_item, action=StoreAll) parser_create.add_argument('mount', nargs='?', help='''Directory to mount the newly create volume to.''') parser_create.set_defaults(func=self.storage.create) return parser_create def _get_parser_list(self): """ List command """ parser_list = self.subcommands.add_parser("list", help='''List information about all detected, devices, pools, volumes and snapshots in the system.''') parser_list.add_argument('type', nargs='?', choices=["volumes", "vol", "dev", "devices", "pool", "pools", "fs", "filesystems", "snap", "snapshots"]) parser_list.set_defaults(func=self.storage.list) return parser_list def _get_parser_add(self): """ Add command """ parser_add = self.subcommands.add_parser("add", help='''Add one or more devices into the pool.''') parser_add.add_argument('-p', '--pool', default="", help='''Pool to add device into. If not specified the default pool is used.''', type=self.storage.is_pool) parser_add.add_argument('device', nargs='+', help="Devices to add into the pool.", type=self.storage.get_bdevice, action=StoreAll) parser_add.set_defaults(func=self.storage.add) return parser_add def _get_parser_remove(self): """ Remove command """ parser_remove = self.subcommands.add_parser("remove", help='''Remove devices from the pool, volumes or pools.''') parser_remove.add_argument('-a', '--all', action="store_true", help="Remove all pools in the system.") parser_remove.add_argument('items', nargs='*', help="Items to remove. Item could be device, pool, or volume.", type=self.storage.check_remove_item) parser_remove.set_defaults(func=self.storage.remove) return parser_remove def _get_parser_snapshot(self): """ Snapshot command """ parser_snapshot = self.subcommands.add_parser("snapshot", help='''Take a snapshot of the existing volume.''') parser_snapshot.add_argument('-s', '--size', help='''Gives the size to allocate for the new snapshot volume A size suffix K|k, M|m, G|g, T|t, P|p, E|e can be used to define 'power of two' units. If no unit is provided, it defaults to kilobytes. This is option and if not give, the size will be determined automatically.''') group = parser_snapshot.add_mutually_exclusive_group() group.add_argument('-d', '--dest', help='''Destination of the snapshot specified with absolute path to be used for the new snapshot. This is optional and if not specified default backend policy will be performed.''') group.add_argument('-n', '--name', help='''Name of the new snapshot. This is optional and if not specified default backend policy will be performed.''') parser_snapshot.add_argument('volume', help="Volume, or mount point to take a snapshot of.", type=self.storage.can_snapshot) parser_snapshot.set_defaults(func=self.storage.snapshot) return parser_snapshot def _get_parser_mount(self): """ Mount command """ parser_mount = self.subcommands.add_parser("mount", help='''Mount a volume with file system to specified locaion.''') parser_mount.add_argument('-o', '--options', help='''Options are specified with a -o flag followed by a comma separated string of options. This option is equivalent to the same mount(8) option.''') parser_mount.add_argument("volume", help="Volume to mount.") parser_mount.add_argument("directory", help="Directory to attach the volume.", type=is_directory) parser_mount.set_defaults(func=self.storage.mount) return parser_mount def main(args=None): if args: sys.argv = args.split() options = Options() PR.set_options(options) storage = StorageHandle(options) ssm_parser = SsmParser(storage) args = ssm_parser.parse() # Check create command dependency if args.func == storage.create: if not args.raid: if (args.stripesize): err = "You can not specify --stripesize without specifying" + \ " RAID level!" ssm_parser.parser_create.error(err) if (args.stripes): err = "You can not specify --stripes without specifying" + \ " RAID level!" ssm_parser.parser_create.error(err) options.verbose = args.verbose options.force = args.force #storage.set_globals(args.force, args.verbose, args.yes, args.config) storage.set_globals(options) # Register clean-up function on exit sys.exitfunc = misc.do_cleanup try: args.func(args) except argparse.ArgumentTypeError, ex: ssm_parser.parser.error(ex) return 0 system-storage-manager-0.4/ssmlib/misc.py0000664000175000017500000003437712200417352022257 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # Miscellaneous functions for use by System Storage Manager import os import re import sys import tempfile import threading import subprocess # List of temporary mount points which should be cleaned up # before exiting TMP_MOUNTED = [] def get_unit_size(string): """ Check the last character of the string for the unit and return the unit value, otherwise return zero. It check only the last character of the string. >>> get_unit_size("B") 1 >>> get_unit_size("k") 1024 >>> get_unit_size("M") 1048576 >>> get_unit_size("g") 1073741824 >>> get_unit_size("T") 1099511627776... >>> get_unit_size("p") 1125899906842624... >>> get_unit_size("") 0 >>> get_unit_size("H") 0 """ mult = 0 units = {'B': 1, 'K': 2 ** 10, 'M': 2 ** 20, 'G': 2 ** 30, 'T': 2 ** 40, 'P': 2 ** 50} if len(string) > 0 and string[-1].upper() in units: mult = units[string[-1].upper()] return mult def is_number(string): """ Check is the string is number and return True or False. >>> is_number("3.14") True >>> is_number("+3.14") True >>> is_number("-3.14") True >>> is_number("314") True >>> is_number("3a14") False """ try: float(string) return True except ValueError: return False def get_real_size(size): """ Get the real number from the size argument. It converts the size with units into the size in kilobytes. Is no unit is specified it defaults to kilobytes. >>> get_real_size("3141") '3141' >>> get_real_size("3141K") '3141.00' >>> get_real_size("3141k") '3141.00' >>> get_real_size("3141M") '3216384.00' >>> get_real_size("3141G") '3293577216.00' >>> get_real_size("3141T") '3372623069184.00' >>> get_real_size("3141P") '3453566022844416.00' >>> get_real_size("3.14") '3.14' >>> get_real_size("+3.14") '+3.14' >>> get_real_size("-3.14") '-3.14' >>> get_real_size("3.14k") '3.14' >>> get_real_size("+3.14K") '+3.14' >>> get_real_size("-3.14k") '-3.14' >>> get_real_size("3.14G") '3292528.64' >>> get_real_size("+3.14g") '+3292528.64' >>> get_real_size("-3.14G") '-3292528.64' >>> get_real_size("G") Traceback (most recent call last): ... Exception: Not supported unit in the size 'G' argument. >>> get_real_size("3141H") Traceback (most recent call last): ... Exception: Not supported unit in the size '3141H' argument. """ if is_number(size): return size elif is_number(size[:-1]): # Always use kilobytes in ssm mult = get_unit_size(size) / 1024 sign = '+' if size[0] == '+' else '' if mult: return '{0}{1:.2f}'.format(sign, float(size[:-1]) * mult) raise Exception("Not supported unit in the " + "size \'{0}\' argument.".format(size)) def get_slaves(devname): return ["/dev/{0}".format(fname) for fname in os.listdir("/sys/block/{0}/slaves".format(devname))] def send_udev_event(device, event): major, minor = get_major_minor(device) with open('/sys/dev/block/{0}:{1}/uevent'.format(major, minor), "w") as f: f.write(event) def get_device_by_uuid(uuid): path = "/dev/disk/by-uuid/{0}".format(uuid) return os.path.abspath(os.path.join(os.path.dirname(path), os.readlink(path))) def get_major_minor(device): real_dev = get_real_device(device) stat = os.stat(real_dev) major = os.major(stat.st_rdev) minor = os.minor(stat.st_rdev) return major, minor def get_file_size(path): """ Get size of the file (even block device) by seeking to the end of the file and returning offset. The returning size is in kilobytes. """ with open(path, 'r') as f: return os.lseek(f.fileno(), os.SEEK_SET, os.SEEK_END) / 1024 def check_binary(name): command = ['which', name] if run(command, can_fail=True)[0]: return False else: return True def do_mount(device, directory, options=None): command = ['mount'] if options: command.extend(['-o', options]) command.extend([device, directory]) run(command) def do_umount(mpoint): command = ['umount', mpoint] try: run(command) except RuntimeError: command = ['umount', '-l', mpoint] run(command) def temp_mount(device, options=None): tmp = tempfile.mkdtemp() do_mount(device, tmp, options) TMP_MOUNTED.append(tmp) return tmp def temp_umount(mpoint=None): if not mpoint: mpoint = TMP_MOUNTED.pop() do_umount(mpoint) os.rmdir(mpoint) def do_cleanup(): while 1: try: temp_umount() except IndexError: break def get_signature(device, types=None): command = ["blkid", "-o", "value", "-p", "-s", "TYPE"] if types is not None: command.extend(['-u', types]) command.append(device) output = run(command, can_fail=True)[1].strip() if len(output) > 0: return output else: return None def get_fs_type(device): return get_signature(device, "filesystem") def get_real_device(device): if os.path.islink(device): return os.path.abspath(os.path.join(os.path.dirname(device), os.readlink(device))) else: return device def get_swaps(): swap = [] with open('/proc/swaps', 'r') as f: for line in f.readlines()[1:]: swap.append(line.split()) return swap def get_partitions(): partitions = [] with open('/proc/partitions', 'r') as f: for line in f.readlines()[2:]: partitions.append(line.split()) return partitions def get_mountinfo(regex=".*"): mounts = {} reg = re.compile(regex) names = ['id', 'parent', 'major_minor', 'root', 'mp', 'options'] with open('/proc/self/mountinfo', 'r') as f: for line in f: m = reg.search(line) if not m: continue array = line.split(None, 6) row = dict([(names[index], array[index]) for index in min(range(len(array) - 1), range(len(names)))]) array = line.rsplit(None, 3) row['fs'] = array[1] row['dev'] = array[2] row['sb_options'] = array[3] dev = get_real_device(row['dev']) if row['root'] != '/': dev = "{0}:{1}".format(dev, row['root']) mounts[dev] = row return mounts def get_mounts_old(regex=".*"): mounts = {} reg = re.compile(regex) with open('/proc/mounts', 'r') as f: for line in f: m = reg.search(line) if m: l = line.split()[:2] dev = get_real_device(l[0]) mounts[dev] = {'dev': l[0], 'mp': l[1]} return mounts def get_mounts(regex=".*"): if os.path.exists("/proc/self/mountinfo"): return get_mountinfo(regex) else: return get_mounts_old(regex) def get_dmnumber(name): reg = re.compile(" {0}$".format(name)) dmnumber = None with open('/proc/devices', 'r') as f: for line in f: m = reg.search(line) if m: dmnumber = line.split()[0] break return dmnumber def wipefs(device, typ): command = ['wipefs', '-p', device] output = run(command)[1] for line in output[1:].split('\n'): if not line: continue array = line.split(",") if array[-1] == typ: command = ['wipefs', '--offset', array[0], device] run(command) def humanize_size(arg): """ Returns the number with power of two units "KiB, MiB, ...etc. Parameter arg should be string of non-zero length, or integer. IMPORTANT: The arg argument is expected to be in KiB. >>> humanize_size(314) '314.00 KB' >>> humanize_size("314") '314.00 KB' >>> humanize_size(314159) '306.80 MB' >>> humanize_size(314159265) '299.61 GB' >>> humanize_size(314159265358) '292.58 TB' >>> humanize_size(314159265358979) '285.73 PB' >>> humanize_size(314159265358979323) '279.03 EB' >>> humanize_size(314159265358979323846) '272.49 ZB' >>> humanize_size(314159265358979323846264) '266.10 YB' >>> humanize_size(314159265358979323846264338) '266103.25 YB' >>> humanize_size(-314159265) '-299.61 GB' >>> humanize_size("") '' >>> humanize_size("hello world") Traceback (most recent call last): ... ValueError: could not convert string to float: hello world """ count = 0 if type(arg) is str and len(arg) == 0: return "" size = float(arg) while abs(size) >= 1024 and count < 7: size /= 1024 count += 1 units = ["KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB"] try: unit = units[count] except IndexError: unit = "???" return ("{0:.2f} {1}").format(size, unit) def run(cmd, show_cmd=False, stdout=False, stderr=True, can_fail=False, stdin_data=None, return_stdout=True): stdin = None if stdin_data is not None: stdin = subprocess.PIPE if stderr: stderr = subprocess.STDOUT else: stderr = subprocess.PIPE if stdout: stdout = None else: stdout = subprocess.PIPE # Convert all parts of cmd into string for i, item in enumerate(cmd): if type(item) is not str: cmd[i] = str(item) proc = subprocess.Popen(cmd, stdout=stdout, stderr=stderr, stdin=stdin) if stdin_data is not None: class StdinThread(threading.Thread): def run(self): proc.stdin.write(stdin_data) proc.stdin.close() stdin_thread = StdinThread() stdin_thread.daemon = True stdin_thread.start() output, error = proc.communicate() if stdin_data is not None: stdin_thread.join() err_msg = "ERROR running command: \"{0}\"".format(" ".join(cmd)) if proc.returncode != 0 and show_cmd: if output is not None: print output if error is not None: print error print >> sys.stderr, err_msg if proc.returncode != 0 and not can_fail: if output is not None: print output if error is not None: print error raise RuntimeError(err_msg) if not return_stdout: output = None return (proc.returncode, output) def chain(*iterables): """ Make an iterator that returns elements from the first iterable until it is exhausted, then proceeds to the next iterable, until all of the iterables are exhausted. Used for treating consecutive sequences as a single sequence. This code has been taken from itertools python module. chain('ABC', 'DEF') --> A B C D E F """ for it in iterables: for element in it: yield element def izip(*iterables): """ Make an iterator that aggregates elements from each of the iterables. Like zip() except that it returns an iterator instead of a list. Used for lock-step iteration over several iterables at a time. This code has been taken from itertools python module. izip('ABCD', 'xy') --> Ax By """ iterators = map(iter, iterables) while iterators: yield tuple(map(next, iterators)) def compress(data, selectors): """ Make an iterator that filters elements from data returning only those that have a corresponding element in selectors that evaluates to True. Stops when either the data or selectors iterables has been exhausted. This code has been taken from itertools python module. compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F """ return (d for d, s in izip(data, selectors) if s) def permutations(iterable, r=None): """ Return successive r length permutations of elements in the iterable. This code has been taken from itertools python module. permutations('ABCD', 2) --> AB AC AD BA BC BD CA CB CD DA DB DC permutations(range(3)) --> 012 021 102 120 201 210 """ pool = tuple(iterable) n = len(pool) r = n if r is None else r if r > n: return indices = range(n) cycles = range(n, n - r, -1) yield tuple(pool[i] for i in indices[:r]) while n: for i in reversed(range(r)): cycles[i] -= 1 if cycles[i] == 0: indices[i:] = indices[i + 1:] + indices[i:i + 1] cycles[i] = n - i else: j = cycles[i] indices[i], indices[-j] = indices[-j], indices[i] yield tuple(pool[i] for i in indices[:r]) break else: return def terminal_size(default=(25, 80)): """ Returns running terminal size. If size cannot be found out default size is returned. """ def _ioctl_GWINSZ(fd): try: import fcntl import termios import struct cr = struct.unpack('hh', fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234')) except: return None return cr try: env = os.environ cr = (env['LINES'], env['COLUMNS']) except: cr = _ioctl_GWINSZ(0) or _ioctl_GWINSZ(1) or _ioctl_GWINSZ(2) if not cr: try: fd = os.open(os.ctermid(), os.O_RDONLY) cr = _ioctl_GWINSZ(fd) os.close(fd) except: pass if not cr: cr = (25, 80) return int(cr[1]), int(cr[0]) system-storage-manager-0.4/bin/0000775000175000017500000000000012200423561020212 5ustar lczernerlczerner00000000000000system-storage-manager-0.4/bin/ssm.local0000775000175000017500000000206612171200774022045 0ustar lczernerlczerner00000000000000#!/bin/bash # # (C)2012 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # # ssm.local - Invoke system storage manager with the local version of modules # Add ../ directory to the PYTHONPATH so when python starts looking for # modules it will be the first location to search as we want local modules # to be imported SSMDIR="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )/../" && pwd )" export PYTHONPATH="$SSMDIR" python $SSMDIR/bin/ssm $@ system-storage-manager-0.4/bin/ssm0000775000175000017500000000233712200411405020741 0ustar lczernerlczerner00000000000000#!/usr/bin/env python # # (C)2011 Red Hat, Inc., Lukas Czerner # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # # ssm - System Storage Manager import os import sys from ssmlib import problem try: from ssmlib import main if __name__ == "__main__": noroot = ['-h', '--help', '--version'] if (len(sys.argv) == 2 and sys.argv[1] not in noroot) or \ (len(sys.argv) >= 3 and sys.argv[2] not in noroot): if not os.geteuid() == 0: sys.exit("\nRoot privileges required to run this script!\n") sys.exit(main.main()) except problem.SsmError, err: sys.exit(err.errcode) system-storage-manager-0.4/INSTALL0000664000175000017500000000232412200423506020473 0ustar lczernerlczerner00000000000000Note: See README for more information! Installation ************ To install System Storage Manager into your system simply run: python setup.py install as root in the System Storage Manager directory. Make sure that your system configuration meet the *requirements* in order for ssm to work correctly. Note that you can run **ssm** even without installation from using the local sources with: bin/ssm.local Requirements ************ Python 2.6 or higher is required to run this tool. System Storage Manager can only be run as root since most of the commands requires root privileges. There are other requirements listed bellow, but note that you do not necessarily need all dependencies for all backends, however if some of the tools required by the backend is missing, the backend would not work. Python modules ============== * os * re * sys * stat * argparse * datetime * threading * subprocess System tools ============ * tune2fs * fsck.SUPPORTED_FS * resize2fs * xfs_db * xfs_check * xfs_growfs * mkfs.SUPPORTED_FS * which * mount * blkid * wipefs Lvm backend =========== * lvm2 binaries Btrfs backend ============= * btrfs progs Crypt backend ============= * dmsetup * cryptsetup system-storage-manager-0.4/CHANGES0000664000175000017500000002755612200423444020454 0ustar lczernerlczerner00000000000000* Wed Aug 7 2013 Lukas Czerner - 0.4 - Stylistic cleanup found by pep8 (Lukas Czerner) - bashtests/005-lvm-snapshot.sh: add test, creation of snapshot when size not enough should fail (Jimmy Pan) - bashtests/005-lvm-snapshot.sh: add test to assure snapshot to a nonexistent volume should fail (Jimmy Pan) - Update Makefile (Lukas Czerner) - bashtests/002-lvm-create.sh: add a test of creation of a volume with filesystem and mountpoint (Jimmy Pan) - bashtests/002-lvm-create.sh: add test to create with single device (Jimmy Pan) - bashtests/002-lvm-create.sh: add test to create volume after removal of a volume (Jimmy Pan) - lib/check.sh: add mountpoint function to check mountpoint (Jimmy Pan) - ssmlib: Print pool name in snapshot list (Lukas Czerner) - btrfs: Always show btrfs snapshot properly (Lukas Czerner) - check: Allow to check file system non interactively (Lukas Czerner) - btrfs: fix btrfs remove test (Lukas Czerner) - btrfs: Fix create test (Lukas Czerner) - btrfs: Force creation of btrfs file system (Lukas Czerner) - check: Print warning when unmount fails (Lukas Czerner) - ssmlib: Allow to force unmount (Lukas Czerner) - ssmlib: Allow to bypass mount point question by force (Lukas Czerner) - ssmlib: Allow to force adjusting the size of the volume (Lukas Czerner) - bashtests/002-lvm-create.sh: prepare mountpoints for use (Jimmy Pan) - bashtests/003-lvm-remove.sh: add fail case of removing nonexistent thing (Jimmy Pan) - bashtests/003-lvm-remove.sh: add test of remove mutilple pools (Jimmy Pan) - bashtests/003-lvm-remove.sh: add test of remove mutilple volumes (Jimmy Pan) - bashtests/003-lvm-remove.sh: add test case of removing mutiple devices (Jimmy Pan) - bashtests/003-lvm-remove.sh: add variable pool3 for future use (Jimmy Pan) - bashtests: add runbashtests.sh to support running independent bash tests (Jimmy Pan) - ssmlib: Allow to force using the device when used in different pool (Lukas Czerner) - ssmlib: Allow to force using the device with existing filesystem (Lukas Czerner) - ssmlib: Inform user when file system exist on device (Lukas Czerner) - ssmlib: Do not allow snapshot when there is not space (Lukas Czerner) - ssmlib: Do not allow to resize to negative size (Lukas Czerner) - ssmlib: Fail if there is no enought space to create volume (Lukas Czerner) - ssmlib: Handle error when stripes > devices (Lukas Czerner) - tests: Search for Traceback in the test output (Lukas Czerner) - tests: Fix failure of list_table in some tests (Lukas Czerner) - ssmlib: Use get_bdevice() helper (Lukas Czerner) - ssmlib: Introduce more generic get_signature() (Lukas Czerner) - ssmlib: Return error from remove when nothing was removed (Lukas Czerner) - remove: Fix remove -a (Lukas Czerner) - resize: Do not free device from the pool if not necessary (Lukas Czerner) - doc: Update add command description (Lukas Czerner) - doc: document SSM_PREFIX_FILTER env variable (Lukas Czerner) - docs: Refresh documentation (Lukas Czerner) - unittests: Create unittests for mount command (Lukas Czerner) - ssmlib: Add new mount command (Lukas Czerner) - Btrfs: adapt to changed 'subvolume list' command (Lukas Czerner) - btrfs: Fix btrfs subvolume removal code (Lukas Czerner) - btrfs: Check for existing path when creating subvolume (Lukas Czerner) - btrfs: Check path when creating subvolume (Lukas Czerner) - btrfs: Get btrfs version (Lukas Czerner) - Fix some typos reported by pep8 (Lukas Czerner) - btrfs: Notify udev when device is removed from fs (Lukas Czerner) - btrfs: Use regex to gather info about subvolumes (Lukas Czerner) - Add ssm output checks into 010-btrfs-snapshot bashtest (Tom Marek) - Fix device count in btrfs remove bashtest (Tom Marek) - Add ssm output checks into 007-btrfs-create bashtest (Tom Marek) - Add ssm output checks into 006-btrfs-add bashtest (Tom Marek) - Add ssm output checks into 005-lvm-snapshot bashtest (Tom Marek) - Add ssm output checks into 002-lvm-create bashtest (Tom Marek) - Add ssm output checks into 001-lvm-add bashtest (Tom Marek) - Set enviromental variable COLUMNS for testing ssm output (Tom Marek) - Add error message to bashtest "check list_table" (Tom Marek) - btrfs: Rework how subvolume info is gathered (Lukas Czerner) - Remove trailing semicolons (Tony Asleson) - Use double quote instead of single quote doc strings (Tony Asleson) - Clean up comparisions (Tony Asleson) - Simplify assignment (Tony Asleson) * Tue Oct 16 2012 Lukas Czerner - 0.3 - Add regression test suite to the source distribution (Lukas Czerner) - ssm: Fix various stylistic problems (Lukas Czerner) - lvm: Use problem interface instead of plain Exception (Lukas Czerner) - main: Fix resizing device without pool (Lukas Czerner) - md: Add new md backend (Lukas Czerner) - problem: Add new not_supported flag (Lukas Czerner) - ssm: fix typo in problem interface (Tomas Racek) - tests: add bashtest for checking ssm lvm output (Tom Marek) - Change priority of ways to gahter terminal dimensions (Tom Marek) - tests: Add ssm output check for bash testing into check.sh (Tom Marek) - Avoid executing non existing fs specific tools (Tom Marek) - Allow non root users to show --help for subcommands (Tom Marek) - Fix of unittests in python2.6 (Tom Marek) - Fix of doctests in python2.6 (Tom Marek) - Wrap table lines when terminal width is too small (Lukas Czerner) * Tue Jun 5 2012 Lukas Czerner - 0.2 - Update version to 0.2 (Lukas Czerner) - Add Makefile to the project (Lukas Czerner) - Remove TODO and ass AUTHORS and INSTALL to the distribution (Lukas Czerner) - tests: Exit successfully even when bashtests are not run (Lukas Czerner) - setup.py: update package name and add url (Lukas Czerner) - Set LC_NUMERIC instead of LANG variable (Lukas Czerner) - Show all file system in the listing (Lukas Czerner) - Add problem.py to __init__ (Lukas Czerner) - Update required python version to 2.6 (Lukas Czerner) - Ask when removing mounted lvm volume (Lukas Czerner) - Print warning if tool to create fs does not exist (Lukas Czerner) - tests: Colorize test result output (Lukas Czerner) - Catch RuntimeError when initializing backend (Lukas Czerner) - lvm: Do not query for lv_path with lvs (Lukas Czerner) - crypt: Use run_cryptsetup() to run cryptsetup binary (Lukas Czerner) - btrfs: Use run_btrfs() to run btrfs binary (Lukas Czerner) - Fix python string formatting (Lukas Czerner) - misc: Fix referencing uninitialized variable (Lukas Czerner) - Remove itertools from dependencies (Lukas Czerner) - Allow ssm to be run locally without installing (Lukas Czerner) - Fix check for the root user (Lukas Czerner) - Catch all ssm specific exceptions and exit (Lukas Czerner) - problem: Create problem specific exceptions (Lukas Czerner) - doc: Document changed create/resize behaviour (Lukas Czerner) - Remove executable section from ssmlib/main.py (Lukas Czerner) - btrfs: Compute pool_free as difference between pool_size and fs_used (Lukas Czerner) - crypt: Do not print unnecessary information (Lukas Czerner) - btrfs: Umount file system only once before wipefs (Lukas Czerner) - tests: Update tests with new problem interface (Lukas Czerner) - Change how create and resize handles optional devices (Lukas Czerner) - Use Options() object to pass options between classes (Lukas Czerner) - main: Wire in problem interface (Lukas Czerner) - lvm: Wire in problem interface (Lukas Czerner) - btrfs: Wire in problem interface (Lukas Czerner) - misc: Do not print unnecessary information (Lukas Czerner) - Add problem.py module for dealing with problems (Lukas Czerner) - Update TODO list (Lukas Czerner) - Update README (Lukas Czerner) - Update setup.py file (Lukas Czerner) - Add INSTALL file (Lukas Czerner) - Add ssm.8 manual page (Lukas Czerner) - doc: Add sphinx documentation directory (Lukas Czerner) - Update argparse description (Lukas Czerner) - Fix ssm version (Lukas Czerner) - Allow help to be shown by non root user (Lukas Czerner) - Wrap argparse code into its own class (Lukas Czerner) - tests: Add bash tests cases to use -b/--backend argument (Lukas Czerner) - tests: Add unittests to use -b/--backend argument (Lukas Czerner) - Add option to choose default backend (Lukas Czerner) - tests: Add btrfs "add" unittest (Lukas Czerner) - tests: Add btrfs resize unittest (Lukas Czerner) - tests: Add btrfs snapshot unittest (Lukas Czerner) - tests: Add btrfs remove unittest (Lukas Czerner) - tests: Add btrfs create unittest (Lukas Czerner) - tests: Add mocked mount handling for unittests (Lukas Czerner) - tests: Add "add" lvm unittests (Lukas Czerner) - tests: Add resize lvm unittests (Lukas Czerner) - tests: Add create lvm unittests (Lukas Czerner) - tests: Add ssm resize unittest (Lukas Czerner) - tests: Remove some unneeded ssm unittest cases (Lukas Czerner) - tests: Add some required backend variables into unittest (Lukas Czerner) - btrfs: Call btrfs with verbose argument when verbose specified (Lukas Czerner) - btrfs: Always use integer when printing block count (Lukas Czerner) - btrfs: Show real mount point of the subvolume (Lukas Czerner) - misc: use /proc/self/mountinfo if exists (Lukas Czerner) - tests: Add basic test for btrfs raid support (Lukas Czerner) - Add --raid argument to the create command (Lukas Czerner) - Do not show cdrom in device listing (Lukas Czerner) - tests: Add btrfs snapshot bash test (Lukas Czerner) - tests: Add btrfs remove bash test (Lukas Czerner) - tests: Add btrfs create bash test (Lukas Czerner) - tests: Add btrfs add bash test (Lukas Czerner) - tests: set SSM_DEFAULT_BACKEND for each test (Lukas Czerner) - btrfs: Mount device temporarily when resizing (Lukas Czerner) - Prevent from using already used devices in add (Lukas Czerner) - btrfs: Mount device temporarily when creating snapshot (Lukas Czerner) - Add new get_device method to checking for block device (Lukas Czerner) - Stylistic clean-up (Lukas Czerner) - tests: Add btrfs specific checks (Lukas Czerner) - btrfs: Add new devices into the pool prior subvolume creation (Lukas Czerner) - btrfs: Create new pool when adding to non-existing pool (Lukas Czerner) - btrfs: Mount device temporarily when shrinking pool (Lukas Czerner) - btrfs: Mount device temporarily when extending pool (Lukas Czerner) - btrfs: Mount device temporarily when creating subvolume (Lukas Czerner) - Add possibility to temporarily mount devices (Lukas Czerner) - btrfs: Test all btrfs devices for mount point (Lukas Czerner) - btrfs: Always use pool name when creating new pool (Lukas Czerner) - tests: Add infrastructure to create temp mount points (Lukas Czerner) - Always translate device name into real device (Lukas Czerner) - Use have backends to define default pool name (Lukas Czerner) - Fix creating btrfs volumes (Lukas Czerner) - Change prefix_filter so it check for more than device name (Lukas Czerner) - btrfs: Fix btrfs create routine (Lukas Czerner) - btrfs: Track subvolume ID correctly (Lukas Czerner) - Try to find internal record for provided device (Lukas Czerner) - misc: fix do_mount so it create string from list of options (Lukas Czerner) - Add type information to the backend (Lukas Czerner) - Allow to reinitialize device/pool/volume/snapshot (Lukas Czerner) - Allow to specify default backend (Lukas Czerner) - tests: Show running test name first (Lukas Czerner) - tests: Rename bash tests to include backend name (Lukas Czerner) - lvm: Do not show dev_free and dev_used if pv is not in pool (Lukas Czerner) - Do not fail on create if DeviceInfo does not recognise device (Lukas Czerner) - Add pool name to the Volume listing (Lukas Czerner) - tests 005: Add bashtests to exercise snapshot command (Lukas Czerner) - backend lvm: Convert lv name into the dev_name (Lukas Czerner) - tests 004: Check if the fs is supported by the kernel (Lukas Czerner) - unittests: Add lvm specific snapshot unit test (Lukas Czerner) - unittests: Add generic snapshot unittests (Lukas Czerner) - tests: Allow specify mount point in mocked volume (Lukas Czerner) - Add "name" parameter to the snapshot command (Lukas Czerner) - tests: Add __init__ into tests directory (Lukas Czerner) system-storage-manager-0.4/README0000664000175000017500000007070012200423506020325 0ustar lczernerlczerner00000000000000 System Storage Manager ********************** A single tool to manage your storage. Description *********** System Storage Manager provides easy to use command line interface to manage your storage using various technologies like lvm, btrfs, encrypted volumes and more. In more sophisticated enterprise storage environments, management with Device Mapper (dm), Logical Volume Manager (LVM), or Multiple Devices (md) is becoming increasingly more difficult. With file systems added to the mix, the number of tools needed to configure and manage storage has grown so large that it is simply not user friendly. With so many options for a system administrator to consider, the opportunity for errors and problems is large. The btrfs administration tools have shown us that storage management can be simplified, and we are working to bring that ease of use to Linux filesystems in general. Licence ******* (C)2011 Red Hat, Inc., Lukas Czerner This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Commands ******** Introduction ************ System Storage Manager have several commands you can specify on the command line as a first argument to the ssm. They all have specific use and its own arguments, but global ssm arguments are propagated to all commands. Create command ************** This command creates a new volume with defined parameters. If **device** is provided it will be used to create a volume, hence it will be added into the **pool** prior the volume creation (See *Add command section*). More devices can be used to create a volume. If the **device** is already used in the different pool, then **ssm** will ask you whether you want to remove it from the original pool. If you decline, or the removal fails, then the **volume** creation fails if the *SIZE* was not provided. On the other hand, if the *SIZE* is provided and some devices can not be added to the **pool** the volume creation might succeed if there is enough space in the **pool**. *POOL* name can be specified as well. If the pool exists new volume will be created from that pool (optionally adding **device** into the pool). However if the *POOL* does not exist **ssm** will attempt to create a new pool with provided **device** and then create a new volume from this pool. If **--backend** argument is omitted, the default **ssm** backend will be used. Default backend is *lvm*. **ssm** also supports creating RAID configuration, however some back- ends might not support all the levels, or it might not support RAID at all. In this case, volume creation will fail. If **mount** point is provided **ssm** will attempt to mount the volume after it is created. However it will fail if mountable file system is not present on the volume. List command ************ List informations about all detected devices, pools, volumes and snapshots found in the system. **list** command can be used either alone to list all the information, or you can request specific section only. Following sections can be specified: {volumes | vol} List information about all **volumes** found in the system. {devices | dev} List information about all **devices** found in the system. Some devices are intentionally hidden, like for example cdrom, or DM/MD devices since those are actually listed as volumes. {pools | pool} List information about all **pools** found in the system. {filesystems | fs} List information about all volumes containing **filesystems** found in the system. {snapshots | snap} List information about all **snapshots** found in the system. Note that some back-ends does not support snapshotting and some can not distinguish between snapshot and regular volume. in this case **ssm** will try to recognize volume name in order to identify **snapshot**, but if the **ssm** regular expression does not match the snapshot pattern, this snapshot will not be recognized. Remove command ************** This command removes **item** from the system. Multiple items can be specified. If the **item** can not be removed for some reason, it will be skipped. **item** can represent: device Remove **device** from the pool. Note that this can not be done in some cases where the device is used by pool. You can use **-f** argument to *force* removal. If the device does not belong to any pool, it will be skipped. pool Remove the **pool** from the system. This will also remove all volumes created from that pool. volume Remove the **volume** from the system. Note that this will fail if the **volume** is mounted and it can not be *forced* with **-f**. Resize command ************** Change size of the **volume** and file system. If there is no file system only the **volume** itself will be resized. You can specify **device** to add into the **volume** pool prior the resize. Note that **device** will only be added into the pool if the **volume** size is going to grow. If the **device** is already used in the different pool, then **ssm** will ask you whether you want to remove it from the original pool. In some cases file system has to be mounted in order to resize. This will be handled by **ssm** automatically by mounting the **volume** temporarily. Check command ************* Check the file system consistency on the **volume**. You can specify multiple volumes to check. If there is no file system on the **volume**, this **volume** will be skipped. In some cases file system has to be mounted in order to check the file system This will be handled by **ssm** automatically by mounting the **volume** temporarily. Snapshot command **************** Take a snapshot of existing **volume**. This operation will fail if back-end which the **volume** belongs to does not support snapshotting. Note that you can not specify both *NAME* and *DESC* since those options are mutually exclusive. In some cases file system has to be mounted in order to take a snapshot of the **volume**. This will be handled by **ssm** automatically by mounting the **volume** temporarily. Add command *********** This command adds **device** into the pool. The **device** will not be added if it's already part of different pool by default, but user will be asked whether to remove the device from it's pool. When multiple devices are provided, all of them are added into the pool. If one of the devices can not be added into the pool for any reason, add command will fail. If no pool is specified, default pool will be chosen. In the case of non existing pool, it will be created using provided devices. Backends ******** Introduction ************ Ssm aims to create unified user interface for various technologies like Device Mapper (dm), Btrfs file system, Multiple Devices (md) and possibly more. In order to do so we have a core abstraction layer in "ssmlib/main.py". This abstraction layer should ideally know nothing about the underlying technology, but rather comply with **device**, **pool** and **volume** abstraction. Various backends can be registered in "ssmlib/main.py" in order to handle specific storage technology implementing methods like *create*, *snapshot*, or *remove* volumes and pools. The core will then call these methods to manage the storage without needing to know what lies underneath it. There are already several backends registered in ssm. Btrfs backend ************* Btrfs is the file system with many advanced features including volume management. This is the reason why btrfs is handled differently than other *conventional* file systems in **ssm**. It is used as a volume management back-end. Pools, volumes and snapshots can be created with btrfs backend and here is what it means from the btrfs point of view: pool Pool is actually a btrfs file system itself, because it can be extended by adding more devices, or shrink by removing devices from it. Subvolumes and snapshots can also be created. When the new btrfs pool should be created **ssm** simply creates a btrfs file system, which means that every new btrfs pool has one volume of the same name as the pool itself which can not be removed without removing the entire pool. Default btrfs pool name is **btrfs_pool**. When creating new btrfs pool, the name of the pool is used as the file system label. If there is already existing btrfs file system in the system without a label, btrfs pool name will be generated for internal use in the following format "btrfs_{device base name}". Btrfs pool is created when **create** or **add** command is used with devices specified and non existing pool name. volume Volume in btrfs back-end is actually just btrfs subvolume with the exception of the first volume created on btrfs pool creation, which is the file system itself. Subvolumes can only be created on btrfs file system when the it is mounted, but user does not have to worry about that, since **ssm** will automatically mount the file system temporarily in order to create a new subvolume. Volume name is used as subvolume path in the btrfs file system and every object in this path must exists in order to create a volume. Volume name for internal tracking and for representing to the user is generated in the format "{pool_name}:{volume name}", but volumes can be also referenced with its mount point. Btrfs volumes are only shown in the *list* output, when the file system is mounted, with the exception of the main btrfs volume - the file system itself. New btrfs volume can be created with **create** command. snapshot Btrfs file system support subvolume snapshotting, so you can take a snapshot of any btrfs volume in the system with **ssm**. However btrfs does not distinguish between subvolumes and snapshots, because snapshot actually is just a subvolume with some block shared with different subvolume. It means, that **ssm** is not able to recognize btrfs snapshot directly, but instead it is trying to recognize special name format of the btrfs volume. However, if the *NAME* is specified when creating snapshot which does not match the special pattern, snapshot will not be recognized by the **ssm** and it will be listed as regular btrfs volume. New btrfs snapshot can be created with **snapshot** command. device Btrfs does not require any special device to be created on. Lvm backend *********** Pools, volumes and snapshots can be created with lvm, which pretty much match the lvm abstraction. pool Lvm pool is just *volume group* in lvm language. It means that it is grouping devices and new logical volumes can be created out of the lvm pool. Default lvm pool name is **lvm_pool**. Lvm pool is created when **create** or **add** command is used with devices specified and non existing pool name. volume Lvm volume is just *logical volume* in lvm language. Lvm volume can be created wit **create** command. snapshot Lvm volumes can be snapshotted as well. When a snapshot is created from the lvm volume, new *snapshot* volume is created, which can be handled as any other lvm volume. Unlike *btrfs* lvm is able to distinguish snapshot from regular volume, so there is no need for a snapshot name to match special pattern. device Lvm requires *physical device* to be created on the device, but with **ssm** this is transparent for the user. Crypt backend ************* Crypt backend in **ssm** is currently limited to only gather the information about encrypted volumes in the system. You can not create or manage encrypted volumes or pools, but it will be extended in the future. Environment variables ********************* SSM_DEFAULT_BACKEND Specify which backend will be used by default. This can be overridden by specifying **-b** or **--backend** argument. Currently only *lvm* and *btrfs* is supported. SSM_LVM_DEFAULT_POOL Name of the default lvm pool to be used if **-p** or **--pool** argument is omitted. SSM_BTRFS_DEFAULT_POOL Name of the default btrfs pool to be used if **-p** or **--pool** argument is omitted. SSM_PREFIX_FILTER When this is set **ssm** will filter out all devices, volumes and pools which name does not start with this prefix. It is used mainly in **ssm** test suite to make sure that we do not scramble local system configuration. Quick examples ************** List system storage: # ssm list ---------------------------------- Device Total Mount point ---------------------------------- /dev/loop0 5.00 GB /dev/loop1 5.00 GB /dev/loop2 5.00 GB /dev/loop3 5.00 GB /dev/loop4 5.00 GB /dev/sda 149.05 GB PARTITIONED /dev/sda1 19.53 GB / /dev/sda2 78.12 GB /dev/sda3 1.95 GB SWAP /dev/sda4 1.00 KB /dev/sda5 49.44 GB /mnt/test ---------------------------------- ------------------------------------------------------------------------------ Volume Pool Volume size FS FS size Free Type Mount point ------------------------------------------------------------------------------ /dev/dm-0 dm-crypt 78.12 GB ext4 78.12 GB 45.01 GB crypt /home /dev/sda1 19.53 GB ext4 19.53 GB 12.67 GB part / /dev/sda5 49.44 GB ext4 49.44 GB 29.77 GB part /mnt/test ------------------------------------------------------------------------------ Creating a volume of defined size with the defined file system. The default back-end is set to lvm and lvm default pool name is lvm_pool: # ssm create --fs ext4 -s 15G /dev/loop0 /dev/loop1 The name of the new volume is '/dev/lvm_pool/lvol001'. Resize the volume to 10GB: # ssm resize -s-5G /dev/lvm_pool/lvol001 Resize the volume to 100G, but it would require to add more devices into the pool: # ssm resize -s 25G /dev/lvm_pool/lvol001 /dev/loop2 Now we can try to create new lvm volume named 'myvolume' from the remaining pool space with xfs file system and mount it to /mnt/test1: # ssm create --fs xfs --name myvolume /mnt/test1 List all volumes with file system: # ssm list filesystems ----------------------------------------------------------------------------------------------- Volume Pool Volume size FS FS size Free Type Mount point ----------------------------------------------------------------------------------------------- /dev/lvm_pool/lvol001 lvm_pool 25.00 GB ext4 25.00 GB 23.19 GB linear /dev/lvm_pool/myvolume lvm_pool 4.99 GB xfs 4.98 GB 4.98 GB linear /mnt/test1 /dev/dm-0 dm-crypt 78.12 GB ext4 78.12 GB 45.33 GB crypt /home /dev/sda1 19.53 GB ext4 19.53 GB 12.67 GB part / /dev/sda5 49.44 GB ext4 49.44 GB 29.77 GB part /mnt/test ----------------------------------------------------------------------------------------------- You can then easily remove the old volume by: # ssm remove /dev/lvm_pool/lvol001 Now lest try to create btrfs volume. Btrfs is separate backend, not just a file system. That is because btrfs itself have integrated volume manager. Defaul btrfs pool name is btrfs_pool.: # ssm -b btrfs create /dev/loop3 /dev/loop4 Now create we btrfs subvolumes. Note that btrfs file system has to be mounted in order to create subvolumes. However ssm will handle it for you.: # ssm create -p btrfs_pool # ssm create -n new_subvolume -p btrfs_pool # ssm list filesystems ----------------------------------------------------------------- Device Free Used Total Pool Mount point ----------------------------------------------------------------- /dev/loop0 0.00 KB 10.00 GB 10.00 GB lvm_pool /dev/loop1 0.00 KB 10.00 GB 10.00 GB lvm_pool /dev/loop2 0.00 KB 10.00 GB 10.00 GB lvm_pool /dev/loop3 8.05 GB 1.95 GB 10.00 GB btrfs_pool /dev/loop4 6.54 GB 1.93 GB 8.47 GB btrfs_pool /dev/sda 149.05 GB PARTITIONED /dev/sda1 19.53 GB / /dev/sda2 78.12 GB /dev/sda3 1.95 GB SWAP /dev/sda4 1.00 KB /dev/sda5 49.44 GB /mnt/test ----------------------------------------------------------------- ------------------------------------------------------- Pool Type Devices Free Used Total ------------------------------------------------------- lvm_pool lvm 3 0.00 KB 29.99 GB 29.99 GB btrfs_pool btrfs 2 3.84 MB 18.47 GB 18.47 GB ------------------------------------------------------- ----------------------------------------------------------------------------------------------- Volume Pool Volume size FS FS size Free Type Mount point ----------------------------------------------------------------------------------------------- /dev/lvm_pool/lvol001 lvm_pool 25.00 GB ext4 25.00 GB 23.19 GB linear /dev/lvm_pool/myvolume lvm_pool 4.99 GB xfs 4.98 GB 4.98 GB linear /mnt/test1 /dev/dm-0 dm-crypt 78.12 GB ext4 78.12 GB 45.33 GB crypt /home btrfs_pool btrfs_pool 18.47 GB btrfs 18.47 GB 18.47 GB btrfs /dev/sda1 19.53 GB ext4 19.53 GB 12.67 GB part / /dev/sda5 49.44 GB ext4 49.44 GB 29.77 GB part /mnt/test ----------------------------------------------------------------------------------------------- Now let's free up some of the loop devices so we cat try to add them into then btrfs_pool. So we'll simply remove lvm mvolume and resize lvol001 so we can remove /dev/loop2. Note that myvolume is mounted so we have to unmount it first.: # umount /mnt/test1 # ssm remove /dev/lvm_pool/myvolume # ssm resize -s-10G /dev/lvm_pool/lvol001 # ssm remove /dev/loop2 Add device to the btrfs file system: # ssm add /dev/loop2 -p btrfs_pool Set' see what happend. Note that to actually see btrfs subvolumes you have to mount the file system first: # mount -L btrfs_pool /mnt/test1/ # ssm list volumes ------------------------------------------------------------------------------------------------------------------------ Volume Pool Volume size FS FS size Free Type Mount point ------------------------------------------------------------------------------------------------------------------------ /dev/lvm_pool/lvol001 lvm_pool 15.00 GB ext4 15.00 GB 13.85 GB linear /dev/dm-0 dm-crypt 78.12 GB ext4 78.12 GB 45.33 GB crypt /home btrfs_pool btrfs_pool 28.47 GB btrfs 28.47 GB 28.47 GB btrfs /mnt/test1 btrfs_pool:2012-05-09-T113426 btrfs_pool 28.47 GB btrfs 28.47 GB 28.47 GB btrfs /mnt/test1/2012-05-09-T113426 btrfs_pool:new_subvolume btrfs_pool 28.47 GB btrfs 28.47 GB 28.47 GB btrfs /mnt/test1/new_subvolume /dev/sda1 19.53 GB ext4 19.53 GB 12.67 GB part / /dev/sda5 49.44 GB ext4 49.44 GB 29.77 GB part /mnt/test ------------------------------------------------------------------------------------------------------------------------ Remove the whole lvm pool and one of the btrfs subvolume, and one unused device from the btrfs pool btrfs_loop3. Note that with btrfs, pool have the same name as the volume: # ssm remove lvm_pool /dev/loop2 /mnt/test1/new_subvolume/ Snapshots can also be done with ssm: # ssm snapshot btrfs_pool # ssm snapshot -n btrfs_snapshot btrfs_pool With lvm, you can also create snapshots: root# ssm create -s 10G /dev/loop[01] # ssm snapshot /dev/lvm_pool/lvol001 Now list all snapshots. Note that btrfs snapshots are actually just subvolumes with some blocks shared with the original subvolume, so there currently no way to distinguish between those. ssm is using a little trick to search for name patters to recognize snapshots, so if you specify your own name for the snapshot ssm will not recognize it as snapshot, but rather as regular volume (subvolume). This problem does not exist with lvm.: # ssm list snapshots ------------------------------------------------------------------------------------------------------------- Snapshot Origin Volume size Size Type Mount point ------------------------------------------------------------------------------------------------------------- /dev/lvm_pool/snap20120509T121611 lvol001 2.00 GB 0.00 KB linear btrfs_pool:snap-2012-05-09-T121313 18.47 GB btrfs /mnt/test1/snap-2012-05-09-T121313 ------------------------------------------------------------------------------------------------------------- Installation ************ To install System Storage Manager into your system simply run: python setup.py install as root in the System Storage Manager directory. Make sure that your system configuration meet the *requirements* in order for ssm to work correctly. Note that you can run **ssm** even without installation from using the local sources with: bin/ssm.local Requirements ************ Python 2.6 or higher is required to run this tool. System Storage Manager can only be run as root since most of the commands requires root privileges. There are other requirements listed bellow, but note that you do not necessarily need all dependencies for all backends, however if some of the tools required by the backend is missing, the backend would not work. Python modules ============== * os * re * sys * stat * argparse * datetime * threading * subprocess System tools ============ * tune2fs * fsck.SUPPORTED_FS * resize2fs * xfs_db * xfs_check * xfs_growfs * mkfs.SUPPORTED_FS * which * mount * blkid * wipefs Lvm backend =========== * lvm2 binaries Btrfs backend ============= * btrfs progs Crypt backend ============= * dmsetup * cryptsetup For developers ************** We are accepting patches! If you're interested contributing to the System Storage Manager code, just checkout the git repository located on SourceForge. Please, base all of your work on the "devel" branch since it is more up-to-date and it will save us some work when merging your patches: git clone --branch devel git://git.code.sf.net/p/storagemanager/code storagemanager-code Any form of contribution - patches, documentation, reviews or rants are appreciated. See *Mailing list section* section. Tests ===== System Storage Manager contains regression testing suite to make sure that we do not break thing that should already work. And we recommend every developer to run tests before sending patches: python test.py Tests in System Storage Manager are divided into four levels. 1. First the doctest is executed. 2. Then we have unittests in "tests/unittests/test_ssm.py" which is testing the core of ssm "ssmlib/main.py". It is checking for basic things like required backend methods and variables, flag propagations, proper class initialization and finally whether commands actually result in the proper backend callbacks. It does not require root permissions and it does not touch your system configuration in any way. It actually should not invoke any shell command, and if it does it's a bug. 3. Second part of unittests is backend testing. We are mainly testing whether ssm commands result in proper backend operations. It does not require root permissions and it does not touch your system configuration in any way. It actually should not invoke any shell command and if it does it's a bug. 4. And finally there are real bash tests located in "tests/bashtests". Bash tests are divided into files. Each file tests one command for one backend and it containing series of test cases followed by checks whether the command created the expected result. In order to test real system commands we have to create system device to test on and not touch any of the existing system configuration. Before each test a number of devices are created using *dmsetup* in the test directory. These devices will be used in test cases instead of real devices. Real operation are performed in those devices as it would on the real system devices. It implies that this phase requires root privileges and it would not be run otherwise. In order to make sure that **ssm** does not touch any existing system configuration, each device, poor and volume name is include special prefix and SSM_PREFIX_FILTER environment variable is set to make **ssm** to exclude all items which does not match this filter. Even though we tried hard to make sure that the bash tests does not change any of your system configuration the recommendation is **not** to run tests as with root privileges on your work or production system, but rather run it on your testing machine. If you change or create new functionality, please make sure that it is covered by the System Storage Manager regression test suite to make sure that we do not break it unintentionally. Important: Please, make sure to run full tests before you send a patch to the mailing list. To do so, simply run "python test.py" as root on your test machine. Documentation ============= System Storage Manager documentation is stored in "doc/" directory. The documentation is build using **sphinx** software which help us not to duplicate texts for different type of documentation (man page, html pages, readme). If you are going to modify documentation, please make sure not to modify manual page, html pages or README directly, but rather modify "doc/*.rst" and "doc/src/*.rst" files accordingly so the change is propagated to all documents. Moreover, parts of the documentation such as *synopsis* or ssm command *options* are parsed directly from the ssm help output. It means that when you're going to add or change argument into **ssm** the only thing you have to do is to add or change it in the "ssmlib/main.py" source code and then run "make dist" in the "doc/" directory and all the documents should be updated automatically. Important: Please make sure you update the documentation when you add or change **ssm** functionality if the format of the change requires it. Then regenerate all the documents using "make dist" and include changes in the patch. Mailing list ============ System Storage Manager developers communicate via the mailing list. Address of our mailing list is storagemanager- devel@lists.sourceforge.net and you can subscribe on the SourceForge project page https://lists.sourceforge.net/lists/listinfo /storagemanager-devel. Mailing list archives can be found here http://sourceforge.net/mailarchive/forum.php?forum_name =storagemanager-devel. This is also the list where to send patches and where the review process is happening. We do not have separate *user* mailing list, so feel free to drop your questions there as well. Posting patches =============== As already mentioned, we are accepting patches! And we are very happy for every contribution. If you're going to send a path in, please make sure to follow some simple rules: 1. Before you're going to post a patch, please run our regression testing suite to make sure that your change does not break someone else work. See *Tests section* 2. If you're making a change that might require documentation update, please update the documentation as well. See *Documentation section* 3. Make sure your patch have all the requisites such as *short description* preferably 50 characters long at max describing the main idea of the change. *Long description* describing what was changed with and why and finally Signed-off-by tag. 4. If you're going to send a patch to the mailing list, please send the patch inlined in the email body. It is much better for review process. Hint: You can use **git** to do all the work for you. "git format-patch" and "git send-email" will help you with creating and sending the patch.