singularity-2.4.2/0000755000175000017500000000000013211621077013027 5ustar mehdimehdisingularity-2.4.2/CONTRIBUTORS.md0000644000175000017500000000255313211621077015313 0ustar mehdimehdi#Project Lead: - Gregory M. Kurtzer #Developers: - Cedric Clerget - Dave Dykstra - Dave Godlove - Krishna Muriki - Michael Bauer - Vanessa Sochat - Yannick Cote #Contributors: - Amanda Duffy - Ángel Bejarano - Bernard Li - Brian Bockelman - Chris Hollowell - Dave Love - Eduardo Arango - Felix Abecassis - George Hartzell - Jarrod Johnson - Jason Stover - Maciej Sieczka - Mark Egan-Fuller - Nathan Lin - Oleksandr Moskalenko - Oliver Freyermuth - Petr Votava - Ralph Castain - Rémy Dernat - Yaroslav Halchenko - Josef Hrabal - Daniele Tamino singularity-2.4.2/INSTALL.md0000644000175000017500000000377113211621077014467 0ustar mehdimehdi# Installing Singularity Since you are reading this from the Singualrity source code, it will be assumed that you are building/compiling. You must first install the development tools and libraries to your host. Assuming a Red Hat compatible system (apply similar to Debian derivitives): ``` $ sudo yum groupinstall "Development Tools" ``` ## To compile and install Singularity from a [released tarball](https://github.com/singularityware/singularity/releases): Here, the version of Singularity that you want to install is given in <version>. Please substitute as necessary. 
$ version=<version>
$ wget "https://github.com/singularityware/singularity/releases/download/${version}/singularity-${version}.tar.gz"
$ tar -xvzf singularity-${version}.tar.gz
$ cd singularity-${version}
$ ./configure --prefix=/usr/local
$ make
$ sudo make install
note: The `sudo` is very important for the `make install`. Failure to do this will result in a non-functioning or semi-functioning installation. ## To compile and install Singularity from a Git clone: Here, the version of Singularity that you want to install is given in <version>. Please substitute as necessary. 
$ version=<version>
$ git clone https://github.com/singularityware/singularity.git
$ cd singularity
$ git checkout tags/${version} -b ${version}
$ ./autogen.sh
$ ./configure --prefix=/usr/local
$ make
$ sudo make install
note: The `sudo` is very important for the `make install`. Failure to do this will result in a non-functioning or semi-functioning installation. ## To build an RPM of Singularity from a Git clone: Here, the version of Singularity that you want to install is given in <version>. Please substitute as necessary. 
$ version=<version>
$ git clone https://github.com/singularityware/singularity.git
$ cd singularity
$ git checkout tags/${version} -b ${version}
$ ./autogen.sh
$ ./configure
$ make dist
$ rpmbuild -ta singularity-*.tar.gz
singularity-2.4.2/README.md0000644000175000017500000001666513211621077014324 0ustar mehdimehdi[![Build Status](https://travis-ci.org/singularityware/singularity.svg?branch=master)](https://travis-ci.org/singularityware/singularity) - [Guidelines for Contributing](CONTRIBUTING.md) - [Pull Request Template](.github/PULL_REQUEST_TEMPLATE.md) - [Project License](LICENSE.md) - [Documentation](http://singularity.lbl.gov/) - [Citation](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459) # Singularity - Enabling users to have full control of their environment. Starting a Singularity container "swaps" out the host operating system environment for one the user controls! Let's say you are running Ubuntu on your workstation or server, but you have an application which only runs on Red Hat Enterprise Linux 6.3. Singularity can instantly virtualize the operating system, without having root access, and allow you to run that application in its native environment! # About Singularity is a container platform focused on supporting "Mobility of Compute" Mobility of Compute encapsulates the development to compute model where developers can work in an environment of their choosing and creation and when the developer needs additional compute resources, this environment can easily be copied and executed on other platforms. Additionally as the primary use case for Singularity is targeted towards computational portability, many of the barriers to entry of other container solutions do not apply to Singularity making it an ideal solution for users (both computational and non-computational) and HPC centers. ## The Container Singularity utilizes container images, which means when you enter and work within the Singularity container, you are physically located inside of this image. The image grows and shrinks in real time as you install or delete files within the container. If you want to copy a container, you copy the image. Using a single image for the container format, has added advantages especially within the context of HPC with large parallel file systems because all metadata operations within the container occur within the container image (and not on the metadata server!). ## Mobility of Compute With Singularity, developers who like to be able to easily control their own environment will love Singularity's flexibility. Singularity does not provide a pathway for escalation of privilege (as do other container platforms which are thus not applicable for multi-tenant resources) so you must be able to become root on the host system (or virtual machine) in order to modify the container. A Singularity container can be launched in a variety of different ways depending on what you wanted to do with it. A simple method might be to launch an interactive shell within the container image as follows: [gmk@centos7-x64 demo]$ singularity shell /tmp/Centos-7.img gmk@Centos-7.img demo> echo "Hello from within the container" Hello from within the container gmk@Centos-7.img demo> whoami gmk gmk@Centos-7.img demo> And if you wanted to do the same thing as root: [gmk@centos7-x64 demo]$ sudo singularity shell -w /tmp/Centos-7.img root@Centos-7.img demo> whoami root root@Centos-7.img demo> *note: By default, Singularity launches the container image in read only mode (so it can be easily launched in parallel). The -w option used above tells Singularity to mount the image in read/write mode such that root can now make changes to the container.* Additionally relevant file systems on your host are automatically shared within the context of your container. This can be demonstrated as follows: [gmk@centos7-x64 demo]$ pwd /home/gmk/demo [gmk@centos7-x64 demo]$ echo "world" > hello [gmk@centos7-x64 demo]$ singularity shell /tmp/Centos-7.img gmk@Centos-7.img demo> pwd /home/gmk/demo gmk@Centos-7.img demo> cat hello world Once the developer has completed their environment the image file can be compressed and copied to any other system that has Singularity installed. If you do not have root on that system, you will not be able to make any changes to the image once on that system. But you will be able to use the container and access the data and files outside the container as easily as you would on your development system or virtual machine. ## Portability of Singularity container images Singularity images are highly portable between Linux distributions (as long as the binary format is the same). You can generate your image on Debian or CentOS, and run it on Mint or Slackware. Within a particular container one can include their programs, data, scripts and pipelines and thus portable to any other architecture compatible Linux system or distribution. ## Bootstrapping new images Generally when bootstrapping an image from scratch you must build it from a compatible host. This is because you must use the distribution specific tools it comes with (e.g. Red Hat does not provide Debian's debootstrap). But once the image has been bootstrapped and includes the necessary bits to be self hosting (e.g. YUM on CentOS and apt-get on Debian/Ubuntu) then the process of managing the container can be implemented from within the container. The process of building a bootstrap starts with a definition specification. The definition file describes how you want the operating system to be built, what should go inside it and any additional modifications necessary. Here is an example of a very simple bootstrap definition file for CentOS: BootStrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/ Include: yum Once you have created your bootstrap definition, you can build your Singularity container image by first creating a blank image, and then bootstrapping using your definition file: [gmk@centos7-x64 demo]$ sudo singularity create /tmp/Centos-7.img [gmk@centos7-x64 demo]$ sudo singularity bootstrap /tmp/Centos-7.img centos.def From there we can immediately start using the container: [gmk@centos7-x64 demo]$ singularity exec /tmp/Centos-7.img cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) [gmk@centos7-x64 demo]$ singularity exec /tmp/Centos-7.img python --version Python 2.7.5 [gmk@centos7-x64 demo]$ singularity exec /tmp/Centos-7.img python hello.py hello world [gmk@centos7-x64 demo]$ And if I do this same process again, while changing the **OSVersion** variable in the bootstrap definition to **6** (where previously it was automatically ascertained by querying the RPM database), we can essentially build a CentOS-6 image in exactly the same manner as above. Doing so reveals this: [gmk@centos7-x64 demo]$ singularity exec /tmp/Centos-6.img cat /etc/redhat-release CentOS release 6.7 (Final) [gmk@centos7-x64 demo]$ singularity exec /tmp/Centos-6.img python --version Python 2.6.6 [gmk@centos7-x64 demo]$ And as expected, the Python version we now see is what comes from by default in CentOS-6. # Cite as: ``` Kurtzer GM, Sochat V, Bauer MW (2017) Singularity: Scientific containers for mobility of compute. PLoS ONE 12(5): e0177459. https://doi.org/10.1371/journal.pone.0177459 ``` We also have a Zenodo citation: ``` Kurtzer, Gregory M.. (2016). Singularity 2.1.2 - Linux application and environment containers for science. 10.5281/zenodo.60736 http://dx.doi.org/10.5281/zenodo.60736 ``` # Webpage We have full documentation at [http://singularity.lbl.gov/](http://singularity.lbl.gov/), and [welcome contributions](http://www.github.com/singularityware/singularityware.github.io). singularity-2.4.2/CHANGELOG.md0000644000175000017500000002366113211621077014650 0ustar mehdimehdi# CHANGELOG This is a manually generated log to track changes to the repository for each release. Each section should include general headers such as ### Implemented enhancements and **Merged pull requests**. All closed issued and bug fixes should be represented by the pull requests that fixed them. This log originated with Singularity 2.4 and changes prior to that are (unfortunately) done retrospectively. Critical items to know are: - renamed, deprecaed, or removed commands - defaults that are changed - backward incompatible changes (recipe file format? image file format?) - migration guidance (how to convert images?) - changed behaviour (recipe sections work differently) ## [v2.4.2](https://github.com/singularityware/singularity/tree/release-2.4) - This fixed an issue for support of older distributions and kernels with regards to `setns()` functionality. - Fixed autofs bug path (lost during merge) ## [v2.4.1](https://github.com/singularityware/singularity/tree/release-2.4) (2017-11-22) ### Security related fixes - Fixed container path and owner limitations (original merge was lost) - Check of overlay upper/work images are symlinks ### Implemented enhancements - This changelog was added. - Addition of APP[app]_[LABELS,ENV,RUNSCRIPT,META] so apps can internally find one another. - Exposing labels for SCI-F in environment ### Bug Fixes - Adjusting environment parsing regular expression for Docker to allow for "=" sign in variable - Try overlayFS now default option - Confirm that localstate directories were properly packaged - Fix when running over NFS with root_squash enabled - Honor the user name request when pulling from Singularity Hub - Allow http_proxy envar for runtime and build - Properly require mksquashfs tools for Debian packaging - Fix for empty docker namespaces in private repositories - Fix Docker environment parsing - Revert lolcow easter egg - Fix "Duplicate bootstrap definition key" triggered by comments and blank spaces - Fix for docker permission error when downloading multiple layers - Fix parsing of registry (including port), namespace, tags, and version - Add "$@" to any CMD/ENTRYPOINT found when building from Docker - Added sqaushfs-tools as a dependency for building deb files - Fix terminal echo problem when using PID namespace and killing shell - Fix SuSE squashFS package name in RPM spec ## [v2.4](https://github.com/singularityware/singularity/tree/v2.4) (2017-10-02) [Full Changelog](https://github.com/singularityware/singularity/compare/2.3.2...2.4) ### Implemented enhancements - a new `build` command was added to replace `create` + `bootstrap` ([build](https://singularityware.github.io/docs-build-container)) - default image format is squashfs, eliminating the need to specify a size - for development build supports `--sandbox` (folder) and `--writable` (ext3) - a `localimage` can be used as a build base, including ext3, sandbox, and other squashfs images - singularity hub can now be used as a base with the uri `shub://` - support has been added for instances (services) including network namespace isolation under the `instances` group of commands. - [singularity registry](https://www.github.com/singularityhub/sregistry) is released and published - [Standard Container Integration Format](https://singularityware.github.io/docs-apps) apps are added to support internal modularity and organization. - [build environment](https://singularityware.github.io/build-environment) is better documented - Persistent Overlay - Container checks - Tests for instance support - Wrapper for create - Group instance commands - Group image commands - Bash completion updates ### Deprecated - the `create` command is being deprecated in favor of `image.create` - `bootstrap` is being deprecated in favor of `build` (will work through 2.4) - `expand` is being deprecated in favor of `image.expand`, and no longer works on images with headers (meaning after they are built). - `export` is being deprecated and added to the image command group, `image.export` - the `shub://` URI no longer supports an integer to reference a container ## [v2.3.2](https://github.com/singularityware/singularity/tree/v2.3.2) (2017-09-15) [Full Changelog](https://github.com/singularityware/singularity/compare/2.3.1...2.3.2) ### Implemented enhancements - Quick fix to support manifest lists when pulling from Docker Hub ## [v2.3.1](https://github.com/singularityware/singularity/tree/v2.3.1) (2017-06-26) [Full Changelog](https://github.com/singularityware/singularity/compare/2.3...2.3.1) ### Security Fix - A fix was implemented to address an escalation pathway and various identified bugs and potential race conditions. ## [v2.3](https://github.com/singularityware/singularity/tree/v2.3) (2017-05-31) [Full Changelog](https://github.com/singularityware/singularity/compare/2.2.1...2.3) ### Implemented enhancements - Lots of backend library changes to accommodate a more flexible API - Restructured Python backend - Updated bootstrap backend to make it much more reliable - Direct support for Singularity-Hub - Ability to run additional commands without root privileges (e.g. create, import, copy, export, etc..). - Added ability to pull images from Singularity Hub and Docker - Containers now have labels, and are inspect'able ## [v2.2.1](https://github.com/singularityware/singularity/tree/v2.2.1) (2017-02-14) [Full Changelog](https://github.com/singularityware/singularity/compare/2.2...2.2.1) ### Security Fix - a security loophole related to mount devices was fixed (thanks @UMU in Sweden) ### Implemented enhancements - Fixed some leaky file descriptors - Cleaned up `*printf()` usage - Catch if user's group is not properly defined ## [v2.2](https://github.com/singularityware/singularity/tree/v2.2) (2016-10-11) [Full Changelog](https://github.com/singularityware/singularity/compare/2.1.2...2.2) ### Implemented enhancements - A complete rework of the back end source code to allow a much larger feature set, sanity, and facilitate contributions - The ability to execute completely unprivileged (does not support Singularity images) (thanks to Brian Bockelman) - Container execute by URI support (file, http, https, docker, etc..) - Integration with the Docker Registry Remote API (thanks to @vsoch), including stateless containers running ad-hoc, bootstrapping, and importing - OverlayFS support - Allows for automatic creation of bind points within containers at runtime (thanks to Amanda Duffy and Jarrod Johnson) - Additional container formats supported (directories and archives) - New bootstrap definition format to handle much more complicated and intuitive recipes - All Singularity 2.x containers continue to be supported with this release. ## [v2.1.2](https://github.com/singularityware/singularity/tree/v2.1.2) (2016-08-04) [Full Changelog](https://github.com/singularityware/singularity/compare/2.1.1...2.1.2) ### Bug Fixes - Fix for kernel panic on corrupt images - Fixes build warning ## [v2.1.1](https://github.com/singularityware/singularity/tree/v2.1.1) (2016-08-03) [Full Changelog](https://github.com/singularityware/singularity/compare/2.1...2.1.1) ### Bug Fixes - Contain option no longer maintains current working directory - Remove need to obtain a shared lock on the image (was failing on some shared file systems) - Move creation of a container's /environment to the beginning of the bootstrap (so it can be modified via a bootstrap definition file ## [v2.1](https://github.com/singularityware/singularity/tree/v2.1) (2016-07-28) [Full Changelog](https://github.com/singularityware/singularity/compare/2.0...2.1) ### Implemented enhancements - Configuration file for system administrator control over what Singularity features users are allowed to use - Support for non Gnu LibC based distributions (e.g. Alpine Linux) - Source file restructuring and refactoring - Added message(), and enabled very verbose debugging - Be smarter about when to avoid separation of the PID namespace - Log container runs to syslog() - Support custom container environments (via container:/environment) - Sanitized source files for Flawfinder ### Bug Fixes - Fix bug with /run and /var directories being read only in some situations - Fix lots of bootstrap definition issues - Fixed issue with /dev/pts not being mounted within a container - Resolved some issues with image file de-looping - Fixed bugs related to very restrictive umasks set ## [v2.0](https://github.com/singularityware/singularity/tree/v2.0) (2016-06-01) [Full Changelog](https://github.com/singularityware/singularity/compare/1.x...2.0) ### Implemented enhancements - Support for non-root container contexts (user outside container, is same user inside container) - Support of “live” container sparse image files - Utilizing the operating system’s build and dependency resolution subsystems (e.g. YUM, Apt, etc.) - Support for Open MPI 2.1 (pre-release) - Updates for usage with non-local file systems - Performance optimizations - Support for native X11 ## [v1.x](https://github.com/singularityware/singularity/tree/v1.x) (2016-04-06) ### Implemented enhancements - Ability to create Singularity containers based on a package specfile - Specfile templates can be generated automatically (singularity specgen …) - Support for various automatic dependency resolution - Dynamic libraries - Perl scripts and modules - Python scripts and modules - R scripts and modules - Basic X11 support - Open MPI (v2.1 - which is not yet released) - Direct execution of Singularity containers (e.g. ./container.sapp [opts]) - Access to files in your home directory and a scratch directory - Existing IO (pipes, stdio, stderr, and stdin) all maintained through container - Singularity internal container cache management - Standard networking access (exactly as it does on the host) - Singularity containers run within existing resource contexts (CGroups and ulimits are maintained) - Support for scalable execution of MPI parallel jobs - Singularity containers are portable between Linux distributions singularity-2.4.2/NEWS0000644000175000017500000000000013211621077013514 0ustar mehdimehdisingularity-2.4.2/singularity.spec.in0000644000175000017500000001146213211621077016666 0ustar mehdimehdi# # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016, The Regents of the University of California, through # Lawrence Berkeley National Laboratory (subject to receipt of any required # approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # %{!?_rel:%{expand:%%global _rel 1}} Summary: Application and environment virtualization Name: singularity Version: @PACKAGE_VERSION@ Release: %{_rel}%{?dist} # https://spdx.org/licenses/BSD-3-Clause-LBNL.html License: BSD-3-Clause-LBNL Group: System Environment/Base URL: http://singularity.lbl.gov/ Source: %{name}-%{version}.tar.gz ExclusiveOS: linux BuildRoot: %{?_tmppath}%{!?_tmppath:/var/tmp}/%{name}-%{version}-%{release}-root BuildRequires: python %if "%{_target_vendor}" == "suse" Requires: squashfs %else Requires: squashfs-tools %endif Requires: %{name}-runtime = %{version}-%{release} %description Singularity provides functionality to make portable containers that can be used across host environments. %package devel Summary: Development libraries for Singularity Group: System Environment/Development %description devel Development files for Singularity %package runtime Summary: Support for running Singularity containers Group: System Environment/Base %description runtime This package contains support for running containers created by the %{name} package. %prep %setup %build if [ ! -f configure ]; then ./autogen.sh fi %configure %{__make} %{?mflags} %install %{__make} install DESTDIR=$RPM_BUILD_ROOT %{?mflags_install} rm -f $RPM_BUILD_ROOT/%{_libdir}/singularity/lib*.la %post runtime -p /sbin/ldconfig %postun runtime -p /sbin/ldconfig %clean rm -rf $RPM_BUILD_ROOT %files %doc examples CONTRIBUTORS.md CONTRIBUTING.md COPYRIGHT.md INSTALL.md LICENSE-LBNL.md LICENSE.md README.md %attr(0755, root, root) %dir %{_sysconfdir}/singularity %attr(0644, root, root) %config(noreplace) %{_sysconfdir}/singularity/* %{_libexecdir}/singularity/cli/apps.* %{_libexecdir}/singularity/cli/bootstrap.* %{_libexecdir}/singularity/cli/build.* %{_libexecdir}/singularity/cli/check.* %{_libexecdir}/singularity/cli/create.* %{_libexecdir}/singularity/cli/image.* %{_libexecdir}/singularity/cli/inspect.* %{_libexecdir}/singularity/cli/mount.* %{_libexecdir}/singularity/cli/pull.* %{_libexecdir}/singularity/cli/selftest.* %{_libexecdir}/singularity/handlers %{_libexecdir}/singularity/helpers %{_libexecdir}/singularity/image-handler.sh %{_libexecdir}/singularity/python # Binaries %{_libexecdir}/singularity/bin/builddef %{_libexecdir}/singularity/bin/cleanupd %{_libexecdir}/singularity/bin/get-section %{_libexecdir}/singularity/bin/mount %{_libexecdir}/singularity/bin/image-type %{_libexecdir}/singularity/bin/prepheader # Directories %{_libexecdir}/singularity/bootstrap-scripts #SUID programs %attr(4755, root, root) %{_libexecdir}/singularity/bin/mount-suid %files runtime %dir %{_libexecdir}/singularity %dir %{_localstatedir}/singularity %dir %{_localstatedir}/singularity/mnt %dir %{_localstatedir}/singularity/mnt/session %dir %{_localstatedir}/singularity/mnt/container %dir %{_localstatedir}/singularity/mnt/overlay %dir %{_localstatedir}/singularity/mnt/final %{_bindir}/singularity %{_bindir}/run-singularity %{_libdir}/singularity/lib*.so.* %{_libexecdir}/singularity/cli/action_argparser.* %{_libexecdir}/singularity/cli/exec.* %{_libexecdir}/singularity/cli/help.* %{_libexecdir}/singularity/cli/instance.* %{_libexecdir}/singularity/cli/run.* %{_libexecdir}/singularity/cli/shell.* %{_libexecdir}/singularity/cli/test.* %{_libexecdir}/singularity/bin/action %{_libexecdir}/singularity/bin/start %{_libexecdir}/singularity/functions %dir %{_sysconfdir}/singularity %config(noreplace) %{_sysconfdir}/singularity/* %{_mandir}/man1/singularity.1* %dir %{_sysconfdir}/bash_completion.d %{_sysconfdir}/bash_completion.d/singularity #SUID programs %attr(4755, root, root) %{_libexecdir}/singularity/bin/action-suid %attr(4755, root, root) %{_libexecdir}/singularity/bin/start-suid %files devel %defattr(-, root, root) %{_libdir}/singularity/lib*.so %{_libdir}/singularity/lib*.a %{_includedir}/singularity/*.h %changelog singularity-2.4.2/LICENSE.md0000644000175000017500000000264313211621077014440 0ustar mehdimehdiRedistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. singularity-2.4.2/configure.ac0000644000175000017500000003307413211621077015324 0ustar mehdimehdiAC_PREREQ(2.59) AC_INIT([singularity],[2.4.2],[gmkurtzer@gmail.com]) if test -z "$prefix" -o "$prefix" = "NONE" ; then prefix=${ac_default_prefix} fi AC_SUBST(PREFIX, $prefix) AC_CANONICAL_TARGET case $target_cpu in x86_64) SINGULARITY_ARCH=x86_64 ;; i?86) SINGULARITY_ARCH=i386 ;; athlon) SINGULARITY_ARCH=i386 ;; *) SINGULARITY_ARCH="$target_cpu" ;; esac AC_SUBST(SINGULARITY_ARCH) # This should be used per autogen.sh output, but we hit this automake bug: # https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1232579.html #AM_INIT_AUTOMAKE([foreign subdir-objects]) AM_INIT_AUTOMAKE([foreign subdir-objects]) ## SEE ABOVE BEFORE CHANGING AC_CONFIG_SRCDIR([src]) AC_CONFIG_HEADERS([src/config.h]) AC_CONFIG_MACRO_DIR([m4]) AC_GNU_SOURCE AC_PROG_INSTALL AC_PROG_LIBTOOL AC_PROG_CC AM_INIT_AUTOMAKE AM_PROG_CC_C_O AC_ENABLE_SHARED AC_PROG_LIBTOOL(libtool) # Setting rpath if necessary if test "$libdir" = "\${exec_prefix}/lib"; then LDFLAGS="$LDFLAGS -Wl,-rpath -Wl,\$(libdir)" fi AC_MSG_CHECKING([for namespace: CLONE_NEWPID]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#define _GNU_SOURCE #include ]], [[unshare(CLONE_NEWPID);]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DNS_CLONE_NEWPID" ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([for namespace: CLONE_PID]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#define _GNU_SOURCE #include ]], [[unshare(CLONE_PID);]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DNS_CLONE_PID" ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([for namespace: CLONE_FS]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#define _GNU_SOURCE #include ]], [[unshare(CLONE_FS);]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DNS_CLONE_FS" ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([for namespace: CLONE_NEWNS]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#define _GNU_SOURCE #include ]], [[unshare(CLONE_NEWNS);]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DNS_CLONE_NEWNS" ], [ AC_MSG_RESULT([no]) echo echo "ERROR!!!!!!" echo echo "This host does not support the CLONE_NEWNS (mount) namespace flag! You" echo "really really really don't want to run Singularity containers without a" echo "Separate mount name namespace!" echo exit 255 ] ) AC_MSG_CHECKING([for namespace: CLONE_NEWUSER]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#define _GNU_SOURCE #include ]], [[unshare(CLONE_NEWUSER);]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DNS_CLONE_NEWUSER" ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([for namespace: CLONE_NEWIPC]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#define _GNU_SOURCE #include ]], [[unshare(CLONE_NEWIPC);]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DNS_CLONE_NEWIPC" ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([for namespace: CLONE_NEWNET]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#define _GNU_SOURCE #include ]], [[unshare(CLONE_NEWNET);]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DNS_CLONE_NEWNET" ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([for feature: NO_NEW_PRIVS]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #include ]], [[prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0); prctl(PR_GET_NO_NEW_PRIVS, 0, 0, 0, 0);]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DSINGULARITY_NO_NEW_PRIVS" ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([for feature: MS_SLAVE]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #include ]], [[#ifndef MS_SLAVE #error failed #endif ]])], [ AC_MSG_RESULT([yes]) SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DSINGULARITY_MS_SLAVE" ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([for feature: MS_REC]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #include ]], [[#ifndef MS_REC #error failed #endif ]])], [ AC_MSG_RESULT([yes]) ], [ AC_MSG_RESULT([no]) echo echo "ERROR!!!!!!" echo echo "This host does not support the MS_REC mount option!" echo exit 255 ] ) AC_MSG_CHECKING([for feature: MS_PRIVATE]) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #include ]], [[#ifndef MS_PRIVATE #error failed #endif ]])], [ AC_MSG_RESULT([yes]) ], [ AC_MSG_RESULT([no]) echo echo "ERROR!!!!!!" echo echo "This host does not support the MS_PRIVATE mount option!" echo exit 255 ] ) AC_ARG_WITH([userns], AS_HELP_STRING([--with-userns], [Enable use of user namespaces]), [ SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DSINGULARITY_USERNS" USER_NS=1 ] ) AC_SUBST(USER_NS) AC_CHECK_FUNCS(setns, [ ], [ SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DSINGULARITY_NO_SETNS" AC_MSG_CHECKING([for setns kernel support]) if test -d "/proc/self/ns"; then SINGULARITY_DEFINES="$SINGULARITY_DEFINES -DSINGULARITY_SETNS_SYSCALL" AC_MSG_RESULT([yes]) else WARN_SETNS=1 AC_MSG_RESULT([no]) fi ] ) AC_SUBST(SINGULARITY_DEFINES) # --------------------------------------------------------------------- # PYTHON # --------------------------------------------------------------------- AC_CHECK_PROG(PYTHON_CHECK,python,yes) if test x"$PYTHON_CHECK" != x"yes" ; then AC_MSG_ERROR([Please install python before installing.]) else PYTHON_MODULES="base64 datetime glob hashlib io itertools json math multiprocessing pickle pwd re shutil signal subprocess stat sys tarfile tempfile time" for PYTHON_MODULE in $PYTHON_MODULES; do AC_MSG_CHECKING([for the $PYTHON_MODULE python module]) python_module_result=`python -c "import $PYTHON_MODULE" 2>&1` if test -z "$python_module_result"; then AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) AC_MSG_ERROR([cannot import Python module $PYTHON_MODULE. Please check your Python installation. The error was: $python_module_result]) fi done fi AC_MSG_CHECKING([--with-slurm]) AC_ARG_WITH([slurm], AS_HELP_STRING([--with-slurm], [This feature will no longer be part of Singularity proper]), ) AS_IF([test "x$with_slurm" == "xyes"], [ AC_MSG_RESULT([yes]) echo echo "ERROR: Slurm support is no longer maintained in Singularity proper and will" echo " be maintained as a subproject on GitHub." exit 1 ], [ AC_MSG_RESULT([no]) ] ) AC_MSG_CHECKING([if suid should be enabled]) AC_ARG_ENABLE([suid], AS_HELP_STRING([--disable-suid], [Disable build for SUID (only works for new kernels)])) AS_IF([test "x$enable_suid" != "xno"], [ AC_MSG_RESULT([yes]) BUILD_SUID="action-suid start-suid mount-suid" ], [ AC_MSG_RESULT([no]) BUILD_SUID="" ]) AC_SUBST(BUILD_SUID) AC_MSG_CHECKING([for git version]) if BRANCH=`git rev-parse --abbrev-ref HEAD 2>/dev/null`; then if HASH=`git rev-parse --short HEAD 2>/dev/null`; then GIT_VERSION="-$BRANCH.g$HASH" else GIT_VERSION="-$BRANCH" fi else GIT_VERSION="-dist" fi AC_MSG_RESULT([$GIT_VERSION]) AC_SUBST(GIT_VERSION) AC_DEFINE(CONTAINER_MOUNTDIR, LOCALSTATEDIR "/singularity/mnt/container", "container mount location") AC_DEFINE(CONTAINER_FINALDIR, LOCALSTATEDIR "/singularity/mnt/final", "location of container post overlay") AC_DEFINE(CONTAINER_OVERLAY, LOCALSTATEDIR "/singularity/mnt/overlay", "location of container post overlay") AC_DEFINE(SESSIONDIR, LOCALSTATEDIR "/singularity/mnt/session", "location of session directory") AC_SUBST(CONTAINER_MOUNTDIR, "$localstatedir/singularity/mnt/container") AC_SUBST(CONTAINER_FINALDIR, "$localstatedir/singularity/mnt/final") AC_SUBST(CONTAINER_OVERLAY, "$localstatedir/singularity/mnt/overlay") AC_SUBST(SESSIONDIR, "$localstatedir/singularity/mnt/session") AC_CONFIG_FILES([ Makefile singularity.spec test.sh src/Makefile src/lib/Makefile src/lib/runtime/Makefile src/lib/runtime/ns/Makefile src/lib/runtime/ns/ipc/Makefile src/lib/runtime/ns/mnt/Makefile src/lib/runtime/ns/net/Makefile src/lib/runtime/ns/pid/Makefile src/lib/runtime/files/Makefile src/lib/runtime/files/group/Makefile src/lib/runtime/files/passwd/Makefile src/lib/runtime/files/resolvconf/Makefile src/lib/runtime/files/libs/Makefile src/lib/runtime/mounts/Makefile src/lib/runtime/mounts/cwd/Makefile src/lib/runtime/mounts/dev/Makefile src/lib/runtime/mounts/binds/Makefile src/lib/runtime/mounts/home/Makefile src/lib/runtime/mounts/hostfs/Makefile src/lib/runtime/mounts/kernelfs/Makefile src/lib/runtime/mounts/tmp/Makefile src/lib/runtime/mounts/userbinds/Makefile src/lib/runtime/mounts/scratch/Makefile src/lib/runtime/enter/Makefile src/lib/runtime/enter/chroot/Makefile src/lib/runtime/environment/Makefile src/lib/runtime/overlayfs/Makefile src/lib/runtime/autofs/Makefile src/lib/image/Makefile src/lib/image/squashfs/Makefile src/lib/image/dir/Makefile src/lib/image/ext3/Makefile src/action-lib/Makefile src/bootstrap-lib/Makefile src/util/Makefile src/util/config_defaults.h etc/Makefile bin/Makefile bin/singularity man/Makefile libexec/Makefile libexec/bootstrap-scripts/Makefile libexec/bootstrap-scripts/environment/Makefile libexec/cli/Makefile libexec/handlers/Makefile libexec/helpers/Makefile libexec/helpers/apps/Makefile libexec/helpers/checks/Makefile libexec/python/Makefile libexec/python/docker/Makefile libexec/python/shub/Makefile libexec/python/helpers/Makefile libexec/python/helpers/json/Makefile ]) AC_OUTPUT if test -n "$WARN_SETNS"; then echo echo "WARNING: instance feature is disabled due to lack of kernel support" echo fi singularity-2.4.2/src/0000755000175000017500000000000013211621077013616 5ustar mehdimehdisingularity-2.4.2/src/action.c0000644000175000017500000001344013211621077015241 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/daemon.h" #include "util/registry.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "util/sessiondir.h" #include "util/cleanupd.h" #include "./action-lib/include.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif int main(int argc, char **argv) { struct image_object image; char *pwd = get_current_dir_name(); char *target_pwd = NULL; char *command = NULL; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_priv_init(); singularity_suid_init(argv); singularity_registry_init(); singularity_priv_userns(); singularity_priv_drop(); singularity_runtime_autofs(); singularity_daemon_init(); if ( singularity_registry_get("WRITABLE") != NULL ) { singularity_message(VERBOSE3, "Instantiating writable container image object\n"); image = singularity_image_init(singularity_registry_get("IMAGE"), O_RDWR); } else { singularity_message(VERBOSE3, "Instantiating read only container image object\n"); image = singularity_image_init(singularity_registry_get("IMAGE"), O_RDONLY); } if ( singularity_registry_get("DAEMON_JOIN") == NULL ) { singularity_cleanupd(); singularity_runtime_ns(SR_NS_ALL); singularity_sessiondir(); singularity_image_mount(&image, CONTAINER_MOUNTDIR); action_ready(); singularity_runtime_overlayfs(); singularity_runtime_mounts(); singularity_runtime_files(); } else { singularity_runtime_ns(SR_NS_ALL); } singularity_runtime_enter(); singularity_runtime_environment(); singularity_priv_drop_perm(); if ( singularity_registry_get("CONTAIN") != NULL ) { singularity_message(DEBUG, "Attempting to chdir to home: %s\n", singularity_priv_home()); if ( chdir(singularity_priv_home()) != 0 ) { singularity_message(WARNING, "Could not chdir to home: %s\n", singularity_priv_home()); if ( chdir("/") != 0 ) { singularity_message(ERROR, "Could not change directory within container.\n"); ABORT(255); } } } else if ( ( target_pwd = singularity_registry_get("TARGET_PWD") ) != NULL ) { singularity_message(DEBUG, "Attempting to chdir to TARGET_PWD: %s\n", target_pwd); if ( chdir(target_pwd) != 0 ) { singularity_message(ERROR, "Could not change directory to: %s\n", target_pwd); ABORT(255); } } else if ( pwd != NULL ) { singularity_message(DEBUG, "Attempting to chdir to CWD: %s\n", pwd); if ( chdir(pwd) != 0 ) { singularity_message(VERBOSE, "Could not chdir to current dir: %s\n", pwd); if ( chdir(singularity_priv_home()) != 0 ) { singularity_message(WARNING, "Could not chdir to home: %s\n", singularity_priv_home()); if ( chdir("/") != 0 ) { singularity_message(ERROR, "Could not change directory within container.\n"); ABORT(255); } } } } else { singularity_message(ERROR, "Could not obtain current directory.\n"); ABORT(255); } free(target_pwd); command = singularity_registry_get("COMMAND"); envar_set("SINGULARITY_CONTAINER", singularity_image_name(&image), 1); // Legacy PS1 support envar_set("SINGULARITY_NAME", singularity_image_name(&image), 1); envar_set("SINGULARITY_SHELL", singularity_registry_get("SHELL"), 1); envar_set("SINGULARITY_APPNAME", singularity_registry_get("APPNAME"), 1); singularity_message(LOG, "USER=%s, IMAGE='%s', COMMAND='%s'\n", singularity_priv_getuser(), singularity_image_name(&image), singularity_registry_get("COMMAND")); if ( command == NULL ) { singularity_message(INFO, "No action command verb was given, invoking 'shell'\n"); action_shell(argc, argv); // Primary Commands } else if ( strcmp(command, "shell") == 0 ) { action_shell(argc, argv); } else if ( strcmp(command, "exec") == 0 ) { action_exec(argc, argv); } else if ( strcmp(command, "inspect") == 0 ) { action_exec(argc, argv); } else if ( strcmp(command, "run") == 0 ) { action_run(argc, argv); } else if ( strcmp(command, "test") == 0 ) { action_test(argc, argv); } else { singularity_message(ERROR, "Unknown action command verb was given\n"); ABORT(255); } return(0); } singularity-2.4.2/src/slurm/0000755000175000017500000000000013211621077014760 5ustar mehdimehdisingularity-2.4.2/src/slurm/README.md0000644000175000017500000000215413211621077016241 0ustar mehdimehdi Singularity plugin for SLURM ============================ This plugin allows users to execute their SLURM jobs within a Singularity container without having to execute Singularity directly. This assists in simplifying the invocation of the container and hiding the implementation details. To enable the plugin, add the following line to the SLURM plugin configuration (`/etc/slurm/plugstack.conf`): ``` required singularity.so ``` This works if Singularity is installed as a system package. If the install prefix is `/opt/singularity`, then one would have: ``` required /opt/singularity/lib/slurm/singularity.so ``` Note that the sysadmin may provide a default image that will be utilized if the user doesn't provide one: ``` required singularity.so default_image=/cvmfs/cernvm-prod.cern.ch/cvm3 ``` Finally, a user may select their image through the `--singularity-image` optional argument: ``` srun --singularity-image=/cvmfs/cms.cern.ch/rootfs/x86_64/centos7/latest ls -lh / ``` Within a batch file, you would append this header: ``` #SBATCH --singularity-image=/cvmfs/cms.cern.ch/rootfs/x86_64/centos7/latest ``` singularity-2.4.2/src/slurm/singularity.c0000644000175000017500000002402013211621077017474 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE 1 #include #include #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/file.h" #include "util/registry.h" #include "lib/image/image.h" #include "util/suid.h" #include "util/sessiondir.h" #include "util/cleanupd.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "action-lib/include.h" #include "slurm/spank.h" SPANK_PLUGIN(singularity, 1); #ifndef SYSCONFDIR #define SYSCONFDIR "/etc" #endif #define INT_MAX_STRING_SIZE 30 // These should only be set post-fork and before exec; // this means reading / writing to these should be thread-safe. static char job_uid_str[INT_MAX_STRING_SIZE]; //Flawfinder: ignore static char job_gid_str[INT_MAX_STRING_SIZE]; //Flawfinder: ignore static char *job_image = NULL; static char *job_bindpath = NULL; static int setup_container_environment(spank_t spank) { uid_t job_uid = -1; gid_t job_gid = -1; char *job_cwd = NULL; char *bindpath = NULL; int argc; char **argv = NULL; int i; setenv("SINGULARITY_MESSAGELEVEL", "1", 0); //Don't overwrite if exists if (ESPANK_SUCCESS != spank_get_item(spank, S_JOB_UID, &job_uid)) { slurm_error("spank/%s: Failed to get job's target UID", plugin_name); return -1; } if (INT_MAX_STRING_SIZE <= snprintf(job_uid_str, INT_MAX_STRING_SIZE, "%u", job_uid)) { // Flawfinder: ignore slurm_error("spank/%s: Failed to serialize job's UID to string", plugin_name); return -1; } if (setenv("SINGULARITY_TARGET_UID", job_uid_str, 1) < 0) { slurm_error("spank/%s: Failed to setenv(\"SINGULARITY_TARGET_UID\")", plugin_name); return -1; } if (ESPANK_SUCCESS != spank_get_item(spank, S_JOB_GID, &job_gid)) { slurm_error("spank/%s: Failed to get job's target GID", plugin_name); return -1; } if (INT_MAX_STRING_SIZE <= snprintf(job_gid_str, INT_MAX_STRING_SIZE, "%u", job_gid)) { // Flawfinder: ignore slurm_error("spank/%s: Failed to serialize job's GID to string", plugin_name); return -1; } if (setenv("SINGULARITY_TARGET_GID", job_gid_str, 1) < 0) { slurm_error("spank/%s: Failed to setenv(\"SINGULARITY_TARGET_GID\")", plugin_name); return -1; } job_cwd = get_current_dir_name(); if (!job_cwd) { slurm_error("spank/%s: Failed to determine job's correct PWD: %s", plugin_name, strerror(errno)); return -1; } if (setenv("SINGULARITY_TARGET_PWD", job_cwd, 1) < 0) { slurm_error("spank/%s: Failed to setenv(\"SINGULARITY_TARGET_PWD\")", plugin_name); return -1; } /* setenv() makes a copy */ free(job_cwd); if (!job_image) { slurm_error("spank/%s: Unable to determine job's image file.", plugin_name); return -1; } if (setenv("SINGULARITY_IMAGE", job_image, 1) < 0) { slurm_error("spank/%s: Failed to setenv(\"SINGULARITY_IMAGE\")", plugin_name); return -1; } if ((job_bindpath) && (setenv("SINGULARITY_BINDPATH", job_bindpath, 1) < 0)) { slurm_error("spank/%s: Failed to setenv(\"SINGULARITY_BINDPATH\")", plugin_name); return -1; } return 0; } static int setup_container_cwd() { singularity_message(DEBUG, "Trying to change directory to where we started\n"); char *target_pwd = singularity_registry_get("TARGET_PWD"); if (!target_pwd || (chdir(target_pwd) < 0)) { singularity_message(ERROR, "Failed to change into correct directory " "(%s) inside container.", target_pwd ? target_pwd : "UNKNOWN"); return -1; } free(target_pwd); return 0; } static int setup_container(spank_t spank) { int rc; struct image_object image; char *command = NULL; if ((rc = setup_container_environment(spank)) != 0) { return rc; } /* * Ugg, singularity_* calls tend to call ABORT(255), which translates to * exit(255), all over the place. The slurm SPANK hook API may not * expect such sudden death of the pending slurm task. I've left * a bunch of following "return rc;" commented out, as the failure * conditions from singularity_* calls isn't clear to me. */ // Before we do anything, check privileges and drop permission singularity_priv_init(); singularity_priv_drop(); singularity_message(VERBOSE, "Running SLURM/Singularity integration " "plugin\n"); if ((rc = singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf"))) != 0) { return rc; } singularity_priv_init(); //TODO singularity_suid_init(argv); singularity_registry_init(); singularity_priv_userns(); singularity_priv_drop(); singularity_cleanupd(); singularity_runtime_ns(SR_NS_ALL); singularity_sessiondir(); image = singularity_image_init(singularity_registry_get("IMAGE")); if ( singularity_registry_get("WRITABLE") == NULL ) { singularity_image_open(&image, O_RDONLY); } else { singularity_image_open(&image, O_RDWR); } singularity_image_check(&image); singularity_image_bind(&image); singularity_image_mount(&image, singularity_runtime_rootfs(NULL)); action_ready(singularity_runtime_rootfs(NULL)); singularity_runtime_overlayfs(); singularity_runtime_mounts(); singularity_runtime_files(); singularity_runtime_enter(); singularity_runtime_environment(); singularity_priv_drop_perm(); if ((rc = setup_container_cwd()) < 0) { singularity_message(ERROR, "Could not obtain current directory.\n"); return rc; } envar_set("SINGULARITY_CONTAINER", singularity_image_name(&image), 1); // Legacy PS1 support envar_set("SINGULARITY_NAME", singularity_image_name(&image), 1); envar_set("SINGULARITY_SHELL", singularity_registry_get("SHELL"), 1); command = singularity_registry_get("COMMAND"); singularity_message(LOG, "USER=%s, IMAGE='%s', COMMAND='%s'\n", singularity_priv_getuser(), singularity_image_name(&image), singularity_registry_get("COMMAND")); // At this point, the current process is in the runtime container environment. // Return control flow back to SLURM: when execv is invoked, it'll be done from // within the container. return 0; } // TODO: When run on the submit host, we should evaluate the URL and create/cache docker images as necessary. static int determine_image(int val, const char *optarg, int remote) { if (val) {} // Suppresses unused error... // TODO: could do some basic path validation here in order to prevent an ABORT() later. job_image = strdup(optarg); return job_image == NULL; } static int determine_bind(int val, const char *optarg, int remote) { if (!job_bindpath) { job_bindpath = strdup(optarg); } return job_bindpath == NULL; } /// SPANK plugin functions. int slurm_spank_init(spank_t spank, int ac, char **av) { int i; struct spank_option image_opt, bind_opt; memset(&image_opt, '\0', sizeof(image_opt)); image_opt.name = "singularity-image"; image_opt.arginfo = "[path]"; image_opt.usage = "Specify a path to a Singularity image, directory tree, " "or Docker image"; image_opt.has_arg = 1; image_opt.val = 0; image_opt.cb = determine_image; if (ESPANK_SUCCESS != spank_option_register(spank, &image_opt)) { slurm_error("spank/%s: Unable to register a new option.", plugin_name); return -1; } memset(&bind_opt, '\0', sizeof(bind_opt)); bind_opt.name = "singularity-bind"; bind_opt.arginfo = "[path || src:dest],..."; bind_opt.usage = "Specify a user-bind path specification. Can either be " "a path or a src:dest pair, specifying the bind mount to " "perform"; bind_opt.has_arg = 1; bind_opt.val = 0; bind_opt.cb = determine_bind; if (ESPANK_SUCCESS != spank_option_register(spank, &bind_opt)) { slurm_error("spank/%s: Unable to register a new option.", plugin_name); return -1; } // Make this a no-op except when starting the task. if (spank_context() == S_CTX_ALLOCATOR || (spank_remote(spank) != 1)) { return 0; } for (i = 0; i < ac; i++) { if (strncmp ("default_image=", av[i], 14) == 0) { const char *optarg = av[i] + 14; job_image = strdup(optarg); } else { slurm_error ("spank/%s: Invalid option: %s", av[i], plugin_name); } } return 0; } int slurm_spank_task_init_privileged(spank_t spank, int ac, char *argv[]) { if (job_image) { return setup_container(spank); } return 0; } singularity-2.4.2/src/slurm/Makefile.am0000644000175000017500000000230113211621077017010 0ustar mehdimehdiAM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) plugindir = $(libdir)/slurm if WITH_SLURM plugin_LTLIBRARIES = singularity_spank.la singularity_spank_la_SOURCES = singularity.c ../util/sessiondir.c ../action-lib/ready.c ../util/cleanupd.c singularity_spank_la_LIBADD = ../lib/image/libsingularity-image.la ../lib/runtime/libsingularity-runtime.la singularity_spank_la_LDFLAGS = -module -no-undefined -avoid-version -export-symbols-regex '^slurm_spank_|^plugin_' endif install-data-hook: cleanup_plugin cleanup_plugin: @if test -e "$(DESTDIR)$(plugindir)/singularity_spank.so"; then \ echo "cp $(DESTDIR)$(plugindir)/singularity_spank.so $(DESTDIR)$(plugindir)/backup_singularity_spank"; \ cp "$(DESTDIR)$(plugindir)/singularity_spank.so" "$(DESTDIR)$(plugindir)/backup_singularity_spank"; \ echo "rm -f $(DESTDIR)$(plugindir)/singularity_spank.*"; \ rm -f "$(DESTDIR)$(plugindir)/"singularity_spank.*; \ echo "mv $(DESTDIR)$(plugindir)/backup_singularity_spank $(DESTDIR)$(plugindir)/singularity_spank.so"; \ mv "$(DESTDIR)$(plugindir)/backup_singularity_spank" "$(DESTDIR)$(plugindir)/singularity.so"; \ fi singularity-2.4.2/src/mount.c0000644000175000017500000000661613211621077015135 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif int main(int argc, char **argv) { struct image_object image; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_priv_init(); singularity_suid_init(argv); singularity_registry_init(); singularity_priv_drop(); singularity_runtime_autofs(); if ( singularity_registry_get("WRITABLE") != NULL ) { singularity_message(VERBOSE3, "Instantiating writable container image object\n"); image = singularity_image_init(singularity_registry_get("IMAGE"), O_RDWR); } else { singularity_message(VERBOSE3, "Instantiating read only container image object\n"); image = singularity_image_init(singularity_registry_get("IMAGE"), O_RDONLY); } if ( is_owner(CONTAINER_MOUNTDIR, 0) != 0 ) { singularity_message(ERROR, "Root must own container mount directory: %s\n", CONTAINER_MOUNTDIR); ABORT(255); } singularity_runtime_ns(SR_NS_MNT); singularity_image_mount(&image, CONTAINER_MOUNTDIR); singularity_runtime_overlayfs(); singularity_priv_drop_perm(); envar_set("SINGULARITY_MOUNTPOINT", CONTAINER_FINALDIR, 1); if ( argc > 1 ) { singularity_message(VERBOSE, "Running command: %s\n", argv[1]); singularity_message(DEBUG, "Calling exec...\n"); execvp(argv[1], &argv[1]); // Flawfinder: ignore (Yes flawfinder, we are exec'ing) singularity_message(ERROR, "Exec failed: %s: %s\n", argv[1], strerror(errno)); ABORT(255); } else { singularity_message(INFO, "%s is mounted at: %s\n\n", singularity_image_name(&image), CONTAINER_FINALDIR); envar_set("PS1", "Singularity> ", 1); execl("/bin/sh", "/bin/sh", NULL); // Flawfinder: ignore (Yes flawfinder, this is what we want, sheesh, so demanding!) singularity_message(ERROR, "Exec of /bin/sh failed: %s\n", strerror(errno)); ABORT(255); } return(0); } singularity-2.4.2/src/create.c0000644000175000017500000000661713211621077015237 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "util/fork.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif int main(int argc, char **argv) { struct image_object image; long int size = 768; char *size_s; char *mkfs_cmd[4]; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_priv_init(); singularity_suid_init(argv); singularity_registry_init(); singularity_priv_drop(); if ( ( size_s = singularity_registry_get("IMAGESIZE") ) != NULL ) { if ( str2int(size_s, &size) == 0 ) { singularity_message(VERBOSE, "Converted size string to long int: %ld\n", size); } else { singularity_message(ERROR, "Could not convert container size to integer\n"); ABORT(255); } } singularity_message(INFO, "Initializing Singularity image subsystem\n"); image = singularity_image_init(singularity_registry_get("IMAGE")); singularity_message(INFO, "Opening image file: %s\n", image.name); // singularity_image_open(&image, O_CREAT | O_RDWR); singularity_message(INFO, "Creating %ldMiB image\n", size); singularity_image_create(&image, size); singularity_message(INFO, "Binding image to loop\n"); singularity_image_bind(&image); if ( singularity_image_loopdev(&image) == NULL ) { singularity_message(ERROR, "Image was not bound correctly.\n"); ABORT(255); } mkfs_cmd[0] = strdup("/sbin/mkfs.ext3"); mkfs_cmd[1] = strdup("-q"); mkfs_cmd[2] = strdup(singularity_image_loopdev(&image)); mkfs_cmd[3] = NULL; singularity_message(DEBUG, "Cleaning environment\n"); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing the environment\n"); ABORT(255); } singularity_priv_escalate(); singularity_message(INFO, "Creating file system within image\n"); singularity_fork_exec(0, mkfs_cmd); singularity_priv_drop(); singularity_message(INFO, "Image is done: %s\n", image.path); return(0); } singularity-2.4.2/src/lib/0000755000175000017500000000000013211621077014364 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/0000755000175000017500000000000013211621077016047 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/environment/0000755000175000017500000000000013211621077020413 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/environment/Makefile.am0000644000175000017500000000100613211621077022444 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie libns_pid_a_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = environment.c EXTRA_DIST = environment.h singularity-2.4.2/src/lib/runtime/environment/environment.h0000644000175000017500000000235513211621077023135 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_ENVIRONMENT_H_ #define __SINGULARITY_RUNTIME_ENVIRONMENT_H_ extern int _singularity_runtime_environment(void); #endif /* __SINGULARITY_RUNTIME_ENVIRONMENT_H */ singularity-2.4.2/src/lib/runtime/environment/environment.c0000644000175000017500000000662013211621077023127 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/registry.h" int _singularity_runtime_environment(void) { char **current_env = environ; char **envclone; int envlen = 0; int i; // Copy and cache environment singularity_message(DEBUG, "Cloning environment\n"); for(envlen = 0; current_env[envlen] != 0; envlen++) { } singularity_message(DEBUG, "Counted %d environment elements\n", envlen); envclone = (char**) malloc(envlen * sizeof(char *)); for(i = 0; i < envlen; i++) { envclone[i] = strdup(current_env[i]); } // Clean environment if ( singularity_registry_get("CLEANENV") != NULL ) { char *term = envar_get("TERM", "_-.", 128); char *home = envar_path("HOME"); singularity_message(DEBUG, "Sanitizing environment\n"); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing environment\n"); ABORT(255); } envar_set("LANG", "C", 1); envar_set("TERM", term, 1); envar_set("HOME", home, 1); } else { singularity_message(DEBUG, "Cleaning environment\n"); for(i = 0; i < envlen; i++) { singularity_message(DEBUG, "Evaluating envar to clean: %s\n", envclone[i]); if ( strncmp(envclone[i], "SINGULARITY_", 12) == 0 ) { char *key, *tok; key = strtok_r(envclone[i], "=", &tok); singularity_message(DEBUG, "Unsetting environment variable: %s\n", key); unsetenv(key); } } } // Transpose environment singularity_message(DEBUG, "Transposing environment\n"); for(i = 0; i < envlen; i++) { if ( strncmp(envclone[i], "SINGULARITYENV_", 15) == 0 ) { char *tok, *key, *val; key = strtok_r(envclone[i], "=", &tok); val = strtok_r(NULL, "\n", &tok); singularity_message(DEBUG, "Converting envar '%s' to '%s' = '%s'\n", key, &key[15], val); envar_set(&key[15], val, 1); unsetenv(key); } } for(i = 0; i < envlen; i++) { free(envclone[i]); } return(0); } singularity-2.4.2/src/lib/runtime/runtime.h0000644000175000017500000000353313211621077017707 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_H_ #define __SINGULARITY_RUNTIME_H_ // The Following functions actually do work: // Unshare namespaces extern int singularity_runtime_ns(unsigned int flags); #define SR_NS_PID 1 #define SR_NS_IPC 2 #define SR_NS_MNT 4 #define SR_NS_NET 8 #define SR_NS_ALL 255 // Setup/initialize the overlayFS extern int singularity_runtime_overlayfs(void); // Setup mount points within container extern int singularity_runtime_mounts(void); // Setup files within the container extern int singularity_runtime_files(void); // Enter container root extern int singularity_runtime_enter(void); // Clean, santize, update environment extern int singularity_runtime_environment(void); // Setup for buggy autofs path extern int singularity_runtime_autofs(void); #endif /* __SINGULARITY_RUNTIME_H */ singularity-2.4.2/src/lib/runtime/enter/0000755000175000017500000000000013211621077017164 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/enter/enter.h0000644000175000017500000000232513211621077020454 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_ENTER_H_ #define __SINGULARITY_RUNTIME_ENTER_H_ extern int _singularity_runtime_enter(void); #endif /* __SINGULARITY_RUNTIME_ENTER_H */ singularity-2.4.2/src/lib/runtime/enter/enter.c0000644000175000017500000000301213211621077020441 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "./chroot/chroot.h" int _singularity_runtime_enter(void) { int retval = 0; singularity_message(VERBOSE, "Containing all rootfs components\n"); retval += _singularity_runtime_enter_chroot(); return(retval); } singularity-2.4.2/src/lib/runtime/enter/chroot/0000755000175000017500000000000013211621077020462 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/enter/chroot/chroot.h0000644000175000017500000000236113211621077022133 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_ENTER_CHROOT_H_ #define __SINGULARITY_RUNTIME_ENTER_CHROOT_H_ extern int _singularity_runtime_enter_chroot(void); #endif /* __SINGULARITY_RUNTIME_ENTER_CHROOT_H */ singularity-2.4.2/src/lib/runtime/enter/chroot/Makefile.am0000644000175000017500000000055313211621077022521 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = chroot.c EXTRA_DIST = chroot.h singularity-2.4.2/src/lib/runtime/enter/chroot/chroot.c0000644000175000017500000000413013211621077022122 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "../../runtime.h" int _singularity_runtime_enter_chroot(void) { char *container_dir = CONTAINER_FINALDIR; singularity_priv_escalate(); singularity_message(VERBOSE, "Entering container file system root: %s\n", container_dir); if ( chroot(container_dir) < 0 ) { // Flawfinder: ignore (yep, yep, yep... we know!) singularity_message(ERROR, "failed chroot to container at: %s\n", container_dir); ABORT(255); } singularity_priv_drop(); singularity_message(DEBUG, "Changing dir to '/' within the new root\n"); if ( chdir("/") < 0 ) { singularity_message(ERROR, "Could not chdir after chroot to /: %s\n", strerror(errno)); ABORT(1); } return(0); } singularity-2.4.2/src/lib/runtime/enter/Makefile.am0000644000175000017500000000107213211621077021220 0ustar mehdimehdiSUBDIRS = chroot MAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie libns_pid_a_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_LIBADD = chroot/libinternal.la libinternal_la_SOURCES = enter.c EXTRA_DIST = enter.h singularity-2.4.2/src/lib/runtime/files/0000755000175000017500000000000013211621077017151 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/files/file-bind.h0000644000175000017500000000232613211621077021156 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_FILE_BIND_H_ #define __SINGULARITY_FILE_BIND_H_ int container_file_bind(char *file, char *dest_path); #endif /* __SINGULARITY_FILE_BIND_H */ singularity-2.4.2/src/lib/runtime/files/file-bind.c0000644000175000017500000000552613211621077021156 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/mount.h" #include "../runtime.h" int container_file_bind(char *source, char *dest_path) { char *dest; char *containerdir = CONTAINER_FINALDIR; singularity_message(DEBUG, "Called file_bind(%s, %s()\n", source, dest_path); if ( containerdir == NULL ) { singularity_message(ERROR, "Failed to obtain container directory\n"); ABORT(255); } dest = joinpath(containerdir, dest_path); if ( is_file(source) < 0 ) { singularity_message(WARNING, "Bind file source does not exist on host: %s\n", source); return(1); } if ( is_file(dest) < 0 ) { singularity_message(VERBOSE, "Skipping bind file, destination does not exist in container: %s\n", dest_path); return(0); } singularity_priv_escalate(); singularity_message(VERBOSE, "Binding file '%s' to '%s'\n", source, dest); if ( singularity_mount(source, dest, NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL) < 0 ) { singularity_priv_drop(); singularity_message(ERROR, "There was an error binding %s to %s: %s\n", source, dest, strerror(errno)); ABORT(255); } if ( singularity_priv_userns_enabled() != 1 ) { if ( singularity_mount(NULL, dest, NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC|MS_REMOUNT, NULL) < 0 ) { singularity_priv_drop(); singularity_message(ERROR, "There was an error remounting %s to %s: %s\n", source, dest, strerror(errno)); ABORT(255); } } singularity_priv_drop(); return(0); } singularity-2.4.2/src/lib/runtime/files/libs/0000755000175000017500000000000013211621077020102 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/files/libs/libs.c0000644000175000017500000001607113211621077021204 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef _GNU_SOURCE #define _GNU_SOURCE #endif #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../file-bind.h" #include "../../runtime.h" int _singularity_runtime_files_libs(void) { char *container_dir = CONTAINER_FINALDIR; char *tmpdir = singularity_registry_get("SESSIONDIR"); char *includelibs_string; char *libdir = joinpath(tmpdir, "/libs"); char *libdir_contained = joinpath(container_dir, "/.singularity.d/libs"); if ( ( includelibs_string = singularity_registry_get("CONTAINLIBS") ) != NULL ) { char *tok = NULL; char *current = strtok_r(strdup(includelibs_string), ",", &tok); #ifndef SINGULARITY_NO_NEW_PRIVS singularity_message(WARNING, "Not mounting libs: host does not support PR_SET_NO_NEW_PRIVS\n"); return(0); #endif singularity_message(DEBUG, "Parsing SINGULARITY_CONTAINLIBS for user-specified libraries to include.\n"); free(includelibs_string); singularity_message(DEBUG, "Checking if libdir in container exists: %s\n", libdir_contained); if ( is_dir(libdir_contained) != 0 ) { singularity_message(WARNING, "Library bind directory not present in container, update container\n"); } singularity_message(DEBUG, "Creating session libdir at: %s\n", libdir); if ( s_mkpath(libdir, 0755) != 0 ) { singularity_message(ERROR, "Failed creating temp lib directory at: %s\n", libdir); ABORT(255); } while (current != NULL ) { char *dest = NULL; char *source = NULL; singularity_message(DEBUG, "Evaluating requested library path: %s\n", current); dest = joinpath(libdir, basename(current)); if ( is_file(dest) == 0 ) { singularity_message(VERBOSE3, "Staged library exists, skipping: %s\n", current); current = strtok_r(NULL, ",", &tok); continue; } if ( is_link(current) == 0 ) { char *link_name; ssize_t len; link_name = (char *) malloc(PATH_MAX); len = readlink(current, link_name, PATH_MAX-1); // Flawfinder: ignore if ( ( len > 0 ) && ( len <= PATH_MAX) ) { link_name[len] = '\0'; singularity_message(VERBOSE3, "Found library link source: %s -> %s\n", current, link_name); if ( link_name[0] == '/' ) { source = strdup(link_name); } else { if ( link_name[0] == '/' ) { source = strdup(link_name); } else { source = joinpath(dirname(strdup(current)), link_name); } } } else { singularity_message(WARNING, "Failed reading library link for %s: %s\n", current, strerror(errno)); ABORT(255); } free(link_name); } else if (is_file(current) == 0 ) { source = strdup(current); singularity_message(VERBOSE3, "Found library source: %s\n", source); } else { singularity_message(WARNING, "Could not find library: %s\n", current); current = strtok_r(NULL, ",", &tok); continue; } singularity_message(DEBUG, "Binding library source here: %s -> %s\n", source, dest); if ( fileput(dest, "") != 0 ) { singularity_message(ERROR, "Failed creating file at %s: %s\n", dest, strerror(errno)); ABORT(255); } singularity_priv_escalate(); singularity_message(VERBOSE, "Binding file '%s' to '%s'\n", source, dest); if ( singularity_mount(source, dest, NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL) < 0 ) { singularity_priv_drop(); singularity_message(ERROR, "There was an error binding %s to %s: %s\n", source, dest, strerror(errno)); ABORT(255); } singularity_priv_drop(); free(source); free(dest); current = strtok_r(NULL, ",", &tok); } if ( is_dir(libdir_contained) != 0 ) { char *ld_path; singularity_message(DEBUG, "Attempting to create contained libdir\n"); singularity_priv_escalate(); if ( s_mkpath(libdir_contained, 0755) != 0 ) { singularity_message(ERROR, "Failed creating directory %s :%s\n", libdir_contained, strerror(errno)); ABORT(255); } singularity_priv_drop(); ld_path = envar_path("LD_LIBRARY_PATH"); if ( ld_path == NULL ) { singularity_message(DEBUG, "Setting LD_LIBRARY_PATH to '/.singularity.d/libs'\n"); envar_set("LD_LIBRARY_PATH", "/.singularity.d/libs", 1); } else { singularity_message(DEBUG, "Prepending '/.singularity.d/libs' to LD_LIBRARY_PATH\n"); envar_set("LD_LIBRARY_PATH", strjoin("/.singularity.d/libs:", ld_path), 1); } } singularity_priv_escalate(); singularity_message(VERBOSE, "Binding libdir '%s' to '%s'\n", libdir, libdir_contained); if ( singularity_mount(libdir, libdir_contained, NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL) < 0 ) { singularity_priv_drop(); singularity_message(ERROR, "There was an error binding %s to %s: %s\n", libdir, libdir_contained, strerror(errno)); ABORT(255); } singularity_priv_drop(); } return(0); } singularity-2.4.2/src/lib/runtime/files/libs/Makefile.am0000644000175000017500000000054713211621077022144 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = libs.c EXTRA_DIST = libs.h singularity-2.4.2/src/lib/runtime/files/libs/libs.h0000644000175000017500000000235113211621077021205 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_FILES_LIBS_H_ #define __SINGULARITY_RUNTIME_FILES_LIBS_H_ extern int _singularity_runtime_files_libs(void); #endif /* __SINGULARITY_RUNTIME_FILES_LIBS_H */ singularity-2.4.2/src/lib/runtime/files/files.h0000644000175000017500000000232513211621077020426 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_FILES_H_ #define __SINGULARITY_RUNTIME_FILES_H_ extern int _singularity_runtime_files(void); #endif /* __SINGULARITY_RUNTIME_FILES_H */ singularity-2.4.2/src/lib/runtime/files/resolvconf/0000755000175000017500000000000013211621077021331 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/files/resolvconf/resolvconf.h0000644000175000017500000000240113211621077023657 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_FILES_RESOLVCONF_H_ #define __SINGULARITY_RUNTIME_FILES_RESOLVCONF_H_ extern int _singularity_runtime_files_resolvconf(void); #endif /* __SINGULARITY_RUNTIME_FILES_RESOLVCONF_H */ singularity-2.4.2/src/lib/runtime/files/resolvconf/Makefile.am0000644000175000017500000000056313211621077023371 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = resolvconf.c EXTRA_DIST = resolvconf.h singularity-2.4.2/src/lib/runtime/files/resolvconf/resolvconf.c0000644000175000017500000000347713211621077023670 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/config_parser.h" #include "util/message.h" #include "util/privilege.h" #include "../file-bind.h" #include "../../runtime.h" int _singularity_runtime_files_resolvconf(void) { char *file = "/etc/resolv.conf"; singularity_message(DEBUG, "Checking configuration option\n"); if ( singularity_config_get_bool(CONFIG_RESOLV_CONF) <= 0 ) { singularity_message(VERBOSE, "Skipping bind of the host's %s\n", file); return(0); } container_file_bind(file, file); return(0); } singularity-2.4.2/src/lib/runtime/files/group/0000755000175000017500000000000013211621077020305 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/files/group/group.c0000644000175000017500000001371513211621077021614 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/config_parser.h" #include "util/message.h" #include "util/privilege.h" #include "util/registry.h" #include "../file-bind.h" #include "../../runtime.h" int _singularity_runtime_files_group(void) { FILE *file_fp; char *source_file; char *tmp_file; int i; uid_t uid = singularity_priv_getuid(); uid_t gid = singularity_priv_getgid(); const gid_t *gids = singularity_priv_getgids(); int gid_count = singularity_priv_getgidcount(); char *containerdir = CONTAINER_FINALDIR; char *tmpdir = singularity_registry_get("SESSIONDIR"); singularity_message(DEBUG, "Called singularity_file_group_create()\n"); if ( uid == 0 ) { singularity_message(VERBOSE, "Not updating group file, running as root!\n"); return(0); } if ( containerdir == NULL ) { singularity_message(ERROR, "Failed to obtain container directory\n"); ABORT(255); } if ( tmpdir == NULL ) { singularity_message(ERROR, "Failed to obtain session directory\n"); ABORT(255); } singularity_message(DEBUG, "Checking configuration option: 'config group'\n"); if ( singularity_config_get_bool(CONFIG_GROUP) <= 0 ) { singularity_message(VERBOSE, "Skipping bind of the host's /etc/group\n"); return(0); } source_file = joinpath(containerdir, "/etc/group"); tmp_file = joinpath(tmpdir, "/group"); if ( is_file(source_file) < 0 ) { singularity_message(VERBOSE, "Group file does not exist in container, not updating\n"); return(0); } errno = 0; struct passwd *pwent = getpwuid(uid); if ( ! pwent ) { // List of potential error codes for unknown name taken from man page. if ( (errno == 0) || (errno == ESRCH) || (errno == EBADF) || (errno == EPERM) || (errno == ENOENT)) { singularity_message(VERBOSE3, "Not updating group file as passwd entry for UID %d not found.\n", uid); return(0); } else { singularity_message(ERROR, "Failed to lookup username for UID %d: %s\n", uid, strerror(errno)); ABORT(255); } } singularity_message(VERBOSE2, "Creating template of /etc/group for containment\n"); if ( ( copy_file(source_file, tmp_file) ) < 0 ) { singularity_message(ERROR, "Failed copying template group file to tmpdir: %s\n", strerror(errno)); ABORT(255); } if ( ( file_fp = fopen(tmp_file, "a") ) == NULL ) { // Flawfinder: ignore singularity_message(ERROR, "Could not open template group file %s: %s\n", tmp_file, strerror(errno)); ABORT(255); } errno = 0; struct group *grent = getgrgid(gid); if ( grent ) { singularity_message(VERBOSE, "Updating group file with user info\n"); fprintf(file_fp, "\n%s:x:%u:%s\n", grent->gr_name, grent->gr_gid, pwent->pw_name); } else if ( (errno == 0) || (errno == ESRCH) || (errno == EBADF) || (errno == EPERM) || (errno == ENOENT) ) { // It's rare, but certainly possible to have a GID that's not a group entry in this system. // According to the man page, all of the above errno's can indicate this situation. singularity_message(VERBOSE3, "Skipping GID %d as group entry does not exist.\n", gid); } else { singularity_message(ERROR, "Failed to lookup GID %d group entry: %s\n", gid, strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Getting supplementary group info\n"); for (i=0; i < gid_count; i++) { if ( gids[i] == gid ) { singularity_message(DEBUG, "Skipping duplicate supplementary group\n"); continue; } if ( gids[i] < UINT_MAX ) { errno = 0; struct group *gr = getgrgid(gids[i]); if ( gr ) { singularity_message(VERBOSE3, "Found supplementary group membership in: %d\n", gids[i]); singularity_message(VERBOSE2, "Adding user's supplementary group ('%s') info to template group file\n", gr->gr_name); fprintf(file_fp, "%s:x:%u:%s\n", gr->gr_name, gr->gr_gid, pwent->pw_name); } else if ( (errno == 0) || (errno == ESRCH) || (errno == EBADF) || (errno == EPERM) ) { singularity_message(VERBOSE3, "Skipping GID %d as group entry does not exist.\n", gids[i]); } else { singularity_message(ERROR, "Failed to lookup GID %d group entry: %s\n", gids[i], strerror(errno)); ABORT(255); } } else { singularity_message(VERBOSE, "Group id '%d' is out of bounds\n", gids[i]); } } fclose(file_fp); container_file_bind(tmp_file, "/etc/group"); return(0); } singularity-2.4.2/src/lib/runtime/files/group/Makefile.am0000644000175000017500000000055113211621077022342 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = group.c EXTRA_DIST = group.h singularity-2.4.2/src/lib/runtime/files/group/group.h0000644000175000017500000000235513211621077021617 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_FILES_GROUP_H_ #define __SINGULARITY_RUNTIME_FILES_GROUP_H_ extern int _singularity_runtime_files_group(void); #endif /* __SINGULARITY_RUNTIME_FILES_GROUP_H */ singularity-2.4.2/src/lib/runtime/files/files.c0000644000175000017500000000336413211621077020425 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "./passwd/passwd.h" #include "./group/group.h" #include "./resolvconf/resolvconf.h" #include "./libs/libs.h" int _singularity_runtime_files(void) { int retval = 0; singularity_message(VERBOSE, "Running file components\n"); retval += _singularity_runtime_files_passwd(); retval += _singularity_runtime_files_group(); retval += _singularity_runtime_files_resolvconf(); retval += _singularity_runtime_files_libs(); return(retval); } singularity-2.4.2/src/lib/runtime/files/Makefile.am0000644000175000017500000000125313211621077021206 0ustar mehdimehdiSUBDIRS = passwd group resolvconf libs MAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie libns_pid_a_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_LIBADD = passwd/libinternal.la group/libinternal.la resolvconf/libinternal.la libs/libinternal.la libinternal_la_SOURCES = files.c file-bind.c EXTRA_DIST = file-bind.h files.h singularity-2.4.2/src/lib/runtime/files/passwd/0000755000175000017500000000000013211621077020452 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/files/passwd/passwd.c0000644000175000017500000000744413211621077022130 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/config_parser.h" #include "util/message.h" #include "util/privilege.h" #include "util/registry.h" #include "../file-bind.h" #include "../../runtime.h" int _singularity_runtime_files_passwd(void) { FILE *file_fp; char *source_file; char *tmp_file; char *homedir = singularity_priv_home(); uid_t uid = singularity_priv_getuid(); struct passwd *pwent = getpwuid(uid); char *containerdir = CONTAINER_FINALDIR; char *tmpdir = singularity_registry_get("SESSIONDIR"); singularity_message(DEBUG, "Called singularity_file_passwd_create()\n"); if ( uid == 0 ) { singularity_message(VERBOSE, "Not updating passwd file, running as root!\n"); return(0); } if ( containerdir == NULL ) { singularity_message(ERROR, "Failed to obtain container directory\n"); ABORT(255); } if ( tmpdir == NULL ) { singularity_message(ERROR, "Failed to obtain session directory\n"); ABORT(255); } singularity_message(DEBUG, "Checking configuration option: 'config passwd'\n"); if ( singularity_config_get_bool(CONFIG_PASSWD) <= 0 ) { singularity_message(VERBOSE, "Skipping bind of the host's /etc/passwd\n"); return(0); } source_file = joinpath(containerdir, "/etc/passwd"); tmp_file = joinpath(tmpdir, "/passwd"); singularity_message(VERBOSE2, "Checking for template passwd file: %s\n", source_file); if ( is_file(source_file) < 0 ) { singularity_message(VERBOSE, "Passwd file does not exist in container, not updating\n"); return(0); } singularity_message(VERBOSE2, "Creating template of /etc/passwd\n"); if ( ( copy_file(source_file, tmp_file) ) < 0 ) { singularity_message(ERROR, "Failed copying template passwd file to tmpdir: %s\n", strerror(errno)); ABORT(255); } singularity_message(VERBOSE, "Creating template passwd file and appending user data: %s\n", tmp_file); if ( ( file_fp = fopen(tmp_file, "a") ) == NULL ) { // Flawfinder: ignore singularity_message(ERROR, "Could not open template passwd file %s: %s\n", tmp_file, strerror(errno)); ABORT(255); } fprintf(file_fp, "%s:x:%d:%d:%s:%s:%s\n", pwent->pw_name, pwent->pw_uid, pwent->pw_gid, pwent->pw_gecos, homedir, pwent->pw_shell); fclose(file_fp); container_file_bind(tmp_file, "/etc/passwd"); // set HOME to the homedir, because it might be different than outside envar_set("HOME", homedir, 1); return(0); } singularity-2.4.2/src/lib/runtime/files/passwd/passwd.h0000644000175000017500000000236113211621077022126 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_FILES_PASSWD_H_ #define __SINGULARITY_RUNTIME_FILES_PASSWD_H_ extern int _singularity_runtime_files_passwd(void); #endif /* __SINGULARITY_RUNTIME_FILES_PASSWD_H */ singularity-2.4.2/src/lib/runtime/files/passwd/Makefile.am0000644000175000017500000000055313211621077022511 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = passwd.c EXTRA_DIST = passwd.h singularity-2.4.2/src/lib/runtime/autofs/0000755000175000017500000000000013211621077017350 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/autofs/autofs.h0000644000175000017500000000233113211621077021021 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_AUTOFS_H_ #define __SINGULARITY_RUNTIME_AUTOFS_H_ extern int _singularity_runtime_autofs(void); #endif /* __SINGULARITY_RUNTIME_AUTOFS_H */ singularity-2.4.2/src/lib/runtime/autofs/Makefile.am0000644000175000017500000000077413211621077021414 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie libns_pid_a_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = autofs.c EXTRA_DIST = autofs.h singularity-2.4.2/src/lib/runtime/autofs/autofs.c0000644000175000017500000000503013211621077021013 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/config_parser.h" #include "util/registry.h" int _singularity_runtime_autofs(void) { char *source; int autofs_fd; const char **tmp_config_string_list = singularity_config_get_value_multi(AUTOFS_BUG_PATH); if ( strlength(*tmp_config_string_list, 1) == 0 ) { singularity_message(VERBOSE, "No autofs bug path in configuration, skipping\n"); return(0); } singularity_message(VERBOSE, "Autofs bug path requested\n"); while ( *tmp_config_string_list != NULL ) { source = strdup(*tmp_config_string_list); tmp_config_string_list++; chomp(source); singularity_message(VERBOSE2, "Autofs bug fix for directory %s\n", source); if ( is_dir(source) < 0 ) { singularity_message(WARNING, "Autofs bug path %s is not a directory\n", source); continue; } autofs_fd = open(source, O_RDONLY); if ( autofs_fd < 0 ) { singularity_message(WARNING, "Failed to open directory '%s'\n", source); continue; } if ( fcntl(autofs_fd, F_SETFD, FD_CLOEXEC) != 0 ) { singularity_message(WARNING, "Failed to set FD_CLOEXEC on directory '%s'\n", source); continue; } } return(0); } singularity-2.4.2/src/lib/runtime/runtime.c0000644000175000017500000000603113211621077017676 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "./ns/ns.h" #include "./mounts/mounts.h" #include "./files/files.h" #include "./enter/enter.h" #include "./overlayfs/overlayfs.h" #include "./environment/environment.h" #include "./autofs/autofs.h" #ifndef LOCALSTATEDIR #error LOCALSTATEDIR not defined #endif int singularity_runtime_ns(unsigned int flags) { /* If a daemon already exists, join existing namespaces instead of creating */ if ( singularity_registry_get("DAEMON_JOIN") ) { return(_singularity_runtime_ns_join(flags)); } return(_singularity_runtime_ns(flags)); } int singularity_runtime_overlayfs(void) { if ( singularity_registry_get("DAEMON_JOIN") ) { singularity_message(ERROR, "Internal Error - This function should not be called when joining an instance\n"); } return(_singularity_runtime_overlayfs()); } int singularity_runtime_environment(void) { return(_singularity_runtime_environment()); } int singularity_runtime_mounts(void) { if ( singularity_registry_get("DAEMON_JOIN") ) { singularity_message(ERROR, "Internal Error - This function should not be called when joining an instance\n"); } return(_singularity_runtime_mounts()); } int singularity_runtime_files(void) { if ( singularity_registry_get("DAEMON_JOIN") ) { singularity_message(ERROR, "Internal Error - This function should not be called when joining an instance\n"); } return(_singularity_runtime_files()); } int singularity_runtime_enter(void) { return(_singularity_runtime_enter()); } int singularity_runtime_autofs(void) { return(_singularity_runtime_autofs()); } singularity-2.4.2/src/lib/runtime/ns/0000755000175000017500000000000013211621077016467 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/ns/mnt/0000755000175000017500000000000013211621077017265 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/ns/mnt/Makefile.am0000644000175000017500000000054513211621077021325 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = mnt.c EXTRA_DIST = mnt.h singularity-2.4.2/src/lib/runtime/ns/mnt/mnt.c0000644000175000017500000001011613211621077020226 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "util/message.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/setns.h" #include "util/mount.h" static int enabled = -1; int _singularity_runtime_ns_mnt(void) { int slave; slave = singularity_config_get_bool(MOUNT_SLAVE); singularity_priv_escalate(); #ifdef NS_CLONE_FS singularity_message(DEBUG, "Virtualizing FS namespace\n"); if ( unshare(CLONE_FS) < 0 ) { singularity_message(ERROR, "Could not virtualize file system namespace: %s\n", strerror(errno)); ABORT(255); } #endif singularity_message(DEBUG, "Virtualizing mount namespace\n"); if ( unshare(CLONE_NEWNS) < 0 ) { singularity_message(ERROR, "Could not virtualize mount namespace: %s\n", strerror(errno)); ABORT(255); } // Privatize the mount namespaces // #ifdef SINGULARITY_MS_SLAVE singularity_message(DEBUG, "Making mounts %s\n", (slave ? "slave" : "private")); // The strange formatting here is to avoid SonarQube complaints about bitwise or of signed operands. unsigned mount_flags = MS_REC; mount_flags |= (unsigned)(slave ? MS_SLAVE : MS_PRIVATE); if ( singularity_mount(NULL, "/", NULL, mount_flags, NULL) < 0 ) { singularity_message(ERROR, "Could not make mountspaces %s: %s\n", (slave ? "slave" : "private"), strerror(errno)); ABORT(255); } #else if ( slave > 0 ) { singularity_message(WARNING, "Requested option 'mount slave' is not available on this host, using private\n"); } singularity_message(DEBUG, "Making mounts private\n"); if ( singularity_mount(NULL, "/", NULL, MS_PRIVATE | MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Could not make mountspaces %s: %s\n", (slave ? "slave" : "private"), strerror(errno)); ABORT(255); } #endif singularity_priv_drop(); enabled = 0; return(0); } int _singularity_runtime_ns_mnt_join(void) { int ns_fd = atoi(singularity_registry_get("DAEMON_NS_FD")); int mnt_fd; /* Attempt to open /proc/[MNT]/ns/mnt */ singularity_priv_escalate(); mnt_fd = openat(ns_fd, "mnt", O_RDONLY); if( mnt_fd == -1 ) { singularity_message(ERROR, "Could not open mount NS fd: %s\n", strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Attempting to join mount namespace\n"); if ( setns(mnt_fd, CLONE_NEWNS) < 0 ) { singularity_message(ERROR, "Could not join mount namespace: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); singularity_message(DEBUG, "Successfully joined mount namespace\n"); close(ns_fd); return(0); } /* int singularity_ns_mnt_enabled(void) { singularity_message(DEBUG, "Checking MNT namespace enabled: %d\n", enabled); return(enabled); } */ singularity-2.4.2/src/lib/runtime/ns/mnt/mnt.h0000644000175000017500000000241413211621077020235 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_NS_MNT_H_ #define __SINGULARITY_RUNTIME_NS_MNT_H_ extern int _singularity_runtime_ns_mnt(void); extern int _singularity_runtime_ns_mnt_join(void); #endif /* __SINGULARITY_RUNTIME_NS_MNT_H */ singularity-2.4.2/src/lib/runtime/ns/ns.c0000644000175000017500000000603513211621077017257 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/config_parser.h" #include "./ipc/ipc.h" #include "./mnt/mnt.h" #include "./pid/pid.h" #include "./net/net.h" #include "../runtime.h" int _singularity_runtime_ns(unsigned int flags) { int retval = 0; if ( flags & SR_NS_IPC ) { singularity_message(DEBUG, "Calling: _singularity_runtime_ns_ipc()\n"); retval += _singularity_runtime_ns_ipc(); } if ( flags & SR_NS_PID ) { singularity_message(DEBUG, "Calling: _singularity_runtime_ns_pid()\n"); retval += _singularity_runtime_ns_pid(); } if ( flags & SR_NS_NET ) { singularity_message(DEBUG, "Calling: _singularity_runtime_ns_net()\n"); retval += _singularity_runtime_ns_net(); } if ( flags & SR_NS_MNT ) { singularity_message(DEBUG, "Calling: _singularity_runtime_ns_mnt()\n"); retval += _singularity_runtime_ns_mnt(); } return(retval); } int _singularity_runtime_ns_join(unsigned int flags) { int retval = 0; if ( flags & SR_NS_IPC ) { singularity_message(DEBUG, "Calling: _singularity_runtime_ns_ipc_join()\n"); retval += _singularity_runtime_ns_ipc_join(); } if ( flags & SR_NS_PID ) { singularity_message(DEBUG, "Calling: _singularity_runtime_ns_pid_join()\n"); retval += _singularity_runtime_ns_pid_join(); } if ( flags & SR_NS_NET ) { singularity_message(DEBUG, "Calling: _singularity_runtime_ns_net_join()\n"); retval += _singularity_runtime_ns_net_join(); } if ( flags & SR_NS_MNT ) { singularity_message(DEBUG, "Calling: _singularity_runtime_ns_mnt_join()\n"); retval += _singularity_runtime_ns_mnt_join(); } return(retval); } singularity-2.4.2/src/lib/runtime/ns/pid/0000755000175000017500000000000013211621077017243 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/ns/pid/pid.h0000644000175000017500000000241413211621077020171 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_NS_PID_H_ #define __SINGULARITY_RUNTIME_NS_PID_H_ extern int _singularity_runtime_ns_pid(void); extern int _singularity_runtime_ns_pid_join(void); #endif /* __SINGULARITY_RUNTIME_NS_PID_H */ singularity-2.4.2/src/lib/runtime/ns/pid/Makefile.am0000644000175000017500000000054513211621077021303 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = pid.c EXTRA_DIST = pid.h singularity-2.4.2/src/lib/runtime/ns/pid/pid.c0000644000175000017500000000701113211621077020162 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/fork.h" #include "util/registry.h" #include "util/setns.h" int _singularity_runtime_ns_pid(void) { #ifdef SINGULARITY_NO_NEW_PRIVS // Use PID namespace when NO_NEW_PRIVS is not supported if ( singularity_config_get_bool(ALLOW_PID_NS) <= 0 ) { singularity_message(VERBOSE2, "Not virtualizing PID namespace by configuration\n"); return(0); } if ( singularity_registry_get("UNSHARE_PID") == NULL ) { singularity_message(VERBOSE2, "Not virtualizing PID namespace on user request\n"); return(0); } #endif /* SINGULARITY_NO_NEW_PRIVS */ #ifdef NS_CLONE_NEWPID singularity_message(DEBUG, "Using PID namespace: CLONE_NEWPID\n"); #else singularity_message(WARNING, "Skipping PID namespace creation, support not available on host\n"); return(0); #endif singularity_message(DEBUG, "Virtualizing PID namespace\n"); if ( singularity_registry_get("DAEMON_START") ) { singularity_fork_daemonize(CLONE_NEWPID); } else { singularity_fork_run(CLONE_NEWPID); } singularity_registry_set("PIDNS_ENABLED", "1"); return(0); } int _singularity_runtime_ns_pid_join(void) { int ns_fd = atoi(singularity_registry_get("DAEMON_NS_FD")); int pid_fd; /* Attempt to open /proc/[PID]/ns/pid */ singularity_priv_escalate(); pid_fd = openat(ns_fd, "pid", O_RDONLY); if( pid_fd == -1 ) { /* Daemons should always have a ns/pid file. If it doesn't exist, something is wrong */ singularity_message(ERROR, "Could not open PID NS fd: %s\n", strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Attempting to join PID namespace\n"); if ( setns(pid_fd, CLONE_NEWPID) < 0 ) { singularity_message(ERROR, "Could not join PID namespace: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); singularity_message(DEBUG, "Successfully joined PID namespace\n"); close(pid_fd); /* Enable PID NS by forking into a child */ singularity_fork_run(0); singularity_registry_set("PIDNS_ENABLED", "1"); return(0); } singularity-2.4.2/src/lib/runtime/ns/ns.h0000644000175000017500000000334013211621077017260 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_NS_H_ #define __SINGULARITY_RUNTIME_NS_H_ extern int _singularity_runtime_ns(unsigned int flags); extern int _singularity_runtime_ns_join(unsigned int flags); #endif /* __SINGULARITY_RUNTIME_NS_H */ /* extern int singularity_ns_unshare(void); extern int singularity_ns_pid_unshare(void); extern int singularity_ns_pid_enabled(void); extern int singularity_ns_mnt_unshare(void); extern int singularity_ns_mnt_enabled(void); extern int singularity_ns_user_unshare(void); extern int singularity_ns_user_enabled(void); extern int singularity_ns_user_configured(void); extern int singularity_ns_join(pid_t attach_pid); */ singularity-2.4.2/src/lib/runtime/ns/ipc/0000755000175000017500000000000013211621077017242 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/ns/ipc/ipc.c0000644000175000017500000000651613211621077020171 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/fork.h" #include "util/registry.h" #include "util/setns.h" static int enabled = -1; int _singularity_runtime_ns_ipc(void) { if ( singularity_config_get_bool(ALLOW_IPC_NS) <= 0 ) { singularity_message(VERBOSE2, "Not virtualizing IPC namespace by configuration\n"); return(0); } if ( singularity_registry_get("UNSHARE_IPC") == NULL ) { singularity_message(VERBOSE2, "Not virtualizing IPC namespace on user request\n"); return(0); } #ifdef NS_CLONE_NEWIPC singularity_message(DEBUG, "Using IPC namespace: CLONE_NEWIPC\n"); singularity_priv_escalate(); singularity_message(DEBUG, "Virtualizing IPC namespace\n"); if ( unshare(CLONE_NEWIPC) < 0 ) { singularity_message(ERROR, "Could not virtualize IPC namespace: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); enabled = 0; #else singularity_message(WARNING, "Skipping IPC namespace creation, support not available on host\n"); return(0); #endif return(0); } int _singularity_runtime_ns_ipc_join(void) { int ns_fd = atoi(singularity_registry_get("DAEMON_NS_FD")); int ipc_fd; /* Attempt to open /proc/[PID]/ns/pid */ singularity_priv_escalate(); ipc_fd = openat(ns_fd, "ipc", O_RDONLY); if( ipc_fd == -1 ) { singularity_message(ERROR, "Could not open IPC NS fd: %s\n", strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Attempting to join IPC namespace\n"); if ( setns(ipc_fd, CLONE_NEWIPC) < 0 ) { singularity_message(ERROR, "Could not join IPC namespace: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); singularity_message(DEBUG, "Successfully joined IPC namespace\n"); close(ipc_fd); return(0); } /* int singularity_ns_ipc_enabled(void) { singularity_message(DEBUG, "Checking IPC namespace enabled: %d\n", enabled); return(enabled); } */ singularity-2.4.2/src/lib/runtime/ns/ipc/Makefile.am0000644000175000017500000000054513211621077021302 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = ipc.c EXTRA_DIST = ipc.h singularity-2.4.2/src/lib/runtime/ns/ipc/ipc.h0000644000175000017500000000241413211621077020167 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_NS_IPC_H_ #define __SINGULARITY_RUNTIME_NS_IPC_H_ extern int _singularity_runtime_ns_ipc(void); extern int _singularity_runtime_ns_ipc_join(void); #endif /* __SINGULARITY_RUNTIME_NS_IPC_H */ singularity-2.4.2/src/lib/runtime/ns/Makefile.am0000644000175000017500000000116413211621077020525 0ustar mehdimehdiSUBDIRS = mnt pid ipc net MAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie libns_pid_a_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_LIBADD = mnt/libinternal.la pid/libinternal.la ipc/libinternal.la net/libinternal.la libinternal_la_SOURCES = ns.c EXTRA_DIST = ns.h singularity-2.4.2/src/lib/runtime/ns/net/0000755000175000017500000000000013211621077017255 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/ns/net/net.c0000644000175000017500000000613213211621077020211 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * This software is licensed under a 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/fork.h" #include "util/registry.h" #include "util/setns.h" static int enabled = -1; int _singularity_runtime_ns_net(void) { int sockfd; struct ifreq req; if ( singularity_registry_get("UNSHARE_NET") == NULL ) { singularity_message(VERBOSE2, "Not virtualizing network namespace on user request\n"); return(0); } #ifdef NS_CLONE_NEWNET singularity_message(DEBUG, "Using network namespace: CLONE_NEWNET\n"); singularity_priv_escalate(); singularity_message(DEBUG, "Virtualizing network namespace\n"); if ( unshare(CLONE_NEWNET) < 0 ) { singularity_message(ERROR, "Could not virtualize network namespace: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); enabled = 0; #else singularity_message(WARNING, "Skipping network namespace creation, support not available on host\n"); return(0); #endif sockfd = socket(AF_INET, SOCK_DGRAM, 0); if ( sockfd < 0 ) { singularity_message(ERROR, "Unable to open AF_INET socket: %s\n", strerror(errno)); ABORT(255); } memset(&req, 0, sizeof(req)); strncpy(req.ifr_name, "lo", IFNAMSIZ); req.ifr_flags |= IFF_UP; singularity_priv_escalate(); singularity_message(DEBUG, "Bringing up network loopback interface\n"); if ( ioctl(sockfd, SIOCSIFFLAGS, &req) < 0 ) { singularity_message(ERROR, "Failed to set flags on interface: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); return(0); } int _singularity_runtime_ns_net_join(void) { int ns_fd = atoi(singularity_registry_get("DAEMON_NS_FD")); int net_fd; /* Attempt to open /proc/[PID]/ns/net */ singularity_priv_escalate(); net_fd = openat(ns_fd, "net", O_RDONLY); if( net_fd == -1 ) { singularity_message(ERROR, "Could not open NET NS fd: %s\n", strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Attempting to join NET namespace\n"); if ( setns(net_fd, CLONE_NEWNET) < 0 ) { singularity_message(ERROR, "Could not join NET namespace: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); singularity_message(DEBUG, "Successfully joined NET namespace\n"); close(net_fd); return(0); } /* int singularity_ns_net_enabled(void) { singularity_message(DEBUG, "Checking NET namespace enabled: %d\n", enabled); return(enabled); } */ singularity-2.4.2/src/lib/runtime/ns/net/Makefile.am0000644000175000017500000000054513211621077021315 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = net.c EXTRA_DIST = net.h singularity-2.4.2/src/lib/runtime/ns/net/net.h0000644000175000017500000000077013211621077020220 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * This software is licensed under a 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * */ #ifndef __SINGULARITY_RUNTIME_NS_NET_H_ #define __SINGULARITY_RUNTIME_NS_NET_H_ extern int _singularity_runtime_ns_net(void); extern int _singularity_runtime_ns_net_join(void); #endif /* __SINGULARITY_RUNTIME_NS_NET_H */ singularity-2.4.2/src/lib/runtime/Makefile.am0000644000175000017500000000302713211621077020105 0ustar mehdimehdiSUBDIRS = files mounts ns enter environment overlayfs autofs MAINTAINERCLEANFILES = Makefile.in config.h config.h.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie -fPIC AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) distlibdir = $(libdir)/singularity distincludedir = $(includedir)/singularity noinst_LTLIBRARIES = libinternal.la libinternal_la_LIBADD = ns/libinternal.la mounts/libinternal.la files/libinternal.la enter/libinternal.la overlayfs/libinternal.la environment/libinternal.la autofs/libinternal.la libinternal_la_SOURCES = runtime.c ../../util/fork.c ../../util/registry.c ../../util/message.c ../../util/config_parser.c ../../util/privilege.c ../../util/util.c ../../util/file.c ../../util/setns.c ../../util/mount.c libinternal_la_CFLAGS = $(AM_CFLAGS) # This fixes duplicate sources in library and progs distinclude_HEADERS = runtime.h distlib_LTLIBRARIES = libsingularity-runtime.la libsingularity_runtime_la_SOURCES = libsingularity_runtime_la_LIBADD = $(noinst_LTLIBRARIES) libsingularity_runtime_la_LDFLAGS = -version-info 1:0:0 libsingularity_runtime_la_CFLAGS = $(AM_CFLAGS) EXTRA_DIST = # These are kludges so they don't remove the $(DEPDIR) in ../../util/ otherwise # the clean will fail when other Makefile tries to remove those directories distclean: distclean-recursive -rm ./$(DEPDIR) -rm -f Makefile maintainer-clean: maintainer-clean-recursive -rm ./$(DEPDIR) -rm -f Makefile singularity-2.4.2/src/lib/runtime/overlayfs/0000755000175000017500000000000013211621077020061 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/overlayfs/overlayfs.c0000644000175000017500000002024413211621077022241 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "util/message.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/mount.h" #include "lib/image/image.h" #include "../runtime.h" int _singularity_runtime_overlayfs(void) { singularity_priv_escalate(); singularity_message(DEBUG, "Creating overlay_final directory: %s\n", CONTAINER_FINALDIR); if ( s_mkpath(CONTAINER_FINALDIR, 0755) < 0 ) { singularity_message(ERROR, "Failed creating overlay_final directory %s: %s\n", CONTAINER_FINALDIR, strerror(errno)); ABORT(255); } singularity_priv_drop(); singularity_message(DEBUG, "Checking if overlayfs should be used\n"); int try_overlay = ( strcmp("try", singularity_config_get_value(ENABLE_OVERLAY)) == 0 ); if ( !try_overlay && ( singularity_config_get_bool_char(ENABLE_OVERLAY) <= 0 ) ) { singularity_message(VERBOSE3, "Not enabling overlayFS via configuration\n"); } else if ( singularity_registry_get("DISABLE_OVERLAYFS") != NULL ) { singularity_message(VERBOSE3, "Not enabling overlayFS via environment\n"); } else if ( singularity_registry_get("WRITABLE") != NULL ) { singularity_message(VERBOSE3, "Not enabling overlayFS, image mounted writable\n"); } else { char *rootfs_source = CONTAINER_MOUNTDIR; char *overlay_final = CONTAINER_FINALDIR; char *overlay_mount = CONTAINER_OVERLAY; char *overlay_upper = joinpath(overlay_mount, "/upper"); char *overlay_work = joinpath(overlay_mount, "/work"); int overlay_options_len = strlength(rootfs_source, PATH_MAX) + strlength(overlay_upper, PATH_MAX) + strlength(overlay_work, PATH_MAX) + 50; char *overlay_options = (char *) malloc(overlay_options_len); char *overlay_path = NULL; if (try_overlay) singularity_message(VERBOSE3, "Trying OverlayFS as requested by configuration\n"); else singularity_message(VERBOSE3, "OverlayFS enabled by configuration\n"); singularity_message(DEBUG, "Setting up overlay mount options\n"); snprintf(overlay_options, overlay_options_len, "lowerdir=%s,upperdir=%s,workdir=%s", rootfs_source, overlay_upper, overlay_work); // Flawfinder: ignore singularity_message(DEBUG, "Checking for existance of overlay directory: %s\n", overlay_mount); if ( is_dir(overlay_mount) < 0 ) { singularity_message(ERROR, "Overlay mount directory does not exist: %s\n", overlay_mount); ABORT(255); } if ( ( overlay_path = singularity_registry_get("OVERLAYIMAGE") ) != NULL ) { struct image_object image; image = singularity_image_init(singularity_registry_get("OVERLAYIMAGE"), O_RDWR); if ( singularity_image_type(&image) != EXT3 ) { if ( singularity_image_type(&image) == DIRECTORY ) { if ( singularity_priv_getuid() == 0 ) { singularity_message(VERBOSE, "Allowing directory based overlay as root user\n"); } else { singularity_message(ERROR, "Only root can use directory based overlays\n"); ABORT(255); } } else { singularity_message(ERROR, "Persistent overlay must be a writable image or directory\n"); ABORT(255); } } if ( singularity_image_mount(&image, overlay_mount) != 0 ) { singularity_message(ERROR, "Could not mount persistent overlay file: %s\n", singularity_image_name(&image)); ABORT(255); } } else { char *size = NULL; if ( singularity_priv_getuid() == 0 ) { size = strdup(""); } else { size = strdup("size=1m"); } singularity_priv_escalate(); singularity_message(DEBUG, "Mounting overlay tmpfs: %s\n", overlay_mount); if ( singularity_mount("tmpfs", overlay_mount, "tmpfs", MS_NOSUID | MS_NODEV, size) < 0 ){ singularity_message(ERROR, "Failed to mount overlay tmpfs %s: %s\n", overlay_mount, strerror(errno)); ABORT(255); } singularity_priv_drop(); free(size); } if ( is_link(overlay_upper) == 0 ) { singularity_message(ERROR, "symlink detected, upper overlay %s must be a directory\n", overlay_upper); ABORT(255); } if ( is_link(overlay_work) == 0 ) { singularity_message(ERROR, "symlink detected, work overlay %s must be a directory\n", overlay_work); ABORT(255); } singularity_priv_escalate(); singularity_message(DEBUG, "Creating upper overlay directory: %s\n", overlay_upper); if ( s_mkpath(overlay_upper, 0755) < 0 ) { singularity_message(ERROR, "Failed creating upper overlay directory %s: %s\n", overlay_upper, strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Creating overlay work directory: %s\n", overlay_work); if ( s_mkpath(overlay_work, 0755) < 0 ) { singularity_message(ERROR, "Failed creating overlay work directory %s: %s\n", overlay_work, strerror(errno)); ABORT(255); } singularity_message(VERBOSE, "Mounting overlay with options: %s\n", overlay_options); int result = singularity_mount("OverlayFS", overlay_final, "overlay", MS_NOSUID | MS_NODEV, overlay_options); if (result < 0) { if ( (errno == EPERM) || ( try_overlay && ( errno == ENODEV ) ) ) { singularity_message(VERBOSE, "Singularity overlay mount did not work (%s), continuing without it\n", strerror(errno)); singularity_message(DEBUG, "Unmounting overlay tmpfs: %s\n", overlay_mount); umount(overlay_mount); } else { singularity_message(ERROR, "Could not mount Singularity overlay: %s\n", strerror(errno)); ABORT(255); } } singularity_priv_drop(); free(overlay_upper); free(overlay_work); free(overlay_options); if (result >= 0) { singularity_registry_set("OVERLAYFS_ENABLED", "1"); return(0); } } // If we got here, assume we are not overlaying, so we must bind to final directory singularity_priv_escalate(); singularity_message(DEBUG, "Binding container directory to final home %s->%s\n", CONTAINER_MOUNTDIR, CONTAINER_FINALDIR); if ( singularity_mount(CONTAINER_MOUNTDIR, CONTAINER_FINALDIR, NULL, MS_BIND|MS_NOSUID|MS_REC|MS_NODEV, NULL) < 0 ) { singularity_message(ERROR, "Could not bind mount container to final home %s->%s: %s\n", CONTAINER_MOUNTDIR, CONTAINER_FINALDIR, strerror(errno)); return 1; } singularity_priv_drop(); return(0); } singularity-2.4.2/src/lib/runtime/overlayfs/overlayfs.h0000644000175000017500000000234513211621077022250 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_OVERLAYFS_H_ #define __SINGULARITY_RUNTIME_OVERLAYFS_H_ extern int _singularity_runtime_overlayfs(void); #endif /* __SINGULARITY_RUNTIME_OVERLAYFS_H */ singularity-2.4.2/src/lib/runtime/overlayfs/Makefile.am0000644000175000017500000000056113211621077022117 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = overlayfs.c EXTRA_DIST = overlayfs.h singularity-2.4.2/src/lib/runtime/mounts/0000755000175000017500000000000013211621077017374 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/home/0000755000175000017500000000000013211621077020324 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/home/home.h0000644000175000017500000000235113211621077021426 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_HOME_H_ #define __SINGULARITY_RUNTIME_MOUNT_HOME_H_ extern int _singularity_runtime_mount_home(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_HOME_H */ singularity-2.4.2/src/lib/runtime/mounts/home/Makefile.am0000644000175000017500000000054713211621077022366 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = home.c EXTRA_DIST = home.h singularity-2.4.2/src/lib/runtime/mounts/home/home.c0000644000175000017500000001737413211621077021434 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" int _singularity_runtime_mount_home(void) { char *home_source = singularity_priv_homedir(); char *home_dest = singularity_priv_home(); char *session_dir = singularity_registry_get("SESSIONDIR"); char *container_dir = CONTAINER_FINALDIR; if ( singularity_config_get_bool(MOUNT_HOME) <= 0 ) { singularity_message(VERBOSE, "Skipping home dir mounting (per config)\n"); return(0); } singularity_message(DEBUG, "Checking that home directry is configured: %s\n", home_dest); if ( home_dest == NULL ) { singularity_message(ERROR, "Could not obtain user's home directory\n"); ABORT(255); } singularity_message(DEBUG, "Checking if home directories are being influenced by user\n"); if ( singularity_registry_get("HOME") != NULL ) { #ifndef SINGULARITY_NO_NEW_PRIVS singularity_message(WARNING, "Not mounting user requested home: host does not support PR_SET_NO_NEW_PRIVS\n"); ABORT(255); #endif singularity_message(DEBUG, "Checking if user bind control is allowed\n"); if ( singularity_config_get_bool(USER_BIND_CONTROL) <= 0 ) { singularity_message(ERROR, "Not mounting user requested home: User bind control is disallowed\n"); ABORT(255); } } singularity_message(DEBUG, "Checking ownership of home directory source: %s\n", home_source); if ( is_owner(home_source, singularity_priv_getuid()) != 0 ) { singularity_message(ERROR, "Home directory is not owned by calling user: %s\n", home_source); ABORT(255); } singularity_message(DEBUG, "Checking to make sure home directory destination is a full path: %s\n", home_dest); if ( home_dest[0] != '/' ) { singularity_message(ERROR, "Home directory must be a full path: %s\n", home_dest); ABORT(255); } singularity_message(DEBUG, "Checking if home directory is already mounted: %s\n", home_dest); if ( check_mounted(home_dest) >= 0 ) { singularity_message(VERBOSE, "Not mounting home directory (already mounted in container): %s\n", home_dest); return(0); } singularity_message(DEBUG, "Creating temporary directory to stage home: %s\n", joinpath(session_dir, home_dest)); if ( s_mkpath(joinpath(session_dir, home_dest), 0755) < 0 ) { singularity_message(ERROR, "Failed creating home directory stage %s: %s\n", joinpath(session_dir, home_dest), strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Checking if SINGULARITY_CONTAIN is set\n"); if ( ( singularity_registry_get("CONTAIN") == NULL ) || ( singularity_registry_get("HOME") != NULL ) ) { singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting home directory source into session directory: %s -> %s\n", home_source, joinpath(session_dir, home_dest)); if ( singularity_mount(home_source, joinpath(session_dir, home_dest), NULL, MS_BIND | MS_NOSUID | MS_NODEV | MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Failed to mount home directory %s -> %s: %s\n", home_source, joinpath(session_dir, home_dest), strerror(errno)); ABORT(255); } if ( singularity_priv_userns_enabled() != 1 ) { if ( singularity_mount(NULL, joinpath(session_dir, home_dest), NULL, MS_BIND | MS_REMOUNT | MS_NODEV | MS_NOSUID | MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Failed to remount home directory base %s: %s\n", joinpath(session_dir, home_dest), strerror(errno)); ABORT(255); } } singularity_priv_drop(); } else { singularity_message(VERBOSE, "Using sessiondir for home directory\n"); } singularity_message(DEBUG, "Checking if overlay is enabled\n"); if ( singularity_registry_get("OVERLAYFS_ENABLED") == NULL ) { char *homedir_base; singularity_message(DEBUG, "Staging home directory base\n"); singularity_message(DEBUG, "Identifying the base home directory: %s\n", home_dest); if ( ( homedir_base = basedir(home_dest) ) == NULL ) { singularity_message(ERROR, "Could not identify base home directory path: %s\n", home_dest); ABORT(255); } singularity_message(DEBUG, "Checking home directory base exists in container: %s\n", homedir_base); if ( is_dir(joinpath(container_dir, homedir_base)) != 0 ) { singularity_message(ERROR, "Base home directory does not exist within the container: %s\n", homedir_base); ABORT(255); } singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting staged home directory base to container's base dir: %s -> %s\n", joinpath(session_dir, homedir_base), joinpath(container_dir, homedir_base)); if ( singularity_mount(joinpath(session_dir, homedir_base), joinpath(container_dir, homedir_base), NULL, MS_BIND | MS_NOSUID | MS_NODEV | MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Failed to mount staged home base: %s -> %s: %s\n", joinpath(session_dir, homedir_base), joinpath(container_dir, homedir_base), strerror(errno)); ABORT(255); } singularity_priv_drop(); free(homedir_base); } else { singularity_message(DEBUG, "Staging home directory\n"); singularity_priv_escalate(); singularity_message(DEBUG, "Creating home directory within container: %s\n", joinpath(container_dir, home_dest)); if ( s_mkpath(joinpath(container_dir, home_dest), 0755) < 0 ) { singularity_message(ERROR, "Failed creating home directory in container %s: %s\n", joinpath(container_dir, home_dest), strerror(errno)); ABORT(255); } singularity_message(VERBOSE, "Mounting staged home directory to container: %s -> %s\n", joinpath(session_dir, home_dest), joinpath(container_dir, home_dest)); if ( singularity_mount(joinpath(session_dir, home_dest), joinpath(container_dir, home_dest), NULL, MS_BIND | MS_NOSUID | MS_NODEV | MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Failed to mount staged home base: %s -> %s: %s\n", joinpath(session_dir, home_dest), joinpath(container_dir, home_dest), strerror(errno)); ABORT(255); } singularity_priv_drop(); } envar_set("HOME", home_dest, 1); free(home_source); free(home_dest); free(session_dir); return(0); } singularity-2.4.2/src/lib/runtime/mounts/tmp/0000755000175000017500000000000013211621077020174 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/tmp/tmp.c0000644000175000017500000001376013211621077021147 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" int _singularity_runtime_mount_tmp(void) { char *container_dir = CONTAINER_FINALDIR; char *tmp_source; char *vartmp_source; if ( singularity_config_get_bool(MOUNT_TMP) <= 0 ) { singularity_message(VERBOSE, "Skipping tmp dir mounting (per config)\n"); return(0); } if ( singularity_registry_get("CONTAIN") == NULL ) { tmp_source = strdup("/tmp"); vartmp_source = strdup("/var/tmp"); } else { char *tmpdirpath; if ( ( tmpdirpath = singularity_registry_get("WORKDIR") ) != NULL ) { if ( singularity_config_get_bool(USER_BIND_CONTROL) <= 0 ) { singularity_message(ERROR, "User bind control is disabled by system administrator\n"); ABORT(5); } #ifndef SINGULARITY_NO_NEW_PRIVS singularity_message(WARNING, "Not mounting workdir: host does not support PR_SET_NO_NEW_PRIVS\n"); ABORT(255); #endif tmp_source = joinpath(tmpdirpath, "/tmp"); vartmp_source = joinpath(tmpdirpath, "/var_tmp"); } else { char *sessiondir = singularity_registry_get("SESSIONDIR"); tmp_source = joinpath(sessiondir, "/tmp"); vartmp_source = joinpath(sessiondir, "/var_tmp"); } free(tmpdirpath); } if ( check_mounted("/tmp") < 0 ) { if ( s_mkpath(tmp_source, 0755) < 0 ) { singularity_message(ERROR, "Could not create source /tmp directory %s: %s\n", tmp_source, strerror(errno)); ABORT(255); } if ( is_dir(tmp_source) == 0 ) { if ( is_dir(joinpath(container_dir, "/tmp")) == 0 ) { singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting directory: /tmp\n"); if ( singularity_mount(tmp_source, joinpath(container_dir, "/tmp"), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Failed to mount %s -> /tmp: %s\n", tmp_source, strerror(errno)); ABORT(255); } if ( singularity_priv_userns_enabled() != 1 ) { if ( singularity_mount(NULL, joinpath(container_dir, "/tmp"), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC|MS_REMOUNT, NULL) < 0 ) { singularity_message(ERROR, "Failed to remount /tmp: %s\n", strerror(errno)); ABORT(255); } } singularity_priv_drop(); } else { singularity_message(VERBOSE, "Could not mount container's /tmp directory: does not exist\n"); } } else { singularity_message(VERBOSE, "Could not mount host's /tmp directory (%s): does not exist\n", tmp_source); } } else { singularity_message(VERBOSE, "Not mounting '/tmp', already mounted\n"); } if ( check_mounted("/var/tmp") < 0 ) { if ( s_mkpath(vartmp_source, 0755) < 0 ) { singularity_message(ERROR, "Could not create source /var/tmp directory %s: %s\n", vartmp_source, strerror(errno)); ABORT(255); } if ( is_dir(vartmp_source) == 0 ) { if ( is_dir(joinpath(container_dir, "/var/tmp")) == 0 ) { singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting directory: /var/tmp\n"); if ( singularity_mount(vartmp_source, joinpath(container_dir, "/var/tmp"), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Failed to mount %s -> /var/tmp: %s\n", vartmp_source, strerror(errno)); ABORT(255); } if ( singularity_priv_userns_enabled() != 1 ) { if ( singularity_mount(NULL, joinpath(container_dir, "/var/tmp"), NULL, MS_BIND|MS_NOSUID|MS_REC|MS_NODEV|MS_REMOUNT, NULL) < 0 ) { singularity_message(ERROR, "Failed to remount /var/tmp: %s\n", strerror(errno)); ABORT(255); } } singularity_priv_drop(); } else { singularity_message(VERBOSE, "Could not mount container's /var/tmp directory: does not exist\n"); } } else { singularity_message(VERBOSE, "Could not mount host's /var/tmp directory (%s): does not exist\n", vartmp_source); } } else { singularity_message(VERBOSE, "Not mounting '/var/tmp', already mounted\n"); } free(tmp_source); free(vartmp_source); return(0); } singularity-2.4.2/src/lib/runtime/mounts/tmp/tmp.h0000644000175000017500000000234513211621077021151 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_TMP_H_ #define __SINGULARITY_RUNTIME_MOUNT_TMP_H_ extern int _singularity_runtime_mount_tmp(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_TMP_H */ singularity-2.4.2/src/lib/runtime/mounts/tmp/Makefile.am0000644000175000017500000000054513211621077022234 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = tmp.c EXTRA_DIST = tmp.h singularity-2.4.2/src/lib/runtime/mounts/cwd/0000755000175000017500000000000013211621077020151 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/cwd/cwd.h0000644000175000017500000000234513211621077021103 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_CWD_H_ #define __SINGULARITY_RUNTIME_MOUNT_CWD_H_ extern int _singularity_runtime_mount_cwd(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_CWD_H */ singularity-2.4.2/src/lib/runtime/mounts/cwd/Makefile.am0000644000175000017500000000054513211621077022211 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = cwd.c EXTRA_DIST = cwd.h singularity-2.4.2/src/lib/runtime/mounts/cwd/cwd.c0000644000175000017500000001322213211621077021072 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef _GNU_SOURCE #define _GNU_SOURCE #endif #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" int _singularity_runtime_mount_cwd(void) { char *container_dir = CONTAINER_FINALDIR; char *cwd_path = NULL; int r; singularity_message(DEBUG, "Checking to see if we should mount current working directory\n"); singularity_message(DEBUG, "Getting current working directory\n"); cwd_path = get_current_dir_name(); if ( cwd_path == NULL ) { singularity_message(ERROR, "Could not obtain current directory path: %s\n", strerror(errno)); ABORT(1); } singularity_message(DEBUG, "Checking for contain option\n"); if ( singularity_registry_get("CONTAIN") != NULL ) { singularity_message(VERBOSE, "Not mounting current directory: contain was requested\n"); free(cwd_path); return(0); } singularity_message(DEBUG, "Checking if current directory already available within container: %s\n", cwd_path); if ( is_dir(joinpath(container_dir, cwd_path)) == 0 ) { char *cwd_fileid = file_devino(cwd_path); char *container_cwd_fileid = file_devino(joinpath(container_dir, cwd_path)); singularity_message(DEBUG, "Checking if container's cwd == host's cwd\n"); if ( strcmp(cwd_fileid, container_cwd_fileid) == 0 ) { singularity_message(VERBOSE, "Not mounting current directory: location already available within container\n"); free(cwd_path); free(cwd_fileid); free(container_cwd_fileid); return(0); } else { singularity_message(DEBUG, "Container's cwd is not the same as the host, continuing on...\n"); } } else { singularity_message(VERBOSE, "Not mounting CWD, directory does not exist within container: %s\n", cwd_path); free(cwd_path); return(0); } singularity_message(DEBUG, "Checking if CWD is already mounted: %s\n", cwd_path); if ( check_mounted(cwd_path) >= 0 ) { singularity_message(VERBOSE, "Not mounting CWD (already mounted in container): %s\n", cwd_path); free(cwd_path); return(0); } singularity_message(DEBUG, "Checking if cwd is in an operating system directory\n"); if ( ( strcmp(cwd_path, "/") == 0 ) || ( strcmp(cwd_path, "/bin") == 0 ) || ( strcmp(cwd_path, "/etc") == 0 ) || ( strcmp(cwd_path, "/mnt") == 0 ) || ( strcmp(cwd_path, "/usr") == 0 ) || ( strcmp(cwd_path, "/var") == 0 ) || ( strcmp(cwd_path, "/opt") == 0 ) || ( strcmp(cwd_path, "/sbin") == 0 ) ) { singularity_message(VERBOSE, "Not mounting CWD within operating system directory: %s\n", cwd_path); free(cwd_path); return(0); } singularity_message(DEBUG, "Checking if cwd is in a virtual directory\n"); if ( ( strncmp(cwd_path, "/sys", 4) == 0 ) || ( strncmp(cwd_path, "/dev", 4) == 0 ) || ( strncmp(cwd_path, "/proc", 5) == 0 ) ) { singularity_message(VERBOSE, "Not mounting CWD within virtual directory: %s\n", cwd_path); free(cwd_path); return(0); } singularity_message(DEBUG, "Checking configuration file for 'user bind control'\n"); if ( singularity_config_get_bool(USER_BIND_CONTROL) <= 0 ) { singularity_message(WARNING, "Not mounting current directory: user bind control is disabled by system administrator\n"); free(cwd_path); return(0); } #ifndef SINGULARITY_NO_NEW_PRIVS singularity_message(WARNING, "Not mounting current directory: host does not support PR_SET_NO_NEW_PRIVS\n"); free(cwd_path); return(0); #endif singularity_message(VERBOSE, "Binding '%s' to '%s/%s'\n", cwd_path, container_dir, cwd_path); singularity_priv_escalate(); r = singularity_mount(cwd_path, joinpath(container_dir, cwd_path), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL); if ( singularity_priv_userns_enabled() != 1 ) { r = singularity_mount(NULL, joinpath(container_dir, cwd_path), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC|MS_REMOUNT, NULL); } singularity_priv_drop(); if ( r < 0 ) { singularity_message(WARNING, "Could not bind CWD to container %s: %s\n", cwd_path, strerror(errno)); } free(cwd_path); return(0); } singularity-2.4.2/src/lib/runtime/mounts/scratch/0000755000175000017500000000000013211621077021023 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/scratch/scratch.c0000644000175000017500000001251213211621077022617 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" int _singularity_runtime_mount_scratch(void) { char *container_dir = CONTAINER_FINALDIR; char *scratchdir_path; char *tmpdir_path; char *sourcedir_path; int r; singularity_message(DEBUG, "Getting SINGULARITY_SCRATCHDIR from environment\n"); if ( ( scratchdir_path = singularity_registry_get("SCRATCHDIR") ) == NULL ) { singularity_message(DEBUG, "Not mounting scratch directory: Not requested\n"); return(0); } singularity_message(DEBUG, "Checking configuration file for 'user bind control'\n"); if ( singularity_config_get_bool(USER_BIND_CONTROL) <= 0 ) { singularity_message(VERBOSE, "Not mounting scratch: user bind control is disabled by system administrator\n"); return(0); } #ifndef SINGULARITY_NO_NEW_PRIVS singularity_message(WARNING, "Not mounting scratch: host does not support PR_SET_NO_NEW_PRIVS\n"); return(0); #endif singularity_message(DEBUG, "Checking SINGULARITY_WORKDIR from environment\n"); if ( ( tmpdir_path = singularity_registry_get("WORKDIR") ) == NULL ) { if ( ( tmpdir_path = singularity_registry_get("SESSIONDIR") ) == NULL ) { singularity_message(ERROR, "Could not identify a suitable temporary directory for scratch\n"); return(0); } } sourcedir_path = joinpath(tmpdir_path, "/scratch"); free(tmpdir_path); char *outside_token = NULL; char *current = strtok_r(strdup(scratchdir_path), ",", &outside_token); free(scratchdir_path); while ( current != NULL ) { char *full_sourcedir_path = joinpath(sourcedir_path, basename(strdup(current))); char *full_destdir_path = joinpath(container_dir, current); if ( s_mkpath(full_sourcedir_path, 0750) < 0 ) { singularity_message(ERROR, "Could not create scratch working directory %s: %s\n", full_sourcedir_path, strerror(errno)); ABORT(255); } if ( is_dir(full_destdir_path) != 0 ) { if ( singularity_registry_get("OVERLAYFS_ENABLED") != NULL ) { singularity_priv_escalate(); singularity_message(DEBUG, "Creating scratch directory inside container\n"); r = s_mkpath(full_destdir_path, 0755); singularity_priv_drop(); if ( r < 0 ) { singularity_message(VERBOSE, "Skipping scratch directory mount, could not create dir inside container %s: %s\n", current, strerror(errno)); current = strtok_r(NULL, ",", &outside_token); continue; } } else { singularity_message(WARNING, "Skipping scratch directory mount, target directory does not exist: %s\n", current); current = strtok_r(NULL, ",", &outside_token); continue; } } singularity_priv_escalate(); singularity_message(VERBOSE, "Binding '%s' to '%s/%s'\n", full_sourcedir_path, container_dir, current); r = singularity_mount(full_sourcedir_path, joinpath(container_dir, current), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL); if ( singularity_priv_userns_enabled() != 1 ) { r += singularity_mount(NULL, joinpath(container_dir, current), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC|MS_REMOUNT, NULL); } singularity_priv_drop(); if ( r < 0 ) { singularity_message(WARNING, "Could not bind scratch directory into container %s: %s\n", full_sourcedir_path, strerror(errno)); ABORT(255); } free(full_sourcedir_path); free(full_destdir_path); current = strtok_r(NULL, ",", &outside_token); // Skip empty directories. while ( current && !strlength(current, 1024) ) { current = strtok_r(NULL, ",", &outside_token); } } return(0); } singularity-2.4.2/src/lib/runtime/mounts/scratch/Makefile.am0000644000175000017500000000055513211621077023064 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = scratch.c EXTRA_DIST = scratch.h singularity-2.4.2/src/lib/runtime/mounts/scratch/scratch.h0000644000175000017500000000236513211621077022631 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_SCRATCH_H_ #define __SINGULARITY_RUNTIME_MOUNT_SCRATCH_H_ extern int _singularity_runtime_mount_scratch(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_SCRATCH_H */ singularity-2.4.2/src/lib/runtime/mounts/mounts.h0000644000175000017500000000232613211621077021075 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_H_ #define __SINGULARITY_RUNTIME_MOUNT_H_ extern int _singularity_runtime_mounts(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_H */ singularity-2.4.2/src/lib/runtime/mounts/hostfs/0000755000175000017500000000000013211621077020702 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/hostfs/hostfs.h0000644000175000017500000000236113211621077022363 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_HOSTFS_H_ #define __SINGULARITY_RUNTIME_MOUNT_HOSTFS_H_ extern int _singularity_runtime_mount_hostfs(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_HOSTFS_H */ singularity-2.4.2/src/lib/runtime/mounts/hostfs/Makefile.am0000644000175000017500000000055313211621077022741 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = hostfs.c EXTRA_DIST = hostfs.h singularity-2.4.2/src/lib/runtime/mounts/hostfs/hostfs.c0000644000175000017500000001672313211621077022365 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" #define MAX_LINE_LEN 4096 int _singularity_runtime_mount_hostfs(void) { FILE *mounts; char *line = NULL; char *container_dir = CONTAINER_FINALDIR; if ( singularity_config_get_bool(MOUNT_HOSTFS) <= 0 ) { singularity_message(DEBUG, "Not mounting host file systems per configuration\n"); return(0); } singularity_message(DEBUG, "Checking to see if /proc/mounts exists\n"); if ( is_file("/proc/mounts") < 0 ) { singularity_message(WARNING, "Can not probe for currently mounted host file systems\n"); return(1); } singularity_message(DEBUG, "Opening /proc/mounts\n"); if ( ( mounts = fopen("/proc/mounts", "r") ) == NULL ) { // Flawfinder: ignore singularity_message(ERROR, "Could not open /proc/mounts for reading: %s\n", strerror(errno)); return(1); } line = (char *)malloc(MAX_LINE_LEN); singularity_message(DEBUG, "Getting line by line\n"); while ( fgets(line, MAX_LINE_LEN, mounts) ) { char *source; char *mountpoint; char *filesystem; if ( line == NULL ) { singularity_message(DEBUG, "Skipping empty line in /proc/mounts\n"); continue; } chomp(line); if ( line[0] == '#' || strlength(line, 2) <= 1 ) { // Flawfinder: ignore singularity_message(VERBOSE3, "Skipping blank or comment line in /proc/mounts\n"); continue; } if ( ( source = strtok(strdup(line), " ") ) == NULL ) { singularity_message(VERBOSE3, "Could not obtain mount source from /proc/mounts: %s\n", line); continue; } if ( ( mountpoint = strtok(NULL, " ") ) == NULL ) { singularity_message(VERBOSE3, "Could not obtain mount point from /proc/mounts: %s\n", line); continue; } if ( ( filesystem = strtok(NULL, " ") ) == NULL ) { singularity_message(VERBOSE3, "Could not obtain file system from /proc/mounts: %s\n", line); continue; } if ( strcmp(mountpoint, "/") == 0 ) { singularity_message(DEBUG, "Skipping root (/): %s,%s,%s\n", source, mountpoint, filesystem); continue; } if ( strncmp(mountpoint, "/sys", 4) == 0 ) { singularity_message(DEBUG, "Skipping /sys based file system: %s,%s,%s\n", source, mountpoint, filesystem); continue; } if ( strncmp(mountpoint, "/boot", 5) == 0 ) { singularity_message(DEBUG, "Skipping /boot based file system: %s,%s,%s\n", source, mountpoint, filesystem); continue; } if ( strncmp(mountpoint, "/proc", 5) == 0 ) { singularity_message(DEBUG, "Skipping /proc based file system: %s,%s,%s\n", source, mountpoint, filesystem); continue; } if ( strncmp(mountpoint, "/dev", 4) == 0 ) { singularity_message(DEBUG, "Skipping /dev based file system: %s,%s,%s\n", source, mountpoint, filesystem); continue; } if ( strncmp(mountpoint, "/run", 4) == 0 ) { singularity_message(DEBUG, "Skipping /run based file system: %s,%s,%s\n", source, mountpoint, filesystem); continue; } if ( strncmp(mountpoint, "/var", 4) == 0 ) { singularity_message(DEBUG, "Skipping /var based file system: %s,%s,%s\n", source, mountpoint, filesystem); continue; } if ( strncmp(mountpoint, container_dir, strlength(container_dir, PATH_MAX)) == 0 ) { singularity_message(DEBUG, "Skipping container_dir (%s) based file system: %s,%s,%s\n", container_dir, source, mountpoint, filesystem); continue; } if ( strcmp(filesystem, "tmpfs") == 0 ) { singularity_message(DEBUG, "Skipping tmpfs file system: %s,%s,%s\n", source, mountpoint, filesystem); continue; } if ( strcmp(filesystem, "cgroup") == 0 ) { singularity_message(DEBUG, "Skipping cgroup file system: %s,%s,%s\n", source, mountpoint, filesystem); continue; } singularity_message(DEBUG, "Checking if host file system is already mounted: %s\n", mountpoint); if ( check_mounted(mountpoint) >= 0 ) { singularity_message(VERBOSE, "Not mounting host FS (already mounted in container): %s\n", mountpoint); continue; } if ( ( is_dir(mountpoint) == 0 ) && ( is_dir(joinpath(container_dir, mountpoint)) < 0 ) ) { if ( singularity_registry_get("OVERLAYFS_ENABLED") != NULL ) { singularity_priv_escalate(); if ( s_mkpath(joinpath(container_dir, mountpoint), 0755) < 0 ) { singularity_priv_drop(); singularity_message(WARNING, "Could not create bind point directory in container %s: %s\n", mountpoint, strerror(errno)); continue; } singularity_priv_drop(); } else { singularity_message(WARNING, "Non existent 'bind point' directory in container: '%s'\n", mountpoint); continue; } } singularity_priv_escalate(); singularity_message(VERBOSE, "Binding '%s'(%s) to '%s/%s'\n", mountpoint, filesystem, container_dir, mountpoint); if ( singularity_mount(mountpoint, joinpath(container_dir, mountpoint), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL) < 0 ) { singularity_message(ERROR, "There was an error binding the path %s: %s\n", mountpoint, strerror(errno)); ABORT(255); } if ( singularity_priv_userns_enabled() != 1 ) { if ( singularity_mount(NULL, joinpath(container_dir, mountpoint), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC|MS_REMOUNT, NULL) < 0 ) { singularity_message(ERROR, "There was an error remounting the path %s: %s\n", mountpoint, strerror(errno)); ABORT(255); } } singularity_priv_drop(); } free(line); fclose(mounts); return(0); } singularity-2.4.2/src/lib/runtime/mounts/binds/0000755000175000017500000000000013211621077020473 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/binds/binds.c0000644000175000017500000001446613211621077021751 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" int _singularity_runtime_mount_binds(void) { char *tmp_config_string; char *container_dir = CONTAINER_FINALDIR; if ( singularity_registry_get("CONTAIN") != NULL ) { singularity_message(DEBUG, "Skipping bind mounts as contain was requested\n"); return(0); } singularity_message(DEBUG, "Checking configuration file for 'bind path'\n"); const char **tmp_config_string_list = singularity_config_get_value_multi(BIND_PATH); if ( strlength(*tmp_config_string_list, 1) == 0 ) { return(0); } while ( *tmp_config_string_list != NULL ) { tmp_config_string = strdup(*tmp_config_string_list); tmp_config_string_list++; char *source = strtok(tmp_config_string, ":"); char *dest = strtok(NULL, ":"); chomp(source); if ( dest == NULL ) { dest = strdup(source); } else { chomp(dest); } singularity_message(VERBOSE2, "Found 'bind path' = %s, %s\n", source, dest); if ( ( is_file(source) < 0 ) && ( is_dir(source) < 0 ) ) { singularity_message(WARNING, "Non existent 'bind path' source: '%s'\n", source); continue; } singularity_message(DEBUG, "Checking if bind point is already mounted: %s\n", dest); if ( check_mounted(dest) >= 0 ) { singularity_message(VERBOSE, "Not mounting bind point (already mounted): %s\n", dest); continue; } if ( ( is_file(source) == 0 ) && ( is_file(joinpath(container_dir, dest)) < 0 ) ) { if ( singularity_registry_get("OVERLAYFS_ENABLED") != NULL ) { char *basedir = dirname(joinpath(container_dir, dest)); singularity_message(DEBUG, "Checking base directory for file %s ('%s')\n", dest, basedir); if ( is_dir(basedir) != 0 ) { singularity_message(DEBUG, "Creating base directory for file bind\n"); singularity_priv_escalate(); if ( s_mkpath(basedir, 0755) != 0 ) { singularity_message(ERROR, "Failed creating base directory to bind file: %s\n", dest); ABORT(255); } singularity_priv_drop(); } free(basedir); singularity_priv_escalate(); singularity_message(VERBOSE3, "Creating bind file on overlay file system: %s\n", dest); FILE *tmp = fopen(joinpath(container_dir, dest), "w+"); // Flawfinder: ignore singularity_priv_drop(); if ( tmp == NULL ) { singularity_message(WARNING, "Could not create bind point file in container %s: %s\n", dest, strerror(errno)); continue; } if ( fclose(tmp) != 0 ) { singularity_message(WARNING, "Could not close bind point file descriptor %s: %s\n", dest, strerror(errno)); continue; } singularity_message(DEBUG, "Created bind file: %s\n", dest); } else { singularity_message(WARNING, "Non existent bind point (file) in container: '%s'\n", dest); continue; } } else if ( ( is_dir(source) == 0 ) && ( is_dir(joinpath(container_dir, dest)) < 0 ) ) { if ( singularity_registry_get("OVERLAYFS_ENABLED") != NULL ) { singularity_priv_escalate(); singularity_message(VERBOSE3, "Creating bind directory on overlay file system: %s\n", dest); if ( s_mkpath(joinpath(container_dir, dest), 0755) < 0 ) { singularity_priv_drop(); singularity_message(WARNING, "Could not create bind point directory in container %s: %s\n", dest, strerror(errno)); continue; } singularity_priv_drop(); } else { singularity_message(WARNING, "Non existent bind point (directory) in container: '%s'\n", dest); continue; } } singularity_priv_escalate(); singularity_message(VERBOSE, "Binding '%s' to '%s/%s'\n", source, container_dir, dest); if ( singularity_mount(source, joinpath(container_dir, dest), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL) < 0 ) { singularity_message(ERROR, "There was an error binding the path %s: %s\n", source, strerror(errno)); ABORT(255); } if ( singularity_priv_userns_enabled() != 1 ) { if ( singularity_mount(NULL, joinpath(container_dir, dest), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC|MS_REMOUNT, NULL) < 0 ) { singularity_message(ERROR, "There was an error remounting the path %s: %s\n", source, strerror(errno)); ABORT(255); } } singularity_priv_drop(); } return(0); } singularity-2.4.2/src/lib/runtime/mounts/binds/Makefile.am0000644000175000017500000000055113211621077022530 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = binds.c EXTRA_DIST = binds.h singularity-2.4.2/src/lib/runtime/mounts/binds/binds.h0000644000175000017500000000235513211621077021750 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_BINDS_H_ #define __SINGULARITY_RUNTIME_MOUNT_BINDS_H_ extern int _singularity_runtime_mount_binds(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_BINDS_H */ singularity-2.4.2/src/lib/runtime/mounts/Makefile.am0000644000175000017500000000143013211621077021426 0ustar mehdimehdiSUBDIRS = binds cwd dev home hostfs kernelfs scratch tmp userbinds MAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie libns_pid_a_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_LIBADD = binds/libinternal.la cwd/libinternal.la dev/libinternal.la home/libinternal.la hostfs/libinternal.la kernelfs/libinternal.la scratch/libinternal.la tmp/libinternal.la userbinds/libinternal.la libinternal_la_SOURCES = mounts.c EXTRA_DIST = mounts.h singularity-2.4.2/src/lib/runtime/mounts/mounts.c0000644000175000017500000000416613211621077021074 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "./binds/binds.h" #include "./home/home.h" #include "./hostfs/hostfs.h" #include "./kernelfs/kernelfs.h" #include "./tmp/tmp.h" #include "./dev/dev.h" #include "./cwd/cwd.h" #include "./userbinds/userbinds.h" #include "./scratch/scratch.h" int _singularity_runtime_mounts(void) { int retval = 0; singularity_message(VERBOSE, "Running all mount components\n"); retval += _singularity_runtime_mount_hostfs(); retval += _singularity_runtime_mount_binds(); retval += _singularity_runtime_mount_kernelfs(); retval += _singularity_runtime_mount_dev(); retval += _singularity_runtime_mount_home(); retval += _singularity_runtime_mount_userbinds(); retval += _singularity_runtime_mount_tmp(); retval += _singularity_runtime_mount_scratch(); retval += _singularity_runtime_mount_cwd(); return(retval); } singularity-2.4.2/src/lib/runtime/mounts/kernelfs/0000755000175000017500000000000013211621077021205 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/kernelfs/kernelfs.h0000644000175000017500000000237113211621077023172 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_KERNELFS_H_ #define __SINGULARITY_RUNTIME_MOUNT_KERNELFS_H_ extern int _singularity_runtime_mount_kernelfs(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_KERNELFS_H */ singularity-2.4.2/src/lib/runtime/mounts/kernelfs/kernelfs.c0000644000175000017500000001056613211621077023172 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" #include "../../ns/ns.h" int _singularity_runtime_mount_kernelfs(void) { char *container_dir = CONTAINER_FINALDIR; // Mount /proc if we are configured singularity_message(DEBUG, "Checking configuration file for 'mount proc'\n"); if ( singularity_config_get_bool(MOUNT_PROC) > 0 ) { if ( is_dir(joinpath(container_dir, "/proc")) == 0 ) { if ( singularity_registry_get("PIDNS_ENABLED") == NULL ) { singularity_priv_escalate(); singularity_message(VERBOSE, "Bind-mounting host /proc\n"); if ( singularity_mount("/proc", joinpath(container_dir, "/proc"), NULL, MS_BIND | MS_NOSUID | MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Could not bind-mount host /proc into container: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); } else { singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting new procfs\n"); if ( singularity_mount("proc", joinpath(container_dir, "/proc"), "proc", MS_NOSUID, NULL) < 0 ) { singularity_message(ERROR, "Could not mount new procfs into container: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); } } else { singularity_message(WARNING, "Not mounting /proc, container has no bind directory\n"); } } else { singularity_message(VERBOSE, "Skipping /proc mount\n"); } // Mount /sys if we are configured singularity_message(DEBUG, "Checking configuration file for 'mount sys'\n"); if ( singularity_config_get_bool(MOUNT_SYS) > 0 ) { if ( is_dir(joinpath(container_dir, "/sys")) == 0 ) { if ( singularity_priv_userns_enabled() == 1 ) { singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting /sys\n"); if ( singularity_mount("/sys", joinpath(container_dir, "/sys"), NULL, MS_BIND | MS_NOSUID | MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Could not mount /sys into container: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); } else { singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting /sys\n"); if ( singularity_mount("sysfs", joinpath(container_dir, "/sys"), "sysfs", MS_NOSUID, NULL) < 0 ) { singularity_message(ERROR, "Could not mount /sys into container: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); } } else { singularity_message(WARNING, "Not mounting /sys, container has no bind directory\n"); } } else { singularity_message(VERBOSE, "Skipping /sys mount\n"); } return(0); } singularity-2.4.2/src/lib/runtime/mounts/kernelfs/Makefile.am0000644000175000017500000000055713211621077023250 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = kernelfs.c EXTRA_DIST = kernelfs.h singularity-2.4.2/src/lib/runtime/mounts/dev/0000755000175000017500000000000013211621077020152 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/dev/dev.c0000644000175000017500000002202113211621077021071 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" static int bind_dev(char *tmpdir, char *dev); int _singularity_runtime_mount_dev(void) { char *container_dir = CONTAINER_FINALDIR; if ( ( singularity_registry_get("CONTAIN") != NULL ) || ( strcmp("minimal", singularity_config_get_value(MOUNT_DEV)) == 0 ) ) { char *sessiondir = singularity_registry_get("SESSIONDIR"); char *devdir = joinpath(sessiondir, "/dev"); if ( is_dir(joinpath(container_dir, "/dev")) < 0 ) { int ret; if ( singularity_registry_get("OVERLAYFS_ENABLED") == NULL ) { singularity_message(WARNING, "Not mounting devices as /dev directory does not exist within container\n"); return(-1); } singularity_priv_escalate(); ret = s_mkpath(joinpath(container_dir, "/dev"), 0755); singularity_priv_drop(); if ( ret < 0 ) { singularity_message(ERROR, "Could not create /dev inside container\n"); ABORT(255); } } singularity_message(DEBUG, "Creating temporary staged /dev\n"); if ( s_mkpath(devdir, 0755) != 0 ) { singularity_message(ERROR, "Failed creating the session device directory %s: %s\n", devdir, strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Creating temporary staged /dev/shm\n"); if ( s_mkpath(joinpath(devdir, "/shm"), 0755) != 0 ) { singularity_message(ERROR, "Failed creating temporary /dev/shm %s: %s\n", joinpath(devdir, "/shm"), strerror(errno)); ABORT(255); } if ( singularity_config_get_bool_char(MOUNT_DEVPTS) > 0 ) { struct stat multi_instance_devpts; if( stat("/dev/pts/ptmx", &multi_instance_devpts) < 0 ) { singularity_message(ERROR, "Multiple devpts instances unsupported and \"%s\" configured\n", MOUNT_DEVPTS); ABORT(255); } singularity_message(DEBUG, "Creating staged /dev/pts\n"); if ( s_mkpath(joinpath(devdir, "/pts"), 0755) != 0 ) { singularity_message(ERROR, "Failed creating /dev/pts %s: %s\n", joinpath(devdir, "/pts"), strerror(errno)); ABORT(255); } bind_dev(sessiondir, "/dev/tty"); } bind_dev(sessiondir, "/dev/null"); bind_dev(sessiondir, "/dev/zero"); bind_dev(sessiondir, "/dev/random"); bind_dev(sessiondir, "/dev/urandom"); singularity_priv_escalate(); singularity_message(DEBUG, "Mounting tmpfs for staged /dev/shm\n"); if ( singularity_mount("/dev/shm", joinpath(devdir, "/shm"), "tmpfs", MS_NOSUID, "") < 0 ) { singularity_message(ERROR, "Failed to mount %s: %s\n", joinpath(devdir, "/shm"), strerror(errno)); ABORT(255); } if ( singularity_config_get_bool_char(MOUNT_DEVPTS) > 0 ) { struct group *ttygid; char *devpts_opts_base="mode=0620,newinstance,ptmxmode=0666,gid="; char *devpts_opts; unsigned int max_sz, gd, gd_n; if ( (ttygid=getgrnam("tty")) == NULL ) { singularity_message(ERROR, "Problem resolving 'tty' group GID: %s\n", strerror(errno)); ABORT(255); } /* number of digits in gid */ if ( ttygid->gr_gid == 0 ) { gd_n = 1; } else { gd_n = 0; gd = ttygid->gr_gid; while ( gd ) { gd /= 10; gd_n++; } } /* length of gid string + mount options + terminator + padding */ max_sz = gd_n + strlen(devpts_opts_base) + 1 + 16; if ( (devpts_opts=malloc(max_sz)) == NULL ) { singularity_message(ERROR, "Memory allocation failure: %s\n", strerror(errno)); ABORT(255); } bzero(devpts_opts, max_sz); snprintf(devpts_opts, max_sz-1, "%s%d", devpts_opts_base, ttygid->gr_gid); singularity_message(DEBUG, "Mounting devpts for staged /dev/pts\n"); if ( singularity_mount("devpts", joinpath(devdir, "/pts"), "devpts", MS_NOSUID|MS_NOEXEC, devpts_opts) < 0 ) { if (errno == EINVAL) { // This is the error when unprivileged on RHEL7.4 singularity_message(VERBOSE, "Couldn't mount %s, continuing\n", joinpath(devdir, "/pts")); } else { singularity_message(ERROR, "Failed to mount %s: %s\n", joinpath(devdir, "/pts"), strerror(errno)); ABORT(255); } } else { singularity_message(DEBUG, "Creating staged /dev/ptmx symlink\n"); if ( symlink("/dev/pts/ptmx", joinpath(devdir, "/ptmx")) < 0 ) { singularity_message(ERROR, "Failed to create /dev/ptmx symlink: %s\n", strerror(errno)); ABORT(255); } } free(devpts_opts); } singularity_message(DEBUG, "Mounting minimal staged /dev into container\n"); if ( singularity_mount(devdir, joinpath(container_dir, "/dev"), NULL, MS_BIND|MS_REC, NULL) < 0 ) { singularity_priv_drop(); singularity_message(WARNING, "Could not stage dev tree: '%s' -> '%s': %s\n", devdir, joinpath(container_dir, "/dev"), strerror(errno)); free(sessiondir); free(devdir); return(-1); } singularity_priv_drop(); free(sessiondir); free(devdir); return(0); } singularity_message(DEBUG, "Checking configuration file for 'mount dev'\n"); if ( singularity_config_get_bool_char(MOUNT_DEV) > 0 ) { if ( is_dir(joinpath(container_dir, "/dev")) == 0 ) { singularity_priv_escalate(); singularity_message(VERBOSE, "Bind mounting /dev\n"); if ( singularity_mount("/dev", joinpath(container_dir, "/dev"), NULL, MS_BIND|MS_NOSUID|MS_REC, NULL) < 0 ) { singularity_message(ERROR, "Could not bind mount container's /dev: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); } else { singularity_message(WARNING, "Not mounting /dev, container has no bind directory\n"); } return(0); } singularity_message(VERBOSE, "Not mounting /dev inside the container\n"); return(0); } static int bind_dev(char *tmpdir, char *dev) { char *path = joinpath(tmpdir, dev); if ( ( is_chr(dev) == 0 ) || ( is_blk(dev) == 0 ) ) { if ( is_file(path) != 0 ) { int ret; singularity_message(VERBOSE2, "Creating bind point within container: %s\n", dev); singularity_priv_escalate(); ret = fileput(path, ""); singularity_priv_drop(); if ( ret < 0 ) { singularity_message(WARNING, "Can not create %s: %s\n", dev, strerror(errno)); return(-1); } } } else { singularity_message(WARNING, "Not setting up contained device: %s\n", dev); return(-1); } singularity_priv_escalate(); singularity_message(DEBUG, "Mounting device %s at %s\n", dev, path); if ( singularity_mount(dev, path, NULL, MS_BIND, NULL) < 0 ) { singularity_priv_drop(); singularity_message(WARNING, "Could not mount %s: %s\n", dev, strerror(errno)); free(path); return(-1); } singularity_priv_drop(); free(path); return(0); } singularity-2.4.2/src/lib/runtime/mounts/dev/Makefile.am0000644000175000017500000000054513211621077022212 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = dev.c EXTRA_DIST = dev.h singularity-2.4.2/src/lib/runtime/mounts/dev/dev.h0000644000175000017500000000234513211621077021105 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_DEV_H_ #define __SINGULARITY_RUNTIME_MOUNT_DEV_H_ extern int _singularity_runtime_mount_dev(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_DEV_H */ singularity-2.4.2/src/lib/runtime/mounts/userbinds/0000755000175000017500000000000013211621077021372 5ustar mehdimehdisingularity-2.4.2/src/lib/runtime/mounts/userbinds/userbinds.c0000644000175000017500000002226713211621077023545 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/config_parser.h" #include "util/registry.h" #include "util/mount.h" #include "../../runtime.h" int _singularity_runtime_mount_userbinds(void) { char *container_dir = CONTAINER_FINALDIR; char *bind_path_string; singularity_message(DEBUG, "Checking for environment variable 'SINGULARITY_BINDPATH'\n"); if ( ( bind_path_string = singularity_registry_get("BINDPATH") ) != NULL ) { singularity_message(DEBUG, "Checking for 'user bind control' in config\n"); if ( singularity_config_get_bool(USER_BIND_CONTROL) <= 0 ) { singularity_message(WARNING, "Ignoring user bind request: user bind control is disabled by system administrator\n"); return(0); } #ifndef SINGULARITY_NO_NEW_PRIVS singularity_message(WARNING, "Ignoring user bind request: host does not support PR_SET_NO_NEW_PRIVS\n"); return(0); #endif singularity_message(DEBUG, "Parsing SINGULARITY_BINDPATH for user-specified bind mounts.\n"); char *outside_token = NULL; char *inside_token = NULL; char *current = strtok_r(strdup(bind_path_string), ",", &outside_token); free(bind_path_string); while ( current != NULL ) { int read_only = 0; char *source = strtok_r(current, ":", &inside_token); char *dest = strtok_r(NULL, ":", &inside_token); char *opts = strtok_r(NULL, ":", &inside_token); current = strtok_r(NULL, ",", &outside_token); if ( dest == NULL ) { dest = source; } singularity_message(DEBUG, "Found bind: %s -> container:%s\n", source, dest); if ( opts != NULL ) { if ( strcmp(opts, "rw") == 0 ) { // This is the default } else if ( strcmp(opts, "ro") == 0 ) { read_only = 1; } else { singularity_message(WARNING, "Not mounting requested bind point, invalid mount option %s: %s\n", opts, dest); continue; } } singularity_message(DEBUG, "Checking if bind point is already mounted: %s\n", dest); if ( check_mounted(dest) >= 0 ) { singularity_message(WARNING, "Not mounting requested bind point (already mounted in container): %s\n", dest); continue; } if ( ( is_file(source) == 0 ) && ( is_file(joinpath(container_dir, dest)) < 0 ) ) { if ( singularity_registry_get("OVERLAYFS_ENABLED") != NULL ) { char *dir = dirname(strdup(dest)); if ( is_dir(joinpath(container_dir, dir)) < 0 ) { singularity_message(VERBOSE3, "Creating bind directory on overlay file system: %s\n", dest); if ( s_mkpath(joinpath(container_dir, dir), 0755) < 0 ) { singularity_priv_escalate(); singularity_message(VERBOSE3, "Retrying with privileges to create bind directory on overlay file system: %s\n", dest); if ( s_mkpath(joinpath(container_dir, dir), 0755) < 0 ) { singularity_message(ERROR, "Could not create basedir for file bind %s: %s\n", dest, strerror(errno)); continue; } singularity_priv_drop(); } } singularity_priv_escalate(); singularity_message(VERBOSE3, "Creating bind file on overlay file system: %s\n", dest); FILE *tmp = fopen(joinpath(container_dir, dest), "w+"); // Flawfinder: ignore singularity_priv_drop(); if ( tmp == NULL ) { singularity_message(WARNING, "Skipping user bind, could not create bind point %s: %s\n", dest, strerror(errno)); continue; } if ( fclose(tmp) != 0 ) { singularity_message(WARNING, "Skipping user bind, could not close bind point file descriptor %s: %s\n", dest, strerror(errno)); continue; } singularity_message(DEBUG, "Created bind file: %s\n", dest); } else { singularity_message(WARNING, "Skipping user bind, non existent bind point (file) in container: '%s'\n", dest); continue; } } else if ( ( is_dir(source) == 0 ) && ( is_dir(joinpath(container_dir, dest)) < 0 ) ) { if ( singularity_registry_get("OVERLAYFS_ENABLED") != NULL ) { singularity_message(VERBOSE3, "Creating bind directory on overlay file system: %s\n", dest); if ( s_mkpath(joinpath(container_dir, dest), 0755) < 0 ) { singularity_priv_escalate(); singularity_message(VERBOSE3, "Retrying with privileges to create bind directory on overlay file system: %s\n", dest); if ( s_mkpath(joinpath(container_dir, dest), 0755) < 0 ) { singularity_priv_drop(); singularity_message(WARNING, "Skipping user bind, could not create bind point %s: %s\n", dest, strerror(errno)); continue; } singularity_priv_drop(); } } else { singularity_message(WARNING, "Skipping user bind, non existent bind point (directory) in container: '%s'\n", dest); continue; } } singularity_priv_escalate(); singularity_message(VERBOSE, "Binding '%s' to '%s/%s'\n", source, container_dir, dest); if ( singularity_mount(source, joinpath(container_dir, dest), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC, NULL) < 0 ) { singularity_message(ERROR, "There was an error binding the path %s: %s\n", source, strerror(errno)); ABORT(255); } if ( read_only ) { if ( singularity_priv_userns_enabled() == 1 ) { singularity_message(WARNING, "Can not make bind mount read only within the user namespace: %s\n", dest); } else { singularity_message(VERBOSE, "Remounting %s read-only\n", dest); if ( singularity_mount(NULL, joinpath(container_dir, dest), NULL, MS_RDONLY|MS_BIND|MS_NOSUID|MS_NODEV|MS_REC|MS_REMOUNT, NULL) < 0 ) { singularity_message(ERROR, "There was an error write-protecting the path %s: %s\n", source, strerror(errno)); ABORT(255); } if ( access(joinpath(container_dir, dest), W_OK) == 0 || (errno != EROFS && errno != EACCES) ) { // Flawfinder: ignore (precautionary confirmation, not necessary) singularity_message(ERROR, "Failed to write-protect the path %s: %s\n", source, strerror(errno)); ABORT(255); } } } else { if ( singularity_priv_userns_enabled() <= 0 ) { if ( singularity_mount(NULL, joinpath(container_dir, dest), NULL, MS_BIND|MS_NOSUID|MS_NODEV|MS_REC|MS_REMOUNT, NULL) < 0 ) { singularity_message(ERROR, "There was an error remounting the path %s: %s\n", source, strerror(errno)); ABORT(255); } } } singularity_priv_drop(); } singularity_message(DEBUG, "Unsetting environment variable 'SINGULARITY_BINDPATH'\n"); unsetenv("SINGULARITY_BINDPATH"); } else { singularity_message(DEBUG, "No user bind mounts specified.\n"); } return(0); } singularity-2.4.2/src/lib/runtime/mounts/userbinds/userbinds.h0000644000175000017500000000237513211621077023550 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_RUNTIME_MOUNT_USERBINDS_H_ #define __SINGULARITY_RUNTIME_MOUNT_USERBINDS_H_ extern int _singularity_runtime_mount_userbinds(void); #endif /* __SINGULARITY_RUNTIME_MOUNT_USERBINDS_H */ singularity-2.4.2/src/lib/runtime/mounts/userbinds/Makefile.am0000644000175000017500000000056113211621077023430 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = userbinds.c EXTRA_DIST = userbinds.h singularity-2.4.2/src/lib/image/0000755000175000017500000000000013211621077015446 5ustar mehdimehdisingularity-2.4.2/src/lib/image/squashfs/0000755000175000017500000000000013211621077017303 5ustar mehdimehdisingularity-2.4.2/src/lib/image/squashfs/mount.c0000644000175000017500000000422613211621077020615 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/mount.h" #include "../image.h" #include "../bind.h" int _singularity_image_squashfs_mount(struct image_object *image, char *mount_point) { char *loop_dev = NULL; if ( ( loop_dev = singularity_image_bind(image) ) == NULL ) { singularity_message(ERROR, "Could not obtain the image loop device\n"); ABORT(255); } singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting squashfs image: %s -> %s\n", loop_dev, mount_point); if ( singularity_mount(loop_dev, mount_point, "squashfs", MS_NOSUID|MS_RDONLY|MS_NODEV, "errors=remount-ro") < 0 ) { singularity_message(ERROR, "Failed to mount squashfs image in (read only): %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); return(0); } singularity-2.4.2/src/lib/image/squashfs/include.h0000644000175000017500000000254313211621077021103 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_IMAGE_SQUASHFS_H_ #define __SINGULARITY_IMAGE_SQUASHFS_H_ extern int _singularity_image_squashfs_init(struct image_object *image, int open_flags); extern int _singularity_image_squashfs_mount(struct image_object *image, char *mount_point); #endif /* __SINGULARITY_IMAGE_SQUASHFS_H_ */ singularity-2.4.2/src/lib/image/squashfs/init.c0000644000175000017500000000574013211621077020420 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include "util/message.h" #include "util/util.h" #include "util/file.h" #include "../image.h" int _singularity_image_squashfs_init(struct image_object *image, int open_flags) { int image_fd; int ret; FILE *image_fp; static char buf[1024]; char *p; singularity_message(DEBUG, "Checking if writable image requested\n"); if ( open_flags == O_RDWR ) { errno = EROFS; return(-1); } singularity_message(DEBUG, "Opening file descriptor to image: %s\n", image->path); if ( ( image_fd = open(image->path, open_flags, 0755) ) < 0 ) { singularity_message(ERROR, "Could not open image %s: %s\n", image->path, strerror(errno)); ABORT(255); } if ( ( image_fp = fdopen(dup(image_fd), "r") ) == NULL ) { singularity_message(ERROR, "Could not associate file pointer from file descriptor on image %s: %s\n", image->path, strerror(errno)); ABORT(255); } singularity_message(VERBOSE3, "Checking that file pointer is a Singularity image\n"); rewind(image_fp); // Get the first line from the config ret = fread(buf, 1, sizeof(buf), image_fp); fclose(image_fp); if ( ret != sizeof(buf) ) { singularity_message(DEBUG, "Could not read the top of the image\n"); return(-1); } singularity_message(DEBUG, "Checking for magic in the top of the file\n"); /* if LAUNCH_STRING is present, figure out squashfs magic offset */ p = strstr(buf, "hsqs"); if ( p != NULL ) { singularity_message(VERBOSE2, "File is a valid SquashFS image\n"); image->offset = p - buf; } else { close(image_fd); singularity_message(VERBOSE, "File is not a valid SquashFS image\n"); return(-1); } image->fd = image_fd; return(0); } singularity-2.4.2/src/lib/image/squashfs/Makefile.am0000644000175000017500000000055413211621077021343 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = init.c mount.c EXTRA_DIST = *.h singularity-2.4.2/src/lib/image/ext3/0000755000175000017500000000000013211621077016331 5ustar mehdimehdisingularity-2.4.2/src/lib/image/ext3/mount.c0000644000175000017500000000463413211621077017646 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/registry.h" #include "util/mount.h" #include "../image.h" #include "../bind.h" int _singularity_image_ext3_mount(struct image_object *image, char *mount_point) { int opts = MS_NOSUID; char *loop_dev; if ( ( loop_dev = singularity_image_bind(image) ) == NULL ) { singularity_message(ERROR, "Could not obtain the image loop device\n"); ABORT(255); } if ( getuid() != 0 ) { singularity_message(DEBUG, "Adding MS_NODEV to mount options\n"); opts |= MS_NODEV; } if ( image->writable <= 0 ) { singularity_message(DEBUG, "Adding MS_RDONLY to mount options\n"); opts |= MS_RDONLY; } singularity_priv_escalate(); singularity_message(VERBOSE, "Mounting '%s' to: '%s'\n", loop_dev, mount_point); if ( singularity_mount(loop_dev, mount_point, "ext3", opts, "errors=remount-ro") < 0 ) { singularity_message(ERROR, "Failed to mount ext3 image: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); return(0); } singularity-2.4.2/src/lib/image/ext3/include.h0000644000175000017500000000251713211621077020132 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_IMAGE_EXT3_H_ #define __SINGULARITY_IMAGE_EXT3_H_ extern int _singularity_image_ext3_init(struct image_object *image, int open_flags); extern int _singularity_image_ext3_mount(struct image_object *image, char *mount_point); #endif /* __SINGULARITY_IMAGE_EXT3_H_ */ singularity-2.4.2/src/lib/image/ext3/init.c0000644000175000017500000001015713211621077017444 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include "util/message.h" #include "util/util.h" #include "util/file.h" #include "util/registry.h" #include "../image.h" #define BUFFER_SIZE (1024*1024) #define MAX_LINE_LEN 2048 #define EXTFS_MAGIC "\123\357" #define COMPAT_HASJOURNAL 0x4 #define INCOMPAT_FILETYPE 0x2 #define INCOMPAT_RECOVER 0x4 #define INCOMPAT_METABG 0x10 #define ROCOMPAT_SPARSESUPER 0x1 #define ROCOMPAT_LARGEFILE 0x2 #define ROCOMPAT_BTREEDIR 0x4 struct extfs_info { unsigned char magic[2]; uint16_t state; uint32_t dummy[8]; uint32_t feat_compat; uint32_t feat_incompat; uint32_t feat_rocompat; }; int _singularity_image_ext3_init(struct image_object *image, int open_flags) { int image_fd; int ret; int magicoff = 1080; FILE *image_fp; static char buf[2048]; struct extfs_info *einfo; singularity_message(DEBUG, "Opening file descriptor to image: %s\n", image->path); if ( ( image_fd = open(image->path, open_flags, 0755) ) < 0 ) { singularity_message(ERROR, "Could not open image %s: %s\n", image->path, strerror(errno)); ABORT(255); } if ( ( image_fp = fdopen(dup(image_fd), "r") ) == NULL ) { singularity_message(ERROR, "Could not associate file pointer from file descriptor on image %s: %s\n", image->path, strerror(errno)); ABORT(255); } singularity_message(VERBOSE3, "Checking that file pointer is a Singularity image\n"); rewind(image_fp); // Get the first line from the config ret = fread(buf, 1, sizeof(buf), image_fp); fclose(image_fp); if ( ret != sizeof(buf) ) { singularity_message(DEBUG, "Could not read the top of the image\n"); return(-1); } /* if LAUNCH_STRING is present, figure out EXTFS magic offset */ if ( strstr(buf, "singularity") != NULL ) { magicoff += strlen(buf); image->offset = strlen(buf); } einfo = (struct extfs_info *)&buf[magicoff]; if ( memcmp(einfo->magic, EXTFS_MAGIC, 2 ) != 0 ) { close(image_fd); singularity_message(VERBOSE, "File is not a valid EXT3 image\n"); return(-1); } /* Check for features supported by EXT3 */ if ( !(einfo->feat_compat & COMPAT_HASJOURNAL) ) { close(image_fd); singularity_message(VERBOSE, "File is not a valid EXT3 image\n"); return(-1); } /* check for unsupported incompat ext3 features */ if ( einfo->feat_incompat & ~(INCOMPAT_FILETYPE|INCOMPAT_RECOVER|INCOMPAT_METABG) ) { close(image_fd); singularity_message(VERBOSE, "File is not a valid EXT3 image\n"); return(-1); } /* check for unsupported rocompat ext3 features */ if ( einfo->feat_rocompat & ~(ROCOMPAT_SPARSESUPER|ROCOMPAT_LARGEFILE|ROCOMPAT_BTREEDIR) ) { close(image_fd); singularity_message(VERBOSE, "File is not a valid EXT3 image\n"); return(-1); } image->fd = image_fd; return(0); } singularity-2.4.2/src/lib/image/ext3/Makefile.am0000644000175000017500000000055413211621077020371 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = init.c mount.c EXTRA_DIST = *.h singularity-2.4.2/src/lib/image/image.c0000644000175000017500000002303213211621077016674 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/registry.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "./image.h" #include "./bind.h" #include "./squashfs/include.h" #include "./dir/include.h" #include "./ext3/include.h" struct image_object singularity_image_init(char *path, int open_flags) { struct image_object image; char *real_path; if ( path == NULL ) { singularity_message(ERROR, "No container image path defined\n"); ABORT(255); } real_path = realpath(path, NULL); // Flawfinder: ignore if ( real_path == NULL ) { singularity_message(ERROR, "Image path doesn't exists\n"); ABORT(255); } image.path = real_path; image.name = basename(strdup(real_path)); image.type = -1; image.fd = -1; image.loopdev = NULL; image.offset = 0; if ( open_flags & ( O_RDWR | O_WRONLY ) ) { image.writable = 1; } else { image.writable = 0; } singularity_message(DEBUG, "Calling image_init for each file system module\n"); if ( _singularity_image_dir_init(&image, open_flags) == 0 ) { singularity_message(DEBUG, "got image_init type for directory\n"); image.type = DIRECTORY; if ( ( singularity_config_get_bool(ALLOW_CONTAINER_DIR) <= 0 ) && ( singularity_priv_getuid() != 0 ) ) { singularity_message(ERROR, "Configuration disallows users from running directory based containers\n"); ABORT(255); } } else if ( _singularity_image_squashfs_init(&image, open_flags) == 0 ) { singularity_message(DEBUG, "got image_init type for squashfs\n"); image.type = SQUASHFS; if ( ( singularity_config_get_bool(ALLOW_CONTAINER_SQUASHFS) <= 0 ) && ( singularity_priv_getuid() != 0 ) ) { singularity_message(ERROR, "Configuration disallows users from running squashFS based containers\n"); ABORT(255); } } else if ( _singularity_image_ext3_init(&image, open_flags) == 0 ) { singularity_message(DEBUG, "got image_init type for ext3\n"); image.type = EXT3; if ( ( singularity_config_get_bool(ALLOW_CONTAINER_EXTFS) <= 0 ) && ( singularity_priv_getuid() != 0 ) ) { singularity_message(ERROR, "Configuration disallows users from running extFS based containers\n"); ABORT(255); } } else { if ( errno == EROFS ) { singularity_message(ERROR, "Unable to open squashfs image in read-write mode: %s\n", strerror(errno)); } else { singularity_message(ERROR, "Unknown image format/type: %s\n", path); } ABORT(255); } if ( fcntl(image.fd, F_SETFD, FD_CLOEXEC) != 0 ) { singularity_message(ERROR, "Failed to set CLOEXEC on image file descriptor\n"); ABORT(255); } if ( ( singularity_suid_enabled() >= 0 ) && ( singularity_priv_getuid() != 0 ) ) { singularity_limit_container_paths(&image); singularity_limit_container_owners(&image); } return(image); } int singularity_image_fd(struct image_object *image) { return(image->fd); } char *singularity_image_loopdev(struct image_object *image) { return(image->loopdev); } char *singularity_image_name(struct image_object *image) { return(image->name); } char *singularity_image_path(struct image_object *image) { return(image->path); } int singularity_image_offset(struct image_object *image) { return(image->offset); } int singularity_image_type(struct image_object *image) { return(image->type); } int singularity_image_writable(struct image_object *image) { return(image->writable); } int singularity_image_mount(struct image_object *image, char *mount_point) { if ( singularity_registry_get("DAEMON_JOIN") ) { singularity_message(ERROR, "Internal Error - This function should not be called when joining an instance\n"); } singularity_message(DEBUG, "Figuring out which mount module to use...\n"); if ( image->type == SQUASHFS ) { singularity_message(DEBUG, "Calling squashfs_mount\n"); return(_singularity_image_squashfs_mount(image, mount_point)); } else if ( image->type == DIRECTORY ) { singularity_message(DEBUG, "Calling dir_mount\n"); return(_singularity_image_dir_mount(image, mount_point)); } else if ( image->type == EXT3 ) { singularity_message(DEBUG, "Calling ext3_mount\n"); return(_singularity_image_ext3_mount(image, mount_point)); } else { singularity_message(ERROR, "Can not mount file system of unknown type\n"); ABORT(255); } return(-1); } void singularity_limit_container_owners(struct image_object *image) { const char *limit_container_owners = singularity_config_get_value(LIMIT_CONTAINER_OWNERS); if ( strcmp(limit_container_owners, "NULL") != 0 ) { struct stat image_stat; char *user_token = NULL; char *current = strtok_r(strdup(limit_container_owners), ",", &user_token); chomp(current); singularity_message(DEBUG, "Limiting container access to allowed users\n"); if ( fstat(image->fd, &image_stat) != 0 ) { singularity_message(ERROR, "Could not fstat() image file descriptor (%d): %s\n", image->fd, strerror(errno)); ABORT(255); } while (1) { struct passwd *user_pw; if ( current[0] == '\0' ) { singularity_message(DEBUG, "Skipping blank user limit entry\n"); } else { singularity_message(DEBUG, "Checking user: '%s'\n", current); if ( ( user_pw = getpwnam(current) ) != NULL ) { if ( user_pw->pw_uid == image_stat.st_uid ) { singularity_message(DEBUG, "Singularity image is owned by required user: %s\n", current); break; } } } current = strtok_r(NULL, ",", &user_token); chomp(current); if ( current == NULL ) { singularity_message(ERROR, "Singularity image is not owned by required user(s)\n"); ABORT(255); } } } } void singularity_limit_container_paths(struct image_object *image) { const char *limit_container_paths = singularity_config_get_value(LIMIT_CONTAINER_PATHS); if ( strcmp(limit_container_paths, "NULL") != 0 ) { char image_path[PATH_MAX]; char *path_token = NULL; char *fd_path = NULL; fd_path = (char *) malloc(PATH_MAX+21); singularity_message(DEBUG, "Obtaining full path to image file descriptor (%d)\n", image->fd); if ( snprintf(fd_path, PATH_MAX+20, "/proc/self/fd/%d", image->fd) > 0 ) { singularity_message(DEBUG, "Checking image path from file descriptor source: %s\n", fd_path); } else { singularity_message(ERROR, "Internal - Failed allocating memory for fd_path string: %s\n", strerror(errno)); ABORT(255); } if ( readlink(fd_path, image_path, PATH_MAX-1) > 0 ) { // Flawfinder: ignore (TOCTOU not an issue within /proc) char *current = strtok_r(strdup(limit_container_paths), ",", &path_token); chomp(current); while (1) { if ( current[0] == '\0' ) { singularity_message(DEBUG, "Skipping blank path limit entry\n"); } else { singularity_message(DEBUG, "Checking image path: '%s'\n", current); if ( strncmp(image_path, current, strlength(current, PATH_MAX)) == 0 ) { singularity_message(VERBOSE, "Singularity image is in an allowed path: %s\n", current); break; } current = strtok_r(NULL, ",", &path_token); chomp(current); if ( current == NULL ) { singularity_message(ERROR, "Singularity image is not in an allowed configured path\n"); ABORT(255); } } } } else { singularity_message(ERROR, "Could not obtain the full system path of the image file: %s\n", strerror(errno)); ABORT(255); } } } singularity-2.4.2/src/lib/image/bind.h0000644000175000017500000000234013211621077016532 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_IMAGE_BIND_H_ #define __SINGULARITY_IMAGE_BIND_H_ extern char *singularity_image_bind(struct image_object *image); #endif /* __SINGULARITY_IMAGE_BIND_H */ singularity-2.4.2/src/lib/image/dir/0000755000175000017500000000000013211621077016224 5ustar mehdimehdisingularity-2.4.2/src/lib/image/dir/mount.c0000644000175000017500000000411113211621077017527 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/mount.h" #include "../image.h" int _singularity_image_dir_mount(struct image_object *image, char *mount_point) { if ( strcmp(image->path, "/") == 0 ) { singularity_message(ERROR, "Naughty naughty naughty...\n"); ABORT(255); } singularity_priv_escalate(); singularity_message(DEBUG, "Mounting container directory %s->%s\n", image->path, mount_point); if ( singularity_mount(image->path, mount_point, NULL, MS_BIND|MS_NOSUID|MS_REC|MS_NODEV, NULL) < 0 ) { singularity_message(ERROR, "Could not mount container directory %s->%s: %s\n", image->path, mount_point, strerror(errno)); return 1; } singularity_priv_drop(); return(0); } singularity-2.4.2/src/lib/image/dir/include.h0000644000175000017500000000251213211621077020020 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_IMAGE_DIR_H_ #define __SINGULARITY_IMAGE_DIR_H_ extern int _singularity_image_dir_init(struct image_object *image, int open_flags); extern int _singularity_image_dir_mount(struct image_object *image, char *mount_point); #endif /* __SINGULARITY_IMAGE_DIR_H_ */ singularity-2.4.2/src/lib/image/dir/init.c0000644000175000017500000000412513211621077017335 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include "util/message.h" #include "util/util.h" #include "util/file.h" #include "../image.h" int _singularity_image_dir_init(struct image_object *image, int open_flags) { int fd = -1; struct stat st; singularity_message(DEBUG, "Opening file descriptor to directory: %s\n", image->path); if ( ( fd = open(image->path, O_RDONLY, 0755) ) < 0 ) { singularity_message(ERROR, "Could not open image %s: %s\n", image->path, strerror(errno)); ABORT(255); } if ( fstat(fd, &st) != 0 ) { singularity_message(ERROR, "Could not stat file descriptor: %s\n", strerror(errno)); ABORT(255); } if ( S_ISDIR(st.st_mode) == 0 ) { singularity_message(DEBUG, "This is not a directory based image\n"); close(fd); return(-1); } // If we got here, we assume things are a directory image->fd = fd; return(0); } singularity-2.4.2/src/lib/image/dir/Makefile.am0000644000175000017500000000055413211621077020264 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = init.c mount.c EXTRA_DIST = *.h singularity-2.4.2/src/lib/image/image.h0000644000175000017500000000456113211621077016707 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_IMAGE_H_ #define __SINGULARITY_IMAGE_H_ #define SQUASHFS 1 #define EXT3 2 #define DIRECTORY 3 struct image_object { char *path; char *name; char *loopdev; int offset; int fd; int type; int writable; }; extern struct image_object singularity_image_init(char *path, int open_flags); int singularity_image_fd(struct image_object *object); char *singularity_image_loopdev(struct image_object *object); char *singularity_image_name(struct image_object *object); char *singularity_image_path(struct image_object *object); char *singularity_image_bind(struct image_object *object); int singularity_image_writable(struct image_object *object); int singularity_image_type(struct image_object *object); extern int singularity_image_mount(struct image_object *image, char *mount_point); void singularity_limit_container_paths(struct image_object *object); void singularity_limit_container_owners(struct image_object *object); #define LAUNCH_STRING "#!/usr/bin/env run-singularity\n" #endif singularity-2.4.2/src/lib/image/bind.c0000644000175000017500000001223313211621077016527 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "lib/image/image.h" #include "util/util.h" #include "util/file.h" #include "util/message.h" #include "util/privilege.h" #include "util/registry.h" #include "util/config_parser.h" #include "./image.h" #ifndef LO_FLAGS_AUTOCLEAR #define LO_FLAGS_AUTOCLEAR 4 #endif char *singularity_image_bind(struct image_object *image) { struct loop_info64 lo64 = {0}; long int max_loop_devs; const char *max_loop_devs_string = singularity_config_get_value(MAX_LOOP_DEVS); char *loop_dev = NULL; int loop_fd = -1; int loop_mount_opts; int i; singularity_message(DEBUG, "Entered singularity_image_bind()\n"); singularity_message(DEBUG, "Converting max_loop_devs_string to int: '%s'\n", max_loop_devs_string); if ( str2int(max_loop_devs_string, &max_loop_devs) != 0 ) { singularity_message(ERROR, "Failed converting config option '%s = %s' to integer\n", MAX_LOOP_DEVS, max_loop_devs_string); ABORT(255); } singularity_message(DEBUG, "Converted max_loop_devs_string to int: '%s' -> %ld\n", max_loop_devs_string, max_loop_devs); singularity_message(DEBUG, "Checking if this image has been properly opened\n"); if ( image->fd <= 0 ) { singularity_message(ERROR, "Image file descriptor not assigned!\n"); ABORT(255); } if ( image->writable <= 0 ) { loop_mount_opts = O_RDONLY; } else { loop_mount_opts = O_RDWR; } singularity_priv_escalate(); singularity_message(DEBUG, "Finding next available loop device...\n"); for( i=0; i < max_loop_devs; i++ ) { char *test_loopdev = strjoin("/dev/loop", int2str(i)); if ( is_blk(test_loopdev) < 0 ) { singularity_message(DEBUG, "Instantiating loop device: %s\n", test_loopdev); if ( mknod(test_loopdev, S_IFBLK | 0644, makedev(7, i)) < 0 ) { if ( errno != EEXIST ) { singularity_message(ERROR, "Could not create %s: %s\n", test_loopdev, strerror(errno)); ABORT(255); } } } if ( ( loop_fd = open(test_loopdev, loop_mount_opts) ) < 0 ) { // Flawfinder: ignore singularity_message(VERBOSE, "Could not open loop device %s: %s\n", test_loopdev, strerror(errno)); continue; } if ( ioctl(loop_fd, LOOP_SET_FD, image->fd)== 0 ) { loop_dev = strdup(test_loopdev); break; } else { if ( errno == EBUSY ) { close(loop_fd); continue; } else { singularity_message(WARNING, "Could not associate image to loop %s: %s\n", test_loopdev, strerror(errno)); close(loop_fd); continue; } } } singularity_priv_drop(); if ( loop_dev == NULL ) { singularity_message(ERROR, "No more available loop devices, try increasing '%s' in singularity.conf\n", MAX_LOOP_DEVS); ABORT(255); } singularity_message(VERBOSE, "Found available loop device: %s\n", loop_dev); singularity_message(DEBUG, "Setting LO_FLAGS_AUTOCLEAR\n"); lo64.lo_flags = LO_FLAGS_AUTOCLEAR; singularity_message(DEBUG, "Using image offset: %d\n", image->offset); lo64.lo_offset = image->offset; singularity_priv_escalate(); singularity_message(DEBUG, "Setting loop device flags\n"); if ( ioctl(loop_fd, LOOP_SET_STATUS64, &lo64) < 0 ) { singularity_message(ERROR, "Failed to set loop flags on loop device: %s\n", strerror(errno)); (void)ioctl(loop_fd, LOOP_CLR_FD, 0); ABORT(255); } singularity_priv_drop(); singularity_message(VERBOSE, "Using loop device: %s\n", loop_dev); if ( fcntl(loop_fd, F_SETFD, FD_CLOEXEC) != 0 ) { singularity_message(ERROR, "Could not set file descriptor flag to close on exit: %s\n", strerror(errno)); ABORT(255); } return(loop_dev); } singularity-2.4.2/src/lib/image/Makefile.am0000644000175000017500000000323313211621077017503 0ustar mehdimehdi#SUBDIRS = bind create check expand offset open mount ext3 dir squashfs SUBDIRS = ext3 dir squashfs MAINTAINERCLEANFILES = Makefile.in config.h config.h.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie -fPIC AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) distlibdir = $(libdir)/singularity distincludedir = $(includedir)/singularity noinst_LTLIBRARIES = libimage.la #libimage_la_LIBADD = bind/libinternal.la create/libinternal.la check/libinternal.la expand/libinternal.la mount/libinternal.la offset/libinternal.la open/libinternal.la ext3/libinternal.la dir/libinternal.la squashfs/libinternal.la libimage_la_LIBADD = ext3/libinternal.la dir/libinternal.la squashfs/libinternal.la libimage_la_SOURCES = image.c bind.c ../../util/registry.c ../../util/message.c ../../util/config_parser.c ../../util/privilege.c ../../util/util.c ../../util/file.c ../../util/suid.c ../../util/mount.c libimage_la_CFLAGS = $(AM_CFLAGS) # This fixes duplicate sources in library and progs distinclude_HEADERS = image.h bind.h distlib_LTLIBRARIES = libsingularity-image.la libsingularity_image_la_SOURCES = libsingularity_image_la_LIBADD = $(noinst_LTLIBRARIES) libsingularity_image_la_LDFLAGS = -version-info 1:0:0 libsingularity_image_la_CFLAGS = $(AM_CFLAGS) # These are kludges so they don't remove the $(DEPDIR) in ../../util/ otherwise # the clean will fail when other Makefile tries to remove those directories distclean: distclean-recursive -rm ./$(DEPDIR) -rm -f Makefile maintainer-clean: maintainer-clean-recursive -rm ./$(DEPDIR) -rm -f Makefile singularity-2.4.2/src/lib/Makefile.am0000644000175000017500000000236113211621077016422 0ustar mehdimehdiSUBDIRS = runtime image #SUBDIRS = ns rootfs action mount file image MAINTAINERCLEANFILES = Makefile.in config.h config.h.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie -fPIC AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) distlibdir = $(libdir)/singularity distincludedir = $(includedir)/singularity #noinst_LTLIBRARIES = libsingularity_internal.la #libsingularity_internal_la_LIBADD = ns/libns.la rootfs/librootfs.la action/libaction.la mount/libmount.la file/libfile.la image/libimage.la #libsingularity_internal_la_SOURCES = singularity.c privilege.c message.c util.c file.c sessiondir.c config_parser.c fork.c loop-control.c #libsingularity_internal_la_CFLAGS = $(AM_CFLAGS) # This fixes duplicate sources in library and progs # #distinclude_HEADERS = singularity.h #distlib_LTLIBRARIES = libsingularity.la # #libsingularity_la_SOURCES = #libsingularity_la_LIBADD = libsingularity_internal.la #libsingularity_la_LDFLAGS = -version-info 1:0:0 #libsingularity_la_CFLAGS = $(AM_CFLAGS) # #EXTRA_DIST = config_parser.h fork.h loop-control.h message.h privilege.h sessiondir.h singularity.h config_defaults.h EXTRA_DIST = singularity-2.4.2/src/image-type.c0000644000175000017500000000556013211621077016031 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/config_parser.h" #include "util/message.h" #include "lib/image/image.h" enum { BUFLEN = 512 }; static unsigned char gzmagic[2] = { 0x1f, 0x8b }; static unsigned char bzmagic[3] = { 0x42, 0x5a, 0x68 }; static unsigned char tarmagic[5] = { 0x75, 0x73, 0x74, 0x61, 0x72 }; char * check_compression_formats(char *fname) { FILE *fp; size_t ret; static unsigned char buf[BUFLEN]; if ( fname == NULL ) return NULL; fp = fopen(fname, "r"); if ( fp == NULL ) return NULL; ret = fread(buf, 1, BUFLEN, fp); fclose(fp); if ( ret >= 2 && memcmp(buf, gzmagic, 2) == 0 ) return "GZIP"; else if ( ret >= 3 && memcmp(buf, bzmagic, 3) == 0 ) return "BZIP2"; else if ( ret >= 263 && memcmp(&buf[257], tarmagic, 5) == 0 ) return "TAR"; return NULL; } int main(int argc, char **argv) { struct image_object image; char *compfmtstr; if ( (compfmtstr = check_compression_formats(argv[1])) != NULL ) { printf("%s\n", compfmtstr); return(0); } singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_message(VERBOSE3, "Instantiating read only container image object\n"); image = singularity_image_init(argv[1], O_RDONLY); if ( singularity_image_type(&image) == SQUASHFS ) { printf("SQUASHFS\n"); } else if ( singularity_image_type(&image) == EXT3 ) { printf("EXT3\n"); } else if ( singularity_image_type(&image) == DIRECTORY ) { printf("DIRECTORY\n"); } else { singularity_message(ERROR, "Unknown image type\n"); return(1); } return(0); } singularity-2.4.2/src/export.c0000644000175000017500000001004113211621077015277 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "util/fork.h" #include "util/sessiondir.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif int main(int argc, char **argv) { int retval = 0; char *tar_cmd[7]; struct image_object image; struct image_object image_test; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_priv_init(); singularity_suid_init(argv); singularity_registry_init(); singularity_priv_drop(); if ( argc == 2 ) { singularity_registry_set("IMAGE", argv[1]); } image = singularity_image_init(singularity_registry_get("IMAGE")); // singularity_image_open(&image, O_RDONLY); // singularity_image_check(&image); if ( image.type != EXT3 ) { singularity_message(ERROR, "Export is only allowed on Singularity image files\n"); ABORT(255); } singularity_runtime_ns(SR_NS_MNT); // singularity_image_bind(&image); // // if ( image.loopdev == NULL ) { // singularity_message(ERROR, "Bind failed to connect to image!\n"); // ABORT(255); // } singularity_image_mount(&image, singularity_runtime_rootfs(NULL)); // Check to make sure the image hasn't been swapped out by a race image_test = singularity_image_init(singularity_registry_get("IMAGE")); // singularity_image_open(&image_test, O_RDONLY); // singularity_image_check(&image_test); if ( image_test.type != EXT3 ) { singularity_message(ERROR, "Import is only allowed on Singularity image files\n"); ABORT(255); } if ( is_exec("/usr/bin/tar") == 0 ) { tar_cmd[0] = strdup("/usr/bin/tar"); } else if ( is_exec("/bin/tar") == 0 ) { tar_cmd[0] = strdup("/bin/tar"); } else { singularity_message(ERROR, "Could not locate the system version of 'tar'\n"); ABORT(255); } tar_cmd[1] = strdup("--exclude=*//*"); tar_cmd[2] = strdup("--exclude=*../*"); tar_cmd[3] = strdup("-cf"); tar_cmd[4] = strdup("-"); tar_cmd[5] = strdup("."); tar_cmd[6] = NULL; if ( chdir(singularity_runtime_rootfs(NULL)) != 0 ) { singularity_message(ERROR, "Could not change to working directory: %s\n", singularity_runtime_rootfs(NULL)); ABORT(255); } singularity_message(DEBUG, "Cleaning environment\n"); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing the environment\n"); ABORT(255); } singularity_priv_escalate(); retval = singularity_fork_exec(0, tar_cmd); singularity_priv_drop(); if ( retval != 0 ) { singularity_message(ERROR, "Tar did not return successful\n"); } return(retval); } singularity-2.4.2/src/simage.c0000644000175000017500000001134713211621077015235 0ustar mehdimehdi/* * Copyright (c) 2016, Michael W. Bauer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "lib/singularity.h" #include "util/file.h" #include "util/util.h" #include "util/config_parser.h" #ifndef SYSCONFDIR #define SYSCONFDIR "/etc" #endif int main(int argc_in, char ** argv_in) { // Note: SonarQube complains when we change the value of parameters, even // in obviously-OK cases like this one... char **argv = argv_in; int argc = argc_in; long int size = 1024; if ( argv[1] == NULL ) { fprintf(stderr, "USAGE: %s [bootstrap/mount/bind/create/expand] [args]\n", argv[0]); return(1); } /* Open the config file for parsing */ singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); /* * Even though we don't have SUID for this binary, singularity_priv_init and * singularity_priv_escalate can be used to ensure that the calling user is root. */ singularity_priv_init(); singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); /* Loop until we've gone through argv and returned */ while ( 1 ) { singularity_message(DEBUG, "Running %s %s workflow\n", argv[0], argv[1]); singularity_priv_escalate(); if ( argv[1] == NULL ) { singularity_message(DEBUG, "Finished running simage command and returning\n"); return(0); } /* Run image mount workflow */ else if ( strcmp(argv[1], "mount") == 0 ) { singularity_ns_mnt_unshare(); if ( singularity_image_mount(argc - 1, &argv[1]) != 0 ) { singularity_priv_drop_perm(); return(1); } } /* Run image bind workflow */ else if ( strcmp(argv[1], "bind") == 0 ) { if ( singularity_image_bind(argc - 1, &argv[1]) != 0 ) { singularity_priv_drop_perm(); return(1); } } /* Run image create workflow */ else if ( strcmp(argv[1], "create") == 0 ) { if ( argv[2] == NULL ) { fprintf(stderr, "USAGE: %s create [singularity container image] [size in MiB]\n", argv[0]); } if ( argv[3] != NULL ) { size = ( strtol(argv[3], (char **)NULL, 10) ); } return(singularity_image_create(argv[2], size)); } /* Run image expand workflow */ else if ( strcmp(argv[1], "expand") == 0 ) { if ( argv[2] == NULL ) { fprintf(stderr, "USAGE: %s expand [singularity container image] [size in MiB]\n", argv[0]); } if ( argv[3] != NULL ) { size = ( strtol(argv[3], (char **)NULL, 10) ); } return(singularity_image_expand(argv[2], size)); } /* Run image bootstrap workflow */ else if ( strcmp(argv[1], "bootstrap") == 0 ) { if ( (argv[2] == NULL) || (argv[3] == NULL) ) { fprintf(stderr, "USAGE: %s bootstrap [singularity container image] [bootstrap definition file]\n", argv[0]); return(1); } return(singularity_bootstrap(argv[2], argv[3])); } /* If there is a trailing arg containing a script, attempt to execute it */ else { singularity_priv_drop_perm(); return(singularity_fork_exec(&argv[1])); } argv++; argc--; singularity_priv_drop(); } } singularity-2.4.2/src/prepheader.c0000644000175000017500000000363413211621077016107 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * This software is licensed under a 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * */ #include #include #include #include #include #include #include #include #include "lib/image/image.h" /* prepend string "str" into file "filename" */ int prepend(char *str, char *filename) { int fd; char *map; struct stat st; if(filename == NULL){ fprintf(stderr, "Error invalid filename\n"); return -1; } fd = open(filename, O_RDWR); if(fd < 0){ perror("Error while opening file"); return -1; } if(lseek(fd, 0, SEEK_END) < 0){ perror("Error while seeking to end of file"); close(fd); return -1; } if(write(fd, str, strlen(str)) != (ssize_t)strlen(str)){ perror("Error writing past end of file"); close(fd); return -1; } if(fstat(fd, &st) < 0){ perror("Error calling fstat()"); close(fd); return -1; } map = mmap(NULL, st.st_size, PROT_WRITE, MAP_SHARED, fd, 0); if(map == MAP_FAILED){ perror("Error mapping file"); close(fd); return -1; } /* the heart of the program is these two lines really */ memmove(map+strlen(str), map, st.st_size-strlen(str)); strncpy(map, str, strlen(str)); if(munmap(map, st.st_size) < 0){ perror("Error tearing down map, file corrupted -- dont use"); close(fd); return -1; } if(fchmod(fd, st.st_mode | S_IXUSR | S_IXGRP | S_IXOTH) < 0){ perror("Error trying to change mode +x\n"); close(fd); return 0; } if(close(fd) < 0){ perror("Error closing file -- weird"); return -1; } return 0; } int main(int argc, char *argv[]) { if(argc != 2){ fprintf(stderr, "usage: %s PART_IMAGE_FILE\n", argv[0]); return -1; } prepend(LAUNCH_STRING, argv[1]); return 0; } singularity-2.4.2/src/start.c0000644000175000017500000001535413211621077015127 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * This software is licensed under a 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "util/fork.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "util/sessiondir.h" #include "util/cleanupd.h" #include "util/daemon.h" #include "util/signal.h" #include "./action-lib/include.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif #define CHILD_FAILED 200 int started = 0; int main(int argc, char **argv) { int i, daemon_fd, cleanupd_fd; struct tempfile *stdout_log, *stderr_log, *singularity_debug; struct image_object image; pid_t child; siginfo_t siginfo; struct stat filestat; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_priv_init(); singularity_suid_init(argv); singularity_registry_init(); singularity_priv_userns(); singularity_priv_drop(); singularity_runtime_autofs(); singularity_registry_set("UNSHARE_PID", "1"); singularity_registry_set("UNSHARE_IPC", "1"); singularity_cleanupd(); if ( singularity_registry_get("WRITABLE") != NULL ) { singularity_message(VERBOSE3, "Instantiating writable container image object\n"); image = singularity_image_init(singularity_registry_get("IMAGE"), O_RDWR); } else { singularity_message(VERBOSE3, "Instantiating read only container image object\n"); image = singularity_image_init(singularity_registry_get("IMAGE"), O_RDONLY); } singularity_runtime_ns(SR_NS_ALL); singularity_sessiondir(); singularity_image_mount(&image, CONTAINER_MOUNTDIR); action_ready(); singularity_runtime_overlayfs(); singularity_runtime_mounts(); singularity_runtime_files(); /* After this point, we are running as PID 1 inside PID NS */ singularity_message(DEBUG, "Preparing sinit daemon\n"); singularity_registry_set("ROOTFS", CONTAINER_FINALDIR); singularity_daemon_init(); singularity_message(DEBUG, "We are ready to recieve jobs, sending signal_go_ahead to parent\n"); singularity_runtime_enter(); singularity_priv_drop_perm(); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing the environment\n"); ABORT(255); } singularity_install_signal_handler(); daemon_fd = atoi(singularity_registry_get("DAEMON_FD")); cleanupd_fd = atoi(singularity_registry_get("CLEANUPD_FD")); /* Close all open fd's that may be present besides daemon info file fd */ singularity_message(DEBUG, "Closing open fd's\n"); for( i = sysconf(_SC_OPEN_MAX); i > 2; i-- ) { if ( i != daemon_fd && i != cleanupd_fd ) { if ( fstat(i, &filestat) == 0 ) { if ( S_ISFIFO(filestat.st_mode) != 0 ) { continue; } } close(i); } } singularity_debug = make_logfile("singularity-debug"); stdout_log = make_logfile("stdout"); stderr_log = make_logfile("stderr"); for( i = 0; i <= 2; i++ ) { close(i); } if ( chdir("/") < 0 ) { singularity_message(ERROR, "Can't change directory to /\n"); } setsid(); umask(0); /* set program name */ if ( prctl(PR_SET_NAME, "sinit", 0, 0, 0) < 0 ) { singularity_message(ERROR, "Failed to set program name\n"); ABORT(255); } child = fork(); if ( child == 0 ) { /* Make standard output and standard error files to log stdout & stderr into */ if ( stdout_log != NULL ) { if ( -1 == dup2(stdout_log->fd, 1) ) { singularity_message(ERROR, "Unable to dup2(): %s\n", strerror(errno)); ABORT(255); } } if ( stderr_log != NULL ) { if ( -1 == dup2(stderr_log->fd, 2) ) { singularity_message(ERROR, "Unable to dup2(): %s\n", strerror(errno)); ABORT(255); } } /* Unblock signals and execute startscript */ singularity_unblock_signals(); if ( is_exec("/.singularity.d/actions/start") == 0 ) { singularity_message(DEBUG, "Exec'ing /.singularity.d/actions/start\n"); if ( execv("/.singularity.d/actions/start", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.singularity.d/actions/start: %s\n", strerror(errno)); ABORT(CHILD_FAILED); } } else { singularity_message(VERBOSE, "Instance start script not found\n"); kill(1, SIGCONT); } } else if ( child > 0 ) { if ( singularity_debug != NULL ) { if ( -1 == dup2(singularity_debug->fd, 2) ) { singularity_message(ERROR, "Unable to dup2(): %s\n", strerror(errno)); ABORT(255); } } singularity_message(DEBUG, "Waiting for signals\n"); /* send a SIGALRM if start script doesn't send SIGCONT within 1 seconds */ alarm(1); while (1) { if ( singularity_handle_signals(&siginfo) < 0 ) { singularity_signal_go_ahead(255); break; } if ( siginfo.si_signo == SIGCHLD ) { singularity_message(DEBUG, "Child exited\n"); if ( siginfo.si_pid == 2 && siginfo.si_status == CHILD_FAILED ) { singularity_signal_go_ahead(CHILD_FAILED); break; } } else if ( siginfo.si_signo == SIGCONT && siginfo.si_pid == 2 ) { /* start script correctly exec */ singularity_signal_go_ahead(0); started = 1; } else if ( siginfo.si_signo == SIGALRM && started == 0 ) { /* don't receive SIGCONT, start script modified/replaced ? */ singularity_message(ERROR, "Start script doesn't send SIGCONT\n"); singularity_signal_go_ahead(255); break; } } } else { singularity_message(ERROR, "Failed to execute start script\n"); singularity_signal_go_ahead(255); } return(0); } singularity-2.4.2/src/get-section.c0000644000175000017500000000511713211621077016207 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #define MAX_LINE_LEN 4096 int main(int argc, char ** argv) { char *section; char *file; int toggle_section = 0; int retval = 1; FILE *input; char *line = (char *)malloc(MAX_LINE_LEN); if ( argc < 3 ) { printf("USAGE: %s [section] [file]\n", argv[0]); exit(0); } section = strdup(argv[1]); file = strdup(argv[2]); if ( is_file(file) < 0 ) { singularity_message(ERROR, "File not found: %s\n", file); ABORT(1); } if ( ( input = fopen(file, "r") ) == NULL ) { // Flawfinder: ignore singularity_message(ERROR, "Could not open file %s: %s\n", file, strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Iterating through file looking for sections matching: %%%s\n", section); while ( fgets(line, MAX_LINE_LEN, input) != NULL ) { if ( strncmp(line, strjoin("%", section), strlength(section, 128) + 1) == 0 ) { toggle_section = 1; retval = 0; } else if ( ( toggle_section == 1 ) && ( strncmp(line, "%", 1) == 0 ) ) { toggle_section = 0; } else if ( toggle_section == 1 ) { printf("%s", line); } } fclose(input); free(section); free(file); free(line); return(retval); } singularity-2.4.2/src/util/0000755000175000017500000000000013211621077014573 5ustar mehdimehdisingularity-2.4.2/src/util/fork.c0000644000175000017500000003610013211621077015700 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * Copyright (c) 2016, Brian Bockelman. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include "util/privilege.h" #include "util/message.h" #include "util/util.h" int generic_signal_rpipe = -1; int generic_signal_wpipe = -1; int sigchld_signal_rpipe = -1; int sigchld_signal_wpipe = -1; int watchdog_rpipe = -1; int watchdog_wpipe = -1; pid_t child_pid; struct pollfd fds[2]; typedef struct fork_state_s { sigjmp_buf env; } fork_state_t; // NOTE: singularity_message is NOT signal handler safe. // Hence, we MUST NOT do any sort of generic logging from these // functions. We might, in the future, add in a signal-safe // version of singularity_message here. void handle_signal(int sig, siginfo_t * _, void * __) { char info = (char)sig; while (-1 == write(generic_signal_wpipe, &info, 1) && errno == EINTR) {} } void handle_sigchld(int sig, siginfo_t *siginfo, void * _) { if ( siginfo->si_pid == child_pid ) { char one = '1'; while (-1 == write(sigchld_signal_wpipe, &one, 1) && errno == EINTR) {} } } /* Setup the communication between parent and child. * * These series of functions force the child to wait for an explicit go-ahead from the * parent before proceeding. * - `prepare_for_fork`: Must be called by both parent and child before fork() is called. * - `wait_for_go_ahead`: Called by child, waits until the parent gives the go-ahead signal. * - `signal_go_ahead`: Called by parent, indicates the child may proceed. * * Keeps global state in the coordination_pipe variable. */ static int pipe_to_child[] = {-1, -1}; static int pipe_to_parent[] = {-1, -1}; static int coordination_pipe[] = {-1, -1}; static void prepare_fork() { singularity_message(DEBUG, "Creating parent/child coordination pipes.\n"); // Note we use pipe and not pipe2 here with CLOEXEC. This is because we eventually want the parent process // to exec a separate unprivileged process and inherit the communication pipe. if ( -1 == pipe(pipe_to_child) ) { singularity_message(ERROR, "Failed to create coordination pipe for fork: %s (errno=%d)\n", strerror(errno), errno); ABORT(255); } if ( -1 == pipe(pipe_to_parent) ) { singularity_message(ERROR, "Failed to create coordination pipe for fork: %s (errno=%d)\n", strerror(errno), errno); ABORT(255); } } /* Put the appropriate read and write pipes into coordination_pipe[] */ static void prepare_pipes_child() { /* Close to child write pipe */ close(pipe_to_child[1]); /* Close to parent read pipe */ close(pipe_to_parent[0]); /* Store read and write pipes into common variable */ coordination_pipe[0] = pipe_to_child[0]; coordination_pipe[1] = pipe_to_parent[1]; } /* Put the appropriate read and write pipes into coordination_pipe[] */ static void prepare_pipes_parent() { /* Close to parent write pipe */ close(pipe_to_parent[1]); /* Close to child read pipe */ close(pipe_to_child[0]); /* Store read and write pipes into common variable */ coordination_pipe[0] = pipe_to_parent[0]; coordination_pipe[1] = pipe_to_child[1]; } /* Updated wait_for_go_ahead() which allows bi-directional wait signaling */ int singularity_wait_for_go_ahead() { if ( (coordination_pipe[0] == -1) || (coordination_pipe[1] == -1)) { singularity_message(ERROR, "Internal error! wait_for_go_ahead invoked with invalid pipe state (%d, %d).\n", coordination_pipe[0], coordination_pipe[1]); ABORT(255); } singularity_message(DEBUG, "Waiting for go-ahead signal\n"); char code = -1; int retval; // Block until other process indicates it is OK to proceed. while ( (-1 == (retval = read(coordination_pipe[0], &code, 1))) && errno == EINTR) {} if (retval == -1) { // Failed to communicate with other process. singularity_message(ERROR, "Failed to communicate with other process: %s (errno=%d)\n", strerror(errno), errno); ABORT(255); } else if (retval == 0) { // Other process closed the write pipe unexpectedly. if ( close(dup(coordination_pipe[1])) == -1 ) { singularity_message(ERROR, "Other process closed write pipe unexpectedly.\n"); ABORT(255); } } singularity_message(DEBUG, "Received go-ahead signal: %d\n", code); return(code); } /* Updated signal_go_ahead() which allows bi-directional wait signaling */ void singularity_signal_go_ahead(int code) { if ( (coordination_pipe[0] == -1) || (coordination_pipe[1] == -1)) { singularity_message(ERROR, "Internal error! signal_go_ahead invoked with invalid pipe state (%d, %d).\n", coordination_pipe[0], coordination_pipe[1]); ABORT(255); } singularity_message(DEBUG, "Sending go-ahead signal: %d\n", code); int retval; while ( (-1 == (retval = write(coordination_pipe[1], &code, 1))) && errno == EINTR) {} if (retval == -1) { if ( errno != EPIPE ) { singularity_message(ERROR, "Failed to send go-ahead to child process: %s (errno=%d)\n", strerror(errno), errno); ABORT(255); } } // Note that we don't test for retval == 0 as we should get a EPIPE instead. } static int wait_child() { int child_ok = 1; int retval, tmpstatus; singularity_message(DEBUG, "Parent process is waiting on child process\n"); do { /* Poll the signal handle read pipes to wait for any written signals */ while ( -1 == (retval = poll(fds, 2, -1)) && errno == EINTR ) {} if ( -1 == retval ) { singularity_message(ERROR, "Failed to wait for file descriptors: %s\n", strerror(errno)); ABORT(255); } /* When SIGCHILD is received, set child_ok = 0 to break out of loop */ if (fds[0].revents) { singularity_message(DEBUG, "SIGCHLD raised, parent is exiting\n"); child_ok = 0; } /* If we catch any other signal, */ if (fds[1].revents) { char signum = SIGKILL; while (-1 == (retval = read(generic_signal_rpipe, &signum, 1)) && errno == EINTR) {} // Flawfinder: ignore if (-1 == retval) { singularity_message(ERROR, "Failed to read from signal handler pipe: %s\n", strerror(errno)); ABORT(255); } singularity_message(VERBOSE2, "Sending signal to child: %d\n", signum); kill(child_pid, signum); } } while( child_ok ); /* Catch the exit status or kill signal of the child process */ waitpid(child_pid, &tmpstatus, 0); if (WIFEXITED(tmpstatus)) { return(WEXITSTATUS(tmpstatus)); } else if (WIFSIGNALED(tmpstatus)) { kill(getpid(), WTERMSIG(tmpstatus)); } return(-1); } /* */ static int clone_fn(void *data_ptr) { fork_state_t *state = (fork_state_t *)data_ptr; siglongjmp(state->env, 1); } /* */ static int fork_ns(unsigned int flags) { fork_state_t state; if ( sigsetjmp(state.env, 1) ) { return 0; } int stack_size = 1024*1024; void *child_stack_ptr = malloc(stack_size); if ( child_stack_ptr == 0 ) { errno = ENOMEM; return -1; } child_stack_ptr += stack_size; int retval = clone(clone_fn, child_stack_ptr, (SIGCHLD|flags), &state ); return retval; } void install_generic_signal_handle() { int pipes[2]; struct sigaction action; sigset_t empty_mask; sigemptyset(&empty_mask); /* Fill action with handle_signal function */ action.sa_sigaction = &handle_signal; action.sa_flags = SA_SIGINFO|SA_RESTART; action.sa_mask = empty_mask; singularity_message(DEBUG, "Assigning generic sigaction()s\n"); if ( -1 == sigaction(SIGINT, &action, NULL) ) { singularity_message(ERROR, "Failed to install SIGINT signal handler: %s\n", strerror(errno)); ABORT(255); } if ( -1 == sigaction(SIGQUIT, &action, NULL) ) { singularity_message(ERROR, "Failed to install SIGQUIT signal handler: %s\n", strerror(errno)); ABORT(255); } if ( -1 == sigaction(SIGTERM, &action, NULL) ) { singularity_message(ERROR, "Failed to install SIGTERM signal handler: %s\n", strerror(errno)); ABORT(255); } if ( -1 == sigaction(SIGHUP, &action, NULL) ) { singularity_message(ERROR, "Failed to install SIGHUP signal handler: %s\n", strerror(errno)); ABORT(255); } if ( -1 == sigaction(SIGUSR1, &action, NULL) ) { singularity_message(ERROR, "Failed to install SIGUSR1 signal handler: %s\n", strerror(errno)); ABORT(255); } if ( -1 == sigaction(SIGUSR2, &action, NULL) ) { singularity_message(ERROR, "Failed to install SIGUSR2 signal handler: %s\n", strerror(errno)); ABORT(255); } /* Open pipes for handle_signal() to write to */ singularity_message(DEBUG, "Creating generic signal pipes\n"); if ( -1 == pipe2(pipes, O_CLOEXEC) ) { singularity_message(ERROR, "Failed to create communication pipes: %s\n", strerror(errno)); ABORT(255); } generic_signal_rpipe = pipes[0]; generic_signal_wpipe = pipes[1]; } void install_sigchld_signal_handle() { int pipes[2]; struct sigaction action; sigset_t empty_mask; sigemptyset(&empty_mask); /* Fill action with handle_sigchld function */ action.sa_sigaction = &handle_sigchld; action.sa_flags = SA_SIGINFO|SA_RESTART; action.sa_mask = empty_mask; singularity_message(DEBUG, "Assigning SIGCHLD sigaction()\n"); if ( -1 == sigaction(SIGCHLD, &action, NULL) ) { singularity_message(ERROR, "Failed to install SIGCHLD signal handler: %s\n", strerror(errno)); ABORT(255); } /* Open pipes for handle_sigchld() to write to */ singularity_message(DEBUG, "Creating sigchld signal pipes\n"); if ( -1 == pipe2(pipes, O_CLOEXEC) ) { singularity_message(ERROR, "Failed to create communication pipes: %s\n", strerror(errno)); ABORT(255); } sigchld_signal_rpipe = pipes[0]; sigchld_signal_wpipe = pipes[1]; } pid_t singularity_fork(unsigned int flags) { int priv_fork = 1; prepare_fork(); if ( flags == 0 || geteuid() == 0 ) { priv_fork = 0; } singularity_message(VERBOSE2, "Forking child process\n"); if ( priv_fork == 1 ) { singularity_priv_escalate(); } child_pid = fork_ns(flags); if ( priv_fork == 1 ) { singularity_priv_drop(); } if ( child_pid == 0 ) { singularity_message(VERBOSE2, "Hello from child process\n"); prepare_pipes_child(); singularity_wait_for_go_ahead(); return(child_pid); } else if ( child_pid > 0 ) { singularity_message(VERBOSE2, "Hello from parent process\n"); prepare_pipes_parent(); /* Set signal mask to block all signals while we set up sig actions */ sigset_t blocked_mask, old_mask; sigfillset(&blocked_mask); sigprocmask(SIG_SETMASK, &blocked_mask, &old_mask); /* Now that we can't receive any signals, install signal handlers for all signals we want to catch */ install_generic_signal_handle(); install_sigchld_signal_handle(); /* Set signal mask back to the original mask, unblocking the blocked signals */ sigprocmask(SIG_SETMASK, &old_mask, NULL); /* Set fds[n].fd to the read pipes created earlier */ fds[0].fd = sigchld_signal_rpipe; fds[0].events = POLLIN; fds[0].revents = 0; fds[1].fd = generic_signal_rpipe; fds[1].events = POLLIN; fds[1].revents = 0; /* Drop privs if we're SUID */ if ( singularity_priv_is_suid() == 0 ) { singularity_message(DEBUG, "Dropping permissions\n"); singularity_priv_drop(); } /* Allow child process to continue */ singularity_signal_go_ahead(0); return(child_pid); } else { singularity_message(ERROR, "Failed to fork child process: %s\n", strerror(errno)); ABORT(255); } } void singularity_fork_run(unsigned int flags) { pid_t child; int retval; child = singularity_fork(flags); if ( child == 0 ) { return; } else if ( child > 0 ) { retval = wait_child(); exit(retval); } } int singularity_fork_exec(unsigned int flags, char **argv) { int retval = 1; int i = 0; pid_t child; child = singularity_fork(0); if ( child == 0 ) { while(1) { if ( argv[i] == NULL ) { break; } else if ( i == 128 ) { singularity_message(ERROR, "singularity_fork_exec() ARGV out of bounds\n"); ABORT(255); } singularity_message(DEBUG, "fork argv[%d] = %s\n", i, argv[i]); i++; } singularity_message(VERBOSE, "Running child program: %s\n", argv[0]); if ( execvp(argv[0], argv) < 0 ) { //Flawfinder: ignore singularity_message(ERROR, "Failed to exec program %s: %s\n", argv[0], strerror(errno)); ABORT(255); } } else if ( child > 0 ) { retval = wait_child(); } singularity_message(DEBUG, "Returning from singularity_fork_exec with: %d\n", retval); return(retval); } int singularity_fork_daemonize(unsigned int flags) { pid_t child; child = singularity_fork(flags); if ( child == 0 ) { return(0); } else if ( child > 0 ) { singularity_message(DEBUG, "Successfully spawned daemon, waiting for signal_go_ahead from child\n"); int code = singularity_wait_for_go_ahead(); if ( code == 0 ) { exit(0); } else { singularity_message(ERROR, "Daemon failed to start\n"); ABORT(code); } } singularity_message(ERROR, "Reached unreachable code. How did you get here?\n"); ABORT(255); return(0); } singularity-2.4.2/src/util/registry.h0000644000175000017500000000435613211621077016624 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_REGISTRY_H_ #define __SINGULARITY_REGISTRY_H_ #define REGISTRY_SIZE 128 #define MAX_KEY_LEN 128 // Initalize the registry. It will automatically be initalized on the // first call to _get() or _set(), but if you want to ensure it gets run // at startup time, do it manually. // // This will also retrieve any environment variables that are prefixed with // "SINGULARITY_" and load that data into the registry automatically. extern void singularity_registry_init(void); // Set a value in the registry. If it already exists, this will overwrite the // previous entry as only one value for each key can be stored. extern int singularity_registry_set(char *key, char *value); // Get any value that is currently being stored in the registry. If the key // is not currently set, it will return with NULL. extern char *singularity_registry_get(char *key); #endif /* __SINGULARITY_REGISTRY_H_ */ singularity-2.4.2/src/util/file.c0000644000175000017500000003325413211621077015665 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _XOPEN_SOURCE 500 // For nftw #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" char *file_id(char *path) { struct stat filestat; char *ret; uid_t uid = singularity_priv_getuid(); singularity_message(DEBUG, "Called file_id(%s)\n", path); // Stat path if (lstat(path, &filestat) < 0) { return(NULL); } ret = (char *) malloc(128); snprintf(ret, 128, "%d.%d.%lu", (int)uid, (int)filestat.st_dev, (long unsigned)filestat.st_ino); // Flawfinder: ignore singularity_message(VERBOSE2, "Generated file_id: %s\n", ret); singularity_message(DEBUG, "Returning file_id(%s) = %s\n", path, ret); return(ret); } char *file_devino(char *path) { struct stat filestat; char *ret; singularity_message(DEBUG, "Called file_devino(%s)\n", path); // Stat path if (lstat(path, &filestat) < 0) { return(NULL); } ret = (char *) malloc(128); snprintf(ret, 128, "%d.%lu", (int)filestat.st_dev, (long unsigned)filestat.st_ino); // Flawfinder: ignore singularity_message(DEBUG, "Returning file_devino(%s) = %s\n", path, ret); return(ret); } int chk_perms(char *path, mode_t mode) { struct stat filestat; singularity_message(DEBUG, "Checking permissions on: %s\n", path); // Stat path if (stat(path, &filestat) < 0) { return(-1); } if ( filestat.st_mode & mode ) { singularity_message(WARNING, "Found appropriate permissions on file: %s\n", path); return(0); } return(-1); } int chk_mode(char *path, mode_t mode, mode_t mask) { struct stat filestat; singularity_message(DEBUG, "Checking exact mode (%o) on: %s\n", mode, path); // Stat path if (stat(path, &filestat) < 0) { return(-1); } if ( ( filestat.st_mode | mask ) == ( mode | mask ) ) { singularity_message(DEBUG, "Found appropriate mode on file: %s\n", path); return(0); } else { singularity_message(VERBOSE, "Found wrong permission on file %s: %o != %o\n", path, mode, filestat.st_mode); } return(-1); } int is_file(char *path) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } // Test path if ( S_ISREG(filestat.st_mode) ) { return(0); } return(-1); } int is_fifo(char *path) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } // Test path if ( S_ISFIFO(filestat.st_mode) ) { return(0); } return(-1); } int is_link(char *path) { struct stat filestat; // Stat path if (lstat(path, &filestat) < 0) { return(-1); } // Test path if ( S_ISLNK(filestat.st_mode) ) { return(0); } return(-1); } int is_dir(char *path) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } // Test path if ( S_ISDIR(filestat.st_mode) ) { return(0); } return(-1); } int is_suid(char *path) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } // Test path if ( (S_ISUID & filestat.st_mode) ) { return(0); } return(-1); } int is_exec(char *path) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } // Test path if ( (S_IXUSR & filestat.st_mode) ) { return(0); } return(-1); } int is_write(char *path) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } // Test path if ( (S_IWUSR & filestat.st_mode) ) { return(0); } return(-1); } int is_owner(char *path, uid_t uid) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } if ( uid == (int)filestat.st_uid ) { return(0); } return(-1); } int is_blk(char *path) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } // Test path if ( S_ISBLK(filestat.st_mode) ) { return(0); } return(-1); } int is_chr(char *path) { struct stat filestat; // Stat path if (stat(path, &filestat) < 0) { return(-1); } // Test path if ( S_ISCHR(filestat.st_mode) ) { return(0); } return(-1); } int s_mkpath(char *dir, mode_t mode) { if (!dir) { return(-1); } if (strcmp(dir, "/") == 0 ) { singularity_message(DEBUG, "Directory is '/', returning '0'\n"); return(0); } if ( is_dir(dir) == 0 ) { singularity_message(DEBUG, "Directory exists, returning '0': %s\n", dir); return(0); } if ( is_dir(dirname(strdupa(dir))) < 0 ) { singularity_message(DEBUG, "Creating parent directory: %s\n", dirname(strdupa(dir))); if ( s_mkpath(dirname(strdupa(dir)), mode) < 0 ) { singularity_message(VERBOSE, "Failed to create parent directory %s\n", dir); return(-1); } } singularity_message(DEBUG, "Creating directory: %s\n", dir); mode_t mask = umask(0); // Flawfinder: ignore int ret = mkdir(dir, mode); umask(mask); // Flawfinder: ignore if ( ret < 0 ) { if ( errno != EEXIST ) { singularity_message(DEBUG, "Opps, could not create directory %s: (%d) %s\n", dir, errno, strerror(errno)); return(-1); } } return(0); } int _unlink(const char *fpath, const struct stat *sb, int typeflag, struct FTW *ftwbuf) { int retval; if ( ( retval = remove(fpath) ) < 0 ) { singularity_message(WARNING, "Failed removing file: %s\n", fpath); } return(retval); } int _writable(const char *fpath, const struct stat *sb, int typeflag, struct FTW *ftwbuf) { int retval; if ( is_link((char *) fpath) == 0 ) { return(0); } if ( ( retval = chmod(fpath, 0700) ) < 0 ) { // Flawfinder: ignore singularity_message(WARNING, "Failed changing permission of file: %s\n", fpath); } // Always return success return(0); } int s_rmdir(char *dir) { singularity_message(DEBUG, "Removing directory: %s\n", dir); if ( nftw(dir, _writable, 32, FTW_MOUNT|FTW_PHYS) < 0 ) { singularity_message(ERROR, "Failed preparing directory for removal: %s\n", dir); ABORT(255); } return(nftw(dir, _unlink, 32, FTW_DEPTH|FTW_MOUNT|FTW_PHYS)); } int copy_file(char * source, char * dest) { struct stat filestat; int c; FILE * fp_s; FILE * fp_d; singularity_message(DEBUG, "Called copy_file(%s, %s)\n", source, dest); if ( is_file(source) < 0 ) { singularity_message(ERROR, "Could not copy from non-existent source: %s\n", source); return(-1); } singularity_message(DEBUG, "Opening source file: %s\n", source); if ( ( fp_s = fopen(source, "r") ) == NULL ) { // Flawfinder: ignore singularity_message(ERROR, "Could not read %s: %s\n", source, strerror(errno)); return(-1); } singularity_message(DEBUG, "Opening destination file: %s\n", dest); if ( ( fp_d = fopen(dest, "w") ) == NULL ) { // Flawfinder: ignore fclose(fp_s); singularity_message(ERROR, "Could not write %s: %s\n", dest, strerror(errno)); return(-1); } singularity_message(DEBUG, "Calling fstat() on source file descriptor: %d\n", fileno(fp_s)); if ( fstat(fileno(fp_s), &filestat) < 0 ) { singularity_message(ERROR, "Could not fstat() on %s: %s\n", source, strerror(errno)); fclose(fp_s); fclose(fp_d); return(-1); } singularity_message(DEBUG, "Cloning permission string of source to dest\n"); if ( fchmod(fileno(fp_d), filestat.st_mode) < 0 ) { singularity_message(ERROR, "Could not set permission mode on %s: %s\n", dest, strerror(errno)); fclose(fp_s); fclose(fp_d); return(-1); } singularity_message(DEBUG, "Copying file data...\n"); while ( ( c = fgetc(fp_s) ) != EOF ) { // Flawfinder: ignore (checked boundries) fputc(c, fp_d); } singularity_message(DEBUG, "Done copying data, closing file pointers\n"); fclose(fp_s); fclose(fp_d); singularity_message(DEBUG, "Returning copy_file(%s, %s) = 0\n", source, dest); return(0); } int fileput(char *path, char *string) { FILE *fd; singularity_message(DEBUG, "Called fileput(%s, %s)\n", path, string); if ( ( fd = fopen(path, "w") ) == NULL ) { // Flawfinder: ignore singularity_message(ERROR, "Could not write to %s: %s\n", path, strerror(errno)); return(-1); } fprintf(fd, "%s", string); fclose(fd); return(0); } char *filecat(char *path) { char *ret; FILE *fd; int c; long length; long pos = 0; singularity_message(DEBUG, "Called filecat(%s)\n", path); if ( is_file(path) < 0 ) { singularity_message(ERROR, "Could not find %s\n", path); return(NULL); } if ( ( fd = fopen(path, "r") ) == NULL ) { // Flawfinder: ignore singularity_message(ERROR, "Could not read from %s: %s\n", path, strerror(errno)); return(NULL); } if ( fseek(fd, 0L, SEEK_END) < 0 ) { singularity_message(ERROR, "Could not seek to end of file %s: %s\n", path, strerror(errno)); fclose(fd); return(NULL); } length = ftell(fd); rewind(fd); ret = (char *) malloc(length+1); while ( ( c = fgetc(fd) ) != EOF ) { // Flawfinder: ignore (checked boundries) ret[pos] = c; pos++; } ret[pos] = '\0'; fclose(fd); return(ret); } /* * Open and exclusive-lock file, creating it (-rw-------) * if necessary. If fdptr is not NULL, the descriptor is * saved there. The descriptor is never one of the standard * descriptors STDIN_FILENO, STDOUT_FILENO, or STDERR_FILENO. * If successful, the function returns 0. * Otherwise, the function returns nonzero errno: * EINVAL: Invalid lock file path * EMFILE: Too many open files * EALREADY: Already locked * or one of the open(2)/creat(2) errors. */ int filelock(const char *const filepath, int *const fdptr) { struct flock lock; int used = 0; /* Bits 0 to 2: stdin, stdout, stderr */ int fd; singularity_message(DEBUG, "Called filelock(%s)\n", filepath); /* In case the caller is interested in the descriptor, * initialize it to -1 (invalid). */ if (fdptr) *fdptr = -1; /* Invalid path? */ if (filepath == NULL || *filepath == '\0') return errno = EINVAL; /* Open the file. */ do { fd = open(filepath, O_RDWR | O_CREAT, 0644); } while (fd == -1 && errno == EINTR); if (fd == -1) { if (errno == EALREADY) errno = EIO; return errno; } /* Move fd away from the standard descriptors. */ while (1) { if( fd == STDIN_FILENO ) { used |= 1; fd = dup(fd); } else if ( fd == STDOUT_FILENO ) { used |= 2; fd = dup(fd); } else if( fd == STDERR_FILENO ) { used |= 4; fd = dup(fd); } else { break; } } /* Close the standard descriptors we temporarily used. */ if (used & 1) close(STDIN_FILENO); if (used & 2) close(STDOUT_FILENO); if (used & 4) close(STDERR_FILENO); /* Did we run out of descriptors? */ if (fd == -1) return errno = EMFILE; /* Exclusive lock, cover the entire file (regardless of size). */ lock.l_type = F_WRLCK; lock.l_whence = SEEK_SET; lock.l_start = 0; lock.l_len = 0; if (fcntl(fd, F_SETLK, &lock) == -1) { /* Lock failed. Close file and report locking failure. */ close(fd); return errno = EALREADY; } if ( fcntl(fd, F_SETFD, FD_CLOEXEC) != 0 ) { close(fd); return errno = EBADF; } /* Save descriptor, if the caller wants it. */ if (fdptr) *fdptr = fd; return 0; } char *basedir(char *dir) { char *testdir = strdup(dir); char *ret = NULL; singularity_message(DEBUG, "Obtaining basedir for: %s\n", dir); while ( ( strcmp(testdir, "/") != 0 ) && ( strcmp(testdir, ".") != 0 ) ) { singularity_message(DEBUG, "Iterating basedir: %s\n", testdir); ret = strdup(testdir); testdir = dirname(strdup(testdir)); } return(ret); } singularity-2.4.2/src/util/config_defaults.h.in0000644000175000017500000000633013211621077020507 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_CONFIG_DEFAULTS_H_ #define __SINGULARITY_CONFIG_DEFAULTS_H_ #define ALLOW_SETUID "allow setuid" #define ALLOW_SETUID_DEFAULT 1 #define ALLOW_PID_NS "allow pid ns" #define ALLOW_PID_NS_DEFAULT 1 // NOTE: This is missing from the default configuration file. #define ALLOW_IPC_NS "allow ipc ns" #define ALLOW_IPC_NS_DEFAULT 1 // NOTE: This is missing from the default configuration file. #define ALLOW_USER_NS "allow user ns" #define ALLOW_USER_NS_DEFAULT 1 #define ENABLE_OVERLAY "enable overlay" #define ENABLE_OVERLAY_DEFAULT "try" #define CONFIG_PASSWD "config passwd" #define CONFIG_PASSWD_DEFAULT 1 #define CONFIG_GROUP "config group" #define CONFIG_GROUP_DEFAULT 1 #define CONFIG_RESOLV_CONF "config resolv_conf" #define CONFIG_RESOLV_CONF_DEFAULT 1 #define MOUNT_PROC "mount proc" #define MOUNT_PROC_DEFAULT 1 #define MOUNT_SYS "mount sys" #define MOUNT_SYS_DEFAULT 1 #define MOUNT_DEV "mount dev" #define MOUNT_DEV_DEFAULT "yes" #define MOUNT_DEVPTS "mount devpts" #define MOUNT_DEVPTS_DEFAULT "yes" #define MOUNT_HOME "mount home" #define MOUNT_HOME_DEFAULT 1 #define MOUNT_TMP "mount tmp" #define MOUNT_TMP_DEFAULT 1 #define MOUNT_HOSTFS "mount hostfs" #define MOUNT_HOSTFS_DEFAULT 0 #define BIND_PATH "bind path" #define BIND_PATH_DEFAULT "" #define USER_BIND_CONTROL "user bind control" #define USER_BIND_CONTROL_DEFAULT 1 #define MOUNT_SLAVE "mount slave" #define MOUNT_SLAVE_DEFAULT 1 #define SESSIONDIR_MAXSIZE "sessiondir max size" #define SESSIONDIR_MAXSIZE_DEFAULT "16" #define LIMIT_CONTAINER_OWNERS "limit container owners" #define LIMIT_CONTAINER_OWNERS_DEFAULT "NULL" #define LIMIT_CONTAINER_PATHS "limit container paths" #define LIMIT_CONTAINER_PATHS_DEFAULT "NULL" #define MAX_LOOP_DEVS "max loop devices" #define MAX_LOOP_DEVS_DEFAULT "256" #define AUTOFS_BUG_PATH "autofs bug path" #define AUTOFS_BUG_PATH_DEFAULT "" #define ALLOW_CONTAINER_DIR "allow container dir" #define ALLOW_CONTAINER_DIR_DEFAULT 1 #define ALLOW_CONTAINER_EXTFS "allow container extfs" #define ALLOW_CONTAINER_EXTFS_DEFAULT 1 #define ALLOW_CONTAINER_SQUASHFS "allow container squashfs" #define ALLOW_CONTAINER_SQUASHFS_DEFAULT 1 #endif // __SINGULARITY_CONFIG_DEFAULTS_H_ singularity-2.4.2/src/util/cleanupd.h0000644000175000017500000000275713211621077016552 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_CLEANUPD_H_ #define __SINGULARITY_CLEANUPD_H_ extern int singularity_cleanupd(void); #endif /* __SINGULARITY_CLEANUPD_H_ */ singularity-2.4.2/src/util/mount.c0000644000175000017500000000610413211621077016102 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * See the COPYRIGHT.md file at the top-level directory of this distribution and at * https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. * * This file is part of the Singularity Linux container project. It is subject to the license * terms in the LICENSE.md file found in the top-level directory of this distribution and * at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part * of Singularity, including this file, may be copied, modified, propagated, or distributed * except according to the terms contained in the LICENSE.md file. * */ #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #define MAX_LINE_LEN 2048 int singularity_mount(const char *source, const char *target, const char *filesystemtype, unsigned long mountflags, const void *data) { if ( ( mountflags & MS_BIND ) ) { setfsuid(singularity_priv_getuid()); } return mount(source, target, filesystemtype, mountflags, data); } int check_mounted(char *mountpoint) { int retval = -1; FILE *mounts; char *line = (char *)malloc(MAX_LINE_LEN); char *rootfs_dir = CONTAINER_FINALDIR; unsigned int mountpoint_len = strlength(mountpoint, PATH_MAX); char *real_mountpoint; singularity_message(DEBUG, "Opening /proc/mounts\n"); if ( ( mounts = fopen("/proc/mounts", "r") ) == NULL ) { // Flawfinder: ignore singularity_message(ERROR, "Could not open /proc/mounts: %s\n", strerror(errno)); ABORT(255); } if ( mountpoint[mountpoint_len-1] == '/' ) { singularity_message(DEBUG, "Removing trailing slash from string: %s\n", mountpoint); mountpoint[mountpoint_len-1] = '\0'; } real_mountpoint = realpath(mountpoint, NULL); // Flawfinder: ignore if ( real_mountpoint == NULL ) { // mountpoint doesn't exists return(retval); } singularity_message(DEBUG, "Iterating through /proc/mounts\n"); while ( fgets(line, MAX_LINE_LEN, mounts) != NULL ) { (void) strtok(strdup(line), " "); char *mount = strtok(NULL, " "); // Check to see if mountpoint is already mounted if ( strcmp(joinpath(rootfs_dir, real_mountpoint), mount) == 0 ) { singularity_message(DEBUG, "Mountpoint is already mounted: %s\n", mountpoint); retval = 1; break; } // Check to see if path is in container root if ( strncmp(rootfs_dir, mount, strlength(rootfs_dir, 1024)) != 0 ) { continue; } // Check to see if path is ot the container root if ( strcmp(mount, rootfs_dir) == 0 ) { continue; } } fclose(mounts); free(line); free(real_mountpoint); return(retval); } singularity-2.4.2/src/util/config_parser.h0000644000175000017500000000534213211621077017571 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_CONFIG_H_ #define __SINGULARITY_CONFIG_H_ #include "config_defaults.h" // Retrieve a single value from the configuration; in the presence of // multiple values in the configuration file, only the last one is // returned. // // If the configuration file does not have a value for the given key, // then the compile-time default is returned. singularity_config_get_value // is actually a macro and should cause a compile-time error if there is no // default specified in the code. // const char *_singularity_config_get_value_impl(const char *key, const char *default_value); #define singularity_config_get_value(NAME) \ _singularity_config_get_value_impl(NAME, NAME ## _DEFAULT) // Retrieve (possibly) multiple values from the configuration file; the char* // array is terminated by NULL. // const char **_singularity_config_get_value_multi_impl(const char *key, const char *default_value); #define singularity_config_get_value_multi(NAME) \ _singularity_config_get_value_multi_impl(NAME, NAME ## _DEFAULT) // Retrieves a boolean value from the configuration file. If there are // multiple values in the configuration file, then only the last one is // returned. int _singularity_config_get_bool_impl(const char *key, int default_value); #define singularity_config_get_bool(NAME) \ _singularity_config_get_bool_impl(NAME, NAME ## _DEFAULT) int _singularity_config_get_bool_char_impl(const char *key, const char *value); #define singularity_config_get_bool_char(NAME) \ _singularity_config_get_bool_char_impl(NAME, NAME ## _DEFAULT) // Initialize the configuration table // int singularity_config_init(char *config_path); #endif /* __SINGULARITY_CONFIG_H_ */ singularity-2.4.2/src/util/sessiondir.c0000644000175000017500000000771313211621077017131 0ustar mehdimehdi/* * Copyright (c) 2016, Brian Bockelman. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include "util/message.h" #include "util/util.h" #include "util/file.h" #include "util/registry.h" #include "util/config_parser.h" #include "util/fork.h" #include "util/privilege.h" #include "util/mount.h" #ifndef LOCALSTATEDIR #error LOCALSTATEDIR not defined #endif int singularity_sessiondir(void) { char *sessiondir = NULL; char *sessiondir_size_str = NULL; long int sessiondir_size = 0; int sessiondir_size_str_len; int sessiondir_size_str_usd; if ( singularity_registry_get("DAEMON_JOIN") ) { singularity_message(ERROR, "Internal Error - This function should not be called when joining an instance\n"); } singularity_message(DEBUG, "Setting sessiondir\n"); sessiondir = joinpath(LOCALSTATEDIR, "/singularity/mnt/session"); singularity_message(VERBOSE, "Using session directory: %s\n", sessiondir); singularity_message(DEBUG, "Checking for session directory: %s\n", sessiondir); if ( is_dir(sessiondir) != 0 ) { singularity_message(ERROR, "Session directory does not exist: %s\n", sessiondir); ABORT(255); } singularity_message(DEBUG, "Obtaining the default sessiondir size\n"); if ( str2int(singularity_config_get_value(SESSIONDIR_MAXSIZE), &sessiondir_size) < 0 ) { singularity_message(ERROR, "Failed converting sessiondir size to integer, check config file\n"); ABORT(255); } singularity_message(DEBUG, "Converted sessiondir size to: %ld\n", sessiondir_size); singularity_message(DEBUG, "Creating the sessiondir size mount option length\n"); sessiondir_size_str_len = intlen(sessiondir_size) + 7; singularity_message(DEBUG, "Got size length of: %d\n", sessiondir_size_str_len); sessiondir_size_str = (char *) malloc(sessiondir_size_str_len); singularity_message(DEBUG, "Creating the sessiondir size mount option string\n"); sessiondir_size_str_usd = snprintf(sessiondir_size_str, sessiondir_size_str_len, "size=%ldm", sessiondir_size); singularity_message(DEBUG, "Checking to make sure the string was allocated correctly\n"); if ( sessiondir_size_str_usd + 1 != sessiondir_size_str_len ) { singularity_message(ERROR, "Failed to allocate string for sessiondir size string (%s): %d + 1 != %d\n", sessiondir_size_str, sessiondir_size_str_usd, sessiondir_size_str_len); ABORT(255); } singularity_priv_escalate(); singularity_message(DEBUG, "Mounting sessiondir tmpfs: %s\n", sessiondir); if ( singularity_mount("tmpfs", sessiondir, "tmpfs", MS_NOSUID, sessiondir_size_str) < 0 ){ singularity_message(ERROR, "Failed to mount sessiondir tmpfs %s: %s\n", sessiondir, strerror(errno)); ABORT(255); } singularity_priv_drop(); singularity_registry_set("SESSIONDIR", sessiondir); return(0); } singularity-2.4.2/src/util/config_parser.c0000644000175000017500000002726013211621077017567 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/message.h" #include "util/file.h" #include "config_parser.h" #define MAX_LINE_LEN (PATH_MAX + 128) #define MAX_CONFIG_ENTRIES 64 #define NULLONE ((char*)1) static int config_initialized = 0; static struct hsearch_data config_table; // Return a new, empty hash entry appropriate for adding to the config hash. // // By default, each hash bucket can have 7 values. We set currently-empty // entries static ENTRY *new_hash_entry(char *key, char *value) { char **hash_value = (char**) malloc(sizeof(char*) * MAX_CONFIG_ENTRIES+1); int idx; hash_value[0] = value; for (idx=1; idx < MAX_CONFIG_ENTRIES; idx++) {hash_value[idx] = (char*)1;} hash_value[MAX_CONFIG_ENTRIES] = NULL; ENTRY *hash_entry = (ENTRY*)malloc(sizeof(ENTRY)); memset(hash_entry, '\0', sizeof(ENTRY)); hash_entry->key = key; hash_entry->data = hash_value; return hash_entry; } static void add_entry(char *key, char *value) { ENTRY search_item; search_item.key = key; search_item.data = NULL; ENTRY * old_entry = NULL; if (hsearch_r(search_item, FIND, &old_entry, &config_table)) { char **hash_value = old_entry->data; int idx = 0; while ( (hash_value[idx] != NULL) && (hash_value[idx] != NULLONE) ) {idx++;} if ( idx >= MAX_CONFIG_ENTRIES ) { singularity_message(ERROR, "Maximum of %d allowed configuration entries for: %s\n", MAX_CONFIG_ENTRIES, key); ABORT(255); } if (hash_value[idx] == NULLONE) { hash_value[idx] = value; return; } if (hash_value[idx] == NULL) { int max_size = 2*(idx+1); hash_value = realloc(hash_value, sizeof(char*)*max_size); hash_value[idx] = value; for (; idxdata; int idx = 0; const char *retval = default_value; while ((values[idx] != NULL) && (values[idx] != NULLONE)) { retval = values[idx]; idx++; } singularity_message(DEBUG, "Returning configuration value %s='%s'\n", key, retval); return retval; } static const char *_default_entry[2]; const char **_singularity_config_get_value_multi_impl(const char *key, const char *default_value) { if (!config_initialized) { singularity_message(ERROR, "Called singularity_config_get_value on uninitialized config subsystem\n"); ABORT(255); } _default_entry[1] = '\0'; _default_entry[0] = default_value; ENTRY search_item; search_item.key = (char*)key; search_item.data = NULL; ENTRY * old_entry = NULL; if (!hsearch_r(search_item, FIND, &old_entry, &config_table)) { singularity_message(DEBUG, "No configuration entry found for '%s'; returning default value '%s'\n", key, default_value); return _default_entry; } char **values = old_entry->data; if ( (values[0] == NULL || values[0] == NULLONE) ) { singularity_message(DEBUG, "No configuration entry found for '%s'; returning default value '%s'\n", key, default_value); return _default_entry; } int idx = 1; while (values[idx] != NULL) { if (values[idx] == NULLONE) { values[idx] = NULL; } idx++; } return (const char **)values; } /* * Gets the associated boolean value of key from config_fp. Passes * key into singularity_get_config_value() and then checks if that * value is yes, no, or NULL. If not yes or no and not NULL, errors out. * * @param char *key pointer to key to search for * @param int def integer representing the default value of key * @returns 1 for yes, 0 for no, def if NULL */ int _singularity_config_get_bool_impl(const char *key, int def) { return _singularity_config_get_bool_char_impl(key, def ? "yes" : "no"); } int _singularity_config_get_bool_char_impl(const char *key, const char *def) { const char *config_value; singularity_message(DEBUG, "Called singularity_config_get_bool(%s, %s)\n", key, def); if ( ( config_value = _singularity_config_get_value_impl(key, def) ) != NULL ) { if ( strcmp(config_value, "yes") == 0 || strcmp(config_value, "y") == 0 || strcmp(config_value, "1") == 0 ) { singularity_message(DEBUG, "Return singularity_config_get_bool(%s, %s) = 1\n", key, def); return(1); } else if ( strcmp(config_value, "no") == 0 || strcmp(config_value, "n") == 0 || strcmp(config_value, "0") == 0 ) { singularity_message(DEBUG, "Return singularity_config_get_bool(%s, %s) = 0\n", key, def); return(0); } else { singularity_message(ERROR, "Unsupported value for configuration boolean key '%s' = '%s'\n", key, config_value); singularity_message(ERROR, "Returning default value: %s\n", def); ABORT(255); } } else { singularity_message(ERROR, "Undefined configuration for '%s'; should have resulted in a compile error.\n", key); ABORT(255); } return(-1); } singularity-2.4.2/src/util/daemon.c0000644000175000017500000001647113211621077016213 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * This software is licensed under a 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/daemon.h" #include "util/registry.h" #include "util/message.c" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/privilege.h" void daemon_file_parse(void) { singularity_message(DEBUG, "reached file parse\n"); char *key, *val; char *line = (char *)malloc(2048 * sizeof(char *)); FILE *file = fopen(singularity_registry_get("DAEMON_FILE"), "r"); while( fgets(line, 2048, file) ) { key = strtok(line, "=\n"); val = strtok(NULL, "=\n"); singularity_message(DEBUG, "Read key-val pair %s=%s\n", key, val); singularity_registry_set(key, val); } } void daemon_file_write(int fd, char *key, char *val) { int retval = 0; errno = 0; singularity_message(DEBUG, "Called daemon_file_write(%d, %s, %s)\n", fd, key, val); retval += write(fd, key, strlength(key, 2048)); retval += write(fd, "=", 1); retval += write(fd, val, strlength(val, 2048)); retval += write(fd, "\n", 1); if ( errno != 0 ) { singularity_message(ERROR, "Unable to write to daemon file: %s\n", strerror(errno)); ABORT(255); } } int daemon_is_owner(char *pid_path) { int retval = 0; char *proc_status = joinpath(pid_path, "/status"); char *uid_check = (char *)malloc(2048); char *line = (char *)malloc(2048); FILE *status = fopen(proc_status, "r"); pid_t uid = singularity_priv_getuid(); if ( status == NULL ) { singularity_message(ERROR, "Failed to open %s to check instance owner\n", proc_status); ABORT(255); } memset(uid_check, 0, 2048); snprintf(uid_check, 2047, "Uid:\t%d\t%d\t%d\t%d\n", uid, uid, uid, uid); while ( fgets(line, 2048, status) ) { if ( strcmp(line, uid_check) == 0 ) { retval = 1; break; } } free(uid_check); free(line); fclose(status); return(retval); } void daemon_init_join(void) { char *ns_path, *ns_fd_str; char *pid_path; int lock_result, ns_fd; int *lock_fd = malloc(sizeof(int)); char *daemon_file = singularity_registry_get("DAEMON_FILE"); char *daemon_name = singularity_registry_get("DAEMON_NAME"); /* Check if there is a lock on daemon file */ singularity_message(DEBUG, "Checking for lock on %s\n", daemon_file); lock_result = filelock(daemon_file, lock_fd); if ( lock_result == 0 ) { /* Successfully obtained lock, no daemon controls this file. */ singularity_message(ERROR, "Unable to join daemon: %s daemon does not exist\n", daemon_name); unlink(daemon_file); close(*lock_fd); ABORT(255); return; } else if ( lock_result == EALREADY ) { /* EALREADY is set when another process has a lock on the file. */ singularity_message(DEBUG, "Another process has lock on daemon file\n"); daemon_file_parse(); pid_path = (char *)malloc(2048 * sizeof(char *)); sprintf(pid_path, "/proc/%s", singularity_registry_get("DAEMON_PID")); //Flawfinder: ignore if ( daemon_is_owner(pid_path) == 0 ) { singularity_message(ERROR, "Unable to join instance: you are not the owner\n"); ABORT(255); } ns_path = joinpath(pid_path, "/ns"); /* Open FD to /proc/[PID]/ns directory to call openat() for ns files */ singularity_priv_escalate(); if ( ( ns_fd = open(ns_path, O_RDONLY | O_CLOEXEC) ) == -1 ) { singularity_message(ERROR, "Unable to open ns directory of PID in daemon file: %s\n", strerror(errno)); ABORT(255); } singularity_priv_drop(); ns_fd_str = int2str(ns_fd); /* Set DAEMON_NS_FD to /proc/[PID]/ns FD in registry */ singularity_registry_set("DAEMON_NS_FD", ns_fd_str); } else { singularity_message(ERROR, "Unable to join daemon: %s daemon does not exist\n", daemon_name); ABORT(255); } } void daemon_init_start(void) { char *daemon_file = singularity_registry_get("DAEMON_FILE"); char *daemon_name = singularity_registry_get("DAEMON_NAME"); char *daemon_file_dir = strdup(daemon_file); char *daemon_pid = (char *)malloc(256 * sizeof(char)); char *daemon_image; int daemon_fd; int lock; /* Check if /var/tmp/.singularity-daemon-[UID]/ directory exists, if not create it */ if ( is_dir(dirname(daemon_file_dir)) == -1 ) { s_mkpath(daemon_file_dir, 0755); } /* Attempt to open lock on daemon file */ lock = filelock(daemon_file, &daemon_fd); if( lock == 0 ) { singularity_message(DEBUG, "Successfully obtained excluse lock on %s\n", daemon_file); /* Calling readlink on /proc/self returns the PID of the thread in the host PID NS */ memset(daemon_pid, 0, 256); if ( readlink("/proc/self", daemon_pid, 256) == -1 ) { //Flawfinder: ignore singularity_message(ERROR, "Unable to open /proc/self: %s\n", strerror(errno)); ABORT(255); } else { singularity_message(DEBUG, "PID in host namespace: %s\n", daemon_pid); } if ( !(daemon_image = realpath(singularity_registry_get("IMAGE"), NULL)) ) { //Flawfinder: ignore singularity_message(DEBUG, "ERROR: %s\n", strerror(errno)); } /* Successfully obtained lock, write to daemon fd */ lseek(daemon_fd, 0, SEEK_SET); if ( ftruncate(daemon_fd, 0) == -1 ) { singularity_message(ERROR, "Unable to truncate %d: %s\n", daemon_fd, strerror(errno)); } daemon_file_write(daemon_fd, "DAEMON_PID", daemon_pid); daemon_file_write(daemon_fd, "DAEMON_IMAGE", daemon_image); daemon_file_write(daemon_fd, "DAEMON_ROOTFS", singularity_registry_get("ROOTFS")); singularity_registry_set("DAEMON_FD", int2str(daemon_fd)); } else if( lock == EALREADY ) { /* Another daemon controls this file already */ singularity_message(ERROR, "Daemon %s already exists: %s\n", daemon_name, strerror(errno)); ABORT(255); } else { singularity_message(ERROR, "Cannot lock %s: %s\n", daemon_file, strerror(errno)); ABORT(255); } } void singularity_daemon_init(void) { if ( singularity_registry_get("DAEMON_START") ) { #if defined (SINGULARITY_NO_SETNS) && !defined (SINGULARITY_SETNS_SYSCALL) singularity_message(ERROR, "Instance feature is disabled, your kernel is too old\n"); ABORT(255); #endif daemon_init_start(); return; } else if ( singularity_registry_get("DAEMON_JOIN") ) { #if defined (SINGULARITY_NO_SETNS) && !defined (SINGULARITY_SETNS_SYSCALL) singularity_message(ERROR, "Instance feature is disabled, your kernel is too old\n"); ABORT(255); #endif daemon_init_join(); return; } else { singularity_message(DEBUG, "Not joining a daemon, daemon join not set\n"); return; } } singularity-2.4.2/src/util/registry.c0000644000175000017500000001071213211621077016610 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/message.h" #include "util/file.h" #include "registry.h" static int registry_initialized = 0; static struct hsearch_data htab; extern char **environ; static ENTRY keypair(char *key, char *value) { ENTRY hash_entry; if ( key == NULL ) { singularity_message(ERROR, "Internal - Called keypair() with NULL key\n"); ABORT(255); } hash_entry.key = strdup(key); if ( value == NULL ) { hash_entry.data = NULL; } else { hash_entry.data = strdup(value); } return hash_entry; } void singularity_registry_init(void) { if ( registry_initialized != 1 ) { char **env = environ; singularity_message(VERBOSE, "Initializing Singularity Registry\n"); if ( hcreate_r(REGISTRY_SIZE, &htab) == 0 ) { singularity_message(ERROR, "Internal error - Unable to initalize registry core: %s\n", strerror(errno)); ABORT(255); } registry_initialized = 1; while (*env) { char *tok; char *string = strdup(*env++); if ( string == NULL ) { continue; } if ( strncmp(string, "SINGULARITY_", 12) != 0 ) { continue; } tok = strchr(string, '='); *tok = '\0'; string += 12; // Move string over so that SINGULARITY_ is skipped over singularity_registry_set(string, tok+1); } } } char *singularity_registry_get(char *key) { ENTRY *found; char *upperkey; int i = 0; int len = strlength(key, MAX_KEY_LEN); upperkey = (char *) malloc(len + 1); singularity_registry_init(); for ( i = 0; i < len; ++i ) upperkey[i] = toupper(key[i]); upperkey[len] = '\0'; if ( hsearch_r(keypair(upperkey, NULL), FIND, &found, &htab) == 0 ) { singularity_message(DEBUG, "Returning NULL on '%s'\n", upperkey); return(NULL); } singularity_message(DEBUG, "Returning value from registry: '%s' = '%s'\n", upperkey, (char *)found->data); return(found->data ? (strdup(found->data)) : NULL); } int singularity_registry_set(char *key, char *value) { ENTRY *prev; char *upperkey; int i = 0; int len = strlength(key, MAX_KEY_LEN); upperkey = (char *) malloc(len + 1); singularity_registry_init(); for ( i = 0; i < len; ++i ) upperkey[i] = toupper(key[i]); upperkey[len] = '\0'; singularity_message(VERBOSE2, "Adding value to registry: '%s' = '%s'\n", upperkey, value); if ( hsearch_r(keypair(upperkey, value), FIND, &prev, &htab) != 0 ) { singularity_message(VERBOSE2, "Found prior value for '%s', overriding with '%s'\n", key, value); prev->data = value ? strdup(value) : NULL; } else { if ( hsearch_r(keypair(upperkey, value), ENTER, &prev, &htab) == 0 ) { singularity_message(ERROR, "Internal error - Unable to set registry entry ('%s' = '%s'): %s\n", key, value, strerror(errno)); ABORT(255); } } singularity_message(DEBUG, "Returning singularity_registry_set(%s, %s) = 0\n", key, value); return(0); } singularity-2.4.2/src/util/sessiondir.h0000644000175000017500000000276713211621077017142 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_SESSIONDIR_H_ #define __SINGULARITY_SESSIONDIR_H_ extern int singularity_sessiondir(void); #endif /* __SINGULARITY_SESSIONDIR_H_ */ singularity-2.4.2/src/util/setns.h0000644000175000017500000000263113211621077016102 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * See the COPYRIGHT.md file at the top-level directory of this distribution and at * https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. * * This file is part of the Singularity Linux container project. It is subject to the license * terms in the LICENSE.md file found in the top-level directory of this distribution and * at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part * of Singularity, including this file, may be copied, modified, propagated, or distributed * except according to the terms contained in the LICENSE.md file. * */ #ifndef __SETNS_H_ #define __SETNS_H_ #ifndef __NR_setns # if defined (__x86_64__) # define __NR_setns 308 # elif defined (__i386__) # define __NR_setns 346 # elif defined (__alpha__) # define __NR_setns 501 # elif defined (__arm__) # define __NR_setns 375 # elif defined (__aarch64__) # define __NR_setns 375 # elif defined (__ia64__) # define __NR_setns 1330 # elif defined (__sparc__) # define __NR_setns 337 # elif defined (__powerpc__) # define __NR_setns 350 # elif defined (__s390__) # define __NR_setns 339 # endif #endif #ifdef __NR_setns extern int setns(int fd, int nstype); #else /* !__NR_setns */ # error Please determine the syscall number for setns on your architecture #endif #endif /* __SETNS_H_ */ singularity-2.4.2/src/util/message.c0000644000175000017500000001405313211621077016366 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/message.h" int messagelevel = -99; extern const char *__progname; static void message_init(void) { char *messagelevel_string = getenv("SINGULARITY_MESSAGELEVEL"); // Flawfinder: ignore (need to get string, validation in atol()) openlog("Singularity", LOG_CONS | LOG_NDELAY, LOG_LOCAL0); if ( messagelevel_string == NULL ) { messagelevel = 5; singularity_message(DEBUG, "SINGULARITY_MESSAGELEVEL undefined, setting level 5 (debug)\n"); } else { messagelevel = atoi(messagelevel_string); // Flawfinder: ignore if ( messagelevel > 9 ) { messagelevel = 9; } singularity_message(VERBOSE, "Set messagelevel to: %d\n", messagelevel); } } int singularity_message_level(void) { if ( messagelevel == -1 ) { message_init(); } return(messagelevel); } void _singularity_message(int level, const char *function, const char *file_in, int line, char *format, ...) { const char *file = file_in; int syslog_level = LOG_NOTICE; char message[512]; // Flawfinder: ignore (messages are truncated to 512 chars) char *prefix = NULL; char *color = NULL; va_list args; va_start (args, format); if (vsnprintf(message, 512, format, args) >= 512) { // Flawfinder: ignore (args are not user modifyable) memcpy(message+496, "(TRUNCATED...)\n", 15); message[511] = '\0'; } va_end (args); if ( messagelevel == -99 ) { message_init(); } while( ( ! isalpha(file[0]) ) && ( file[0] != '\0') ) { file++; } switch (level) { case ABRT: prefix = "ABORT"; color = ANSI_COLOR_RED; syslog_level = LOG_ALERT; break; case ERROR: prefix = "ERROR"; color = ANSI_COLOR_LIGHTRED; syslog_level = LOG_ERR; break; case WARNING: prefix = "WARNING"; color = ANSI_COLOR_YELLOW; syslog_level = LOG_WARNING; break; case LOG: prefix = "LOG"; color = ANSI_COLOR_BLUE; break; case DEBUG: prefix = "DEBUG"; color = ""; break; case INFO: prefix = "INFO"; color = ""; break; default: prefix = "VERBOSE"; color = ""; break; } if ( level <= LOG ) { // Note __progname comes from the linker; the UID can be 5 characters and PID can be // 10-or-so characters. char syslog_string[560]; // Flawfinder: ignore (512 max message length + 48 for header). if (snprintf(syslog_string, 540, "%s (U=%d,P=%d)> %s", __progname, geteuid(), getpid(), message) >= 540) { // This case shouldn't happen unless we have a very strange __progname; nul-terminating to be sure. syslog_string[559] = '\0'; } syslog(syslog_level, "%s", syslog_string); } if ( level <= messagelevel ) { char header_string[100]; if ( messagelevel >= DEBUG ) { char debug_string[25]; char location_string[60]; char tmp_header_string[86]; // snprintf(location_string, 60, "%s:%d:%s()", basename(strdup(file)), line, function); // Flawfinder: ignore // snprintf(location_string, 60, "%s:%d ", basename(strdup(file)), line); // Flawfinder: ignore if ( function[0] == '_' ) { function++; } snprintf(location_string, 60, "%s()", function); // Flawfinder: ignore location_string[59] = '\0'; snprintf(debug_string, 25, "[U=%d,P=%d]", geteuid(), getpid()); // Flawfinder: ignore debug_string[24] = '\0'; snprintf(tmp_header_string, 86, "%-18s %s", debug_string, location_string); // Flawfinder: ignore tmp_header_string[85] = '\0'; snprintf(header_string, 100, "%s%-7s %-60s ", color, prefix, tmp_header_string); // Flawfinder: ignore // header_string[94] = '\0'; } else { snprintf(header_string, 15, "%s%-7s: ", color, prefix); // Flawfinder: ignore // header_string[9] = '\0'; } if ( level == INFO && messagelevel == INFO ) { printf("%s" ANSI_COLOR_RESET, message); // Flawfinder: ignore (false alarm, format is constant) } else if ( level == INFO ) { printf("%s%s" ANSI_COLOR_RESET, header_string, message); // Flawfinder: ignore (false alarm, format is constant) } else if ( level == LOG && messagelevel <= INFO ) { // Don't print anything... } else { fprintf(stderr, "%s%s" ANSI_COLOR_RESET, header_string, message); // Flawfinder: ignore (false alarm, format is constant) } fflush(stdout); fflush(stderr); } } singularity-2.4.2/src/util/suid.h0000644000175000017500000000302613211621077015711 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_SUID_H_ #define __SINGULARITY_SUID_H_ extern int singularity_suid_init(char **argv); extern int singularity_suid_enabled(void); #endif /* __SINGULARITY_SUID_H_ */ singularity-2.4.2/src/util/daemon.h0000644000175000017500000000061613211621077016212 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * This software is licensed under a 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * */ #ifndef __SINGULARITY_DAEMON_H_ #define __SINGULARITY_DAEMON_H_ void singularity_daemon_init(void); #endif singularity-2.4.2/src/util/signal.c0000644000175000017500000000444013211621077016216 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * This software is licensed under a 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/signal.h" #include "util/fork.h" static sigset_t old_mask; static sigset_t sig_mask; static const int all_signals[] = { SIGHUP, SIGINT, SIGQUIT, SIGTRAP, SIGIOT, SIGUSR1, SIGUSR2, SIGPIPE, SIGALRM, SIGTERM, SIGSTKFLT, SIGCHLD, SIGCONT, SIGTSTP, SIGTTIN, SIGTTOU, SIGURG, SIGXCPU, SIGXFSZ, SIGVTALRM, SIGPROF, SIGWINCH, SIGIO, SIGPOLL, SIGPWR, SIGSYS }; static void handle_sig_sigchld(siginfo_t *siginfo) { while(1) { if ( waitpid(-1, NULL, WNOHANG) <= 0 ) break; } } static void handle_sig_generic(siginfo_t *siginfo) { singularity_message(DEBUG, "Generic sig received: %d\n", siginfo->si_signo); if ( siginfo->si_signo != SIGALRM && siginfo->si_signo != SIGCONT ) { kill(-1, siginfo->si_signo); } } void singularity_install_signal_handler() { int i = 0; singularity_message(DEBUG, "Creating signal handler\n"); sigemptyset(&sig_mask); while( all_signals[i] != 0 ) { sigaddset(&sig_mask, all_signals[i]); ++i; } if ( -1 == sigprocmask(SIG_SETMASK, &sig_mask, &old_mask) ) { singularity_message(ERROR, "Unable to block signals: %s\n", strerror(errno)); ABORT(255); } } /* Never returns. Will always read from sig_fd and wait for signal events */ int singularity_handle_signals(siginfo_t *siginfo) { if ( sigwaitinfo(&sig_mask, siginfo) < 0 ) { singularity_message(ERROR, "Unable to get siginfo: %s\n", strerror(errno)); return(-1); } if ( siginfo->si_signo == SIGCHLD ) { handle_sig_sigchld(siginfo); } else { handle_sig_generic(siginfo); } return(0); } void singularity_unblock_signals() { sigprocmask(SIG_SETMASK, &old_mask, NULL); } singularity-2.4.2/src/util/mount.h0000644000175000017500000000167213211621077016114 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * See the COPYRIGHT.md file at the top-level directory of this distribution and at * https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. * * This file is part of the Singularity Linux container project. It is subject to the license * terms in the LICENSE.md file found in the top-level directory of this distribution and * at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part * of Singularity, including this file, may be copied, modified, propagated, or distributed * except according to the terms contained in the LICENSE.md file. * */ #ifndef __MOUNT_H_ #define __MOUNT_H_ int singularity_mount(const char *source, const char *target, const char *filesystemtype, unsigned long mountflags, const void *data); int check_mounted(char *mountpoint); #endif /* __MOUNT_H_ */ singularity-2.4.2/src/util/suid.c0000644000175000017500000001230313211621077015702 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/message.h" #include "util/file.h" #include "util/registry.h" #include "util/config_parser.h" #include "util/privilege.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif #ifndef LIBEXECDIR #error LIBEXECDIR not defined #endif int singularity_suid_init(char **argv) { #ifdef SINGULARITY_SUID singularity_message(VERBOSE2, "Running SUID program workflow\n"); singularity_message(VERBOSE2, "Checking program has appropriate permissions\n"); if ( ( is_owner("/proc/self/exe", 0) < 0 ) || ( is_suid("/proc/self/exe") < 0 ) ) { char *path = (char*) malloc(PATH_MAX); int len = readlink("/proc/self/exe", path, PATH_MAX - 1); // Flawfinder: ignore (TOCTOU not an issue here) if ( len <= 0 ) { singularity_abort(255, "Could not obtain link target of self\n"); } if ( len == PATH_MAX - 1 ) { singularity_abort(255, "Link length error!\n"); } path[len] = '\0'; singularity_message(ERROR, "Installation error, run the following commands as root to fix:\n"); singularity_message(ERROR, " sudo chown root:root %s\n", path); singularity_message(ERROR, " sudo chmod 4755 %s\n", path); if ( getuid() == 0 ) { singularity_message(INFO, "\n"); } else { ABORT(255); } } singularity_message(VERBOSE2, "Checking configuration file is properly owned by root\n"); if ( is_owner(joinpath(SYSCONFDIR, "/singularity/singularity.conf"), 0 ) < 0 ) { singularity_abort(255, "Running in privileged mode, root must own the Singularity configuration file: %s\n", joinpath(SYSCONFDIR, "/singularity/singularity.conf")); } singularity_message(VERBOSE2, "Checking if singularity.conf allows us to run as suid\n"); if ( ( singularity_config_get_bool(ALLOW_SETUID) <= 0 ) || ( singularity_registry_get("NOSUID") != NULL ) ) { char *self; char *self_tail; self = (char *) malloc(PATH_MAX); if ( readlink("/proc/self/exe", self, PATH_MAX) <= 0 ) { // Flawfinder: ignore (TOCTOU not an issue here) singularity_message(ERROR, "Could not dereference our own program name\n"); ABORT(255); } if ( ( self_tail = strstr(self, "-suid") ) == NULL ) { singularity_message(ERROR, "Could not identify non-SUID operation path: %s\n", self); ABORT(255); } *self_tail = '\0'; if ( is_exec(self) == 0 ) { singularity_message(VERBOSE, "Invoking non-SUID program flow: %s\n", self); argv[0] = strdup(self); singularity_priv_drop_perm(); execv(argv[0], argv); // Flawfinder: ignore (all covered with sand) singularity_message(ERROR, "Failed exec'ing non-SUID program flow: %s\n", strerror(errno)); ABORT(255); } else { singularity_message(ERROR, "Could not locate non-SUID program flow: %s\n", self); ABORT(255); } singularity_message(ERROR, "We never should have gotten here...\n"); ABORT(255); } if ( geteuid() != 0 ) { singularity_message(ERROR, "Singularity is not running with appropriate privileges!\n"); singularity_message(ERROR, "Check installation path is not mounted with 'nosuid', and/or consult manual.\n"); ABORT(255); } #else singularity_message(VERBOSE, "Running NON-SUID program workflow\n"); singularity_message(DEBUG, "Checking program has appropriate permissions\n"); if ( is_suid("/proc/self/exe") >= 0 ) { singularity_message(ERROR, "This program must **NOT** be SUID\n"); ABORT(255); } #endif /* SINGULARITY_SUID */ return(0); } int singularity_suid_enabled(void) { if ( is_owner("/proc/self/exe", 0) < 0 ) { singularity_message(DEBUG, "Executable is not root owned\n"); return(-1); } if ( is_suid("/proc/self/exe") < 0 ) { singularity_message(DEBUG, "Executable is not SUID\n"); return(-1); } return(1); } singularity-2.4.2/src/util/setns.c0000644000175000017500000000251413211621077016075 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * See the COPYRIGHT.md file at the top-level directory of this distribution and at * https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. * * This file is part of the Singularity Linux container project. It is subject to the license * terms in the LICENSE.md file found in the top-level directory of this distribution and * at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part * of Singularity, including this file, may be copied, modified, propagated, or distributed * except according to the terms contained in the LICENSE.md file. * */ #define _GNU_SOURCE #include #include #include #include "util/message.h" #if defined (SINGULARITY_NO_SETNS) && defined (SINGULARITY_SETNS_SYSCALL) #include "util/setns.h" int setns(int fd, int nstype) { singularity_message(DEBUG, "Using syscall() wrapped __NR_setns\n"); return syscall(__NR_setns, fd, nstype); } #elif defined (SINGULARITY_NO_SETNS) && !defined (SINGULARITY_SETNS_SYSCALL) int setns(int fd, int nstype) { singularity_message(VERBOSE, "setns() not supported at compile time by kernel at time of building\n"); errno = ENOSYS; return -1; } #endif /* SINGULARITY_NO_SETNS && SINGULARITY_SETNS_SYSCALL */ singularity-2.4.2/src/util/Makefile.am0000644000175000017500000000140113211621077016623 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in config.h config.h.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la *.o AM_CFLAGS = -Wall -fpie -fPIC AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) EXTRA_DIST = cleanupd.c \ cleanupd.h \ config_parser.c \ config_parser.h \ daemon.c \ daemon.h \ file.c \ file.h \ fork.c \ fork.h \ message.c \ message.h \ mount.c \ mount.h \ privilege.c \ privilege.h \ registry.c \ registry.h \ sessiondir.c \ sessiondir.h \ suid.c \ suid.h \ util.c \ util.h \ setns.c \ setns.h \ signal.c \ signal.h \ config_defaults.h singularity-2.4.2/src/util/file.h0000644000175000017500000000341313211621077015664 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __FILE_H_ #define __FILE_H_ char *file_id(char *path); char *file_devino(char *path); #include int chk_perms(char *path, mode_t mode); int chk_mode(char *path, mode_t mode, mode_t mask); int is_file(char *path); int is_fifo(char *path); int is_link(char *path); int is_dir(char *path); int is_exec(char *path); int is_write(char *path); int is_suid(char *path); int is_owner(char *path, uid_t uid); int is_blk(char *path); int is_chr(char *path); int s_mkpath(char *dir, mode_t mode); int s_rmdir(char *dir); int copy_file(char * source, char * dest); char *filecat(char *path); int fileput(char *path, char *string); int filelock(const char *const filepath, int *const fdptr); char *basedir(char *dir); #endif singularity-2.4.2/src/util/util.c0000644000175000017500000002650413211621077015723 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _XOPEN_SOURCE 500 // For nftw #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" #include "util/registry.h" char *envar_get(char *name, char *allowed, int len) { char *ret; char *env = getenv(name); // Flawfinder: ignore int count; singularity_message(VERBOSE2, "Checking input from environment: '%s'\n", name); singularity_message(DEBUG, "Checking environment variable is defined: %s\n", name); if ( env == NULL ) { singularity_message(VERBOSE2, "Environment variable is NULL: %s\n", name); return(NULL); } singularity_message(DEBUG, "Checking environment variable length (<= %d): %s\n", len, name); if ( strlength(env, len+1) > len) { singularity_message(ERROR, "Input length of '%s' is larger then allowed: %d\n", name, len); ABORT(255); } singularity_message(DEBUG, "Checking environment variable has allowed characters: %s\n", name); ret = (char *) malloc(len+1); for(count=0; count <= len && env[count] != '\0'; count++) { int test_char = env[count]; int c, success = 0; if ( isalnum(test_char) > 0 ) { success = 1; } else { if ( allowed != NULL ) { for (c=0; allowed[c] != '\0'; c++) { if ( test_char == allowed[c] ) { success = 1; continue; } } } } if ( success == 0 ) { singularity_message(ERROR, "Illegal input character '%c' in: '%s=%s'\n", test_char, name, env); ABORT(255); } ret[count] = test_char; } ret[count] = '\0'; singularity_message(VERBOSE2, "Obtained input from environment '%s' = '%s'\n", name, ret); return(ret); } int envar_defined(char *name) { singularity_message(DEBUG, "Checking if environment variable is defined: %s\n", name); if ( getenv(name) == NULL ) { // Flawfinder: ignore singularity_message(VERBOSE2, "Environment variable is undefined: %s\n", name); return(-1); } singularity_message(VERBOSE2, "Environment variable is defined: %s\n", name); return(0); } char *envar_path(char *name) { singularity_message(DEBUG, "Checking environment variable is valid path: '%s'\n", name); return(envar_get(name, "/._+-=,:@", PATH_MAX)); } int envar_set(char *key, char *value, int overwrite) { if ( key == NULL ) { singularity_message(VERBOSE2, "Not setting envar, null key\n"); return(-1); } if ( value == NULL ) { singularity_message(DEBUG, "Unsetting environment variable: %s\n", key); return(unsetenv(key)); } singularity_message(DEBUG, "Setting environment variable: '%s' = '%s'\n", key, value); return(setenv(key, value, overwrite)); } int intlen(int input_int) { unsigned int len = 1; int input = input_int; while (input /= 10) { len ++; } return(len); } char *uppercase(char *string) { int len = strlength(string, 4096); char *upperkey = strdup(string); int i = 0; while ( i <= len ) { upperkey[i] = toupper(string[i]); i++; } singularity_message(DEBUG, "Transformed to uppercase: '%s' -> '%s'\n", string, upperkey); return(upperkey); } char *int2str(int num) { char *ret; ret = (char *) malloc(intlen(num) + 1); snprintf(ret, intlen(num) + 1, "%d", num); // Flawfinder: ignore return(ret); } char *joinpath(const char * path1, const char * path2_in) { if ( path1 == NULL ) { singularity_message(ERROR, "joinpath() called with NULL path1\n"); ABORT(255); } if ( path2_in == NULL ) { singularity_message(ERROR, "joinpath() called with NULL path2\n"); ABORT(255); } const char *path2 = path2_in; char *tmp_path1 = strdup(path1); int path1_len = strlength(tmp_path1, 4096); char *ret = NULL; if ( tmp_path1[path1_len - 1] == '/' ) { tmp_path1[path1_len - 1] = '\0'; } if ( path2[0] == '/' ) { path2++; } size_t ret_pathlen = strlength(tmp_path1, PATH_MAX) + strlength(path2, PATH_MAX) + 2; ret = (char *) malloc(ret_pathlen); if (snprintf(ret, ret_pathlen, "%s/%s", tmp_path1, path2) >= ret_pathlen) { // Flawfinder: ignore singularity_message(ERROR, "Overly-long path name.\n"); ABORT(255); } return(ret); } char *strjoin(char *str1, char *str2) { char *ret; int len = strlength(str1, 2048) + strlength(str2, 2048) + 1; ret = (char *) malloc(len); if (snprintf(ret, len, "%s%s", str1, str2) >= len) { // Flawfinder: ignore singularity_message(ERROR, "Overly-long string encountered.\n"); ABORT(255); } return(ret); } void chomp_noline(char *str) { int len; int i; len = strlength(str, 4096); while ( str[0] == ' ' ) { for ( i = 1; i < len; i++ ) { str[i-1] = str[i]; } str[len] = '\0'; len--; } while ( str[len - 1] == ' ' ) { str[len - 1] = '\0'; len--; } } void chomp(char *str) { if (!str) {return;} int len; int i; len = strlength(str, 4096); // Trim leading whitespace by shifting array. i = 0; while ( isspace(str[i]) ) {i++;} if (i) { len -= i; memmove(str, str+i, len); str[len] = '\0'; } // Trim trailing whitespace and redefine NULL while ( str[len - 1] == ' ' ) { str[len - 1] = '\0'; len--; } // If str starts with a new line, there is nothing here if ( str[0] == '\n' ) { str[0] = '\0'; } if ( str[len - 1] == '\n' ) { str[len - 1] = '\0'; } } void chomp_comments(char *str) { if (!str) {return;} char* comment = strchr(str, '#'); if (comment) { *comment = '\0'; // terminate string at comment } chomp(str); } int strlength(const char *string, int max_len) { int len; for (len=0; string[len] && len < max_len; len++) { // Do nothing in the loop } return(len); } char *random_string(int length) { static const char characters[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"; char *ret; int i; int pid = getpid(); ret = (char *) malloc(length); srand(time(NULL) * pid); // Flawfinder: ignore (complete mathmetical randomness is not required) for (i = 0; i < length; ++i) { ret[i] = characters[rand() % (sizeof(characters) - 1)]; } ret[length] = '\0'; return(ret); } int str2int(const char *input_str, long int *output_num) { long int result; char *endptr; errno = 0; // Empty string is an error: if ( *input_str == '\0' ) { errno = EINVAL; return -1; } result = strtol(input_str, &endptr, 10); // In the case of overflow / underflow or (possibly) // no digits consumed. if (errno) {return -1;} if ( *endptr == '\0' ) { // All data was consumed. if (output_num) {*output_num = result;} return 0; } errno = EINVAL; return -1; } int envclean(void) { int retval = 0; char **env = environ; char **envclone; int i; int envlen = 0; for(i = 0; env[i] != 0; i++) { envlen++; } envclone = (char**) malloc(i * sizeof(char *)); for(i = 0; env[i] != 0; i++) { envclone[i] = strdup(env[i]); } for(i = 0; i < envlen; i++) { char *tok = NULL; char *key = NULL; key = strtok_r(envclone[i], "=", &tok); if ( (strcasecmp(key, "http_proxy") == 0) || (strcasecmp(key, "https_proxy") == 0) || (strcasecmp(key, "no_proxy") == 0) || (strcasecmp(key, "all_proxy") == 0) ) { singularity_message(DEBUG, "Leaving environment variable set: %s\n", key); } else { singularity_message(DEBUG, "Unsetting environment variable: %s\n", key); unsetenv(key); } } return(retval); } void free_tempfile(struct tempfile *tf) { if (fclose(tf->fp)) { singularity_message(ERROR, "Error while closing temp file %s\n", tf->filename); ABORT(255); } if (unlink(tf->filename) < 0) { singularity_message(ERROR, "Could not remove temp file %s\n", tf->filename); ABORT(255); } free(tf); } struct tempfile *make_tempfile(void) { int fd; struct tempfile *tf; tf = malloc(sizeof(struct tempfile)); if (tf == NULL) { singularity_message(ERROR, "Could not allocate memory for tempfile\n"); ABORT(255); } strncpy(tf->filename, "/tmp/vb.XXXXXXXXXX", sizeof(tf->filename) - 1); tf->filename[sizeof(tf->filename) - 1] = '\0'; if ((fd = mkstemp(tf->filename)) == -1 || (tf->fp = fdopen(fd, "w+")) == NULL) { if (fd != -1) { unlink(tf->filename); close(fd); } singularity_message(ERROR, "Could not create temp file\n"); ABORT(255); } return tf; } struct tempfile *make_logfile(char *label) { struct tempfile *tf; char *daemon = singularity_registry_get("DAEMON_NAME"); char *image = basename(singularity_registry_get("IMAGE")); tf = malloc(sizeof(struct tempfile)); if (tf == NULL) { singularity_message(ERROR, "Could not allocate memory for tempfile\n"); ABORT(255); } if ( snprintf(tf->filename, sizeof(tf->filename) - 1, "/tmp/%s.%s.%s.XXXXXX", image, daemon, label) > sizeof(tf->filename) - 1 ) { singularity_message(ERROR, "Label string too long\n"); ABORT(255); } tf->filename[sizeof(tf->filename) - 1] = '\0'; if ( (tf->fd = mkstemp(tf->filename)) == -1 || (tf->fp = fdopen(tf->fd, "w+")) == NULL ) { if (tf->fd != -1) { unlink(tf->filename); close(tf->fd); } singularity_message(DEBUG, "Could not create log file, running silently\n"); return(NULL); } singularity_message(DEBUG, "Logging container's %s at: %s\n", label, tf->filename); return(tf); } singularity-2.4.2/src/util/fork.h0000644000175000017500000000537413211621077015716 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * Copyright (c) 2016, Brian Bockelman. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_FORK_H_ #define __SINGULARITY_FORK_H_ // SINGULARITY_FORK() // Wrap the fork() system call and create the necessary communication // pipes and signal handlers so that signals are correctly passed around // between children and parents. pid_t singularity_fork(unsigned int flags); // SINGLARITY_FORK_RUN() // Fork() and automatically have the parent wait on the child while // allowing the child to continue on through the code path. The parent // will automatically wait in the background until the child exits, and // then the parent will also exit with the same exit code as the parent. // Similar to singularity_fork() above, this will maintain the proper // communication channels for signal handling. void singularity_fork_run(unsigned int flags); // SINGULARITY_FORK_EXEC // Fork and exec a child system command, wait for it to return, and then // return with the appropriate exit value. int singularity_fork_exec(unsigned int flags, char **argv); // SINGULARITY_FORK_DAEMONIZE_WAIT // Fork and wait for the child to send the go-ahead signal via // singularity_signal_go_ahead() before exiting. void singularity_fork_daemonize(unsigned int flags); // SINGULARITY_SIGNAL_GO_AHEAD() // Send a go-ahead signal via pipes to the partner process // to indicate that it is allowed to move forward. Requires // that prepare_fork() & prepare_pipes_[child/parent]() are // called first to work properly. int singularity_signal_go_ahead(int code); // SINGULARITY_WAIT_FOR_GO_AHEAD() // Wait for the go-ahead signal described above void singularity_wait_for_go_ahead(); #endif /* __SINGULARITY_FORK_H_ */ singularity-2.4.2/src/util/util.h0000644000175000017500000000477713211621077015740 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include "util/message.h" #ifndef __UTIL_H_ #define __UTIL_H_ #define TRUE 1 #define FALSE 0 struct tempfile { FILE *fp; int fd; char filename[64]; }; char *envar_get(char *name, char *allowed, int len); char *envar_path(char *name); int envar_defined(char *name); int envar_set(char *key, char *value, int overwrite); int intlen(int input); char *int2str(int num); char *joinpath(const char * path1, const char * path2); char *strjoin(char *str1, char *str2); char *uppercase(char *string); void chomp_noline(char *str); void chomp_comments(char *str); void chomp(char *str); int strlength(const char *string, int max_len); int envclean(void); char *random_string(int length); void free_tempfile(struct tempfile *tf); struct tempfile *make_tempfile(void); struct tempfile *make_logfile(char *label); // Given a const char * string containing a base-10 integer, // try to convert to an C integer. // This is a bit less error prone (and stricter!) than strtoll: // - Returns -1 on error and sets errno appropriately. // - On failure, output_num is not touched. // - On success, sets output_num to the parsed value (if output_num // is not null). // - If the whole string isn't consumed, then -1 is returned and // errno is set to EINVAL int str2int(const char *input_str, long int *output_num); struct passwd; #define ABORT(a) do {singularity_message(ABRT, "Retval = %d\n", a); exit(a);} while (0) #endif singularity-2.4.2/src/util/privilege.h0000644000175000017500000000413513211621077016735 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __PRIVILEGE_H_ #define __PRIVILEGE_H_ void singularity_priv_init(void); void singularity_priv_userns(void); void singularity_priv_escalate(void); void singularity_priv_drop(void); void singularity_priv_drop_perm(void); uid_t singularity_priv_getuid(void); gid_t singularity_priv_getgid(void); const gid_t *singularity_priv_getgids(); int singularity_priv_getgidcount(void); int singularity_priv_userns_enabled(void); int singularity_priv_is_suid(void); char *singularity_priv_home(void); char *singularity_priv_homedir(void); char *singularity_priv_getuser(void); // Escalate privileges to the 'singularity' user. void singularity_priv_escalate_singularity(void); // Returns 1 if invoking user is a member of `gid`, 0 otherwise int singularity_priv_has_gid(gid_t gid); // Returns the UID of the singularity user. uid_t singularity_priv_singularity_uid(); // Returns the GID of the singularity user. gid_t singularity_priv_singularity_gid(); #endif /* __PRIVILEGE_H_ */ singularity-2.4.2/src/util/cleanupd.c0000644000175000017500000001045313211621077016535 0ustar mehdimehdi/* * Copyright (c) 2016, Brian Bockelman. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include "util/message.h" #include "util/util.h" #include "util/file.h" #include "util/registry.h" #include "util/fork.h" #include "util/privilege.h" #ifndef LIBEXECDIR #error LIBEXECDIR not defined #endif char *trigger = NULL; int singularity_cleanupd(void) { char *cleanup_dir = singularity_registry_get("CLEANUPDIR"); int trigger_fd = -1; singularity_registry_set("CLEANUPD_FD", "-1"); if ( singularity_registry_get("DAEMON_JOIN") ) { singularity_message(ERROR, "Internal Error - This function should not be called when joining an instance\n"); } if ( ( singularity_registry_get("NOSESSIONCLEANUP") != NULL ) || ( singularity_registry_get("NOCLEANUP") != NULL ) ) { singularity_message(DEBUG, "Not running a cleanup thread, requested not to\n"); return(0); } if ( cleanup_dir == NULL ) { singularity_message(DEBUG, "Not running a cleanup thread, no 'SINGULARITY_CLEANUPDIR' defined\n"); return(0); } if ( is_dir(cleanup_dir) != 0 ) { singularity_message(WARNING, "Cleanup path must be a directory: %s\n", cleanup_dir); return(-1); } if ( trigger == NULL ) { char *rand = NULL; if ( ( rand = random_string(8) ) == NULL ) { singularity_message(ERROR, "Failed obtaining a random string for temporary cleanup trigger\n"); ABORT(255); } trigger = strjoin("/tmp/.singularity-cleanuptrigger.", rand); singularity_message(DEBUG, "Creating new cleanup trigger file: %s\n", trigger); singularity_message(DEBUG, "Opening cleanup trigger file: %s\n", trigger); if ( ( trigger_fd = open(trigger, O_WRONLY | O_CREAT, 00644) ) < 0 ) { singularity_message(ERROR, "Failed opening trigger file %s: %s\n", trigger, strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Gaining an exclusive flock on FD %d\n", trigger_fd); if ( flock(trigger_fd, LOCK_EX | LOCK_NB) < 0 ) { singularity_message(ERROR, "Could not obtain flock() on cleanup trigger file\n"); ABORT(255); } singularity_registry_set("CLEANUPD_FD", int2str(trigger_fd)); } else { singularity_message(DEBUG, "Using existing cleanup trigger file: %s\n", trigger); } int child = fork(); if ( child == 0 ) { close(trigger_fd); singularity_message(VERBOSE, "Exec'ing cleanupd thread: %s\n", joinpath(LIBEXECDIR, "/singularity/bin/cleanupd")); envar_set("SINGULARITY_CLEANUPDIR", cleanup_dir, 1); envar_set("SINGULARITY_CLEANUPTRIGGER", trigger, 1); execl(joinpath(LIBEXECDIR, "/singularity/bin/cleanupd"), "Singularity: cleanup", NULL); // Flawfinder: ignore (on top of old smokey...) singularity_message(ERROR, "Exec of cleanupd process failed %s: %s\n", joinpath(LIBEXECDIR, "/singularity/bin/cleanupd"), strerror(errno)); exit(255); } else if ( child > 0 ) { int tmpstatus; waitpid(child, &tmpstatus, 0); if ( WEXITSTATUS(tmpstatus) != 0 ) { ABORT(255); } } return(0); } singularity-2.4.2/src/util/signal.h0000644000175000017500000000075313211621077016226 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * This software is licensed under a 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * */ #ifndef __SINGULARITY_SIGNAL_H_ #define __SINGULARITY_SIGNAL_H_ void singularity_install_signal_handler(); int singularity_handle_signals(siginfo_t *siginfo); void singularity_unblock_signals(); #endif singularity-2.4.2/src/util/message.h0000644000175000017500000000506513211621077016376 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __SINGULARITY_MESSAGE_H_ #define __SINGULARITY_MESSAGE_H_ #define ABRT -4 #define ERROR -3 #define WARNING -2 #define LOG -1 #define INFO 1 #define VERBOSE 2 #define VERBOSE1 2 #define VERBOSE2 3 #define VERBOSE3 4 #define DEBUG 5 #define ANSI_COLOR_RED "\x1b[31m" #define ANSI_COLOR_GREEN "\x1b[32m" #define ANSI_COLOR_YELLOW "\x1b[33m" #define ANSI_COLOR_BLUE "\x1b[34m" #define ANSI_COLOR_MAGENTA "\x1b[35m" #define ANSI_COLOR_CYAN "\x1b[36m" #define ANSI_COLOR_GRAY "\x1b[37m" #define ANSI_COLOR_LIGHTGRAY "\x1b[90m" #define ANSI_COLOR_LIGHTRED "\x1b[91m" #define ANSI_COLOR_LIGHTGREEN "\x1b[92m" #define ANSI_COLOR_LIGHTYELLOW "\x1b[93m" #define ANSI_COLOR_LIGHTBLUE "\x1b[94m" #define ANSI_COLOR_LIGHTMAGENTA "\x1b[95m" #define ANSI_COLOR_LIGHTCYAN "\x1b[96m" #define ANSI_COLOR_RESET "\x1b[0m" int singularity_message_level(void); void _singularity_message(int level, const char *function, const char *file, int line, char *format, ...) __attribute__ ((__format__(printf, 5, 6))); // Flawfinder: ignore #define singularity_message(a,b...) _singularity_message(a, __func__, __FILE__, __LINE__, b) #define singularity_abort(a,b...) do {_singularity_message(ABRT, __func__, __FILE__, __LINE__, b); _singularity_message(ABRT, __func__, __FILE__, __LINE__, "Retval = %d\n", a); exit(a);} while(0) #endif /*__SINGULARITY_MESSAGE_H_ */ singularity-2.4.2/src/util/privilege.c0000644000175000017500000004566313211621077016743 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "util/privilege.h" #include "util/message.h" #include "util/config_parser.h" static struct PRIV_INFO { int ready; uid_t uid; gid_t gid; gid_t *gids; size_t gids_count; int userns_ready; uid_t orig_uid; uid_t orig_gid; pid_t orig_pid; char *home; char *homedir; char *username; int dropped_groups; int target_mode; // Set to 1 if we are running in "target mode" (admin specifies UID/GID) } uinfo; // Cache of UID / GID of the 'singularity' user. static struct SINGULARITY_PRIV_INFO { int ready; uid_t uid; gid_t gid; } sinfo; void singularity_priv_init(void) { long int target_uid = -1; long int target_gid = -1; memset(&uinfo, '\0', sizeof(uinfo)); memset(&sinfo, '\0', sizeof(sinfo)); char *home_tmp = singularity_registry_get("HOME"); char *target_uid_str = singularity_registry_get("TARGET_UID"); char *target_gid_str = singularity_registry_get("TARGET_GID"); struct passwd *pwent; singularity_message(DEBUG, "Initializing user info\n"); if ( target_uid_str && !target_gid_str ) { singularity_message(ERROR, "A target UID is set (%s) but a target GID is not set (SINGULARITY_TARGET_GID). Both must be specified.\n", target_uid_str); ABORT(255); } if (target_uid_str) { if ( -1 == str2int(target_uid_str, &target_uid) ) { singularity_message(ERROR, "Unable to convert target UID (%s) to integer: %s\n", target_uid_str, strerror(errno)); ABORT(255); } if (target_uid < 500) { singularity_message(ERROR, "Target UID (%ld) must be 500 or greater to avoid system users.\n", target_uid); ABORT(255); } if (target_uid > UINT_MAX) { // Avoid anything greater than the traditional overflow UID. singularity_message(ERROR, "Target UID (%ld) cannot be greater than UINT_MAX.\n", target_uid); ABORT(255); } } if ( !target_uid_str && target_gid_str ) { singularity_message(ERROR, "A target GID is set (%s) but a target UID is not set (SINGULARITY_TARGET_UID). Both must be specified.\n", target_gid_str); ABORT(255); } if (target_gid_str) { if ( -1 == str2int(target_gid_str, &target_gid) ) { singularity_message(ERROR, "Unable to convert target GID (%s) to integer: %s\n", target_gid_str, strerror(errno)); ABORT(255); } if (target_gid < 500) { singularity_message(ERROR, "Target GID (%ld) must be 500 or greater to avoid system groups.\n", target_gid); ABORT(255); } if (target_gid > UINT_MAX) { // Avoid anything greater than the traditional overflow GID. singularity_message(ERROR, "Target GID (%ld) cannot be greater than UINT_MAX.\n", target_gid); ABORT(255); } } if ( (target_uid >= 500) && (target_gid >= 500) ) { if ( getuid() != 0 ) { singularity_message(ERROR, "Unable to use TARGET UID/GID mode when not running as root.\n"); ABORT(255); } uinfo.target_mode = 1; uinfo.uid = target_uid; uinfo.gid = target_gid; uinfo.gids_count = 0; uinfo.gids = NULL; } else { uinfo.uid = getuid(); uinfo.gid = getgid(); uinfo.gids_count = getgroups(0, NULL); uinfo.gids = (gid_t *) malloc(sizeof(gid_t) * uinfo.gids_count); if ( getgroups(uinfo.gids_count, uinfo.gids) < 0 ) { singularity_message(ERROR, "Could not obtain current supplementary group list: %s\n", strerror(errno)); ABORT(255); } } if ( ( pwent = getpwuid(uinfo.uid) ) == NULL ) { singularity_message(VERBOSE, "Failed obtaining user information for uid: %i\n", uinfo.uid); uinfo.username = strdup("NULL"); } else { if ( ( uinfo.username = strdup(pwent->pw_name) ) != NULL ) { singularity_message(DEBUG, "Set the calling user's username to: %s\n", uinfo.username); } else { singularity_message(ERROR, "Failed obtaining the calling user's username\n"); ABORT(255); } } singularity_message(DEBUG, "Marking uinfo structure as ready\n"); uinfo.ready = 1; singularity_message(DEBUG, "Obtaining home directory\n"); if ( home_tmp != NULL ) { char *colon = strchr(home_tmp, ':'); if ( colon == NULL ) { uinfo.home = strdup(home_tmp); uinfo.homedir = uinfo.home; singularity_message(VERBOSE2, "Set home and homedir (via SINGULARITY_HOME) to: %s\n", uinfo.home); } else { *colon = '\0'; uinfo.home = strdup(&colon[1]); singularity_message(VERBOSE2, "Set home (via SINGULARITY_HOME) to: %s\n", uinfo.home); uinfo.homedir = strdup(home_tmp); singularity_message(VERBOSE2, "Set the home directory (via SINGULARITY_HOME) to: %s\n", uinfo.homedir); } } else if ( pwent != NULL ) { if ( ( uinfo.home = strdup(pwent->pw_dir) ) != NULL ) { singularity_message(VERBOSE2, "Set home (via getpwuid()) to: %s\n", uinfo.home); uinfo.homedir = uinfo.home; } else { singularity_message(ERROR, "Failed obtaining the calling user's home directory\n"); ABORT(255); } } else { uinfo.home = strdup("/"); uinfo.homedir = uinfo.home; } return; } void singularity_priv_userns(void) { singularity_message(VERBOSE, "Invoking the user namespace\n"); if ( singularity_config_get_bool(ALLOW_USER_NS) <= 0 ) { singularity_message(VERBOSE, "Not virtualizing USER namespace by configuration: 'allow user ns' = no\n"); } else if ( getuid() == 0 ) { singularity_message(VERBOSE, "Not virtualizing USER namespace: running as root\n"); } else if ( singularity_priv_is_suid() == 0 ) { singularity_message(VERBOSE, "Not virtualizing USER namespace: running as SUID\n"); } else { uid_t uid = singularity_priv_getuid(); gid_t gid = singularity_priv_getgid(); singularity_message(DEBUG, "Attempting to virtualize the USER namespace\n"); if ( unshare(CLONE_NEWUSER) != 0 ) { singularity_message(ERROR, "Failed invoking the NEWUSER namespace runtime: %s\n", strerror(errno)); ABORT(255); // If we are configured to use CLONE_NEWUSER, we should abort if that fails } singularity_message(DEBUG, "Enabled user namespaces\n"); { singularity_message(DEBUG, "Setting setgroups to: 'deny'\n"); char *map_file = (char *) malloc(PATH_MAX); snprintf(map_file, PATH_MAX-1, "/proc/%d/setgroups", getpid()); // Flawfinder: ignore FILE *map_fp = fopen(map_file, "w+"); // Flawfinder: ignore if ( map_fp != NULL ) { singularity_message(DEBUG, "Updating setgroups: %s\n", map_file); fprintf(map_fp, "deny\n"); if ( fclose(map_fp) < 0 ) { singularity_message(ERROR, "Failed to write deny to setgroup file %s: %s\n", map_file, strerror(errno)); ABORT(255); } } else { singularity_message(ERROR, "Could not write info to setgroups: %s\n", strerror(errno)); ABORT(255); } free(map_file); } { singularity_message(DEBUG, "Setting GID map to: '%i %i 1'\n", gid, gid); char *map_file = (char *) malloc(PATH_MAX); snprintf(map_file, PATH_MAX-1, "/proc/%d/gid_map", getpid()); // Flawfinder: ignore FILE *map_fp = fopen(map_file, "w+"); // Flawfinder: ignore if ( map_fp != NULL ) { singularity_message(DEBUG, "Updating the parent gid_map: %s\n", map_file); fprintf(map_fp, "%i %i 1\n", gid, gid); if ( fclose(map_fp) < 0 ) { singularity_message(ERROR, "Failed to write to GID map %s: %s\n", map_file, strerror(errno)); ABORT(255); } } else { singularity_message(ERROR, "Could not write parent info to gid_map: %s\n", strerror(errno)); ABORT(255); } free(map_file); } { singularity_message(DEBUG, "Setting UID map to: '%i %i 1'\n", uid, uid); char *map_file = (char *) malloc(PATH_MAX); snprintf(map_file, PATH_MAX-1, "/proc/%d/uid_map", getpid()); // Flawfinder: ignore FILE *map_fp = fopen(map_file, "w+"); // Flawfinder: ignore if ( map_fp != NULL ) { singularity_message(DEBUG, "Updating the parent uid_map: %s\n", map_file); fprintf(map_fp, "%i %i 1\n", uid, uid); if ( fclose(map_fp) < 0 ) { singularity_message(ERROR, "Failed to write to UID map %s: %s\n", map_file, strerror(errno)); ABORT(255); } } else { singularity_message(ERROR, "Could not write parent info to uid_map: %s\n", strerror(errno)); ABORT(255); } free(map_file); } uinfo.userns_ready = 1; } singularity_message(DEBUG, "Returning singularity_priv_init(void)\n"); } void singularity_priv_escalate(void) { if ( uinfo.ready != 1 ) { singularity_message(ERROR, "User info is not available\n"); ABORT(255); } if ( uinfo.userns_ready == 1 ) { singularity_message(DEBUG, "Not escalating privileges, user namespace enabled\n"); return; } if ( uinfo.uid == 0 ) { singularity_message(DEBUG, "Running as root, not changing privileges\n"); return; } singularity_message(DEBUG, "Temporarily escalating privileges (U=%d)\n", getuid()); if ( ( seteuid(0) < 0 ) || ( setegid(0) < 0 ) ) { singularity_message(ERROR, "The feature you are requesting requires privilege you do not have\n"); ABORT(255); } singularity_message(DEBUG, "Clearing supplementary GIDs.\n"); if ( setgroups(0, NULL) == -1 ) { singularity_message(ERROR, "Unable to clear the supplementary group IDs: %s (errno=%d).\n", strerror(errno), errno); ABORT(255); } uinfo.dropped_groups = 1; } void singularity_priv_drop(void) { if ( uinfo.ready != 1 ) { singularity_message(ERROR, "User info is not available\n"); ABORT(255); } if ( uinfo.userns_ready == 1 ) { singularity_message(DEBUG, "Not dropping privileges, user namespace enabled\n"); return; } if ( uinfo.uid == 0 ) { singularity_message(DEBUG, "Running as root, not changing privileges\n"); return; } // If we escalated privileges to user singularity (!=0), we need to set the EUID back to 0 first before // we can switch back to the invoking user. if ( (geteuid() != 0) && (seteuid(0) < 0) ) { singularity_message(VERBOSE, "Could not restore EUID to 0: %s (errno=%d).\n", strerror(errno), errno); } singularity_message(DEBUG, "Dropping privileges to UID=%d, GID=%d (%lu supplementary GIDs)\n", uinfo.uid, uinfo.gid, uinfo.gids_count); singularity_message(DEBUG, "Restoring supplementary groups\n"); if ( uinfo.dropped_groups && (setgroups(uinfo.gids_count, uinfo.gids) < 0) ) { singularity_message(ERROR, "Could not reset supplementary group list: %s\n", strerror(errno)); ABORT(255); } uinfo.dropped_groups = 0; if ( setegid(uinfo.gid) < 0 ) { singularity_message(ERROR, "Could not drop effective group privileges to gid %d: %s\n", uinfo.gid, strerror(errno)); ABORT(255); } if ( seteuid(uinfo.uid) < 0 ) { singularity_message(ERROR, "Could not drop effective user privileges to uid %d: %s\n", uinfo.uid, strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Confirming we have correct UID/GID\n"); if ( getgid() != uinfo.gid ) { if ( uinfo.target_mode && getgid() != 0 ) { singularity_message(ERROR, "Non-zero real GID for target mode: %d\n", getgid()); ABORT(255); } else if ( !uinfo.target_mode ) { singularity_message(ERROR, "Failed to drop effective group privileges to gid %d (currently %d)\n", uinfo.gid, getgid()); ABORT(255); } } if ( getuid() != uinfo.uid ) { if ( uinfo.target_mode && getuid() != 0 ) { singularity_message(ERROR, "Non-zero real UID for target mode: %d\n", getuid()); ABORT(255); } else if ( !uinfo.target_mode ) { singularity_message(ERROR, "Failed to drop effective user privileges to uid %d (currently %d)\n", uinfo.uid, getuid()); ABORT(255); } } } void singularity_priv_drop_perm(void) { singularity_message(DEBUG, "Called singularity_priv_drop_perm(void)\n"); if ( uinfo.ready != 1 ) { singularity_message(ERROR, "User info is not available\n"); ABORT(255); } if ( uinfo.userns_ready == 1 ) { singularity_message(VERBOSE2, "User namespace called, no privilges to drop\n"); return; } if ( uinfo.uid == 0 ) { singularity_message(VERBOSE2, "Calling user is root, no privileges to drop\n"); return; } singularity_message(DEBUG, "Escalating permissison so we can properly drop permission\n"); singularity_priv_escalate(); singularity_message(DEBUG, "Resetting supplementary groups\n"); if ( setgroups(uinfo.gids_count, uinfo.gids) < 0 ) { singularity_message(ERROR, "Could not reset supplementary group list (perm): %s\n", strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Dropping to group ID '%d'\n", uinfo.gid); if ( setgid(uinfo.gid) < 0 ) { singularity_message(ERROR, "Could not dump group privileges: %s\n", strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Dropping real and effective privileges to GID = '%d'\n", uinfo.gid); if ( setregid(uinfo.gid, uinfo.gid) < 0 ) { singularity_message(ERROR, "Could not dump real and effective group privileges: %s\n", strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Dropping real and effective privileges to UID = '%d'\n", uinfo.uid); if ( setreuid(uinfo.uid, uinfo.uid) < 0 ) { singularity_message(ERROR, "Could not dump real and effective user privileges: %s\n", strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Confirming we have correct GID\n"); if ( getgid() != uinfo.gid ) { singularity_message(ERROR, "Failed to drop effective group privileges to gid %d: %s\n", uinfo.gid, strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Confirming we have correct UID\n"); if ( getuid() != uinfo.uid ) { singularity_message(ERROR, "Failed to drop effective user privileges to uid %d: %s\n", uinfo.uid, strerror(errno)); ABORT(255); } #ifdef SINGULARITY_NO_NEW_PRIVS // Prevent the following processes to increase privileges singularity_message(DEBUG, "Setting NO_NEW_PRIVS to prevent future privilege escalations.\n"); if ( prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) != 0 ) { singularity_message(ERROR, "Could not set NO_NEW_PRIVS safeguard: %s\n", strerror(errno)); ABORT(255); } #else // SINGULARITY_NO_NEW_PRIVS singularity_message(VERBOSE2, "Not enabling NO_NEW_PRIVS flag due to lack of compile-time support.\n"); #endif singularity_message(DEBUG, "Finished dropping privileges\n"); } int singularity_priv_userns_enabled(void) { return uinfo.userns_ready; } /* Return 0 if program is SUID, -1 if not SUID */ int singularity_priv_is_suid(void) { if ( ( is_suid("/proc/self/exe") == 0 ) && ( is_owner("/proc/self/exe", 0) == 0) ) { return(0); } else { return(-1); } } char *singularity_priv_home(void) { if ( !uinfo.ready ) { singularity_message(ERROR, "Invoked before privilege info initialized!\n"); ABORT(255); } return(strdup(uinfo.home)); } char *singularity_priv_homedir(void) { if ( !uinfo.ready ) { singularity_message(ERROR, "Invoked before privilege info initialized!\n"); ABORT(255); } return(strdup(uinfo.homedir)); } char *singularity_priv_getuser(void) { if ( !uinfo.ready ) { singularity_message(ERROR, "Invoked before privilege info initialized!\n"); ABORT(255); } return uinfo.username; } uid_t singularity_priv_getuid(void) { if ( !uinfo.ready ) { singularity_message(ERROR, "Invoked before privilege info initialized!\n"); ABORT(255); } return uinfo.uid; } gid_t singularity_priv_getgid(void) { if ( !uinfo.ready ) { singularity_message(ERROR, "Invoked before privilege info initialized!\n"); ABORT(255); } return uinfo.gid; } const gid_t *singularity_priv_getgids(void) { if ( !uinfo.ready ) { singularity_message(ERROR, "Invoked before privilege info initialized!\n"); ABORT(255); } return uinfo.gids; } int singularity_priv_getgidcount(void) { if ( !uinfo.ready ) { singularity_message(ERROR, "Invoked before privilege info initialized!\n"); ABORT(255); } return uinfo.gids_count; } int singularity_priv_has_gid(gid_t gid) { if ( !uinfo.ready ) { singularity_message(ERROR, "Invoked singularity_priv_has_gid before privilege info initialized!\n"); ABORT(255); } int gid_idx; for (gid_idx=0; gid_idx #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "util/fork.h" #include "util/sessiondir.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif int main(int argc, char **argv) { int retval = 0; char *tar_cmd[4]; struct image_object image; struct image_object image_test; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_priv_init(); singularity_suid_init(argv); singularity_registry_init(); singularity_priv_drop(); if ( argc == 2 ) { singularity_registry_set("IMAGE", argv[1]); } image = singularity_image_init(singularity_registry_get("IMAGE")); // singularity_image_open(&image, O_RDWR); // // singularity_image_check(&image); if ( image.type != EXT3 ) { singularity_message(ERROR, "Import is only allowed on Singularity image files\n"); ABORT(255); } singularity_registry_set("WRITABLE", "1"); singularity_runtime_ns(SR_NS_MNT); // singularity_image_bind(&image); if ( image.loopdev == NULL ) { singularity_message(ERROR, "Bind failed to connect to image!\n"); ABORT(255); } singularity_image_mount(&image, singularity_runtime_rootfs(NULL)); // Check to make sure the image hasn't been swapped out by a race image_test = singularity_image_init(singularity_registry_get("IMAGE")); // singularity_image_open(&image_test, O_RDONLY); // singularity_image_check(&image_test); if ( image_test.type != EXT3 ) { singularity_message(ERROR, "Import is only allowed on Singularity image files\n"); ABORT(255); } if ( is_exec("/usr/bin/tar") == 0 ) { tar_cmd[0] = strdup("/usr/bin/tar"); } else if ( is_exec("/bin/tar") == 0 ) { tar_cmd[0] = strdup("/bin/tar"); } else { singularity_message(ERROR, "Could not locate the system version of 'tar'\n"); ABORT(255); } tar_cmd[1] = strdup("-xf"); tar_cmd[2] = strdup("-"); tar_cmd[3] = NULL; if ( chdir(singularity_runtime_rootfs(NULL)) != 0 ) { singularity_message(ERROR, "Could not change to working directory: %s\n", singularity_runtime_rootfs(NULL)); ABORT(255); } singularity_message(DEBUG, "Cleaning environment\n"); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing the environment\n"); ABORT(255); } singularity_priv_escalate(); singularity_message(VERBOSE, "Opening STDIN for tar stream\n"); retval = singularity_fork_exec(0, tar_cmd); singularity_priv_drop(); if ( retval != 0 ) { singularity_message(ERROR, "Tar did not return successful\n"); } return(retval); } singularity-2.4.2/src/builddef.c0000644000175000017500000001507313211621077015546 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/sessiondir.h" #include "./bootstrap-lib/include.h" #ifndef LIBEXECDIR #error LIBEXECDIR not defined #endif #ifndef BINDIR #error BINDIR not defined #endif #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif #define MAX_LINE_LEN 4096 int main(int argc, char **argv) { struct image_object image; FILE *bootdef_fp; char *line; char *builddef; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_registry_init(); singularity_priv_init(); singularity_message(INFO, "Sanitizing environment\n"); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing the environment\n"); ABORT(255); } singularity_registry_set("WRITABLE", "1"); if ( singularity_registry_get("WRITABLE") != NULL ) { singularity_message(VERBOSE3, "Instantiating writable container image object\n"); image = singularity_image_init(singularity_registry_get("IMAGE"), O_RDWR); } else { singularity_message(VERBOSE3, "Instantiating read only container image object\n"); image = singularity_image_init(singularity_registry_get("IMAGE"), O_RDONLY); } singularity_runtime_ns(SR_NS_MNT); singularity_image_mount(&image, CONTAINER_MOUNTDIR); builddef = singularity_registry_get("BUILDDEF"); if ( is_file(builddef) != 0 ) { singularity_message(ERROR, "Bootstrap definition file not found: %s\n", builddef); ABORT(255); } if ( ( bootdef_fp = fopen(builddef, "r") ) == NULL ) { singularity_message(ERROR, "Could not open bootstrap definition file %s: %s\n", builddef, strerror(errno)); ABORT(255); } line = (char *)malloc(MAX_LINE_LEN); while ( fgets(line, MAX_LINE_LEN, bootdef_fp) ) { char *bootdef_key; chomp_comments(line); // skip empty lines (do this after 'chomp') if (line[0] == '\0') { continue; } if ( line[0] == '%' ) { // We hit a section, stop parsing for keyword tags break; } else if ( ( bootdef_key = strtok(line, ":") ) != NULL ) { chomp(bootdef_key); char *bootdef_value; bootdef_value = strtok(NULL, "\n"); char empty[] = ""; if (bootdef_value == NULL) { bootdef_value = empty; } else { chomp(bootdef_value); } singularity_message(VERBOSE2, "Got bootstrap definition key/val '%s' = '%s'\n", bootdef_key, bootdef_value); if ( envar_defined(strjoin("SINGULARITY_DEFFILE_", uppercase(bootdef_key))) == 0 ) { singularity_message(ERROR, "Duplicate bootstrap definition key found: '%s'\n", bootdef_key); ABORT(255); } if ( strcasecmp(bootdef_key, "import") == 0 ) { // Do this again for an imported deffile bootstrap_keyval_parse(bootdef_value); } if ( strcasecmp(bootdef_key, "bootstrap") == 0 ) { singularity_registry_set("DRIVER", bootdef_value); } // Cool little feature, every key defined in def file is transposed // to environment envar_set(uppercase(bootdef_key), bootdef_value, 1); envar_set(strjoin("SINGULARITY_DEFFILE_", uppercase(bootdef_key)), bootdef_value, 1); } } free(line); fclose(bootdef_fp); envar_set("PATH", "/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin", 1); envar_set("SINGULARITY_ROOTFS", CONTAINER_MOUNTDIR, 1); envar_set("SINGULARITY_libexecdir", LIBEXECDIR, 1); envar_set("SINGULARITY_bindir", BINDIR, 1); envar_set("SINGULARITY_IMAGE", singularity_registry_get("IMAGE"), 1); envar_set("SINGULARITY_BUILDDEF", singularity_registry_get("BUILDDEF"), 1); envar_set("SINGULARITY_CHECKS", singularity_registry_get("CHECKS"), 1); envar_set("SINGULARITY_CHECKLEVEL", singularity_registry_get("CHECKLEVEL"), 1); envar_set("SINGULARITY_CHECKTAGS", singularity_registry_get("CHECKTAGS"), 1); envar_set("SINGULARITY_MESSAGELEVEL", singularity_registry_get("MESSAGELEVEL"), 1); envar_set("SINGULARITY_NOTEST", singularity_registry_get("NOTEST"), 1); envar_set("SINGULARITY_BUILDSECTION", singularity_registry_get("BUILDSECTION"), 1); envar_set("SINGULARITY_BUILDNOBASE", singularity_registry_get("BUILDNOBASE"), 1); envar_set("SINGULARITY_DOCKER_PASSWORD", singularity_registry_get("DOCKER_PASSWORD"), 1); envar_set("SINGULARITY_DOCKER_USERNAME", singularity_registry_get("DOCKER_USERNAME"), 1); envar_set("SINGULARITY_CACHEDIR", singularity_registry_get("CACHEDIR"), 1); envar_set("SINGULARITY_NOHTTPS", singularity_registry_get("NOHTTPS"), 1); envar_set("SINGULARITY_version", singularity_registry_get("VERSION"), 1); envar_set("HOME", singularity_priv_home(), 1); envar_set("LANG", "C", 1); char *bootstrap = joinpath(LIBEXECDIR, "/singularity/bootstrap-scripts/main-deffile.sh"); execl(bootstrap, bootstrap, NULL); //Flawfinder: ignore (Yes, yes, we know, and this is required) singularity_message(ERROR, "Exec of bootstrap code failed: %s\n", strerror(errno)); ABORT(255); return(0); } singularity-2.4.2/src/Makefile.am0000644000175000017500000000617713211621077015665 0ustar mehdimehdiSUBDIRS = lib action-lib bootstrap-lib util MAINTAINERCLEANFILES = Makefile.in config.h config.h.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie -fPIC AM_LDFLAGS = -pie AM_CPPFLAGS = -DBINDIR=\"$(bindir)\" -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) lexecdir = $(libexecdir)/singularity/bin lexec_PROGRAMS = action builddef cleanupd get-section image-type mount prepheader start $(BUILD_SUID) EXTRA_PROGRAMS = action-suid mount-suid start-suid cleanupd_SOURCES = cleanupd.c util/util.c util/file.c util/message.c util/privilege.c util/config_parser.c util/registry.c cleanupd_CPPFLAGS = $(AM_CPPFLAGS) action_SOURCES = action.c util/util.c util/file.c util/registry.c util/privilege.c util/sessiondir.c util/suid.c util/cleanupd.c util/daemon.c util/mount.c action_LDADD = lib/image/libsingularity-image.la lib/runtime/libsingularity-runtime.la action-lib/libinternal.la action_CPPFLAGS = $(AM_CPPFLAGS) builddef_SOURCES = builddef.c util/util.c util/file.c util/registry.c util/sessiondir.c builddef_LDADD = lib/image/libsingularity-image.la lib/runtime/libsingularity-runtime.la bootstrap-lib/libinternal.la builddef_CPPFLAGS = $(AM_CPPFLAGS) builddef_LDFLAGS = -static start_SOURCES = start.c util/util.c util/file.c util/registry.c util/privilege.c util/sessiondir.c util/suid.c util/cleanupd.c util/fork.c util/daemon.c util/signal.c util/mount.c start_LDADD = lib/image/libsingularity-image.la lib/runtime/libsingularity-runtime.la action-lib/libinternal.la start_CPPFLAGS = $(AM_CPPFLAGS) mount_SOURCES = mount.c util/util.c util/file.c util/registry.c util/suid.c util/privilege.c util/mount.c mount_LDADD = lib/image/libsingularity-image.la lib/runtime/libsingularity-runtime.la mount_CPPFLAGS = $(AM_CPPFLAGS) prepheader_SOURCES = prepheader.c prepheader_LDADD = prepheader_CPPFLAGS = $(AM_CPPFLAGS) get_section_SOURCES = get-section.c util/util.c util/file.c util/message.c util/privilege.c util/config_parser.c util/registry.c get_section_CPPFLAGS = $(AM_CPPFLAGS) image_type_SOURCES = image-type.c util/util.c util/message.c util/config_parser.c util/file.c image_type_LDADD = lib/image/libsingularity-image.la image_type_CPPFLAGS = $(AM_CPPFLAGS) action_suid_SOURCES = $(action_SOURCES) action_suid_LDADD = $(action_LDADD) action_suid_LDFLAGS = -static action_suid_CPPFLAGS = -DSINGULARITY_SUID $(AM_CPPFLAGS) start_suid_SOURCES = $(start_SOURCES) start_suid_LDADD = $(start_LDADD) start_suid_LDFLAGS = -static start_suid_CPPFLAGS = -DSINGULARITY_SUID $(AM_CPPFLAGS) mount_suid_SOURCES = $(mount_SOURCES) mount_suid_LDADD = $(mount_LDADD) mount_suid_LDFLAGS = -static mount_suid_CPPFLAGS = -DSINGULARITY_SUID $(AM_CPPFLAGS) install-data-hook: make_suid make_suid: @if test `id -ru` = "0"; then \ for i in $(BUILD_SUID); do \ echo " /bin/chown root:root $(DESTDIR)$(libexecdir)/singularity/bin/$$i"; \ /bin/chown root:root $(DESTDIR)$(libexecdir)/singularity/bin/$$i ; \ echo " /bin/chmod 4755 $(DESTDIR)$(libexecdir)/singularity/bin/$$i"; \ /bin/chmod 4755 $(DESTDIR)$(libexecdir)/singularity/bin/$$i; \ done; \ fi EXTRA_DIST = config.h singularity-2.4.2/src/copy.c0000644000175000017500000000563513211621077014745 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "util/fork.h" #include "util/sessiondir.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif int main(int argc, char **argv) { int retval = 0; int i; char *cp_cmd[argc]; struct image_object image; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_priv_init(); singularity_suid_init(argv); singularity_registry_init(); singularity_priv_drop(); singularity_sessiondir(); image = singularity_image_init(singularity_registry_get("IMAGE")); // singularity_image_open(&image, O_RDWR); // singularity_image_check(&image); singularity_registry_set("WRITABLE", "1"); singularity_runtime_ns(SR_NS_MNT); singularity_image_bind(&image); singularity_image_mount(&image, singularity_runtime_rootfs(NULL)); cp_cmd[0] = strdup("/bin/cp"); for(i=1; i < argc; i++) { if ( i == argc-1 ) { cp_cmd[i] = joinpath(singularity_runtime_rootfs(NULL), argv[i]); } else { cp_cmd[i] = strdup(argv[i]); } } cp_cmd[argc] = NULL; singularity_message(DEBUG, "Cleaning environment\n"); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing the environment\n"); ABORT(255); } singularity_priv_escalate(); retval = singularity_fork_exec(0, cp_cmd); singularity_priv_drop(); if ( retval != 0 ) { singularity_message(ERROR, "/bin/cp did not return successful\n"); } return(retval); } singularity-2.4.2/src/bootstrap.c0000644000175000017500000000747413211621077016013 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/sessiondir.h" #include "./bootstrap-lib/include.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif int main(int argc, char **argv) { struct image_object image; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_registry_init(); singularity_priv_init(); singularity_message(INFO, "Sanitizing environment\n"); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing the environment\n"); ABORT(255); } singularity_registry_set("WRITABLE", "1"); image = singularity_image_init(singularity_registry_get("IMAGE")); // singularity_image_open(&image, O_RDWR); // // singularity_image_check(&image); singularity_runtime_ns(SR_NS_MNT); // singularity_image_bind(&image); singularity_image_mount(&image, CONTAINER_MOUNTDIR); envar_set("PATH", "/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin", 1); envar_set("SINGULARITY_ROOTFS", CONTAINER_MOUNTDIR, 1); envar_set("SINGULARITY_libexecdir", singularity_registry_get("LIBEXECDIR"), 1); envar_set("SINGULARITY_IMAGE", singularity_registry_get("IMAGE"), 1); envar_set("SINGULARITY_BUILDDEF", singularity_registry_get("BUILDDEF"), 1); envar_set("SINGULARITY_CHECKS", singularity_registry_get("CHECKS"), 1); envar_set("SINGULARITY_CHECKLEVEL", singularity_registry_get("CHECKLEVEL"), 1); envar_set("SINGULARITY_CHECKTAGS", singularity_registry_get("CHECKTAGS"), 1); envar_set("SINGULARITY_MESSAGELEVEL", singularity_registry_get("MESSAGELEVEL"), 1); envar_set("SINGULARITY_NOTEST", singularity_registry_get("NOTEST"), 1); envar_set("SINGULARITY_BUILDSECTION", singularity_registry_get("BUILDSECTION"), 1); envar_set("SINGULARITY_BUILDNOBASE", singularity_registry_get("BUILDNOBASE"), 1); envar_set("SINGULARITY_DOCKER_PASSWORD", singularity_registry_get("DOCKER_PASSWORD"), 1); envar_set("SINGULARITY_DOCKER_USERNAME", singularity_registry_get("DOCKER_USERNAME"), 1); envar_set("SINGULARITY_CACHEDIR", singularity_registry_get("CACHEDIR"), 1); envar_set("SINGULARITY_version", singularity_registry_get("VERSION"), 1); envar_set("HOME", singularity_priv_home(), 1); envar_set("LANG", "C", 1); // At this point, the container image is mounted at // CONTAINER_MOUNTDIR, and bootstrap code can be added // in the bootstrap-lib/ directory. bootstrap_init(argc, argv); return(0); } singularity-2.4.2/src/get-keyvals.c0000644000175000017500000000520313211621077016215 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #define MAX_LINE_LEN 4096 int main(int argc, char ** argv) { FILE *bootdef_fp; char *line; char *path; if ( argc < 2 ) { printf("USAGE: %s [file]\n", argv[0]); exit(0); } path = argv[1]; if ( is_file(path) != 0 ) { singularity_message(ERROR, "Bootstrap definition file not found: %s\n", path); ABORT(255); } if ( ( bootdef_fp = fopen(path, "r") ) == NULL ) { singularity_message(ERROR, "Could not open bootstrap definition file %s: %s\n", path, strerror(errno)); ABORT(255); } line = (char *)malloc(MAX_LINE_LEN); while ( fgets(line, MAX_LINE_LEN, bootdef_fp) ) { char *bootdef_key; if ( line[0] == '%' ) { // We hit a section, stop parsing for keyword tags break; } else if ( ( bootdef_key = strtok(line, ":") ) != NULL ) { char *bootdef_value; chomp(bootdef_key); if ( ( bootdef_value = strtok(NULL, "\n") ) != NULL ) { chomp_comments(bootdef_value); singularity_message(VERBOSE2, "Got bootstrap definition key/val '%s' = '%s'\n", bootdef_key, bootdef_value); printf("declare -x '%s'='%s'\n", bootdef_key, bootdef_value); } } } free(line); fclose(bootdef_fp); return(0); } singularity-2.4.2/src/bootstrap-lib/0000755000175000017500000000000013211621077016377 5ustar mehdimehdisingularity-2.4.2/src/bootstrap-lib/include.h0000644000175000017500000000241113211621077020171 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __BOOTSTRAP_LIB_H_ #define __BOOTSTRAP_LIB_H_ extern int bootstrap_init(int argc, char **argv); extern int bootstrap_keyval_parse(char *path); extern int bootstrap_deffile(void); #endif /* __BOOTSTRAP_LIB_H */ singularity-2.4.2/src/bootstrap-lib/init.c0000644000175000017500000000643613211621077017517 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/registry.h" #include "./include.h" #ifndef LIBEXECDIR #error LIBEXECDIR is not defined #endif int bootstrap_init(int argc, char **argv) { char *builddef = singularity_registry_get("BUILDDEF"); if ( strncmp(builddef, "docker://", 9) == 0 ) { char *bootstrap = joinpath(LIBEXECDIR, "/singularity/bootstrap-scripts/main-dockerhub.sh"); singularity_message(INFO, "Building from DockerHub container\n"); execl(bootstrap, bootstrap, NULL); // Flawfinder: ignore (this is necessary) } else if ( strncmp(builddef, "self", 4) == 0 ) { char *bootstrap = joinpath(LIBEXECDIR, "/singularity/bootstrap-scripts/main-deffile.sh"); singularity_message(INFO, "Self clone with bootstrap definition recipe\n"); if ( bootstrap_keyval_parse(builddef) != 0 ) { singularity_message(ERROR, "Failed parsing the bootstrap definition file: %s\n", singularity_registry_get("BUILDDEF")); ABORT(255); } execl(bootstrap, bootstrap, NULL); // Flawfinder: ignore (this is necessary) } else if ( is_file(builddef) == 0 ) { char *bootstrap = joinpath(LIBEXECDIR, "/singularity/bootstrap-scripts/main-deffile.sh"); singularity_message(INFO, "Building from bootstrap definition recipe\n"); if ( bootstrap_keyval_parse(builddef) != 0 ) { singularity_message(ERROR, "Failed parsing the bootstrap definition file: %s\n", singularity_registry_get("BUILDDEF")); ABORT(255); } execl(bootstrap, bootstrap, NULL); // Flawfinder: ignore (this is necessary) } else if ( builddef == NULL || builddef[0] == '\0' ) { char *bootstrap = joinpath(LIBEXECDIR, "/singularity/bootstrap-scripts/main-null.sh"); singularity_message(INFO, "Running bootstrap with no recipe\n"); execl(bootstrap, bootstrap, NULL); // Flawfinder: ignore (this is necessary) } else { singularity_message(ERROR, "Unsupported bootstrap definition format: '%s'\n", builddef); ABORT(255); } return(0); } singularity-2.4.2/src/bootstrap-lib/Makefile.am0000644000175000017500000000056313211621077020437 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = init.c keyval.c EXTRA_DIST = include.h singularity-2.4.2/src/bootstrap-lib/keyval.c0000644000175000017500000000644413211621077020046 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/registry.h" #define MAX_LINE_LEN 4096 int bootstrap_keyval_parse(char *path) { FILE *bootdef_fp; char *line; if ( is_file(path) != 0 ) { singularity_message(ERROR, "Bootstrap definition file not found: %s\n", path); ABORT(255); } if ( ( bootdef_fp = fopen(path, "r") ) == NULL ) { singularity_message(ERROR, "Could not open bootstrap definition file %s: %s\n", path, strerror(errno)); ABORT(255); } line = (char *)malloc(MAX_LINE_LEN); while ( fgets(line, MAX_LINE_LEN, bootdef_fp) ) { char *bootdef_key; if ( line[0] == '%' ) { // We hit a section, stop parsing for keyword tags break; } else if ( ( bootdef_key = strtok(line, ":") ) != NULL ) { chomp(bootdef_key); char *bootdef_value; if ( ( bootdef_value = strtok(NULL, "\n") ) != NULL ) { chomp_comments(bootdef_value); singularity_message(VERBOSE2, "Got bootstrap definition key/val '%s' = '%s'\n", bootdef_key, bootdef_value); if ( envar_defined(strjoin("SINGULARITY_DEFFILE_", uppercase(bootdef_key))) == 0 ) { singularity_message(ERROR, "Duplicate bootstrap definition key found: '%s'\n", bootdef_key); ABORT(255); } if ( strcasecmp(bootdef_key, "import") == 0 ) { // Do this again for an imported deffile bootstrap_keyval_parse(bootdef_value); } if ( strcasecmp(bootdef_key, "bootstrap") == 0 ) { singularity_registry_set("DRIVER", bootdef_value); } // Cool little feature, every key defined in def file is transposed // to environment envar_set(uppercase(bootdef_key), bootdef_value, 1); envar_set(strjoin("SINGULARITY_DEFFILE_", uppercase(bootdef_key)), bootdef_value, 1); } } } free(line); fclose(bootdef_fp); return(0); } singularity-2.4.2/src/action-lib/0000755000175000017500000000000013211621077015637 5ustar mehdimehdisingularity-2.4.2/src/action-lib/exec.c0000644000175000017500000000504413211621077016732 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" void action_exec(int argc, char **argv) { if ( argc <= 1 ) { singularity_message(ERROR, "No program name to exec\n"); ABORT(255); } singularity_message(DEBUG, "Checking for: /.singularity.d/actions/exec\n"); if ( is_exec("/.singularity.d/actions/exec") == 0 ) { singularity_message(VERBOSE, "Exec'ing /.singularity.d/actions/exec\n"); if ( execv("/.singularity.d/actions/exec", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.singularity.d/actions/exec: %s\n", strerror(errno)); } } singularity_message(DEBUG, "Checking for: /.exec\n"); if ( is_exec("/.exec") == 0 ) { singularity_message(VERBOSE, "Exec'ing /.exec\n"); if ( execv("/.exec", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.exec: %s\n", strerror(errno)); } } singularity_message(WARNING, "Container does not have an exec helper script, calling '%s' directly\n", argv[1]); if ( execvp(argv[1], &argv[1]) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execvp() %s: %s\n", argv[1], strerror(errno)); ABORT(255); } singularity_message(ERROR, "We should never get here... Grrrrrr!\n"); ABORT(255); } singularity-2.4.2/src/action-lib/ready.c0000644000175000017500000000317513211621077017115 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" void action_ready(void) { singularity_message(DEBUG, "Checking if container is valid at: %s\n", CONTAINER_MOUNTDIR); if ( is_exec(joinpath(CONTAINER_MOUNTDIR, "/bin/sh")) != 0 && is_link(joinpath(CONTAINER_MOUNTDIR, "/bin/sh")) != 0 ) { singularity_message(ERROR, "No valid /bin/sh in container\n"); ABORT(255); } return; } singularity-2.4.2/src/action-lib/shell.c0000644000175000017500000000515113211621077017114 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" #include "util/privilege.h" void action_shell(int argc, char **argv) { singularity_message(INFO, "Singularity: Invoking an interactive shell within container...\n\n"); if ( is_exec("/.singularity.d/actions/shell") == 0 ) { singularity_message(DEBUG, "Exec'ing /.singularity.d/actions/shell\n"); if ( execv("/.singularity.d/actions/shell", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.singularity.d/actions/shell, continuing to /bin/sh: %s\n", strerror(errno)); } } else if ( is_exec("/.shell") == 0 ) { singularity_message(DEBUG, "Exec'ing /.shell\n"); if ( execv("/.shell", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.shell, continuing to /bin/sh: %s\n", strerror(errno)); } } singularity_message(VERBOSE, "Invoking the container's /bin/sh\n"); if ( is_exec("/bin/sh") == 0 ) { singularity_message(DEBUG, "Exec'ing /bin/sh\n"); argv[0] = strdup("/bin/sh"); if ( execv("/bin/sh", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /bin/sh: %s\n", strerror(errno)); ABORT(255); } } singularity_message(ERROR, "What are you doing %s, this is highly irregular!\n", singularity_priv_getuser()); ABORT(255); } singularity-2.4.2/src/action-lib/test.c0000644000175000017500000000422513211621077016765 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" void action_test(int argc, char **argv) { singularity_message(VERBOSE, "Starting test code\n"); if ( is_exec("/.singularity.d/actions/test") == 0 ) { singularity_message(DEBUG, "Exec'ing /.singularity.d/actions/test\n"); if ( execv("/.singularity.d/actions/test", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.singularity.d/actions/test: %s\n", strerror(errno)); ABORT(255); } } else if ( is_exec("/.test") == 0 ) { if ( execv("/.test", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.test, continuing to /bin/sh: %s\n", strerror(errno)); } } else { singularity_message(ERROR, "No test driver found inside container\n"); ABORT(255); } singularity_message(ERROR, "We should never get here... Grrrrrr!\n"); ABORT(255); } singularity-2.4.2/src/action-lib/include.h0000644000175000017500000000303713211621077017436 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #ifndef __ACTION_LIB_H_ #define __ACTION_LIB_H_ extern void action_ready(void); extern int action_shell(int argc, char **argv); extern int action_exec(int argc, char **argv); extern int action_run(int argc, char **argv); extern int action_test(int argc, char **argv); extern int action_appexec(int argc, char **argv); extern int action_apprun(int argc, char **argv); extern int action_appshell(int argc, char **argv); extern int action_apptest(int argc, char **argv); #endif /* __ACTION_LIB_H */ singularity-2.4.2/src/action-lib/run.c0000644000175000017500000000470613211621077016616 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #include #include #include #include #include #include #include #include "util/file.h" #include "util/util.h" #include "util/message.h" void action_run(int argc, char **argv) { singularity_message(VERBOSE, "Starting runscript\n"); if ( is_exec("/.singularity.d/actions/run") == 0 ) { singularity_message(DEBUG, "Exec'ing /.singularity.d/actions/run\n"); if ( execv("/.singularity.d/actions/run", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.singularity.d/actions/run: %s\n", strerror(errno)); ABORT(255); } } else if ( is_exec("/.run") == 0 ) { if ( execv("/.run", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /.run, continuing to /bin/sh: %s\n", strerror(errno)); } } else if ( is_exec("/singularity") == 0 ) { singularity_message(DEBUG, "Exec'ing /singularity\n"); if ( execv("/singularity", argv) < 0 ) { // Flawfinder: ignore singularity_message(ERROR, "Failed to execv() /singularity: %s\n", strerror(errno)); ABORT(255); } } else { singularity_message(ERROR, "No run driver found inside container\n"); ABORT(255); } singularity_message(ERROR, "We should never get here... Grrrrrr!\n"); ABORT(255); } singularity-2.4.2/src/action-lib/Makefile.am0000644000175000017500000000060713211621077017676 0ustar mehdimehdiMAINTAINERCLEANFILES = Makefile.in DISTCLEANFILES = Makefile CLEANFILES = core.* *~ *.la AM_CFLAGS = -Wall -fpie AM_LDFLAGS = -pie AM_CPPFLAGS = -DSYSCONFDIR=\"$(sysconfdir)\" -DLOCALSTATEDIR=\"$(localstatedir)\" -DLIBEXECDIR=\"$(libexecdir)\" $(SINGULARITY_DEFINES) noinst_LTLIBRARIES = libinternal.la libinternal_la_SOURCES = exec.c test.c ready.c run.c shell.c EXTRA_DIST = include.h singularity-2.4.2/src/cleanupd.c0000644000175000017500000000624013211621077015557 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/message.h" int main(int argc, char **argv) { int retval = 0; char *cleanup_dir = envar_path("SINGULARITY_CLEANUPDIR"); char *trigger = envar_path("SINGULARITY_CLEANUPTRIGGER"); int trigger_fd; int daemon_options = 0; singularity_message(DEBUG, "Starting cleanup process\n"); if ( singularity_message_level() > 1 ) { daemon_options = 1; } if ( ( cleanup_dir == NULL ) || ( trigger == NULL ) ) { singularity_message(ERROR, "Environment is not properly setup\n"); ABORT(255); } if ( is_dir(cleanup_dir) != 0 ) { singularity_message(ERROR, "Cleanup location is not a directory: %s\n", cleanup_dir); ABORT(255); } singularity_message(DEBUG, "Opening cleanup trigger file: %s\n", trigger); if ( ( trigger_fd = open(trigger, O_RDONLY, 00644) ) < 0 ) { singularity_message(ERROR, "Failed opening trigger file %s: %s\n", trigger, strerror(errno)); ABORT(255); } singularity_message(DEBUG, "Checking to see if we need to daemonize\n"); if ( flock(trigger_fd, LOCK_EX | LOCK_NB) != 0 ) { singularity_message(VERBOSE, "Daemonizing cleandir cleanup process\n"); if ( daemon(daemon_options, daemon_options) != 0 ) { singularity_message(ERROR, "Failed daemonizing cleanup process: %s\n", strerror(errno)); ABORT(255); } } singularity_message(DEBUG, "Waiting for exclusive flock() on trigger file descriptor: %d\n", trigger_fd); if ( flock(trigger_fd, LOCK_EX) == 0 ) { singularity_message(VERBOSE, "Cleaning directory: %s\n", cleanup_dir); if ( s_rmdir(cleanup_dir) < 0 ) { unlink(trigger); singularity_message(ERROR, "Could not remove directory %s: %s\n", cleanup_dir, strerror(errno)); ABORT(255); } close(trigger_fd); unlink(trigger); } return(retval); } singularity-2.4.2/src/expand.c0000644000175000017500000000747013211621077015251 0ustar mehdimehdi/* * Copyright (c) 2017, SingularityWare, LLC. All rights reserved. * * Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. * * Copyright (c) 2016-2017, The Regents of the University of California, * through Lawrence Berkeley National Laboratory (subject to receipt of any * required approvals from the U.S. Dept. of Energy). All rights reserved. * * This software is licensed under a customized 3-clause BSD license. Please * consult LICENSE file distributed with the sources of this project regarding * your rights to use or distribute this software. * * NOTICE. This Software was developed under funding from the U.S. Department of * Energy and the U.S. Government consequently retains certain rights. As such, * the U.S. Government has been granted for itself and others acting on its * behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software * to reproduce, distribute copies to the public, prepare derivative works, and * perform publicly and display publicly, and to permit other to do so. * */ #define _GNU_SOURCE #include #include #include #include #include #include #include "config.h" #include "util/file.h" #include "util/util.h" #include "util/registry.h" #include "lib/image/image.h" #include "lib/runtime/runtime.h" #include "util/config_parser.h" #include "util/privilege.h" #include "util/suid.h" #include "util/fork.h" #ifndef SYSCONFDIR #error SYSCONFDIR not defined #endif int main(int argc, char **argv) { struct image_object image; long int size = 768; char *size_s; char *e2fsck_cmd[4]; char *resize2fs_cmd[3]; singularity_config_init(joinpath(SYSCONFDIR, "/singularity/singularity.conf")); singularity_priv_init(); singularity_suid_init(argv); singularity_registry_init(); singularity_priv_drop(); if ( ( size_s = singularity_registry_get("IMAGESIZE") ) != NULL ) { if ( str2int(size_s, &size) == 0 ) { singularity_message(VERBOSE, "Converted size string to long int: %ld\n", size); } else { singularity_message(ERROR, "Could not convert container size to integer\n"); ABORT(255); } } singularity_message(INFO, "Initializing Singularity image subsystem\n"); image = singularity_image_init(singularity_registry_get("IMAGE")); singularity_message(INFO, "Opening image file: %s\n", image.name); singularity_image_open(&image, O_RDWR); singularity_image_check(&image); singularity_message(INFO, "Expanding image by %ldMiB\n", size); singularity_image_expand(&image, size); singularity_message(INFO, "Binding image to loop\n"); singularity_image_bind(&image); e2fsck_cmd[0] = strdup("/sbin/e2fsck"); e2fsck_cmd[1] = strdup("-fy"); e2fsck_cmd[2] = strdup(image.loopdev); e2fsck_cmd[3] = NULL; resize2fs_cmd[0] = strdup("/sbin/resize2fs"); resize2fs_cmd[1] = strdup(image.loopdev); resize2fs_cmd[2] = NULL; singularity_message(DEBUG, "Cleaning environment\n"); if ( envclean() != 0 ) { singularity_message(ERROR, "Failed sanitizing the environment\n"); ABORT(255); } singularity_priv_escalate(); singularity_message(INFO, "Checking file system\n"); if ( singularity_fork_exec(0, e2fsck_cmd) != 0 ) { singularity_message(ERROR, "Failed running %s\n", e2fsck_cmd[0]); ABORT(255); } singularity_priv_drop(); singularity_priv_escalate(); singularity_message(INFO, "Resizing file system\n"); if ( singularity_fork_exec(0, resize2fs_cmd) != 0 ) { singularity_message(ERROR, "Failed running '%s' '%s'\n", resize2fs_cmd[0], resize2fs_cmd[1]); ABORT(255); } singularity_priv_drop(); singularity_message(INFO, "Image is done: %s\n", image.path); return(0); } singularity-2.4.2/CONTRIBUTING.md0000644000175000017500000001515713211621077015271 0ustar mehdimehdi# Contributor's Agreement You are under no obligation whatsoever to provide any bug fixes, patches, or upgrades to the features, functionality or performance of the source code ("Enhancements") to anyone; however, if you choose to make your Enhancements available either publicly, or directly to the project, without imposing a separate written license agreement for such Enhancements, then you hereby grant the following license: a non-exclusive, royalty-free perpetual license to install, use, modify, prepare derivative works, incorporate into other computer software, distribute, and sublicense such enhancements or derivative works thereof, in binary and source code form. # Contributing When contributing to Singularity, it is important to properly communicate the gist of the contribution. If it is a simple code or editorial fix, simply explaining this within the GitHub Pull Request (PR) will suffice. But if this is a larger fix or Enhancement, you are advised to first discuss the change with the project leader or developers. Please note we have a code of conduct, described below. Please follow it in all your interactions with the project members and users. ## Pull Requests (PRs) ### Process 1. Essential bug fix PRs should be sent to both master and devopment branches. 2. Small bug fix and feature enhancement PRs should be sent to development only. 3. Follow the existing code style precedent. This does not need to be strictly defined as there are many thousands of lines of examples. Note the lack of tabs anywhere in the project, parentheses and spacing, curly bracket locations, source code layout, variable scoping, etc. and follow the project's standards. 4. Ensure any install or build dependencies are removed before doing a build to test your PR locally. 5. For any new functionality, please write a test to be added to Continuous Integration (Travis) to test it (tests can be found in the `tests/` directory). 6. The project's default copyright and header have been included in any new source files. 7. Make sure you have implemented a local `make test` and all tests succeed before submitting the PR. 8. Is the code human understandable? This can be accomplished via a clear code style as well as documentation and/or comments. 9. The pull request will be reviewed by others, and the final merge must be done by the Singularity project lead, @gmkurtzer (or approved by him). 10. Documentation must be provided if necessary (next section) ### Documentation 1. If you are changing any of the following: - renamed commands - deprecated / removed commands - changed defaults - backward incompatible changes (recipe file format? image file format?) - migration guidance (how to convert images?) - changed behaviour (recipe sections work differently) You are **required** to document it in the [changelog](CHANGELOG.md) for the next release. You are also required to provide documentation or a direct pull request to the (upcoming) version of the [singularityware.github.io](https://www.github.io/singularityware/singularityware.github.io) docs. Ask for help if you aren't sure where your contribution should go. 2. If necessary, update the README.md, and check the `*.help` scripts under [libexec/cli](libexec/cli) that provide the command line helper output. If you make changes to the internal Python API, make sure to check those changes into the [libexec/python/README.md](libexec/python/README.md) as well. # Code of Conduct ## Our Pledge In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. ## Our Standards Examples of behavior that contributes to creating a positive environment include: * Using welcoming and inclusive language * Being respectful of differing viewpoints and experiences * Gracefully accepting constructive criticism * Focusing on what is best for the community * Showing empathy towards other community members Examples of unacceptable behavior by participants include: * The use of sexualized language or imagery and unwelcome sexual attention or advances * Trolling, insulting/derogatory comments, and personal or political attacks * Public or private harassment * Publishing others' private information, such as a physical or electronic address, without explicit permission * Other conduct which could reasonably be considered inappropriate in a professional setting ### Our Responsibilities Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. ## Scope This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. ## Enforcement Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project leader (gmkurtzer@gmail.com). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. Project maintainers, contributors and users who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions with their involvement in the project as determined by the project's leader(s). ## Attribution This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version] [homepage]: http://contributor-covenant.org [version]: http://contributor-covenant.org/version/1/4/ singularity-2.4.2/alpinelinux/0000755000175000017500000000000013211621077015357 5ustar mehdimehdisingularity-2.4.2/alpinelinux/singularity-2.2/0000755000175000017500000000000013211621077020230 5ustar mehdimehdisingularity-2.4.2/alpinelinux/singularity-2.2/APKBUILD0000644000175000017500000000370413211621077021352 0ustar mehdimehdi# Contributor: "Tru Huynh " # Maintainer: "Tru Huynh " pkgname=singularity pkgver=2.2 pkgrel=0 pkgdesc="Singularity: Application containers for Linux" url="http://singularity.lbl.gov" arch="x86_64" license="LNL" depends="" depends_dev="autoconf automake gcc make libtool linux-headers" makedepends="$depends_dev" install="" subpackages="$pkgname-doc $pkgname-examples" #source="${pkgname}-${pkgver}.tar.gz" source="${pkgname}-${pkgver}.tar.gz::https://github.com/singularityware/singularity/releases/download/${pkgver}/${pkgname}-${pkgver}.tar.gz" options="suid" builddir=$srcdir/${pkgname}-${pkgver} build() { cd "$builddir" ./configure \ --build=$CBUILD \ --host=$CHOST \ --prefix=/usr \ --sysconfdir=/etc \ --mandir=/usr/share/man \ --localstatedir=/var \ || return 1 make || return 1 } package() { cd "$builddir" make DESTDIR="$pkgdir" install || return 1 } doc() { arch="noarch" cd "$builddir" mkdir -p "$subpkgdir"/usr/share/doc/"$pkgname" || return 1 # Doc files _docs="AUTHORS COPYING ChangeLog INSTALL NEWS README.md" for _doc in $_docs; do install -Dm644 "$srcdir"/$pkgname-$pkgver/$_doc \ "$subpkgdir"/usr/share/doc/$pkgname/$_doc || return 1 done } examples() { arch="noarch" # Put the examples into a seperate package cd "$builddir" mkdir -p "$subpkgdir"/usr/share/doc/"$pkgname"/examples || return 1 mv "$builddir"/examples/* "$subpkgdir"/usr/share/doc/"$pkgname"/examples || return 1 } md5sums="74d05c2f2275bbfde36a2f05a736b7fe singularity-2.2.tar.gz" sha256sums="3dcb23300d6a5a248659880cbcd98a073bc4a49d19c279ba0460256ed480f3e9 singularity-2.2.tar.gz" sha512sums="ae22a2a33dd7d013f4fd12e751d83aeaf3b0acfe98d79d4f827e1380703cc17e624b67afde1f8af6e5a762d493a8748d073570133207b82db8f8e3483055379b singularity-2.2.tar.gz" singularity-2.4.2/alpinelinux/singularity-2.1.2/0000755000175000017500000000000013211621077020367 5ustar mehdimehdisingularity-2.4.2/alpinelinux/singularity-2.1.2/APKBUILD0000644000175000017500000000372113211621077021510 0ustar mehdimehdi# Contributor: "Tru Huynh " # Maintainer: "Tru Huynh " pkgname=singularity pkgver=2.1.2 pkgrel=0 pkgdesc="Singularity: Application containers for Linux" url="http://singularity.lbl.gov" arch="x86_64" license="LNL" depends="" depends_dev="autoconf automake gcc make libtool linux-headers" makedepends="$depends_dev" install="" subpackages="$pkgname-doc $pkgname-examples" source="${pkgname}-${pkgver}.tar.gz::https://github.com/singularityware/singularity/archive/${pkgver}.tar.gz" options="suid" builddir=$srcdir/${pkgname}-${pkgver} build() { cd "$builddir" ./autogen.sh ./configure \ --build=$CBUILD \ --host=$CHOST \ --prefix=/usr \ --sysconfdir=/etc \ --mandir=/usr/share/man \ --localstatedir=/var \ || return 1 make || return 1 } package() { cd "$builddir" make DESTDIR="$pkgdir" install || return 1 } doc() { arch="noarch" cd "$builddir" mkdir -p "$subpkgdir"/usr/share/doc/"$pkgname" || return 1 # Doc files _docs="AUTHORS COPYING ChangeLog INSTALL NEWS README.md" for _doc in $_docs; do # install -Dm644 "$srcdir"/$pkgname-master/$_doc \ install -Dm644 "$srcdir"/$pkgname-$pkgver/$_doc \ "$subpkgdir"/usr/share/doc/$pkgname/$_doc || return 1 done } examples() { arch="noarch" # Put the examples into a seperate package cd "$builddir" mkdir -p "$subpkgdir"/usr/share/doc/"$pkgname"/examples || return 1 mv "$builddir"/examples/* "$subpkgdir"/usr/share/doc/"$pkgname"/examples || return 1 } md5sums="d581dc080e6d5e2e055e4cc91572c829 singularity-2.1.2.tar.gz" sha256sums="8175adb404ea402b73333eb909dc6b63135444390a8f632900e7113030563458 singularity-2.1.2.tar.gz" sha512sums="6d90e613d50692d8b72d92f02df3aae34190f99ce123361f8db8e8ec2104a8d1a75a2956fc8b01bef572508f0b2ae6e87aaf6067efdf2278fceed52220890e8e singularity-2.1.2.tar.gz" singularity-2.4.2/alpinelinux/README.txt0000644000175000017500000001245513211621077017064 0ustar mehdimehdiPorting singularity to the lightweight musl libc. Building requirements: $ sudo apk update && sudo apk upgrade $ sudo apk add alpine-sdk autoconf automake libtool linux-headers $ mkdir singularity-build $ cd singularity-build fetch the APKBUILD $ apbuild -r if you get something like: ~/singularity-build$ abuild -r No private key found. Use 'abuild-keygen' to generate the keys. Then you can either: * set the PACKAGER_PRIVKEY in /home/tru/.abuild/abuild.conf ('abuild-keygen -a' does this for you) * set the PACKAGER_PRIVKEY in /etc/abuild.conf * specify the key with the -k option to abuild-sign >>> ERROR: singularity: all failed you need to properly setup your alpine build environment :P ~/singularity-build$ abuild-keygen -a >>> Generating public/private rsa key pair for abuild Enter file in which to save the key [/home/tru/.abuild/tru-5805cc3f.rsa]: Generating RSA private key, 2048 bit long modulus ..................................................................................................+++ .................+++ e is 65537 (0x10001) writing RSA key >>> >>> You'll need to install /home/tru/.abuild/tru-5805cc3f.rsa.pub into >>> /etc/apk/keys to be able to install packages and repositories signed with >>> /home/tru/.abuild/tru-5805cc3f.rsa >>> >>> Please remember to make a safe backup of your private key: >>> /home/tru/.abuild/tru-5805cc3f.rsa >>> ~/singularity-build$ abuild -r >>> singularity: Checking sanity of /home/tru/singularity-build/APKBUILD... >>> WARNING: singularity: depends_dev found but no development subpackage found >>> singularity: Analyzing dependencies... abuild-apk: User tru is not a member of group abuild >>> ERROR: singularity: all failed >>> singularity: Uninstalling dependencies... abuild-apk: User tru is not a member of group abuild Just add yourself to the abuild group, logout and login. ~/singularity-build$ abuild -r >>> singularity: Checking sanity of /home/tru/singularity-build/APKBUILD... >>> WARNING: singularity: depends_dev found but no development subpackage found >>> singularity: Analyzing dependencies... WARNING: Ignoring /home/tru/packages//tru/x86_64/APKINDEX.tar.gz: No such file or directory (1/1) Installing .makedepends-singularity (0) OK: 249 MiB in 84 packages >>> singularity: Cleaning temporary build dirs... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 134 0 134 0 0 284 0 --:--:-- --:--:-- --:--:-- 292 100 47328 0 47328 0 0 45085 0 --:--:-- 0:00:01 --:--:-- 125k >>> singularity: Checking sha512sums... singularity-2.1.2.tar.gz: OK >>> singularity: Unpacking /var/cache/distfiles/singularity-2.1.2.tar.gz... +autoreconf -i -f libtoolize: putting auxiliary files in '.'. libtoolize: copying file './ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, '.'. ... >>> singularity-doc*: Preparing subpackage singularity-doc... fatal: Not a git repository (or any of the parent directories): .git fatal: Not a git repository (or any of the parent directories): .git >>> singularity*: Running postcheck for singularity-doc >>> singularity*: Running split function examples... >>> singularity-examples*: Preparing subpackage singularity-examples... fatal: Not a git repository (or any of the parent directories): .git fatal: Not a git repository (or any of the parent directories): .git >>> singularity*: Running postcheck for singularity-examples >>> WARNING: singularity*: Found /usr/share/doc but package name doesn't end with -doc >>> singularity*: Running postcheck for singularity >>> singularity*: Preparing package singularity... >>> singularity*: Stripping binaries fatal: Not a git repository (or any of the parent directories): .git fatal: Not a git repository (or any of the parent directories): .git >>> singularity-doc*: Scanning shared objects >>> singularity-examples*: Scanning shared objects >>> singularity*: Scanning shared objects >>> singularity-doc*: Tracing dependencies... >>> singularity-doc*: Package size: 40.0 KB >>> singularity-doc*: Compressing data... >>> singularity-doc*: Create checksum... >>> singularity-doc*: Create singularity-doc-2.1.2-r0.apk >>> singularity-examples*: Tracing dependencies... >>> singularity-examples*: Package size: 48.0 KB >>> singularity-examples*: Compressing data... >>> singularity-examples*: Create checksum... >>> singularity-examples*: Create singularity-examples-2.1.2-r0.apk >>> singularity*: Tracing dependencies... so:libc.musl-x86_64.so.1 >>> singularity*: Package size: 364.0 KB >>> singularity*: Compressing data... >>> singularity*: Create checksum... >>> singularity*: Create singularity-2.1.2-r0.apk >>> singularity: Cleaning up srcdir >>> singularity: Cleaning up pkgdir >>> singularity: Uninstalling dependencies... >>> singularity: Updating the cached abuild repository index... fatal: Not a git repository (or any of the parent directories): .git >>> singularity: Signing the index... You should have a your freshly cooked packages in ~/packages. $ find ~/packages/ /home/tru/packages/ /home/tru/packages/tru /home/tru/packages/tru/x86_64 /home/tru/packages/tru/x86_64/singularity-2.1.2-r0.apk /home/tru/packages/tru/x86_64/singularity-doc-2.1.2-r0.apk /home/tru/packages/tru/x86_64/singularity-examples-2.1.2-r0.apk /home/tru/packages/tru/x86_64/APKINDEX.tar.gz singularity-2.4.2/alpinelinux/singularity-master/0000755000175000017500000000000013211621077021222 5ustar mehdimehdisingularity-2.4.2/alpinelinux/singularity-master/APKBUILD0000644000175000017500000000356113211621077022345 0ustar mehdimehdi# Contributor: "Tru Huynh " # Maintainer: "Tru Huynh " pkgname=singularity pkgver=2 pkgrel=0 pkgdesc="Singularity: Application containers for Linux" url="http://singularity.lbl.gov" arch="x86_64" license="LNL" depends="" depends_dev="autoconf automake gcc make libtool linux-headers" makedepends="$depends_dev" install="" subpackages="$pkgname-doc $pkgname-examples" source="${pkgname}-${pkgver}.zip::https://github.com/gmkurtzer/singularity/archive/master.zip" options="suid" builddir=$srcdir/singularity-master build() { cd "$builddir" ./autogen.sh ./configure \ --build=$CBUILD \ --host=$CHOST \ --prefix=/usr \ --sysconfdir=/etc \ --mandir=/usr/share/man \ --localstatedir=/var \ || return 1 make || return 1 } package() { cd "$builddir" make DESTDIR="$pkgdir" install || return 1 } doc() { arch="noarch" cd "$builddir" mkdir -p "$subpkgdir"/usr/share/doc/"$pkgname" || return 1 # Doc files _docs="AUTHORS COPYING ChangeLog INSTALL NEWS README.md" for _doc in $_docs; do install -Dm644 "$srcdir"/$pkgname-master/$_doc \ "$subpkgdir"/usr/share/doc/$pkgname/$_doc || return 1 done } examples() { arch="noarch" # Put the examples into a seperate package cd "$builddir" mkdir -p "$subpkgdir"/usr/share/doc/"$pkgname"/examples || return 1 mv "$builddir"/examples/* "$subpkgdir"/usr/share/doc/"$pkgname"/examples || return 1 } md5sums="23030140f8945148b83bdb0eadd0ed40 singularity-2.zip" sha256sums="1b027788abc2e5ba3aae1b5786d97373857cb70b526d37c34ade02c4e6b380f3 singularity-2.zip" sha512sums="fb148069f59afd205b90ae4189b01e1b7cbe3f0c755b032c00cb4841e491cf46b5ed586bc5a1898018c6b717de10f5a65f4a1106b1e5c237750794c41b0cc6b0 singularity-2.zip" singularity-2.4.2/test.sh.in0000755000175000017500000000630113211621077014752 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016, The Regents of the University of California, through # Lawrence Berkeley National Laboratory (subject to receipt of any required # approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # prefix="@prefix@" exec_prefix="@exec_prefix@" libexecdir="@libexecdir@" sysconfdir="@sysconfdir@" localstatedir="@localstatedir@" bindir="@bindir@" SINGULARITY_USER_NS="@USER_NS@" SINGULARITY_OVERLAY_FS="0" KVERS=`uname -r` if test -f "/lib/modules/$KVERS/modules.dep"; then if grep -q 'overlay.ko' "/lib/modules/$KVERS/modules.dep"; then SINGULARITY_OVERLAY_FS="1" fi fi SINGULARITY_libexecdir="$libexecdir" SINGULARITY_sysconfdir="$sysconfdir" SINGULARITY_localstatedir="$localstatedir" SINGULARITY_PATH="$bindir" export SINGULARITY_libexecdir SINGULARITY_sysconfdir SINGULARITY_localstatedir SINGULARITY_PATH SINGULARITY_OVERLAY_FS SINGULARITY_USER_NS if [ -z "$CLEAN_SHELL" ]; then /bin/echo "Building/Installing Singularity to temporary directory" /bin/echo "Reinvoking in a clean shell" sleep 1 # Keep docker host for travis under centos 7, which runs the whole test suite in a docker container exec env -i CLEAN_SHELL=1 PATH="$SINGULARITY_PATH:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin" \ DOCKER_HOST="$DOCKER_HOST" bash "$0" "$@" fi if [ ! -d "tests" ]; then /bin/echo "ERROR: Run this from the singularity source root" exit 1 fi if [ ! -x "$SINGULARITY_PATH/singularity" ]; then /bin/echo "ERROR: Could not locate singularity program at: $SINGULARITY_PATH/singularity" exit 1 fi if [ ! -d "./tests/" ]; then /bin/echo "ERROR: Could not locate singularity test directory" exit 1 fi if ! cd tests; then /bin/echo "ERROR: Could not change into the Singularity test directory" exit 1 fi if [ -n "$1" ]; then for i in $@; do test=`basename "$i"` if [ -f "$test" ]; then if ! /bin/sh "$test"; then /bin/echo "ERROR: Failed running test: $test" exit 1 fi else echo "ERROR: Could not find test: '$test'" exit 1 fi done else for test in *.sh; do if [ -f "$test" ]; then if ! /bin/sh "$test"; then /bin/echo "ERROR: Failed running test: $test" exit 1 fi fi done fi echo echo "All tests passed" singularity-2.4.2/autogen.sh0000755000175000017500000000046713211621077015037 0ustar mehdimehdi#!/bin/sh # set PS4 explicitly to be POSIX shell compatible for set -x export PS4=+ mkdir m4 >/dev/null 2>&1 if autoreconf -V >/dev/null 2>&1 ; then set -x autoreconf -i -f else set -x libtoolize -c aclocal autoheader autoconf automake -ca -Wno-portability --add-missing fi singularity-2.4.2/.mailmap0000644000175000017500000000033313211621077014447 0ustar mehdimehdiGregory M. Kurtzer Krishna Muriki Vanessa Sochat Rémy Dernat remy.d1 Rémy Dernat singularity-2.4.2/docs/0000755000175000017500000000000013211621077013757 5ustar mehdimehdisingularity-2.4.2/docs/README.md0000644000175000017500000000036213211621077015237 0ustar mehdimehdi**NOTE**: Documentation is no longer maintained in this repo. Please submit documentation updates as a pull request to the `master` branch of the [singularity docs repo](https://github.com/singularityware/singularityware.github.io). Thanks!! singularity-2.4.2/man/0000755000175000017500000000000013211621077013602 5ustar mehdimehdisingularity-2.4.2/man/singularity.10000644000175000017500000000260313211621077016237 0ustar mehdimehdi.TH SINGULARITY 1 .SH NAME Singularity \- Linux Application Containers .SH SYNOPSIS .B singularity [global options...] .B [command options...] .B ... .SH DESCRIPTION Singularity provides an application virtualization layer enabling mobility of compute via both application and environment portability. With Singularity one is capable of building a root file system and running that root file system on any other Linux system where Singularity is installed. .SH OPTIONS .IP "-h/--help/help" Print usage summary .IP "--version" Print Singularity version .IP "-v/--verbose" For each pass of verbose, increase verbosity .IP "-d/--debug" Print LOTS of debugging output (including user and process information) .SH FILES .I ${sysconfdir}/singularity/singularity.conf .RS The system wide configuration file. This file must be root owned for Singularity to parse it. .SH EXAMPLES .IP "$ singularity help" Will print a generalized usage summary and available commands. .IP "$ singularity help " Additional help for any of the Singularity commands can be seen by appending the command name to the above command. .SH DOCUMENTATION Full documentation can be found on the Singularity web site at: .B http://singularity.lbl.gov/ .SH BUGS Please submit bug reports to the GitHub issue tracker at .B https://github.com/singularityware/singularity/issues/new .SH AUTHOR Gregory M. Kurtzer singularity-2.4.2/man/Makefile.am0000644000175000017500000000003613211621077015635 0ustar mehdimehdidist_man_MANS = singularity.1 singularity-2.4.2/bin/0000755000175000017500000000000013211621077013577 5ustar mehdimehdisingularity-2.4.2/bin/singularity.in0000644000175000017500000001032613211621077016503 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016, The Regents of the University of California, through # Lawrence Berkeley National Laboratory (subject to receipt of any required # approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # prefix="@prefix@" exec_prefix="@exec_prefix@" libexecdir="@libexecdir@" sysconfdir="@sysconfdir@" localstatedir="@localstatedir@" bindir="@bindir@" home=`dirname "$0"` SINGULARITY_version="@VERSION@@GIT_VERSION@" SINGULARITY_libexecdir="$libexecdir" SINGULARITY_sysconfdir="$sysconfdir" SINGULARITY_localstatedir="$localstatedir" SINGULARITY_bindir="$bindir" SINGULARITY_MESSAGELEVEL=1 PATH="/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin" export SINGULARITY_bindir SINGULARITY_sysconfdir SINGULARITY_libexecdir SINGULARITY_localstatedir SINGULARITY_version SINGULARITY_MESSAGELEVEL PATH # the Modules package always calls unset module if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi message 5 "Starting argument loop\n" while true; do case ${1:-} in -h|--help|help) shift exec $SINGULARITY_libexecdir/singularity/cli/help.exec "$@" ;; -q|--quiet) SINGULARITY_MESSAGELEVEL=0 shift ;; --version) message 1 "$SINGULARITY_version\n" exit ;; -s|--silent) SINGULARITY_MESSAGELEVEL=-3 shift ;; -d|--debug) SINGULARITY_MESSAGELEVEL=5 message 4 "Enabling debugging\n" shift ;; -x|--sh-debug) SHELL_DEBUG=1 export SHELL_DEBUG message 4 "Enabling shell debugging\n" shift ;; -v|--verbose) SINGULARITY_MESSAGELEVEL=`expr $SINGULARITY_MESSAGELEVEL + 1` message 2 "Increasing verbosity level ($SINGULARITY_MESSAGELEVEL)\n" shift ;; -vv) SINGULARITY_MESSAGELEVEL=`expr $SINGULARITY_MESSAGELEVEL + 2` message 2 "Increasing verbosity level ($SINGULARITY_MESSAGELEVEL)\n" shift ;; -vvv) SINGULARITY_MESSAGELEVEL=`expr $SINGULARITY_MESSAGELEVEL + 3` message 2 "Increasing verbosity level ($SINGULARITY_MESSAGELEVEL)\n" shift ;; -vvvv) SINGULARITY_MESSAGELEVEL=`expr $SINGULARITY_MESSAGELEVEL + 4` message 2 "Increasing verbosity level ($SINGULARITY_MESSAGELEVEL)\n" shift ;; -*) message ERROR "Unknown argument: $1\n" exit 1 ;; *) message 5 "Ending argument loop\n" break ;; esac done message 2 "Singularity version: $SINGULARITY_version\n" SINGULARITY_COMMAND="${1:-}" export SINGULARITY_COMMAND shift if [ -z "$SINGULARITY_COMMAND" ]; then exec $SINGULARITY_libexecdir/singularity/cli/help.exec "$@" exit 0 fi if [ -x "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.exec" ]; then message 2 "Exec'ing: $SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.exec\n" exec $SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.exec "$@" else message ERROR "Unknown command '$SINGULARITY_COMMAND'\n" exit 1 fi # We should never get here... exit 255 singularity-2.4.2/bin/Makefile.am0000644000175000017500000000012513211621077015631 0ustar mehdimehdi dist_bin_SCRIPTS = singularity run-singularity MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/bin/run-singularity0000644000175000017500000000220313211621077016673 0ustar mehdimehdi#!/bin/sh # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016-2017, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # # Simple wrapper for 'singularity run' exec /usr/bin/env singularity --quiet run "$@" exit 255 singularity-2.4.2/COPYRIGHT.md0000644000175000017500000000053313211621077014722 0ustar mehdimehdiCopyright (c) 2017, SingularityWare, LLC. All rights reserved. Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. Copyright (c) 2016-2017, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. singularity-2.4.2/.gitignore0000644000175000017500000000226013211621077015017 0ustar mehdimehdi*~ #* *# src/*.o singularity-*.tar.gz *.m4 config.* *.pyc Makefile Makefile.in autom4te.cache/ bin/Makefile bin/Makefile.in bin/singularity compile configure depcomp etc/singularity.conf etc/Makefile etc/Makefile.in install-sh libexec/Makefile libexec/Makefile.in libexec/cli/Makefile libexec/cli/Makefile.in libexec/mods/Makefile libexec/mods/Makefile.in libexec/bootstrap-scripts/environment/environment.tar libtool ltmain.sh missing singularity.spec src/util/config_defaults.h test.sh src/action src/action-suid src/start src/start-suid src/builddef src/cleanupd src/image-type src/mount src/mount-suid src/prepheader src/sinit src/get-section src/stamp-h1 src/include/stamp-h1 src/.dirstamp src/sinit src/start src/start-suid src/.deps/ src/.libs/ src/lib*.la src/*.o src/*.lo src/slurm/singularity_spank.la src/*/.deps/ src/*/.libs/ src/*/lib*.la src/*/*.o src/*/*.lo src/*/.dirstamp src/*/*/.deps/ src/*/*/.libs/ src/*/*/lib*.la src/*/*/*.o src/*/*/*.lo src/*/*/.dirstamp src/*/*/*/.deps/ src/*/*/*/.libs/ src/*/*/*/lib*.la src/*/*/*/*.o src/*/*/*/*.lo src/*/*/*/.dirstamp src/*/*/*/*/.deps/ src/*/*/*/*/.libs/ src/*/*/*/*/lib*.la src/*/*/*/*/*.o src/*/*/*/*/*.lo src/*/*/*/*/.dirstamp singularity-2.4.2/.github/0000755000175000017500000000000013211621077014367 5ustar mehdimehdisingularity-2.4.2/.github/SUPPORT.md0000644000175000017500000000355413211621077016074 0ustar mehdimehdi# Singularity Support Need help? We have several ways to reach us, depending on your preferences and needs. ## Documentation If you haven't already, search our [user quick start](http://singularity.lbl.gov/quickstart), [docs base](http://singularity.lbl.gov/docs-quick-start-installation), and [other guides](http://singularity.lbl.gov/links) from the community. These docs have common use cases, and might be helpful to browse before submitting an issue. If you have a good resource, please [let us know](https://github.com/singularityware/singularity/issues). ## Github For issues with code (and especially if you need to share debug output) we recommend Github issues boards. - [Singularity Issues](https://github.com/singularityware/singularity/issues): is recommended for most issues with the Singularity software. - [Singularity Hub Issues](https://github.com/singularityhub/singularityhub.github.io/issues): is the board for issues relevant to Singularity Hub. - [Singularity Registry Issues](https://github.com/singularityhub/sregistry/issues): is the board for issues relevant to Singularity Registry - [Documentation Issues](https://github.com/singularityware/singularityware.github.io/issues): documentation questions, feedback, and suggestions should go here. Feel free to create an issue on a board and additionally request updated content here. ## Google Group You can reach the community quickly by way of joining our [Google Group](https://groups.google.com/a/lbl.gov/forum/#!forum/singularity). ## Slack For real time support from the community, you can join our community on slack at [https://singularity-container.slack.com/](https://singularity-container.slack.com/). Ping the Google Group or one of the admins here to request to be added. Is there something missing here you'd like to see? Please [let us know](https://github.com/singularityware/singularity/issues). singularity-2.4.2/.github/PULL_REQUEST_TEMPLATE.md0000644000175000017500000000211013211621077020162 0ustar mehdimehdi**Description of the Pull Request (PR):** Write your description of the PR here. Be sure to include as much background, and details necessary for the reviewers to understand exactly what this is fixing or enhancing. **This fixes or addresses the following GitHub issues:** - Ref: # **Checkoff for all PRs:** - [ ] I have read the [Guidelines for Contributing](https://github.com/singularityware/singularity/blob/master/CONTRIBUTING.md), and this PR conforms to the stated requirements. - [ ] I have added changes to the [CHANGELOG](https://github.com/singularityware/singularity/blob/development/CHANGELOG.md) and and documentation updates to the [singularityware](https://www.github.com/singularityware/singularityware.github.io) documentation base. - [ ] I have tested this PR locally with a `make test` - [ ] This PR is NOT against the project's master branch - [ ] I have added myself as a contributor to the [contributors's file](https://github.com/singularityware/singularity/blob/master/CONTRIBUTORS.md) - [ ] This PR is ready for review and/or merge Attn: @singularityware-admin singularity-2.4.2/.github/RELEASE_CHECKLIST.md0000644000175000017500000000114513211621077017363 0ustar mehdimehdi# Release Checklist ## Code - [] Ensure docs exist at singularityware.github.io for new/updated features - [] Update version number in `configure.ac` - [] Update changelog and version in `debian/changelog` - [] Confirm tests exist for new features, and tests pass - [] Commit the changes: ``` git add . git commit -m "Release 2.3.3" ``` ## Github - [] Tag the last git commit with the version number: ``` git tag -a 2.3.3 ``` ## Announcement - [] Initial announcement to list (@gmk) - [] Announcement for singularityware.github.io (ensure links work) (@vsoch) - [] If release, @SingularityWare on twitter singularity-2.4.2/.github/ISSUE_TEMPLATE.md0000644000175000017500000000023713211621077017076 0ustar mehdimehdi ## Version of Singularity: Write here. ### Expected behavior Write here. ### Actual behavior Write here. ### Steps to reproduce behavior Write here. singularity-2.4.2/libexec/0000755000175000017500000000000013211621077014442 5ustar mehdimehdisingularity-2.4.2/libexec/functions0000644000175000017500000003271213211621077016402 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. set -u if [ -z "${SINGULARITY_libexecdir:-}" ]; then echo "ERROR: SINGULARITY_libexecdir not defined in environment" exit 2 fi if [ -z "${SINGULARITY_MESSAGELEVEL:-}" ]; then echo "Warning: SINGULARITY_MESSAGELEVEL is undefined, temporarily setting to '5' (all messages)" SINGULARITY_MESSAGELEVEL=5 fi if [ -z "${USER:-}" ]; then USER=`id -un` export USER fi if [ -z "${HOME:-}" ]; then HOME=`getent passwd "$USER" | cut -d : -f 6` export HOME fi message() { LEVEL="${1:-}" MESSAGE="${2:-}" if [ -z "$MESSAGE" ]; then return 0 fi shift shift # if level is symbolic, get a numeric value case "$LEVEL" in INFO) LEVNUM=1 ;; VERBOSE) LEVNUM=4 ;; DEBUG) LEVNUM=5 ;; [1-5]) LEVNUM="$LEVEL" ;; *) LEVNUM=1 ;; # default to INFO esac case "$LEVEL" in e|error|E|ERROR) tput -Txterm setaf 1 2>/dev/null printf "ERROR: $MESSAGE" "$@" 1>&2 tput -Txterm sgr0 2>/dev/null ;; a|ABORT) tput -Txterm setaf 1 2>/dev/null printf "ABORT: $MESSAGE" "$@" 1>&2 tput -Txterm sgr0 2>/dev/null ;; w|warn|warning|W|WARN|WARNING) tput -Txterm setaf 3 2>/dev/null printf "WARNING: $MESSAGE" "$@" 1>&2 tput -Txterm sgr0 2>/dev/null ;; 1|INFO) if [ "$LEVNUM" -le "$SINGULARITY_MESSAGELEVEL" ]; then printf "$MESSAGE" "$@" fi ;; p|P|CHECKPASS|PASS) tput -Txterm setaf 2 2>/dev/null printf "PASS: $MESSAGE" "$@" 1>&2 tput -Txterm sgr0 2>/dev/null ;; NOTIFY|notify) tput -Txterm setaf 6 2>/dev/null printf "$MESSAGE" "$@" 1>&2 tput -Txterm sgr0 2>/dev/null ;; f|F|CHECKFAIL|FAIL) tput -Txterm setaf 1 2>/dev/null printf "FAIL: $MESSAGE\n" "$@" 1>&2 tput -Txterm sgr0 2>/dev/null ;; [2-5]|VERBOSE|DEBUG) if [ "$LEVNUM" -le "$SINGULARITY_MESSAGELEVEL" ]; then printf "$MESSAGE" "$@" 1>&2 fi ;; esac return 0 } singularity_key_get() { KEY="${1:-}" FILE="${2:-}" if OUT=`egrep -i "^$KEY:" $FILE`; then echo "$OUT" | head -n 1 | sed -e "s@^$KEY:\s*@@i" | sed -e "s@\s*#.*@@" return 0 fi return 1 } singularity_keys_get() { KEY="${1:-}" FILE="${2:-}" egrep -i "^$KEY:" "$FILE" | while read i; do echo "$i" | sed -e "s@^$KEY:\s*@@i" | sed -e "s@\s*#.*@@" done | tr '\n' ' ' echo return 0 } singularity_daemon_glob() { if ! USERID=`id -ru`; then message ERROR "Could not ascertain user ID\n" exit 255 fi if ! HOST=`hostname`; then message ERROR "Could not ascertain hostname\n" ABORT 255 fi if ! HOME=`getent passwd ${USERID} | cut -d: -f6`; then message ERROR "Could not discover user's home directory\n" ABORT 255 fi for arg in "$@"; do for i in ${HOME}/.singularity/daemon/${HOST}/${arg}; do echo "$i" done done return 0 } singularity_daemon_file() { SINGULARITY_DAEMON_NAME="${1:-}" if [ -z "${SINGULARITY_DAEMON_NAME:-}" ]; then message ERROR "singularity_daemon_file() called with no process name\n" ABORT 255 fi if ! USERID=`id -ru`; then message ERROR "Could not ascertain user ID\n" ABORT 255 fi if ! HOST=`hostname`; then message ERROR "Could not ascertain hostname\n" ABORT 255 fi if ! HOME=`getent passwd ${USERID} | cut -d: -f6`; then message ERROR "Could not discover user's home directory\n" ABORT 255 fi # This is a configurable option, needs to be set accordingly! SINGULARITY_DAEMON_FILE="${HOME}/.singularity/daemon/${HOST}/${SINGULARITY_DAEMON_NAME}" export SINGULARITY_DAEMON_FILE SINGULARITY_DAEMON_NAME message 2 "Using Daemon configuration file: ${SINGULARITY_DAEMON_FILE}\n" return 0 } singularity_calculate_size() { FOLDER="${1:-}" if [ ! -d "${FOLDER}" ]; then message ERROR "Folder not found ($FOLDER)\n" fi eval du -sm ${FOLDER} | cut -f1 # MB return 0 } ABORT() { RETVAL="${1:-}" if [ -z "$RETVAL" ]; then RETVAL=1 fi message ABORT "Aborting with RETVAL=$RETVAL\n" exit $RETVAL } check_pattern() { STRING="${1:-}" PATTERN="${2:-}" case "$PATTERN" in $STRING) true ;; *) return 1 ;; esac return 0 } cmd() { message 2 " + %-68.68s" "$*" "$@" >/dev/null 2>&1 RETVAL=$? if [ $RETVAL -eq 0 ]; then message 2 "OK\n" else message 2 "ERROR\n" fi return $RETVAL } eval_abort() { eval "$@" RETVAL=$? if [ $RETVAL -ne 0 ]; then exit $RETVAL fi return 0 } stest() { ERROR="${1:-}" TMPFILE=`mktemp` shift message 2 " + %-80.80s " "$*" "$@" >$TMPFILE 2>&1 CODE="$?" if [ "$ERROR" = "0" -a "$CODE" != "0" ]; then message 2 "%13s ERROR\n" "(retval=$CODE)" tail "$TMPFILE" echo "Full output in: $TMPFILE" exit 1 elif [ "$ERROR" != "0" -a "$CODE" = "0" ]; then message 2 "%13s ERROR\n" "(retval=$CODE)" tail "$TMPFILE" echo "Full output in: $TMPFILE" exit 1 else message 2 "%13s OK\n" "(retval=$CODE)" fi rm -f "$TMPFILE" } singularity_import() { MOD="${1:-}" if [ -z "$SINGULARITY_libexecdir" ]; then message ERROR "libexecdir not defined, are you running this from within Singularity?\n" exit 1 fi if [ -f "$SINGULARITY_libexecdir/singularity/mods/$MOD.smod" ]; then . "$SINGULARITY_libexecdir/singularity/mods/$MOD.smod" else message ERROR "Could not load Singularity module: $MOD\n" exit 255 fi return 0 } # Different versions of which respond differently (print aliases, or take # different arguments) singularity_which() { i="${1:-}" # Avoid non-pathnames with . not in path, and directories case $i in .* | /*) if [ -f "$i" -a -x "$i" ]; then echo "$i" return 0 fi esac for p in `echo $PATH | sed -e 's/:/ /g'`; do if [ -f "$p/$i" -a -x "$p/$i" ]; then echo "$p/$i" return 0 fi done return 1 } parse_opts() { NEWOPTS="" while [ -n "${1:-}" ]; do case "${1:-}" in -*=*) ARG1=`echo -n "${1:-}" | cut -d = -f 1` ARG2=`echo -n "${1:-}" | cut -d = -f 2` NEWOPTS="$NEWOPTS \"$ARG1\" \"$ARG2\"" shift continue ;; --*) NEWOPTS="$NEWOPTS \"${1:-}\"" shift continue ;; -*) for o in `echo "${1:-}"| sed 's/^-//' | sed 's/./-& /g'`; do NEWOPTS="$NEWOPTS \"$o\"" done shift continue ;; *) NEWOPTS="$NEWOPTS $@" break ;; esac done # Eww, this is bad I know... Would be better just to pass the variable # around without making ad-hoc modifications... Got a better idea, let # me know! (gmk) echo "$NEWOPTS" | sed -e 's/\\/\\\\/g' } compare_envs() { # during bootstrap, if we are bootstrapping from a local image, compare the # environment in the new image to that in the old image and alert the user # of any changes REMOVED=$(comm -23 ${SINGULARITY_STARTING_ENVIRONMENT} ${SINGULARITY_ENDING_ENVIRONMENT}) ADDED=$(comm -13 ${SINGULARITY_STARTING_ENVIRONMENT} ${SINGULARITY_ENDING_ENVIRONMENT}) if [ -n "${REMOVED:-}" -o -n "${ADDED:-}" ]; then message 1 "Environment variables were added, removed, and/or changed during bootstrap.\n" elif [ "${SINGULARITY_STARTING_ENVSHA1:-}" != "${SINGULARITY_ENDING_ENVSHA1:-}" ]; then message 1 "The environment of ${SINGULARITY_IMAGE} may differ from the environment of ${FROM}\n" message 1 "${FROM} sha1 sum of environment is ${SINGULARITY_STARTING_ENVSHA1}\n" message 1 "${SINGULARITY_IMAGE} sha1 sum of environment is ${SINGULARITY_ENDING_ENVSHA1}\n" fi if [ -n "${REMOVED:-}" ]; then message 1 "Variables unique to original image (${FROM})\n" message 1 "$REMOVED\n" fi if [ -n "${ADDED:-}" ]; then message 1 "Variables unique to new image (${SINGULARITY_IMAGE})\n" message 1 "$ADDED\n" fi } replace_string() { ORIGINAL="${1:-}" MATCH="${2:-}" REPLACE="${3:-}" echo $ORIGINAL | sed -e s/${MATCH}/${REPLACE}/g } zcat_compat() { if [ -z "${1:-}" ]; then message FAIL "zcat_compat: missing argument\n" exit 1 fi FILE=$1 HEAD_CHUNK=`head -c 3 $FILE | od -A n -t x1 -w3 | tr -d ' '` if [ "$HEAD_CHUNK" == "1f8b08" ]; then zcat "$1" else cat "$1" fi } nonroot_build_warning() { USERID=`id -ru` if [ "$USERID" != "0" ]; then message WARNING "Building container as an unprivileged user. If you run this container as root\n"; message WARNING "it may be missing some functionality.\n" fi } is_deffile() { # check if a file looks like, walks like, and quacks like a def file FILE2CHECK=$1 # first make a guess based on naming convention case ${FILE2CHECK:-} in *.img) return 1 ;; *.tar*) return 1 ;; *.def) return 0 ;; Singularity) return 0 ;; *) ;; esac # name is ambiguous. is it a text file with def file words in it? if ! `file $FILE2CHECK | grep -q "ASCII\|Unicode"`; then return 1 fi grep -qiE "^bootstrap:|^%setup|^%post|^%test|^%labels|^%files|^%runscript|^%environment" "$FILE2CHECK" RETVAL=$? return $RETVAL } is_image() { # check if a file looks like, walks like, and quacks like an image file if [ -z "${1:-}" ]; then return 1 fi eval "$SINGULARITY_libexecdir/singularity/bin/image-type" "$1" >/dev/null 2>&1 RETVAL=$? return $RETVAL } is_tar() { # check if a file looks like, walks like,... oh you get the idea FILE2CHECK=$1 TARMAGIC="7573746172202000" # "ustar \x00" zcat_compat "$FILE2CHECK" | head -c 320 | od -A n -t x1 -w320 | tr -d ' ' | grep -q "$TARMAGIC" RETVAL=$? return $RETVAL } ############################################################################## # CHECKS ############################################################################## # Given a script and an assigned level and tag, determine if it should be # executed to perform the check # [LEVEL] [SCRIPT] [TAGS] exec_check() { CHECKSUCCESS="${1:-}" CHECKLEVEL="${2:-}" CHECKSCRIPT="${3:-}" shift shift shift CHECKTAGS="${@}" # The default case is that we don't run RETVAL=0 # Don't perform check if tag not indicated PERFORM_CHECK=1 # if not "all," check if user wants check performed if [ "$SINGULARITY_CHECKTAGS" != "all" ]; then has_tag "$CHECKTAGS" "$SINGULARITY_CHECKTAGS" || PERFORM_CHECK=0 fi if [ $PERFORM_CHECK -eq 0 ]; then return $RETVAL fi RUNCHECK_LEVEL=3 # Perform all checks case "$CHECKLEVEL" in l|low|L|LOW) RUNCHECK_LEVEL=3 ;; m|med|M|MED) RUNCHECK_LEVEL=2 ;; h|high|H|HIGH) RUNCHECK_LEVEL=1 ;; esac if [ "$RUNCHECK_LEVEL" -le "$SINGULARITY_CHECKLEVEL" ]; then message NOTIFY "START ${CHECKSCRIPT##*/} tags[${CHECKTAGS}] level[$CHECKLEVEL]\n" scheck 0 $CHECKSCRIPT # script must have interpreter specified printf " $CHECKSCRIPT \n" fi return $RETVAL } has_tag() { taglist="$1" tag="$2" if test "${taglist#*$tag}" != "$taglist" then return 0 # has tag, perform test else return 1 # no tag, skip test fi } scheck() { ERROR="${1:-}" shift message 2 " + %-80.80s " "$*" "$@" CODE="$?" if [ "$ERROR" = "0" -a "$CODE" != "0" ]; then message FAIL "(retval=$CODE)" elif [ "$ERROR" != "0" -a "$CODE" = "0" ]; then message FAIL "(retval=$CODE)" else message PASS "(retval=$CODE)" fi return $CODE } atexit() { trap "$1" EXIT } if [ -n "${SHELL_DEBUG:-}" ]; then set -x fi singularity-2.4.2/libexec/bootstrap-scripts/0000755000175000017500000000000013211621077020144 5ustar mehdimehdisingularity-2.4.2/libexec/bootstrap-scripts/main-deffile.sh0000644000175000017500000000672413211621077023031 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/pre.sh" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/environment.sh" if [ -n "${BOOTSTRAP:-}" -a -z "${SINGULARITY_BUILDNOBASE:-}" ]; then if [ -x "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/deffile-driver-$BOOTSTRAP.sh" ]; then if [ ! -f "${SINGULARITY_ROOTFS}/.coredone" ]; then eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/deffile-driver-$BOOTSTRAP.sh" touch "${SINGULARITY_ROOTFS}/.coredone" fi else message ERROR "'Bootstrap' type not supported: $BOOTSTRAP\n" exit 1 fi fi # take a snapshot of the environment for later comparison if [ "${BOOTSTRAP:-}" = "localimage" -o "${BOOTSTRAP:-}" = "shub" ]; then SINGULARITY_STARTING_ENVIRONMENT=$(eval_abort env -i ${SINGULARITY_libexecdir}/singularity/helpers/record-env.sh ${SINGULARITY_ROOTFS}) SINGULARITY_STARTING_ENVSHA1=$(eval_abort sha1sum ${SINGULARITY_ROOTFS}/.singularity.d/env/*.sh | sha1sum) export SINGULARITY_STARTING_ENVIRONMENT SINGULARITY_STARTING_ENVSHA1 fi eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/deffile-sections.sh" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/post.sh" # take another snapshot and compare to see what changed if [ "${BOOTSTRAP:-}" = "localimage" -o "${BOOTSTRAP:-}" = "shub" ]; then SINGULARITY_ENDING_ENVIRONMENT=$(eval_abort env -i ${SINGULARITY_libexecdir}/singularity/helpers/record-env.sh ${SINGULARITY_ROOTFS}) SINGULARITY_ENDING_ENVSHA1=$(eval_abort sha1sum ${SINGULARITY_ROOTFS}/.singularity.d/env/*.sh | sha1sum) export SINGULARITY_ENDING_ENVIRONMENT SINGULARITY_ENDING_ENVSHA1 compare_envs rm $SINGULARITY_STARTING_ENVIRONMENT $SINGULARITY_ENDING_ENVIRONMENT fi # If checks specified, export variable if [ "${SINGULARITY_CHECKS:-}" = "no" ]; then message 1 "Skipping checks\n" else message 1 "Running checks\n" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/checks.sh" fi singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-self.sh0000644000175000017500000000502113211621077024314 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi ########## BEGIN BOOTSTRAP SCRIPT ########## umask 0002 if ! GUNZIP_PATH=`singularity_which gunzip`; then message ERROR "gunzip is not in PATH... Perhaps 'apt-get install' it?\n" exit 1 fi # By default, we clone from root unless specified otherwise if [ -z "${FROM:-}" ]; then FROM='/' fi message 1 "Cloning from $FROM\m" message 1 "Preparing contents to bootstrap image by self clone with base $FROM\n" SINGULARITY_DUMP=`mktemp /tmp/.singularity-layers.XXXXXXXX.tgz` export SINGULARITY_DUMP # The user can specify custom exclusions if [ -z "${EXCLUDE:-}" ]; then EXCLUDE='' else message 1 "Custom exclusions: $EXCLUDE\n" fi CUSTOM_EXCLUSIONS=$(echo "$EXCLUDE" | sed 's/[^ ]* */--exclude &/g') # Extract the host into a container tar --one-file-system -czvSf $SINGULARITY_DUMP --exclude $SINGULARITY_DUMP --exclude $HOME --exclude $SINGULARITY_libexecdir --exclude ${TMPDIR-/tmp} --exclude $SINGULARITY_libexecdir/singularity $CUSTOM_EXCLUSIONS --exclude /usr/src $FROM eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/pre.sh" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/environment.sh" message 1 "Extracting self into new image\n" cd $SINGULARITY_ROOTFS && gunzip -dc $SINGULARITY_DUMP rm -f "$SINGULARITY_DUMP" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/post.sh" singularity-2.4.2/libexec/bootstrap-scripts/functions0000644000175000017500000001264213211621077022104 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # ############################################################################## # SECTIONS ############################################################################## singularity_section_exists() { SECTION="${1:-}" FILE="${2:-}" TOGGLE="" if [ ! -f "$FILE" ]; then message ERROR "File not found ($FILE)\n" exit 1 fi if ! egrep -i -q -- "^%$SECTION\W*" "$FILE"; then return 1 fi return 0 } singularity_section_args() { SECTION="${1:-}" FILE="${2:-}" TOGGLE="" if [ ! -f "$FILE" ]; then message ERROR "File not found ($FILE)\n" exit 1 fi egrep -i -- "^%$SECTION\W*" "$FILE" | sed -e "s@%$SECTION\s*@@i" return 0 } singularity_section_get() { SECTION="${1:-}" FILE="${2:-}" if [ ! -x "$SINGULARITY_libexecdir/singularity/bin/get-section" ]; then message ERROR "Could not locate get-section program\n" exit 255 fi if [ ! -f "$FILE" ]; then message ERROR "File not found ($FILE)\n" exit 1 fi eval "$SINGULARITY_libexecdir/singularity/bin/get-section" "$SECTION" "$FILE" return 0 } get_section() { SECTION="$1" RECIPE="$2" FOUND="no" while read line do if echo $line | grep -q -e "^%$SECTION$"; then # If not found, continue if [[ "$FOUND" == "no" ]]; then FOUND="yes" else return 0 fi # We didn't match this line but... else # It's the start of a new section if echo $line | grep -q -e "^%"; then # We already found it, we're done if [[ "$FOUND" == "yes" ]]; then return 0 fi else # We already found it, still in section if [[ "$FOUND" == "yes" ]]; then if [ ! -z "${line}" ]; then echo $line fi fi fi fi done < $RECIPE } ############################################################################## # APPS ############################################################################## singularity_app_install_get() { APPNAME="${1:-}" FILE="${2:-}" if [ ! -f "$FILE" ]; then message ERROR "File not found ($FILE)\n" exit 1 fi # To install application, much change to its base and back so paths are relative SECTION="appinstall ${APPNAME}" printf "cd /scif/apps/${APPNAME}\n" get_section "$SECTION" "$FILE" return 0 } singularity_app_save() { APPNAME="${1:-}" FILE="${2:-}" OUTPUT_FILE="${3:-}" if [ ! -x "$SINGULARITY_libexecdir/singularity/bin/get-section" ]; then message ERROR "Could not locate get-section program\n" exit 255 fi if [ ! -f "$FILE" ]; then message ERROR "File not found ($FILE)\n" exit 1 fi SECTION_FOUND="No" while read line do if [[ "$line" == "%appinstall $APPNAME" ]]; then # start of section echo "$line" >> $OUTPUT_FILE SECTION_FOUND="Yes" elif [[ "$line" == %* ]]; then # end of section if [[ "$SECTION_FOUND" == "Yes" ]]; then return 0 fi else if [[ "$SECTION_FOUND" == "Yes" ]]; then echo "$line" >> $OUTPUT_FILE # line in section fi fi done < "$FILE" return 0 } singularity_app_init() { APPNAME="${1:-}" APPROOT="${2:-}" APPBASE="${APPROOT}/scif/apps/${APPNAME}" APPMETA="${APPBASE}/scif" if [ ! -d "${APPBASE}" ]; then mkdir -p "${APPBASE}/scif" mkdir -p "${APPBASE}/bin" mkdir -p "${APPBASE}/lib" mkdir -p "${APPMETA}" mkdir -p "${APPMETA}/env" message 2 "Creating ${APPBASE}...\n" fi APPDATA="${APPROOT}/scif/data/${APPNAME}" if [ ! -d "${APPDATA}" ]; then mkdir -p ${APPDATA} mkdir -p "${APPDATA}/input" mkdir -p "${APPDATA}/output" message 2 "Creating ${APPDATA}...\n" fi # App specific variables should be exposed at run, shell, exec echo " SINGULARITY_APPNAME=$APPNAME SINGULARITY_APPROOT=\"/scif/apps/${APPNAME}\" SINGULARITY_APPMETA=\"/scif/apps/${APPNAME}/scif\" SINGULARITY_DATA=\"/scif/data\" SINGULARITY_APPDATA=\"/scif/data/${APPNAME}\" SINGULARITY_APPINPUT=\"/scif/data/${APPNAME}/input\" SINGULARITY_APPOUTPUT=\"/scif/data/${APPNAME}/output\" export SINGULARITY_APPDATA SINGULARITY_APPNAME SINGULARITY_APPROOT SINGULARITY_APPMETA SINGULARITY_APPINPUT SINGULARITY_APPOUTPUT SINGULARITY_DATA " > "${APPMETA}/env/01-base.sh" } singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-docker.sh0000644000175000017500000000556613211621077024650 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi if [ -z "${FROM:-}" ]; then message ERROR "Required Definition tag 'From:' not defined.\n" exit 1 fi ################################################################################ # Docker Customizations ################################################################################ if [ ! -z "${REGISTRY:-}" ]; then message DEBUG "Custom Docker Registry 'Registry:' ${REGISTRY}.\n" export REGISTRY fi # Note: NAMESPACE can be set to an empty string, and that's a valid namespace # for Docker (not so for shub://) if [ -n "${NAMESPACE+set}" ]; then message DEBUG "Custom Docker Namespace 'Namespace:' ${NAMESPACE}.\n" export NAMESPACE fi if [ -z "${INCLUDECMD:-}" ]; then export SINGULARITY_INCLUDECMD="yes" fi SINGULARITY_CONTAINER="docker://$FROM" SINGULARITY_LABELFILE="$SINGULARITY_ROOTFS/.singularity.d/labels.json" if ! SINGULARITY_CONTENTS=`mktemp ${TMPDIR:-/tmp}/.singularity-layers.XXXXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi export SINGULARITY_CONTAINER SINGULARITY_CONTENTS SINGULARITY_LABELFILE eval_abort "$SINGULARITY_libexecdir/singularity/python/import.py" umask 0002 for i in `cat "$SINGULARITY_CONTENTS"`; do name=`basename "$i"` message 1 "Exploding layer: $name\n" zcat "$i" | (cd "$SINGULARITY_ROOTFS"; tar --exclude=dev/* -xf -) || exit $? done rm -f "$SINGULARITY_CONTENTS" exit 0 singularity-2.4.2/libexec/bootstrap-scripts/environment/0000755000175000017500000000000013211621077022510 5ustar mehdimehdisingularity-2.4.2/libexec/bootstrap-scripts/environment/01-base.sh0000644000175000017500000000204513211621077024175 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016-2017, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # singularity-2.4.2/libexec/bootstrap-scripts/environment/90-environment.sh0000644000175000017500000000005713211621077025640 0ustar mehdimehdi# Custom environment shell code should follow singularity-2.4.2/libexec/bootstrap-scripts/environment/runscript0000644000175000017500000000010613211621077024461 0ustar mehdimehdi#!/bin/sh echo "There is no runscript defined for this container\n"; singularity-2.4.2/libexec/bootstrap-scripts/environment/run0000644000175000017500000000113313211621077023235 0ustar mehdimehdi#!/bin/sh for script in /.singularity.d/env/*.sh; do if [ -f "$script" ]; then . "$script" fi done if test -n "${SINGULARITY_APPNAME:-}"; then if test -x "/scif/apps/${SINGULARITY_APPNAME:-}/scif/runscript"; then exec "/scif/apps/${SINGULARITY_APPNAME:-}/scif/runscript" "$@" else echo "No Singularity runscript for contained app: ${SINGULARITY_APPNAME:-}" exit 1 fi elif test -x "/.singularity.d/runscript"; then exec "/.singularity.d/runscript" "$@" else echo "No Singularity runscript found, executing /bin/sh" exec /bin/sh "$@" fi singularity-2.4.2/libexec/bootstrap-scripts/environment/exec0000644000175000017500000000017713211621077023364 0ustar mehdimehdi#!/bin/sh for script in /.singularity.d/env/*.sh; do if [ -f "$script" ]; then . "$script" fi done exec "$@" singularity-2.4.2/libexec/bootstrap-scripts/environment/95-apps.sh0000644000175000017500000000377213211621077024253 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. if test -n "${SINGULARITY_APPNAME:-}"; then # The active app should be exported export SINGULARITY_APPNAME if test -d "/scif/apps/${SINGULARITY_APPNAME:-}/"; then SINGULARITY_APPS="/scif/apps" SINGULARITY_APPROOT="/scif/apps/${SINGULARITY_APPNAME:-}" export SINGULARITY_APPROOT SINGULARITY_APPS PATH="/scif/apps/${SINGULARITY_APPNAME:-}:$PATH" # Automatically add application bin to path if test -d "/scif/apps/${SINGULARITY_APPNAME:-}/bin"; then PATH="/scif/apps/${SINGULARITY_APPNAME:-}/bin:$PATH" fi # Automatically add application lib to LD_LIBRARY_PATH if test -d "/scif/apps/${SINGULARITY_APPNAME:-}/lib"; then LD_LIBRARY_PATH="/scif/apps/${SINGULARITY_APPNAME:-}/lib:$LD_LIBRARY_PATH" export LD_LIBRARY_PATH fi # Automatically source environment if [ -f "/scif/apps/${SINGULARITY_APPNAME:-}/scif/env/01-base.sh" ]; then . "/scif/apps/${SINGULARITY_APPNAME:-}/scif/env/01-base.sh" fi if [ -f "/scif/apps/${SINGULARITY_APPNAME:-}/scif/env/90-environment.sh" ]; then . "/scif/apps/${SINGULARITY_APPNAME:-}/scif/env/90-environment.sh" fi export PATH else echo "Could not locate the container application: ${SINGULARITY_APPNAME}" exit 1 fi fi singularity-2.4.2/libexec/bootstrap-scripts/environment/99-base.sh0000644000175000017500000000234713211621077024223 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016-2017, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # if [ -z "$LD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="/.singularity.d/libs" else LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/.singularity.d/libs" fi PS1="Singularity> " export LD_LIBRARY_PATH PS1 singularity-2.4.2/libexec/bootstrap-scripts/environment/test0000644000175000017500000000111013211621077023403 0ustar mehdimehdi#!/bin/sh for script in /.singularity.d/env/*.sh; do if [ -f "$script" ]; then . "$script" fi done if test -z "${SINGULARITY_APPNAME:-}"; then if test -x "/scif/apps/${SINGULARITY_APPNAME:-}/scif/test"; then exec "/scif/apps/${SINGULARITY_APPNAME:-}/scif/test" "$@" else echo "No Singularity tests for contained app: ${SINGULARITY_APPNAME:-}" exit 1 fi elif test -x "/.singularity.d/test"; then exec "/.singularity.d/test" "$@" else echo "No Singularity container test found, executing /bin/sh" exec /bin/sh "$@" fi singularity-2.4.2/libexec/bootstrap-scripts/environment/startscript0000644000175000017500000000001213211621077025006 0ustar mehdimehdi#!/bin/sh singularity-2.4.2/libexec/bootstrap-scripts/environment/Makefile.am0000644000175000017500000000361613211621077024552 0ustar mehdimehdienvironmentdir = $(libexecdir)/singularity/bootstrap-scripts/ CLEANFILES = environment.tar EXTRA_DIST = 01-base.sh 90-environment.sh 95-apps.sh 99-base.sh exec run runscript shell test start startscript dist_environment_DATA = environment.tar .PHONY: environment.tar environment.tar: install -d -m 0755 newroot install -d -m 0755 newroot/.singularity.d install -d -m 0755 newroot/.singularity.d/env install -d -m 0755 newroot/.singularity.d/actions install -d -m 0755 newroot/.singularity.d/libs install -d -m 0755 newroot/dev install -d -m 0755 newroot/etc install -d -m 0755 newroot/home install -d -m 0755 newroot/proc install -d -m 0755 newroot/root install -d -m 0755 newroot/sys install -d -m 0755 newroot/tmp install -d -m 0755 newroot/var install -d -m 0755 newroot/var/tmp install -m 0755 /dev/null newroot/etc/hosts install -m 0755 /dev/null newroot/etc/resolv.conf install -m 0755 runscript newroot/.singularity.d/ install -m 0755 startscript newroot/.singularity.d/ install -m 0755 exec newroot/.singularity.d/actions/ install -m 0755 run newroot/.singularity.d/actions/ install -m 0755 shell newroot/.singularity.d/actions/ install -m 0755 test newroot/.singularity.d/actions/ install -m 0755 start newroot/.singularity.d/actions/ install -m 0755 01-base.sh newroot/.singularity.d/env/ install -m 0755 90-environment.sh newroot/.singularity.d/env/ install -m 0755 95-apps.sh newroot/.singularity.d/env/ install -m 0755 99-base.sh newroot/.singularity.d/env/ ln -sf .singularity.d/env/90-environment.sh newroot/environment ln -sf .singularity.d/runscript newroot/singularity ln -sf .singularity.d/actions/exec newroot/.exec ln -sf .singularity.d/actions/run newroot/.run ln -sf .singularity.d/actions/shell newroot/.shell ln -sf .singularity.d/actions/test newroot/.test cd newroot; tar czf ../environment.tar . --owner=0 --group=0 rm -rf newroot MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/bootstrap-scripts/environment/start0000644000175000017500000000043313211621077023570 0ustar mehdimehdi#!/bin/sh # if we are here start notify PID 1 to continue # DON'T REMOVE kill -CONT 1 for script in /.singularity.d/env/*.sh; do if [ -f "$script" ]; then . "$script" fi done if test -x "/.singularity.d/startscript"; then exec "/.singularity.d/startscript" fi singularity-2.4.2/libexec/bootstrap-scripts/environment/shell0000644000175000017500000000115213211621077023541 0ustar mehdimehdi#!/bin/sh for script in /.singularity.d/env/*.sh; do if [ -f "$script" ]; then . "$script" fi done if test -n "$SINGULARITY_SHELL" -a -x "$SINGULARITY_SHELL"; then exec $SINGULARITY_SHELL "$@" echo "ERROR: Failed running shell as defined by '\$SINGULARITY_SHELL'" 1>&2 exit 1 elif test -x /bin/bash; then SHELL=/bin/bash PS1="Singularity $SINGULARITY_CONTAINER:\\w> " export SHELL PS1 exec /bin/bash --norc "$@" elif test -x /bin/sh; then SHELL=/bin/sh export SHELL exec /bin/sh "$@" else echo "ERROR: /bin/sh does not exist in container" 1>&2 fi exit 1 singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-debootstrap.sh0000644000175000017500000000535613211621077025724 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi ########## BEGIN BOOTSTRAP SCRIPT ########## umask 0002 if ! DEBOOTSTRAP_PATH=`singularity_which debootstrap`; then message ERROR "debootstrap is not in PATH... Perhaps 'apt-get install' it?\n" exit 1 fi if [ -n "${ARCH:-}" ]; then ARCH=`echo ${ARCH:-} | sed -e 's/\s//g'` else ARCH=`uname -m` if [ "$ARCH" == "x86_64" ]; then ARCH=amd64 elif [ "$ARCH" == "ppc64le" ]; then ARCH=ppc64el elif [ "$ARCH" == "aarch64" ]; then ARCH=arm64 elif [ "$ARCH" == "armv6l" ]; then ARCH=armhf elif [ "$ARCH" == "armv7l" ]; then ARCH=armhf fi fi if [ -z "${MIRRORURL:-}" ]; then message ERROR "No 'MirrorURL' defined in bootstrap definition\n" ABORT 1 fi if [ -z "${OSVERSION:-}" ]; then message ERROR "No 'OSVersion' defined in bootstrap definition\n" ABORT 1 fi REQUIRES=`echo "${INCLUDE:-}" | sed -e 's/\s/,/g'` # The excludes save 25M or so with jessie. (Excluding udev avoids # systemd, for instance.) There are a few more we could exclude # to save a few MB. I see 182M cleaned with this, v. 241M with # the default debootstrap. if ! eval "$DEBOOTSTRAP_PATH --variant=minbase --exclude=openssl,udev,debconf-i18n,e2fsprogs --include=apt,$REQUIRES --arch=$ARCH '$OSVERSION' '$SINGULARITY_ROOTFS' '$MIRRORURL'"; then ABORT 255 fi singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-yum.sh0000644000175000017500000002020113211621077024172 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi ########## BEGIN BOOTSTRAP SCRIPT ########## umask 0002 install -d -m 0755 "$SINGULARITY_ROOTFS/dev" cp -a /dev/null "$SINGULARITY_ROOTFS/dev/null" 2>/dev/null || > "$SINGULARITY_ROOTFS/dev/null" cp -a /dev/zero "$SINGULARITY_ROOTFS/dev/zero" 2>/dev/null || > "$SINGULARITY_ROOTFS/dev/zero" cp -a /dev/random "$SINGULARITY_ROOTFS/dev/random" 2>/dev/null || > "$SINGULARITY_ROOTFS/dev/random" cp -a /dev/urandom "$SINGULARITY_ROOTFS/dev/urandom" 2>/dev/null || > "$SINGULARITY_ROOTFS/dev/urandom" # dnf should probably be preferred if it's present, at some point we will make # a dnf specific bootstrap module. if INSTALL_CMD=`singularity_which dnf`; then message 1 "Found DNF at: $INSTALL_CMD\n" elif INSTALL_CMD=`singularity_which yum`; then message 1 "Found YUM at: $INSTALL_CMD\n" INSTALL_CMD="$INSTALL_CMD --tolerant" else message ERROR "Neither yum nor dnf in PATH!\n" ABORT 1 fi # Check for RPM's dbpath not being /var/lib/rpm RPM_CMD=`singularity_which rpm` if [ -z "${RPM_CMD:-}" ]; then message ERROR "rpm not in PATH!\n" ABORT 1 fi RPM_DBPATH=$(rpm --showrc | grep -E ":\s_dbpath\s" | cut -f2) if [ "$RPM_DBPATH" != '%{_var}/lib/rpm' ]; then message ERROR "RPM database is using a weird path: %s\n" "$RPM_DBPATH" message WARNING "You are probably running this bootstrap on Debian or Ubuntu.\n" message WARNING "There is a way to work around this problem:\n" message WARNING "Create a file at path %s/.rpmmacros.\n" "$HOME" message WARNING "Place the following lines into the '.rpmmacros' file:\n" message WARNING "%s\n" '%_var /var' message WARNING "%s\n" '%_dbpath %{_var}/lib/rpm' message WARNING "After creating the file, re-run the bootstrap.\n" message WARNING "More info: https://github.com/singularityware/singularity/issues/241\n" ABORT 1 fi if [ -z "${OSVERSION:-}" ]; then if [ -f "/etc/redhat-release" ]; then OSVERSION=`rpm -qf --qf '%{VERSION}' /etc/redhat-release` else OSVERSION=7 fi fi MIRROR=`echo "${MIRRORURL:-}" | sed -r "s/%\{?OSVERSION\}?/$OSVERSION/gi"` MIRROR_META=`echo "${METALINK:-}" | sed -r "s/%\{?OSVERSION\}?/$OSVERSION/gi"` if [ -z "${MIRROR:-}" ] && [ -z "${MIRROR_META:-}" ]; then message ERROR "No 'MirrorURL' or 'MetaLink' defined in bootstrap definition\n" ABORT 1 fi MIRROR_UPDATES=`echo "${UPDATEURL:-}" | sed -r "s/%\{?OSVERSION\}?/$OSVERSION/gi"` MIRROR_UPDATES_META=`echo "${UPDATEMETALINK:-}" | sed -r "s/%\{?OSVERSION\}?/$OSVERSION/gi"` if [ -n "${MIRROR_UPDATES:-}" ] || [ -n "${MIRROR_UPDATES_META:-}" ]; then message 1 "'UpdateURL' or 'UpdateMetaLink' defined in bootstrap definition\n" fi REPO_COUNT=0 YUM_CONF="/etc/bootstrap-yum.conf" export YUM_CONF # Create the main portion of yum config mkdir -p "$SINGULARITY_ROOTFS" YUM_CONF_DIRNAME=`dirname $YUM_CONF` mkdir -m 0755 -p "$SINGULARITY_ROOTFS/$YUM_CONF_DIRNAME" > "$SINGULARITY_ROOTFS/$YUM_CONF" echo "[main]" >> "$SINGULARITY_ROOTFS/$YUM_CONF" if [ -n "${http_proxy:-}" ]; then echo "proxy=$http_proxy" >> "$SINGULARITY_ROOTFS/$YUM_CONF" fi echo 'cachedir=/var/cache/yum-bootstrap' >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "keepcache=0" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "debuglevel=2" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "logfile=/var/log/yum.log" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "syslog_device=/dev/null" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "exactarch=1" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "obsoletes=1" >> "$SINGULARITY_ROOTFS/$YUM_CONF" if [ -n "${GPG:-}" ]; then echo "gpgcheck=1" >> "$SINGULARITY_ROOTFS/$YUM_CONF" else echo "gpgcheck=0" >> "$SINGULARITY_ROOTFS/$YUM_CONF" fi echo "plugins=1" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "reposdir=0" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "deltarpm=0" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo "[base]" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo 'name=Linux $releasever - $basearch' >> "$SINGULARITY_ROOTFS/$YUM_CONF" if [ -n "${MIRROR:-}" ]; then echo "baseurl=$MIRROR" >> "$SINGULARITY_ROOTFS/$YUM_CONF" fi if [ -n "${MIRROR_META:-}" ]; then echo "metalink=$MIRROR_META" >> "$SINGULARITY_ROOTFS/$YUM_CONF" fi echo "enabled=1" >> "$SINGULARITY_ROOTFS/$YUM_CONF" if [ -n "${GPG:-}" ]; then echo "gpgcheck=1" >> "$SINGULARITY_ROOTFS/$YUM_CONF" else echo "gpgcheck=0" >> "$SINGULARITY_ROOTFS/$YUM_CONF" fi if [ -n "${MIRROR_UPDATES:-}" ] || [ -n "${MIRROR_UPDATES_META:-}" ]; then echo "[updates]" >> "$SINGULARITY_ROOTFS/$YUM_CONF" echo 'name=Linux $releasever - $basearch updates' >> "$SINGULARITY_ROOTFS/$YUM_CONF" if [ -n "${MIRROR_UPDATES:-}" ]; then echo "baseurl=${MIRROR_UPDATES}" >> "$SINGULARITY_ROOTFS/$YUM_CONF" fi if [ -n "${MIRROR_UPDATES_META:-}" ]; then echo "metalink=$MIRROR_UPDATES_META" >> "$SINGULARITY_ROOTFS/$YUM_CONF" fi echo "enabled=1" >> "$SINGULARITY_ROOTFS/$YUM_CONF" if [ -n "${GPG:-}" ]; then echo "gpgcheck=1" >> "$SINGULARITY_ROOTFS/$YUM_CONF" else echo "gpgcheck=0" >> "$SINGULARITY_ROOTFS/$YUM_CONF" fi fi echo "" >> "$SINGULARITY_ROOTFS/$YUM_CONF" # If GPG is specified, then we need to import a key from somewhere. if [ -n "${GPG:-}" ]; then message 1 "We have a GPG key! Preparing RPM database.\n" if ! eval rpm --root $SINGULARITY_ROOTFS --initdb; then message ERROR "Failed to create rpmdb!" ABORT 255 fi # RPM will import from the web, if curl is installed, so check for it. if [ ${GPG:0:8} = 'https://' ]; then if CURL_CMD=`singularity_which curl`; then message 1 "Found curl at: $CURL_CMD\n" else message ERROR "curl not in PATH!\n" ABORT 1 fi fi # Before importing, check for (and fail on) HTTP URLs. # Then let RPM handle everything for us! if [ ${GPG:0:7} = 'http://' ]; then message ERROR "It is unsafe to fetch a GPG key with an HTTP URL.\n" ABORT 255 else if ! eval $RPM_CMD --root $SINGULARITY_ROOTFS --import $GPG; then message ERROR "Failed to import downloaded GPG key.\n" ABORT 255 fi message 1 "GPG key import complete!\n" fi else message 1 "Skipping GPG key import.\n" fi # Do the install! if ! eval "$INSTALL_CMD --noplugins -c $SINGULARITY_ROOTFS/$YUM_CONF --installroot $SINGULARITY_ROOTFS --releasever=${OSVERSION} -y install /etc/redhat-release coreutils ${INCLUDE:-}"; then message ERROR "Bootstrap failed... exiting\n" ABORT 255 fi if [ -f "/etc/yum.conf" ]; then if [ -n "${http_proxy:-}" ]; then sed -i -e "s/\[main\]/\[main\]\nproxy=$http_proxy/" /etc/yum.conf elif [ -n "${HTTP_PROXY:-}" ]; then sed -i -e "s/\[main\]/\[main\]\nproxy=$HTTP_PROXY/" /etc/yum.conf fi fi if ! eval "rm -rf $SINGULARITY_ROOTFS/var/cache/yum-bootstrap"; then message WARNING "Failed cleaning Bootstrap packages\n" fi # If we got here, exit... exit 0 singularity-2.4.2/libexec/bootstrap-scripts/pre.sh0000644000175000017500000000342313211621077021270 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -f "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/functions" ]; then . "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/bootstrap-scripts/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi umask 0002 install -d -m 0755 "$SINGULARITY_ROOTFS" install -d -m 0755 "$SINGULARITY_ROOTFS/.singularity.d" install -d -m 0755 "$SINGULARITY_ROOTFS/.singularity.d/env" if [ -f "$SINGULARITY_BUILDDEF" ]; then ARGS=`singularity_section_args "pre" "$SINGULARITY_BUILDDEF"` singularity_section_get "pre" "$SINGULARITY_BUILDDEF" | /bin/sh -e -x $ARGS || ABORT 255 fi exit 0 singularity-2.4.2/libexec/bootstrap-scripts/checks.sh0000644000175000017500000000317413211621077021745 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . $SINGULARITY_libexecdir/singularity/functions else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi SINGULARITY_MOUNTPOINT=$SINGULARITY_ROOTFS RETVAL=0 # Only run if --checks/--check flag present if [ -z "${SINGULARITY_CHECKS:-}" ]; then exit $RETVAL fi # If no tag specified, run default if [ -z "${SINGULARITY_CHECKTAGS:-}" ]; then SINGULARITY_CHECKTAGS=default fi export SINGULARITY_CHECKTAGS SINGULARITY_CHECKLEVEL SINGULARITY_ROOTFS SINGULARITY_MOUNTPOINT eval "$SINGULARITY_libexecdir/singularity/helpers/check.sh" exit $RETVAL singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-zypper.sh0000644000175000017500000001167613211621077024731 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi ########## BEGIN BOOTSTRAP SCRIPT ########## umask 0002 install -d -m 0755 "$SINGULARITY_ROOTFS/dev" cp -a /dev/null "$SINGULARITY_ROOTFS/dev/null" 2>/dev/null || > "$SINGULARITY_ROOTFS/dev/null" cp -a /dev/zero "$SINGULARITY_ROOTFS/dev/zero" 2>/dev/null || > "$SINGULARITY_ROOTFS/dev/zero" cp -a /dev/random "$SINGULARITY_ROOTFS/dev/random" 2>/dev/null || > "$SINGULARITY_ROOTFS/dev/random" cp -a /dev/urandom "$SINGULARITY_ROOTFS/dev/urandom" 2>/dev/null || > "$SINGULARITY_ROOTFS/dev/urandom" # dnf should probably be preferred if it's present, at some point we will make # a dnf specific bootstrap module. if INSTALL_CMD=`singularity_which zypper`; then message 1 "Found Zypper at: $INSTALL_CMD\n" else message ERROR "Zypper not found in PATH!\n" ABORT 1 fi # Check for RPM's dbpath not being /var/lib/rpm RPM_CMD=`singularity_which rpm` if [ -z "${RPM_CMD:-}" ]; then message ERROR "rpm not in PATH!\n" ABORT 1 fi RPM_DBPATH=$(rpm --showrc | grep -E ":\s_dbpath\s" | cut -f2) if [ "$RPM_DBPATH" != '%{_var}/lib/rpm' ]; then message ERROR "RPM database is using a weird path: %s\n" "$RPM_DBPATH" message WARNING "You are probably running this bootstrap on Debian or Ubuntu.\n" message WARNING "There is a way to work around this problem:\n" message WARNING "Create a file at path %s/.rpmmacros.\n" "$HOME" message WARNING "Place the following lines into the '.rpmmacros' file:\n" message WARNING "%s\n" '%_var /var' message WARNING "%s\n" '%_dbpath %{_var}/lib/rpm' message WARNING "After creating the file, re-run the bootstrap.\n" message WARNING "More info: https://github.com/singularityware/singularity/issues/241\n" ABORT 1 fi if [ -z "${OSVERSION:-}" ]; then if [ -f "/etc/os-release" ]; then OSVERSION=`rpm -qf --qf '%{VERSION}' /etc/os-release` else OSVERSION=12.2 fi fi MIRROR=`echo "${MIRRORURL:-}" | sed -r "s/%\{?OSVERSION\}?/$OSVERSION/gi"` MIRROR_META=`echo "${METALINK:-}" | sed -r "s/%\{?OSVERSION\}?/$OSVERSION/gi"` if [ -z "${MIRROR:-}" ] && [ -z "${MIRROR_META:-}" ]; then message ERROR "No 'MirrorURL' or 'MetaLink' defined in bootstrap definition\n" ABORT 1 fi MIRROR_UPDATES=`echo "${UPDATEURL:-}" | sed -r "s/%\{?OSVERSION\}?/$OSVERSION/gi"` MIRROR_UPDATES_META=`echo "${UPDATEMETALINK:-}" | sed -r "s/%\{?OSVERSION\}?/$OSVERSION/gi"` if [ -n "${MIRROR_UPDATES:-}" ] || [ -n "${MIRROR_UPDATES_META:-}" ]; then message 1 "'UpdateURL' or 'UpdateMetaLink' defined in bootstrap definition\n" fi ZYPP_CONF="/etc/zypp/zypp.conf" export ZYPP_CONF # Create the main portion of zypper config mkdir -p "$SINGULARITY_ROOTFS" ZYPP_CONF_DIRNAME=`dirname $ZYPP_CONF` mkdir -m 0755 -p "$SINGULARITY_ROOTFS/$ZYPP_CONF_DIRNAME" > "$SINGULARITY_ROOTFS/$ZYPP_CONF" echo "[main]" >> "$SINGULARITY_ROOTFS/$ZYPP_CONF" echo 'cachedir=/var/cache/zypp-bootstrap' >> "$SINGULARITY_ROOTFS/$ZYPP_CONF" echo "" >> "$SINGULARITY_ROOTFS/$ZYPP_CONF" # Import zypper repos $INSTALL_CMD --root $SINGULARITY_ROOTFS ar $MIRROR repo-oss $INSTALL_CMD --root $SINGULARITY_ROOTFS --gpg-auto-import-keys refresh # Do the install! if ! eval "$INSTALL_CMD -c $SINGULARITY_ROOTFS/$ZYPP_CONF --root $SINGULARITY_ROOTFS --releasever=${OSVERSION} -n install --auto-agree-with-licenses aaa_base ${INCLUDE:-}"; then message ERROR "Bootstrap failed... exiting\n" ABORT 255 fi if ! eval "rm -rf $SINGULARITY_ROOTFS/var/cache/zypp-bootstrap"; then message WARNING "Failed cleaning Bootstrap packages\n" fi # If we got here, exit... exit 0 singularity-2.4.2/libexec/bootstrap-scripts/environment.sh0000644000175000017500000000320413211621077023043 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi message 1 "Adding base Singularity environment to container\n" umask 0002 zcat $SINGULARITY_libexecdir/singularity/bootstrap-scripts/environment.tar | ( cd $SINGULARITY_ROOTFS; tar -xf - >/dev/null) singularity-2.4.2/libexec/bootstrap-scripts/main-null.sh0000644000175000017500000000362313211621077022400 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/pre.sh" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/environment.sh" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/post.sh" # If checks specified, export variable if [ "${SINGULARITY_CHECKS:-}" = "no" ]; then message 1 "Skipping checks\n" else message 1 "Running checks\n" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/checks.sh" fi singularity-2.4.2/libexec/bootstrap-scripts/main-dockerhub.sh0000644000175000017500000000513713211621077023376 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi SINGULARITY_CONTAINER="$SINGULARITY_BUILDDEF" if ! SINGULARITY_CONTENTS=`mktemp ${TMPDIR:-/tmp}/.singularity-layers.XXXXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi export SINGULARITY_CONTAINER SINGULARITY_CONTENTS eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/pre.sh" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/environment.sh" eval_abort "$SINGULARITY_libexecdir/singularity/python/import.py" for i in `cat "$SINGULARITY_CONTENTS"`; do name=`basename "$i"` message 1 "Exploding layer: $name\n" zcat "$i" | (cd "$SINGULARITY_ROOTFS"; tar --exclude=dev/* -xf -) || exit $? done rm -f "$SINGULARITY_CONTENTS" # If checktags not defined, default to docker if [ -z "${SINGULARITY_CHECKTAGS:-}" ]; then SINGULARITY_CHECKTAGS=docker export SINGULARITY_CHECKTAGS fi eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/post.sh" # If checks specified, export variable if [ "${SINGULARITY_CHECKS:-}" = "no" ]; then message 1 "Skipping checks\n" else message 1 "Running checks\n" eval_abort "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/checks.sh" fi singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-shub.sh0000644000175000017500000000615213211621077024332 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi if [ -z "${FROM:-}" ]; then message ERROR "Required Definition tag 'From:' not defined.\n" exit 1 fi ################################################################################ # Singularity Hub/Registry Customizations ################################################################################ if [ ! -z "${REGISTRY:-}" ]; then message DEBUG "Custom Singularity Registry 'Registry:' ${REGISTRY}.\n" export REGISTRY fi if [ ! -z "${NAMESPACE:-}" ]; then message DEBUG "Custom Singularity Registry Namespace 'Namespace:' ${NAMESPACE}.\n" export NAMESPACE fi ########## BEGIN BOOTSTRAP SCRIPT ########## SINGULARITY_CONTAINER="shub://${FROM}" if ! SINGULARITY_CONTENTS=`mktemp ${TMPDIR:-/tmp}/.singularity-layerfile.XXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi # If cache is set, set pull folder to it (first priority) if [ -n "${SINGULARITY_CACHEDIR:-}" ]; then SINGULARITY_PULLFOLDER="$SINGULARITY_CACHEDIR" else # Only set the pull folder to be $PWD if not set by user if [ ! -n "${SINGULARITY_PULLFOLDER:-}" ]; then SINGULARITY_PULLFOLDER="." fi fi export SINGULARITY_CONTENTS SINGULARITY_CONTAINER SINGULARITY_PULLFOLDER umask 0002 # relying on pull.py for error checking here ${SINGULARITY_libexecdir}/singularity/python/pull.py message 1 "Exporting contents of ${SINGULARITY_CONTAINER} to ${SINGULARITY_IMAGE}\n" # switch $SINGULARITY_CONTAINER from remote to local SINGULARITY_CONTAINER=`cat $SINGULARITY_CONTENTS` rm -r $SINGULARITY_CONTENTS #if ! eval "${SINGULARITY_bindir}"/singularity image.export "${SINGULARITY_CONTAINER}" | (cd "${SINGULARITY_ROOTFS}" && tar xBf -); then if ! eval "${SINGULARITY_bindir}"/singularity image.export "${SINGULARITY_CONTAINER}" | tar xBf - -C "${SINGULARITY_ROOTFS}"; then message ERROR "Failed to export contents of ${SINGULARITY_CONTAINER} to ${SINGULARITY_ROOTFS}\n" rm $SINGULARITY_CONTAINER ABORT 255 fi rm $SINGULARITY_CONTAINER singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-localimage.sh0000644000175000017500000000324613211621077025467 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi if [ -z "${FROM:-}" ]; then message ERROR "Required Definition tag 'From:' not defined.\n" exit 1 elif [ ! -e "${FROM:-}" ]; then message ERROR "${FROM} does not exist\n" exit 1 fi ########## BEGIN BOOTSTRAP SCRIPT ########## umask 0002 message 1 "Exporting contents of ${FROM} to ${SINGULARITY_IMAGE}\n" if ! eval "${SINGULARITY_bindir}"/singularity image.export "${FROM}" | tar xBf - -C "${SINGULARITY_ROOTFS}"; then message ERROR "Failed to export contents of ${FROM} to ${SINGULARITY_ROOTFS}\n" ABORT 255 fi singularity-2.4.2/libexec/bootstrap-scripts/Makefile.am0000644000175000017500000000154413211621077022204 0ustar mehdimehdiSUBDIRS = environment modsdir = $(libexecdir)/singularity/bootstrap-scripts dist_mods_SCRIPTS = functions checks.sh \ deffile-driver-arch.sh \ deffile-driver-busybox.sh \ deffile-driver-debootstrap.sh \ deffile-driver-docker.sh \ deffile-driver-localimage.sh \ deffile-driver-self.sh \ deffile-driver-shub.sh \ deffile-driver-yum.sh \ deffile-driver-zypper.sh \ deffile-sections.sh \ deffile-post.sh \ environment.sh \ post.sh \ pre.sh \ main-deffile.sh \ main-dockerhub.sh \ main-null.sh MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/bootstrap-scripts/post.sh0000644000175000017500000000703613211621077021473 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . $SINGULARITY_libexecdir/singularity/functions else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi message 1 "Finalizing Singularity container\n" umask 0002 test -L "$SINGULARITY_ROOTFS/etc/mtab" && rm -f "$SINGULARITY_ROOTFS/etc/mtab" cat > "$SINGULARITY_ROOTFS/etc/mtab" << EOF singularity / rootfs rw 0 0 EOF # Populate the labels. # NOTE: We have to be careful to quote stuff that we know isn't quoted. SINGULARITY_LABELFILE=$(printf "%q" "$SINGULARITY_ROOTFS/.singularity.d/labels.json") SINGULARITY_ADD_SCRIPT=$(printf "%q" "$SINGULARITY_libexecdir/singularity/python/helpers/json/add.py") ########################################################################################## # # LABEL SCHEMA: http://label-schema.org/rc1/ # ########################################################################################## eval $SINGULARITY_ADD_SCRIPT -f --key "org.label-schema.schema-version" --value "1.0" --file $SINGULARITY_LABELFILE eval $SINGULARITY_ADD_SCRIPT -f --key "org.label-schema.build-date" --value $(date -R | sed 's/ /_/g') --file $SINGULARITY_LABELFILE if [ -f "${SINGULARITY_ROOTFS}/.singularity.d/runscript.help" ]; then eval $SINGULARITY_ADD_SCRIPT -f --key "org.label-schema.usage" --value "/.singularity.d/runscript.help" --file $SINGULARITY_LABELFILE fi eval $SINGULARITY_ADD_SCRIPT -f --key "org.label-schema.usage.singularity.deffile" --value $(printf "%q" "$SINGULARITY_BUILDDEF") --file $SINGULARITY_LABELFILE eval $SINGULARITY_ADD_SCRIPT -f --key "org.label-schema.usage.singularity.version" --value $(printf "%q" "$SINGULARITY_version") --file $SINGULARITY_LABELFILE # Calculate image final size message 1 "Calculating final size for metadata...\n" IMAGE_SIZE=$(du -x -sm $SINGULARITY_ROOTFS 2>/dev/null | cut -f 1) eval $SINGULARITY_ADD_SCRIPT -f --key "org.label-schema.build-size" --value "${IMAGE_SIZE}MB" --file $SINGULARITY_LABELFILE env | egrep "^SINGULARITY_DEFFILE_" | while read i; do KEY=`echo $i | cut -f1 -d =` KEY=$(replace_string "$KEY" "SINGULARITY_DEFFILE_" "") VAL=`echo $i | cut -f2- -d =` eval $SINGULARITY_ADD_SCRIPT -f --key $(printf "org.label-schema.usage.singularity.deffile.%q" "$KEY") --value $(printf "%q" "$VAL") --file $SINGULARITY_LABELFILE done singularity-2.4.2/libexec/bootstrap-scripts/deffile-post.sh0000644000175000017500000000363213211621077023065 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi if [ -z "${SINGULARITY_BUILDDEF:-}" ]; then message ERROR "Singularity bootstrap definition file not defined!\n" exit 1 fi if [ ! -f "${SINGULARITY_BUILDDEF:-}" ]; then message ERROR "Singularity bootstrap definition file not found!\n" exit 1 fi if [ ! -d "$SINGULARITY_ROOTFS/.singularity.d/bootstrap" ]; then mkdir -p "$SINGULARITY_ROOTFS/.singularity.d/bootstrap" fi cp "$SINGULARITY_BUILDDEF" "$SINGULARITY_ROOTFS/.singularity.d/bootstrap/Singularity" singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-busybox.sh0000644000175000017500000000440513211621077025063 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi ########## BEGIN BOOTSTRAP SCRIPT ########## if [ -z "${MIRRORURL:-}" ]; then MIRRORURL="https://www.busybox.net/downloads/binaries/1.26.1-defconfig-multiarch/busybox-x86_64" fi umask 0002 mkdir -p -m 0755 "$SINGULARITY_ROOTFS/bin" mkdir -p -m 0755 "$SINGULARITY_ROOTFS/etc" echo "root:!:0:0:root:/root:/bin/sh" > "$SINGULARITY_ROOTFS/etc/passwd" echo " root:x:0:" > "$SINGULARITY_ROOTFS/etc/group" echo "127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4" > "$SINGULARITY_ROOTFS/etc/hosts" curl -f "$MIRRORURL" > "$SINGULARITY_ROOTFS/bin/busybox" if [ $? -ne 0 ]; then message ERROR "Failed fetching MirrorURL: $MIRRORURL\n" exit 1 fi chmod 0755 "$SINGULARITY_ROOTFS/bin/busybox" eval "$SINGULARITY_ROOTFS/bin/busybox" --install "$SINGULARITY_ROOTFS/bin/" # If we got here, exit... exit 0 singularity-2.4.2/libexec/bootstrap-scripts/deffile-sections.sh0000644000175000017500000004415113211621077023730 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -f "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/functions" ]; then . "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/bootstrap-scripts/functions" exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined\n" exit 1 fi if [ -z "${SINGULARITY_BUILDDEF:-}" ]; then message ERROR "Singularity bootstrap definition file not defined!\n" exit 1 fi if [ ! -f "${SINGULARITY_BUILDDEF:-}" ]; then message ERROR "Singularity bootstrap definition file not found!\n" exit 1 fi umask 0002 # First priority goes to runscript defined in build file runscript_command=$(singularity_section_get "runscript" "$SINGULARITY_BUILDDEF") # If the command is not empty, write to file. if [ ! -z "$runscript_command" ]; then echo "User defined %runscript found! Taking priority." echo "$runscript_command" > "$SINGULARITY_ROOTFS/singularity" fi test -d "$SINGULARITY_ROOTFS/proc" || install -d -m 755 "$SINGULARITY_ROOTFS/proc" test -d "$SINGULARITY_ROOTFS/sys" || install -d -m 755 "$SINGULARITY_ROOTFS/sys" test -d "$SINGULARITY_ROOTFS/tmp" || install -d -m 755 "$SINGULARITY_ROOTFS/tmp" test -d "$SINGULARITY_ROOTFS/dev" || install -d -m 755 "$SINGULARITY_ROOTFS/dev" mount --no-mtab -t proc proc "$SINGULARITY_ROOTFS/proc" mount --no-mtab -t sysfs sysfs "$SINGULARITY_ROOTFS/sys" mount --no-mtab --rbind "/tmp" "$SINGULARITY_ROOTFS/tmp" mount --no-mtab --rbind "/dev" "$SINGULARITY_ROOTFS/dev" cp /etc/hosts "$SINGULARITY_ROOTFS/etc/hosts" cp /etc/resolv.conf "$SINGULARITY_ROOTFS/etc/resolv.conf" ### EXPORT ENVARS DEBIAN_FRONTEND=noninteractive SINGULARITY_ENVIRONMENT="/.singularity.d/env/91-environment.sh" export DEBIAN_FRONTEND SINGULARITY_ENVIRONMENT # Script helper paths ADD_LABEL=$SINGULARITY_libexecdir/singularity/python/helpers/json/add.py ########################################################################################## # # MAIN SECTIONS # ########################################################################################## ### SETUP if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "setup" ]; then if singularity_section_exists "setup" "$SINGULARITY_BUILDDEF"; then ARGS=`singularity_section_args "setup" "$SINGULARITY_BUILDDEF"` singularity_section_get "setup" "$SINGULARITY_BUILDDEF" | /bin/sh -e -x $ARGS || ABORT 255 fi if [ ! -x "$SINGULARITY_ROOTFS/bin/sh" -a ! -L "$SINGULARITY_ROOTFS/bin/sh" ]; then message ERROR "Could not locate /bin/sh inside the container\n" exit 255 fi else message 2 "Skipping setup section\n" fi ### FILES if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "files" ]; then if singularity_section_exists "files" "$SINGULARITY_BUILDDEF"; then message 1 "Adding files to container\n" singularity_section_get "files" "$SINGULARITY_BUILDDEF" | sed -e 's/#.*//' | while read origin dest; do if [ -n "${origin:-}" ]; then if [ -z "${dest:-}" ]; then dest="$origin" fi message 1 "Copying '$origin' to '$dest'\n" if ! /bin/cp -fLr $origin "$SINGULARITY_ROOTFS/$dest"; then message ERROR "Failed copying file(s) into container\n" exit 255 fi fi done fi else message 2 "Skipping files section\n" fi ### ENVIRONMENT if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "environment" ]; then if singularity_section_exists "environment" "$SINGULARITY_BUILDDEF"; then message 1 "Adding environment to container\n" singularity_section_get "environment" "$SINGULARITY_BUILDDEF" >> "$SINGULARITY_ROOTFS/.singularity.d/env/90-environment.sh" fi else message 2 "Skipping environment section\n" fi ### RUN POST if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "post" ]; then if singularity_section_exists "post" "$SINGULARITY_BUILDDEF"; then message 1 "Running post scriptlet\n" ARGS=`singularity_section_args "post" "$SINGULARITY_BUILDDEF"` singularity_section_get "post" "$SINGULARITY_BUILDDEF" | chroot "$SINGULARITY_ROOTFS" /bin/sh -e -x $ARGS || ABORT 255 fi else message 2 "Skipping post section\n" fi ### LABELS if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "labels" ]; then if singularity_section_exists "labels" "$SINGULARITY_BUILDDEF"; then message 1 "Adding deffile section labels to container\n" singularity_section_get "labels" "$SINGULARITY_BUILDDEF" | while read KEY VAL; do if [ -n "$KEY" -a -n "$VAL" ]; then if [ "${SINGULARITY_DEFFILE_BOOTSTRAP:-}" = "shub" -o "${SINGULARITY_DEFFILE_BOOTSTRAP:-}" = "localimage" ]; then $ADD_LABEL --key "$KEY" --value "$VAL" --file "$SINGULARITY_ROOTFS/.singularity.d/labels.json" -f else $ADD_LABEL --key "$KEY" --value "$VAL" --file "$SINGULARITY_ROOTFS/.singularity.d/labels.json" fi fi done fi else message 2 "Skipping labels section\n" fi ### RUNSCRIPT if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "startscript" ]; then if singularity_section_exists "startscript" "$SINGULARITY_BUILDDEF"; then message 1 "Adding startscript\n" echo -n "#!/bin/sh " > "$SINGULARITY_ROOTFS/.singularity.d/startscript" singularity_section_args "startscript" "$SINGULARITY_BUILDDEF" >> "$SINGULARITY_ROOTFS/.singularity.d/startscript" echo "" >> "$SINGULARITY_ROOTFS/.singularity.d/startscript" singularity_section_get "startscript" "$SINGULARITY_BUILDDEF" >> "$SINGULARITY_ROOTFS/.singularity.d/startscript" fi else message 2 "Skipping startscript section\n" fi ### RUNSCRIPT if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "runscript" ]; then if singularity_section_exists "runscript" "$SINGULARITY_BUILDDEF"; then message 1 "Adding runscript\n" echo -n "#!/bin/sh " > "$SINGULARITY_ROOTFS/.singularity.d/runscript" singularity_section_args "runscript" "$SINGULARITY_BUILDDEF" >> "$SINGULARITY_ROOTFS/.singularity.d/runscript" echo "" >> "$SINGULARITY_ROOTFS/.singularity.d/runscript" singularity_section_get "runscript" "$SINGULARITY_BUILDDEF" >> "$SINGULARITY_ROOTFS/.singularity.d/runscript" fi else message 2 "Skipping runscript section\n" fi ### HELP FOR RUNSCRIPT if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "help" ]; then if singularity_section_exists "help" "$SINGULARITY_BUILDDEF"; then message 1 "Adding runscript help\n" singularity_section_args "help" "$SINGULARITY_BUILDDEF" > "$SINGULARITY_ROOTFS/.singularity.d/runscript.help" echo "" >> "$SINGULARITY_ROOTFS/.singularity.d/runscript.help" singularity_section_get "help" "$SINGULARITY_BUILDDEF" >> "$SINGULARITY_ROOTFS/.singularity.d/runscript.help" # Add label for the file HELPLABEL="org.label-schema.usage.singularity.runscript.help" HELPFILE="/.singularity.d/runscript.help" $SINGULARITY_libexecdir/singularity/python/helpers/json/add.py --key "$HELPLABEL" --value "$HELPFILE" --file "$SINGULARITY_ROOTFS/.singularity.d/labels.json" fi else message 2 "Skipping runscript help section\n" fi ### RUN TEST if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "test" ]; then if [ -z "${SINGULARITY_NOTEST:-}" ]; then if singularity_section_exists "test" "$SINGULARITY_BUILDDEF"; then message 1 "Running test scriptlet\n" ARGS=`singularity_section_args "test" "$SINGULARITY_BUILDDEF"` echo "#!/bin/sh" > "$SINGULARITY_ROOTFS/.singularity.d/test" echo "" >> "$SINGULARITY_ROOTFS/.singularity.d/test" singularity_section_get "test" "$SINGULARITY_BUILDDEF" >> "$SINGULARITY_ROOTFS/.singularity.d/test" chmod 0755 "$SINGULARITY_ROOTFS/.singularity.d/test" chroot "$SINGULARITY_ROOTFS" /bin/sh -e -x $ARGS "/.singularity.d/test" "$@" || ABORT 255 fi fi else message 2 "Skipping test section\n" fi ########################################################################################## # # APP SECTIONS # ########################################################################################## ### APPFILES if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "appfiles" ]; then if singularity_section_exists "appfiles" "$SINGULARITY_BUILDDEF"; then APPNAMES=(`singularity_section_args "appfiles" "$SINGULARITY_BUILDDEF"`) for APPNAME in "${APPNAMES[@]}"; do message 1 "Adding files to ${APPNAME}\n" singularity_app_init "${APPNAME}" "${SINGULARITY_ROOTFS}" get_section "appfiles ${APPNAME}" "$SINGULARITY_BUILDDEF" | sed -e 's/#.*//' | while read origin dest; do if [ -n "${origin:-}" ]; then if [ -z "${dest:-}" ]; then # files must be relative to app dest="scif/apps/${APPNAME}" else dest="scif/apps/${APPNAME}/$dest" fi message 1 "+ $origin to $dest\n" if ! /bin/cp -fLr $origin "$SINGULARITY_ROOTFS/$dest"; then message ERROR "Failed copying file(s) for app ${APPNAME} into container\n" exit 255 fi fi done done fi fi ### APPHELP if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "apphelp" ]; then if singularity_section_exists "apphelp" "$SINGULARITY_BUILDDEF"; then APPNAMES=(`singularity_section_args "apphelp" "$SINGULARITY_BUILDDEF"`) for APPNAME in "${APPNAMES[@]}"; do message 1 "${APPNAME} has help section\n" singularity_app_init "${APPNAME}" "${SINGULARITY_ROOTFS}" APPHELP=$(get_section "apphelp ${APPNAME}" "$SINGULARITY_BUILDDEF") if [ ! -z "$APPHELP" ]; then echo "$APPHELP" > "$SINGULARITY_ROOTFS/scif/apps/${APPNAME}/scif/runscript.help" fi done fi fi ### APPRUNSCRIPT if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "apprun" ]; then if singularity_section_exists "apprun" "$SINGULARITY_BUILDDEF"; then APPNAMES=(`singularity_section_args "apprun" "$SINGULARITY_BUILDDEF"`) for APPNAME in "${APPNAMES[@]}"; do message 1 "${APPNAME} has runscript definition\n" singularity_app_init "${APPNAME}" "${SINGULARITY_ROOTFS}" APPRUN=$(get_section "apprun ${APPNAME}" "$SINGULARITY_BUILDDEF") if [ ! -z "$APPRUN" ]; then echo "$APPRUN" > "$SINGULARITY_ROOTFS/scif/apps/${APPNAME}/scif/runscript" chmod 0755 "$SINGULARITY_ROOTFS/scif/apps/${APPNAME}/scif/runscript" fi # Make sure we have metadata APPBASE="$SINGULARITY_ROOTFS/scif/apps/${APPNAME}" APPFOLDER_SIZE=$(singularity_calculate_size "${APPBASE}") $ADD_LABEL --key "SINGULARITY_APP_SIZE" --value "${APPFOLDER_SIZE}MB" --file "$APPBASE/scif/labels.json" $ADD_LABEL --key "SINGULARITY_APP_NAME" --value "${APPNAME}" --file "${APPBASE}/scif/labels.json" done fi fi ### APPENVIRONMENT if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "appenv" ]; then if singularity_section_exists "appenv" "$SINGULARITY_BUILDDEF"; then APPNAMES=(`singularity_section_args "appenv" "$SINGULARITY_BUILDDEF"`) for APPNAME in "${APPNAMES[@]}"; do message 1 "Adding custom environment to ${APPNAME}\n" singularity_app_init "${APPNAME}" "${SINGULARITY_ROOTFS}" get_section "appenv ${APPNAME}" "$SINGULARITY_BUILDDEF" >> "$SINGULARITY_ROOTFS/scif/apps/${APPNAME}/scif/env/90-environment.sh" done fi fi ### APPLABELS if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "appfiles" ]; then if singularity_section_exists "applabels" "$SINGULARITY_BUILDDEF"; then APPNAMES=(`singularity_section_args "applabels" "$SINGULARITY_BUILDDEF"`) for APPNAME in "${APPNAMES[@]}"; do message 1 "Adding labels to ${APPNAME}\n" singularity_app_init "${APPNAME}" "${SINGULARITY_ROOTFS}" get_section "applabels ${APPNAME}" "$SINGULARITY_BUILDDEF" | while read KEY VAL; do if [ -n "$KEY" -a -n "$VAL" ]; then $ADD_LABEL --key "$KEY" --value "$VAL" --file "$SINGULARITY_ROOTFS/scif/apps/${APPNAME}/scif/labels.json" fi done done fi fi ### APPINSTALL if [ -z "${SINGULARITY_BUILDSECTION:-}" -o "${SINGULARITY_BUILDSECTION:-}" == "appinstall" ]; then if singularity_section_exists "appinstall" "$SINGULARITY_BUILDDEF"; then APPNAMES=(`singularity_section_args "appinstall" "$SINGULARITY_BUILDDEF"`) for APPNAME in "${APPNAMES[@]}"; do message 1 "Installing ${APPNAME}\n" APPBASE="$SINGULARITY_ROOTFS/scif/apps/${APPNAME}" SINGULARITY_APPROOT="/scif/apps/${APPNAME}" export SINGULARITY_APPROOT singularity_app_init "${APPNAME}" "${SINGULARITY_ROOTFS}" singularity_app_save "${APPNAME}" "$SINGULARITY_BUILDDEF" "${APPBASE}/scif/Singularity" singularity_app_install_get "${APPNAME}" "$SINGULARITY_BUILDDEF" | chroot "$SINGULARITY_ROOTFS" /bin/sh -xe || ABORT 255 APPFOLDER_SIZE=$(singularity_calculate_size "${APPBASE}") $ADD_LABEL --key "SINGULARITY_APP_SIZE" --value "${APPFOLDER_SIZE}MB" --file "$APPBASE/scif/labels.json" --quiet -f $ADD_LABEL --key "SINGULARITY_APP_NAME" --value "${APPNAME}" --file "${APPBASE}/scif/labels.json" --quiet -f done fi else message 2 "No applications detected for install\n" fi ## APPGLOBAL APPGLOBAL="${SINGULARITY_ROOTFS}/.singularity.d/env/94-appsbase.sh" for app in ${SINGULARITY_ROOTFS}/scif/apps/*; do if [ -d "$app" ]; then app="${app##*/}" app=(`echo $app | sed -e "s/-/_/g"`) appbase="${SINGULARITY_ROOTFS}/scif/apps/$app" appmeta="${appbase}/scif" # Export data, root, metadata, labels, environment echo "APPDATA_$app=/scif/data/$app" >> "${APPGLOBAL}" echo "APPMETA_$app=/scif/apps/$app/scif" >> "${APPGLOBAL}" echo "APPROOT_$app=/scif/apps/$app" >> "${APPGLOBAL}" echo "APPBIN_$app=/scif/apps/$app/bin" >> "${APPGLOBAL}" echo "APPLIB_$app=/scif/apps/$app/lib" >> "${APPGLOBAL}" echo "export APPDATA_$app APPROOT_$app APPMETA_$app APPBIN_$app APPLIB_$app" >> "${APPGLOBAL}" # Environment if [ -e "${appmeta}/env/90-environment.sh" ]; then echo "APPENV_${app}=/scif/apps/$app/scif/env/90-environment.sh" >> "${APPGLOBAL}" echo "export APPENV_${app}" >> "${APPGLOBAL}" fi # Labels if [ -e "${appmeta}/labels.json" ]; then echo "APPLABELS_${app}=/scif/apps/$app/scif/labels.json" >> "${APPGLOBAL}" echo "export APPLABELS_${app}" >> "${APPGLOBAL}" fi # Runscript if [ -e "${appmeta}/runscript" ]; then echo "APPRUN_${app}=/scif/apps/$app/scif/runscript" >> "${APPGLOBAL}" echo "export APPRUN_${app}" >> "${APPGLOBAL}" fi fi done ########################################################################################## # # Finalizing # ########################################################################################## > "$SINGULARITY_ROOTFS/etc/hosts" > "$SINGULARITY_ROOTFS/etc/resolv.conf" # If we have a runscript, whether docker, user defined, change permissions if [ -s "$SINGULARITY_ROOTFS/.singularity.d/runscript" ]; then chmod 0755 "$SINGULARITY_ROOTFS/.singularity.d/runscript" fi # Copy the definition file into the container. If one already exists, archive. if [ -f "$SINGULARITY_ROOTFS/.singularity.d/Singularity" ]; then message 1 "Found an existing definition file\n" message 1 "Adding a bootstrap_history directory\n" mkdir -p "$SINGULARITY_ROOTFS/.singularity.d/bootstrap_history" count=0 while true; do if [ ! -f "$SINGULARITY_ROOTFS/.singularity.d/bootstrap_history/Singularity${count}" ]; then mv "$SINGULARITY_ROOTFS/.singularity.d/Singularity" "$SINGULARITY_ROOTFS/.singularity.d/bootstrap_history/Singularity${count}" break fi count=`expr $count + 1` done fi install -m 644 "$SINGULARITY_BUILDDEF" "$SINGULARITY_ROOTFS/.singularity.d/Singularity" singularity-2.4.2/libexec/bootstrap-scripts/deffile-driver-arch.sh0000644000175000017500000001025613211621077024306 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2016, Maciej Sieczka. All rights reserved # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. # Basic sanity. if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi # Load functions. if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions." exit 1 fi if [ -z "${SINGULARITY_ROOTFS:-}" ]; then message ERROR "Singularity root file system not defined.\n" ABORT 255 fi ########## BEGIN BOOTSTRAP SCRIPT ########## if ! PACSTRAP=`singularity_which pacstrap`; then message ERROR "\`pacstrap' is not in PATH. You can install it with \`pacman -S arch-install-scripts'.\n" ABORT 1 fi if ! WGET=`singularity_which wget`; then message ERROR "\`wget' is not in PATH. You can install it with \`pacman -S wget'.\n" ABORT 1 fi ARCHITECTURE="`uname -m`" if [ "$ARCHITECTURE" != 'x86_64' -a "$ARCHITECTURE" != 'i686' ]; then message ERROR "Architecture \`$ARCHITECTURE' is not supported." ABORT 1 fi PACMAN_CONF_URL="https://git.archlinux.org/svntogit/packages.git/plain/trunk/pacman.conf.${ARCHITECTURE}?h=packages/pacman" # `pacstrap' installs the whole "base" package group, unless told otherwise. # $BASE_TO_SKIP are "base" packages that won't be normally needed on a # container system. $BASE_TO_INST are "base" packages not present in # $BASE_TO_SKIP. The list of packages included in "base" group may (it surely # will, one day) change in future, so $BASE_TO_SKIP will need an update from # time to time. Here I'm referring to `base' group contents as of 30.08.2016. BASE_TO_SKIP='cryptsetup\|device-mapper\|dhcpcd\|iproute2\|jfsutils\|linux\|lvm2\|man-db\|man-pages\|mdadm\|nano\|netctl\|openresolv\|pciutils\|pcmciautils\|reiserfsprogs\|s-nail\|systemd-sysvcompat\|usbutils\|vi\|xfsprogs' BASE_TO_INST=`pacman -Sgq base | grep -xv $BASE_TO_SKIP | tr '\n' ' '` # TODO: Try choosing fastest mirror(s) with rankmirrors? # https://wiki.archlinux.org/index.php/Mirrors#List_by_speed PACMAN_CONF="/tmp/pacman.conf.$$" # TODO: Use mktemp instead? if ! eval "'$WGET' --no-verbose -O '$PACMAN_CONF' '$PACMAN_CONF_URL'"; then message ERROR "Failed to download \`$PACMAN_CONF_URL' to \`$PACMAN_CONF'.\n" ABORT 255 fi # In addition to selected `base' packages `haveged' has to be installed. It's # required to generate enough entropy for Pacman package signing setup without # having to wait for ages until entropy accumulates. See # https://wiki.archlinux.org/index.php/Install_from_Existing_Linux, # https://wiki.archlinux.org/index.php/Pacman/Package_signing. if ! eval "'$PACSTRAP' -C '$PACMAN_CONF' -c -d -G -M '$SINGULARITY_ROOTFS' haveged $BASE_TO_INST"; then rm -f "$PACMAN_CONF" message ERROR "\`$PACSTRAP' failed.\n" ABORT 255 fi rm -f "$PACMAN_CONF" # Pacman package signing setup. if ! eval "arch-chroot '$SINGULARITY_ROOTFS' /bin/sh -c 'haveged -w 1024; pacman-key --init; pacman-key --populate archlinux'"; then message ERROR "Pacman package signing setup failed.\n" ABORT 255 fi # Cleanup. if ! eval "arch-chroot '$SINGULARITY_ROOTFS' pacman -Rs --noconfirm haveged"; then message ERROR "Bootstrap packages cleanup failed.\n" ABORT 255 fi singularity-2.4.2/libexec/handlers/0000755000175000017500000000000013211621077016242 5ustar mehdimehdisingularity-2.4.2/libexec/handlers/image-docker.sh0000644000175000017500000000402013211621077021121 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. NAME=`echo "$SINGULARITY_IMAGE" | sed -e 's@^docker://@@'` if [ -z "${SINGULARITY_LOCALCACHEDIR:-}" ]; then SINGULARITY_LOCALCACHEDIR="${TMPDIR:-/tmp}" fi if ! SINGULARITY_TMPDIR=`mktemp -d $SINGULARITY_LOCALCACHEDIR/.singularity-runtime.XXXXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi SINGULARITY_ROOTFS="$SINGULARITY_TMPDIR/$NAME" if ! mkdir -p "$SINGULARITY_ROOTFS"; then message ERROR "Failed to create named SINGULARITY_ROOTFS=$SINGULARITY_ROOTFS\n" ABORT 255 fi SINGULARITY_CONTAINER="$SINGULARITY_IMAGE" SINGULARITY_IMAGE="$SINGULARITY_ROOTFS" SINGULARITY_CLEANUPDIR="$SINGULARITY_TMPDIR" if ! SINGULARITY_CONTENTS=`mktemp ${TMPDIR:-/tmp}/.singularity-layers.XXXXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi export SINGULARITY_ROOTFS SINGULARITY_IMAGE SINGULARITY_CONTAINER SINGULARITY_CONTENTS SINGULARITY_CLEANUPDIR eval_abort "$SINGULARITY_libexecdir/singularity/python/import.py" message 1 "Creating container runtime...\n" message 2 "Importing: base Singularity environment\n" zcat $SINGULARITY_libexecdir/singularity/bootstrap-scripts/environment.tar | (cd $SINGULARITY_ROOTFS; tar -xf -) || exit $? for i in `cat "$SINGULARITY_CONTENTS"`; do name=`basename "$i"` message 2 "Exploding layer: $name\n" # Settings of file privileges must be buffered files=$( zcat "$i" | (cd "$SINGULARITY_ROOTFS"; tar --overwrite --exclude=dev/* -xvf -)) || exit $? for file in $files; do if [ -L "$SINGULARITY_ROOTFS/$file" ]; then # Skipping symlinks true elif [ -f "$SINGULARITY_ROOTFS/$file" ]; then chmod u+rw "$SINGULARITY_ROOTFS/$file" >/dev/null 2>&1 elif [ -d "$SINGULARITY_ROOTFS/$file" ]; then chmod u+rwx "$SINGULARITY_ROOTFS/${file%/}" >/dev/null 2>&1 fi done done rm -f "$SINGULARITY_CONTENTS" singularity-2.4.2/libexec/handlers/image-shub.sh0000644000175000017500000000170313211621077020620 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. if ! SINGULARITY_CONTENTS=`mktemp ${TMPDIR:-/tmp}/.singularity-layerfile.XXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi if [ -n "${SINGULARITY_CACHEDIR:-}" ]; then SINGULARITY_PULLFOLDER="$SINGULARITY_CACHEDIR" else SINGULARITY_PULLFOLDER="." fi SINGULARITY_CONTAINER="$SINGULARITY_IMAGE" export SINGULARITY_PULLFOLDER SINGULARITY_CONTAINER SINGULARITY_CONTENTS if ! eval "$SINGULARITY_libexecdir/singularity/python/pull.py"; then ABORT 255 fi # The python script saves names to files in CONTAINER_DIR SINGULARITY_IMAGE=`cat $SINGULARITY_CONTENTS` export SINGULARITY_IMAGE rm -f "$SINGULARITY_CONTENTS" if [ -f "$SINGULARITY_IMAGE" ]; then chmod +x "$SINGULARITY_IMAGE" else message ERROR "Could not locate downloaded image\n" ABORT 255 fi singularity-2.4.2/libexec/handlers/image-instance.sh0000644000175000017500000000142613211621077021465 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. NAME=`echo "$SINGULARITY_IMAGE" | sed -e 's@^instance://@@'` singularity_daemon_file "${NAME}" if [ ! -f "${SINGULARITY_DAEMON_FILE}" ]; then message ERROR "A daemon process is not running with this name: ${NAME}\n" ABORT 255 fi . "${SINGULARITY_DAEMON_FILE}" if [ -z "${DAEMON_IMAGE}" ]; then message ERROR "Image for daemon is not defined\n" ABORT 255 fi if [ ! -f "${DAEMON_IMAGE}" -a ! -d "${DAEMON_IMAGE}" ]; then message ERROR "Image for daemon is not found: ${DAEMON_IMAGE}\n" ABORT 255 fi SINGULARITY_IMAGE="${DAEMON_IMAGE}" SINGULARITY_DAEMON_JOIN=1 export SINGULARITY_DAEMON_JOIN SINGULARITY_IMAGE singularity-2.4.2/libexec/handlers/archive-cpio.sh0000644000175000017500000000242113211621077021146 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. NAME=`basename "$SINGULARITY_IMAGE"` if [ -z "${SINGULARITY_CACHEDIR:-}" ]; then SINGULARITY_CACHEDIR="${TMPDIR:-/tmp}" fi if [ ! -d "$SINGULARITY_CACHEDIR" ]; then message ERROR "Cache directory does not exist: $SINGULARITY_CACHEDIR\n" ABORT 1 fi if ! SINGULARITY_TMPDIR=`mktemp -d $SINGULARITY_CACHEDIR/singularity-rundir.XXXXXXXX`; then message ERROR "Failed to create tmpdir\n" ABORT 255 fi CONTAINER_DIR="$SINGULARITY_TMPDIR/$NAME" if ! mkdir -p "$CONTAINER_DIR"; then message ERROR "Could not create cache directory: $CONTAINER_DIR\n" ABORT 255 fi message 1 "Opening cpio archive, stand by...\n" case "$SINGULARITY_IMAGE" in *.cpioz|*.vnfs) # this almost always gives permission errors, so ignore them when # running as a user. zcat "$SINGULARITY_IMAGE" | ( cd "$CONTAINER_DIR"; cpio -id >/dev/null 2>&1 ) ;; *.cpio) cat "$SINGULARITY_IMAGE" | ( cd "$CONTAINER_DIR"; cpio -id >/dev/null 2>&1 ) ;; esac chmod -R +w "$CONTAINER_DIR" SINGULARITY_IMAGE="$CONTAINER_DIR" SINGULARITY_CLEANUPDIR="$SINGULARITY_TMPDIR" export SINGULARITY_CLEANUPDIR SINGULARITY_IMAGE singularity-2.4.2/libexec/handlers/image-http.sh0000644000175000017500000000101313211621077020630 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. NAME=`basename "$SINGULARITY_IMAGE"` if [ -f "$NAME" ]; then message 2 "Using cached container in current working directory: $NAME\n" SINGULARITY_IMAGE="$NAME" else message 1 "Caching container to current working directory: $NAME\n" if curl -L -k "$SINGULARITY_IMAGE" > "$NAME"; then SINGULARITY_IMAGE="$NAME" else ABORT 255 fi fi singularity-2.4.2/libexec/handlers/Makefile.am0000644000175000017500000000032513211621077020276 0ustar mehdimehdiscriptlibexecdir = $(libexecdir)/singularity/handlers dist_scriptlibexec_SCRIPTS = archive-tar.sh archive-cpio.sh image-docker.sh image-shub.sh image-http.sh image-instance.sh MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/handlers/archive-tar.sh0000644000175000017500000000326413211621077021010 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. NAME=`basename "$SINGULARITY_IMAGE"` if [ -z "${SINGULARITY_CACHEDIR:-}" ]; then SINGULARITY_CACHEDIR="${TMPDIR:-/tmp}" fi if [ ! -d "$SINGULARITY_CACHEDIR" ]; then message ERROR "Cache directory does not exist: $SINGULARITY_CACHEDIR\n" ABORT 1 fi if ! SINGULARITY_TMPDIR=`mktemp -d $SINGULARITY_CACHEDIR/singularity-rundir.XXXXXXXX`; then message ERROR "Failed to create tmpdir\n" ABORT 255 fi CONTAINER_DIR="$SINGULARITY_TMPDIR/$NAME" if ! mkdir -p "$CONTAINER_DIR"; then message ERROR "Could not create cache directory: $CONTAINER_DIR\n" ABORT 255 fi case "$SINGULARITY_IMAGE" in *.tar) message 1 "Opening tar archive, stand by...\n" # this almost always gives permission errors, so ignore them when # running as a user. tar -C "$CONTAINER_DIR" -xf "$SINGULARITY_IMAGE" 2>/dev/null ;; *.tgz|*.tar.gz) message 1 "Opening gzip compressed archive, stand by...\n" # this almost always gives permission errors, so ignore them when # running as a user. tar -C "$CONTAINER_DIR" -xzf "$SINGULARITY_IMAGE" 2>/dev/null ;; *.tbz|*.tar.bz) message 1 "Opening bzip compressed archive, stand by...\n" # this almost always gives permission errors, so ignore them when # running as a user. tar -C "$CONTAINER_DIR" -xjf "$SINGULARITY_IMAGE" 2>/dev/null ;; esac chmod -R +w "$CONTAINER_DIR" SINGULARITY_IMAGE="$CONTAINER_DIR" SINGULARITY_CLEANUPDIR="$SINGULARITY_TMPDIR" export SINGULARITY_CLEANUPDIR SINGULARITY_IMAGE singularity-2.4.2/libexec/cli/0000755000175000017500000000000013211621077015211 5ustar mehdimehdisingularity-2.4.2/libexec/cli/selftest.info0000644000175000017500000000016513211621077017721 0ustar mehdimehdiNAME="selftest" SECTION="general" SUMMARY="Run some self tests for singularity install" USAGE="singularity selftest" singularity-2.4.2/libexec/cli/instance.list.exec0000644000175000017500000000506313211621077020641 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # This software is licensed under a 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi ## Init Singularity environment if [ -f "$SINGULARITY_sysconfdir/singularity/init" ]; then . "$SINGULARITY_sysconfdir/singularity/init" fi if ! USERID=`id -ru`; then message ERROR "Could not ascertain user ID\n" exit 255 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -u|--user) if [ "$USERID" = 0 ]; then shift USERID=${1:-} shift else message ERROR "Must be root to list with -u/--user option\n" exit 1 fi ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done COUNT=0 if [ -z "${1:-}" ]; then DAEMONS=`singularity_daemon_glob '*' | sort | uniq` else DAEMONS=`singularity_daemon_glob "$@" | sort | uniq` fi message 1 "%-16s %-8s %s\n" "DAEMON NAME" "PID" "CONTAINER IMAGE" for i in $DAEMONS; do if [ -f "$i" ]; then if [ ! -s "$i" ]; then rm "$i" continue fi FILE_NAME=`basename "$i"` DAEMON_NAME=${FILE_NAME##*-} . "$i" if [ -n "${DAEMON_PID:-}" ]; then PROC_CMDLINE="/proc/${DAEMON_PID:-}/cmdline" if [ -e "$PROC_CMDLINE" ]; then PROGNAME=$(<"${PROC_CMDLINE}" tr -d \\0) if [ "${PROGNAME}" != "singularity-instance: $USER [$DAEMON_NAME]" ]; then message WARN "Removing stale daemon file: $i\n" rm -f "$i" continue fi else rm -f "$i" continue fi fi printf "%-16s %-8s %s\n" "${FILE_NAME}" "${DAEMON_PID}" "${DAEMON_IMAGE:-}" let "COUNT++" fi done if [ "$COUNT" == 0 ]; then exit 1 fi singularity-2.4.2/libexec/cli/instance.stop.info0000644000175000017500000000232613211621077020661 0ustar mehdimehdiNAME="instance.stop" GROUP="instance" SUMMARY="Stop a running container instance" USAGE="singularity [...] instance.stop [...] " print_help() { cat< If running as root, stop the specified instance of -a|--all Stop all user's running instances -f|--force Force kill instance -s|--signal Signal sent to the instance EXAMPLES: $ singularity instance.start my-sql.img mysql1 $ singularity instance.start my-sql.img mysql2 $ singularity instance.stop mysql* Stopping mysql1 instance of my-sql.img (PID=23845) Stopping mysql2 instance of my-sql.img (PID=23858) $ singularity instance.start my-sql.img mysql1 Force instance to shutdown $ singularity instance.stop -f mysql1 (may corrupt data) Send SIGTERM to the instance $ singularity instance.stop -s SIGTERM mysql1 $ singularity instance.stop -s TERM mysql1 $ singularity instance.stop -s 15 mysql1 For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/cli/image.export.exec0000644000175000017500000000575213211621077020472 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -f|--file) shift SINGULARITY_EXPORT_FILE="${1:-}" export SINGULARITY_EXPORT_FILE shift ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi SINGULARITY_IMAGE="${1:-}" export SINGULARITY_IMAGE shift if [ -n "${1:-}" ]; then SINGULARITY_EXPORT_FILE="${1:-}" export SINGULARITY_EXPORT_FILE shift fi if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/mount-suid" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/mount-suid" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/image.sh" elif [ -x "$SINGULARITY_libexecdir/singularity/bin/mount" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/mount" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/image.sh" else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/mount\n" exit 1 fi message ERROR "We should never have gotten here..." exit 1 singularity-2.4.2/libexec/cli/pull.exec0000644000175000017500000001276513211621077017046 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -n|--name|name) shift SHUB_CONTAINERNAME="${1:-}" export SHUB_CONTAINERNAME shift ;; -C|--commit|commit) shift if [ -z "${SINGULARITY_IMAGE+x}" ]; then SHUB_NAMEBYCOMMIT="yes" unset SHUB_NAMEBYHASH export SHUB_NAMEBYCOMMIT fi ;; -H|--hash|hash) shift if [ -z "${SINGULARITY_IMAGE+x}" ]; then if [ -z "${SHUB_NAMEBYCOMMIT+x}" ]; then SHUB_NAMEBYHASH="yes" export SHUB_NAMEBYHASH fi fi ;; -s|--size) shift SINGULARITY_IMAGESIZE="${1:-}" export SINGULARITY_IMAGESIZE shift ;; -F|--force) shift OVERWRITE=1 ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi RETVAL=0 SINGULARITY_CONTAINER="${1:-}" if ! SINGULARITY_CONTENTS=`mktemp ${TMPDIR:-/tmp}/.singularity-layerfile.XXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi # If cache is set, set pull folder to it (first priority) if [ -n "${SINGULARITY_CACHEDIR:-}" ]; then SINGULARITY_PULLFOLDER="$SINGULARITY_CACHEDIR" else # Only set the pull folder to be $PWD if not set by user if [ ! -n "${SINGULARITY_PULLFOLDER:-}" ]; then SINGULARITY_PULLFOLDER="." fi fi shift export SINGULARITY_CONTAINER SINGULARITY_PULLFOLDER SINGULARITY_CONTENTS case "$SINGULARITY_CONTAINER" in docker://*) if [ -z "${SINGULARITY_IMAGE:-}" ]; then # If name given, use that if [ -n "${SHUB_CONTAINERNAME:-}" ]; then SINGULARITY_IMAGE=`basename "$SHUB_CONTAINERNAME"` else SINGULARITY_IMAGE=`basename "$SINGULARITY_CONTAINER.simg" | sed -e 's/:/-/g'` fi SINGULARITY_IMAGE="$SINGULARITY_PULLFOLDER/$SINGULARITY_IMAGE" message 3 "Singularity Image: $SINGULARITY_IMAGE" export SINGULARITY_IMAGE fi message WARNING "pull for Docker Hub is not guaranteed to produce the\n" message WARNING "same image on repeated pull. Use Singularity Registry\n" message WARNING "(shub://) to pull exactly equivalent images.\n" if [ -n "${SINGULARITY_IMAGESIZE:-}" ]; then message WARNING "Using the --size option when building from Docker Hub is deprecated.\n" message WARNING "Container will be sized appropriately for Docker image.\n" fi if [ -f "$SINGULARITY_IMAGE" ]; then if [ -n "${OVERWRITE:-}" ]; then message 2 "Removing existing file\n" rm -f "$SINGULARITY_IMAGE" else message ERROR "Image file exists, not overwriting.\n" exit 1 fi fi if [ -x "${SINGULARITY_bindir}/singularity" ]; then ${SINGULARITY_bindir}/singularity build ${SINGULARITY_IMAGE} ${SINGULARITY_CONTAINER} RETVAL=$? else message ERROR "Could not locate the Singularity binary: $SINGULARITY_home/singularity\n" exit 1 fi ;; shub://*) eval_abort $SINGULARITY_libexecdir/singularity/python/pull.py RETVAL=$? SINGULARITY_IMAGE=`cat $SINGULARITY_CONTENTS` ;; *) message ERROR "pull is only supported for shub URIs\n" exit 255 ;; esac rm -f "$SINGULARITY_CONTENTS" if [ $RETVAL -eq 0 -a -f "$SINGULARITY_IMAGE" ]; then chmod +x "$SINGULARITY_IMAGE" echo "Done. Container is at: $SINGULARITY_IMAGE" else message ERROR "pulling container failed!\n" fi exit $RETVAL singularity-2.4.2/libexec/cli/selftest.exec0000644000175000017500000000373613211621077017721 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi if [ -z "$SINGULARITY_bindir" ]; then echo "Could not identify the Singularity bindir" exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; *) break ;; esac done SINGULARITY_MESSAGELEVEL=5 SUID_FILES=" $SINGULARITY_libexecdir/singularity/bin/action-suid $SINGULARITY_libexecdir/singularity/bin/mount-suid $SINGULARITY_libexecdir/singularity/bin/start-suid " stest 0 sh -c "test -f $SINGULARITY_sysconfdir/singularity/singularity.conf" for i in $SUID_FILES; do stest 0 test -u "$i" done singularity-2.4.2/libexec/cli/inspect.info0000644000175000017500000000244613211621077017541 0ustar mehdimehdiNAME="inspect" SECTION="management" SUMMARY="Display container's metadata" USAGE="singularity [...] inspect [exec options...] " print_help() { cat< A user-bind path specification. spec has the format src[:dest[:opts]], where src and dest are outside and inside paths. If dest is not given, it is set equal to src. Mount options ('opts') may be specified as 'ro' (read-only) or 'rw' (read/write, which is the default). This option can be called multiple times. -c|--contain Use minimal /dev and empty other directories (e.g. /tmp and $HOME) instead of sharing filesystems on your host -H|--home A home directory specification. spec can either be a src path or src:dest pair. src is the source path of the home directory outside the container and dest overrides the home directory within the container -n|--net Run container in a new network namespace (loopback is only network device active) --nv Enable experimental Nvidia support -o|--overlay Use a persistent overlayFS via a writable image -S|--scratch Include a scratch directory within the container that is linked to a temporary dir (use -W to force location) -W|--workdir Working directory to be used for /tmp, /var/tmp and $HOME (if -c/--contain was also used) -w|--writable By default all Singularity containers are available as read only. This option makes the file system accessible as read/write. CONTAINER FORMATS SUPPORTED: *.img This is the native Singularity image format for all Singularity versions 2.x. *.sqsh SquashFS format, note the suffix is required! *.tar* Tar archives are exploded to a temporary directory and run within that directory (and cleaned up after). The contents of the archive is a root file system with root being in the current directory. Compression suffixes as '.gz' and '.bz2' are supported. directory/ Container directories that contain a valid root file system. EXAMPLES: $ singularity instance.start /tmp/my-sql.img mysql $ singularity shell instance://mysql Singularity my-sql.img> pwd /home/mibauer/mysql Singularity my-sql.img> ps PID TTY TIME CMD 1 pts/0 00:00:00 sinit 2 pts/0 00:00:00 bash 3 pts/0 00:00:00 ps Singularity my-sql.img> $ singularity instance.stop /tmp/my-sql.img mysql Stopping /tmp/my-sql.img mysql For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/cli/image.export.info0000644000175000017500000000124013211621077020465 0ustar mehdimehdiNAME="image.export" GROUP="image" SUMMARY="Export an image to a tar pipe" USAGE="singularity [...] image.export [...] " print_help() { cat< /tmp/Debian.tar $ singularity image.export /tmp/Debian.img | gzip -9 > /tmp/Debian.tar.gz $ singularity image.export -f Debian.tar /tmp/Debian.img For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/cli/create.exec0000644000175000017500000000415113211621077017323 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi PATH=/sbin:/usr/sbin:${PATH:-} while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; *) break; ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi # relying on image.create for error checking message WARNING "The create command is deprecated, and will be removed\n" message WARNING "To create, use the image.create command.\n" message WARNING "Use the build command to create and build an image in a single step.\n" exec "$SINGULARITY_bindir"/singularity image.create "$@" message ERROR "We should never have gotten here... ARRRGGGG!!!!\n" exit 255 singularity-2.4.2/libexec/cli/image.import.exec0000644000175000017500000000604113211621077020453 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -f|--file) shift SINGULARITY_IMPORT_FILE="${1:-}" export SINGULARITY_IMPORT_FILE shift ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi SINGULARITY_IMAGE="${1:-}" SINGULARITY_WRITABLE=1 shift export SINGULARITY_IMAGE SINGULARITY_WRITABLE if [ -n "${1:-}" ]; then SINGULARITY_EXPORT_FILE="${1:-}" export SINGULARITY_IMPORT_FILE shift fi if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/mount-suid" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/mount-suid" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/image.sh" "$@" elif [ -x "$SINGULARITY_libexecdir/singularity/bin/mount" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/mount" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/image.sh" "$@" else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/mount\n" exit 1 fi message ERROR "We should never have gotten here..." exit 1 singularity-2.4.2/libexec/cli/image.expand.info0000644000175000017500000000102413211621077020423 0ustar mehdimehdiNAME="image.expand" GROUP="image" SUMMARY="Expand a disk image" USAGE="singularity [...] image.expand [expand options...] " print_help() { cat< A user-bind path specification. spec has the format src[:dest[:opts]], where src and dest are outside and inside paths. If dest is not given, it is set equal to src. Mount options ('opts') may be specified as 'ro' (read-only) or 'rw' (read/write, which is the default). This option can be called multiple times. -c|--contain Use minimal /dev and empty other directories (e.g. /tmp and $HOME) instead of sharing filesystems on your host -C|--containall Contain not only file systems, but also PID and IPC -e|--cleanenv Clean environment before running container -H|--home A home directory specification. spec can either be a src path or src:dest pair. src is the source path of the home directory outside the container and dest overrides the home directory within the container -i|--ipc Run container in a new IPC namespace -n|--net Run container in a new network namespace (loopback is only network device active) --nv Enable experimental Nvidia support -o|--overlay Use a persistent overlayFS via a writable image -p|--pid Run container in a new PID namespace --pwd Initial working directory for payload process inside the container -S|--scratch Include a scratch directory within the container that is linked to a temporary dir (use -W to force location) -u|--userns Run container in a new user namespace (this allows Singularity to run completely unprivileged on recent kernels and doesn't support all features) -W|--workdir Working directory to be used for /tmp, /var/tmp and $HOME (if -c/--contain was also used) -w|--writable By default all Singularity containers are available as read only. This option makes the file system accessible as read/write. CONTAINER FORMATS SUPPORTED: *.sqsh SquashFS format. Native to Singularity 2.4+ *.img This is the native Singularity image format for all Singularity versions < 2.4. *.tar* Tar archives are exploded to a temporary directory and run within that directory (and cleaned up after). The contents of the archive is a root file system with root being in the current directory. Compression suffixes as '.gz' and '.bz2' are supported. directory/ Container directories that contain a valid root file system. instance://* A local running instance of a container. (See the instance command group.) shub://* A container hosted on Singularity Hub docker://* A container hosted on Docker Hub EXAMPLES: # Here we see that the runscript prints "Hello world: $@" $ singularity exec /tmp/Debian.img cat /singularity #!/bin/sh echo "Hello world: $@" # It runs with our inputs when we run the image $ singularity run /tmp/Debian.img one two three Hello world: one two three # Note that this does the same thing $ ./tmp/Debian.img one two three For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/cli/shell.exec0000644000175000017500000000561313211621077017173 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -f "${HOME:-}/.singularity-init" ]; then . "${HOME:-}/.singularity-init" fi if [ -f "$SINGULARITY_libexecdir/singularity/cli/action_argparser.sh" ]; then . "$SINGULARITY_libexecdir/singularity/cli/action_argparser.sh" else message ERROR "Could not find the action argument parser\n" exit 1 fi if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${SINGULARITY_IMAGE:-}" ]; then SINGULARITY_IMAGE="${1:-}" export SINGULARITY_IMAGE shift fi if [ -z "${SINGULARITY_IMAGE:-}" ]; then exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" fi if [ -n "${SINGULARITY_DAEMON_JOIN:-}" ]; then singularity_daemon_file "${SINGULARITY_IMAGE}" fi if [ -f "$SINGULARITY_sysconfdir/singularity/init" ]; then . "$SINGULARITY_sysconfdir/singularity/init" fi if [ -x "$SINGULARITY_libexecdir/singularity/image-handler.sh" ]; then . "$SINGULARITY_libexecdir/singularity/image-handler.sh" fi if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/action-suid" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/action-suid" "$@" <&0 elif [ -x "$SINGULARITY_libexecdir/singularity/bin/action" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/action" "$@" <&0 else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/action\n" exit 1 fi singularity-2.4.2/libexec/cli/check.exec0000644000175000017500000000673313211621077017145 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi # If no arguments # default SINGULARITY_CHECKLEVEL is highest (3) # define check tags to be default SINGULARITY_CHECKLEVEL=3 # LOW SINGULARITY_CHECKTAGS="default" while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -l|--low) # levels 3,2,1 SINGULARITY_CHECKLEVEL=3 # LOW shift; ;; -m|--med|--medium) # levels 2,1 SINGULARITY_CHECKLEVEL=2 # MED shift; ;; --high) SINGULARITY_CHECKLEVEL=1 # level 1 only shift; # HIGH ;; -t|--tag) shift; SINGULARITY_CHECKTAGS="${1:-}" echo $SINGULARITY_CHECKTAGS shift; ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi RETVAL=0 SINGULARITY_IMAGE="${1:-}" # We will inspect the image by mounting it, and this should also work for # bootstrap since we will have the SINGULARITY_ROOTFS SINGULARITY_MOUNTPOINT=`mktemp -d /tmp/.singularity-check.XXXXXXXX` export SINGULARITY_IMAGE SINGULARITY_MOUNTPOINT SINGULARITY_CHECKLEVEL SINGULARITY_CHECKTAGS shift if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/mount-suid" ]; then eval "$SINGULARITY_libexecdir/singularity/bin/mount-suid" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/check.sh" RETVAL=$? elif [ -x "$SINGULARITY_libexecdir/singularity/bin/mount" ]; then eval "$SINGULARITY_libexecdir/singularity/bin/mount" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/check.sh" RETVAL=$? else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/mount\n" exit 1 fi rmdir "$SINGULARITY_MOUNTPOINT" exit $RETVAL singularity-2.4.2/libexec/cli/test.info0000644000175000017500000000155113211621077017047 0ustar mehdimehdiNAME="test" SECTION="action" SUMMARY="Launch a testscript within container" USAGE="singularity [...] test [exec options...] " print_help() { cat< Debian.tar.gz | singularity image.import /tmp/Debian.img $ tar -cvf - -C data_dir/ . | singularity image.import /tmp/data.img For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/cli/apps.exec0000644000175000017500000000513613211621077017027 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; *) break ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi RETVAL=0 SINGULARITY_IMAGE="${1:-}" if ! SINGULARITY_MOUNTPOINT=`mktemp -d ${TMPDIR:-/tmp}/.singularity-inspect.XXXXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi export SINGULARITY_IMAGE SINGULARITY_MOUNTPOINT shift if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/mount-suid" ]; then eval "$SINGULARITY_libexecdir/singularity/bin/mount-suid" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/apps/list.sh" RETVAL=$? elif [ -x "$SINGULARITY_libexecdir/singularity/bin/mount" ]; then eval "$SINGULARITY_libexecdir/singularity/bin/mount" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/apps/list.sh" RETVAL=$? else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/mount\n" exit 1 fi rmdir "$SINGULARITY_MOUNTPOINT" exit $RETVAL singularity-2.4.2/libexec/cli/help.info0000644000175000017500000000017013211621077017014 0ustar mehdimehdiNAME="help" SECTION="general" SUMMARY="Show additional help for a command or container" USAGE="singularity help [apps]" singularity-2.4.2/libexec/cli/create.info0000644000175000017500000000130113211621077017324 0ustar mehdimehdiUSAGE="singularity [...] image.create [create options...] " print_help() { cat< If running as root, list instances from EXAMPLES: $ singularity instance.list /home/mibauer/singularity/sinstance/test.img [2056.132716] Instances: Instance PID: 11963 Instance Name: test $ sudo singularity instance.list -u mibauer /home/mibauer/singularity/sinstance/test.img [2056.132716] Instances: Instance PID: 16219 Instance Name: test Instance PID: 11963 Instance Name: test For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/cli/action_argparser.sh0000644000175000017500000001244213211621077021073 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. message 2 "Evaluating args: '$*'\n" while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -o|--overlay) shift SINGULARITY_OVERLAYIMAGE="${1:-}" export SINGULARITY_OVERLAYIMAGE shift if [ ! -e "${SINGULARITY_OVERLAYIMAGE:-}" ]; then message ERROR "Overlay image must be a file or directory!\n" ABORT 255 fi ;; -s|--shell) shift SINGULARITY_SHELL="${1:-}" export SINGULARITY_SHELL shift ;; -u|--user|--userns) SINGULARITY_NOSUID=1 export SINGULARITY_NOSUID shift ;; -w|--writable) shift SINGULARITY_WRITABLE=1 export SINGULARITY_WRITABLE ;; -H|--home) shift SINGULARITY_HOME="$1" export SINGULARITY_HOME shift ;; -W|--wdir|--workdir|--workingdir) shift SINGULARITY_WORKDIR="$1" export SINGULARITY_WORKDIR shift ;; -S|--scratchdir|--scratch-dir|--scratch) shift SINGULARITY_SCRATCHDIR="$1,${SINGULARITY_SCRATCHDIR:-}" export SINGULARITY_SCRATCHDIR shift ;; app|--app|-a) shift SINGULARITY_APPNAME="${1:-}" export SINGULARITY_APPNAME shift ;; -B|--bind) shift SINGULARITY_BINDPATH="${SINGULARITY_BINDPATH:-},${1:-}" export SINGULARITY_BINDPATH shift ;; -c|--contain) shift SINGULARITY_CONTAIN=1 export SINGULARITY_CONTAIN ;; -C|--containall|--CONTAIN) shift SINGULARITY_CONTAIN=1 SINGULARITY_UNSHARE_PID=1 SINGULARITY_UNSHARE_IPC=1 SINGULARITY_CLEANENV=1 export SINGULARITY_CONTAIN SINGULARITY_UNSHARE_PID SINGULARITY_UNSHARE_IPC SINGULARITY_CLEANENV ;; -e|--cleanenv) shift SINGULARITY_CLEANENV=1 export SINGULARITY_CLEANENV ;; -p|--pid) shift SINGULARITY_UNSHARE_PID=1 export SINGULARITY_UNSHARE_PID ;; -i|--ipc) shift SINGULARITY_UNSHARE_IPC=1 export SINGULARITY_UNSHARE_IPC ;; -n|--net) shift SINGULARITY_UNSHARE_NET=1 export SINGULARITY_UNSHARE_NET ;; --pwd) shift SINGULARITY_TARGET_PWD="$1" export SINGULARITY_TARGET_PWD shift ;; --nv) shift SINGULARITY_NVLIBLIST=`mktemp ${TMPDIR:-/tmp}/.singularity-nvliblist.XXXXXXXX` cat $SINGULARITY_sysconfdir"/singularity/nvliblist.conf" | grep -Ev "^#|^\s*$" > $SINGULARITY_NVLIBLIST for i in $(ldconfig -p | grep -f "${SINGULARITY_NVLIBLIST}"); do if [ -f "$i" ]; then message 2 "Found NV library: $i\n" if [ -z "${SINGULARITY_CONTAINLIBS:-}" ]; then SINGULARITY_CONTAINLIBS="$i" else SINGULARITY_CONTAINLIBS="$SINGULARITY_CONTAINLIBS,$i" fi fi done rm $SINGULARITY_NVLIBLIST if [ -z "${SINGULARITY_CONTAINLIBS:-}" ]; then message WARN "Could not find any Nvidia libraries on this host!\n"; else export SINGULARITY_CONTAINLIBS fi if NVIDIA_SMI=$(which nvidia-smi); then if [ -n "${SINGULARITY_BINDPATH:-}" ]; then SINGULARITY_BINDPATH="${SINGULARITY_BINDPATH},${NVIDIA_SMI}" else SINGULARITY_BINDPATH="${NVIDIA_SMI}" fi export SINGULARITY_BINDPATH else message WARN "Could not find the Nvidia SMI binary to bind into container\n" fi ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break; ;; esac done singularity-2.4.2/libexec/cli/legacy/0000755000175000017500000000000013211621077016455 5ustar mehdimehdisingularity-2.4.2/libexec/cli/legacy/bootstrap.help0000644000175000017500000000615513211621077021353 0ustar mehdimehdiUSAGE: singularity [...] bootstrap The bootstrap command is useful for creating a new bootstrap or modifying an existing one using a definition file that describes how to build the container. The quickest, easiest Bootstrap is dumping Docker layers into a container: BootStrap: docker From: library/ubuntu:14.04.1 Optionally, you can perform container checks with --checks. For more documentation on definition file recipes, look below. note: This command must be executed as root. CREATE OPTIONS: -T|--notest Bootstrap without running tests in %test section -f|--force Force a rebootstrap of an OS (note: this does not delete what is currently in the image, just causes the core to be reinstalled) -s|--section Only run a given section within the bootstrap (setup, post, files, environment, test, labels, none) --nobase Skip the generation of the base OS CHECKS OPTIONS: -c|--checks enable checks -t|--tag specify a check tag (not default) -l/--low Specify low threshold (all checks, default) -m/--med Perform medium and high checks -h/--high Perform only checks at level high See singularity check --help for available tags DEFFILE BASEOS EXAMPLES: Docker: Bootstrap: docker From: tensorflow/tensorflow:latest IncludeCmd: yes # Use the CMD as runscript instead of ENTRYPOINT YUM/RHEL: Bootstrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/ Include: yum #This is important to include inside the base container Debian/Ubuntu: Bootstrap: debootstrap OSVersion: trusty MirrorURL: http://us.archive.ubuntu.com/ubuntu/ Local Image: Bootstrap: localimage From: /home/dave/starter.img DEFFILE SECTION EXAMPLES: %setup echo "This is a scriptlet that will be executed on the host, as root, after" echo "the container has been bootstrapped. To install things into the container" echo "reference the file system location with \$SINGULARITY_BUILDROOT" %post echo "This scriptlet section will be executed from within the container after" echo "the bootstrap/base has been created and setup" %test echo "Define any test commands that should be executed after container has been" echo "built. This scriptlet will be executed from within the running container" echo "as the root user. Pay attention to the exit/return value of this scriptlet" echo "as any non-zero exit code will be assumed as failure" exit 0 %labels HELLO MOTO KEY VALUE %files /path/on/host/file.txt /path/on/container/file.txt relative_file.txt /path/on/container/relative_file.txt EXAMPLES: $ singularity create /tmp/Debian.img $ sudo singularity bootstrap /tmp/Debian.img debian.def For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ singularity-2.4.2/libexec/cli/legacy/copy.help0000644000175000017500000000127113211621077020302 0ustar mehdimehdiUSAGE: singularity [...] copy [cp options...] This command will allow you to easily install files into your container from your host. It is a Singularity wrapper for /bin/cp and thus you can, and should, pass any 'cp' arguments to it. EXAMPLES: $ singularity copy /tmp/Debian.img -p script.sh /usr/bin/ $ singularity copy /tmp/Debian.img -r dir /etc/ We strongly recommend using the -p or --preserve option to keep the mode and ownership of the file constant. You should look at "man cp" to see other arguments that might be useful For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ singularity-2.4.2/libexec/cli/legacy/copy.exec0000644000175000017500000000504413211621077020300 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) if [ -e "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.help" ]; then cat "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.help" else message ERROR "No help exists for this command\n" exit 1 fi exit ;; -*) message ERROR "Unknown option: $1\n" exit 1 ;; *) break; ;; esac done if [ -z "${1:-}" ]; then if [ -e "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.help" ]; then head -n 1 "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.help" else message ERROR "To see usage summary, try: singularity help $SINGULARITY_COMMAND\n" fi exit 0 fi SINGULARITY_IMAGE="${1:-}" export SINGULARITY_IMAGE shift if [ -x "$SINGULARITY_libexecdir/singularity/bin/copy" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/copy" "$@" else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/copy\n" exit 1 fi message ERROR "We should never have gotten here... ARRRGGGG!!!!\n" exit 255 singularity-2.4.2/libexec/cli/legacy/bootstrap.exec0000644000175000017500000000770213211621077021346 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi SINGULARITY_CHECKS="no" SINGULARITY_CHECKTAGS=bootstrap SINGULARITY_CHECKLEVEL=3 while true; do case ${1:-} in -h|--help|help) if [ -e "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.help" ]; then cat "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.help" else message ERROR "No help exists for this command\n" exit 1 fi exit ;; -T|--notest) shift SINGULARITY_NOTEST=1 export SINGULARITY_NOTEST ;; -f|--force) shift SINGULARITY_REBOOTSTRAP=1 export SINGULARITY_REBOOTSTRAP ;; -s|--section) shift SINGULARITY_BUILDSECTION="$1" export SINGULARITY_BUILDSECTION shift ;; --nobase) shift SINGULARITY_BUILDNOBASE=1 export SINGULARITY_BUILDNOBASE ;; -c|--check|--checks) SINGULARITY_CHECKS="yes" shift; ;; -l|--low) # levels 3,2,1 SINGULARITY_CHECKLEVEL=3 # LOW shift; ;; -m|--med|--medium) # levels 2,1 SINGULARITY_CHECKLEVEL=2 # MED shift; ;; -h|--high) SINGULARITY_CHECKLEVEL=1 # level 1 only shift; # HIGH ;; -t|--tag) shift; SINGULARITY_CHECKTAGS="${1:-}" shift; ;; -*) message ERROR "Unknown option: $1\n" exit 1 ;; *) break; ;; esac done if [ -z "${1:-}" ]; then if [ -e "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.help" ]; then head -n 1 "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.help" else message ERROR "To see usage summary, try: singularity help $SINGULARITY_COMMAND\n" fi exit 0 fi SINGULARITY_IMAGE="${1:-}" SINGULARITY_BUILDDEF="${2:-}" SINGULARITY_WRITABLE=1 export SINGULARITY_IMAGE SINGULARITY_WRITABLE SINGULARITY_BUILDDEF SINGUARITY_libexecdir \ SINGULARITY_CHECKS SINGULARITY_CHECKTAGS SINGULARITY_CHECKLEVEL shift shift if [ -x "$SINGULARITY_libexecdir/singularity/bin/bootstrap" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/bootstrap" else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/bootstrap\n" exit 1 fi message ERROR "We should never have gotten here... Ahhh!!!!\n" exit 255 singularity-2.4.2/libexec/cli/mount.exec0000644000175000017500000000610713211621077017225 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -w|--writable) SINGULARITY_WRITABLE=1 export SINGULARITY_WRITABLE shift ;; -o|--overlay) shift SINGULARITY_OVERLAYIMAGE="${1:-}" export SINGULARITY_OVERLAYIMAGE shift if [ ! -e "${SINGULARITY_OVERLAYIMAGE:-}" ]; then message ERROR "Overlay image must be a file or directory!\n" ABORT 255 fi ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi SINGULARITY_IMAGE="${1:-}" export SINGULARITY_IMAGE shift if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/mount-suid" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/mount-suid" "$@" elif [ -x "$SINGULARITY_libexecdir/singularity/bin/mount" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/mount" "$@" else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/mount\n" exit 1 fi message ERROR "We should never have gotten here... ARRRGGGG!!!!\n" exit 255 singularity-2.4.2/libexec/cli/build.exec0000644000175000017500000003222413211621077017161 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016-2017, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if ! singularity_which mksquashfs > /dev/null 2>&1; then message ERROR "You must install squashfs-tools to build images\n" ABORT 255 fi build_cleanup() { message 1 "Cleaning up...\n"; if [ -f "${SINGULARITY_CONTENTS:-}" ]; then rm -f "${SINGULARITY_CONTENTS:-}" fi if [ -n "${SINGULARITY_CLEANUP:-}" ]; then rm -f "${SINGULARITY_BUILDDEF:-}" fi if [ -z "${SINGULARITY_NOCLEANUP:-}" -a -z "${SINGULARITY_SANDBOX:-}" ]; then if [ -d "${SINGULARITY_ROOTFS:-}" ]; then rm -rf "${SINGULARITY_ROOTFS:-}" fi fi } atexit build_cleanup USERID=`id -ru` SINGULARITY_CHECKS="no" SINGULARITY_CHECKTAGS=bootstrap SINGULARITY_CHECKLEVEL=3 while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -s|--sandbox) SINGULARITY_SANDBOX=1 shift ;; -w|--writable) if [ "$USERID" != "0" ]; then message ERROR "Writable images must be created as root\n" message ERROR "Use --sandbox (does not required sudo) for a writable folder.\n" exit 1 fi SINGULARITY_WRITABLE=1 shift ;; -f|-F|--force) SINGULARITY_FORCE=1 shift ;; -T|--notest) shift SINGULARITY_NOTEST=1 export SINGULARITY_NOTEST ;; -s|--section) shift SINGULARITY_BUILDSECTION="$1" export SINGULARITY_BUILDSECTION shift ;; -c|--check|--checks) SINGULARITY_CHECKS="yes" shift; ;; -l|--low) # levels 3,2,1 SINGULARITY_CHECKLEVEL=3 # LOW shift; ;; -m|--med|--medium) # levels 2,1 SINGULARITY_CHECKLEVEL=2 # MED shift; ;; -h|--high) SINGULARITY_CHECKLEVEL=1 # level 1 only shift; # HIGH ;; -t|--tag) shift; SINGULARITY_CHECKTAGS="${1:-}" shift; ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done ################################################################################ # Source Usage and Help ################################################################################ if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${2:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi SINGULARITY_CONTAINER_OUTPUT="${1:-}" shift ################################################################################ # Are the commands and permissions valid? ################################################################################ SINGULARITY_BUILDDEF="${1:-}" # not allowed to use --writable and --sandbox if [ -n "${SINGULARITY_WRITABLE:-}" -a -n "${SINGULARITY_SANDBOX:-}" ]; then message ERROR "--writable and --sandbox are conflicting options. There can be only one.\n" ABORT 255 fi # The user is not root if [ "$USERID" != "0" ]; then # They want sandbox if [ -n "${SINGULARITY_SANDBOX:-}" ]; then # Sandbox with deffile is no go. if eval is_deffile "${SINGULARITY_BUILDDEF}"; then message ERROR "You must be the root user to sandbox from a Singularity recipe file\n" exit 1 # Sandbox with anything else is ok else message WARNING "Building sandbox as non-root may result in wrong file permissions\n" fi fi fi ################################################################################ # Sandbox Directory and Image Creation ################################################################################ # if the container exists, build into it (or delete it if -F was passed) if [ -e "$SINGULARITY_CONTAINER_OUTPUT" ]; then if eval is_image "${SINGULARITY_CONTAINER_OUTPUT}"; then if [ -z "${SINGULARITY_FORCE:-}" ]; then message 1 "Building into existing container: $SINGULARITY_CONTAINER_OUTPUT\n" SINGULARITY_BTSTRP_2EXISTING=1 elif [ -n "${SINGULARITY_FORCE:-}" ]; then # this may be a directory, so rm -rf rm -rf "$SINGULARITY_CONTAINER_OUTPUT" fi else message ERROR "$SINGULARITY_CONTAINER_OUTPUT is not an image file\n" exit 1 fi fi if [ -n "${SINGULARITY_SANDBOX:-}" ]; then SINGULARITY_ROOTFS="$SINGULARITY_CONTAINER_OUTPUT" SINGULARITY_IMAGE="$SINGULARITY_CONTAINER_OUTPUT" if [ "$USERID" == "0" ]; then SINGULARITY_ROOTFS_DIRNAME=`dirname "$SINGULARITY_ROOTFS"` SINGULARITY_ROOTFS_DIRNAME_OWNER=`stat -c '%u' $SINGULARITY_ROOTFS_DIRNAME` fi if [ ! -d "${SINGULARITY_ROOTFS:-}" ]; then if ! mkdir -p "${SINGULARITY_ROOTFS:-}"; then message ERROR "Could not create sandbox!\n"; exit 1 fi fi else if ! SINGULARITY_ROOTFS=`mktemp -d ${SINGULARITY_TMPDIR:-/tmp}/.singularity-build.XXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi SINGULARITY_IMAGE="$SINGULARITY_ROOTFS" fi export SINGULARITY_IMAGE SINGULARITY_ROOTFS \ SINGULARITY_CHECKS SINGULARITY_CHECKTAGS SINGULARITY_CHECKLEVEL ################################################################################ # Image Handlers # Note this code is redundant, needs to be consolidated with handlers/ folder ################################################################################ case $SINGULARITY_BUILDDEF in docker://*) SINGULARITY_CONTAINER="$SINGULARITY_BUILDDEF" if ! SINGULARITY_CONTENTS=`mktemp ${TMPDIR:-/tmp}/.singularity-layers.XXXXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi export SINGULARITY_CONTAINER SINGULARITY_CONTENTS eval_abort "$SINGULARITY_libexecdir/singularity/python/import.py" message 1 "Importing: base Singularity environment\n" zcat "$SINGULARITY_libexecdir/singularity/bootstrap-scripts/environment.tar" | tar xBf - -C "${SINGULARITY_ROOTFS}" || exit $? for i in `cat "$SINGULARITY_CONTENTS"`; do message 1 "Importing: $i\n" zcat "$i" | (cd "$SINGULARITY_ROOTFS"; tar --exclude=dev/* -xf - ) || exit $? done SINGULARITY_BUILDDEF="$SINGULARITY_ROOTFS" SINGULARITY_QUIET_SANDBOXMESSAGE=1 ;; shub://*) SINGULARITY_CONTAINER="$SINGULARITY_BUILDDEF" if ! SINGULARITY_CONTENTS=`mktemp ${TMPDIR:-/tmp}/.singularity-layerfile.XXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi # If not set, don't export PULLFOLDER. PULL in shub/main.py will default to it if [ ! -n "${SINGULARITY_PULLFOLDER:-}" ]; then export SINGULARITY_CONTENTS SINGULARITY_CONTAINER else export SINGULARITY_PULLFOLDER SINGULARITY_CONTENTS SINGULARITY_CONTAINER fi # relying on pull.py for error checking here ${SINGULARITY_libexecdir}/singularity/python/pull.py # switch $SINGULARITY_CONTAINER from remote to local SINGULARITY_BUILDDEF=`cat $SINGULARITY_CONTENTS` SINGULARITY_CLEANUP=1 ;; esac if [ -f "${SINGULARITY_BUILDDEF:-}" ]; then if eval is_tar "${SINGULARITY_BUILDDEF}"; then message 1 "Building from local tar file: $SINGULARITY_BUILDDEF\n" nonroot_build_warning if ! eval "zcat_compat ${SINGULARITY_BUILDDEF} 2>/dev/null | tar xf - -C ${SINGULARITY_ROOTFS}" >/dev/null 2>&1; then message ERROR "Failed to export contents of ${SINGULARITY_BUILDDEF} to ${SINGULARITY_ROOTFS}\n" ABORT 255 fi elif eval is_image "${SINGULARITY_BUILDDEF}"; then message 1 "Building from local image: $SINGULARITY_BUILDDEF\n" nonroot_build_warning if ! eval "${SINGULARITY_bindir}"/singularity image.export "${SINGULARITY_BUILDDEF}" 2>/dev/null | tar xBf - -C "${SINGULARITY_ROOTFS}" >/dev/null 2>&1; then message ERROR "Failed to export contents of ${SINGULARITY_BUILDDEF} to ${SINGULARITY_ROOTFS}\n" ABORT 255 fi elif eval is_deffile "${SINGULARITY_BUILDDEF}"; then message 1 "Using container recipe deffile: $SINGULARITY_BUILDDEF\n" if [ "$USERID" != "0" ]; then message ERROR "You must be the root user to build from a Singularity recipe file\n" exit 1 fi if [ -n "${SINGULARITY_BTSTRP_2EXISTING:-}" ]; then # we are bootstrapping to an existing image and need to populate the rootfs if ! eval "${SINGULARITY_bindir}"/singularity image.export "${SINGULARITY_CONTAINER_OUTPUT}" | tar xBf - -C "${SINGULARITY_ROOTFS}"; then message ERROR "Failed to export contents of ${SINGULARITY_BUILDDEF} to ${SINGULARITY_ROOTFS}\n" ABORT 255 fi fi export SINGULARITY_BUILDDEF eval_abort "$SINGULARITY_libexecdir/singularity/bin/builddef" else message ERROR "Unsupported file type: $SINGULARITY_BUILDDEF\n" exit 1 fi elif [ -d "$SINGULARITY_BUILDDEF" ]; then nonroot_build_warning if [ -z ${SINGULARITY_QUIET_SANDBOXMESSAGE:-} ]; then message 1 "Building image from sandbox: $SINGULARITY_BUILDDEF\n" SINGULARITY_NOCLEANUP=1 # don't delete when building directly from sandbox fi SINGULARITY_ROOTFS="$SINGULARITY_BUILDDEF" else message ERROR "Unknown container build Singularity recipe format: $SINGULARITY_BUILDDEF\n" ABORT 255 fi if [ "$USERID" != "0" ]; then # This is required as some files in a container are not readable by owner and # thus fail to build when not root. chmod u+rw -R "$SINGULARITY_ROOTFS" fi if [ -z "${SINGULARITY_SANDBOX:-}" ]; then if [ -n "${SINGULARITY_WRITABLE:-}" ]; then if [ -z "${SINGULARITY_BTSTRP_2EXISTING:-}" ]; then CONTAINER_SIZE=`du -sm ${SINGULARITY_ROOTFS} | cut -f 1` # using the let builtin instead of bc which is not guaranteed let "IMAGE_SIZE=(${CONTAINER_SIZE:-768} * 125)/100" if [ "$IMAGE_SIZE" -lt 10 ]; then IMAGE_SIZE=10 fi message 1 "Creating empty Singularity writable container ${CONTAINER_SIZE}MB\n" eval_abort ${SINGULARITY_bindir}/singularity image.create --size ${IMAGE_SIZE} ${SINGULARITY_CONTAINER_OUTPUT} fi message 1 "Building Singularity image...\n" tar -cf - -C "$SINGULARITY_ROOTFS" . | eval_abort ${SINGULARITY_bindir}/singularity image.import ${SINGULARITY_CONTAINER_OUTPUT} else message 1 "Building Singularity image...\n" if [ "$USERID" != "0" ]; then OPTS="-all-root" else OPTS="" fi if ! mksquashfs "$SINGULARITY_ROOTFS/" "$SINGULARITY_CONTAINER_OUTPUT" -noappend $OPTS > /dev/null; then message ERROR "Failed squashing image, left template directory at: $SINGULARITY_ROOTFS\n" exit 1 fi fi if ! ${SINGULARITY_libexecdir}/singularity/bin/prepheader "$SINGULARITY_CONTAINER_OUTPUT" > /dev/null; then message ERROR "Failed to prepend singularity header\n" exit 1 fi chmod a+x "$SINGULARITY_CONTAINER_OUTPUT" fi message 1 "Singularity container built: $SINGULARITY_CONTAINER_OUTPUT\n" singularity-2.4.2/libexec/cli/exec.exec0000644000175000017500000000566013211621077017012 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -f "${HOME:-}/.singularity-init" ]; then . "${HOME:-}/.singularity-init" fi if [ -f "$SINGULARITY_libexecdir/singularity/cli/action_argparser.sh" ]; then . "$SINGULARITY_libexecdir/singularity/cli/action_argparser.sh" else message ERROR "Could not find the action argument parser\n" exit 1 fi if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${SINGULARITY_IMAGE:-}" ]; then SINGULARITY_IMAGE="${1:-}" export SINGULARITY_IMAGE shift fi if [ -z "${SINGULARITY_IMAGE:-}" ]; then exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" fi if [ -n "${SINGULARITY_DAEMON_JOIN:-}" ]; then SINGULARITY_DAEMON_FILE=`singularity_daemon_file` export SINGULARITY_DAEMON_FILE fi if [ -f "$SINGULARITY_sysconfdir/singularity/init" ]; then . "$SINGULARITY_sysconfdir/singularity/init" fi if [ -x "$SINGULARITY_libexecdir/singularity/image-handler.sh" ]; then . "$SINGULARITY_libexecdir/singularity/image-handler.sh" fi if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/action-suid" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/action-suid" "$@" <&0 elif [ -x "$SINGULARITY_libexecdir/singularity/bin/action" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/action" "$@" <&0 else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/action\n" exit 1 fi singularity-2.4.2/libexec/cli/instance.stop.exec0000644000175000017500000000647013211621077020656 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # This software is licensed under a 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi ## Init Singularity environment if [ -f "$SINGULARITY_sysconfdir/singularity/init" ]; then . "$SINGULARITY_sysconfdir/singularity/init" fi if ! USERID=`id -ru`; then message ERROR "Could not ascertain user ID\n" exit 255 fi KILL_VAL="15" KILL_WAIT="10" while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -u|--user) if [ "$USERID" = 0 ]; then shift USERID=${1:-} shift else message ERROR "Must be root to stop with -u/--user option\n" exit 1 fi ;; -a|--all) ALL_DAEMONS=1 export ALL_DAEMONS shift ;; -f|--force) FORCE=1 shift ;; -s|--signal) shift KILL_VAL=${1:-} shift ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" -a -z "${ALL_DAEMONS:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi COUNT=0 if [ -n "${ALL_DAEMONS:-}" ]; then DAEMONS=`singularity_daemon_glob '*' | sort | uniq` else DAEMONS=`singularity_daemon_glob "$@" | sort | uniq` fi for i in $DAEMONS; do if [ -f "$i" ]; then if [ ! -s "$i" ]; then rm "$i" continue fi FILE_NAME=`basename "$i"` DAEMON_NAME=${FILE_NAME##*-} . "$i" echo "Stopping ${DAEMON_NAME} instance of ${DAEMON_IMAGE} (PID=${DAEMON_PID})" if [ -n "${FORCE:-}" ]; then kill -s 9 ${DAEMON_PID} 2> /dev/null else kill -s ${KILL_VAL} ${DAEMON_PID} 2> /dev/null if [ $? = 0 ]; then { sleep "${KILL_WAIT}"; test -d "/proc/${DAEMON_PID}" && kill -s 9 "${DAEMON_PID}"; } & else message ERROR "No instance running with pid ${DAEMON_PID}\n" exit 1 fi fi rm $i let "COUNT++" fi done if [ "$COUNT" == 0 ]; then message ERROR "No instances found with the name(s): $*\n" exit 1 else message 2 "Stopped $COUNT instances\n" fi singularity-2.4.2/libexec/cli/mount.info0000644000175000017500000000146013211621077017231 0ustar mehdimehdiNAME="mount" SECTION="management" SUMMARY="Mount a Singularity container image" USAGE="singularity [...] mount [mount options...] " print_help() { cat< [command options...] ... GLOBAL OPTIONS: -d|--debug Print debugging information -h|--help Display usage summary -s|--silent Only print errors -q|--quiet Suppress all normal output --version Show application version -v|--verbose Increase verbosity +1 -x|--sh-debug Print shell wrapper debugging information EOF echo echo "GENERAL COMMANDS:" egrep -l '^SECTION="?general"?' "$SINGULARITY_libexecdir/singularity/cli/"*.info | while read i; do if [ -f "$i" ]; then . "$i" if [ -n "$USAGE" -a -n "$NAME" ]; then printf " %-10s %-65.65s\n" "$NAME" "$SUMMARY" fi fi done echo echo "CONTAINER USAGE COMMANDS:" egrep -l '^SECTION="?action"?' "$SINGULARITY_libexecdir/singularity/cli/"*.info | while read i; do if [ -f "$i" ]; then . "$i" if [ -n "$USAGE" -a -n "$NAME" ]; then printf " %-10s %-65.65s\n" "$NAME" "$SUMMARY" fi fi done echo echo "CONTAINER MANAGEMENT COMMANDS:" egrep -l '^SECTION="?management"?' "$SINGULARITY_libexecdir/singularity/cli/"*.info | while read i; do if [ -f "$i" ]; then . "$i" if [ -n "$USAGE" -a -n "$NAME" ]; then printf " %-10s %-65.65s\n" "$NAME" "$SUMMARY" fi fi done echo echo "COMMAND GROUPS:" egrep -l '^SECTION="?group"?' "$SINGULARITY_libexecdir/singularity/cli/"*.info | while read i; do if [ -f "$i" ]; then . "$i" if [ -n "$USAGE" -a -n "$NAME" ]; then printf " %-10s %-65.65s\n" "$NAME" "$SUMMARY" fi fi done echo echo cat < For any additional help or support visit the Singularity website: http://singularity.lbl.gov/ EOF exit 0 elif [ -f "$SINGULARITY_libexecdir/singularity/cli/${HELP_ASK}.info" ]; then print_help() { echo "No help defined for ${HELP_ASK}."; } . "$SINGULARITY_libexecdir/singularity/cli/${HELP_ASK}.info" if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" fi print_help echo exit 0 elif [ -e "${HELP_ASK}" ]; then RETVAL=0 SINGULARITY_IMAGE="${HELP_ASK:-}" if ! SINGULARITY_MOUNTPOINT=`mktemp -d ${TMPDIR:-/tmp}/.singularity-inspect.XXXXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi export SINGULARITY_IMAGE SINGULARITY_MOUNTPOINT shift if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/mount-suid" ]; then eval "$SINGULARITY_libexecdir/singularity/bin/mount-suid" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/help.sh" RETVAL=$? elif [ -x "$SINGULARITY_libexecdir/singularity/bin/mount" ]; then eval "$SINGULARITY_libexecdir/singularity/bin/mount" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/help.sh" RETVAL=$? else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/mount\n" exit 1 fi rmdir "$SINGULARITY_MOUNTPOINT" exit $RETVAL else echo "No help provided for: ${HELP_ASK}" exit 1 fi singularity-2.4.2/libexec/cli/exec.info0000644000175000017500000000771413211621077017023 0ustar mehdimehdiNAME="exec" SECTION="action" SUMMARY="Execute a command within container" USAGE="singularity [...] exec [exec options...] " print_help() { cat< A user-bind path specification. spec has the format src[:dest[:opts]], where src and dest are outside and inside paths. If dest is not given, it is set equal to src. Mount options ('opts') may be specified as 'ro' (read-only) or 'rw' (read/write, which is the default). This option can be called multiple times. -c|--contain Use minimal /dev and empty other directories (e.g. /tmp and $HOME) instead of sharing filesystems on your host -C|--containall Contain not only file systems, but also PID and IPC -e|--cleanenv Clean environment before running container -H|--home A home directory specification. spec can either be a src path or src:dest pair. src is the source path of the home directory outside the container and dest overrides the home directory within the container -i|--ipc Run container in a new IPC namespace -n|--net Run container in a new network namespace (loopback is only network device active) --nv Enable experimental Nvidia support -o|--overlay Use a persistent overlayFS via a writable image -p|--pid Run container in a new PID namespace --pwd Initial working directory for payload process inside the container -S|--scratch Include a scratch directory within the container that is linked to a temporary dir (use -W to force location) -u|--userns Run container in a new user namespace (this allows Singularity to run completely unprivileged on recent kernels and doesn't support all features) -W|--workdir Working directory to be used for /tmp, /var/tmp and $HOME (if -c/--contain was also used) -w|--writable By default all Singularity containers are available as read only. This option makes the file system accessible as read/write. CONTAINER FORMATS SUPPORTED: *.sqsh SquashFS format. Native to Singularity 2.4+ *.img This is the native Singularity image format for all Singularity versions < 2.4. *.tar* Tar archives are exploded to a temporary directory and run within that directory (and cleaned up after). The contents of the archive is a root file system with root being in the current directory. Compression suffixes as '.gz' and '.bz2' are supported. directory/ Container directories that contain a valid root file system. instance://* A local running instance of a container. (See the instance command group.) shub://* A container hosted on Singularity Hub docker://* A container hosted on Docker Hub EXAMPLES: $ singularity exec /tmp/Debian.img cat /etc/debian_version $ singularity exec /tmp/Debian.img python ./hello_world.py $ cat hello_world.py | singularity exec /tmp/Debian.img python $ sudo singularity exec --writable /tmp/Debian.img apt-get update $ singularity exec instance://my_instance ps -ef For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/cli/Makefile.am0000644000175000017500000000333713211621077017253 0ustar mehdimehdicliexecdir = $(libexecdir)/singularity/cli dist_cliexec_SCRIPTS = action_argparser.sh \ apps.exec \ bootstrap.exec \ build.exec \ check.exec \ create.exec \ exec.exec \ help.exec \ instance.exec \ instance.list.exec \ instance.start.exec \ instance.stop.exec \ image.exec \ image.create.exec \ image.expand.exec \ image.export.exec \ image.import.exec \ inspect.exec \ mount.exec \ pull.exec \ run.exec \ shell.exec \ test.exec \ selftest.exec dist_cliexec_DATA = apps.info \ bootstrap.info \ build.info \ check.info \ create.info \ exec.info \ help.info \ instance.info \ instance.list.info \ instance.start.info \ instance.stop.info \ image.info \ image.create.info \ image.expand.info \ image.export.info \ image.import.info \ inspect.info \ mount.info \ pull.info \ run.info \ selftest.info \ shell.info \ test.info MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/cli/bootstrap.exec0000644000175000017500000000755113211621077020104 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi SINGULARITY_CHECKS="no" SINGULARITY_CHECKTAGS=bootstrap SINGULARITY_CHECKLEVEL=3 while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -T|--notest) shift SINGULARITY_NOTEST=1 export SINGULARITY_NOTEST ;; -f|-F|--force) shift SINGULARITY_REBOOTSTRAP=1 export SINGULARITY_REBOOTSTRAP ;; -s|--section) shift SINGULARITY_BUILDSECTION="$1" export SINGULARITY_BUILDSECTION shift ;; -c|--check|--checks) SINGULARITY_CHECKS="yes" shift; ;; -l|--low) # levels 3,2,1 SINGULARITY_CHECKLEVEL=3 # LOW shift; ;; -m|--med|--medium) # levels 2,1 SINGULARITY_CHECKLEVEL=2 # MED shift; ;; -h|--high) SINGULARITY_CHECKLEVEL=1 # level 1 only shift; # HIGH ;; -t|--tag) shift; SINGULARITY_CHECKTAGS="${1:-}" shift; ;; -*) message ERROR "Unknown option: $1\n" exit 1 ;; *) break; ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${1:-}" ]; then if [ -n "${USAGE:-}" ]; then echo "USAGE: $USAGE" else echo "To see usage summary type: singularity help $SINGULARITY_COMMAND" fi exit 0 fi SINGULARITY_IMAGE="${1:-}" SINGULARITY_BUILDDEF="${2:-}" SINGULARITY_WRITABLE=1 export SINGULARITY_IMAGE SINGULARITY_WRITABLE SINGULARITY_BUILDDEF SINGUARITY_libexecdir \ SINGULARITY_CHECKS SINGULARITY_CHECKTAGS SINGULARITY_CHECKLEVEL shift shift message WARNING "The bootstrap command is deprecated and will be removed in a future release.\n" message WARNING "Use the build command like so:\n" message WARNING "singularity build ${SINGULARITY_IMAGE} ${SINGULARITY_BUILDDEF}\n" if [ -x "${SINGULARITY_bindir}/singularity" -a -n "${SINGULARITY_REBOOTSTRAP:-}" ]; then ${SINGULARITY_bindir}/singularity build -f -w ${SINGULARITY_IMAGE} ${SINGULARITY_BUILDDEF} exit 0 elif [ -x "${SINGULARITY_bindir}/singularity" ]; then ${SINGULARITY_bindir}/singularity build -w ${SINGULARITY_IMAGE} ${SINGULARITY_BUILDDEF} exit 0 else message ERROR "Could not locate the Singularity binary: $SINGULARITY_home/singularity\n" exit 1 fi message ERROR "We should never have gotten here... Ahhh!!!!\n" exit 255 singularity-2.4.2/libexec/cli/test.exec0000644000175000017500000000600513211621077017037 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ -f "${HOME:-}/.singularity-init" ]; then . "${HOME:-}/.singularity-init" fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; -a|--app) shift SINGULARITY_APPNAME="$1" shift export SINGULARITY_APPNAME ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done if [ -f "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" ]; then . "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.info" else message ERROR "Could not find the info file for: $SINGULARITY_COMMAND\n" ABORT 255 fi if [ -z "${SINGULARITY_IMAGE:-}" ]; then SINGULARITY_IMAGE="${1:-}" export SINGULARITY_IMAGE shift fi if [ -z "${SINGULARITY_IMAGE:-}" ]; then exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" fi if [ -f "$SINGULARITY_sysconfdir/singularity/init" ]; then . "$SINGULARITY_sysconfdir/singularity/init" fi if [ -x "$SINGULARITY_libexecdir/singularity/image-handler.sh" ]; then . "$SINGULARITY_libexecdir/singularity/image-handler.sh" fi if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/action-suid" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/action-suid" "$@" <&0 elif [ -x "$SINGULARITY_libexecdir/singularity/bin/action" ]; then exec "$SINGULARITY_libexecdir/singularity/bin/action" "$@" <&0 else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/action\n" exit 1 fi singularity-2.4.2/libexec/cli/image.exec0000644000175000017500000000402013211621077017135 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # This file also contains content that is covered under the LBNL/DOE/UC modified # 3-clause BSD license and is subject to the license terms in the LICENSE-LBNL.md # file found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/LICENSE-LBNL.md. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; *) break ;; esac done if [ -n "${1:-}" ]; then if [ -x "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.$1.exec" ]; then SINGULARITY_COMMAND="$SINGULARITY_COMMAND.$1" export SINGULARITY_COMMAND shift exec "$SINGULARITY_libexecdir/singularity/cli/$SINGULARITY_COMMAND.exec" "$@" else message ERROR "Unknown subcommand: $SINGULARITY_COMMAND.$1\n" ABORT 255 fi else exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" fi singularity-2.4.2/libexec/cli/pull.info0000644000175000017500000000414113211621077017042 0ustar mehdimehdiNAME="pull" SECTION="management" SUMMARY="Pull a Singularity/Docker container to \$PWD" USAGE="singularity [...] pull shub://[unique id]" print_help() { cat</dev/null` if [ "$IMAGE_TYPE" != "EXT3" ]; then message ERROR "This is not a writable Singularity image format\n" ABORT 255 fi if ! touch "$SINGULARITY_IMAGE" >/dev/null 2>&1; then message ERROR "Inappropriate permission to modify: $SINGULARITY_IMAGE\n" ABORT 255 fi message 1 "Expanding image by ${SINGULARITY_IMAGESIZE:-768}MB\n" if ! dd if=/dev/zero bs=1M count=${SINGULARITY_IMAGESIZE:-768} status=none >> "$SINGULARITY_IMAGE"; then message ERROR "Failed expanding image!\n" exit 1 fi message 1 "Checking image's file system\n" if ! /sbin/e2fsck -fy "$SINGULARITY_IMAGE"; then exit 1 fi message 1 "Resizing image's file system\n" if ! /sbin/resize2fs "$SINGULARITY_IMAGE"; then exit 1 fi message 1 "Image is done: $SINGULARITY_IMAGE\n" exit 0 singularity-2.4.2/libexec/cli/inspect.exec0000644000175000017500000000714313211621077017531 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi while true; do case ${1:-} in -h|--help|help) exec "$SINGULARITY_libexecdir/singularity/cli/help.exec" "$SINGULARITY_COMMAND" ;; app|--app|-a) shift SINGULARITY_APPNAME="${1:-}" export SINGULARITY_APPNAME shift ;; -j|--json) SINGULARITY_PRINT_STRUCTURED=1 export SINGULARITY_PRINT_STRUCTURED shift; ;; -l|--label|--labels) SELECTED=1 SINGULARITY_INSPECT_LABELS=1 export SINGULARITY_INSPECT_LABELS shift; ;; -d|--deffile) SELECTED=1 SINGULARITY_INSPECT_DEFFILE=1 export SINGULARITY_INSPECT_DEFFILE shift; ;; -hf|--helpfile) SELECTED=1 SINGULARITY_INSPECT_HELP=1 export SINGULARITY_INSPECT_HELP shift; ;; -r|--run|--runscript) SELECTED=1 SINGULARITY_INSPECT_RUNSCRIPT=1 export SINGULARITY_INSPECT_RUNSCRIPT shift; ;; -t|--test) SELECTED=1 SINGULARITY_INSPECT_TEST=1 export SINGULARITY_INSPECT_TEST shift; ;; -e|--environment) SELECTED=1 SINGULARITY_INSPECT_ENVIRONMENT=1 export SINGULARITY_INSPECT_ENVIRONMENT shift; ;; -*) message ERROR "Unknown option: ${1:-}\n" exit 1 ;; *) break ;; esac done if [ -z "${SELECTED:-}" ]; then SINGULARITY_INSPECT_LABELS=1 export SINGULARITY_INSPECT_LABELS fi RETVAL=0 SINGULARITY_IMAGE="${1:-}" if ! SINGULARITY_MOUNTPOINT=`mktemp -d ${TMPDIR:-/tmp}/.singularity-inspect.XXXXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi export SINGULARITY_IMAGE SINGULARITY_MOUNTPOINT shift if [ -z "${SINGULARITY_NOSUID:-}" -a -u "$SINGULARITY_libexecdir/singularity/bin/mount-suid" ]; then eval "$SINGULARITY_libexecdir/singularity/bin/mount-suid" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/inspect.sh" RETVAL=$? elif [ -x "$SINGULARITY_libexecdir/singularity/bin/mount" ]; then eval "$SINGULARITY_libexecdir/singularity/bin/mount" /bin/bash "$SINGULARITY_libexecdir/singularity/helpers/inspect.sh" RETVAL=$? else message ERROR "Could not locate the Singularity binary: $SINGULARITY_libexecdir/singularity/bin/mount\n" exit 1 fi rmdir "$SINGULARITY_MOUNTPOINT" exit $RETVAL singularity-2.4.2/libexec/cli/shell.info0000644000175000017500000001110613211621077017174 0ustar mehdimehdiNAME="shell" SECTION="action" SUMMARY="Run a Bourne shell within container" USAGE="singularity [...] shell [shell options...] " print_help() { cat< A user-bind path specification. spec has the format src[:dest[:opts]], where src and dest are outside and inside paths. If dest is not given, it is set equal to src. Mount options ('opts') may be specified as 'ro' (read-only) or 'rw' (read/write, which is the default). This option can be called multiple times. -c|--contain Use minimal /dev and empty other directories (e.g. /tmp and $HOME) instead of sharing filesystems on your host -C|--containall Contain not only file systems, but also PID and IPC -e|--cleanenv Clean environment before running container -H|--home A home directory specification. spec can either be a src path or src:dest pair. src is the source path of the home directory outside the container and dest overrides the home directory within the container -i|--ipc Run container in a new IPC namespace -j|--join Join a running named instance of the given container image -n|--net Run container in a new network namespace (loopback is only network device active) --nv Enable experimental Nvidia support -o|--overlay Use a persistent overlayFS via a writable image -p|--pid Run container in a new PID namespace (creates child) --pwd Initial working directory for payload process inside the container -S|--scratch Include a scratch directory within the container that is linked to a temporary dir (use -W to force location) -s|--shell Path to program to use for interactive shell -u|--userns Run container in a new user namespace (this allows Singularity to run completely unprivileged on recent kernels and doesn't support all features) -W|--workdir Working directory to be used for /tmp, /var/tmp and $HOME (if -c/--contain was also used) -w|--writable By default all Singularity containers are available as read only. This option makes the file system accessible as read/write. CONTAINER FORMATS SUPPORTED: *.sqsh SquashFS format. Native to Singularity 2.4+ *.img This is the native Singularity image format for all Singularity versions < 2.4. *.tar* Tar archives are exploded to a temporary directory and run within that directory (and cleaned up after). The contents of the archive is a root file system with root being in the current directory. Compression suffixes as '.gz' and '.bz2' are supported. directory/ Container directories that contain a valid root file system. instance://* A local running instance of a container. (See the instance command group.) shub://* A container hosted on Singularity Hub docker://* A container hosted on Docker Hub EXAMPLES: $ singularity shell /tmp/Debian.img Singularity/Debian.img> pwd /home/gmk/test Singularity/Debian.img> exit $ singularity shell -C /tmp/Debian.img Singularity/Debian.img> pwd /home/gmk Singularity/Debian.img> ls -l total 0 Singularity/Debian.img> exit $ sudo singularity shell -w /tmp/Debian.img $ sudo singularity shell --writable /tmp/Debian.img $ singularity shell instance://my_instance $ singularity shell instance://my_instance Singularity: Invoking an interactive shell within container... Singularity container:~> ps -ef UID PID PPID C STIME TTY TIME CMD ubuntu 1 0 0 20:00 ? 00:00:00 /usr/local/bin/singularity/bin/sinit ubuntu 2 0 0 20:01 pts/8 00:00:00 /bin/bash --norc ubuntu 3 2 0 20:02 pts/8 00:00:00 ps -ef For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/cli/check.info0000644000175000017500000000170213211621077017143 0ustar mehdimehdiNAME="check" SECTION="management" SUMMARY="Perform container lint checks" USAGE="singularity [...] check [exec options...] " print_help() { cat< With all of the above, the following environment variables are available to you when called from the shell inside the container. The top variables are relevant to the active app (--app ) and the bottom available for all apps regardless of the active: ACTIVE APP ENVIRONMENT: SINGULARITY_APPNAME the name of the application SINGULARITY_APPROOT the application base (/scif/apps/) SINGULARITY_APPMETA the application metadata folder SINGULARITY_APPDATA the data base folder for active app SINGULARITY_APPINPUT expected input folder within data base folder SINGULARITY_APPOUTPUT the output data folder within data base folder GLOBAL APP ENVIRONMENT: SINGULARITY_DATA scif defined data base for all apps (/scif/data) SINGULARITY_APPS scif defined install bases for all apps (/scif/apps) APPROOT_ root for application APPDATA_ data root for application For additional help, please visit our public documentation pages which are found at: http://singularity.lbl.gov/ EOF } singularity-2.4.2/libexec/helpers/0000755000175000017500000000000013211621077016104 5ustar mehdimehdisingularity-2.4.2/libexec/helpers/README.md0000644000175000017500000000735213211621077017372 0ustar mehdimehdi# Helpers ## Checks Broadly, a check is a script that is run over a mounted filesystem, primary with the purpose of checking for some security issue. This process is tighly controlled, meaning that the script names in the [checks](checks) folder are hard coded into the script [check.sh](check.sh). The flow of checks is the following: - the user calls `singularity check container.img` to invoke [check.exec](../cli/check.exec) - specification of `--low` (3), `--med` (2), or `--high` (1) sets the level to perform. The level is a filter, meaning that a level of 3 will include 3,2,1, and a level of 1 (high) will only call checks of high priority. - specification of `-t/--tag` will allow the user (or execution script) to specify a kind of check. This is primarily to allow for extending the checks to do other types of things. For example, for this initial batch, these are all considered `default` checks. The [check.help](../cli/check.help) displays examples of how the user specifies a tag: ``` # Perform all default checks, these are the same $ singularity check ubuntu.img $ singularity check --tag default ubuntu.img # Perform checks with tag "clean" $ singularity check --tag clean ubuntu.img ``` ### Adding a Check A check should be a bash (or other) script that will perform some action. The following is required: **Relative to SINGULARITY_ROOTFS** The script must perform check actions relative to `$SINGULARITY_ROOTFS`. For example, in python you might change directory to this location: ``` import os base = os.environ["SINGULARITY_ROOTFS"] os.chdir(base) ``` or do the same in bash: ``` cd $SINGULARITY_ROOTFS ls $SINGULARITY_ROOTFS/var ``` Since we are doing a mount, all checks must be static relative to this base, otherwise you are likely checking the host system. **Verbose** The script should indicate any warning/message to the user if the check is found to have failed. If pass, the check's name and status will be printed, with any relevant information. For more thorough checking, you might want to give more verbose output. **Return Code** The script return code of "success" is defined in [check.sh](check.sh), and other return codes are considered not success. When a non success return code is found, the rest of the checks continue running, and no action is taken. We might want to give some admin an ability to specify a check, a level, and prevent continuation of the build/bootstrap given a fail. **Check.sh** The script level, path, and tags should be added to [check.sh](check.sh) in the following format: ``` ################################################################################## # CHECK SCRIPTS ################################################################################## # [SUCCESS] [LEVEL] [SCRIPT] [TAGS] execute_check 0 HIGH "bash $SINGULARITY_libexecdir/singularity/helpers/checks/1-hello-world.sh" security execute_check 0 LOW "python $SINGULARITY_libexecdir/singularity/helpers/checks/2-cache-content.py" clean execute_check 0 HIGH "python $SINGULARITY_libexecdir/singularity/helpers/checks/3-cve.py" security ``` The function `execute_check` will compare the level (`[LEVEL]`) with the user specified (or default) `SINGULARITY_CHECKLEVEL` and execute the check only given it is under the specified threshold, and (not yet implemented) has the relevant tag. The success code is also set here with `[SUCCESS]`. Currently, we aren't doing anything with `[TAGS]` and thus perform all checks. ## Inspect Inspect is called from [inspect.exec](../cli/inspect.exec), and serves to mount an image and then cat/return some subset of content from the singularity metadata folder (`.singularity.d`). singularity-2.4.2/libexec/helpers/inspect.sh0000644000175000017500000000346713211621077020117 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ ! -d "${SINGULARITY_MOUNTPOINT}" ]; then message ERROR "The mount point does not exist: ${SINGULARITY_MOUNTPOINT}\n" ABORT 255 fi if [ ! -d "${SINGULARITY_MOUNTPOINT}/.singularity.d" ]; then message ERROR "The Singularity metadata directory does not exist in image\n" ABORT 255 fi SINGULARITY_ROOTFS=${SINGULARITY_MOUNTPOINT} export SINGULARITY_MOUNTPOINT SINGULARITY_INSPECT_LABELS SINGULARITY_INSPECT_DEFFILE SINGULARITY_INSPECT_RUNSCRIPT SINGULARITY_INSPECT_TEST SINGULARITY_INSPECT_ENVIRONMENT SINGULARITY_ROOTFS SINGULARITY_PRINT_STRUCTURED SINGULARITY_INSPECT_HELP if [ ! -z ${SINGULARITY_APPNAME+x} ]; then export SINGULARITY_APPNAME fi eval_abort "$SINGULARITY_libexecdir/singularity/python/helpers/json/inspect.py" singularity-2.4.2/libexec/helpers/help.sh0000644000175000017500000000276413211621077017401 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ ! -d "${SINGULARITY_MOUNTPOINT}" ]; then message ERROR "The mount point does not exist: ${SINGULARITY_MOUNTPOINT}\n" ABORT 255 fi if [ ! -d "${SINGULARITY_MOUNTPOINT}/.singularity.d" ]; then message ERROR "The Singularity metadata directory does not exist in image\n" ABORT 255 fi SINGULARITY_ROOTFS=${SINGULARITY_MOUNTPOINT} export SINGULARITY_MOUNTPOINT SINGULARITY_ROOTFS if [ -n "${SINGULARITY_APPNAME:-}" ]; then if [ -f "${SINGULARITY_MOUNTPOINT}/scif/apps/${SINGULARITY_APPNAME}/scif/runscript.help" ]; then eval_abort cat "${SINGULARITY_MOUNTPOINT}/scif/apps/${SINGULARITY_APPNAME}/scif/runscript.help" else echo "No runscript help is defined for this application." fi elif [ -f "${SINGULARITY_MOUNTPOINT}/.singularity.d/runscript.help" ]; then eval_abort cat "${SINGULARITY_MOUNTPOINT}/.singularity.d/runscript.help" else echo "No runscript help is defined for this image." fi singularity-2.4.2/libexec/helpers/checks/0000755000175000017500000000000013211621077017344 5ustar mehdimehdisingularity-2.4.2/libexec/helpers/checks/1-cache-content.py0000755000175000017500000000436713211621077022604 0ustar mehdimehdi#!/usr/bin/env python # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this # distribution and at https://github.com/singularityware/singularity # # This file is part of the Singularity Linux container project. # It is subject to the license terms in the LICENSE.md file # found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity. No part # of Singularity, including this file, may be copied, modified, # propagated, or distributed except according to the terms # contained in the LICENSE.md file. # # Alert the user about files in the cache import platform import sys import os base = os.environ["SINGULARITY_ROOTFS"] os.chdir(base) os_base, os_name, os_version = platform.linux_distribution() os_base = os_base.lower() returncode = 0 def check_cache(returncode): '''check cache will look for archives in /var/cache, and return 1 if files are found''' # The cache should only have apt debconf ldconfig skip = ["apt", "debconf", "ldconfig"] cache_dirs = [x for x in os.listdir("var/cache") if x not in skip] if len(cache_dirs) > 3: to_remove = "\n".join(["rm -rf /var/cache/%s" % x for x in cache_dirs]) print("PROBLEM: /var/cache has uneccessary entries") print("RESOLVE: \n%s" % to_remove) returncode = 1 def check_apt(returncode): '''check apt will look for files in apt archives that need to be cleaned. Return 1 if files are found''' if os.path.exists('var/cache/apt/archives'): skip = ['partial', 'lock'] count = len([x for x in os.listdir("var/cache/apt/archives/") if x not in skip]) # The apt cache should be cleaned if count > 0: print("PROBLEM: apt-get cache should be cleaned.") print("RESOLVE: sudo apt-get clean") returncode = 1 return returncode # Debian Cache if os_base in ["debian", "ubuntu"]: if os.path.exists("var/cache"): returncode = check_cache(returncode) if os.path.exists("var/cache/apt/archives"): returncode = check_apt(returncode) sys.exit(returncode) singularity-2.4.2/libexec/helpers/checks/1-docker.py0000755000175000017500000000253213211621077021330 0ustar mehdimehdi#!/usr/bin/env python # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this # distribution and at https://github.com/singularityware/singularity # # This file is part of the Singularity Linux container project. # It is subject to the license terms in the LICENSE.md file # found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity. No part # of Singularity, including this file, may be copied, modified, # propagated, or distributed except according to the terms # contained in the LICENSE.md file. # # Yell at the user for bad practices from Docker containers import sys import os base = os.environ["SINGULARITY_ROOTFS"] os.chdir(base) returncode = 0 if os.geteuid() != 0: print("You must run this test as sudo, skipping") sys.exit(returncode) # Apt-get cache skip = ['.profile', '.bashrc', '.tcshc', '.cshrc', '.bash_history', '.bash_profile'] root = [x for x in os.listdir('root') if x not in skip] # The user should not put content in root! if len(root) > 0: print("PROBLEM: You should not save content in roots home.") print("RESOLVE: Install to /opt or /usr/local") print("\n".join(root)) returncode = 1 sys.exit(returncode) singularity-2.4.2/libexec/helpers/checks/3-cve.py0000755000175000017500000000514113211621077020637 0ustar mehdimehdi#!/usr/bin/env python # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this # distribution and at https://github.com/singularityware/singularity # # This file is part of the Singularity Linux container project. # It is subject to the license terms in the LICENSE.md file # found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity. No part # of Singularity, including this file, may be copied, modified, # propagated, or distributed except according to the terms # contained in the LICENSE.md file. import os import platform import sys import re try: from urllib.request import urlopen, Request, unquote except ImportError: from urllib2 import urlopen, Request, HTTPError os_base, os_name, os_version = platform.linux_distribution() os_base = os_base.lower() os_names = "|".join([x.lower() for x in os_name.split('/')]) base = os.environ["SINGULARITY_ROOTFS"] os.chdir(base) ################################################################## # Common Vulnerabilities Database, High Risk ################################################################## base = "https://security-tracker.debian.org/tracker/status/release" filters = "?filter=1&filter=high_urgency" release = "stable" url = Request('%s/%s/%s' % (base, release, filters)) response = urlopen(url).read().decode('utf-8') cve_codes = re.findall(">CVE-(.*?)<", response) returncode = 0 # We are only testing debian if os_base not in ['debian', 'ubuntu']: print("OS not in debian/ubuntu family, skipping test.") sys.exit(returncode) # Iterate through the CVE codes, and assess if the distribution matches print("Checking %s system for %s CVE vulnerabilities..." % (os_base, len(cve_codes))) for cve_code in cve_codes: url = "https://security-tracker.debian.org/tracker/CVE-%s" % cve_code request = Request(url) try: response = urlopen(request) except HTTPError: pass html = response.read().decode('utf-8') table = html.replace('PTS', '').split('')[2] title = table.split('')[2] title = re.findall('">(.*?)', title)[0] print("CVE-%s: %s" % (cve_code, title)) rows = table.replace('', '').split('') for row in rows: if row: if re.search(os_names, row): print("PROBLEM: Vulnerability CVE-%s" % cve_code) print("RESOLVE: %s" % url) returncode = 1 sys.exit(returncode) singularity-2.4.2/libexec/helpers/checks/1-hello-world.sh0000755000175000017500000000007113211621077022267 0ustar mehdimehdi#!/bin/bash echo "Hello World, security level 1" exit 0 singularity-2.4.2/libexec/helpers/checks/Makefile.am0000644000175000017500000000043413211621077021401 0ustar mehdimehdichecksdir = $(libexecdir)/singularity/helpers/checks dist_checks_SCRIPTS = 1-hello-world.sh \ 1-cache-content.py \ 1-bash-hiddens.py \ 3-cve.py \ 1-docker.py MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/helpers/checks/1-bash-hiddens.py0000755000175000017500000000440513211621077022413 0ustar mehdimehdi#!/usr/bin/env python # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this # distribution and at https://github.com/singularityware/singularity # # This file is part of the Singularity Linux container project. # It is subject to the license terms in the LICENSE.md file # found in the top-level directory of this distribution and at # https://github.com/singularityware/singularity. No part # of Singularity, including this file, may be copied, modified, # propagated, or distributed except according to the terms # contained in the LICENSE.md file. # # Alert the user about history and profiles import platform import sys import os base = os.environ["SINGULARITY_ROOTFS"] os.chdir(base) os_base, os_name, os_version = platform.linux_distribution() os_base = os_base.lower() returncode = 0 if os.geteuid() != 0: print("You must run this test as sudo, skipping") sys.exit(returncode) def find_history(returncode): '''find_history will return 1 if any history files are found''' if os.path.exists('root'): history = [x for x in os.listdir('root') if x.endswith('history') or 'hist' in x] # The apt cache should be cleaned if len(history) > 0: print("PROBLEM: history at /root home found.") print("RESOLVE: check for sensitive content.") print('\n'.join(history)) returncode = 1 return returncode def find_profiles(returncode): '''find_profiles will return 1 if any profile files are found''' if os.path.exists('root'): profiles = [x for x in os.listdir('root') if x.endswith('rc') or 'profile' in x] # The apt cache should be cleaned if len(profiles) > 0: print("PROBLEM: profiles at /root home found.") print("RESOLVE: check for sensitive content.") print('\n'.join(profiles)) returncode = 1 return returncode # Debian Cache if os_base in ["debian", "ubuntu", "centos", "redhat"]: if os.path.exists("root"): returncode = find_history(returncode) returncode = find_profiles(returncode) sys.exit(returncode) singularity-2.4.2/libexec/helpers/Makefile.am0000644000175000017500000000026613211621077020144 0ustar mehdimehdiSUBDIRS = apps checks helpersdir = $(libexecdir)/singularity/helpers dist_helpers_SCRIPTS = check.sh help.sh inspect.sh image.sh record-env.sh MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/helpers/apps/0000755000175000017500000000000013211621077017047 5ustar mehdimehdisingularity-2.4.2/libexec/helpers/apps/Makefile.am0000644000175000017500000000016313211621077021103 0ustar mehdimehdiappsdir = $(libexecdir)/singularity/helpers/apps dist_apps_SCRIPTS = list.sh MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/helpers/apps/list.sh0000644000175000017500000000303013211621077020352 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ ! -d "${SINGULARITY_MOUNTPOINT}" ]; then message ERROR "The mount point does not exist: ${SINGULARITY_MOUNTPOINT}\n" ABORT 255 fi if [ ! -d "${SINGULARITY_MOUNTPOINT}/.singularity.d" ]; then message ERROR "The Singularity metadata directory does not exist in image\n" ABORT 255 fi for app in ${SINGULARITY_MOUNTPOINT}/scif/apps/*; do if [ -d "$app/scif" ]; then APPNAME=`basename $app` echo "$APPNAME" fi done exit 0 singularity-2.4.2/libexec/helpers/record-env.sh0000755000175000017500000000203413211621077020506 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # # when calling this script, please use the following syntax to clean env # env -i ./record-env.sh $SINGULARITY_ROOTFS # because env should be clean upon entry, no sanity checking or functions SINGULARITY_ROOTFS=${1:-} for file in $(ls ${SINGULARITY_ROOTFS}/.singularity.d/env/*.sh); do . ${file} > /dev/null 2>&1 done tmpfile=$(mktemp) printenv | sort >$tmpfile echo $tmpfile singularity-2.4.2/libexec/helpers/image.sh0000644000175000017500000000371213211621077017525 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ ! -d "${SINGULARITY_MOUNTPOINT}" ]; then message ERROR "The mount point does not exist: ${SINGULARITY_MOUNTPOINT}\n" ABORT 255 fi if ! which tar >/dev/null; then message ERROR "Could not find the program: tar\n" ABORT 255 fi case "$SINGULARITY_COMMAND" in image.import) if [ -n "${SINGULARITY_IMPORT_FILE:-}" ]; then exec zcat "${SINGULARITY_IMPORT_FILE}" | tar --ignore-failed-read -xf - -C "$SINGULARITY_MOUNTPOINT" else exec tar --ignore-failed-read -xf - -C "$SINGULARITY_MOUNTPOINT" fi ;; image.export) if [ -n "${SINGULARITY_EXPORT_FILE:-}" ]; then exec tar --ignore-failed-read -cf - -C "$SINGULARITY_MOUNTPOINT" . > "${SINGULARITY_EXPORT_FILE}" else exec tar --ignore-failed-read -cf - -C "$SINGULARITY_MOUNTPOINT" . fi ;; *) message ERROR "Unknown image class command\n" ABORT 255 ;; esac singularity-2.4.2/libexec/helpers/check.sh0000644000175000017500000000665213211621077017526 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. ## Basic sanity if [ -z "$SINGULARITY_libexecdir" ]; then echo "Could not identify the Singularity libexecdir." exit 1 fi ## Load functions if [ -f "$SINGULARITY_libexecdir/singularity/functions" ]; then . "$SINGULARITY_libexecdir/singularity/functions" else echo "Error loading functions: $SINGULARITY_libexecdir/singularity/functions" exit 1 fi if [ ! -d "${SINGULARITY_MOUNTPOINT}" ]; then message ERROR "The mount point does not exist: ${SINGULARITY_MOUNTPOINT}\n" ABORT 255 fi if [ -z "${SINGULARITY_MOUNTPOINT}" ]; then message ERROR "The mount point does not exist: ${SINGULARITY_MOUNTPOINT}\n" ABORT 255 fi SINGULARITY_ROOTFS=${SINGULARITY_MOUNTPOINT} export SINGULARITY_MOUNTPOINT SINGULARITY_CHECKLEVEL \ SINGULARITY_ROOTFS SINGULARITY_CHECKTAGS message DEFAULT "Checking tags $SINGULARITY_CHECKTAGS" ################################################################################## # USER TAGS ################################################################################## ################################################################################## # CHECK SCRIPTS ################################################################################## # [SUCCESS] [LEVEL] [SCRIPT] [TAGS] exec_check 0 LOW "python $SINGULARITY_libexecdir/singularity/helpers/checks/1-bash-hiddens.py" security clean bootstrap exec_check 0 LOW "bash $SINGULARITY_libexecdir/singularity/helpers/checks/1-hello-world.sh" testing exec_check 0 LOW "python $SINGULARITY_libexecdir/singularity/helpers/checks/1-cache-content.py" default clean bootstrap exec_check 0 HIGH "python $SINGULARITY_libexecdir/singularity/helpers/checks/3-cve.py" security exec_check 0 LOW "python $SINGULARITY_libexecdir/singularity/helpers/checks/1-docker.py" docker ################################################################################## # Checks we want to add ################################################################################## # 1) history cleanup (no trace of credentials like "echo changemenow| sudo passwd --stdin root") # 2) remove any file specific to the machine and not required on the container # (same files as for cloning instances) e.g /etc/ssh/host, # cleanup /etc/hosts, /etc/# sysconfig/network-scripts/ifcfg-, # but not ifcfg-lo, /etc/udev/rules.d/persistent ) /var/log/{secure,messages*,...} # 3) yum clean all and the apt-get equivalent # 4) SElinux or ACLs transfer? tar --selinux --xattrs # 5) sparse file handling # 6) remove any non required users via userdel? # 7) maybe reuse|have a peak at cloud-init ? # 8) maybe http://libguestfs.org/virt-sysprep.1.html#operations would be a better start singularity-2.4.2/libexec/Makefile.am0000644000175000017500000000030413211621077016473 0ustar mehdimehdiSUBDIRS = bootstrap-scripts cli helpers handlers python scriptlibexecdir = $(libexecdir)/singularity dist_scriptlibexec_SCRIPTS = functions image-handler.sh MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/image-handler.sh0000644000175000017500000000405413211621077017476 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016-2017, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # # This script is designed to be sourced rather then executed, as a result we do # not load functions or basic sanity. if [ -z "${SINGULARITY_IMAGE:-}" ]; then message ERROR "SINGULARITY_IMAGE is undefined...\n" ABORT 255 fi if [ -z "${SINGULARITY_COMMAND:-}" ]; then message ERROR "SINGULARITY_COMMAND is undefined...\n" ABORT 255 fi case "$SINGULARITY_IMAGE" in instance://*) . "$SINGULARITY_libexecdir/singularity/handlers/image-instance.sh" ;; docker://*) . "$SINGULARITY_libexecdir/singularity/handlers/image-docker.sh" ;; http://*|https://*) . "$SINGULARITY_libexecdir/singularity/handlers/image-http.sh" ;; shub://*) . "$SINGULARITY_libexecdir/singularity/handlers/image-shub.sh" ;; *.cpioz|*.vnfs|*.cpio) . "$SINGULARITY_libexecdir/singularity/handlers/archive-cpio.sh" ;; *.tar|*.tgz|*.tar.gz|*.tbz|*.tar.bz) . "$SINGULARITY_libexecdir/singularity/handlers/archive-tar.sh" ;; esac singularity-2.4.2/libexec/python/0000755000175000017500000000000013211621077015763 5ustar mehdimehdisingularity-2.4.2/libexec/python/shub/0000755000175000017500000000000013211621077016724 5ustar mehdimehdisingularity-2.4.2/libexec/python/shub/api.py0000644000175000017500000002361413211621077020055 0ustar mehdimehdi#!/usr/bin/env python ''' api.py: Singularity Hub helper functions for python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir))) # noqa sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # noqa from shell import ( parse_image_uri, remove_image_uri, ) from sutils import ( add_http, clean_up, get_image_format, read_file, run_command ) from base import ApiConnection from helpers.json.main import ADD from defaults import ( SHUB_API_BASE ) from message import bot import json import re try: from urllib import unquote except Exception: from urllib.parse import unquote # Shub API Class ---------------------------------------------- class SingularityApiConnection(ApiConnection): def __init__(self, **kwargs): self.token = None self.api_base = SHUB_API_BASE if 'image' in kwargs: self.load_image(kwargs['image']) if 'token' in kwargs: self.token = kwargs['token'] super(SingularityApiConnection, self).__init__(**kwargs) def load_image(self, image): self.image = parse_image_uri(image=image, uri='shub://', default_registry=SHUB_API_BASE, quiet=True) # parse_image_uri may return an empty namespace cause that's allowed # with docker://, but not with shub:// if len(self.image["namespace"]) == 0: bot.error("Namespace cannot be empty for shub:// url!") sys.exit(1) def get_manifest(self): '''get_image will return a json object with image metadata based on a unique id. Parameters ========== :param image: the image name, either an id or a repo name, tag, etc. Returns ======= manifest: a json manifest from the registry ''' # make sure we have a complete url registry = add_http(self.image['registry']) base = "%s/api/container/%s/%s:%s" % (registry, self.image['namespace'], self.image['repo_name'], self.image['repo_tag']) # ------------------------------------------------------ # If we need to authenticate, will do it here # ------------------------------------------------------ # If the Hub returns 404, the image name is likely wrong response = self.get(base, return_response=True) if response.code == 404: msg = "Cannot find image." msg += " Is your capitalization correct?" bot.error(msg) sys.exit(1) try: response = response.read().decode('utf-8') response = json.loads(response) except Exception: print("Error getting image manifest using url %s" % base) sys.exit(1) return response def download_image(self, manifest, download_folder=None, extract=False): ''' download_image will download a singularity image from singularity hub to a download_folder, named based on the image version (commit id) Parameters ========== :param manifest: the manifest obtained with get_manifest :param download_folder: the folder to download to, if None, will be pwd :param extract: if True, will extract image to .img and return that. Returns ======= image_file: the full path to the downloaded image ''' from defaults import SHUB_CONTAINERNAME # Returns just basename, no extension image_name = get_image_name(manifest) if not bot.is_quiet(): print("Found image %s:%s" % (manifest['name'], manifest['branch'])) print("Downloading image... %s" % image_name) url = manifest['image'] if url is None: bot.error("%s is not ready for download" % image_name) bot.error("please try when build completes or specify tag.") sys.exit(1) if download_folder is not None: image_name = "%s/%s" % (download_folder, image_name) # Download image file atomically, streaming image_file = self.download_atomically(url=url, file_name=image_name, show_progress=True) # Compressed ext3 images need extraction image_type = get_image_format(image_file) extension = "simg" if image_type == "GZIP" or extract is True: extension = "img" if not image_file.endswith('.gz'): os.rename(image_file, "%s.gz" % image_file) image_file = "%s.gz" % image_file if not bot.is_quiet(): print("Decompressing %s" % image_file) output = run_command(['gzip', '-d', '-f', image_file]) image_file = image_file.replace('.gz', '') # Any error in extraction (return code not 0) will return None if output is None: bot.error('Error extracting image, cleaning up.') clean_up([image_file, "%s.gz" % image_file]) # If the user has provided a default name, be true to it if SHUB_CONTAINERNAME is not None: folder = os.path.dirname(image_file) image_name = "%s/%s" % (folder, SHUB_CONTAINERNAME) os.rename(image_file, image_name) image_file = image_name # Otherwise rename to have extension matching image type else: image_name = "%s.%s" % (image_file, extension) os.rename(image_file, image_name) image_file = image_name return image_file # Various Helpers ----------------------------------------------- def get_image_name(manifest): '''return the image name for a manifest. Does not return extension, and does not account for a custom name provided by the user. :param manifest: the image manifest with 'image' as key with download link ''' from defaults import (SHUB_CONTAINERNAME, SHUB_NAMEBYCOMMIT, SHUB_NAMEBYHASH) if SHUB_CONTAINERNAME is not None: return SHUB_CONTAINERNAME # Second preference goes to commit elif SHUB_NAMEBYCOMMIT is not None: if manifest.get("commit") is not None: return manifest['commit'] elif manifest['version'] is not None: return manifest['version'] elif SHUB_NAMEBYHASH is not None: image_url = os.path.basename(unquote(manifest['image'])) image_name = re.findall('([-\w]+\.(?:img|simg|img.gz))', image_url) if len(image_name) > 0: return image_name[0].split('.')[0] return get_default_name(manifest) def get_default_name(manifest, source="Hub"): ''' get manifest name returns the default name that is discovered via the image manifest. This is the fallback option in the case that the user doesn't ask to name by commit, hash, or custom name ''' # Singularity Hub v2.0 if "tag" in manifest and "branch" in manifest: version = "%s-%s" % (manifest["branch"], manifest["tag"]) # Singularity Registry elif "tag" in manifest: version = manifest["tag"] # Singularity Hub v1.0 else: version = manifest['branch'] # Remove slashes version = version.replace('/', '-') # sregistry images store collection/name separately name = manifest['name'] if 'frozen' in manifest: source = "Registry" name = '%s-%s' % (manifest['collection'], name) image_name = "%s-%s" % (name.replace('/', '-'), version) if not bot.is_quiet(): print("Singularity %s Image: %s" % (source, image_name)) return image_name def extract_metadata(manifest, labelfile=None, prefix=None): '''extract_metadata will write a file of metadata from shub :param manifest: the manifest to use ''' if prefix is None: prefix = "" prefix = prefix.upper() source = 'Hub' if 'frozen' in manifest: source = 'Registry' metadata = manifest.copy() remove_fields = ['files', 'spec', 'metrics'] for remove_field in remove_fields: if remove_field in metadata: del metadata[remove_field] if labelfile is not None: for key, value in metadata.items(): key = "%s%s" % (prefix, key) value = ADD(key=key, value=value, jsonfile=labelfile, force=True) bot.verbose("Saving Singularity %s metadata to %s" % (source, labelfile)) return metadata singularity-2.4.2/libexec/python/shub/__init__.py0000644000175000017500000000000013211621077021023 0ustar mehdimehdisingularity-2.4.2/libexec/python/shub/main.py0000644000175000017500000001060213211621077020221 0ustar mehdimehdi''' main.py: Singularity Hub helper functions for Singularity in Python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # noqa from shell import parse_image_uri from .api import ( extract_metadata, get_image_name, SingularityApiConnection ) from sutils import ( get_cache, write_file ) from defaults import SHUB_PREFIX from message import bot import json import re import os def SIZE(image, contentfile=None): '''size is intended to be run before an import, to return to the contentfile a list of sizes (one per layer) corresponding with the layers that will be downloaded for image ''' bot.debug("Starting SHub SIZE, will get size from manifest") bot.debug("Singularity Hub image: %s" % image) client = SingularityApiConnection(image=image) manifest = client.get_manifest() if 'size_mb' in manifest: # sregistry size = manifest['size_mb'] else: size = manifest['metrics']['size'] if contentfile is not None: write_file(contentfile, str(size), mode="w") return size def PULL(image, download_folder=None, layerfile=None): '''PULL will retrieve a Singularity Hub image and download to the local file system, to the variable specified by SINGULARITY_PULLFOLDER. :param image: the singularity hub image name :param download folder: the folder to pull the image to. :param layerfile: if defined, write pulled image to file ''' client = SingularityApiConnection(image=image) manifest = client.get_manifest() if download_folder is None: cache_base = get_cache(subfolder="shub") else: cache_base = os.path.abspath(download_folder) bot.debug("Pull folder set to %s" % cache_base) # The image name is the md5 hash, download if it's not there image_name = get_image_name(manifest) # Did the user specify an absolute path? custom_folder = os.path.dirname(image_name) if custom_folder not in [None, ""]: cache_base = custom_folder image_name = os.path.basename(image_name) image_file = "%s/%s" % (cache_base, image_name) bot.debug('Pulling to %s' % image_file) if not os.path.exists(image_file): image_file = client.download_image(manifest=manifest, download_folder=cache_base) else: if not bot.is_quiet(): # not --quiet print("Image already exists at %s, skipping download" % image_file) if not bot.is_quiet(): # not --quiet print("Singularity Hub Image Download: %s" % image_file) manifest = {'image_file': image_file, 'manifest': manifest, 'cache_base': cache_base, 'image': image} if layerfile is not None: bot.debug("Writing Singularity Hub image path to %s" % layerfile) write_file(layerfile, image_file, mode="w") return manifest def IMPORT(image, layerfile, labelfile=None): '''IMPORT takes one more step than ADD, returning the image written to a layerfile plus metadata written to the metadata base in rootfs. :param image: the singularity hub image name ''' manifest = PULL(image, layerfile=layerfile) # Write metadata to base manifest['metadata'] = extract_metadata(manifest=manifest['manifest'], labelfile=labelfile, prefix="%s_" % SHUB_PREFIX) return manifest singularity-2.4.2/libexec/python/shub/Makefile.am0000644000175000017500000000022313211621077020755 0ustar mehdimehdiscriptlibexecdir = $(libexecdir)/singularity/python/shub dist_scriptlibexec_DATA = api.py main.py __init__.py MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/python/message.py0000644000175000017500000002211013211621077017755 0ustar mehdimehdi# -*- coding: utf-8 -*- ''' message.py: simple logger for Singularity python helper The error names (prefix) and integer level are assigned as follows: ABRT -4 ERROR -3 WARNING -2 LOG -1 INFO 1 QUIET 0 VERBOSE 2 VERBOSE1 2 VERBOSE2 3 VERBOSE3 4 DEBUG 5 VERBOSE is equivalent to VERBOSE1 (this is mirroring the C code) and for each level, calling it corresponds to calling the class' function for it. E.g., DEBUG --> bot.debug('This is the message!') The following levels are to stderr: 5,4,3,2,1,-1,-2,-3,-4 The following levels are only to stdout 1 The following levels do nothing (quiet) 0 Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import sys ABRT = -4 ERROR = -3 WARNING = -2 LOG = -1 INFO = 1 QUIET = 0 VERBOSE = VERBOSE1 = 2 VERBOSE2 = 3 VERBOSE3 = 4 DEBUG = 5 class SingularityMessage: def __init__(self, MESSAGELEVEL=None): self.level = get_logging_level() self.history = [] self.errorStream = sys.stderr self.outputStream = sys.stdout self.colorize = self.useColor() self.colors = {ABRT: "\033[31m", # dark red ERROR: "\033[91m", # red WARNING: "\033[93m", # dark yellow LOG: "\033[95m", # purple DEBUG: "\033[36m", # cyan 'OFF': "\033[0m"} # end sequence # Colors -------------------------------------------- def useColor(self): '''useColor will determine if color should be added to a print. Will check if being run in a terminal, and if has support for asci''' COLORIZE = get_user_color_preference() if COLORIZE is not None: return COLORIZE streams = [self.errorStream, self.outputStream] for stream in streams: if not hasattr(stream, 'isatty'): return False if not stream.isatty(): return False return True def addColor(self, level, text): '''addColor to the prompt (usually prefix) if terminal supports, and specified to do so''' if self.colorize: if level in self.colors: text = "%s%s%s" % (self.colors[level], text, self.colors["OFF"]) return text def emitError(self, level): '''determine if a level should print to stderr, includes all levels but INFO and QUIET''' if level in [ABRT, ERROR, WARNING, LOG, VERBOSE, VERBOSE1, VERBOSE2, VERBOSE3, DEBUG]: return True return False def emitOutput(self, level): '''determine if a level should print to stdout only includes INFO''' if level in [INFO]: return True return False def isEnabledFor(self, messageLevel): '''check if a messageLevel is enabled to emit a level ''' if messageLevel <= self.level: return True return False def emit(self, level, message, prefix=None): '''emit is the main function to print the message optionally with a prefix :param level: the level of the message :param message: the message to print :param prefix: a prefix for the message ''' if prefix is not None: prefix = self.addColor(level, "%s " % (prefix)) else: prefix = "" message = self.addColor(level, message) # Add the prefix message = "%s%s" % (prefix, message) if not message.endswith('\n'): message = "%s\n" % (message) # If the level is quiet, only print to error if self.level == QUIET: pass # Otherwise if in range print to stdout and stderr elif self.isEnabledFor(level): if self.emitError(level): self.write(self.errorStream, message) else: self.write(self.outputStream, message) # Add all log messages to history self.history.append(message) def write(self, stream, message): '''write will write a message to a stream, first checking the encoding ''' if isinstance(message, bytes): message = message.decode('utf-8') stream.write(message) def get_logs(self, join_newline=True): ''''get_logs will return the complete history, joined by newline (default) or as is. ''' if join_newline: return '\n'.join(self.history) return self.history def show_progress(self, iteration, total, length=40, min_level=0, prefix=None, carriage_return=True, suffix=None, symbol=None): '''create a terminal progress bar, default bar shows for verbose+ :param iteration: current iteration (Int) :param total: total iterations (Int) :param length: character length of bar (Int) ''' perc = 100 * (iteration / float(total)) progress = int(length * iteration // total) if suffix is None: suffix = '' if prefix is None: prefix = 'Progress' # Download sizes can be imperfect, setting carriage_return to False # and writing newline with caller cleans up the UI if perc >= 100: perc = 100 progress = length if symbol is None: symbol = "=" if progress < length: bar = symbol * progress + '|' + '-' * (length - progress - 1) else: bar = symbol * progress + '-' * (length - progress) # Only show progress bar for level > min_level if self.level > min_level: perc = "%5s" % ("{0:.1f}").format(perc) output = '\r' + prefix + " |%s| %s%s %s" % (bar, perc, '%', suffix) sys.stdout.write(output), if iteration == total and carriage_return: sys.stdout.write('\n') sys.stdout.flush() def abort(self, message): self.emit(ABRT, message, 'ABRT') def error(self, message): self.emit(ERROR, message, 'ERROR') def warning(self, message): self.emit(WARNING, message, 'WARNING') def log(self, message): self.emit(LOG, message, 'LOG') def info(self, message): self.emit(INFO, message) def verbose(self, message): self.emit(VERBOSE, message, "VERBOSE") def verbose1(self, message): self.emit(VERBOSE, message, "VERBOSE1") def verbose2(self, message): self.emit(VERBOSE2, message, 'VERBOSE2') def verbose3(self, message): self.emit(VERBOSE3, message, 'VERBOSE3') def debug(self, message): self.emit(DEBUG, message, 'DEBUG') def is_quiet(self): '''is_quiet returns true if the level is under 1 ''' if self.level < 1: return False return True def get_logging_level(): '''get_logging_level will configure a logging to standard out based on the user's selected level, which should be in an environment variable called SINGULARITY_MESSAGELEVEL. if SINGULARITY_MESSAGELEVEL is not set, the maximum level (5) is assumed (all messages). #define ABRT -4 #define ERROR -3 #define WARNING -2 #define LOG -1 #define INFO 1 implied define: QUIET 0 #define VERBOSE 2 #define VERBOSE1 2 #define VERBOSE2 3 #define VERBOSE3 4 #define DEBUG 5 ''' return int(os.environ.get("SINGULARITY_MESSAGELEVEL", 5)) def get_user_color_preference(): COLORIZE = os.environ.get('SINGULARITY_COLORIZE', None) if COLORIZE is not None: COLORIZE = convert2boolean(COLORIZE) return COLORIZE def convert2boolean(arg): '''convert2boolean is used for environmental variables that must be returned as boolean''' if not isinstance(arg, bool): return arg.lower() in ("yes", "true", "t", "1", "y") return arg bot = SingularityMessage() singularity-2.4.2/libexec/python/README.md0000644000175000017500000005503513211621077017252 0ustar mehdimehdi# Python Developer Documentation This document explains how to use the Python functions from any calling Singularity (C) process. For the first version we used a client [cli.py](https://github.com/singularityware/singularity/blob/6433ddd67b6abfdf1faa3eaa7b0338aa8fe55b31/libexec/python/cli.py), with optparse, and this strategy made sense given that the python module had a limited functionality to import Docker layers. When the need arose to support more than one uri, and different optional imports, it was decided that a more modular strategy was needed, and one that mirrored how the core Singularity software worked. Specifically: - variables are passed around via the environment - commands are simple and modular, named intuitively In the current version, this old client was removed in favor of a modular, environment variable-based approach. The following sections are about the python api endpoints available, including bootstrap modules, and general utilities. For each, the following standards are used for parsing environmental variables (the source of the inputs for the functions): # Table of Contents 1. [Environment Varialbes](#environment-variables) 2. [Cache](#cache) 3. [Bootstrap API](#bootstrap-modules) 4. [Util Functions API](#utility-modules) 5. [Future Additions](#future-additions) ## Environment Variables All environmental variables are parsed in [defaults.py](defaults.py), which is a gateway between variables defined at runtime, and defaults. By way of import from the file, variables set at runtime do not change if re-imported. This was done intentionally to prevent changes during the execution, and could be changed if needed. For all variables, the order of operations works as follows: 1. First preference goes to environment variable set at runtime 2. Second preference goes to default defined in this file 3. Then, if neither is found, null is returned except in the case that `required=True`. A `required=True` variable not found will system exit with an error. 4. Variables that should not be dispayed in debug logger are set with `silent=True`, and are only reported to be defined. For boolean variables, the following are acceptable for True, with any kind of capitalization or not: ("yes", "true", "t", "1","y") ## Cache The location and usage of the cache is also determined by environment variables. **SINGULARITY_DISABLE_CACHE** If the user wants to disable the cache, all this means is that the layers are written to a temporary directory. The python functions do nothing to actually remove images, as they are needed by the calling process. It should be responsibility of the calling process to remove layers given that `SINGULARITY_DISABLE_CACHE` is set to any true/yes value. By default, the cache is not disabled. **SINGULARITY_CACHE** Is the base folder for caching layers and singularity hub images. If not defined, it uses default of `$HOME/.singularity`, and subfolders for docker layers are `$HOME` If defined, the defined location is used instead. If `DISABLE_CACHE` is set to True, this value is ignored in favor of a temporary directory. For specific subtypes of things to cache, subdirectories are created (by python), including `$SINGULARITY_CACHE/docker` for docker layers and `$SINGULARITY_CACHE/shub` for Singularity Hub images. If the cache is not created, the Python script creates it. **SINGULARITY_CONTENT** The layerfile is important for both docker ADD and IMPORT, as it is the file where .tar.gz layer files are written for the calling process to extract. If `SINGULARITY_CONTENT` is not defined, then it will be generated as `$SINGULARITY_METADATA_BASE/.layers`. **SINGULARITY_ENVBASE** The environment base folder is the folder name within the metadata folder to hold environment variable files to be sourced. If not defined, it defaults to `$SINGULARITY_METADATA_BASE/.env`, and python carries it around in the variable `ENV_BASE`. **SINGULARITY_LABELBASE** The label base is akin to the `ENV_BASE`, except it is for labels from the docker image. If not defined, it defaults to `$SINGULARITY_METADATA_BASE/labels` **SINGULARITY_PULLFOLDER** By default, images are pulled to the present working directory. The user can change this variable to change that. Currently, the "pull" command is only relevant for Singularity Hub. ## Bootstrap Modules A boostrap module is a set of functions that allow importing contents, metadata, and environment variables from other containers. In the current version, the old client was removed, and each module (currently we have support for [shub](shub) and [docker](docker) has a set of functions not exposed for command line use (typically in a `main.py` and `api.py` module, and then a main client (an executable script) that is meant to be called from the command line. For example: - `libexec/python/import.py` - `libexec/python/pull.py` For each of the above, the python takes car of parsing the uri, meaning that a uri of `docker://` or `shub://` can be passed to `python/import.py` and it will be directed to the correct module to handle it. This basic structure is meant to put more responsibility on python for parsing and handling uris, with possibility of easily adding other endpoints in the future. This means that the Singularity (calling process), given that required environmental variables are defined, can call a function like: eval $SINGULARITY_libexecdir/singularity/python/import.py and the environment might hold and image uri for `docker://` or `shub://`. For each, the details of required arguments are detailed in the scripts, and discussed below. First, we will review the environmental variables. ### Defaults The following variables in [defaults.py](defaults.py) are a combination of static values, and variables that can be customized by the user via environment variables at runtime. #### Docker **DOCKER_API_BASE** Set as `index.docker.io`, which is the name of the registry. In the first version of Singularity we parsed the Registry argument from the build spec file, however now this is removed because it can be obtained directly from the image name (eg, `registry/namespace/repo:tag`). If you don't specify a registry name for your image, this default is used. **DOCKER_API_VERSION** Is the version of the Docker Registry API currently being used, by default now is `v2`. **DOCKER_OS** This is exposed via the exported environment variable `SINGULARITY_DOCKER_OS` and pertains to images that reveal a version 2 manifest with a [manifest list](https://docs.docker.com/registry/spec/manifest-v2-2/#manifest-list). In the case that the list is present, we must choose an operating system (this variable) and an architecture (below). The default is `linux`. **DOCKER_ARCHITECTURE** This is exposed via the exported environment variable `SINGULARITY_DOCKER_ARCHITECTURE` and the same applies as for the `DOCKER_OS` with regards to being used in context of a list of manifests. In the case that the list is present, we must choose an architecture (this variable) and an os (above). The default is `amd64`, and other common ones include `arm`, `arm64`, `ppc64le`, `386`, and `s390x`. **DOCKER_PREFIX** Whenever a new Docker container is imported, it brings its environment. This means that we must write the environmental variables to a file where they can be preserved. To keep a record of Docker imports, we generate a file starting with `DOCKER_PREFIX` in the environment metadata folder (see environment variable `ENV_BASE`) (default is `docker`). **DOCKER_NUMBER** To support multiple imports, we must number this file (eg, `10-docker.sh`). The `DOCKER_NUMBER` is the starting count for this file, with default `10` to allow more important environment variables to come first. A note of caution to the calling script - this would mean we source them in reverse, otherwise higher numbers (which should be lower priority) overwrite. We probably should import in reverse always, but maintain 1..N as priority ordering so it is intuitive. **NAMESPACE** Is the default namespace, `library`. **RUNSCRIPT_COMMAND** Is not obtained from the environment, but is a hard coded default (`"/bin/bash"`). This is the fallback command used in the case that the docker image does not have a `CMD` or `ENTRYPOINT`. **TAG** Is the default tag, `latest`. **DISABLE_HTTPS** If you export the variable `SINGULARITY_NOHTTPS` you can force the software to not use https when interacting with a Docker registry. This use case is typically for use of a local registry. #### Singularity Hub **SHUB_PREFIX** Singularity images are imported in entirety (meaning no environmental data to parse) so we only need the prefix to write metadata for. **SHUB_API_BASE** The default base for the Singularity Hub API, which is `https://singularity-hub.org/api` **SHUB_CONTAINERNAME** The user is empowered to define a custom name for the singularity image downloaded. The first preference goes to specifying an `SHUB_CONTAINERNAME`. For example: ```bash export SHUB_CONTAINERNAME="meatballs.img" singularity pull shub://vsoch/singularity-images ... Done. Container is at: ./meatballs.img ``` **SHUB_NAMEBYCOMMIT** Second preference goes to naming the container by commit. If this variable is found in the environment, regardless of the value, it will be done! ```bash unset SHUB_CONTAINERNAME export SHUB_NAMEBYCOMMIT=yesplease singularity pull shub://vsoch/singularity-images Done. Container is at: ./7a75cd7a32192e5d50f267982e0c30aff794076b.img ``` **SHUB_NAMEBYHASH** Finally, we can name the container based on the file hash. ```bash unset SHUB_NAMEBYCOMMIT export SHUB_NAMEBYHASH=yesplease singularity pull shub://vsoch/singularity-images Done. Container is at: ./a989bc72cb154d007aa47a5034978328.img ``` If none of the above are selected, the default is to use the username and reponame ```bash unset SHUB_NAMEBYHASH singularity pull shub://vsoch/singularity-images Done. Container is at: ./vsoch-singularity-images-mongo.img ``` ### Formatting Formatting refers to the user interface for the command line tool. **SINGULARITY_COLORIZE** By default, debug messages (and other types) use ascii escape sequences to various commands with colors. This helps to distinguish them, and makes the user interface a bit more pleasant. If the output is not going to a terminal, or if the terminal does not support the ascii escape sequences, this variable is set to False. The user can always override this by setting this variable. ### Plugins Singularity plugins are custom environment variables that can be set to turn bootstrap (and other building) customizations on and off. Currently, we just have one plugin that will, when turned on, have the Python API backend change permissions for the tarballs. **SINGULARITY_FIX_PERMS** If set to `True/true/1/yes`, the Python back end will parse through the tar files from Docker in memory, and fix permissions. This sets the variable `PLUGIN_FIXPERMS` in the script, and is by default False. ### General **SINGULARITY_PYTHREADS** The Python modules use threads (workers) to download layer files for Docker, and change permissions. By default, we will use 9 workers, unless the environment variable `SINGULARITY_PYTHREADS` is defined. **SINGULARITY_COMMAND_ASIS** By default, we want to make sure the container running process gets passed forward as the current process, so we want to prefix whatever the Docker command or entrypoint is with `exec`. We also want to make sure that following arguments get passed, so we append `"$@"`. Thus, some entrypoint or cmd might look like this: /usr/bin/python and we would parse it into the runscript as: exec /usr/bin/python "$@" However, it might be the case that the user does not want this. For this reason, we have the environmental variable `RUNSCRIPT_COMMAND_ASIS`. If defined as yes/y/1/True/true, etc., then the runscript will remain as `/usr/bin/python`. **SINGULARITY_METADATA_FOLDER** Goes into the variable `METADATA_BASE`, and is the directory location to write the metadata file structure. Specifically, this means folders for environmental variable and layers files. The default looks like this: `$SINGULARITY_ROOTFS` .singularity.d/ env/ labels.json **SINGULARITY_ROOTFS** This is the root file system location of the container. There are various checks in all calling functions so the script should never get to this point without having it defined. If the environmental variable `$SINGULARITY_METADATA_FOLDER` is defined, the metadata folder doesn't even need to live in the container. This could be useful if the calling API wants to skip over it's generation, however care should be taken given that the files are some kind of dependency to produce `/environment`. If the variable isn't defined, then the default metadata folder is set to be `$SINGULARITY_ROOTFS/.singularity.d`. The variable is required, an extra precaution, but probably not necessary since a default is provided. ### Example Usage #### Docker ##### Docker Import The Docker commands include `IMPORT`. Import means returning a layerfile with paths (separated by newlines) to a complete list of layers for import, one of those layers being a tarfile with Docker runscript, environment, and labels. For all Docker commands, by way of needing to use the Docker Registry, the user can optionally specifying a username and password for authentication: export SINGULARITY_DOCKER_USERNAME='mickeymouse' export SINGULARITY_DOCKER_PASSWORD='cheeseftw' The user and Singularity calling functions also have some control over the cache. Specifically: - `SINGULARITY_DISABLE_CACHE`: will write layers to a temporary directory. Note that since the functions aren't actually doing the import, they do not remove the layers (as was done in previous versions) - `SINGULARITY_CACHE`: Is a specific path to the cache. Since we now are also (potentially) parsing a runscript, the user has the choice to use `CMD` instead of `ENTRYPOINT` by way of the variable `SINGULARITY_INCLUDECMD` parsed from `Cmd` in the build spec file, and `SINGULARITY_COMMAND_ASIS` to not include `exec` and `$@`. As with ADD, the user can again specify a `SINGULARITY_DOCKER_USERNAME` and `SINGULARITY_DOCKER_PASSWORD` if authentication is needed. The required environment exports are: - `SINGULARITY_CONTAINER`: (eg, docker://ubuntu:latest) - `SINGULARITY_CONTENTS`: the file to write the list of layers to. The python function previously extracted the layers, but now to support user import without sudo, and consistency across import/shell/bootstrap, the calling function takes care of this. Thus, it is also important that the calling function write metadata to the `labels.json` and any of the user's preferences for the `runscript` after the layers are extracted, in the case that the user wants to overwrite something that came from the Docker dump. An example use case is the following: #!/bin/bash # This is an example of the base usage for the docker/add.py command # We need, minimally, a docker container and rootfs defined export SINGULARITY_CONTAINER="docker://ubuntu:latest" export SINGULARITY_CONTENTS=/tmp/hello-kitty.txt cd libexec/python/tests python ../import.py After the script runs, the file `/tmp/hello-kitty.txt` will contain the list of layers to import. Something like: /home/vanessa/.singularity/docker/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4.tar.gz /home/vanessa/.singularity/docker/sha256:2508cbcde94b50cd53356e8730bb508ddb43c76664a35dc29e69bb8b56a0f531.tar.gz /home/vanessa/.singularity/docker/sha256:bc7277e579f03a13476b4d2dc6607124f7e67341dbd58f9d1cd6555bec086c04.tar.gz /home/vanessa/.singularity/docker/sha256:946d6c48c2a7d60cb2f4d1c4d3a8131086b412d11a9def59d0bcc0892428dde9.tar.gz /home/vanessa/.singularity/docker/sha256:695f074e24e392178d364af5ea2405dda7ab0035284001b49939afac5106c187.tar.gz /home/vanessa/.singularity/docker/sha256:8aec416115fdbd74102c9090bcfe03bfe8926876642d8846c8b917959ea9b552.tar.gz The last of the layers will be the tarfile created by the python with the metadata. #### Singularity Hub The Singularity Hub python functions include IMPORT, and PULL. - PULL: is the most basic of the three, pulling an image from Singularity Hub to the cache (default) or if defined, the `SINGULARITY_PULLFOLDER`. It names the image with the format `username-repo-tag.img.gz`, where the tag is optional. The path to the image is returned to the calling process via the `SINGULARITY_LAYERFILE` environmental variable. - IMPORT: is the most robust, doing the same as PULL, but additionally extracting metadata about the image to the `SINGULARITY_LABELDIR` folder. Examples are included below. ##### PULL Pull must minimally have a container defined in `SINGULARITY_CONTAINER` #!/bin/bash cd libexec/python/tests # We need, minimally, a singularity hub container, default pulls to cache export SINGULARITY_CONTAINER="shub://vsoch/singularity-images" python ../pull.py # If we specify a different folder, we will specify it export SINGULARITY_HUB_PULL_FOLDER=$PWD python ../pull.py ## Utility Modules Included in bootstrap, but not specifically for it, we have a set of utility modules, which do things like: - get, add, delete a key from a json file specified - get the size of a container ### Size The size function will return the size of a container. Required environment variables are: - `SINGULARITY_CONTAINER`: the path (and uri) to the container (shub:// or docker://) - `SINGULARITY_CONTENTS` = the layerfile to write the size to Example usage is as follows: # Singularity Hub export SINGULARITY_CONTAINER=shub://vsoch/singularity-hello-world export SINGULARITY_CONTENTS=/tmp/hello-kitty.txt # from within tests, for example python ../size.py # Docker export SINGULARITY_CONTAINER=docker://ubuntu:latest export SINGULARITY_CONTENTS=/tmp/hello-kitty.txt python ../size.py The size is obtained via reading the (version 2.0) manifest size of each layer, and adding them together. For example, I could use the API internally in Python as follows: ``` cd libexec/python ipython from docker.api import DockerApiConnection client=DockerApiConnection(image='ubuntu') DEBUG Headers found: Accept,Content-Type VERBOSE Registry: index.docker.io VERBOSE Namespace: library VERBOSE Repo Name: ubuntu VERBOSE Repo Tag: latest VERBOSE Version: None VERBOSE Obtaining tags: https://index.docker.io/v2/library/ubuntu/tags/list DEBUG GET https://index.docker.io/v2/library/ubuntu/tags/list DEBUG Http Error with code 401 DEBUG GET https://auth.docker.io/token?service=registry.docker.io&expires_in=9000&scope=repository:library/ubuntu:pull DEBUG Headers found: Accept,Authorization,Content-Type client.get_size() VERBOSE Obtaining manifest: https://index.docker.io/v2/library/ubuntu/manifests/latest DEBUG GET https://index.docker.io/v2/library/ubuntu/manifests/latest Out[3]: 46795242 ``` Given that the container does not have a version 2.0 manifest (not sure if this is possible, but it could be) or if the manifest is malformed in any way, a size of None is returned. ### Json The json module serves two functions. First, it writes a key value store of labels specific to a container using supporting functions [add](helpers/json/add.py), [helpers/json/delete.py](helpers/json/delete.py) and [helpers/json/get.py](helpers/json/get.py). Second, it servers to return a [JSON API](http://jsonapi.org/) manifest with one or more metrics of interest. #### Inspect Inspect will return (print to the screen, or stdout) a json data structure with the fields asked for by the user, specifically which can be in the set defined in [inspect.help](../cli/inspect.help). ``` -l/--labels Show the labels associated with the image (default) -d/--deffile Show the bootstrap definition file which was used to generate this image -r/--runscript Show the runscript for this image -t/--test Show the test script for this image -e/--environment Show the environment settings for this container ``` The bash client handles parsing these command line arguments into the following environment variables that are exported, and found by python: ``` SINGULARITY_MOUNTPOINT SINGULARITY_INSPECT_LABELS SINGULARITY_INSPECT_DEFFILE SINGULARITY_INSPECT_RUNSCRIPT SINGULARITY_INSPECT_TEST SINGULARITY_INSPECT_ENVIRONMENT ``` Essentially, if any of the above for `SINGULARITY_INSPECT_*` are found to be defined, this is interpreted as "yes/True." Otherwise, if the environment variable is not defined, python finds it as `None` and doesn't do anything. In this manner, we can call the executable as follows (e.g., from [inspect.sh](../helpers/inspect.sh)). ``` eval_abort "$SINGULARITY_libexecdir/singularity/python/helpers/json/inspect.py" ``` and the python will look for the environmental variables specified above. This is different from the other json functions (below) that expect different parameters to be passed into the script call. #### Json Functions The functions `get`, `dump`, and `add` are primarily used to write and read `.json` files in the `SINGULARITY_METADATA/labels` folder, with each file mapping to it's source (eg, docker, shub, etc). Given that the calling (C) function has specified the label file (`SINGULARITY_LABELBASE`) The general use would be the following: # Add a key value to labelfile. The key must not exist exec $SINGULARITY_libexecdir/singularity/python/utils/json/add.py --key $KEY --value $VALUE --file $LABELFILE # If it exists, you can force add exec $SINGULARITY_libexecdir/singularity/python/utils/json/add.py --key $KEY --value $VALUE --file $LABELFILE -f # Remove a key from labelfile. If the file is empty after, it is removed exec $SINGULARITY_libexecdir/singularity/python/utils/json/delete.py --key $KEY --file $LABELFILE # Get a stream / list of all labels (in single file, one per line, separated by :) exec $SINGULARITY_libexecdir/singularity/python/utils/json/dump.py --file $LABELFILE # Get a single key from label file, returns empty and exists if not defined exec $SINGULARITY_libexecdir/singularity/python/utils/json/get.py --key $KEY ## Future Additions #### Python Internal API URIs The internal python modules, in the case of returning a `SINGULARITY_CONTENTS` file with a list of contents to be parsed by the calling function, will prefix each content (line in the file) with a uri to tell the calling script how to manage it. Currently, we have the following defined: - URI_IMAGE: img:// - intended for Singularity Hub images, which are downloaded as .img.gz, but returned after decompression. - URI_TAR: tar:// - for a tar (not compressed) - URI_TARGZ: targz:// - for a compressed tarball For each of the above, a path would add an extra slash (e.g. `img:///home/vanessa/image.img.gz`) singularity-2.4.2/libexec/python/defaults.py0000644000175000017500000001436613211621077020156 0ustar mehdimehdi''' defaults.py: this script acts as a gateway between variables defined at runtime, and defaults. Any variable that has an unchanging default value can be found here. The order of operations works as follows: 1. First preference goes to environment variable set at runtime 2. Second preference goes to default defined in this file 3. Then, if neither is found, null is returned except in the case that required = True. A required = True variable not found will system exit with an error. Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' from message import bot import tempfile import os import pwd import sys def getenv(variable_key, default=None, required=False, silent=False): '''getenv will attempt to get an environment variable. If the variable is not found, None is returned. :param variable_key: the variable name :param required: exit with error if not found :param silent: Do not print debugging information for variable ''' variable = os.environ.get(variable_key, default) if variable is None and required: bot.error("Cannot find environment variable %s, exiting." % (variable_key)) sys.exit(1) if silent: if variable is not None: bot.verbose2("%s found" % (variable_key)) else: if variable is not None: bot.verbose2("%s found as %s" % (variable_key, variable)) else: bot.verbose2("%s not defined (None)" % (variable_key)) return variable def convert2boolean(arg): '''convert2boolean is used for environmental variables that must be returned as boolean''' if not isinstance(arg, bool): return arg.lower() in ("yes", "true", "t", "1", "y") return arg ####################################################################### # Singularity ####################################################################### # Filled in to exec %s "$@" RUNSCRIPT_COMMAND_ASIS = convert2boolean(getenv("SINGULARITY_COMMAND_ASIS", default=False)) SINGULARITY_ROOTFS = getenv("SINGULARITY_ROOTFS") METADATA_FOLDER_NAME = ".singularity.d" _metadata_base = "%s/%s" % (SINGULARITY_ROOTFS, METADATA_FOLDER_NAME) METADATA_BASE = getenv("SINGULARITY_METADATA_FOLDER", _metadata_base, required=True) ####################################################################### # Plugins and Formatting ####################################################################### PLUGIN_FIXPERMS = convert2boolean(getenv("SINGULARITY_FIX_PERMS", False)) COLORIZE = getenv("SINGULARITY_COLORIZE", None) if COLORIZE is not None: COLORIZE = convert2boolean(COLORIZE) ####################################################################### # Cache ####################################################################### DISABLE_CACHE = convert2boolean(getenv("SINGULARITY_DISABLE_CACHE", default=False)) if DISABLE_CACHE is True: SINGULARITY_CACHE = tempfile.mkdtemp() else: userhome = pwd.getpwuid(os.getuid())[5] _cache = os.path.join(userhome, ".singularity") SINGULARITY_CACHE = getenv("SINGULARITY_CACHEDIR", default=_cache) ####################################################################### # Docker ####################################################################### # API DOCKER_API_BASE = "index.docker.io" # registry CUSTOM_REGISTRY = getenv("REGISTRY") NAMESPACE = "library" CUSTOM_NAMESPACE = getenv('NAMESPACE') DOCKER_API_VERSION = "v2" DOCKER_ARCHITECTURE = getenv("SINGULARITY_DOCKER_ARCHITECTURE", "amd64") DOCKER_OS = getenv("SINGULARITY_DOCKER_OS", "linux") TAG = "latest" # Container Metadata DOCKER_NUMBER = 10 # number to start docker files at in ENV_DIR DOCKER_PREFIX = "docker" SHUB_PREFIX = "shub" # Defaults for environment, runscript, labels _envbase = "%s/env" % (METADATA_BASE) _runscript = "%s/singularity" % (SINGULARITY_ROOTFS) _environment = "%s/90-environment.sh" % (_envbase) _labelfile = "%s/labels.json" % (METADATA_BASE) _helpfile = "%s/runscript.help" % (METADATA_BASE) _deffile = "%s/Singularity" % (METADATA_BASE) _testfile = "%s/test" % (METADATA_BASE) ENVIRONMENT = getenv("SINGULARITY_ENVIRONMENT", _environment) RUNSCRIPT = getenv("SINGULARITY_RUNSCRIPT", _runscript) TESTFILE = getenv("SINGULARITY_TESTFILE", _testfile) DEFFILE = getenv("SINGULARITY_DEFFILE", _deffile) HELPFILE = getenv("SINGULARITY_HELPFILE", _helpfile) ENV_BASE = getenv("SINGULARITY_ENVBASE", _envbase) LABELFILE = getenv("SINGULARITY_LABELFILE", _labelfile) INCLUDE_CMD = convert2boolean(getenv("SINGULARITY_INCLUDECMD", False)) DISABLE_HTTPS = convert2boolean(getenv("SINGULARITY_NOHTTPS", False)) ####################################################################### # Singularity Hub ####################################################################### SINGULARITY_PULLFOLDER = getenv("SINGULARITY_PULLFOLDER", os.getcwd()) SHUB_API_BASE = "www.singularity-hub.org" SHUB_NAMEBYHASH = getenv("SHUB_NAMEBYHASH") SHUB_NAMEBYCOMMIT = getenv("SHUB_NAMEBYCOMMIT") SHUB_CONTAINERNAME = getenv("SHUB_CONTAINERNAME") ####################################################################### # Python Internal API URI Handling ####################################################################### _layerfile = "%s/.layers" % (METADATA_BASE) LAYERFILE = getenv("SINGULARITY_CONTENTS", _layerfile) SINGULARITY_WORKERS = int(getenv("SINGULARITY_PYTHREADS", 9)) singularity-2.4.2/libexec/python/__init__.py0000644000175000017500000000000013211621077020062 0ustar mehdimehdisingularity-2.4.2/libexec/python/sutils.py0000644000175000017500000004225513211621077017670 0ustar mehdimehdi# -*- coding: utf-8 -*- ''' utils.py: python helper for singularity command line tool Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' from defaults import ( SINGULARITY_CACHE, DISABLE_HTTPS ) from message import bot import datetime import errno import hashlib import json import os import subprocess import stat import sys import tempfile import tarfile import base64 import re from io import ( BytesIO, StringIO ) # Python less than version 3 must import OSError if sys.version_info[0] < 3: from exceptions import OSError #################################################################### # HTTP OPERATIONS ################################################## #################################################################### def add_http(url, use_https=True): '''add_http will add a http / https prefix to a url, in case the user didn't specify :param url: the url to add the prefix to :param use_https: should we default to https? default is True ''' scheme = "https://" if use_https is False or DISABLE_HTTPS is True: scheme = "http://" # remove scheme from url # urlparse is buggy in Python 2.6 # https://bugs.python.org/issue754016, use regex instead parsed = re.sub('.*//', '', url) return "%s%s" % (scheme, parsed.rstrip('/')) def basic_auth_header(username, password): '''basic_auth_header will return a base64 encoded header object to generate a token :param username: the username :param password: the password ''' s = "%s:%s" % (username, password) if sys.version_info[0] >= 3: s = bytes(s, 'utf-8') credentials = base64.b64encode(s).decode('utf-8') else: credentials = base64.b64encode(s) auth = {"Authorization": "Basic %s" % credentials} return auth #################################################################### # COMMAND LINE OPERATIONS ########################################## #################################################################### def run_command(cmd, env=None, quiet=False): '''run_command uses subprocess to send a command to the terminal. :param cmd: the command to send, should be a list for subprocess :param env: an optional environment to include, a dictionary of key/values ''' try: if quiet is False: bot.verbose2("Running command %s with subprocess" % " ".join(cmd)) if env is None: process = subprocess.Popen(cmd, stdout=subprocess.PIPE) else: process = subprocess.Popen(cmd, stdout=subprocess.PIPE, env=env) except OSError as error: bot.error("Error with subprocess: %s, returning None" % error) return None output = process.communicate()[0] if process.returncode != 0: return None return output def clean_up(files): '''clean up will delete a list of files, only if they exist ''' if not isinstance(files, list): files = [files] for f in files: if os.path.exists(f): bot.verbose3("Cleaning up %s" % f) os.remove(f) ############################################################################ # TAR/COMPRESSION ########################################################## ############################################################################ def extract_tar(archive, output_folder): '''extract a tar archive to a specified output folder :param archive: the archive file to extract :param output_folder: the output folder to extract to ''' # If extension is .tar.gz, use -xzf args = '-xf' if archive.endswith(".tar.gz"): args = '-xzf' # Just use command line, more succinct. command = ["tar", args, archive, "-C", output_folder, "--exclude=dev/*"] if not bot.is_quiet(): print("Extracting %s" % archive) return run_command(command) def create_tar(files, output_folder=None): '''create_memory_tar will take a list of files (each a dictionary with name, permission, and content) and write the tarfile (a sha256 sum name is used) to the output_folder. If there is no output folde specified, the tar is written to a temporary folder. ''' if output_folder is None: output_folder = tempfile.mkdtemp() finished_tar = None additions = [] contents = [] for entity in files: info = tarfile.TarInfo(name=entity['name']) info.mode = entity['mode'] info.mtime = int(datetime.datetime.now().strftime('%s')) info.uid = entity["uid"] info.gid = entity["gid"] info.uname = entity["uname"] info.gname = entity["gname"] # Get size from stringIO write filey = StringIO() content = None try: # python3 info.size = filey.write(entity['content']) content = BytesIO(entity['content'].encode('utf8')) except Exception: # python2 info.size = int(filey.write(entity['content'].decode('utf-8'))) content = BytesIO(entity['content'].encode('utf8')) pass if content is not None: addition = {'content': content, 'info': info} additions.append(addition) contents.append(content) # Now generate the sha256 name based on content if len(additions) > 0: hashy = get_content_hash(contents) finished_tar = "%s/sha256:%s.tar.gz" % (output_folder, hashy) # Warn the user if it already exists if os.path.exists(finished_tar): msg = "metadata file %s already exists " % finished_tar msg += "will over-write." bot.debug(msg) # Add all content objects to file tar = tarfile.open(finished_tar, "w:gz") for a in additions: tar.addfile(a["info"], a["content"]) tar.close() else: msg = "No contents, environment or labels" msg += " for tarfile, will not generate." bot.debug(msg) return finished_tar #################################################################### # HASHES/FORMAT #################################################### #################################################################### def get_content_hash(contents): '''get_content_hash will return a hash for a list of content (bytes/other) ''' hasher = hashlib.sha256() for content in contents: if isinstance(content, BytesIO): content = content.getvalue() if not isinstance(content, bytes): content = bytes(content) hasher.update(content) return hasher.hexdigest() def get_image_format(image_file): ''' get image format will use the image-format executable to return the kind of file type for the image Parameters ========== image_file: full path to the image file to inspect Returns ======= GZIP, DIRECTORY, SQUASHFS, EXT3 ''' if image_file.endswith('gz'): bot.debug('Found compressed image') return "GZIP" here = os.path.abspath(os.path.dirname(__file__)) sbin = here.replace("python", "bin/image-type") custom_env = os.environ.copy() custom_env["SINGULARITY_MESSAGELEVEL"] = "1" image_format = run_command([sbin, image_file], env=custom_env, quiet=True) if image_format is not None: if isinstance(image_format, bytes): image_format = image_format.decode('utf-8') image_format = str(image_format).strip('\n') bot.debug('Found %s image' % image_format) return image_format #################################################################### # FOLDERS ########################################################## #################################################################### def get_cache(subfolder=None, quiet=False): '''get_cache will return the user's cache for singularity. The path returned is generated at the start of the run, and returned optionally with a subfolder :param subfolder: a subfolder in the cache base to retrieve ''' # Clean up the path and create cache_base = clean_path(SINGULARITY_CACHE) # Does the user want to get a subfolder in cache base? if subfolder is not None: cache_base = "%s/%s" % (cache_base, subfolder) # Create the cache folder(s), if don't exist create_folders(cache_base) if not quiet: bot.info("Cache folder set to %s" % cache_base) return cache_base def create_folders(path): '''create_folders attempts to get the same functionality as mkdir -p :param path: the path to create. ''' try: os.makedirs(path) except OSError as e: if e.errno == errno.EEXIST and os.path.isdir(path): pass else: bot.error("Error creating path %s, exiting." % path) sys.exit(1) ############################################################################ # PERMISSIONS ############################################################## ############################################################################ def has_permission(file_path, permission=None): '''has_writability will check if a file has writability using bitwise operations. file_path can be a tar member :param file_path: the path to the file, or tar member :param permission: the stat permission to check for is False) ''' if permission is None: permission = stat.S_IWUSR if isinstance(file_path, tarfile.TarInfo): has_permission = file_path.mode & permission else: st = os.stat(file_path) has_permission = st.st_mode & permission if has_permission > 0: return True return False def change_tar_permissions(tar_file, file_permission=None, folder_permission=None): '''change_tar_permissions changes a permission if any member in a tarfile file does not have it :param file_path the path to the file :param file_permission: stat permission to use for files :param folder_permission: stat permission to use for folders ''' tar = tarfile.open(tar_file, "r:gz") # Owner read, write (o+rw) if file_permission is None: file_permission = stat.S_IRUSR | stat.S_IWUSR # Owner read, write execute (o+rwx) if folder_permission is None: folder_permission = stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR # Add owner write permission to all, not symlinks members = tar.getmembers() if len(members) > 0: bot.verbose("Fixing permission for %s" % tar_file) # Add all content objects to file fd, tmp_tar = tempfile.mkstemp(prefix=("%s.fixperm." % tar_file)) os.close(fd) fixed_tar = tarfile.open(tmp_tar, "w:gz") for member in members: # add o+rwx for directories if member.isdir() and not member.issym(): member.mode = folder_permission | member.mode extracted = tar.extractfile(member) fixed_tar.addfile(member, extracted) # add o+rw for plain files elif member.isfile() and not member.issym(): member.mode = file_permission | member.mode extracted = tar.extractfile(member) fixed_tar.addfile(member, extracted) else: fixed_tar.addfile(member) fixed_tar.close() tar.close() # Rename the fixed tar to be the old name os.rename(tmp_tar, tar_file) else: tar.close() bot.warning("Tar file %s is empty, skipping." % tar_file) return tar_file ############################################################################ # FILES #################################################################### ############################################################################ def write_file(filename, content, mode="w"): '''write_file will open a file, "filename" and write content, "content" and properly close the file ''' bot.verbose2("Writing file %s with mode %s." % (filename, mode)) with open(filename, mode) as filey: filey.writelines(content) return filename def write_json(json_obj, filename, mode="w", print_pretty=True): '''write_json will (optionally,pretty print) a json object to file :param json_obj: the dict to print to json :param filename: the output file to write to :param pretty_print: if True, will use nicer formatting ''' bot.verbose2("Writing json file %s with mode %s." % (filename, mode)) with open(filename, mode) as filey: if print_pretty is True: filey.writelines(print_json(json_obj)) else: filey.writelines(json.dumps(json_obj)) return filename def read_json(filename, mode='r'): '''read_json reads in a json file and returns the data structure as dict. ''' with open(filename, mode) as filey: data = json.load(filey) return data def read_file(filename, mode="r", readlines=True): '''read_file will open a file, "filename" and read content, "content" and properly close the file ''' bot.verbose3("Reading file %s with mode %s." % (filename, mode)) with open(filename, mode) as filey: if readlines: content = filey.readlines() else: content = filey.read() return content def print_json(content, print_console=False): '''print_json is intended to pretty print a json :param content: the dictionary to print :param print_console: if False, return the dump as string (default) ''' if print_console: print(json.dumps(content, indent=4, separators=(',', ': '))) else: return json.dumps(content, indent=4, separators=(',', ': ')) def clean_path(path): '''clean_path will canonicalize the path :param path: the path to clean ''' return os.path.realpath(path.strip(" ")) def get_fullpath(file_path, required=True): '''get_fullpath checks if a file exists, and returns the full path to it if it does. If required is true, an error is triggered. :param file_path: the path to check :param required: is the file required? If True, will exit with error ''' file_path = os.path.abspath(file_path) if os.path.exists(file_path): return file_path # If file is required, we exit if required is True: bot.error("Cannot find file %s, exiting." % file_path) sys.exit(1) # If file isn't required and doesn't exist, return None bot.warning("Cannot find file %s" % file_path) return None def get_next_infos(base_dir, prefix, start_number, extension): '''get_next infos will browse some directory and return the next available file ''' output_file = None counter = start_number found = False while not found: output_file = "%s/%s-%s%s" % (base_dir, counter, prefix, extension) if not os.path.exists(output_file): found = True counter += 1 return output_file def write_singularity_infos(base_dir, prefix, start_number, content, extension=None): '''write_singularity_infos will write some metadata object to a file in some base, starting at some default number. For example, we would want to write dockerN files with docker environment exports to some directory ENV_DIR and increase N until we find an available path :param base_dir: the directory base to write the file to :param prefix: the name of the file prefix (eg, docker) :param start_number: the number to start looking for available file at :param content: the content to write :param extension: the extension to use. If not defined, uses .sh ''' if extension is None: extension = "" else: extension = ".%s" % extension # if the base directory doesn't exist, exit with error. if not os.path.exists(base_dir): msg = "Cannot find required metadata directory" msg = "%s %s. Exiting!" % (msg, base_dir) bot.warning(msg) sys.exit(1) # Get the next available number output_file = get_next_infos(base_dir, prefix, start_number, extension) write_file(output_file, content) return output_file singularity-2.4.2/libexec/python/shell.py0000644000175000017500000002277013211621077017454 0ustar mehdimehdi''' shell.py: General shell parsing functions for Singularity in Python Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os from message import bot from defaults import ( DOCKER_API_BASE, NAMESPACE, CUSTOM_REGISTRY, CUSTOM_NAMESPACE, TAG ) import json import re import os def get_image_uri(image, quiet=False): '''get_image_uri will parse a uri sent from Singularity to determine if it's singularity (shub://) or docker (docker://) :param image: the complete image uri (example: docker://ubuntu:latest ''' image_uri = None image = image.replace(' ', '') match = re.findall('^[A-Za-z0-9-]+[:]//', image) if len(match) == 0: if not quiet: bot.warning("Could not detect any uri in %s" % image) else: image_uri = match[0].lower() if not quiet: bot.debug("Found uri %s" % (image_uri)) return image_uri def remove_image_uri(image, image_uri=None, quiet=False): '''remove_image_uri will return just the image name. ::note this will also remove all spaces from the uri. ''' if image_uri is None: image_uri = get_image_uri(image, quiet=quiet) image = image.replace(' ', '') if image_uri is not None: image = image.replace(image_uri, '') return image # Quick python regex syntax reference for referring back to matched groups: # (?:stuff) matches stuff, just like (stuff) but won't appear in m.groups() # (?Pstuff) matches stuff, and can be retrieved by m.group('name') # Regular expression used to parse docker:// uris, which have slightly # different rules than shub://, namely: # - registry must include :port or a . in the name (e.g. docker.io) # - namespace is completely optional _docker_uri_re = re.compile( # Optionally match a registry if it contains a '.' or a ':', # may contain any character but '/' or '@', and terminated by '/' # Note the use of (?:) to make sure the group "registry" does # not contain the / at the end. Also, registry includes the port "(?:(?P[^/@]+[.:][^/@]*)/)?" # Optionally match a namespace, matched as any characters but # ':' or '/', followed by '/'. Note the match will include the final / "(?P(?:[^:@/]+/)+)?" # Match a repo name, mandatory. Any character but ':' or '/' "(?P[^:@/]+)" # Match :tag (optional) "(?::(?P[^:@]+))?" # Match @digest (optional) "(?:@(?P.+))?" # we need to match the whole string, make sure there's no leftover "$" ) # Reduced regex, matches registry:port/repo or registry.com/repo # (registry must include : or .) with optional tag or version. # Also matches repo only, e.g. reponame[:tag|@version]. # This is tried before _default_uri_re, so that namespace/repo takes # precedence over registry/repo. _reduced_uri_no_ns_re = re.compile( # match a registry, optional, may include a : or ., but not a @ "(?:(?P[^/@]+[.:][^/@]*)/)?" # Match a repo name, mandatory. Any character but ':' or '/' "(?P[^:@/]+)" # Match :tag (optional) "(?::(?P[^:@]+))?" # Match @digest (optional) "(?:@(?P.+))?" # we need to match the whole string, make sure there's no leftover "$" # dummy group that will never match, but will add a 'namespace' entry "(?P.)?" ) # Regular expression used to parse other images and uris (e.g. shub://) # This version expects a namespace, and won't match otherwise # but the registry is optional. In cases like a/b/c, the first component # is taken to be the registry in this regex. _default_uri_re = re.compile( # must be either registry[:port]/namespace[/more/namespaces]/repo # or namespace/repo, at least one namespace is required # Match registry, if specified "(?:(?P[^/@]+)/)?" # Match a namespace, matched as any characters but # ':' or '@', ended by a '/'. Note the match will include the final / "(?P(?:[^:@/]+/)+)" # Match a repo name, mandatory. Any character but ':' or '/' "(?P[^:@/]+)" # Match :tag (optional) "(?::(?P[^:@]+))?" # Match @digest (optional) "(?:@(?P.+))?" # we need to match the whole string, make sure there's no leftover "$" ) def parse_image_uri(image, uri=None, quiet=False, default_registry=None, default_namespace=None, default_tag=None): '''parse_image_uri will parse a docker or shub uri and return a json structure with a registry, repo name, tag, namespace and version. Tag and version are optional, namespace is optional for docker:// uris. URIs are of this general form: myuri://[registry.com:port/][namespace/nested/]repo[:tag][@version] (parts in [] are optional) Parsing rules are slightly different if the uri is a docker:// uri: - registry must include a :port or a . in its name, else will be parsed as a namespace. For non-docker uris, instead, if there are three or more parts separated by /, the first one is taken to be the registry - namespace can be empty if a registry is specified, else default namespace will be used (e.g. docker://registry.com/repo:tag). For non-docker uris, namespace cannot be empty and default will be used :param image: the string provided on command line for the image name, eg: ubuntu:latest or docker://local.registry/busybox@12345 :param uri: the uri type (eg, docker://), default autodetects ::note uri is maintained as variable so we have some control over allowed :param quiet: If True, don't show verbose messages with the parsed values :default_registry: Which registry to use if image doesn't contain one. if None, use defaults.REGISTRY :default_namespace: Which namespace to use if image doesn't contain one. if None, use defaults.NAMESPACE. Also, check out the note above about docker:// and empty namespaces. :default_tag: Which tag to use if image doesn't contain one. if None, use defaults.REPO_TAG :returns parsed: a json structure with repo_name, repo_tag, and namespace ''' # Default to most common use case, Docker if default_registry is None: default_registry = DOCKER_API_BASE if default_namespace is None: default_namespace = NAMESPACE if default_tag is None: default_tag = TAG # if user gave custom registry / namespace, make them the default if CUSTOM_NAMESPACE is not None: default_namespace = CUSTOM_NAMESPACE if CUSTOM_REGISTRY is not None: default_registry = CUSTOM_REGISTRY # candidate regex for matching, in order of preference uri_regexes = [ _reduced_uri_no_ns_re, _default_uri_re ] # Be absolutely sure there are no comments image = image.split('#')[0] if not uri: uri = get_image_uri(image, quiet=True) # docker images require slightly different rules if uri == "docker://": uri_regexes = [ _docker_uri_re ] image = remove_image_uri(image, uri) for r in uri_regexes: match = r.match(image) if match: break if not match: bot.error('Could not parse image "%s"! Exiting.' % image) sys.exit(1) registry = match.group('registry') namespace = match.group('namespace') repo_name = match.group('repo') repo_tag = match.group('tag') version = match.group('version') if namespace: # strip trailing / namespace = namespace.rstrip('/') # repo_name is required and enforced by the re (in theory) # if not provided, re should not match assert(repo_name) # replace empty fields with defaults if not namespace: # for docker, if a registry was specified, don't # inject a namespace in between if registry and uri == "docker://": namespace = "" else: namespace = default_namespace if not registry: registry = default_registry if not repo_tag: repo_tag = default_tag # version is not mandatory, don't check that if not quiet: bot.verbose("Registry: %s" % registry) bot.verbose("Namespace: %s" % namespace) bot.verbose("Repo Name: %s" % repo_name) bot.verbose("Repo Tag: %s" % repo_tag) bot.verbose("Version: %s" % version) parsed = {'registry': registry, 'namespace': namespace, 'repo_name': repo_name, 'repo_tag': repo_tag, 'version': version } return parsed singularity-2.4.2/libexec/python/size.py0000644000175000017500000000551113211621077017311 0ustar mehdimehdi#!/usr/bin/env python ''' size.py: python helper for Singularity size ENVIRONMENTAL VARIABLES that are required for this executable: SINGULARITY_CONTAINER SINGULARITY_CONTENTS For Docker, layer sizes are determined from the tarballs, and written to the SINGULARITY_CONTENTS (contentfile). For Singularity Hub, the image size is read from the manifest Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys from shell import ( get_image_uri, remove_image_uri ) from defaults import getenv from message import bot def main(): '''this function will run the main size functions and call shub clients ''' container = getenv("SINGULARITY_CONTAINER",required=True) image_uri = get_image_uri(container) container = remove_image_uri(container) from defaults import LAYERFILE ############################################################################## # Singularity Hub ############################################################ ############################################################################## if image_uri == "shub://": bot.debug("\n*** STARTING SINGULARITY HUB SIZE PYTHON ****") from shub.main import SIZE SIZE(image=container, contentfile=LAYERFILE) elif image_uri == "docker://": from sutils import basic_auth_header from docker.main import SIZE bot.debug("Docker sizes will be written to: %s" %LAYERFILE) username = getenv("SINGULARITY_DOCKER_USERNAME") password = getenv("SINGULARITY_DOCKER_PASSWORD",silent=True) auth = None if username is not None and password is not None: auth = basic_auth_header(username, password) SIZE(image=container, auth=auth, contentfile=LAYERFILE) else: bot.error("uri %s is not a currently supported uri for size. Exiting." %image_uri) sys.exit(1) if __name__ == '__main__': main() singularity-2.4.2/libexec/python/import.py0000644000175000017500000000671113211621077017654 0ustar mehdimehdi#!/usr/bin/env python ''' import.py: python helper for Singularity import ENVIRONMENTAL VARIABLES that are required for this executable: SINGULARITY_CONTAINER SINGULARITY_CONTENTS Given that SINGULARITY_ROOTFS is defined, a full import is done that includes environment, labels, and extraction of layers. If SINGULARITY_ROOTFS is not defined, then SINGULARITY_CONTENTS must be defined, which returns a list of layer contents. Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os from shell import ( get_image_uri, remove_image_uri ) from defaults import getenv from message import bot def main(): '''this function will run a docker import, returning a list of layers and environmental variables and metadata to the metadata base ''' container = getenv("SINGULARITY_CONTAINER", required=True) image_uri = get_image_uri(container, quiet=True) container = remove_image_uri(container) ######################################################################## # Docker Image ######################################################### ######################################################################## if image_uri == "docker://": bot.debug("\n*** STARTING DOCKER IMPORT PYTHON ****") from sutils import basic_auth_header from defaults import LAYERFILE bot.debug("Docker layers and metadata will be written to: %s" % (LAYERFILE)) username = getenv("SINGULARITY_DOCKER_USERNAME") password = getenv("SINGULARITY_DOCKER_PASSWORD", silent=True) auth = None if username is not None and password is not None: auth = basic_auth_header(username, password) from docker.main import IMPORT manifest = IMPORT(auth=auth, image=container, layerfile=LAYERFILE) ######################################################################## # Singularity Hub ###################################################### ######################################################################## elif image_uri == "shub://": bot.debug("\n*** STARTING SINGULARITY HUB IMPORT PYTHON ****") from defaults import LAYERFILE, LABELFILE from shub.main import IMPORT IMPORT(image=container, layerfile=LAYERFILE, labelfile=LABELFILE) else: bot.error("uri %s is not supported for import. Exiting." % (image_uri)) sys.exit(1) if __name__ == '__main__': main() singularity-2.4.2/libexec/python/docker/0000755000175000017500000000000013211621077017232 5ustar mehdimehdisingularity-2.4.2/libexec/python/docker/api.py0000644000175000017500000004253713211621077020370 0ustar mehdimehdi''' api.py: Docker helper functions for Singularity in Python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import math import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir))) # noqa sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # noqa from base import ApiConnection from sutils import ( add_http, get_cache, change_tar_permissions, create_tar, write_singularity_infos ) from defaults import ( DOCKER_API_BASE, DOCKER_API_VERSION, DOCKER_ARCHITECTURE, DOCKER_OS, DOCKER_NUMBER, DOCKER_PREFIX, ENV_BASE, LABELFILE, METADATA_FOLDER_NAME, RUNSCRIPT_COMMAND_ASIS ) from helpers.json.main import ADD from message import bot from shell import parse_image_uri from templates import get_template import json import re import tempfile try: from urllib.error import HTTPError except ImportError: from urllib2 import HTTPError # Docker API Class --------------------------------------- class DockerApiConnection(ApiConnection): def __init__(self, **kwargs): self.auth = None self.token = None self.token_url = None self.schemaVersion = None self.reverseLayers = False self.api_base = DOCKER_API_BASE self.api_version = DOCKER_API_VERSION self.manifest = None self.manifestv1 = None if 'auth' in kwargs: self.auth = kwargs['auth'] if 'token' in kwargs: self.token = kwargs['token'] super(DockerApiConnection, self).__init__(**kwargs) if 'image' in kwargs: self.load_image(kwargs['image']) def assemble_uri(self, sep=None): '''re-assemble the image uri, for components defined. ''' if sep is None: sep = "-" image_uri = "%s%s%s" % (self.registry, sep, self.repo_name) if self.version is not None: image_uri = "%s@%s" % (image_uri, self.version) else: image_uri = "%s:%s" % (image_uri, self.repo_tag) return image_uri def _init_headers(self): # specify wanting version 2 schema # meaning the correct order of digests # returned (base to child) return {"Accept": 'application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.list.v2+json,application/json', # noqa 'Content-Type': 'application/json; charset=utf-8'} # noqa def check_errors(self, response, exit=True): '''take a response with errors key, iterate through errors in expected format and exit upon completion ''' if "errors" in response: for error in response['errors']: bot.error("%s: %s" % (error['code'], error['message'])) if error['code'] == "UNAUTHORIZED": msg = "Check existence, naming, and permissions" bot.error(msg) if exit: sys.exit(1) return response def load_image(self, image): '''load_image parses the image uri, and loads the different image parameters into the client. The image should be a docker uri (eg docker://) or name of docker image''' image = parse_image_uri(image=image, uri="docker://") if len(image['namespace']) > 0: self.repo_name = "%s/%s" % (image['namespace'], image['repo_name']) else: self.repo_name = image['repo_name'] self.repo_tag = image['repo_tag'] self.version = image['version'] self.registry = image['registry'] self.update_token() def update_token(self, response=None, auth=None): '''update_token uses HTTP basic authentication to get a token for Docker registry API V2 operations. We get here if a 401 is returned for a request. https://docs.docker.com/registry/spec/auth/token/ ''' if self.token_url is None: if response is None: response = self.get_tags(return_response=True) if not isinstance(response, HTTPError): bot.verbose3('Response on obtaining token is None.') return None not_asking_auth = "Www-Authenticate" not in response.headers if response.code != 401 or not_asking_auth: bot.error("Authentication error, exiting.") sys.exit(1) challenge = response.headers["Www-Authenticate"] regexp = '^Bearer\s+realm="(.+)",service="(.+)",scope="(.+)",?' match = re.match(regexp, challenge) if not match: bot.error("Unrecognized authentication challenge, exiting.") sys.exit(1) realm = match.group(1) service = match.group(2) scope = match.group(3).split(',')[0] self.token_url = ("%s?service=%s&expires_in=9000&scope=%s" % (realm, service, scope)) headers = dict() # First priority comes to auth supplied directly to function if auth is not None: headers.update(auth) # Second priority is default if supplied at init elif self.auth is not None: headers.update(self.auth) response = self.get(self.token_url, default_headers=False, headers=headers) try: token = json.loads(response)["token"] token = {"Authorization": "Bearer %s" % token} self.token = token self.update_headers(token) except Exception: bot.error("Error getting token for repository %s, exiting." % self.repo_name) sys.exit(1) def get_images(self, manifest=None): '''get_images will return a list of layers from a manifest. The function is intended to work with both version 1 and 2 of the schema. All layers (including redundant) are returned. For version 1 manifests: extraction is reversed :param manifest: the manifest to read_layers from ''' if manifest is None: self.update_manifests() manifest = self.manifest digests = [] layer_key = 'layers' digest_key = 'digest' # Docker manifest-v2-2.md#image-manifest if 'layers' in manifest: bot.debug('Image manifest version 2.2 found.') # Docker manifest-v2-1.md#example-manifest # noqa elif 'fsLayers' in manifest: layer_key = 'fsLayers' digest_key = 'blobSum' bot.debug('Image manifest version 2.1 found.') else: msg = "Improperly formed manifest, " msg += "layers, manifests, or fsLayers must be present" bot.error(msg) sys.exit(1) for layer in manifest[layer_key]: if digest_key in layer: bot.debug("Adding digest %s" % layer[digest_key]) digests.append(layer[digest_key]) # Reverse layer order for manifest version 1.0 if self.reverseLayers is True: message = 'v%s manifest, reversing layers' % self.schemaVersion bot.debug(message) digests.reverse() return digests def get_tags(self, return_response=False): '''get_tags will return the tags for a repo using the Docker Version 2.0 Registry API ''' registry = self.registry if registry is None: registry = self.api_base registry = add_http(registry) # make sure we have a complete url base = "%s/%s/%s/tags/list" % (registry, self.api_version, self.repo_name) bot.verbose("Obtaining tags: %s" % base) # We use get_tags for a testing endpoint in update_token response = self.get(base, return_response=return_response) if return_response: return response try: response = json.loads(response) return response['tags'] except Exception: bot.error("Error obtaining tags: %s" % base) sys.exit(1) def get_manifest(self, old_version=False, version=None): '''get_manifest should return an image manifest for a particular repo and tag. The image details are extracted when the client is generated. :param old_version: return version 1 (for cmd/entrypoint), default False ''' registry = self.registry if registry is None: registry = self.api_base # make sure we have a complete url registry = add_http(registry) base = "%s/%s/%s/manifests" % (registry, self.api_version, self.repo_name) # First priority given to calling function if version is not None: base = "%s/%s" % (base, version) elif self.version is not None: base = "%s/%s" % (base, self.version) else: base = "%s/%s" % (base, self.repo_tag) bot.verbose("Obtaining manifest: %s" % base) headers = self.headers.copy() if old_version is True: headers['Accept'] = 'application/json' response = self.get(base, headers=headers) try: response = json.loads(response) except Exception: # If the call fails, give the user a list of acceptable tags tags = self.get_tags() print("\n".join(tags)) repo_uri = "%s:%s" % (self.repo_name, self.repo_tag) bot.error("Error getting manifest for %s, exiting." % repo_uri) sys.exit(1) # If we have errors, don't continue return self.check_errors(response) def update_manifests(self): '''update manifests ensures that each of a version1 and version2 manifest are present ''' bot.debug('Updating manifests.') if self.repo_name is None: bot.error("Insufficient metadata to get manifest.") sys.exit(1) # Get full image manifest, using version 2.0 of Docker Registry API if self.manifest is None: bot.debug('MANIFEST (Primary): not found, making initial call.') self.manifest = self.get_manifest() # This is the primary manifest schema version, determines if we # need to reverse layers self.schemaVersion = self.manifest['schemaVersion'] if self.schemaVersion == 1: self.reverseLayers = True if self.manifestv1 is None: bot.debug('MANIFEST (Metadata): not found, making initial call.') self.manifestv1 = self.get_manifest(old_version=True) # https://docs.docker.com/registry/spec/manifest-v2-2/#manifest-list if "manifests" in self.manifest: for entry in self.manifest['manifests']: if entry['platform']['architecture'] == DOCKER_ARCHITECTURE: if entry['platform']['os'] == DOCKER_OS: digest = entry['digest'] bot.debug('Image manifest version 2.2 list found.') bot.debug('Obtaining architecture: %s, OS: %s' % (DOCKER_ARCHITECTURE, DOCKER_OS)) # Obtain specific os, architecture self.manifest = self.get_manifest(version=digest) break # If we didn't get a new manifest, fall back to version 1 if "manifests" in self.manifest: self.manifest = self.manifestv1 def get_layer(self, image_id, download_folder=None, change_perms=False, return_tmp=False): '''get_layer will download an image layer (.tar.gz) to a specified download folder. :param download_folder: if specified, download to folder. Otherwise return response with raw data :param change_perms: change permissions additionally (default False to support multiprocessing) :param return_tmp: If true, return the temporary file name (and don't rename to the file's final name). Default is False, should be True for multiprocessing that requires extra permission changes ''' registry = self.registry if registry is None: registry = self.api_base # make sure we have a complete url registry = add_http(registry) # The variable is repo_name base = "%s/%s/%s/blobs/%s" % (registry, self.api_version, self.repo_name, image_id) bot.verbose("Downloading layers from %s" % base) if download_folder is None: download_folder = tempfile.mkdtemp() download_folder = "%s/%s.tar.gz" % (download_folder, image_id) # Update user what we are doing bot.debug("Downloading layer %s" % image_id) # Step 1: Download the layer atomically file_name = "%s.%s" % (download_folder, next(tempfile._get_candidate_names())) tar_download = self.download_atomically(url=base, file_name=file_name) bot.debug('Download of raw file (pre permissions fix) is %s' % tar_download) # Step 2: Fix Permissions? if change_perms: tar_download = change_tar_permissions(tar_download) if return_tmp is True: return tar_download try: os.rename(tar_download, download_folder) except Exception: msg = "Cannot untar layer %s," % tar_download msg += " was there a problem with download?" bot.error(msg) sys.exit(1) return download_folder def get_size(self, add_padding=True, round_up=True, return_mb=True): '''get_size will return the image size (must use v.2.0 manifest) :add_padding: if true, return reported size * 5 :round_up: if true, round up to nearest integer :return_mb: if true, defaults bytes are converted to MB ''' self.update_manifests() size = None if "layers" in self.manifest: size = 0 for layer in self.manifest["layers"]: if "size" in layer: size += layer['size'] if add_padding is True: size = size * 5 if return_mb is True: size = size / (1024 * 1024) # 1MB = 1024*1024 bytes if round_up is True: size = math.ceil(size) size = int(size) return size def get_config(self, spec="Entrypoint", delim=None, old_version=False): '''get_config returns a particular spec (default is Entrypoint) from a VERSION 1 manifest obtained with get_manifest. :param manifest: the manifest obtained from get_manifest :param spec: the key of the spec to return, default is "Entrypoint" :param delim: Given a list, the delim to use to join the entries. Default is newline ''' manifest = self.get_manifest(old_version=old_version) cmd = None # Version 1 of the manifest has more detailed metadata if old_version: if "history" in manifest: for entry in manifest['history']: if 'v1Compatibility' in entry: entry = json.loads(entry['v1Compatibility']) if "config" in entry: if spec in entry["config"]: cmd = entry["config"][spec] # Standard is to include commands like ['/bin/sh'] if isinstance(cmd, list): if delim is None: delim = "\n" cmd = delim.join(cmd) bot.verbose("Found Docker config (%s) %s" % (spec, cmd)) else: if "config" in manifest: if spec in manifest['config']: cmd = manifest['config'][spec] return cmd singularity-2.4.2/libexec/python/docker/__init__.py0000644000175000017500000000000013211621077021331 0ustar mehdimehdisingularity-2.4.2/libexec/python/docker/main.py0000644000175000017500000001174613211621077020541 0ustar mehdimehdi''' main.py: Docker helper functions for Singularity in Python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os from defaults import ( INCLUDE_CMD, PLUGIN_FIXPERMS ) from base import MultiProcess from sutils import ( get_cache, write_file ) from .api import ( DockerApiConnection, ) from .tasks import ( download_layer, change_permissions, extract_runscript, extract_metadata_tar ) from message import bot def SIZE(image, auth=None, contentfile=None): '''size is intended to be run before an import, to return to the contentfile a list of sizes (one per layer) corresponding with the layers that will be downloaded for image ''' bot.debug("Starting Docker SIZE, will get size from manifest") bot.verbose("Docker image: %s" % image) client = DockerApiConnection(image=image, auth=auth) size = client.get_size() if contentfile is not None: write_file(contentfile, str(size), mode="w") return size def IMPORT(image, auth=None, layerfile=None): '''IMPORT is the main script that will obtain docker layers, runscript information (either entrypoint or cmd), and environment and return a list of tarballs to extract into the image :param auth: if needed, an authentication header (default None) :param layerfile: The file to write layers to extract into ''' bot.debug("Starting Docker IMPORT, includes env, runscript, and metadata.") bot.verbose("Docker image: %s" % image) # Does the user want to override default of using ENTRYPOINT? if INCLUDE_CMD: bot.verbose2("Specified Docker CMD as %runscript.") else: bot.verbose2("Specified Docker ENTRYPOINT as %runscript.") # Input Parsing ---------------------------- # Parse image name, repo name, and namespace client = DockerApiConnection(image=image, auth=auth) docker_image_uri = "Docker image path: %s" % client.assemble_uri("/") bot.info(docker_image_uri) # IMAGE METADATA ------------------------------------------- # Use Docker Registry API (version 2.0) to get images ids, manifest images = client.get_images() # DOWNLOAD LAYERS ------------------------------------------- # Each is a .tar.gz file, obtained from registry with curl # Get the cache (or temporary one) for docker cache_base = get_cache(subfolder="docker") download_client = MultiProcess() # Generate a queue of tasks to run with MultiProcess layers = [] tasks = [] for ii in range(len(images)): image_id = images[ii] targz = "%s/%s.tar.gz" % (cache_base, image_id) if not os.path.exists(targz): tasks.append((client, image_id, cache_base)) layers.append(targz) # Does the user want to change permissions of tar? func2 = None if PLUGIN_FIXPERMS: func2 = change_permissions if len(tasks) > 0: download_layers = download_client.run(func=download_layer, func2=func2, tasks=tasks) # Get Docker runscript runscript = extract_runscript(manifest=client.manifestv1, includecmd=INCLUDE_CMD) # Add the environment export tar_file = extract_metadata_tar(client.manifestv1, client.assemble_uri(), runscript=runscript) bot.verbose2('Tar file with Docker env and labels: %s' % tar_file) # Write all layers to the layerfile if layerfile is not None: bot.verbose3("Writing Docker layers files to %s" % layerfile) write_file(layerfile, "\n".join(layers), mode="w") if tar_file is not None: write_file(layerfile, "\n%s" % tar_file, mode="a") # Return additions dictionary additions = {"layers": layers, "image": image, "manifest": client.manifest, "manifestv1": client.manifestv1, "cache_base": cache_base, "metadata": tar_file} bot.debug("*** FINISHING DOCKER IMPORT PYTHON PORTION ****\n") return additions singularity-2.4.2/libexec/python/docker/tasks.py0000644000175000017500000002203613211621077020734 0ustar mehdimehdi''' tasks.py: Tasks for the Docker API Copyright (c) 2017, Vanessa Sochat. All rights reserved. ''' import sys import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), # noqa os.path.pardir))) # noqa sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # noqa from sutils import ( add_http, get_cache, create_tar, change_tar_permissions, print_json, write_singularity_infos ) from defaults import ( DOCKER_NUMBER, DOCKER_PREFIX, ENV_BASE, LABELFILE, PLUGIN_FIXPERMS, METADATA_FOLDER_NAME, RUNSCRIPT_COMMAND_ASIS ) from helpers.json.main import ADD from message import bot from templates import get_template import json import re def download_layer(client, image_id, cache_base): '''download_layer is a function (external to the client) to download a layer and update the client's token after. This is intended to be used by the multiprocessing function. If return_tmp is True, the temporary file is returned (intended to be renamed later)''' targz = client.get_layer(image_id=image_id, download_folder=cache_base, # return tmp when return_tmp=PLUGIN_FIXPERMS) # fix permissions client.update_token() return targz def change_permissions(tar_file, file_permission=None, folder_permission=None): '''change_permissions is a wrapper for change_tar_permissions, intended for use as a function for multiprocessing. To ensure atomic download and permission changes, the input file here is expected to have a temporary extension. This wrapper simply calls the function to change_tar_permissions, and then renames to the final file ''' fixed_tar = change_tar_permissions(tar_file, file_permission=file_permission, folder_permission=folder_permission) final_tar = "%s.tar.gz" % fixed_tar.split('.tar.gz')[0] os.rename(fixed_tar, final_tar) return final_tar def extract_runscript(manifest, includecmd=False): '''create_runscript will write a bash script with default "ENTRYPOINT" into the base_dir. If includecmd is True, CMD is used instead. For both. if the result is found empty, the other is tried, and then a default used. :param manifest: the manifest to use to get the runscript :param includecmd: overwrite default command (ENTRYPOINT) default is False ''' cmd = None # Does the user want to use the CMD instead of ENTRYPOINT? commands = ["Entrypoint", "Cmd"] if includecmd is True: commands.reverse() configs = get_configs(manifest, commands) # Look for non "None" command for command in commands: if configs[command] is not None: cmd = configs[command] break if cmd is not None: bot.verbose3("Adding Docker %s as Singularity runscript..." % command.upper()) # If the command is a list, join. (eg ['/usr/bin/python','hello.py'] if not isinstance(cmd, list): cmd = [cmd] cmd = " ".join(['"%s"' % x for x in cmd]) if not RUNSCRIPT_COMMAND_ASIS: cmd = 'exec %s "$@"' % cmd cmd = "#!/bin/sh\n\n%s\n" % cmd return cmd bot.debug("CMD and ENTRYPOINT not found, skipping runscript.") return cmd def extract_metadata_tar(manifest, image_name, include_env=True, include_labels=True, runscript=None): '''extract_metadata_tar will write a tarfile with the environment, labels, and runscript. include_env and include_labels should be booleans, and runscript should be None or a string to write to the runscript. ''' tar_file = None files = [] if include_env or include_labels: # Extract and add environment if include_env: environ = extract_env(manifest) if environ not in [None, ""]: bot.verbose3('Adding Docker environment to metadata tar') template = get_template('tarinfo') template['name'] = './%s/env/%s-%s.sh' % (METADATA_FOLDER_NAME, DOCKER_NUMBER, DOCKER_PREFIX) template['content'] = environ files.append(template) # Extract and add labels if include_labels: labels = extract_labels(manifest) if labels is not None: if isinstance(labels, dict): labels = print_json(labels) bot.verbose3('Adding Docker labels to metadata tar') template = get_template('tarinfo') template['name'] = "./%s/labels.json" % METADATA_FOLDER_NAME template['content'] = labels files.append(template) if runscript is not None: bot.verbose3('Adding Docker runscript to metadata tar') template = get_template('tarinfo') template['name'] = "./%s/runscript" % METADATA_FOLDER_NAME template['content'] = runscript files.append(template) if len(files) > 0: output_folder = get_cache(subfolder="metadata", quiet=True) tar_file = create_tar(files, output_folder) else: bot.warning("No metadata will be included.") return tar_file def extract_env(manifest): '''extract the environment from the manifest, or return None. Used by functions env_extract_image, and env_extract_tar ''' environ = get_config(manifest, 'Env') if environ is not None: if not isinstance(environ, list): environ = [environ] lines = [] for line in environ: line = re.findall("(?P.+?)=(?P.+)", line) line = ['export %s="%s"' % (x[0], x[1]) for x in line] lines = lines + line environ = "\n".join(lines) bot.verbose3("Found Docker container environment!") return environ def env_extract_image(manifest): '''env_extract_image will write a file of key value pairs of the environment to export. The manner to export must be determined by the calling process depending on the OS type. :param manifest: the manifest to use ''' environ = extract_env(manifest) if environ is not None: environ_file = write_singularity_infos(base_dir=ENV_BASE, prefix=DOCKER_PREFIX, start_number=DOCKER_NUMBER, content=environ, extension='sh') return environ def extract_labels(manifest, labelfile=None, prefix=None): '''extract_labels will write a file of key value pairs including maintainer, and labels. :param manifest: the manifest to use :param labelfile: if defined, write to labelfile (json) :param prefix: an optional prefix to add to the names ''' if prefix is None: prefix = "" labels = get_config(manifest, 'Labels') if labels is not None and len(labels) is not 0: bot.verbose3("Found Docker container labels!") if labelfile is not None: for key, value in labels.items(): key = "%s%s" % (prefix, key) value = ADD(key, value, labelfile, force=True) return labels def get_config(manifest, spec="Entrypoint", delim=None): '''get_config returns a particular spec (default is Entrypoint) from a VERSION 1 manifest obtained with get_manifest. :param manifest: the manifest obtained from get_manifest :param spec: the key of the spec to return, default is "Entrypoint" :param delim: Given a list, the delim to use to join the entries. Default is newline ''' cmd = None if "history" in manifest: history = manifest['history'] for entry in manifest['history']: if 'v1Compatibility' in entry: entry = json.loads(entry['v1Compatibility']) if "config" in entry: if spec in entry["config"]: if entry["config"][spec] is not None: cmd = entry["config"][spec] break # Standard is to include commands like ['/bin/sh'] if isinstance(cmd, list): if delim is not None: cmd = delim.join(cmd) bot.verbose3("Found Docker command (%s) %s" % (spec, cmd)) return cmd def get_configs(manifest, keys, delim=None): '''get_configs is a wrapper for get_config to return a dictionary with multiple config items. :param manifest: the complete manifest :param keys: the key to find :param delim: given a list, combine based on this delim ''' configs = dict() if not isinstance(keys, list): keys = [keys] for key in keys: configs[key] = get_config(manifest, key, delim=delim) return configs singularity-2.4.2/libexec/python/docker/Makefile.am0000644000175000017500000000023713211621077021270 0ustar mehdimehdiscriptlibexecdir = $(libexecdir)/singularity/python/docker dist_scriptlibexec_DATA = api.py main.py tasks.py __init__.py MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/python/base.py0000644000175000017500000002635613211621077017263 0ustar mehdimehdi''' python: base template for making a connection to an API Copyright (c) 2017, Vanessa Sochat. All rights reserved. ''' from message import bot from defaults import SINGULARITY_WORKERS import multiprocessing import itertools import tempfile import time import signal import sys import re import os try: from urllib.parse import urlencode, urlparse from urllib.request import urlopen, Request, unquote from urllib.error import HTTPError except ImportError: from urllib import urlencode, unquote from urlparse import urlparse from urllib2 import urlopen, Request, HTTPError class MultiProcess(object): def __init__(self, workers=None): if workers is None: workers = SINGULARITY_WORKERS self.workers = workers bot.debug("Using %s workers for multiprocess." % (self.workers)) def start(self): bot.debug("Starting multiprocess") self.start_time = time.time() def end(self): self.end_time = time.time() self.runtime = self.runtime = self.end_time - self.start_time bot.debug("Ending multiprocess, runtime: %s sec" % (self.runtime)) def run(self, func, tasks, func2=None): '''run will send a list of tasks, a tuple with arguments, through a function. the arguments should be ordered correctly. :param func: the function to run with multiprocessing.pool :param tasks: a list of tasks, each a tuple of arguments to process :param func2: filter function to run result from func through (optional) ''' # Keep track of some progress for the user progress = 1 total = len(tasks) # If two functions are run per task, double total jobs if func2 is not None: total = total * 2 finished = [] level1 = [] results = [] try: prefix = "[%s/%s]" % (progress, total) bot.show_progress(0, total, length=35, prefix=prefix) pool = multiprocessing.Pool(self.workers, init_worker) self.start() for task in tasks: result = pool.apply_async(multi_wrapper, multi_package(func, [task])) results.append(result) level1.append(result._job) while len(results) > 0: result = results.pop() result.wait() bot.show_progress(progress, total, length=35, prefix=prefix) progress += 1 prefix = "[%s/%s]" % (progress, total) # Pass the result through a second function? if func2 is not None and result._job in level1: result = pool.apply_async(multi_wrapper, multi_package(func2, [(result.get(),)])) results.append(result) else: finished.append(result.get()) self.end() pool.close() pool.join() except (KeyboardInterrupt, SystemExit): bot.error("Keyboard interrupt detected, terminating workers!") pool.terminate() sys.exit(1) except Exception as e: bot.error(e) return finished # Supporting functions for MultiProcess def init_worker(): signal.signal(signal.SIGINT, signal.SIG_IGN) def multi_wrapper(func_args): function, args = func_args return function(*args) def multi_package(func, args): return zip(itertools.repeat(func), args) class ApiConnection(object): def __init__(self, **kwargs): self.headers = dict() self.update_headers() def get_headers(self): return self.headers def _init_headers(self): return {'Accept': 'application/json', 'Content-Type': 'application/json; charset=utf-8'} def update_headers(self, fields=None): '''get_headers will return a simple default header for a json post. This function will be adopted as needed. ''' if len(self.headers) == 0: headers = self._init_headers() else: headers = self.headers if fields is not None: for key, value in fields.items(): headers[key] = value header_names = ",".join(list(headers.keys())) bot.debug("Headers found: %s" % (header_names)) self.headers = headers def update_token(self, response=None): '''empty update_token to be defined by subclasses, if necessary ''' return None def stream(self, url, file_name, data=None, headers=None, default_headers=True, show_progress=False): '''stream is a get that will stream to file_name :param data: a dictionary of key:value items to add to the data args variable :param url: the url to get :param show_progress: if True, show a progress bar with the bot :returns response: the requests response object, or stream ''' bot.debug("GET (stream) %s" % (url)) # If we use default headers, start with client's request_headers = dict() if default_headers and len(self.headers) > 0: request_headers = self.headers if headers is not None: request_headers.update(headers) request = self.prepare_request(headers=request_headers, data=data, url=url) response = self.submit_request(request) # Keep user updated with Progress Bar? if show_progress: content_size = None not_error = response.code not in [400, 401] if 'Content-Length' in response.headers and not_error: progress = 0 content_size = int(response.headers['Content-Length']) bot.show_progress(progress, content_size, length=35) chunk_size = 1 << 20 with open(file_name, 'wb') as filey: while True: chunk = response.read(chunk_size) if not chunk: break try: filey.write(chunk) if show_progress: if content_size is not None: progress += chunk_size bot.show_progress(iteration=progress, total=content_size, length=35, carriage_return=False) except Exception as error: msg = "Error writing to %s:" % (file_name) msg += " %s exiting" % (error) bot.error(msg) sys.exit(1) # Newline to finish download if show_progress: sys.stdout.write('\n') return file_name def get(self, url, data=None, headers=None, default_headers=True, return_response=False): '''get will use requests to get a particular url :param data: a dictionary of key:value items to add to the data args variable :param url: the url to get :returns response: the requests response object, or stream ''' bot.debug("GET %s" % (url)) # If we use default headers, start with client's request_headers = dict() if default_headers and len(self.headers) > 0: request_headers = self.headers if headers is not None: request_headers.update(headers) request = self.prepare_request(headers=request_headers, data=data, url=url) response = self.submit_request(request, return_response=return_response) if return_response is True: return response return response.read().decode('utf-8') def submit_request(self, request, return_response=False): '''submit_request will make the request, via a stream or not. If return response is True, the response is returned as is without further parsing. Given a 401 error, the update_token function is called to try the request again, and only then the error returned. ''' try: response = urlopen(request) # If we have an HTTPError, try to follow the response except HTTPError as error: # Case 1: we have an http 401 error, and need to refresh token bot.debug('Http Error with code %s' % (error.code)) if error.code == 401: self.update_token(response=error) try: request = self.prepare_request(request.get_full_url(), headers=self.headers) response = urlopen(request) except HTTPError as error: bot.debug('Http Error with code %s' % (error.code)) return error else: return error return response def prepare_request(self, url, data=None, headers=None): '''prepare the request object, determining if there is data (making it a POST) or if we should stream the result. ''' if data is not None: args = urlencode(data) request = Request(url=url, data=args, headers=headers) else: request = Request(url=url, headers=headers) return request def download_atomically(self, url, file_name, headers=None, show_progress=False): '''download stream atomically will stream to a temporary file, and rename only upon successful completion. This is to ensure that errored downloads are not found as complete in the cache :param file_name: the file name to stream to :param url: the url to stream from :param headers: additional headers to add to the get (default None) ''' try: tmp_file = "%s.%s" % (file_name, next(tempfile._get_candidate_names())) response = self.stream(url, file_name=tmp_file, headers=headers, show_progress=show_progress) os.rename(tmp_file, file_name) except Exception: download_folder = os.path.dirname(os.path.abspath(file_name)) msg = "Error downloading %s. " % (url) msg += "Do you have permission to write to %s?" % (download_folder) bot.error(msg) try: os.remove(tmp_file) except Exception: pass sys.exit(1) return file_name singularity-2.4.2/libexec/python/helpers/0000755000175000017500000000000013211621077017425 5ustar mehdimehdisingularity-2.4.2/libexec/python/helpers/__init__.py0000644000175000017500000000000013211621077021524 0ustar mehdimehdisingularity-2.4.2/libexec/python/helpers/Makefile.am0000644000175000017500000000022713211621077021462 0ustar mehdimehdiSUBDIRS = json scriptlibexecdir = $(libexecdir)/singularity/python/helpers dist_scriptlibexec_DATA = __init__.py MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/python/helpers/json/0000755000175000017500000000000013211621077020376 5ustar mehdimehdisingularity-2.4.2/libexec/python/helpers/json/inspect.py0000644000175000017500000000312413211621077022415 0ustar mehdimehdi#!/usr/bin/env python ''' inspect.py: python helper for Singularity inspect ENVIRONMENTAL VARIABLES that are used for this executable: SINGULARITY_MOUNTPOINT SINGULARITY_INSPECT_LABELS SINGULARITY_INSPECT_DEFFILE SINGULARITY_INSPECT_RUNSCRIPT SINGULARITY_INSPECT_TEST SINGULARITY_INSPECT_ENVIRONMENT SINGULARITY_APPNAME Copyright (c) 2017, Vanessa Sochat. All rights reserved. ''' import sys import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir, os.path.pardir))) # noqa from helpers.json.main import INSPECT from message import bot from defaults import getenv def main(): '''this function will run the main inspect function ''' labels = getenv("SINGULARITY_INSPECT_LABELS", None) deffile = getenv("SINGULARITY_INSPECT_DEFFILE", None) runscript = getenv("SINGULARITY_INSPECT_RUNSCRIPT", None) test = getenv("SINGULARITY_INSPECT_TEST", None) environment = getenv("SINGULARITY_INSPECT_ENVIRONMENT", None) helpfile = getenv("SINGULARITY_INSPECT_HELP", None) structured = getenv("SINGULARITY_PRINT_STRUCTURED", None) app = getenv("SINGULARITY_APPNAME", None) pretty_print = True if structured is not None: pretty_print = False INSPECT(inspect_app=app, inspect_labels=labels, inspect_def=deffile, inspect_runscript=runscript, inspect_test=test, inspect_help=helpfile, inspect_env=environment, pretty_print=pretty_print) if __name__ == '__main__': main() singularity-2.4.2/libexec/python/helpers/json/__init__.py0000644000175000017500000000000013211621077022475 0ustar mehdimehdisingularity-2.4.2/libexec/python/helpers/json/main.py0000644000175000017500000002572013211621077021702 0ustar mehdimehdi''' main.py: main utility functions for json module in Singularity python API Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir))) # noqa from sutils import ( print_json, read_file, read_json, write_json ) from defaults import ( ENVIRONMENT, HELPFILE, LABELFILE, RUNSCRIPT, TESTFILE, DEFFILE ) from message import bot import json import re def DUMP(jsonfile): '''DUMP will return the entire layfile as text, key value pairs :param jsonfile_path: the path to the jsonfile ''' bot.debug("Reading %s to prepare dump to STDOUT" % jsonfile) if not os.path.exists(jsonfile): bot.error("Cannot find %s, exiting." % jsonfile) sys.exit(1) contents = read_json(jsonfile) dump = "" for key, value in contents.items(): dump = '%s%s:"%s"\n' % (dump, key, value) dump = dump.strip('\n') print(dump) return dump def GET(key, jsonfile): '''GET will return a key from the jsonfile if it exists. If it doesn't, returns None. ''' key = format_keyname(key) bot.debug("GET %s from %s" % (key, jsonfile)) if not os.path.exists(jsonfile): bot.error("Cannot find %s, exiting." % jsonfile) sys.exit(1) contents = read_json(jsonfile) if key in contents: value = contents[key] print(value) bot.debug('%s is %s' % (key, value)) else: bot.error("%s is not defined in file. Exiting" % key) sys.exit(1) return value def ADD(key, value, jsonfile, force=False, quiet=False): '''ADD will write or update a key in a json file ''' # Check that key is not empty if key.strip() in ['#', '', None]: bot.verbose('Empty key %s, skipping' % key) sys.exit(0) key = format_keyname(key) bot.debug("Adding label: '%s' = '%s'" % (key, value)) bot.debug("ADD %s from %s" % (key, jsonfile)) if os.path.exists(jsonfile): contents = read_json(jsonfile) if key in contents: msg = 'Warning, %s is already set. ' % key msg += 'Overwrite is set to %s' % force if not quiet: bot.debug(msg) if force is True: contents[key] = value else: msg = '%s found in %s ' % (key, jsonfile) msg += 'and overwrite set to %s.' % force bot.error(msg) sys.exit(1) else: contents[key] = value else: contents = {key: value} bot.debug('%s is %s' % (key, value)) write_json(contents, jsonfile) return value def DELETE(key, jsonfile): '''DELETE will remove a key from a json file ''' key = format_keyname(key) bot.debug("DELETE %s from %s" % (key, jsonfile)) if not os.path.exists(jsonfile): bot.error("Cannot find %s, exiting." % jsonfile) sys.exit(1) contents = read_json(jsonfile) if key in contents: del contents[key] if len(contents) > 0: write_json(contents, jsonfile) else: bot.debug('%s is empty, deleting.' % jsonfile) os.remove(jsonfile) return True else: bot.debug('Warning, %s not found in %s' % (key, jsonfile)) return False def INSPECT(inspect_labels=None, inspect_def=None, inspect_app=None, inspect_runscript=None, inspect_test=None, inspect_help=None, inspect_env=None, pretty_print=True): '''INSPECT will print a "manifest" for an image, with one or more variables asked for by the user. The default prints a human readable format (text and json) and if pretty_print is turned to False, the entire thing will be returned as json via the JSON API standard. The base is checked for the metadata folder, and if it does not exist, the links are searched for and parsed (to support older versions). For all of the following, each is returned if it exists :param inspect_runscript: if not None, will include runscript :param inspect_labels: if not None, will include labels :param inspect_def: if not None, will include definition file :param inspect_test: if not None, will include test :param inspect_env: if not None, will include environment :param inspect_help: if not None, include helpfile :param pretty_print: if False, return all JSON API spec ''' from defaults import (LABELFILE, HELPFILE, RUNSCRIPT, TESTFILE, ENVIRONMENT) if inspect_app is not None: LABELBASE = "scif/apps/%s/scif/labels.json" % inspect_app LABELFILE = LABELFILE.replace('.singularity.d/labels.json', LABELBASE) HELPBASE = "scif/apps/%s/scif/runscript.help" % inspect_app HELPFILE = HELPFILE.replace('.singularity.d/runscript.help', HELPBASE) RUNBASE = "scif/apps/%s/scif/runscript" % inspect_app RUNSCRIPT = "%s/%s" % (RUNSCRIPT.strip('singularity'), RUNBASE) TESTBASE = "scif/apps/%s/scif/test" % inspect_app TESTFILE = TESTFILE.replace(".singularity.d/test", TESTBASE) ENVBASE = "scif/apps/%s/scif/" % inspect_app ENVIRONMENT = ENVIRONMENT.replace(".singularity.d/", ENVBASE) data = dict() errors = dict() # Labels if inspect_labels: bot.verbose2("Inspection of labels selected.") if os.path.exists(LABELFILE): data["labels"] = read_json(LABELFILE) else: data["labels"] = None error_detail = "This container does not have labels" errors["labels"] = generate_error(404, detail=error_detail, title="Labels Undefined") # Helpfile if inspect_help: bot.verbose2("Inspection of helpfile selected.") if os.path.exists(HELPFILE): data["help"] = read_file(HELPFILE, readlines=False) else: data["help"] = None error_detail = "This container does not have a helpfile" errors["help"] = generate_error(404, detail=error_detail, title="Help Undefined") # Definition File if inspect_def: bot.verbose2("Inspection of deffile selected.") if os.path.exists(DEFFILE): data["deffile"] = read_file(DEFFILE, readlines=False) else: error_detail = "This container doesn't have a bootstrap recipe" data["deffile"] = None errors["deffile"] = generate_error(404, title="Recipe Undefined", detail=error_detail) # Runscript if inspect_runscript: bot.verbose2("Inspection of runscript selected.") if os.path.exists(RUNSCRIPT): data["runscript"] = read_file(RUNSCRIPT, readlines=False) else: error_detail = "This container does not have any runscript defined" data["runscript"] = None errors["runscript"] = generate_error(404, title="Runscript Undefined", detail=error_detail) # Test if inspect_test: bot.verbose2("Inspection of test selected.") if os.path.exists(TESTFILE): data["test"] = read_file(TESTFILE, readlines=False) else: error_detail = "This container does not have any tests defined" data["test"] = None errors["test"] = generate_error(404, title="Tests Undefined", detail=error_detail) # Environment if inspect_env: bot.verbose2("Inspection of environment selected.") if os.path.exists(ENVIRONMENT): data["environment"] = read_file(ENVIRONMENT, readlines=False) else: error_detail = "This container doesn't have environment defined" data["environment"] = None errors["environment"] = generate_error(404, title="Tests Undefined", detail=error_detail) if pretty_print: bot.verbose2("Structured printing specified.") for dtype, content in data.items(): if content is not None: if isinstance(content, dict): print_json(content, print_console=True) else: bot.info(content) else: print_json(errors[dtype], print_console=True) else: bot.verbose2("Unstructed printing specified") # Only define a type if there is data to return # else return errors result = dict() if len(data) > 0: result["data"] = {"attributes": data, "type": "container"} else: result["errors"] = [] for dtype, content in errors.items(): result["errors"].append(content) print_json(result, print_console=True) def generate_error(status=None, title=None, detail=None): '''generate_error will return a JSON API error object, intended to be added to the list of "errors" at the top level. Errors and data cannot both be returned, and so these data structures are only returned given no data. :param status: the status code to return (http status codes) :param title: a short title to describe the error :param detail: detail about the error ''' params = locals() error = dict() for key, value in params.items(): if value is not None: error[key] = value return error def format_keyname(key): '''format keyname will ensure that all keys are uppcase, with no special characters ''' if key.startswith('org.label-schema'): return re.sub('[^A-Za-z0-9-.]+', '_', key).lower() return re.sub('[^A-Za-z0-9]+', '_', key).upper() singularity-2.4.2/libexec/python/helpers/json/dump.py0000644000175000017500000000375213211621077021724 0ustar mehdimehdi#!/usr/bin/env python ''' dump.py: wrapper for dump a a json file for Singularity Hub command line tool. This function takes input arguments of the following: --file: should be the json file to read Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir, os.path.pardir))) # noqa import optparse from helpers.json.main import DUMP from message import bot def get_parser(): parser = optparse.OptionParser(description="DUMP json") parser.add_option("--file", dest='file', help="Path to json file to dump from", type=str, default=None) return parser def main(): parser = get_parser() try: (args, options) = parser.parse_args() except Exception: sys.exit(0) if args.file is not None: dump = DUMP(args.file) else: bot.error("--file must be defined for DUMP. Exiting") sys.exit(1) if __name__ == '__main__': main() singularity-2.4.2/libexec/python/helpers/json/delete.py0000644000175000017500000000450413211621077022215 0ustar mehdimehdi#!/usr/bin/env python ''' delete.py: wrapper for "delete" of a json file for Singularity Hub command line tool. This function takes input arguments (not environment variables) of the following: --key: should be the key to delete from the json file --file: should be the json file to read Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir, os.path.pardir))) import optparse from helpers.json.main import DELETE from message import bot def get_parser(): parser = optparse.OptionParser(description="GET key from json") parser.add_option("--key", dest='key', help="key to delete from json", type=str, default=None) parser.add_option("--file", dest='file', help="Path to json file to delete from", type=str, default=None) return parser def main(): parser = get_parser() try: (args,options) = parser.parse_args() except: sys.exit(0) if args.key is not None and args.file is not None: success = DELETE(key=args.key, jsonfile=args.file) else: bot.error("--key and --file must be defined for DELETE. Exiting") sys.exit(1) if __name__ == '__main__': main() singularity-2.4.2/libexec/python/helpers/json/Makefile.am0000644000175000017500000000033413211621077022432 0ustar mehdimehdiscriptlibexecdir = $(libexecdir)/singularity/python/helpers/json dist_scriptlibexec_SCRIPTS = add.py dump.py delete.py get.py inspect.py dist_scriptlibexec_DATA = main.py __init__.py MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/python/helpers/json/add.py0000644000175000017500000000602313211621077021501 0ustar mehdimehdi#!/usr/bin/env python ''' add.py: wrapper for "add" of a key to a json file for Singularity Hub command line tool. This function takes input arguments of the following: --key: should be the key to lookup from the json file --value: the value to add to the key --file: should be the json file to read Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir, os.path.pardir))) # noqa import optparse from helpers.json.main import ADD from message import bot def get_parser(): parser = optparse.OptionParser(description="GET key from json") parser.add_option("--key", dest='key', help="key to add to json", type=str, default=None) parser.add_option("--value", dest='value', help="value to add to the json", type=str, default=None) parser.add_option("--file", dest='file', help="Path to json file to add to", type=str, default=None) parser.add_option('-f', dest="force", help="force add (overwrite if exists)", default=False, action='store_true') parser.add_option('--quiet', dest="quiet", help="do not display debug", default=False, action='store_true') return parser def main(): parser = get_parser() try: (args, options) = parser.parse_args() except Exception: sys.exit(0) if args.key is not None and args.file is not None: if args.value is not None: value = ADD(key=args.key, value=args.value, jsonfile=args.file, force=args.force, quiet=args.quiet) else: bot.error("--key and --file and --value must be defined for ADD.") sys.exit(1) if __name__ == '__main__': main() singularity-2.4.2/libexec/python/helpers/json/get.py0000644000175000017500000000445513211621077021537 0ustar mehdimehdi#!/usr/bin/env python ''' get.py: wrapper for "get" of a json file for Singularity Hub command line tool. This function takes input arguments (not environment variables) of the following: --key: should be the key to lookup from the json file --file: should be the json file to read Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys import os sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir, os.path.pardir))) import optparse from helpers.json.main import GET from message import bot def get_parser(): parser = optparse.OptionParser(description="GET key from json") parser.add_option("--key", dest='key', help="key to get from json", type=str, default=None) parser.add_option("--file", dest='file', help="Path to json file to retrieve from", type=str, default=None) return parser def main(): parser = get_parser() try: (args,options) = parser.parse_args() except: sys.exit(0) if args.key is not None and args.file is not None: value = GET(key=args.key, jsonfile=args.file) else: bot.error("--key and --file must be defined for GET. Exiting") sys.exit(1) if __name__ == '__main__': main() singularity-2.4.2/libexec/python/Makefile.am0000644000175000017500000000044713211621077020024 0ustar mehdimehdiSUBDIRS = docker shub helpers EXTRA_DIST = tests scriptlibexecdir = $(libexecdir)/singularity/python dist_scriptlibexec_DATA = defaults.py __init__.py message.py shell.py sutils.py base.py templates.py dist_scriptlibexec_SCRIPTS = import.py pull.py size.py MAINTAINERCLEANFILES = Makefile.in singularity-2.4.2/libexec/python/templates.py0000644000175000017500000000341013211621077020331 0ustar mehdimehdi''' templates.py: template data structures for Singularity python Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' from message import bot def get_template(template_name): '''get_template will return a default template for some function in Singularity Python. This is to reduce redundancy if data structures are used multiple times, etc. If there is no template, None is returned. ''' template_name = template_name.lower() templates = dict() templates['tarinfo'] = {"gid": 0, "uid": 0, "uname": "root", "gname": "root", "mode": 493} if template_name in templates: bot.debug("Found template for %s" % (template_name)) return templates[template_name] else: bot.warning("Cannot find template %s" % (template_name)) return None singularity-2.4.2/libexec/python/tests/0000755000175000017500000000000013211621077017125 5ustar mehdimehdisingularity-2.4.2/libexec/python/tests/test_docker_api.py0000644000175000017500000000776513211621077022655 0ustar mehdimehdi''' test_docker_api.py: Docker testing functions for Singularity in Python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import re import sys sys.path.append('..') # noqa from unittest import TestCase import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s API TESTING START ***" % VERSION) class TestApi(TestCase): def setUp(self): self.image = 'docker://ubuntu:latest' self.tmpdir = tempfile.mkdtemp() os.environ['SINGULARITY_ROOTFS'] = self.tmpdir os.mkdir('%s/.singularity.d' % self.tmpdir) from docker.api import DockerApiConnection self.client = DockerApiConnection(image=self.image) print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_get_token(self): '''test_get_token will obtain a token from the Docker registry for a namepspace and repo. ''' from docker.api import DockerApiConnection docker_image = "gcr.io/tensorflow/tensorflow:1.0.0" client = DockerApiConnection(image=docker_image) print("Case 1: Ask when we don't need token returns None") token = client.update_token() self.assertEqual(token, None) def test_get_manifest(self): '''test_get_manifest will obtain a library/repo manifest ''' from docker.api import DockerApiConnection print("Case 1: Obtain manifest for %s" % self.client.repo_name) manifest = self.client.get_manifest(old_version=True) # Default tag should be latest self.assertTrue("fsLayers" in manifest) # Giving a bad tag sould return error print("Case 3: Bad tag should print valid tags and exit") client = DockerApiConnection(image="ubuntu:mmm.avocado") def test_get_images(self): '''test_get_images will obtain a list of images ''' from docker.api import DockerApiConnection images = self.client.get_images() self.assertTrue(isinstance(images, list)) self.assertTrue(len(images) > 1) def test_get_tags(self): '''test_get_tags will obtain a list of tags ''' from docker.api import DockerApiConnection print("Case 1: Ask for tags from standard %s" % self.client.repo_name) tags = self.client.get_tags() self.assertTrue(isinstance(tags, list)) self.assertTrue(len(tags) > 1) ubuntu_tags = ['xenial', 'latest', 'trusty', 'yakkety'] [self.assertTrue(x in tags) for x in ubuntu_tags] def test_get_layer(self): '''test_get_layer will download docker layers ''' from docker.api import DockerApiConnection images = self.client.get_images() print("Case 1: Download an existing layer, should succeed") layer_file = self.client.get_layer(image_id=images[0], download_folder=self.tmpdir) self.assertTrue(os.path.exists(layer_file)) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_custom_cache.py0000644000175000017500000000377713211621077023211 0ustar mehdimehdi''' test_custom_cache.py: Testing custom cache function Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import re import sys sys.path.append('..') # noqa from unittest import TestCase import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s CLIENT TESTING START ***" % VERSION) class TestCustomCache(TestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() self.custom_cache = '%s/cache' % self.tmpdir os.environ['SINGULARITY_CACHEDIR'] = self.custom_cache os.environ['SINGULARITY_ROOTFS'] = self.tmpdir print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_get_cache_custom(self): '''test_run_command tests sending a command to commandline using subprocess ''' print("Testing get_cache with environment set") from defaults import SINGULARITY_CACHE self.assertEqual(self.custom_cache, SINGULARITY_CACHE) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_disable_cache.py0000644000175000017500000000351413211621077023267 0ustar mehdimehdi''' test_disable_cache.py: Testing disabled cache function Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import re import sys sys.path.append('..') # noqa from unittest import TestCase import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s CLIENT TESTING START ***" % VERSION) class TestDisabledCache(TestCase): def setUp(self): os.environ['SINGULARITY_DISABLE_CACHE'] = "yes" self.tmpdir = tempfile.mkdtemp() os.environ['SINGULARITY_ROOTFS'] = self.tmpdir print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_cache_disabled(self): print("Case 3: Testing that cache gets disabled.") from defaults import DISABLE_CACHE self.assertEqual(DISABLE_CACHE, True) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_shub_api.py0000644000175000017500000001032013211621077022324 0ustar mehdimehdi''' test_shub.py: Singularity Hub testing functions for Singularity in Python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import sys sys.path.append('..') # noqa from unittest import TestCase from sutils import read_file from glob import glob import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s SINGULARITY HUB API TESTING START ***" % VERSION) class TestApi(TestCase): def setUp(self): from shub.api import SingularityApiConnection self.connect = SingularityApiConnection self.image = "shub://vsoch/singularity-images" self.client = SingularityApiConnection(image=self.image) self.user_name = "vsoch" self.repo_name = "singularity-images" self.tmpdir = tempfile.mkdtemp() os.environ['SINGULARITY_ROOTFS'] = self.tmpdir os.mkdir('%s/.singularity.d' % self.tmpdir) def tearDown(self): shutil.rmtree(self.tmpdir) def test_get_manifest(self): '''test_get_manifest should return the shub manifest ''' print("Testing singularity hub manifest retrieval") print("...case 1: Testing retrieval of singularity-hub manifest") manifest = self.client.get_manifest() keys = ['version', 'tag', 'branch', 'name', 'id', 'commit', 'image'] [self.assertTrue(x in manifest) for x in keys] def test_download_image(self): '''test_download_image will ensure that an image is downloaded to an appropriate location (tmpdir) or cache ''' print("Testing singularity hub pull") print("...case 1: Specifying a directory downloads to it") manifest = self.client.get_manifest() image = self.client.download_image(manifest=manifest, download_folder=self.tmpdir) self.assertEqual(os.path.dirname(image), self.tmpdir) print("...case 2: Not specifying a directory downloads to PWD") os.chdir(self.tmpdir) image = self.client.download_image(manifest) self.assertEqual(os.getcwd(), self.tmpdir) print(image) self.assertTrue(image in glob("*")) os.remove(image) def test_uri(self): '''test_uri will make sure that the endpoint returns the equivalent image for all different uri options ''' print("Testing singularity hub uris") from shub.api import get_image_name manifest = self.client.get_manifest() image_name = get_image_name(manifest) fullname = "%s/%s" % (self.user_name, self.repo_name) print("...case 1: ask for image by tag.") client = self.connect(image="%s:latest" % fullname) manifest = client.get_manifest() image_name = get_image_name(manifest) self.assertEqual(image_name, get_image_name(manifest)) def test_get_image_name(self): '''test_get_image_name will return the image name from the manifest ''' print("Testing singularity hub image naming") from shub.api import get_image_name manifest = self.client.get_manifest() print("...case 1: return an image name corresponding to repo") image_name = get_image_name(manifest) self.assertEqual('vsoch-singularity-images-master-latest', image_name) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_docker_tasks.py0000644000175000017500000000666013211621077023222 0ustar mehdimehdi''' test_docker_tasks.py: Docker tasks testing for Singularity in Python Copyright (c) 2017, Vanessa Sochat. All rights reserved. ''' import os import re import sys sys.path.append('..') # noqa from unittest import TestCase import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s TASKS TESTING START ***" % VERSION) class TestApi(TestCase): def setUp(self): self.image = 'docker://ubuntu:latest' self.tmpdir = tempfile.mkdtemp() os.environ['SINGULARITY_ROOTFS'] = self.tmpdir os.mkdir('%s/.singularity.d' % self.tmpdir) from docker.api import DockerApiConnection self.client = DockerApiConnection(image=self.image) print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_create_runscript(self): '''test_create_runscript should ensure that a runscript is generated with some command ''' from docker.api import DockerApiConnection print('Testing creation of runscript') from docker.tasks import extract_runscript manifest = self.client.get_manifest(old_version=True) print("Case 1: Asking for CMD when none defined") default_cmd = 'exec "/bin/bash"' runscript = extract_runscript(manifest=manifest, includecmd=True) self.assertTrue(default_cmd in runscript) print("Case 2: Asking for ENTRYPOINT when none defined") runscript = extract_runscript(manifest=manifest) self.assertTrue(default_cmd in runscript) client = DockerApiConnection(image="docker://bids/mriqc:0.0.2") manifest = client.get_manifest(old_version=True) print("Case 3: Asking for ENTRYPOINT when defined") runscript = extract_runscript(manifest=manifest) self.assertTrue('exec "/run_mriqc"' in runscript) print("Case 4: Asking for CMD when defined") runscript = extract_runscript(manifest=manifest, includecmd=True) self.assertTrue('exec "--help"' in runscript) print("Case 5: Asking for ENTRYPOINT when None, should return CMD") from docker.tasks import get_configs client = DockerApiConnection(image="tensorflow/tensorflow:1.0.0") manifest = client.get_manifest(old_version=True) configs = get_configs(manifest, ['Cmd', 'Entrypoint']) self.assertEqual(configs['Entrypoint'], None) runscript = extract_runscript(manifest=manifest) self.assertTrue(configs['Cmd'][0] in runscript) def test_get_config(self): '''test_get_config will obtain parameters from the DOcker configuration json ''' from docker.api import DockerApiConnection from docker.tasks import get_config # Default should return entrypoint print("Case 1: Ask for default command (Entrypoint)") manifest = self.client.get_manifest(old_version=True) entrypoint = get_config(manifest=manifest) # Ubuntu latest should have None self.assertEqual(entrypoint, None) print("Case 2: Ask for custom command (Cmd)") entrypoint = get_config(manifest=manifest, spec="Cmd") self.assertTrue('/bin/bash' in entrypoint) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_default_cache.py0000644000175000017500000000431313211621077023306 0ustar mehdimehdi''' test_default_cache.py: Testing default cache function Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import re import sys sys.path.append('..') # noqa from unittest import TestCase import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s CLIENT TESTING START ***" % VERSION) class TestDefaultCache(TestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() os.environ['SINGULARITY_ROOTFS'] = self.tmpdir if "SINGULARITY_CACHE_DIR" in os.environ: del os.environ['SINGULARITY_CACHE_DIR'] print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_get_cache_default(self): '''test_run_command tests sending a command to commandline using subprocess ''' print("Testing get_cache...") # If there is no cache_base, we should get default print("Case 1: No cache base in environment returns default") from defaults import SINGULARITY_CACHE home = os.environ['HOME'] self.assertEqual("%s/.singularity" % home, SINGULARITY_CACHE) self.assertTrue(os.path.exists(SINGULARITY_CACHE)) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/__init__.py0000644000175000017500000000000013211621077021224 0ustar mehdimehdisingularity-2.4.2/libexec/python/tests/test_json.py0000644000175000017500000002421613211621077021514 0ustar mehdimehdi''' test_json.py: Singularity Hub testing functions for Singularity in Python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import sys sys.path.append('..') # noqa from unittest import TestCase import shutil import tempfile from subprocess import ( Popen, PIPE, STDOUT ) VERSION = sys.version_info[0] print("*** PYTHON VERSION %s UTIL HELPERS TESTING START ***" % VERSION) class TestJson(TestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() self.here = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)) self.file = "%s/meatballs.json" % self.tmpdir print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def format_keyname(self): '''test the function to format the key name ''' from helpers.json.main import format_keyname print("Testing formatting of key function.") print('Case 1: Testing that key returns all caps') key = format_keyname('dry_meatball') self.assertEqual(key, 'DRY_MEATBALL') print('Case 2: Testing that key replaced invalid characters with _') key = format_keyname('big!!meatball)#$FTW') self.assertEqual(key, 'DRY_MEATBALL_FTW') key = format_keyname('ugly-meatball') self.assertEqual(key, 'UGLY_MEATBALL') def test_get(self): '''test_get will test the get function ''' print('Testing json GET') print('Case 1: Get exiting key') from sutils import write_json write_json({"PASTA": "rigatoni!"}, self.file) self.assertTrue(os.path.exists(self.file)) script_path = "%s/helpers/json/get.py" % self.here if VERSION == 2: testing_command = ["python2", script_path, '--key', 'PASTA', '--file', self.file] else: testing_command = ["python3", script_path, '--key', 'PASTA', '--file', self.file] output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 0) output = result['message'] if isinstance(output, bytes): output = output.decode(encoding='UTF-8') self.assertEqual('rigatoni!', output.strip('\n').split('\n')[-1]) print('Case 2: Get non-existing key exits') if VERSION == 2: testing_command = ["python2", script_path, '--key', 'LASAGNA', '--file', self.file] else: testing_command = ["python3", script_path, '--key', 'LASAGNA', '--file', self.file] output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 1) def test_add_delete(self): '''test_add_delete will test the add and delete functions ''' print('Testing json ADD') from sutils import write_json, read_json print('Case 1: Adding to new file, force not needed') self.assertTrue(os.path.exists(self.file) is False) script_path = "%s/helpers/json/add.py" % self.here if VERSION == 2: testing_command = ["python2", script_path, '--key', 'LEGO', '--value', 'RED', '--file', self.file] else: testing_command = ["python3", script_path, '--key', 'LEGO', '--value', 'RED', '--file', self.file] output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 0) self.assertTrue(os.path.exists(self.file)) # Check the contents contents = read_json(self.file) self.assertTrue('LEGO' in contents) self.assertTrue(contents['LEGO'] == 'RED') print('Case 2: Adding to existing key without force should error.') output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 1) print('Case 3: Adding to existing key with force should work.') if VERSION == 2: testing_command = ["python2", script_path, '--key', 'LEGO', '--value', 'BLUE', '--file', self.file, '-f'] else: testing_command = ["python3", script_path, '--key', 'LEGO', '--value', 'BLUE', '--file', self.file, '-f'] output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 0) # Check the updated contents contents = read_json(self.file) self.assertTrue('LEGO' in contents) self.assertTrue(contents['LEGO'] == 'BLUE') if VERSION == 2: testing_command = ["python2", script_path, '--key', 'PASTA', '--value', 'rigatoni!', '--file', self.file] else: testing_command = ["python3", script_path, '--key', 'PASTA', '--value', 'rigatoni!', '--file', self.file] output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} print('Case 4: Deleting key from file') script_path = "%s/helpers/json/delete.py" % self.here if VERSION == 2: testing_command = ["python2", script_path, '--key', 'LEGO', '--file', self.file] else: testing_command = ["python3", script_path, '--key', 'LEGO', '--file', self.file] output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 0) # Check the key was deleted contents contents = read_json(self.file) self.assertTrue('LEGO' not in contents) print('Case 5: Checking that empty file is removed.') if VERSION == 2: testing_command = ["python2", script_path, '--key', 'PASTA', '--file', self.file] else: testing_command = ["python3", script_path, '--key', 'PASTA', '--file', self.file] output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertTrue(os.path.exists(self.file) is False) def test_dump(self): '''test_add_delete will test the add and delete functions ''' print('Testing json DUMP') from sutils import write_json, read_json print('Case 1: Dumping file.') jsondump = {'HELLO': 'KITTY', 'BATZ': 'MARU', 'MY': 'MELODY'} write_json(jsondump, self.file) self.assertTrue(os.path.exists(self.file)) script_path = "%s/helpers/json/dump.py" % self.here if VERSION == 2: testing_command = ["python2", script_path, '--file', self.file] else: testing_command = ["python3", script_path, '--file', self.file] output = Popen(testing_command, stderr=PIPE, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 0) output = result['message'] if isinstance(output, bytes): output = output.decode(encoding='UTF-8') dump = ['HELLO:"KITTY"', 'BATZ:"MARU"', 'MY:"MELODY"'] result = output.strip('\n').split('\n')[-3:] for res in result: self.assertTrue(res in dump) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_core.py0000644000175000017500000007362013211621077021476 0ustar mehdimehdi''' test_core.py: Python testing for core functions for Singularity in Python, including defaults, utils, and shell functions. Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import re import sys import tarfile sys.path.append('..') # noqa from unittest import TestCase import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s BASE TESTING START ***" % VERSION) class TestShell(TestCase): def setUp(self): # Test repo information self.registry = "registry" self.repo_name = "repo" self.namespace = "namespace" self.tag = "tag" # Default repo information self.REGISTRY = 'index.docker.io' self.NAMESPACE = 'library' self.REPO_TAG = 'latest' self.tmpdir = tempfile.mkdtemp() os.environ['SINGULARITY_ROOTFS'] = self.tmpdir print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_get_image_uri(self): '''test_get_image_uri ensures that the correct uri is returned for a user specified uri, registry, namespace. ''' from shell import get_image_uri print("Case 1: No image uri should return None") image_uri = get_image_uri('namespace/repo:tag') self.assertEqual(image_uri, None) print("Case 2: testing return of shub://") image_uri = get_image_uri('shub://namespace/repo:tag') self.assertEqual(image_uri, 'shub://') print("Case 3: testing return of docker uri") image_uri = get_image_uri('docker://namespace/repo:tag') self.assertEqual(image_uri, 'docker://') print("Case 4: weird capitalization should return lowercase") image_uri = get_image_uri('DocKer://namespace/repo:tag') self.assertEqual(image_uri, 'docker://') def test_remove_image_uri(self): '''test_remove_image_uri removes the uri ''' from shell import remove_image_uri print("Case 1: No image_uri should estimate first") image = remove_image_uri('myuri://namespace/repo:tag') self.assertEqual(image, "namespace/repo:tag") print("Case 2: Missing image uri should return image") image = remove_image_uri('namespace/repo:tag') self.assertEqual(image, "namespace/repo:tag") def test_parse_image_uri(self): '''test_parse_image_uri ensures that the correct namespace, repo name, and tag (or unique id) is returned. ''' from shell import parse_image_uri print("Case 1: Empty repo_name should return error") with self.assertRaises(SystemExit) as cm: image = parse_image_uri(image="") self.assertEqual(cm.exception.code, 1) print("Case 2: Checking for correct output tags in digest...") image_name = "%s/%s" % (self.namespace, self.repo_name) digest = parse_image_uri(image=image_name) for tag in ['registry', 'repo_name', 'repo_tag', 'namespace']: self.assertTrue(tag in digest) print("Case 3: Specifying only an image should return defaults") image = parse_image_uri(image="shub://lizardleezle", uri="shub://") self.assertTrue(isinstance(image, dict)) self.assertEqual(image["namespace"], self.NAMESPACE) self.assertEqual(image["repo_tag"], self.REPO_TAG) self.assertEqual(image["repo_name"], 'lizardleezle') self.assertEqual(image["registry"], self.REGISTRY) print("Case 4: Tag when specified should be returned.") image_name = "%s/%s:%s" % (self.namespace, self.repo_name, "pusheenasaurus") digest = parse_image_uri(image_name) self.assertTrue(digest['repo_tag'] == 'pusheenasaurus') print("Case 5: Repo name and tag without namespace...") image_name = "%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['repo_tag'] == self.tag) self.assertTrue(digest['namespace'] == self.NAMESPACE) self.assertTrue(digest['repo_name'] == self.repo_name) print("Case 6: Changing namespace should not use default.") image_name = "meow/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['namespace'] == 'meow') print("Case 7: Changing registry shouldn't use index.docker.io.") image_name = "meow/mix/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow') self.assertTrue(digest['namespace'] == 'mix') print("Case 8: Custom uri should use it.") image_name = "catdog://meow/mix/tenders/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="catdog://") self.assertTrue(digest['registry'] == 'meow') self.assertTrue(digest['namespace'] == 'mix/tenders') print("Case 9: Digest version should be parsed") image_name = ("catdog://meow/mix/original/choice/%s:%s@sha:256xxxxxxxxxxxxxxx" # noqa % (self.repo_name, self.tag)) digest = parse_image_uri(image_name, uri="catdog://") self.assertTrue(digest['registry'] == 'meow') self.assertTrue(digest['namespace'] == 'mix/original/choice') self.assertTrue(digest['version'] == 'sha:256xxxxxxxxxxxxxxx') # now test some tricky cases print("Case 10: registry and namespace, @version contains / and : (docker://)") image_name = "some.registry.com/mix/mux/repo@me/version-1:3:2" digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == 'some.registry.com') self.assertTrue(digest['repo_tag'] == self.REPO_TAG) self.assertTrue(digest['namespace'] == 'mix/mux') self.assertTrue(digest['repo_name'] == 'repo') self.assertTrue(digest['version'] == 'me/version-1:3:2') print("Case 11: registry and namespace, @version contains / and : (generic)") image_name = "registry/mix/mux/repo@me/version-1:3:2" digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'registry') self.assertTrue(digest['repo_tag'] == self.REPO_TAG) self.assertTrue(digest['namespace'] == 'mix/mux') self.assertTrue(digest['repo_name'] == 'repo') self.assertTrue(digest['version'] == 'me/version-1:3:2') print("Case 12: Namespaces can include / characters i.e. can be nested") image_name = "meow/mix/barf/baz/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow') self.assertTrue(digest['namespace'] == 'mix/barf/baz') print("Case 13: Namespaces can include '.'") image_name = "meow/mix.max/barf.baz/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow') self.assertTrue(digest['namespace'] == 'mix.max/barf.baz') self.assertTrue(digest['repo_tag'] == self.tag) self.assertTrue(digest['repo_name'] == self.repo_name) print("Case 14: registry contains ., default namespace") image_name = "meow.io/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == self.NAMESPACE) self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 15: registry contains :port, default namespace") # namespace is not allowed to be empty except for docker:// uris # so in this case, the default namespace is used image_name = "meow:123/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow:123') self.assertTrue(digest['namespace'] == self.NAMESPACE) self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 16: Namespace cannot be empty with full non-docker uri") image_name = "myuri://meow.io/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == self.NAMESPACE) self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 17: comments at the end of the line") image_name = "myuri://meow.io/mix/%s:%s # comment hel.lo/test:blah@stuff" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 18: tag cannot contain @, version without : should be parsed") # apparently there was a bug where if @version doesn't contain any : character # it will get mis-parsed as part of the tag, which is clearly wrong image_name = "myuri://meow.io/mix/my-repo:tag-1.2.3@master" digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == 'my-repo') self.assertTrue(digest['repo_tag'] == 'tag-1.2.3') self.assertTrue(digest['version'] == 'master') print("Case 19: tag cannot contain @, version without : should be parsed") # apparently there was a bug where if @version doesn't contain any : character # it will get mis-parsed as part of the tag, which is clearly wrong image_name = "myuri://meow.io/mix/my-repo:tag@2.2" digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == 'my-repo') self.assertTrue(digest['repo_tag'] == 'tag') self.assertTrue(digest['version'] == '2.2') def test_parse_image_uri_docker(self): """ Docker-specific uri parsing rules """ from shell import parse_image_uri print("Case 1: just image, default everything else") digest = parse_image_uri(self.repo_name, uri="docker://") self.assertTrue(digest['registry'] == self.REGISTRY) self.assertTrue(digest['namespace'] == self.NAMESPACE) self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.REPO_TAG) print("Case 2: just image and tag, default everything else") image_name = "%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == self.REGISTRY) self.assertTrue(digest['namespace'] == self.NAMESPACE) self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 3: image, namespace and tag") image_name = "mix/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == self.REGISTRY) self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 4: image, namespace, tag and version") image_name = "mix/%s:%s@sha:256xxxxxxxxxxxxxxx" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == self.REGISTRY) self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) self.assertTrue(digest['version'] == 'sha:256xxxxxxxxxxxxxxx') print("Case 5: image, several namespaces and tag") # for docker, registry must have a . or a :port, else it's parsed # as a namespace. In this case, no registry is specified image_name = "mix/max/blitz/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == self.REGISTRY) self.assertTrue(digest['namespace'] == 'mix/max/blitz') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 6: image, namespace, tag and version") image_name = "mix/max/blitz/%s:%s@sha:256xxxxxxxxxxxxxxx" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == self.REGISTRY) self.assertTrue(digest['namespace'] == 'mix/max/blitz') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) self.assertTrue(digest['version'] == 'sha:256xxxxxxxxxxxxxxx') print("Case 7: registry with ., image and tag, empty namespace") # with docker://, if registry is present in uri, and namespace is empty, # we parse the namespace as empty image_name = "meow.io/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == '') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 8: registry with :port, image and tag, empty namespace") image_name = "meow:5000/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == 'meow:5000') self.assertTrue(digest['namespace'] == '') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 9: registry with . and :port, image and tag") image_name = "meow.io:5000/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == 'meow.io:5000') self.assertTrue(digest['namespace'] == '') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 10: registry with . and :port, image, no tag. empty namespace") image_name = "meow.io:5000/%s" % (self.repo_name) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == 'meow.io:5000') self.assertTrue(digest['namespace'] == '') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.REPO_TAG) print("Case 11: registry, image, namespace and tag") image_name = "meow.io/mix/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 12: registry, image, several namespaces and tag") image_name = "meow:5000/mix/max/blitz/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == 'meow:5000') self.assertTrue(digest['namespace'] == 'mix/max/blitz') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 13: no registry, several namespaces containing . and tag") # a registry is matched if it contains : or ., but it must be the first one # before any other namespace, else it's just a namespace image_name = "mix/max.nix/blitz.krieg/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name, uri="docker://") self.assertTrue(digest['registry'] == self.REGISTRY) self.assertTrue(digest['namespace'] == 'mix/max.nix/blitz.krieg') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 15: full docker:// uri with registry") image_name = "docker://meow.io/mix/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 16: full docker uri with registry, empty namespace") image_name = "docker://meow:5000/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow:5000') self.assertTrue(digest['namespace'] == '') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 17: Namespace can be empty when full docker:// uri specified") # note: registry must include a . or a :port, else will be parsed as a namespace image_name = "docker://meow.io/%s:%s" % (self.repo_name, self.tag) digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == '') self.assertTrue(digest['repo_name'] == self.repo_name) self.assertTrue(digest['repo_tag'] == self.tag) print("Case 18: tag cannot contain @, version without : should be parsed") # apparently there was a bug where if @version doesn't contain any : character # it will get mis-parsed as part of the tag, which is clearly wrong image_name = "docker://meow.io/mix/my-repo:tag-1.2.3@master" digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == 'my-repo') self.assertTrue(digest['repo_tag'] == 'tag-1.2.3') self.assertTrue(digest['version'] == 'master') print("Case 19: tag cannot contain @, version without : should be parsed") # apparently there was a bug where if @version doesn't contain any : character # it will get mis-parsed as part of the tag, which is clearly wrong image_name = "docker://meow.io/mix/my-repo:tag@2.2" digest = parse_image_uri(image_name) self.assertTrue(digest['registry'] == 'meow.io') self.assertTrue(digest['namespace'] == 'mix') self.assertTrue(digest['repo_name'] == 'my-repo') self.assertTrue(digest['repo_tag'] == 'tag') self.assertTrue(digest['version'] == '2.2') class TestUtils(TestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_add_http(self): '''test_add_http ensures that http is added to a url ''' from sutils import add_http url_http = 'http://registry.docker.io' url_https = 'https://registry.docker.io' print("Case 1: adding https to url with nothing specified...") # Default is https url = 'registry.docker.io' http = add_http(url) self.assertEqual(url_https, http) # http print("Case 2: adding http to url with nothing specified...") http = add_http(url, use_https=False) self.assertEqual(url_http, http) # This should not change. Note - is url is http, stays http print("Case 3: url already has https, should not change...") url = 'https://registry.docker.io' http = add_http(url) self.assertEqual(url_https, http) # This should not change. Note - is url is http, stays http print("Case 4: url already has http, should not change...") url = 'http://registry.docker.io' http = add_http(url, use_https=False) self.assertEqual(url_http, http) print("Case 5: url has http, should change to https") url = 'http://registry.docker.io' http = add_http(url) self.assertEqual(url_https, http) print("Case 6: url has https, should change to http") url = 'https://registry.docker.io' http = add_http(url, use_https=False) self.assertEqual(url_http, http) print("Case 7: url should have trailing slash stripped") url = 'https://registry.docker.io/' http = add_http(url, use_https=False) self.assertEqual(url_http, http) def test_headers(self): '''test_add_http ensures that http is added to a url ''' print("Testing utils header functions...") from sutils import basic_auth_header # Basic auth header print("Case 4: ask for custom authentication header") auth = basic_auth_header(username='vanessa', password='pancakes') self.assertEqual(auth['Authorization'], 'Basic dmFuZXNzYTpwYW5jYWtlcw==') def test_run_command(self): '''test_run_command tests sending a command to commandline using subprocess ''' print("Testing utils.run_command...") from sutils import run_command # An error should return None print("Case 1: Command errors returns None ") none = run_command(['exec', 'whaaczasd']) self.assertEqual(none, None) # A success should return console output print("Case 2: Command success returns output") hello = run_command(['echo', 'hello']) if not isinstance(hello, str): # python 3 support hello = hello.decode('utf-8') self.assertEqual(hello, 'hello\n') def test_extract_tar(self): '''test_extract_tar will test extraction of a tar.gz file ''' print("Testing utils.extract_tar...") # First create a temporary tar file from sutils import extract_tar from glob import glob import tarfile # Create and close a temporary tar.gz print("Case 1: Testing tar.gz...") creation_dir = tempfile.mkdtemp() archive, files = create_test_tar(creation_dir) # Extract to different directory extract_dir = tempfile.mkdtemp() extract_tar(archive=archive, output_folder=extract_dir) extracted_files = [x.replace(extract_dir, '') for x in glob("%s/tmp/*" % extract_dir)] [self.assertTrue(x in files) for x in extracted_files] # Clean up for dirname in [extract_dir, creation_dir]: shutil.rmtree(dirname) print("Case 2: Testing tar...") creation_dir = tempfile.mkdtemp() archive, files = create_test_tar(creation_dir, compressed=False) # Extract to different directory extract_dir = tempfile.mkdtemp() extract_tar(archive=archive, output_folder=extract_dir) extracted_files = [x.replace(extract_dir, '') for x in glob("%s/tmp/*" % extract_dir)] [self.assertTrue(x in files) for x in extracted_files] print("Case 3: Testing that extract_tar returns None on error...") creation_dir = tempfile.mkdtemp() archive, files = create_test_tar(creation_dir, compressed=False) extract_dir = tempfile.mkdtemp() shutil.rmtree(extract_dir) output = extract_tar(archive=archive, output_folder=extract_dir) self.assertEqual(output, None) def test_write_read_files(self): '''test_write_read_files will test the functions write_file and read_file ''' print("Testing utils.write_file...") from sutils import write_file import json tmpfile = tempfile.mkstemp()[1] os.remove(tmpfile) write_file(tmpfile, "hello!") self.assertTrue(os.path.exists(tmpfile)) print("Testing utils.read_file...") from sutils import read_file content = read_file(tmpfile)[0] self.assertEqual("hello!", content) from sutils import write_json print("Testing utils.write_json...") print("Case 1: Providing bad json") bad_json = {"Wakkawakkawakka'}": [{True}, "2", 3]} tmpfile = tempfile.mkstemp()[1] os.remove(tmpfile) with self.assertRaises(TypeError) as cm: write_json(bad_json, tmpfile) print("Case 2: Providing good json") good_json = {"Wakkawakkawakka": [True, "2", 3]} tmpfile = tempfile.mkstemp()[1] os.remove(tmpfile) write_json(good_json, tmpfile) content = json.load(open(tmpfile, 'r')) self.assertTrue(isinstance(content, dict)) self.assertTrue("Wakkawakkawakka" in content) def test_clean_path(self): '''test_clean_path will test the clean_path function ''' print("Testing utils.clean_path...") from sutils import clean_path ideal_path = '/home/vanessa/stuff' self.assertEqual(clean_path('/home/vanessa/stuff/'), ideal_path) self.assertEqual(clean_path('/home/vanessa/stuff//'), ideal_path) self.assertEqual(clean_path('/home/vanessa//stuff/'), ideal_path) def test_get_fullpath(self): '''test_get_fullpath will test the get_fullpath function ''' print("Testing utils.get_fullpath...") from sutils import get_fullpath tmpfile = tempfile.mkstemp()[1] print("Case 1: File exists, should return full path") self.assertEqual(get_fullpath(tmpfile), tmpfile) print("Case 2: File doesn't exist, should return error") os.remove(tmpfile) with self.assertRaises(SystemExit) as cm: get_fullpath(tmpfile) self.assertEqual(cm.exception.code, 1) print("Case 3: File doesn't exist, should return None") self.assertEqual(get_fullpath(tmpfile, required=False), None) def test_write_singularity_infos(self): '''test_get_fullpath will test the get_fullpath function ''' print("Testing utils.write_singuarity_infos...") from sutils import write_singularity_infos base_dir = '%s/ROOTFS' % self.tmpdir prefix = 'docker' start_number = 0 content = "export HELLO=MOTO" print("Case 1: Metadata base doesn't exist, should return error") with self.assertRaises(SystemExit) as cm: info_file = write_singularity_infos(base_dir=base_dir, prefix=prefix, start_number=start_number, content=content) self.assertEqual(cm.exception.code, 1) print("Case 2: Metadata base does exist, should return path.") os.mkdir(base_dir) info_file = write_singularity_infos(base_dir=base_dir, prefix=prefix, start_number=start_number, content=content) self.assertEqual(info_file, "%s/%s-%s" % (base_dir, start_number, prefix)) print("Case 3: Adding another equivalent prefix should return next") info_file = write_singularity_infos(base_dir=base_dir, prefix=prefix, start_number=start_number, content=content) self.assertEqual(info_file, "%s/%s-%s" % (base_dir, start_number+1, prefix)) print("Case 4: Files have correct content.") with open(info_file, 'r') as filey: written_content = filey.read() self.assertEqual(content, written_content) # Supporting Test Functions def create_test_tar(tmpdir, compressed=True): archive = "%s/toodles.tar.gz" % tmpdir if compressed is False: archive = "%s/toodles.tar" % tmpdir mode = "w:gz" if compressed is False: mode = "w" print("Creating %s" % archive) tar = tarfile.open(archive, mode) files = [tempfile.mkstemp()[1] for x in range(3)] [tar.add(x) for x in files] tar.close() return archive, files if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_helpers.py0000644000175000017500000000750013211621077022202 0ustar mehdimehdi''' test_helpers.py: Helpers testing functions for Singularity in Python Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import sys sys.path.append('..') # noqa from unittest import TestCase import shutil import tempfile from subprocess import ( Popen, PIPE, STDOUT ) VERSION = sys.version_info[0] print("*** PYTHON VERSION %s UTIL HELPERS TESTING START ***" % VERSION) class TestJson(TestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() self.here = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)) self.file = "%s/meatballs.json" % self.tmpdir print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_docker_size(self): '''test the function to return Docker size ''' print('Testing Docker Size') from sutils import read_file sha256 = "476959f29a17423a24a17716e058352ff" sha256 += "6fbf13d8389e4a561c8ccc758245937" sha256 = "docker://debian@sha256:%s" % sha256 os.environ['SINGULARITY_CONTAINER'] = sha256 os.environ['SINGULARITY_CONTENTS'] = self.file script_path = "%s/size.py" % self.here if VERSION == 2: testing_command = ["python2", script_path] else: testing_command = ["python3", script_path] output = Popen(testing_command, stderr=STDOUT, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 0) result = int(read_file(self.file)[0]) self.assertTrue(250 <= result <= 251) def test_shub_size(self): '''test the function to return Singularity Hub Image Size ''' print('Testing Singularity Hub Size') from sutils import read_file shub_cont = "shub://vsoch/singularity-hello-world" os.environ['SINGULARITY_CONTAINER'] = shub_cont os.environ['SINGULARITY_CONTENTS'] = self.file script_path = "%s/size.py" % self.here if VERSION == 2: testing_command = ["python2", script_path] else: testing_command = ["python3", script_path] print(' '.join(testing_command)) output = Popen(testing_command, stderr=STDOUT, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} if result['return_code'] != 0: print(result['message']) self.assertEqual(result['return_code'], 0) result = read_file(self.file)[0] self.assertEqual('260', result) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_base.py0000644000175000017500000000361613211621077021456 0ustar mehdimehdi''' test_base.py: Test client initialized with python modules Copyright (c) 2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import re import sys import tarfile sys.path.append('..') # noqa from unittest import TestCase from base import ApiConnection import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s BASE TESTING START ***" % VERSION) class TestShell(TestCase): def setUp(self): self.client = ApiConnection() print("\n---START----------------------------------------") def tearDown(self): print("---END------------------------------------------") def test_client_headers(self): '''test_load_client will load an empty client ''' print("Testing client default headers") required = ['Accept', 'Content-Type'] for required_header in required: self.assertTrue(required_header in self.client.headers) self.assertEqual(self.client.update_token(), None) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_docker_import.py0000644000175000017500000000512213211621077023377 0ustar mehdimehdi''' test_docker_import_user.py: Docker testing functions for Singularity in Python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import sys sys.path.append('..') # noqa from subprocess import ( Popen, PIPE, STDOUT ) from unittest import TestCase import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s DOCKER IMPORT TESTING START ***" % VERSION) class TestImport(TestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() self.contents_file = "%s/hello-kitty" % self.tmpdir self.here = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)) # Variables are obtained from environment os.environ["SINGULARITY_CONTAINER"] = "docker://ubuntu:latest" os.environ["SINGULARITY_CONTENTS"] = self.contents_file print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_IMPORT(self): '''test_import will test the IMPORT USER function ''' script_path = "%s/import.py" % self.here if VERSION == 2: testing_command = ["python2", script_path] else: testing_command = ["python3", script_path] output = Popen(testing_command, stderr=STDOUT, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} self.assertEqual(result['return_code'], 0) self.assertTrue(os.path.exists(self.contents_file)) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/tests/test_shub_pull.py0000644000175000017500000000530513211621077022536 0ustar mehdimehdi''' test_shub_pull.py: Singularity Hub testing functions for Singularity in Python Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import sys sys.path.append('..') # noqa from subprocess import ( Popen, PIPE, STDOUT ) from unittest import TestCase import shutil import tempfile VERSION = sys.version_info[0] print("*** PYTHON VERSION %s SINGULARITY HUB PULL TESTING START ***" % VERSION) class TestImport(TestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() self.here = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)) # Variables are obtained from environment os.environ["SINGULARITY_CONTAINER"] = "shub://vsoch/singularity-images" os.environ["SINGULARITY_PULLFOLDER"] = self.tmpdir os.environ["SINGULARITY_CONTENTS"] = "%s/.layers" % self.tmpdir print("\n---START----------------------------------------") def tearDown(self): shutil.rmtree(self.tmpdir) print("---END------------------------------------------") def test_PULL(self): '''test_PULL will test the PULL function ''' script_path = "%s/pull.py" % self.here if VERSION == 2: testing_command = ["python2", script_path] else: testing_command = ["python3", script_path] print(' '.join(testing_command)) output = Popen(testing_command, stderr=STDOUT, stdout=PIPE) t = output.communicate()[0], output.returncode result = {'message': t[0], 'return_code': t[1]} if result['return_code'] != 0: print(result['message']) self.assertEqual(result['return_code'], 0) if __name__ == '__main__': unittest.main() singularity-2.4.2/libexec/python/pull.py0000644000175000017500000000450613211621077017316 0ustar mehdimehdi#!/usr/bin/env python ''' pull.py: general "pull" wrapper for Singularity Hub command line tool. Currently, only supported endpoint is shub:// ENVIRONMENTAL VARIABLES that are found for this executable: SINGULARITY_CONTAINER: container name: shub://vsoch/singularity-images SINGULARITY_PULLFOLDER: location to pull image to SINGULARITY_METADATA_DIR: if defined, write paths to files here Copyright (c) 2016-2017, Vanessa Sochat. All rights reserved. "Singularity" Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import sys from shell import ( get_image_uri, remove_image_uri ) from message import bot import sys def main(): '''main is a wrapper for the client to hand the parser to the executable functions This makes it possible to set up a parser in test cases ''' bot.debug("\n*** STARTING SINGULARITY PYTHON PULL ****") from defaults import LAYERFILE, DISABLE_CACHE, getenv # What image is the user asking for? container = getenv("SINGULARITY_CONTAINER", required=True) pull_folder = getenv("SINGULARITY_PULLFOLDER") image_uri = get_image_uri(container) container = remove_image_uri(container, quiet=True) if image_uri == "shub://": from shub.main import PULL manifest = PULL(image=container, download_folder=pull_folder, layerfile=LAYERFILE) else: bot.error("uri %s is not supported for pull. Exiting." % (image_uri)) sys.exit(1) if __name__ == '__main__': main() singularity-2.4.2/Makefile.am0000644000175000017500000000205013211621077015060 0ustar mehdimehdiSUBDIRS = bin etc libexec man src MAINTAINERCLEANFILES = Makefile.in aclocal.m4 configure ltmain.sh depcomp install-sh missing config.* *.m4 singularity-*.tar.gz singularity-*.rpm m4/* test.sh DISTCLEANFILES = Makefile test.sh CLEANFILES = EXTRA_DIST = singularity.spec autogen.sh examples debian CONTRIBUTORS.md CONTRIBUTING.md COPYRIGHT.md INSTALL.md LICENSE-LBNL.md LICENSE.md README.md tests maintainer-clean-local: rm -rf m4 distclean-local: maintainer-clean-local test: test.sh sh ./test.sh install-perms: @echo @echo "install-perms is no longer required" @echo install-data-hook: install -d -m 0755 $(DESTDIR)$(CONTAINER_MOUNTDIR) install -d -m 0755 $(DESTDIR)$(CONTAINER_FINALDIR) install -d -m 0755 $(DESTDIR)$(CONTAINER_OVERLAY) install -d -m 0755 $(DESTDIR)$(SESSIONDIR) test -f $(DESTDIR)$(libexecdir)/singularity/sexec-suid && rm -f $(DESTDIR)$(libexecdir)/singularity/sexec-suid || : test -f $(DESTDIR)$(libexecdir)/singularity/bin/copy-suid && rm -f $(DESTDIR)$(libexecdir)/singularity/bin/copy-suid || : ACLOCAL_AMFLAGS = -I m4 singularity-2.4.2/examples/0000755000175000017500000000000013211621077014645 5ustar mehdimehdisingularity-2.4.2/examples/shub/0000755000175000017500000000000013211621077015606 5ustar mehdimehdisingularity-2.4.2/examples/shub/Singularity0000644000175000017500000000034513211621077020045 0ustar mehdimehdiBootStrap: shub From: GodloveD/busybox # This is a comment %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" echo "Install additional software here" singularity-2.4.2/examples/raspbian/0000755000175000017500000000000013211621077016444 5ustar mehdimehdisingularity-2.4.2/examples/raspbian/Singularity0000644000175000017500000000030313211621077020675 0ustar mehdimehdiBootStrap: debootstrap OSVersion: stable MirrorURL: http://ftp.acc.umu.se/mirror/raspbian/raspbian/ %post apt-get update apt-get -y install gcc vim apache2 apt-get clean exit 0 singularity-2.4.2/examples/asciinema/0000755000175000017500000000000013211621077016576 5ustar mehdimehdisingularity-2.4.2/examples/asciinema/Singularity0000644000175000017500000000046513211621077021040 0ustar mehdimehdiBootStrap: docker From: ubuntu:latest %post apt-get update apt-get -y install python3-pip locales pip3 install asciinema locale-gen en_US.UTF-8 %environment LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8 export LANG LANGUAGE LC_ALL %runscript exec asciinema "$@" singularity-2.4.2/examples/centos/0000755000175000017500000000000013211621077016140 5ustar mehdimehdisingularity-2.4.2/examples/centos/Singularity0000644000175000017500000000110713211621077020374 0ustar mehdimehdiBootStrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/ Include: yum # If you want the updates (available at the bootstrap date) to be installed # inside the container during the bootstrap instead of the General Availability # point release (7.x) then uncomment the following line #UpdateURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/updates/$basearch/ %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" yum -y install vim-minimal singularity-2.4.2/examples/README.md0000644000175000017500000000317013211621077016125 0ustar mehdimehdi# Examples The example bootstrap definition files, each called `Singularity`, are located in their respectively named folders in this directory. These files can be used to create new container images on a variety of Linux distributions or become the basis for customization to build reproducible containers for a specific purpose. While many of these examples use core mirrors and OS distributions, keep in mind that you can use a Docker bootstrap to create almost any of them. ## Contributing If you have a specific scientific (or other) container, we suggest that you consider [singularity hub](https://singularity-hub.org) to serve it. If you do not intend to build or use the container, or want to provide a base template, then you might also want to send a pull request to add it here. ### contrib If you wish to contribute a definition file that does not fall within one of the folders here, it should go into [contrib](contrib). In this case, please send a pull request and contribute it to the examples/contribs directory with the format being hyphen ('-') delimited of the following format: 1. Base distribution name and version if applicable (e.g. centos7 or ubuntu_trusty) 2. Target nomenclature that describes the container (e.g. tensorflow) 3. Any relevant version strings to the application or work-flow 4. Always end in .def An example of this: examples/contrib/debian84-tensorflow-0.10.def ### base If your contribution is more appropriate for one of the base or template distributions, then please make a respective folder in the [examples](.) directory, and name the definition file `Singularity`. singularity-2.4.2/examples/ubuntu/0000755000175000017500000000000013211621077016167 5ustar mehdimehdisingularity-2.4.2/examples/ubuntu/Singularity0000644000175000017500000000046113211621077020425 0ustar mehdimehdiBootStrap: debootstrap OSVersion: trusty MirrorURL: http://us.archive.ubuntu.com/ubuntu/ %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" sed -i 's/$/ universe/' /etc/apt/sources.list apt-get -y --force-yes install vim singularity-2.4.2/examples/arch/0000755000175000017500000000000013211621077015562 5ustar mehdimehdisingularity-2.4.2/examples/arch/README.md0000644000175000017500000000243013211621077017040 0ustar mehdimehdi# Arch for Singularity This bootstrap spec will generate an arch linux distribution using Singularity 2.3 (current development branch). Note that you can also just bootstrap a Docker image: If you want to move forward with the raw, old school, jeans and hard toes bootstrap, here is what to do. I work on an Ubuntu machine, so I had to use a Docker Arch Linux image to do this. This first part you should do on your local machine (if not arch linux) is to use Docker to interactively work in an Arch Linux image. If you don't want to copy paste the build spec file, you can use `--volume` to mount a directory from your host to a folder in the image (I would recommend `/tmp` or similar). Here we run the docker image: ```bash docker run -it --privileged pritunl/archlinux bash ``` ```bash pacman -S -y git autoconf libtool automake gcc python make sudo vim arch-install-scripts wget git clone https://github.com/singularityware/singularity cd singularity git checkout -b development git pull origin development ./autogen.sh ./configure --prefix=/usr/local ``` You can add the [Singularity](Singularity) build spec here, or cd to where it is if you have mounted a volume. ```bash cd /tmp singularity create arch.img sudo singularity bootstrap arch.img Singularity ``` That should do the trick! singularity-2.4.2/examples/arch/Singularity0000644000175000017500000000331313211621077020017 0ustar mehdimehdi# Copyright (c) 2015-2016, Maciej Sieczka, Gregory M. Kurtzer. All rights # reserved. # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Minimal installation process is defined in # libexec/bootstrap/modules-v2/dist-arch.sh. A couple extra actions are called # from here in `%post' section. Adjust them as needed. # https://wiki.archlinux.org/index.php/Installation_Guide may come in handy. Bootstrap: arch %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" # Set time zone. Use whatever you prefer instead of UTC. ln -s /usr/share/zoneinfo/UTC /etc/localtime # Set locale. Use whatever you prefer instead of en_US. echo 'en_US.UTF-8 UTF-8' > /etc/locale.gen # Add more locales as needed, eg: # echo 'pl_PL.UTF-8 UTF-8' >> /etc/locale.gen locale-gen echo 'LANG=en_US.UTF-8' > /etc/locale.conf # Mind that Singularity's shell will use host's locale no matter what # anyway, as of version 2.1.2. This may change in a future release. # Set the package mirror server(s). This is only for the output image's # mirrorlist. `pacstrap' can only use your hosts's package mirrors. echo 'Server = https://mirrors.kernel.org/archlinux/$repo/os/$arch' > /etc/pacman.d/mirrorlist # Add any number of fail-over servers, eg: echo 'Server = https://archlinux.honkgong.info/$repo/os/$arch' >> /etc/pacman.d/mirrorlist # I need VIM and Bash completion. Specify your extra packages as needed. pacman -Sy --noconfirm vim bash-completion # Remove the packages downloaded to image's Pacman cache dir. paccache -r -k0 singularity-2.4.2/examples/scientific/0000755000175000017500000000000013211621077016765 5ustar mehdimehdisingularity-2.4.2/examples/scientific/Singularity0000644000175000017500000000043713211621077021226 0ustar mehdimehdiBootStrap: yum OSVersion: 7 MirrorURL: http://ftp.scientificlinux.org/linux/scientific/%{OSVERSION}x/$basearch/os/ Include: yum %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" yum -y install vim-minimal singularity-2.4.2/examples/busybox/0000755000175000017500000000000013211621077016340 5ustar mehdimehdisingularity-2.4.2/examples/busybox/Singularity0000644000175000017500000000033613211621077020577 0ustar mehdimehdiBootStrap: busybox MirrorURL: https://www.busybox.net/downloads/binaries/1.26.1-defconfig-multiarch/busybox-x86_64 %post echo "Hello from inside the container" %runscript echo "Running command: $*" exec "$@" singularity-2.4.2/examples/docker/0000755000175000017500000000000013211621077016114 5ustar mehdimehdisingularity-2.4.2/examples/docker/Singularity0000644000175000017500000000034513211621077020353 0ustar mehdimehdiBootStrap: docker From: busybox:latest # This is a comment %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" echo "Install additional software here" singularity-2.4.2/examples/legacy/0000755000175000017500000000000013211621077016111 5ustar mehdimehdisingularity-2.4.2/examples/legacy/2.2/0000755000175000017500000000000013211621077016412 5ustar mehdimehdisingularity-2.4.2/examples/legacy/2.2/debian.def0000644000175000017500000000107513211621077020317 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: debootstrap OSVersion: stable MirrorURL: http://ftp.us.debian.org/debian/ %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" apt-get update apt-get -y --force-yes install vim singularity-2.4.2/examples/legacy/2.2/scientific.def0000644000175000017500000000111613211621077021211 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: yum OSVersion: 7 MirrorURL: http://ftp.scientificlinux.org/linux/scientific/%{OSVERSION}x/$basearch/os/ Include: yum %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" yum -y install vim-minimal singularity-2.4.2/examples/legacy/2.2/README0000644000175000017500000000140413211621077017271 0ustar mehdimehdiThe example bootstrap definition files listed in this directory can be used to create new container images on a variety of Linux distributions or become the basis for customization to build reproducible containers for a specific purpose. If you wish to contribute a definition file, please send a pull request and contribute it to the examples/contribs/ directory with the format being hyphen ('-') delimited of the following format: 1. Base distribution name and version if applicable (e.g. centos7 or ubuntu_trusty) 2. Target nomenclature that describes the container (e.g. tensorflow) 3. Any relevant version strings to the application or work-flow 4. Always end in .def An example of this: examples/contrib/debian84-tensorflow-0.10.def singularity-2.4.2/examples/legacy/2.2/contrib/0000755000175000017500000000000013211621077020052 5ustar mehdimehdisingularity-2.4.2/examples/legacy/2.2/contrib/ubuntu16-tensorflow-0.12.1-gpu.def0000644000175000017500000001126213211621077025733 0ustar mehdimehdi# Defines a Singularity container with TensorFlow pre-installed # # # Before bootstrapping this container, you must ensure that the following files # are present in the current directory (alongside this definition file): # # * cuda-linux64-rel-8.0.44-21122537.run (* see below) # * NVIDIA-Linux-x86_64-375.20.run (* see below) # * cudnn-8.0-linux-x64-v5.1.tgz (https://developer.nvidia.com/cudnn) # # * The cuda-linux64 and NVIDIA-Linux files can be obtained by downloading the # NVIDIA CUDA local runfile `cuda_8.0.44_linux.run` from: # # https://developer.nvidia.com/cuda-downloads # # Then extract the necessary files by running: # # sh cuda_8.0.44_linux.run --extract= # # IF YOUR HPC SYSTEM IS USING A DIFFERENT VERSION OF CUDA AND/OR NVIDIA DRIVERS # YOU WILL NEED TO ADJUST THE ABOVE VERSION NUMBERS TO MATCH YOUR SYSTEM # # YOU WILL ALSO NEED TO DOWNLOAD THE APPROPRIATE DRIVER. For example, # cuda_8.0.44_linux.run returns driver version 367.48. # # If you use this to create a container inside a virtual machine with no access to # a GPU, comment out the final test. BootStrap: docker From: ubuntu:16.10 %runscript # When executed, the container will run Python with the TensorFlow module # Check the current environment chk_nvidia_uvm=$(grep nvidia_uvm /proc/modules) if [ -z "$chk_nvidia_uvm" ]; then echo "Problem detected on the host: the Linux kernel module nvidia_uvm is not loaded" exit 1 fi exec /usr/bin/python "$@" %setup # Runs from outside the container during Bootstrap NV_DRIVER_VERSION=375.20 NV_CUDA_FILE=cuda-linux64-rel-8.0.44-21122537.run NV_CUDNN_FILE=cudnn-8.0-linux-x64-v5.1.tgz NV_DRIVER_FILE=NVIDIA-Linux-x86_64-${NV_DRIVER_VERSION}.run working_dir=$(pwd) echo "Unpacking NVIDIA driver into container..." cd ${SINGULARITY_ROOTFS}/usr/local/ sh ${working_dir}/${NV_DRIVER_FILE} -x mv NVIDIA-Linux-x86_64-${NV_DRIVER_VERSION} NVIDIA-Linux-x86_64 cd NVIDIA-Linux-x86_64/ for n in *.$NV_DRIVER_VERSION; do ln -v -s $n ${n%.367.48} done ln -v -s libnvidia-ml.so.$NV_DRIVER_VERSION libnvidia-ml.so.1 ln -v -s libcuda.so.$NV_DRIVER_VERSION libcuda.so.1 cd $working_dir echo "Running NVIDIA CUDA installer..." sh $NV_CUDA_FILE -noprompt -nosymlink -prefix=${SINGULARITY_ROOTFS}/usr/local/cuda-8.0 ln -r -s ${SINGULARITY_ROOTFS}/usr/local/cuda-8.0 ${SINGULARITY_ROOTFS}/usr/local/cuda echo "Unpacking cuDNN..." tar xvf $NV_CUDNN_FILE -C ${SINGULARITY_ROOTFS}/usr/local/ echo "Adding NVIDIA PATHs to /environment..." NV_DRIVER_PATH=/usr/local/NVIDIA-Linux-x86_64 echo " LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:$NV_DRIVER_PATH:\$LD_LIBRARY_PATH PATH=$NV_DRIVER_PATH:\$PATH export PATH LD_LIBRARY_PATH " >> $SINGULARITY_ROOTFS/environment %post # Runs within the container during Bootstrap # Set up some required environment defaults export LC_ALL=C export PATH=/bin:/sbin:/usr/bin:/usr/sbin:$PATH # Install the necessary packages (from repo) apt-get update && apt-get install -y --no-install-recommends \ build-essential \ curl \ git \ libcurl4-openssl-dev \ libfreetype6-dev \ libpng-dev \ libzmq3-dev \ python-pip \ pkg-config \ python-dev \ rsync \ software-properties-common \ unzip \ zip \ zlib1g-dev apt-get clean # Update to the latest pip (newer than repo) pip install --no-cache-dir --upgrade pip # Install other commonly-needed packages pip install --no-cache-dir --upgrade \ future \ matplotlib \ scipy \ sklearn # TensorFlow package versions as listed here: # https://www.tensorflow.org/get_started/os_setup#test_the_tensorflow_installation # # Ubuntu/Linux 64-bit, GPU enabled, Python 2.7 (Requires CUDA toolkit 8.0 and CuDNN v5) export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.1-cp27-none-linux_x86_64.whl pip install --no-cache-dir --ignore-installed --upgrade $TF_BINARY_URL %test # Sanity check that the container is operating export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/NVIDIA-Linux-x86_64:$LD_LIBRARY_PATH # Ensure that TensorFlow can be imported /usr/bin/python -c "import tensorflow as tf" # Runs in less than 30 minutes on low-end CPU; in less than 2 minutes on GPU # Comment the following line if building the container inside a VM with no access to a GPU /usr/bin/python -m tensorflow.models.image.mnist.convolutional singularity-2.4.2/examples/legacy/2.2/contrib/linuxbrew_and_non-root_software_example.def0000644000175000017500000000251713211621077030660 0ustar mehdimehdiBootStrap: debootstrap OSVersion: trusty MirrorURL: http://us.archive.ubuntu.com/ubuntu/ %runscript # print out software versions installed by linuxbrew find /Software/brew/Cellar -maxdepth 2 -print | sed 's|/Software/brew/Cellar||g' | sed 's|^/||' | grep "/" | sed 's|/|\t|' | sort | awk '{print $1, $2, "Homebrew"}' | column -t | sort -u --ignore-case %post sed -i 's/$/ universe/' /etc/apt/sources.list locale-gen "en_US.UTF-8" dpkg-reconfigure locales export LANGUAGE="en_US.UTF-8" echo 'LANGUAGE="en_US.UTF-8"' >> /etc/default/locale echo 'LC_ALL="en_US.UTF-8"' >> /etc/default/locale mkdir /Software chmod 777 /tmp chmod +t /tmp chmod 777 /Software apt-get update apt-get install -y apt-transport-https build-essential cmake curl libsm6 libxrender1 libfontconfig1 wget vim git unzip python-setuptools ruby apt-get clean useradd -m singularity su -c 'cd /Software && git clone https://github.com/Linuxbrew/brew.git' singularity su -c '/Software/brew/bin/brew install bsdmainutils parallel util-linux' singularity su -c '/Software/brew/bin/brew tap homebrew/science' singularity su -c '/Software/brew/bin/brew install art bwa samtools' singularity sed -i 's|PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin|PATH="/Software/brew/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"|' /environment singularity-2.4.2/examples/legacy/2.2/contrib/centos-minimal.def0000644000175000017500000000113113211621077023445 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/ %post echo "Hello from inside the container" %test echo "This is the test code running" | sed -e 's/code //' false || true # This will fail without the double pipe true singularity-2.4.2/examples/legacy/2.2/contrib/centos7-ompi_cuda.def0000644000175000017500000002100413211621077024047 0ustar mehdimehdi# Copyright (c) 2017, Paulo Souza. All rights reserved. # # This definition file needs a 2GB image file. # # What is included: # - Infiniband drivers # - CUDA 7.5 runtime # - CUDA-aware OpenMPI # - Linux perf tool # - Dstat tool # # We do recommend a CentOS 7.3 host to build this image. # # How to build the image: # sudo singularity create -s 2000 centos7-ompi_cuda.img # sudo singularity bootstrap centos7-ompi_cuda.img centos7-ompi_cuda.def # # After image creation you do not need sudo to use the image. # BootStrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/ Include: yum # If you want the updates (available at the bootstrap date) to be installed # inside the container during the bootstrap instead of the General Availability # point release (7.x) then uncomment the following line #UpdateURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/updates/$basearch/ # runscript: %runscript export LD_LIBRARY_PATH=/liboverride:$LD_LIBRARY_PATH:/libfallback export LD_RUN_PATH=/liboverride:$LD_RUN_PATH:/libfallback eval "$@" # post: %post export ctnlicense=/LICENSE mkdir -p /app /liboverride /libfallback /prj /localscratch /localflash /opt $ctnlicense alias install_pkg="yum -y install" # Reference: https://www.liquidweb.com/kb/enable-epel-repository/ install_pkg epel-release yum -y update alias clean_pkg='echo "clean not needed."' # dev-tools: yum -y install gcc yum -y install gcc-c++ yum -y install gcc-gfortran yum -y install python-devel install_pkg make install_pkg cmake install_pkg autoconf # basic-tools: install_pkg vim install_pkg curl hostname install_pkg wget install_pkg unzip install_pkg tar install_pkg gzip install_pkg bc install_pkg less install_pkg util-linux install_pkg strace install_pkg which perl-Digest-SHA man # perf-tools: yum install -y flex flex-devel yum install -y bison bison-devel yum install -y elfutils-libelf-devel elfutils-libelf yum install -y systemtap-sdt-devel yum install -y audit-libs audit-libs-devel audit-libs-python yum install -y openssl-devel openssl openssl-libs yum install -y slang slang-devel slang-slsh yum install -y perl perl-ExtUtils-Embed yum install -y xz-devel yum install -y numactl numactl-devel numactl-libs yum install -y cpuid yum install -y elfutils elfutils-devel elfutils-libelf elfutils-libelf-devel elfutils-libs yum install -y libunwind libunwind-devel yum install -y binutils binutils-devel yum install -y e2fsprogs #install_pkg dstat #install_pkg perf echo '2ca34f40067b0c914dbe474bd4ce44dbaec068d684d870c93953480062ce0e42ae32d70392ed572b1cf683ef06683f73c9d98e883405fb75ccbf614a732ce009 -' > dstat.zip.sha512 curl -sSL "https://github.com/dagwieers/dstat/archive/0.7.3.zip" | tee dstat.zip | sha512sum -c dstat.zip.sha512 unzip dstat.zip && rm -d dstat.zip* cat dstat-*/COPYING > $ctnlicense/dstat.COPYING cd dstat-* make install cd - rm -rf dstat-* echo '5aee3e0b77ff9895ee7e214552c14124f90defa72f15811db81a89e7e09c66ee643a3d1c90a97358b049c863ebe8a811355d758729ca6012bc52e822e54cc044 -' > perf.zip.sha512 curl -sSL "https://github.com/torvalds/linux/archive/v4.10.zip" | tee perf.zip | sha512sum -c perf.zip.sha512 unzip perf.zip && rm -f perf.zip* cat linux-*/COPYING > $ctnlicense/perf.COPYING cd linux-*/tools/perf make mv perf /usr/bin/ cd - rm -rf linux-* echo '600a0161d7eca085594191b133cb25ec6bb6db41d0857cfac7f6b3bdf45647961f329c4cd423fd3dacb34fd09183b762c0b70bcd7b06a878666c8a241c3a7e53 -' > pmu-tools.zip.sha512 curl -sSL "https://github.com/andikleen/pmu-tools/archive/r105.zip" | tee pmu-tools.zip | sha512sum -c pmu-tools.zip.sha512 unzip pmu-tools.zip -d /app && rm -rf pmu-tools.zip* cat /app/pmu-tools*/COPYING > $ctnlicense/pmu-tools.COPYING echo '739dc803dd90b34e7b0a817eeb32d9a737a6809e8f2ad6bcfca16505d4934c1ac62b6fec1e90f59cbf6aabf8d38852c980115c9c47758c9fb3b0539dc2f6abb2 -' > likwid.tgz.sha512 curl -sSL "http://ftp.fau.de/pub/likwid/likwid-stable.tar.gz" |tee likwid.tgz|sha512sum -c likwid.tgz.sha512 tar -zxvf likwid.tgz && rm -f likwid.tgz* cd likwid-* cat COPYING > $ctnlicense/likwid.COPYING make make install cd - rm -rf likwid-* # perf reference: http://www.tecmint.com/perf-performance-monitoring-and-analysis-tool-for-linux/ # Additionally, keep in mind that some perf commands may be restricted to # `root by default, which can be disabled (until the system is rebooted) by doing: # echo 0 > /proc/sys/kernel/perf_event_paranoid # If you need to disable paranoid mode permanently, update the following setting in /etc/sysctl.conf file. # kernel.perf_event_paranoid = 0 # /proc/sys/kernel/kptr_restrict # sudo sysctl -w kernel.perf_event_paranoid=0 # infiniband: # Infiniband + RDMA Reference: # https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-InfiniBand_and_RDMA_related_software_packages.html yum -y install rdma yum -y install libibverbs-devel libsysfs-devel # required by custom openmpi yum -y install infinipath-psm libcxgb3 libcxgb4 libipathverbs libmthca libmlx4 libmlx5 libnes libocrdma yum -y install librdmacm librdmacm-utils ibacm yum -y install libibverbs-utils yum -y install infiniband-diags ibutils yum -y install perftest qperf #required by OpenMPI yum install -y numactl numactl-devel numactl-libs yum install -y environment-modules hwloc hwloc-libs hwloc-devel libfabric libpsm2 libfabric-devel # cuda75: echo '40315500010e078d7cfb71534ac8d0a52c5da9dfbe24ecb6fd7c7d1dcef60ae0db309a51925e395c13af390327f5a9f4682516d9d4994beaa3684b69d459b96b -' > cuda-repo.rpm.sha512 curl -sSL "http://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-repo-rhel7-7.5-18.x86_64.rpm" |\ tee cuda-repo.rpm | sha512sum -c cuda-repo.rpm.sha512 rpm -i cuda-repo.rpm && rm -f cuda-repo.rpm* yum -y update export cuda_ver=7-5 export CUDA_VERSION=7.5.18 install_pkg cuda-minimal-build-$cuda_ver install_pkg cuda-command-line-tools-$cuda_ver ln -s /usr/local/cuda-*/ /usr/local/cuda export CUDA_PATH=/usr/local/cuda export CUDA_HOME=$CUDA_PATH export CUDA_INC=$CUDA_PATH/include export CUDADIR=$CUDA_PATH export CUDA_ROOT=$CUDA_PATH env | grep "^CUDA.*="|sed -e "s/^/export /" >> /environment echo 'export PATH=$CUDA_PATH/bin:$PATH' >> /environment echo 'export LD_LIBRARY_PATH=$CUDA_PATH/lib64:$LD_LIBRARY_PATH' >> /environment export LD_LIBRARY_PATH=$CUDA_PATH/lib64:$LD_LIBRARY_PATH cat /usr/local/cuda/doc/EULA.txt > $ctnlicense/CUDA.EULA.txt # cuda-aware-mpi: echo '970e48d19b544a401511a1fe2c6ff9655593607ef5db8cfcb3a913e34d2362d58fe537c356642bd98d2f2355daf3556d7ca37e1b90c14ceabbaae73840716cdb -' > openmpi.tgz.sha512 curl -sSL "https://www.open-mpi.org/software/ompi/v2.0/downloads/openmpi-2.0.2.tar.gz" | \ tee openmpi.tgz | sha512sum -c openmpi.tgz.sha512 tar -zxvf openmpi.tgz && rm -f openmpi.tgz* cd openmpi-* cat LICENSE > $ctnlicense/OpenMPI.LICENSE ./configure --prefix=/app/openmpi --with-verbs --with-cuda \ CFLAGS=-I$CUDA_PATH/include CPPFLAGS=-I$CUDA_PATH/include \ CXXFLAGS=-I$CUDA_PATH/include LDFLAGS=-L$CUDA_PATH/lib64 make make install cd - rm -rf openmpi-* export PATH=/app/openmpi/bin:$PATH export LD_LIBRARY_PATH=/app/openmpi/lib:$LD_LIBRARY_PATH export LD_RUN_PATH=/app/openmpi/lib:$LD_RUN_PATH echo 'export PATH=/app/openmpi/bin:$PATH' >> /environment echo 'export LD_LIBRARY_PATH=/app/openmpi/lib:$LD_LIBRARY_PATH' >> /environment echo 'export LD_RUN_PATH=/app/openmpi/lib:$LD_RUN_PATH' >> /environment echo '74561a004160e064b44848044c9595b337337b60d8f23c87608cbbceee52e8737f4770c2d8fc003f9e0478ea13e6451e29fd13796b45803ccd56f765e6fe058c -' > osu-micro-benchmarks.tgz.sha512 curl -sSL "http://mvapich.cse.ohio-state.edu/download/mvapich/osu-micro-benchmarks-5.3.2.tar.gz" | \ tee osu-micro-benchmarks.tgz | sha512sum -c osu-micro-benchmarks.tgz.sha512 tar -zxvf osu-micro-benchmarks.tgz && rm -f osu-micro-benchmarks.tgz* cd osu-micro-benchmarks-* cat COPYRIGHT > $ctnlicense/OSU.COPYRIGHT ./configure --prefix=/app/openmpi CC=/app/openmpi/bin/mpicc CXX=/app/openmpi/bin/mpicxx make make install cd - rm -rf osu-micro-benchmarks-* ln -s /app/openmpi/libexec/osu-micro-benchmarks/mpi/*/osu_* /app/openmpi/bin/ singularity-2.4.2/examples/legacy/2.2/contrib/ubuntu-openfoam.def0000644000175000017500000000151413211621077023657 0ustar mehdimehdi# Copyright (c) 2016, Ontropos, Inc. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: debootstrap OSVersion: trusty MirrorURL: http://archive.ubuntu.com/ubuntu/ Include: bash %post apt-get -y install wget apt-transport-https sed -i 's/main/main restricted universe/g' /etc/apt/sources.list echo 'deb http://download.openfoam.org/ubuntu trusty main' >> /etc/apt/sources.list wget -O - http://dl.openfoam.org/gpg.key | apt-key add - apt-get update apt-get -y install openfoam4 echo ". /opt/openfoam4/etc/bashrc" >> /environment singularity-2.4.2/examples/legacy/2.2/contrib/fedora.def0000644000175000017500000000122213211621077021767 0ustar mehdimehdiBootStrap: yum OSVersion: 25 MetaLink: https://mirrors.fedoraproject.org/metalink?repo=fedora-%{OSVERSION}&arch=$basearch # if you want a complete basic image, add @Core Include: dnf # If you want the updates (available at the bootstrap date) to be installed # inside the container during the bootstrap instead of the General Availability # point release then uncomment the following line #UpdateMetaLink: https://mirrors.fedoraproject.org/metalink?repo=updates-released-f%{OSVERSION}&arch=$basearch %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" dnf -y install vim-minimal singularity-2.4.2/examples/legacy/2.2/contrib/r_python_julia.def0000644000175000017500000000317513211621077023566 0ustar mehdimehdiBootStrap: debootstrap OSVersion: trusty MirrorURL: http://us.archive.ubuntu.com/ubuntu/ %runscript %post sed -i 's/$/ universe/' /etc/apt/sources.list locale-gen "en_US.UTF-8" dpkg-reconfigure locales export LANGUAGE="en_US.UTF-8" echo 'LANGUAGE="en_US.UTF-8"' >> /etc/default/locale echo 'LC_ALL="en_US.UTF-8"' >> /etc/default/locale mkdir /Software apt-get update apt-get install -y build-essential cmake curl ed git libsm6 libxrender1 libfontconfig1 lsb-release nettle-dev python-setuptools ruby software-properties-common vim wget zlib1g-dev apt-transport-https libgtk-3-0 bzip2 gfortran gettext libcairo2-dev libgit2-dev apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 echo "deb https://cloud.r-project.org/bin/linux/ubuntu trusty/" >> /etc/apt/sources.list apt-get update apt-get install -y r-base-dev # can add command to install desired R packages here apt-get clean cd /Software wget http://repo.continuum.io/archive/Anaconda3-4.1.1-Linux-x86_64.sh bash Anaconda3-4.1.1-Linux-x86_64.sh -b -p /Software/anaconda3 rm Anaconda3-4.1.1-Linux-x86_64.sh export PATH="/Software/anaconda3/bin:$PATH" conda update conda conda update anaconda git clone git://github.com/JuliaLang/julia.git julia-0.5 cd julia-0.5 git checkout release-0.5 make MARCH=x86-64 cd .. export PATH="/Software/julia-0.5/bin:$PATH" # can add command to install desired julia packages here, remember to pre-compile! sed -i 's|PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin|PATH="/Software/julia-0.5:/Software/anaconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"|' /environment singularity-2.4.2/examples/legacy/2.2/contrib/ubuntu-root.def0000644000175000017500000000137213211621077023040 0ustar mehdimehdiBootstrap: docker From: ubuntu:trusty IncludeCmd: no %post ## Install dependencies for ROOT (gcc cmake etc) ## apt-get install -y software-properties-common add-apt-repository ppa:ubuntu-toolchain-r/test apt-get update apt-get install -y gcc-4.8 g++-4.8 apt-get install -y git dpkg-dev cmake binutils libx11-dev libxpm-dev libxft-dev libxext-dev wget python ## Setup for ROOT install ## mkdir /usr/local/ROOT && cd /usr/local/ROOT && mkdir 6.06.08 ## Download ROOT from CERN and install binary ## wget https://root.cern.ch/download/root_v6.06.08.Linux-ubuntu14-x86_64-gcc4.8.tar.gz tar -zxvf root_*.tar.gz -C /usr/local/ROOT/6.06.08 %runscript source /usr/local/ROOT/6.06.08/root/bin/thisroot.sh /bin/bashsingularity-2.4.2/examples/legacy/2.2/contrib/debian85-tensorflow-0.10.def0000644000175000017500000000164013211621077024706 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # SINGULARITY BOOTSTRAP: TENSORFLOW # Installation proceedure copied from: # https://www.tensorflow.org/versions/r0.10/get_started/os_setup.html#pip-installation BootStrap: debootstrap OSVersion: stable MirrorURL: http://ftp.us.debian.org/debian/ %runscript exec /usr/bin/python %post apt-get update apt-get -y install vim python-pip python-dev pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0-cp27-none-linux_x86_64.whl %test # This runs usually less then 30 minutes depending on your host type python -m tensorflow.models.image.mnist.convolutional singularity-2.4.2/examples/legacy/2.2/contrib/centos7-ompi_master.def0000644000175000017500000000164113211621077024433 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/ Include: yum %post echo "Installing Development Tools YUM group" yum -y groupinstall "Development Tools" echo "Installing OpenMPI into container..." mkdir /tmp/git cd /tmp/git git clone https://github.com/open-mpi/ompi.git cd ompi ./autogen.pl ./configure --prefix=/usr/local make make install /usr/local/bin/mpicc examples/ring_c.c -o /usr/bin/mpi_ring cd / rm -rf /tmp/git exit 0 %test /usr/local/bin/mpirun --allow-run-as-root /usr/bin/mpi_ring singularity-2.4.2/examples/legacy/2.2/contrib/ubuntu-bio.def0000644000175000017500000000342713211621077022631 0ustar mehdimehdi# Copyright (c) 2016, Ontropos, Inc. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: debootstrap OSVersion: trusty MirrorURL: http://archive.ubuntu.com/ubuntu/ Include: bash %post # Prep work mkdir -p /tmp/work cd /tmp/work sed -i 's/main/main restricted universe/g' /etc/apt/sources.list apt-get update apt-get -y install wget bzip2 build-essential zlib1g-dev software-properties-common libncurses5-dev # Install BWA wget "http://downloads.sourceforge.net/project/bio-bwa/bwa-0.7.15.tar.bz2?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fbio-bwa%2Ffiles%2F&ts=1473712599&use_mirror=heanet" -O bwa-0.7.15.tar.bz2 tar -xvjf bwa-0.7.15.tar.bz2 cd bwa-0.7.15/ make cp bwa /usr/bin/bwa cd /tmp/work # Install SAM Tools wget "http://downloads.sourceforge.net/project/samtools/samtools/1.3.1/samtools-1.3.1.tar.bz2?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fsamtools%2Ffiles%2Fsamtools%2F1.3.1%2F&ts=1473713899&use_mirror=pilotfiber" -O samtools-1.3.1.tar.bz2 tar -xvjf samtools-1.3.1.tar.bz2 cd samtools-1.3.1 ./configure make cp samtools /usr/bin/samtools # Install java prereq apt-add-repository -y ppa:webupd8team/java apt-get update echo 'oracle-java8-installer shared/accepted-oracle-license-v1-1 select true' | debconf-set-selections apt-get -y install oracle-java8-installer # This is now ready to have GATK (Genomic Analysis Took Kit) installed # from: https://software.broadinstitute.org/gatk/download/ singularity-2.4.2/examples/legacy/2.2/contrib/ubuntu16-tensorflow-0.12.1.def0000644000175000017500000000336013211621077025142 0ustar mehdimehdi# Defines a Singularity container with TensorFlow pre-installed # BootStrap: docker From: ubuntu:16.04 %runscript # When executed, the container will run Python with the TensorFlow module exec /usr/bin/python "$@" %post # Runs within the container during Bootstrap # Set up some required environment defaults export LC_ALL=C export PATH=/bin:/sbin:/usr/bin:/usr/sbin:$PATH # Install the necessary packages (from repo) apt-get update && apt-get install -y --no-install-recommends \ build-essential \ curl \ git \ libcurl3-dev \ libfreetype6-dev \ libpng12-dev \ libzmq3-dev \ python-pip \ pkg-config \ python-dev \ rsync \ software-properties-common \ unzip \ zip \ zlib1g-dev apt-get clean # Update to the latest pip (newer than repo) pip install --no-cache-dir --upgrade pip # Install other commonly-needed packages pip install --no-cache-dir --upgrade \ future \ matplotlib \ scipy \ sklearn # TensorFlow package versions as listed here: # https://www.tensorflow.org/get_started/os_setup#test_the_tensorflow_installation # # Ubuntu/Linux 64-bit, CPU only, Python 2.7 export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.12.1-cp27-none-linux_x86_64.whl pip install --no-cache-dir --ignore-installed --upgrade $TF_BINARY_URL %test # Sanity check that the container is operating # Ensure that TensorFlow can be imported /usr/bin/python -c "import tensorflow as tf" # Runs in less than 30 minutes on low-end CPU /usr/bin/python -m tensorflow.models.image.mnist.convolutional singularity-2.4.2/examples/legacy/2.2/docker.def0000644000175000017500000000102513211621077020337 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: docker From: busybox:latest # This is a comment %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" echo "Install additional software here" singularity-2.4.2/examples/legacy/2.2/busybox.def0000644000175000017500000000072613211621077020572 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: busybox MirrorURL: https://www.busybox.net/downloads/binaries/1.26.1-defconfig-multiarch/busybox-x86_64 %post echo "Hello from inside the container" singularity-2.4.2/examples/legacy/2.2/centos.def0000644000175000017500000000157113211621077020371 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/ Include: yum # If you want the updates (available at the bootstrap date) to be installed # inside the container during the bootstrap instead of the General Availability # point release (7.x) then uncomment the following line #UpdateURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/updates/$basearch/ %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" yum -y install vim-minimal singularity-2.4.2/examples/legacy/2.2/ubuntu.def0000644000175000017500000000114013211621077020410 0ustar mehdimehdi# Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. BootStrap: debootstrap OSVersion: trusty MirrorURL: http://us.archive.ubuntu.com/ubuntu/ %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" sed -i 's/$/ universe/' /etc/apt/sources.list apt-get -y --force-yes install vim singularity-2.4.2/examples/legacy/2.2/arch.def0000644000175000017500000000773513211621077020023 0ustar mehdimehdi# Copyright (c) 2016, Maciej Sieczka. All rights reserved # # Minimal installation process is defined in mods/linux_build_arch.smod. A # couple extra actions are called from here using RunCmd and InstallPkgs # Singularity bootstrap keywords. Adjust them as needed. # https://wiki.archlinux.org/index.php/Installation_Guide may come in handy. DistType "arch" # "Setup" keyword doesn't do anything anymore, at least as of Singularity # 2.1.2. Undocumented "PreSetup" replaced it and is called at each # `singularity bootstrap' run. # Setup Bootstrap # Set time zone. Use whatever you prefer instead of UTC. RunCmd ln -s /usr/share/zoneinfo/UTC /etc/localtime # WARNING: Make sure you prefix the STDOUT redirection target (ie. that guy # after `>' or `>>`) with $SINGULARITY_BUILD_ROOT in RunCmd calls, as below. # Otherwise such command will write to host's filesystem, at least as of # Singularity 2.1.2! This is a known issue, going to be fixed. See # https://github.com/gmkurtzer/singularity/issues/200. # Set locale. Use whatever you prefer instead of en_US. RunCmd echo 'en_US.UTF-8 UTF-8' > "$SINGULARITY_BUILD_ROOT"/etc/locale.gen # Add more locales as needed, eg: # RunCmd echo 'pl_PL.UTF-8 UTF-8' >> "$SINGULARITY_BUILD_ROOT"/etc/locale.gen RunCmd locale-gen RunCmd echo 'LANG=en_US.UTF-8' > "$SINGULARITY_BUILD_ROOT"/etc/locale.conf # Mind that Singularity's shell will use host's locale no matter what anyway, # as of version 2.1.2. This may change in a future release. # Set the package mirror server(s). This is only for the output image's # mirrorlist. `pacstrap' can only use your hosts's package mirrors. RunCmd echo 'Server = https://mirrors.kernel.org/archlinux/$repo/os/$arch' > "$SINGULARITY_BUILD_ROOT"/etc/pacman.d/mirrorlist # Add any number of fail-over servers, eg: RunCmd echo 'Server = https://archlinux.honkgong.info/$repo/os/$arch' >> "$SINGULARITY_BUILD_ROOT"/etc/pacman.d/mirrorlist # I need VIM and Bash completion. Specify your extra packages as needed. InstallPkgs vim bash-completion # Remove the packages downloaded to image's Pacman cache dir during # InstallPkgs. BTW, `pacstrap', used in Bootstrap step, uses host's package # cache rather than the image's. RunCmd paccache -r -k0 Cleanup ======= # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # Minimal installation process is defined in # libexec/bootstrap/modules-v2/dist-arch.sh. A couple extra actions are called # from here in '%post' section. Adjust them as needed. # https://wiki.archlinux.org/index.php/Installation_Guide may come in handy. BootStrap: arch %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" # Set time zone. Use whatever you prefer instead of UTC. ln -s /usr/share/zoneinfo/UTC /etc/localtime # Set locale. Use whatever you prefer instead of en_US. echo 'en_US.UTF-8 UTF-8' > /etc/locale.gen # Add more locales as needed, eg: # echo 'pl_PL.UTF-8 UTF-8' >> /etc/locale.gen locale-gen echo 'LANG=en_US.UTF-8' > /etc/locale.conf # Mind that Singularity's shell will use host's locale no matter what # anyway, as of version 2.1.2. This may change in a future release. # Set the package mirror server(s). This is only for the output image's # mirrorlist. `pacstrap' can only use your hosts's package mirrors. echo 'Server = https://mirrors.kernel.org/archlinux/$repo/os/$arch' > /etc/pacman.d/mirrorlist # Add any number of fail-over servers, eg: echo 'Server = https://archlinux.honkgong.info/$repo/os/$arch' >> /etc/pacman.d/mirrorlist # I need VIM and Bash completion. Specify your extra packages as needed. pacman -Sy --noconfirm vim bash-completion # Remove the packages downloaded to image's Pacman cache dir. paccache -r -k0 singularity-2.4.2/examples/legacy/2.3/0000755000175000017500000000000013211621077016413 5ustar mehdimehdisingularity-2.4.2/examples/legacy/2.3/contrib/0000755000175000017500000000000013211621077020053 5ustar mehdimehdisingularity-2.4.2/examples/legacy/2.3/contrib/raspbian.def0000644000175000017500000000030313211621077022326 0ustar mehdimehdiBootStrap: debootstrap OSVersion: stable MirrorURL: http://ftp.acc.umu.se/mirror/raspbian/raspbian/ %post apt-get update apt-get -y install gcc vim apache2 apt-get clean exit 0 singularity-2.4.2/examples/legacy/2.1/0000755000175000017500000000000013211621077016411 5ustar mehdimehdisingularity-2.4.2/examples/legacy/2.1/debian.def0000644000175000017500000000017513211621077020316 0ustar mehdimehdiDistType "debian" MirrorURL "http://ftp.us.debian.org/debian/" OSVersion "stable" Setup Bootstrap InstallPkgs vim Cleanup singularity-2.4.2/examples/legacy/2.1/scientific.def0000644000175000017500000000037313211621077021214 0ustar mehdimehdiRELEASE=7 if [ -n "${1:-}" ]; then RELEASE="$1" fi echo "Setting RELEASE=$RELEASE" DistType "redhat" MirrorURL "http://ftp.scientificlinux.org/linux/scientific/${RELEASE}x/\$basearch/os/" Setup Bootstrap #InstallPkgs yum vim-minimal Cleanup singularity-2.4.2/examples/legacy/2.1/fedora.def0000644000175000017500000000040313211621077020326 0ustar mehdimehdiRELEASE=23 if [ -n "${1:-}" ]; then RELEASE="$1" fi echo "Setting RELEASE=$RELEASE" DistType "redhat" MirrorURL "http://download.fedoraproject.org/pub/fedora/linux/releases/$RELEASE/Everything/\$basearch/os/" Setup Bootstrap InstallPkgs dnf Cleanup singularity-2.4.2/examples/legacy/2.1/busybox.def0000644000175000017500000000021613211621077020563 0ustar mehdimehdiDistType "busybox" MirrorURL "https://www.busybox.net/downloads/binaries/1.26.1-defconfig-multiarch/busybox-x86_64" Setup Bootstrap Cleanup singularity-2.4.2/examples/legacy/2.1/centos.def0000644000175000017500000000036413211621077020367 0ustar mehdimehdiRELEASE=7 if [ -n "${1:-}" ]; then RELEASE="$1" fi echo "Setting RELEASE=$RELEASE" DistType "redhat" MirrorURL "http://mirror.centos.org/centos-${RELEASE}/${RELEASE}/os/\$basearch/" Setup Bootstrap InstallPkgs yum vim-minimal Cleanup singularity-2.4.2/examples/legacy/2.1/ubuntu.def0000644000175000017500000000031513211621077020412 0ustar mehdimehdiDistType "debian" MirrorURL "http://us.archive.ubuntu.com/ubuntu/" OSVersion "trusty" Setup Bootstrap RunCmd sed -i 's/$/ universe/' /etc/apt/sources.list RunCmd apt-get update InstallPkgs vim Cleanup singularity-2.4.2/examples/legacy/2.1/arch.def0000644000175000017500000000427313211621077020014 0ustar mehdimehdi# Copyright (c) 2016, Maciej Sieczka. All rights reserved # # Minimal installation process is defined in mods/linux_build_arch.smod. A # couple extra actions are called from here using RunCmd and InstallPkgs # Singularity bootstrap keywords. Adjust them as needed. # https://wiki.archlinux.org/index.php/Installation_Guide may come in handy. DistType "arch" # "Setup" keyword doesn't do anything anymore, at least as of Singularity # 2.1.2. Undocumented "PreSetup" replaced it and is called at each # `singularity bootstrap' run. # Setup Bootstrap # Set time zone. Use whatever you prefer instead of UTC. RunCmd ln -s /usr/share/zoneinfo/UTC /etc/localtime # WARNING: Make sure you prefix the STDOUT redirection target (ie. that guy # after `>' or `>>`) with $SINGULARITY_BUILD_ROOT in RunCmd calls, as below. # Otherwise such command will write to host's filesystem, at least as of # Singularity 2.1.2! This is a known issue, going to be fixed. See # https://github.com/gmkurtzer/singularity/issues/200. # Set locale. Use whatever you prefer instead of en_US. RunCmd echo 'en_US.UTF-8 UTF-8' > "$SINGULARITY_BUILD_ROOT"/etc/locale.gen # Add more locales as needed, eg: # RunCmd echo 'pl_PL.UTF-8 UTF-8' >> "$SINGULARITY_BUILD_ROOT"/etc/locale.gen RunCmd locale-gen RunCmd echo 'LANG=en_US.UTF-8' > "$SINGULARITY_BUILD_ROOT"/etc/locale.conf # Mind that Singularity's shell will use host's locale no matter what anyway, # as of version 2.1.2. This may change in a future release. # Set the package mirror server(s). This is only for the output image's # mirrorlist. `pacstrap' can only use your hosts's package mirrors. RunCmd echo 'Server = https://mirrors.kernel.org/archlinux/$repo/os/$arch' > "$SINGULARITY_BUILD_ROOT"/etc/pacman.d/mirrorlist # Add any number of fail-over servers, eg: RunCmd echo 'Server = https://archlinux.honkgong.info/$repo/os/$arch' >> "$SINGULARITY_BUILD_ROOT"/etc/pacman.d/mirrorlist # I need VIM and Bash completion. Specify your extra packages as needed. InstallPkgs vim bash-completion # Remove the packages downloaded to image's Pacman cache dir during # InstallPkgs. BTW, `pacstrap', used in Bootstrap step, uses host's package # cache rather than the image's. RunCmd paccache -r -k0 Cleanup singularity-2.4.2/examples/self/0000755000175000017500000000000013211621077015576 5ustar mehdimehdisingularity-2.4.2/examples/self/README.md0000644000175000017500000000146013211621077017056 0ustar mehdimehdi# Bootstrap Self A self bootstrap means packaging the current operating system that you live on into an image. Since we assume the root, you don't need to define a `From`. It looks like this: ## Options ``` Bootstrap: self ``` If you really wanted to specify some root, you could do this: ``` Bootstrap: self From: / ``` And we highly recommend that you exclude paths that you don't want added to the tar. For example, Docker stores a lot of data in `/var`, so I chose to exclude that, along with some of the applications in `/opt`: ``` Bootstrap: self Exclude: /var/lib/docker /home/vanessa /opt/* ``` ## Build Example so we could do the following with the specification build file in this folder: ``` singularity create --size 8000 container.img sudo singularity bootstrap container.img Singularity ``` singularity-2.4.2/examples/self/Singularity0000644000175000017500000000026513211621077020036 0ustar mehdimehdiBootstrap: self Exclude: /opt /usr /var /home /boot /run %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" singularity-2.4.2/examples/apps/0000755000175000017500000000000013211621077015610 5ustar mehdimehdisingularity-2.4.2/examples/apps/README.md0000644000175000017500000000101213211621077017061 0ustar mehdimehdi# Singularity SCI-F Apps Build your image ``` sudo singularity build cowsay.img Singularity.cowsay ``` What apps are installed? ``` singularity apps cowsay.img cowsay fortune lolcat ``` Ask for help for a specific app! ``` singularity help --app fortune cowsay.img fortune is the best app ``` Run a particular app ``` singularity run --app fortune cowsay.img ``` Inspect an app ``` singularity inspect --app fortune cowsay.img { "SINGULARITY_APP_NAME": "fortune", "SINGULARITY_APP_SIZE": "1MB" } ``` singularity-2.4.2/examples/apps/Singularity0000644000175000017500000000066013211621077020047 0ustar mehdimehdiBootstrap: docker From: ubuntu:14.04 %setup echo "SETUP" %appinstall foo echo "INSTALLING FOO" touch filefoo.exec %appinstall bar echo "INSTALLING BAR" touch filebar.exec %apphelp foo This is the help for foo! %applabels foo HELLOTHISIS foo %applabels bar HELLOTHISIS bar %appenv foo HELLOTHISIS=foo export HELLOTHISIS %apprun foo echo "RUNNING FOO" %runscript echo "RUNSCRIPT" %post echo "POST" singularity-2.4.2/examples/apps/Singularity.cowsay0000644000175000017500000000164413211621077021356 0ustar mehdimehdiBootstrap: docker From: ubuntu:16.04 # ======================= # global # ======================= %post apt-get -y update %environment export LC_ALL=C export PATH=/usr/games:$PATH # ======================= # fortune # ======================= %appinstall fortune apt-get -y install fortune %appenv fortune BEST_APP=fortune export BEST_APP %apphelp fortune fortune is the best app %apprun fortune fortune "$@" # ======================= # cowsay # ======================= %appinstall cowsay apt-get -y install cowsay %appenv cowsay BEST_APP=cowsay export BEST_APP %apphelp cowsay cowsay is the best app %apprun cowsay cowsay "$@" # ======================= # lolcat # ======================= %appinstall lolcat apt-get -y install lolcat %appenv lolcat BEST_APP=lolcat export BEST_APP %apphelp lolcat lolcat is the best app %apprun lolcat lolcat "$@" singularity-2.4.2/examples/opensuse/0000755000175000017500000000000013211621077016506 5ustar mehdimehdisingularity-2.4.2/examples/opensuse/Singularity0000644000175000017500000000043113211621077020741 0ustar mehdimehdiBootStrap: zypper OSVersion: 42.2 MirrorURL: http://download.opensuse.org/distribution/leap/%{OSVERSION}/repo/oss/ Include: zypper %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" zypper -n install bc singularity-2.4.2/tests/0000755000175000017500000000000013211621077014171 5ustar mehdimehdisingularity-2.4.2/tests/29-instance.sh0000755000175000017500000000403713211621077016570 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # . ./functions if [ ! -d "/proc/self/ns" ]; then echo "Instance is not supported on your host, skipping tests" exit 0 fi test_init "Instance command group tests" CONTAINER="$SINGULARITY_TESTDIR/container" stest 0 sudo singularity build "$CONTAINER" "../examples/busybox/Singularity" stest 0 singularity -x -d instance.start "$CONTAINER" service1 stest 0 sleep 5 stest 0 singularity -x exec instance://service1 true stest 1 singularity -x exec instance://service1 false stest 1 singularity instance.start "$CONTAINER" service1 stest 0 singularity instance.start "$CONTAINER" service2 stest 0 singularity instance.start "$CONTAINER" service3 stest 0 singularity instance.start "$CONTAINER" t1 stest 0 singularity instance.start "$CONTAINER" t2 stest 0 singularity instance.start "$CONTAINER" t22 stest 0 singularity instance.start "$CONTAINER" t3 stest 0 singularity instance.start "$CONTAINER" t4 stest 0 singularity instance.list service1 stest 0 singularity instance.stop service1 stest 1 singularity instance.list service1 stest 0 singularity instance.stop service\* stest 1 singularity instance.list service\* stest 0 singularity instance.list stest 0 singularity instance.list t\* stest 0 singularity instance.stop t1 t2\* t3 stest 0 singularity instance.list t\* stest 0 singularity instance.stop --all stest 1 singularity instance.list t\* stest 0 sudo rm -rf "$CONTAINER" test_cleanup singularity-2.4.2/tests/23-pull.sh0000755000175000017500000000323413211621077015730 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # . ./functions test_init "Pull tests" cd "$SINGULARITY_TESTDIR" stest 0 sudo singularity pull --size 10 docker://busybox CONTAINER=busybox.simg stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false stest 0 singularity exec "$CONTAINER" test -f /.singularity.d/runscript stest 0 singularity exec "$CONTAINER" test -f /.singularity.d/env/01-base.sh stest 0 singularity exec "$CONTAINER" test -f /.singularity.d/actions/shell stest 0 singularity exec "$CONTAINER" test -f /.singularity.d/actions/exec stest 0 singularity exec "$CONTAINER" test -f /.singularity.d/actions/run stest 0 singularity exec "$CONTAINER" test -L /environment stest 0 singularity exec "$CONTAINER" test -L /singularity # should fail b/c we already pulled busybox stest 1 sudo singularity pull --size 10 docker://busybox # force should fix stest 0 sudo singularity pull --force --size 10 docker://busybox stest 0 sudo rm -rf "${CONTAINER}" stest 1 singularity pull docker://this_should_not/exist test_cleanup singularity-2.4.2/tests/10-help.sh0000755000175000017500000000431013211621077015674 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Help and usage tests" alias cmd_check="echo echo \"Testing command usage: '\${cmd}'\" stest 0 singularity --help \$cmd stest 0 singularity -h \$cmd stest 0 singularity help \$cmd stest 0 singularity \$cmd help stest 0 singularity \$cmd -h stest 0 singularity \$cmd --help" MOST_COMMANDS=" apps bootstrap build check create exec image image.create image.expand image.export image.import inspect mount pull run shell test instance.start instance.list instance.stop " # Testing singularity internal commands (one word) stest 0 singularity stest 0 singularity --help stest 0 singularity --version # Testing one word commands for cmd in $MOST_COMMANDS; do cmd_check done # Testing two word commands cmd="image create" cmd_check cmd="image expand" cmd_check cmd="image export" cmd_check cmd="image import" cmd_check cmd="instance start" cmd_check cmd="instance list" cmd_check cmd="instance stop" cmd_check /bin/echo /bin/echo "Testing error on bad commands" stest 1 singularity help bogus stest 1 singularity bogus help stest 1 singularity help instance bogus stest 1 singularity image bogus help test_cleanup singularity-2.4.2/tests/30-actions.sh0000755000175000017500000000500113211621077016404 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Basic container action tests" CONTAINER="$SINGULARITY_TESTDIR/container.img" # Creating a new container stest 0 sudo singularity build "$CONTAINER" "../examples/busybox/Singularity" # Testing shell command stest 0 singularity shell "$CONTAINER" -c "true" stest 0 sh -c "echo true | singularity shell '$CONTAINER'" stest 1 singularity shell "$CONTAINER" -c "false" stest 1 sh -c "echo false | singularity shell '$CONTAINER'" # Testing exec command stest 0 singularity exec "$CONTAINER" true stest 0 singularity exec "$CONTAINER" /bin/true stest 1 singularity exec "$CONTAINER" false stest 1 singularity exec "$CONTAINER" /bin/false stest 1 singularity exec "$CONTAINER" /blahh stest 1 singularity exec "$CONTAINER" blahh stest 0 sh -c "echo hi | singularity exec $CONTAINER grep hi" stest 1 sh -c "echo bye | singularity exec $CONTAINER grep hi" # Testing run command stest 0 singularity run "$CONTAINER" true stest 1 singularity run "$CONTAINER" false # Testing run command properly hands arguments stest 0 sh -c "singularity run '$CONTAINER' foo | grep foo" # Testing singularity properly handles STDIN stest 0 sh -c "echo true | singularity shell '$CONTAINER'" stest 1 sh -c "echo false | singularity shell '$CONTAINER'" stest 0 sh -c "echo true | singularity exec '$CONTAINER' /bin/sh" stest 1 sh -c "echo false | singularity exec '$CONTAINER' /bin/sh" # Checking permissions stest 0 sh -c "singularity exec $CONTAINER id -u | grep `id -u`" stest 0 sh -c "sudo singularity exec $CONTAINER id -u | grep 0" test_cleanup singularity-2.4.2/tests/21-build_docker.sh0000755000175000017500000001007613211621077017402 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, Michael W. Bauer. All rights reserved. # Copyright (c) 2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Docker bootstrap tests" CONTAINER="$SINGULARITY_TESTDIR/container.img" DEFFILE="$SINGULARITY_TESTDIR/example.def" # Make sure the examples/docker/Singularity is pointing to busybox:latest (nobody mess with the examples! LOL) stest 0 grep busybox:latest ../examples/docker/Singularity stest 0 cp ../examples/docker/Singularity "$DEFFILE" stest 0 sudo singularity build "$CONTAINER" "$DEFFILE" stest 0 sed -i -e 's@busybox:latest@ubuntu:latest@' "$DEFFILE" stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false stest 0 sed -i -e 's@ubuntu:latest@centos:latest@' "$DEFFILE" stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false stest 0 sed -i -e 's@centos:latest@dock0/arch:latest@' "$DEFFILE" stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false stest 0 sudo singularity build -F "$CONTAINER" docker://busybox stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false stest 1 sudo singularity build -F "$CONTAINER" docker://something_that_doesnt_exist_ever stest 1 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false if singularity_which docker >/dev/null 2>&1; then # make sure local test does not exist, ignore errors sudo docker kill registry >/dev/null 2>&1 sudo docker rm registry >/dev/null 2>&1 # start local docker registry stest 0 sudo docker run -d -p 5000:5000 --restart=always --name registry registry:2 # pull busybox from docker and push to local registry stest 0 sudo docker pull busybox stest 0 sudo docker tag busybox localhost:5000/my-busybox stest 0 sudo docker push localhost:5000/my-busybox # alright, now we have a local registry to test with # test with custom registry and custom namespace (including empty namespace) # from squashfs to squashfs (via def file) cat >"$DEFFILE" <"$DEFFILE" <"$DEFFILE" < "$DEFFILE" Bootstrap: docker From: busybox %runscript true EOF stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 0 singularity run "$CONTAINER" cat < "$DEFFILE" Bootstrap: docker From: busybox %runscript false EOF stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 1 singularity run "$CONTAINER" cat < "$DEFFILE" Bootstrap: docker From: busybox %files # Spaces and comments $DEFFILE /deffile ../Makefile /makefile %post touch /testfile EOF stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 0 singularity exec "$CONTAINER" test -f /deffile stest 0 singularity exec "$CONTAINER" test -f /testfile stest 0 singularity exec "$CONTAINER" test -f /makefile cat < "$DEFFILE" Bootstrap: docker From: busybox %environment echo "hi from environment" EOF stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false stest 0 sh -c "echo true | singularity shell "$CONTAINER" | grep 'hi from environment'" stest 0 sh -c "singularity exec "$CONTAINER" true | grep 'hi from environment'" # Test for #1103: Duplicate bootstrap definition key found: '#UpdateURL' cat < "$DEFFILE" Bootstrap: docker From: busybox#comment that needs to be ignored #DuplicateKey: commented value #DuplicateKey: other commented value # also include exact example found in #1103 #UpdateURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/updates/\$basearch/ #UpdateURL: http://mirror.centos.org/centos-7/7.4.1708/updates/x86_64/ EOF stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false # Test multiple empty lines cat < "$DEFFILE" Bootstrap: docker From: busybox # this is a comment EOF stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false # Test comments inside runscript cat < "$DEFFILE" Bootstrap: docker From: busybox %runscript #this is a comment EOF # expected output of test below cat <"$SINGULARITY_TESTDIR/expected.txt" #!/bin/sh #this is a comment EOF stest 0 sudo singularity build -F "$CONTAINER" "$DEFFILE" stest 0 singularity exec "$CONTAINER" cat /.singularity.d/runscript # save output of last test, need it to compare with expected cp "$SINGULARITY_TESTDIR/output" "$SINGULARITY_TESTDIR/actual.txt" stest 0 cmp "$SINGULARITY_TESTDIR/actual.txt" "$SINGULARITY_TESTDIR/expected.txt" test_cleanupsingularity-2.4.2/tests/functions0000644000175000017500000000510113211621077016121 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016-2017, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # SINGULARITY_MESSAGELEVEL=5 . ../libexec/functions if [ `id -ru` = "0" ]; then message ERROR "Don't make test as root\n" ABORT 255 fi alias singularity="$SINGULARITY_PATH/singularity" test_init() { if [ -z "${1:-}" ]; then echo "Called test_init without a title" exit 1 fi TITLE="$1" shift if [ -n "${SINGULARITY_TESTDIR:-}" ]; then test_cleanup fi if ! SINGULARITY_TESTDIR=`mktemp -d ${TMPDIR:-/tmp}/stest.XXXXXX`; then message ERROR "Failed to create temporary directory\n" ABORT 255 fi echo echo "################################################################################" echo "$TITLE (script: $0, testdir: $SINGULARITY_TESTDIR)" echo } test_cleanup() { if [ -d "$SINGULARITY_TESTDIR" ]; then rm -rf "$SINGULARITY_TESTDIR" fi exit 0 } exit_cleanup() { return 0 } stest() { ERROR="${1:-}" OUTPUT="$SINGULARITY_TESTDIR/output" shift pwd > "$SINGULARITY_TESTDIR/pwd" echo "$@" > "$SINGULARITY_TESTDIR/cmd" message 2 " + %-90.90s " "$*" "$@" >$OUTPUT 2>&1 RETVAL="$?" if [ "$ERROR" = "0" -a "$RETVAL" != "0" ]; then message 2 "%13s ERROR\n" "(retval=$RETVAL)" cat "$OUTPUT" echo "Full output in: $SINGULARITY_TESTDIR" exit_cleanup exit 1 elif [ "$ERROR" != "0" -a "$RETVAL" = "0" ]; then message 2 "%13s ERROR\n" "(retval=$RETVAL)" cat "$OUTPUT" echo "Full output in: $SINGULARITY_TESTDIR" exit_cleanup exit 1 else message 2 "%13s OK\n" "(retval=$RETVAL)" fi } singularity-2.4.2/tests/28-importexport.sh0000755000175000017500000000302713211621077017535 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, Michael W. Bauer. All rights reserved. # Copyright (c) 2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Import/Export tests" CONTAINER="$SINGULARITY_TESTDIR/container.img" stest 0 touch "$SINGULARITY_TESTDIR/hello_world" stest 0 singularity image.create -s 32 "$CONTAINER" stest 0 sh -c "tar cf - -C $SINGULARITY_TESTDIR hello_world | sudo singularity image.import $CONTAINER" stest 0 /bin/rm "$SINGULARITY_TESTDIR/hello_world" stest 0 sh -c "singularity image.export $CONTAINER | tar xf - -C $SINGULARITY_TESTDIR" stest 0 test -f "$SINGULARITY_TESTDIR/hello_world" test_cleanup singularity-2.4.2/tests/35-userbinds.sh0000755000175000017500000000275413211621077016763 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Testing user binds" CONTAINER="$SINGULARITY_TESTDIR/container.img" # Creating a new container stest 0 sudo singularity build "$CONTAINER" "../examples/busybox/Singularity" stest 0 touch /tmp/hello_world_test stest 0 singularity exec -B /tmp:/opt "$CONTAINER" test -f /opt/hello_world_test if [ -n "$SINGULARITY_OVERLAY_FS" ]; then stest 0 singularity exec -B /tmp:/nonexistent "$CONTAINER" test -f /nonexistent/hello_world_test fi stest 0 rm -f /tmp/hello_world_test test_cleanup singularity-2.4.2/tests/32-action_options.sh0000755000175000017500000000512613211621077020006 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Testing action options" CONTAINER="$SINGULARITY_TESTDIR/container.img" TESTDIR="$SINGULARITY_TESTDIR/home_test" # Creating a new container stest 0 sudo singularity build "$CONTAINER" "../examples/busybox/Singularity" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false # Checking if Singularity properly handles custom shells stest 0 singularity shell -s /bin/true "$CONTAINER" stest 1 singularity shell -s /bin/false "$CONTAINER" # Testing --workdir stest 0 touch "$SINGULARITY_TESTDIR/testfile" stest 0 singularity exec --workdir "$SINGULARITY_TESTDIR" "$CONTAINER" test -f "$SINGULARITY_TESTDIR/testfile" stest 1 singularity exec --workdir "$SINGULARITY_TESTDIR" --contain "$CONTAINER" test -f "$SINGULARITY_TESTDIR/testfile" # Testing --pwd stest 0 singularity exec --pwd /etc "$CONTAINER" true stest 1 singularity exec --pwd /non-existent-dir "$CONTAINER" true stest 0 sh -c "singularity exec --pwd /etc '$CONTAINER' pwd | egrep '^/etc'" # Testing --home stest 0 mkdir -p "$TESTDIR" stest 0 touch "$TESTDIR/testfile" stest 0 singularity exec --home "$TESTDIR" "$CONTAINER" test -f "$TESTDIR/testfile" stest 0 singularity exec --home "$TESTDIR:/home" "$CONTAINER" test -f "/home/testfile" if [ -n "${SINGULARITY_OVERLAY_FS:-}" ]; then stest 0 singularity exec --contain --home "$TESTDIR:/blah" "$CONTAINER" test -f "/blah/testfile" fi stest 0 sh -c "echo 'cd; test -f testfile' | singularity exec --home '$TESTDIR' '$CONTAINER' /bin/sh" stest 1 singularity exec --home "/tmp" "$CONTAINER" true stest 1 singularity exec --home "/tmp:/home" "$CONTAINER" true test_cleanup singularity-2.4.2/tests/15-selftest.sh0000755000175000017500000000206413211621077016606 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "selftest" stest 0 singularity selftest test_cleanup singularity-2.4.2/tests/01-sanity.sh0000755000175000017500000000225113211621077016255 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Checking environment" stest 0 sudo true stest 1 sudo false stest 0 which singularity stest 0 test -f "$SINGULARITY_sysconfdir/singularity/singularity.conf" test_cleanup singularity-2.4.2/tests/23-bootstrap_apps.sh0000644000175000017500000000655413211621077020021 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # Copyright (c) 2017, Vanessa Sochat. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. . ./functions test_init "Standard Integration Format (SCI-F) Apps bootstrap tests" CONTAINER="$SINGULARITY_TESTDIR/container.img" DEFFILE="$SINGULARITY_TESTDIR/example.def" # Be consistent to bootstrap from Ubuntu 14.04 stest 0 grep ubuntu:14.04 ../examples/apps/Singularity # Create the container with apps recipe stest 0 cp ../examples/apps/Singularity "$DEFFILE" stest 0 singularity create -F -s 568 "$CONTAINER" stest 0 sudo singularity bootstrap "$CONTAINER" "$DEFFILE" # Testing exec command stest 0 singularity exec "$CONTAINER" true stest 0 singularity exec "$CONTAINER" /bin/true stest 1 singularity exec "$CONTAINER" false stest 1 singularity exec "$CONTAINER" /bin/false # Testing folder organization stest 0 singularity exec "$CONTAINER" test -d "/scif" stest 0 singularity exec "$CONTAINER" test -d "/scif/apps" stest 0 singularity exec "$CONTAINER" test -d "/scif/data" stest 0 singularity exec "$CONTAINER" test -d "/scif/apps/foo" stest 0 singularity exec "$CONTAINER" test -d "/scif/apps/bar" stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/foo/filefoo.exec" stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/bar/filebar.exec" stest 0 singularity exec "$CONTAINER" test -d "/scif/data/foo/output" stest 0 singularity exec "$CONTAINER" test -d "/scif/data/foo/input" # Metadata folder stest 0 singularity exec "$CONTAINER" test -d "/scif/apps/foo/scif" stest 0 singularity exec "$CONTAINER" test -d "/scif/apps/foo/scif/env" stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/foo/scif/Singularity" stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/foo/scif/env/01-base.sh" stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/foo/scif/labels.json" stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/foo/scif/runscript" stest 0 singularity exec "$CONTAINER" test -f "/scif/apps/foo/scif/runscript.help" # Testing help stest 0 sh -c "singularity help '$CONTAINER' | grep 'No runscript help is defined for this image.'" stest 0 sh -c "singularity help --app foo '$CONTAINER' | grep 'This is the help for foo!'" stest 0 sh -c "singularity help --app bar '$CONTAINER' | grep 'No runscript help is defined for this application.'" # Testing apps stest 0 sh -c "singularity apps '$CONTAINER' | grep 'foo'" stest 0 sh -c "singularity apps '$CONTAINER' | grep 'bar'" # Testing inspect stest 0 sh -c "singularity inspect --app foo '$CONTAINER' | grep HELLOTHISIS" stest 0 sh -c "singularity inspect --app foo '$CONTAINER' | grep foo" # Testing run stest 0 sh -c "singularity run --app foo '$CONTAINER' | grep 'RUNNING FOO'" stest 0 sh -c "singularity run --app bar '$CONTAINER' | grep 'No Singularity runscript for contained app: bar'" test_cleanup singularity-2.4.2/tests/20-build.sh0000755000175000017500000001133113211621077016045 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # See the COPYRIGHT.md file at the top-level directory of this distribution and at # https://github.com/singularityware/singularity/blob/master/COPYRIGHT.md. # # This file is part of the Singularity Linux container project. It is subject to the license # terms in the LICENSE.md file found in the top-level directory of this distribution and # at https://github.com/singularityware/singularity/blob/master/LICENSE.md. No part # of Singularity, including this file, may be copied, modified, propagated, or distributed # except according to the terms contained in the LICENSE.md file. # # # the order of these tests is important because one container often # builds from another . ./functions test_init "Build tests" CONTAINER="$SINGULARITY_TESTDIR/container" CONTAINER2="$SINGULARITY_TESTDIR/container2" alias container_check="stest 0 singularity exec \"$CONTAINER\" true ; stest 1 singularity exec \"$CONTAINER\" false ; stest 0 singularity exec \"$CONTAINER\" test -f /.singularity.d/runscript ; stest 0 singularity exec \"$CONTAINER\" test -f /.singularity.d/env/01-base.sh ; stest 0 singularity exec \"$CONTAINER\" test -f /.singularity.d/actions/shell ; stest 0 singularity exec \"$CONTAINER\" test -f /.singularity.d/actions/exec ; stest 0 singularity exec \"$CONTAINER\" test -f /.singularity.d/actions/run ; stest 0 singularity exec \"$CONTAINER\" test -L /environment ; stest 0 singularity exec \"$CONTAINER\" test -L /singularity" # from definition file to squashfs stest 0 sudo singularity build "$CONTAINER" "../examples/busybox/Singularity" container_check # from definition file to sandbox sudo rm "$CONTAINER" stest 0 sudo singularity build --sandbox "$CONTAINER" "../examples/busybox/Singularity" container_check # from ridicolus to squashfs stest 1 sudo singularity build "$CONTAINER" "/some/dumb/path" # from sandbox to squashfs sudo mv "$CONTAINER" "$CONTAINER2" stest 0 sudo singularity build "$CONTAINER" "$CONTAINER2" container_check # from definition file to image rm -rf "$CONTAINER" stest 0 sudo singularity build --writable "$CONTAINER" "../examples/busybox/Singularity" container_check # from image to squasfs sudo mv "$CONTAINER" "$CONTAINER2" stest 0 sudo singularity build "$CONTAINER" "$CONTAINER2" container_check # from docker to squashfs sudo rm "$CONTAINER" stest 0 singularity build "$CONTAINER" "docker://busybox" container_check # from sqaushfs to squashfs sudo mv "$CONTAINER" "$CONTAINER2" stest 0 sudo singularity build "$CONTAINER" "$CONTAINER2" container_check # from shub to squashfs sudo rm "$CONTAINER" stest 0 singularity build "$CONTAINER" "shub://GodloveD/busybox" container_check # from docker to squashfs (via def file) sudo rm "$CONTAINER" stest 0 sudo singularity build "$CONTAINER" "../examples/docker/Singularity" container_check # from shub to squashfs (via def file) sudo rm "$CONTAINER" stest 0 sudo singularity build "$CONTAINER" "../examples/shub/Singularity" container_check # from squashfs to squashfs (via def file) cat >"${SINGULARITY_TESTDIR}/Singularity" <>"${SINGULARITY_TESTDIR}/Singularity" < '${CONTAINER2}.tar'" stest 0 sudo rm "$CONTAINER" stest 0 sudo singularity build "$CONTAINER" "${CONTAINER2}.tar" container_check # from tar.gx to squashfs stest 0 sh -c "singularity image.export '$CONTAINER' | gzip -9 > '${CONTAINER2}.tar.gz'" sudo rm "$CONTAINER" stest 0 sudo singularity build "$CONTAINER" "${CONTAINER2}.tar.gz" container_check stest 0 sudo rm -rf "${CONTAINER}" stest 0 sudo rm -rf "${CONTAINER2}" stest 0 sudo rm -rf "${CONTAINER2}".tar stest 0 sudo rm -rf "${CONTAINER2}".tar.gz stest 0 sudo rm -rf "${SINGULARITY_TESTDIR}/Singularity" test_cleanup singularity-2.4.2/tests/41-configownership.sh0000755000175000017500000000310313211621077020153 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, Michael W. Bauer. All rights reserved. # Copyright (c) 2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Checking configuration file ownership" CONTAINER="$SINGULARITY_TESTDIR/container.img" stest 0 sudo singularity build "$CONTAINER" "../examples/busybox/Singularity" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false stest 0 sudo chown `id -un` "$SINGULARITY_sysconfdir/singularity/singularity.conf" stest 1 singularity exec "$CONTAINER" true stest 0 sudo chown root.root "$SINGULARITY_sysconfdir/singularity/singularity.conf" stest 0 singularity exec "$CONTAINER" true test_cleanup singularity-2.4.2/tests/40-privblock.sh0000755000175000017500000000270513211621077016750 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, Michael W. Bauer. All rights reserved. # Copyright (c) 2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Checking escalation block" CONTAINER="$SINGULARITY_TESTDIR/container.img" stest 0 sudo singularity build "$CONTAINER" docker://centos:7 stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false # Checking no new privs with capabilities stest 0 sudo singularity exec "$CONTAINER" chsh -s /bin/sh stest 1 singularity exec "$CONTAINER" chsh -s /bin/sh test_cleanup singularity-2.4.2/tests/05-python-units.sh0000755000175000017500000000525413211621077017441 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Checking Python unit tests" cd ../libexec/python if which python2 >/dev/null 2>&1; then stest 0 python2 -m unittest tests.test_json stest 0 python2 -m unittest tests.test_helpers stest 0 python2 -m unittest tests.test_base stest 0 python2 -m unittest tests.test_core stest 0 python2 -m unittest tests.test_docker_import stest 0 python2 -m unittest tests.test_docker_api stest 0 python2 -m unittest tests.test_docker_tasks stest 0 python2 -m unittest tests.test_shub_pull stest 0 python2 -m unittest tests.test_shub_api stest 0 python2 -m unittest tests.test_custom_cache stest 0 python2 -m unittest tests.test_default_cache stest 0 python2 -m unittest tests.test_disable_cache else echo "Skipping python2 tests: not installed" fi if which python3 >/dev/null 2>&1; then stest 0 python3 -m unittest tests.test_json stest 0 python3 -m unittest tests.test_helpers stest 0 python3 -m unittest tests.test_base stest 0 python3 -m unittest tests.test_core stest 0 python3 -m unittest tests.test_docker_import stest 0 python3 -m unittest tests.test_docker_api stest 0 python3 -m unittest tests.test_docker_tasks stest 0 python3 -m unittest tests.test_shub_pull stest 0 python3 -m unittest tests.test_shub_api stest 0 python3 -m unittest tests.test_custom_cache stest 0 python3 -m unittest tests.test_default_cache stest 0 python3 -m unittest tests.test_disable_cache else echo "Skipping python3 tests: not installed" fi if which pylint >/dev/null 2>&1; then stest 0 pylint $PWD --errors-only --ignore tests --disable=E0401,E0611,E1101 else echo "Skipping pylint tests: not installed" fi test_cleanup singularity-2.4.2/tests/02-flawfinder.sh0000755000175000017500000000252713211621077017076 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions if ! which flawfinder 2>/dev/null; then echo "Not testing with flawfinder, not installed" exit 0 fi test_init "Running Flawfinder" stest 0 sh -c "flawfinder -SQ --minlevel=3 ../src > '$SINGULARITY_TESTDIR/ff.out'" if ! grep -q 'No hits found.' "$SINGULARITY_TESTDIR/ff.out"; then cat "$SINGULARITY_TESTDIR/ff.out" exit 1 fi test_cleanup singularity-2.4.2/tests/45-targetmode.sh0000755000175000017500000000323113211621077017110 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2017, Michael W. Bauer. All rights reserved. # Copyright (c) 2017, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Checking target UID/GID mode" CONTAINER="$SINGULARITY_TESTDIR/container.img" stest 0 sudo singularity build "$CONTAINER" "../examples/busybox/Singularity" stest 0 singularity exec "$CONTAINER" true stest 1 singularity exec "$CONTAINER" false stest 0 sh -c "sudo SINGULARITY_TARGET_GID=`id -g` SINGULARITY_TARGET_UID=`id -u` singularity exec $CONTAINER whoami | grep `id -un`" stest 1 sh -c "SINGULARITY_TARGET_GID=99 SINGULARITY_TARGET_UID=99 singularity exec $CONTAINER whoami | grep 99" stest 1 sh -c "SINGULARITY_TARGET_GID=99 SINGULARITY_TARGET_UID=99 singularity exec $CONTAINER true" test_cleanup singularity-2.4.2/tests/60-legacy.sh0000755000175000017500000000557113211621077016227 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # if [ ! -d /usr/local/singularity-2.2.1-legacytests ]; then exit fi . ./functions test_init "DISABLED: Checking new container with legacy Singularity" # #CONTAINER="$SINGULARITY_TESTDIR/container.img" #CONTAINERDIR="$SINGULARITY_TESTDIR/container.dir" #LEGACYDIR="/usr/local/singularity-2.2.1-legacytests/bin" # #stest 0 cp "../examples/busybox/Singularity" $SINGULARITY_TESTDIR #stest 0 echo '%environment' >> "$SINGULARITY_TESTDIR/Singularity" #stest 0 echo ' export FOO=bar' >> $SINGULARITY_TESTDIR"/Singularity" #stest 0 sudo singularity build --writable "$CONTAINER" "$SINGULARITY_TESTDIR/Singularity" #stest 0 singularity exec "$CONTAINER" true #stest 1 singularity exec "$CONTAINER" false #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" true #stest 1 $LEGACYDIR/singularity exec "$CONTAINER" false # #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" test -f /.singularity.d/runscript #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" test -f /.singularity.d/labels.json #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" test -f /.singularity.d/env/01-base.sh #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" test -f /.singularity.d/actions/shell #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" test -f /.singularity.d/actions/exec #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" test -f /.singularity.d/actions/run #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" test -L /environment #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" test -L /singularity #stest 0 $LEGACYDIR/singularity exec "$CONTAINER" env | grep -cw FOO=bar > /dev/null # #stest 0 mkdir "$CONTAINERDIR" #stest 0 sudo singularity bootstrap "$CONTAINERDIR" "$SINGULARITY_TESTDIR/Singularity" #stest 0 $LEGACYDIR/singularity exec "$CONTAINERDIR" true #stest 1 $LEGACYDIR/singularity exec "$CONTAINERDIR" false #stest 0 $LEGACYDIR/singularity exec "$CONTAINERDIR" env | grep -cw FOO=bar > /dev/null # #stest 0 sudo rm -rf "$CONTAINERDIR" test_cleanup singularity-2.4.2/tests/31-action_uris.sh0000755000175000017500000000226313211621077017273 0ustar mehdimehdi#!/bin/bash # # Copyright (c) 2015-2016, Gregory M. Kurtzer. All rights reserved. # # "Singularity" Copyright (c) 2016, The Regents of the University of California, # through Lawrence Berkeley National Laboratory (subject to receipt of any # required approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # . ./functions test_init "Action URI tests" CONTAINER="$SINGULARITY_TESTDIR/container.img" stest 0 singularity exec docker://busybox true stest 1 singularity exec docker://busybox false test_cleanup singularity-2.4.2/etc/0000755000175000017500000000000013211621077013602 5ustar mehdimehdisingularity-2.4.2/etc/nvliblist.conf0000644000175000017500000000154213211621077016461 0ustar mehdimehdi# NVLIBLIST.CONF # This configuration file determines which NVIDIA libraries to search for on # the host system when the --nv option is invoked. You can edit it if you have # different libraries on your host system. libcuda.so libEGL_installertest.so libEGL_nvidia.so libEGL.so libGLdispatch.so libGLESv1_CM_nvidia.so libGLESv1_CM.so libGLESv2_nvidia.so libGLESv2.so libGL.so libGLX_installertest.so libGLX_nvidia.so libglx.so libGLX.so libnvcuvid.so libnvidia-cfg.so libnvidia-compiler.so libnvidia-eglcore.so libnvidia-egl-wayland.so libnvidia-encode.so libnvidia-fatbinaryloader.so libnvidia-fbc.so libnvidia-glcore.so libnvidia-glsi.so libnvidia-gtk2.so libnvidia-gtk3.so libnvidia-ifr.so libnvidia-ml.so libnvidia-opencl.so libnvidia-ptxjitcompiler.so libnvidia-tls.so libnvidia-wfb.so libOpenCL.so libOpenGL.so libvdpau_nvidia.so nvidia_drv.so tls_test_.so singularity-2.4.2/etc/default-nsswitch.conf0000644000175000017500000000012713211621077017735 0ustar mehdimehdipasswd: files shadow: files group: files hosts: files dns myhostname singularity-2.4.2/etc/configure_transform.py0000755000175000017500000001064313211621077020237 0ustar mehdimehdi#!/usr/bin/env python ''' Copyright (c) 2017, SingularityWare, LLC. All rights reserved. Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. Copyright (c) 2017, Vanessa Sochat All rights reserved. Copyright (c) 2016, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. This software is licensed under a customized 3-clause BSD license. Please consult LICENSE file distributed with the sources of this project regarding your rights to use or distribute this software. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. ''' import os import re import sys sys.path.append('../libexec/python') # noqa from sutils import ( get_fullpath, read_file, write_file ) from message import bot import optparse def get_parser(): description = "singularity configuration parsing helper in python" parser = optparse.OptionParser(description=description) # Configuration defaults header help = "configuration defaults header file (../src/lib/config_defaults.h)" parser.add_option("--defaults", dest='defaults', help=help, type=str) # input configuration file help = "the configuration input file path (singularity.conf.in)" parser.add_option("--infile", dest='infile', help=help, type=str) # Output configuration file help = "the configuration output file path (singularity.conf)" parser.add_option("--outfile", dest='outfile', help=help, type=str) return parser def main(): '''parse configuration options and produce configuration output file ''' bot.info("\n*** STARTING PYTHON CONFIGURATION HELPER ****") parser = get_parser() try: (args, options) = parser.parse_args() except Exception: bot.error("Input args to %s improperly set, exiting." % os.path.abspath(__file__)) parser.print_help() sys.exit(1) # Check for required args [check_required(parser, arg) for arg in [args.defaults, args.infile, args.outfile]] # Run the configuration configure(args) def check_required(parser, arg): '''check_required arg checks that an argument is defined. It is a workaround for missing required parameter of argparse :param parser: the parser :param arg: the argument ''' if not arg: # if filename is not given parser.error('Missing required argument.') parser.print_help() sys.exit(1) def configure(args): # Get fullpath to each file, and concurrently check that exists defaultfile = get_fullpath(args.defaults) # ../src/lib/config_defaults.h infile = get_fullpath(args.infile) # singularity.conf.in # Find define statements define_re = re.compile("#define ([A-Z_]+) (.*)") # Read in input and default files defaultfile = read_file(defaultfile) data = "".join(read_file(infile)) # Lookup for values we want replaced lookup = {'0': 'no', '1': 'yes'} defaults = {} # Read in defaults to dictionary for line in defaultfile: match = define_re.match(line) if match: key, value = match.groups() # Maintain the original default set by user defaults[key] = value # Use parsed value for final config new_value = value.replace('"', '') if new_value in lookup: new_value = lookup[new_value] data = data.replace("@" + key + "@", new_value) # Write to output file outfile = "%s.tmp" % args.outfile write_file(outfile, data) os.rename(outfile, args.outfile) bot.info("*** FINISHED PYTHON CONFIGURATION HELPER ****\n") if __name__ == '__main__': main() singularity-2.4.2/etc/Makefile.am0000644000175000017500000000105213211621077015634 0ustar mehdimehdiconfdir = $(sysconfdir)/singularity/ completiondir = $(sysconfdir)/bash_completion.d/ all: singularity.conf singularity.conf: singularity.conf.in ../src/util/config_defaults.h ./configure_transform.py --defaults ../src/util/config_defaults.h --infile singularity.conf.in --outfile singularity.conf dist_conf_DATA = default-nsswitch.conf singularity.conf init nvliblist.conf dist_completion_DATA = bash_completion.d/singularity CLEANFILES = singularity.conf MAINTAINERCLEANFILES = Makefile.in EXTRA_DIST = singularity.conf.in configure_transform.py singularity-2.4.2/etc/bash_completion.d/0000755000175000017500000000000013211621077017172 5ustar mehdimehdisingularity-2.4.2/etc/bash_completion.d/singularity0000644000175000017500000003502213211621077021471 0ustar mehdimehdi#!/bin/bash ## Only here for syntax highlighting # # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. # # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. # # Copyright (c) 2016, The Regents of the University of California, through # Lawrence Berkeley National Laboratory (subject to receipt of any required # approvals from the U.S. Dept. of Energy). All rights reserved. # # This software is licensed under a customized 3-clause BSD license. Please # consult LICENSE file distributed with the sources of this project regarding # your rights to use or distribute this software. # # NOTICE. This Software was developed under funding from the U.S. Department of # Energy and the U.S. Government consequently retains certain rights. As such, # the U.S. Government has been granted for itself and others acting on its # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software # to reproduce, distribute copies to the public, prepare derivative works, and # perform publicly and display publicly, and to permit other to do so. # # _singularity_next_nonopt() { cmd_idx=$1 while [ $cmd_idx -lt $COMP_CWORD ]; do cmd="${COMP_WORDS[$cmd_idx]}" if [[ ${cmd} != -* ]] ; then break; fi cmd_idx=$(( $cmd_idx + 1 )) done echo $cmd_idx return 0 } _singularity() { local cur cmd opts cmd_idx container_idx COMPREPLY=() cur="${COMP_WORDS[COMP_CWORD]}" prev_opt_idx=$(( $COMP_CWORD - 2)) prev_opt="${COMP_WORDS[$prev_opt_idx]}" # TODO: This can be dynamically generated by the following: # find /opt/singularity/libexec/singularity/cli -maxdepth 1 -mindepth 1 -name '*.exec' -type f -exec basename {} \; | sed -e 's|.exec||' | sort -u # but we then need to auto-generate this file with autoconf. local -r cmds="help selftest exec run shell test apps bootstrap build check inspect mount pull instance.start instance.list instance.stop image image.create image.import image.export image.expand" # Find the first command (skipping any global options) cmd_idx=( $(_singularity_next_nonopt 1) ) cmd="${COMP_WORDS[$cmd_idx]}" # In this case, no command is present. if [ $cmd_idx -eq $COMP_CWORD ]; then if [[ ${cur} == -* ]] ; then opts="--debug --help --silent --quiet --version --verbose --sh-debug" COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) ) return 0 else COMPREPLY=( $(compgen -W "${opts} ${cmds}" -- ${cur}) ) return 0 fi fi # If another non-option has been specified we will tentatively assume # that it is a container container_idx=$(( $cmd_idx + 1 )) container_idx=( $(_singularity_next_nonopt $container_idx ) ) case "${cmd}" in help) COMPREPLY=( $(compgen -W "$cmds" -- ${cur}) ) return 0 ;; selftest) _filedir return 0 ;; run|shell|exec) # If we specified a container, treat this as a command (excepting opts with args) if [[ ${prev_opt} != -S && ${prev_opt} != --scratch && ${prev_opt} != --pwd && ${prev_opt} != -a && ${prev_opt} != --app && ${prev_opt} != -o && ${prev_opt} != --overlay ]]; then if [ $container_idx -lt $COMP_CWORD ]; then if [[ ${cmd} == run || ${cmd} == exec ]]; then _command_offset $(( $COMP_CWORD )) return 0 fi fi fi if [[ ${cur} == -B || ${cur} == --bind ]]; then #TODO: Not really a path, but this should be a bind spec... _filedir elif [[ ${cur} == -S || ${cur} == --scratch* || ${cur} == --pwd || ${cur} == -a || ${cur} == --app || ${cur} == -o || ${cur} == --overlay ]]; then _filedir elif [[ ${cur} == -H || ${cur} == --home ]]; then COMPREPLY=( $(compgen -f -- ${cur}) ) elif [[ ${cur} == -W || ${cur} == --workdir || ${cur} == --wdir ]]; then COMPREPLY=( $(compgen -f -- ${cur}) ) elif [[ ${cur} == -* ]] ; then opts="--app --bind --contain --containall --cleanenv --home --ipc --net --nv --overlay --pid --pwd --scratch --user --workdir --writable --help " case "${cmd}" in shell) opts="$opts --shell" esac COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) ) else _filedir fi return 0 ;; bootstrap) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--notest --section --checks --tag --low --med --high --force --help" -- ${cur}) ) else _filedir fi return 0 ;; build) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--sandbox --writable --force --notest --section --checks --tag --low --med --high --help" -- ${cur}) ) else _filedir fi return 0 ;; apps) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--help" -- ${cur}) ) else _filedir fi return 0 ;; test) if [[ ${cur} == -a || ${cur} == --app ]]; then _filedir elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--app --help" -- ${cur}) ) else _filedir fi return 0 ;; mount) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--writable --overlay --help" -- ${cur}) ) else _filedir fi _filedir return 0 ;; create) if [[ ${cur} == -s || ${cur} == --size ]]; then # Specify a number; no completion. return 0 elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--size --force --help" -- ${cur}) ) elif [ $container_idx -lt $COMP_CWORD ]; then return 0 else _filedir fi return 0 ;; inspect) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--app --labels --deffile --runscript --test --environment --json --helpfile --help" -- ${cur}) ) elif [ $container_idx -lt $COMP_CWORD ]; then return 0 else _filedir fi return 0 ;; check) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--low --med --high --tag --help" -- ${cur}) ) elif [ $container_idx -lt $COMP_CWORD ]; then return 0 else _filedir fi return 0 ;; pull) if [[ ${cur} == -n || ${cur} == --name ]]; then _filedir elif [[ ${cur} == -C || ${cur} == --commit ]]; then _filedir elif [[ ${cur} == -H || ${cur} == --hash ]]; then _filedir elif [[ ${cur} == -s || ${cur} == --size ]]; then _filedir elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--name --commit --hash --size --force --help" -- ${cur}) ) else _filedir fi return 0 ;; instance.start) if [[ ${cur} == -B || ${cur} == --bind ]]; then #TODO: Not really a path, but this should be a bind spec... _filedir elif [[ ${cur} == -S || ${cur} == --scratch* || ${cur} == --pwd ]]; then _filedir elif [[ ${cur} == -H || ${cur} == --home ]]; then COMPREPLY=( $(compgen -f -- ${cur}) ) elif [[ ${cur} == -W || ${cur} == --workdir || ${cur} == --wdir ]]; then COMPREPLY=( $(compgen -f -- ${cur}) ) elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--bind --contain --home --net --nv --overlay --scratch --workdir --writable --help" -- ${cur}) ) else _filedir fi return 0 ;; instance.list) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W " --user --help" -- ${cur}) ) else _filedir fi return 0 ;; instance.stop) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--all --user --help" -- ${cur}) ) elif [ $container_idx -lt $COMP_CWORD ]; then return 0 else _filedir fi return 0 ;; image.create) if [[ ${cur} == -s || ${cur} == --size ]]; then _filedir elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--size --force --help" -- ${cur}) ) else _filedir fi return 0 ;; image.import) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W " --help" -- ${cur}) ) else _filedir fi return 0 ;; image.expand) if [[ ${cur} == -s || ${cur} == --size ]]; then _filedir elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--size --help" -- ${cur}) ) else _filedir fi return 0 ;; image.export) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--file --help" -- ${cur}) ) elif [ $container_idx -lt $COMP_CWORD ]; then return 0 else _filedir fi return 0 ;; instance) # In this case, no sub-command is present. if [ $container_idx -eq $COMP_CWORD ]; then COMPREPLY=( $(compgen -W "start list stop" -- ${cur}) ) return 0 # In this case the next non-option is a subcommand not an image elif [ $container_idx -lt $COMP_CWORD ]; then subcmd="${COMP_WORDS[$container_idx]}" container_idx=$(( $container_idx + 1 )) container_idx=( $(_singularity_next_nonopt $container_idx ) ) elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--help" -- ${cur}) ) return 0 fi case "${subcmd}" in start) if [[ ${cur} == -B || ${cur} == --bind ]]; then #TODO: Not really a path, but this should be a bind spec... _filedir elif [[ ${cur} == -S || ${cur} == --scratch* || ${cur} == --pwd ]]; then _filedir elif [[ ${cur} == -H || ${cur} == --home ]]; then COMPREPLY=( $(compgen -f -- ${cur}) ) elif [[ ${cur} == -W || ${cur} == --workdir || ${cur} == --wdir ]]; then COMPREPLY=( $(compgen -f -- ${cur}) ) elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--bind --contain --home --net --nv --overlay --scratch --workdir --writable --help" -- ${cur}) ) else _filedir fi return 0 ;; list) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W " --user --help" -- ${cur}) ) else _filedir fi return 0 ;; stop) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--all --user --help" -- ${cur}) ) elif [ $container_idx -lt $COMP_CWORD ]; then return 0 else _filedir fi return 0 ;; esac ;; image) # In this case, no sub-command is present. if [ $container_idx -eq $COMP_CWORD ]; then COMPREPLY=( $(compgen -W "create import export expand" -- ${cur}) ) return 0 # In this case the next non-option is a subcommand not an image elif [ $container_idx -lt $COMP_CWORD ]; then subcmd="${COMP_WORDS[$container_idx]}" container_idx=$(( $container_idx + 1 )) container_idx=( $(_singularity_next_nonopt $container_idx ) ) elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--help" -- ${cur}) ) return 0 fi case "${subcmd}" in create) if [[ ${cur} == -s || ${cur} == --size ]]; then _filedir elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--size --force --help" -- ${cur}) ) else _filedir fi return 0 ;; import) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W " --help" -- ${cur}) ) else _filedir fi return 0 ;; export) if [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--file --help" -- ${cur}) ) elif [ $container_idx -lt $COMP_CWORD ]; then return 0 else _filedir fi return 0 ;; expand) if [[ ${cur} == -s || ${cur} == --size ]]; then _filedir elif [[ ${cur} == -* ]] ; then COMPREPLY=( $(compgen -W "--size --help" -- ${cur}) ) else _filedir fi return 0 ;; esac ;; esac } complete -F _singularity singularity singularity-2.4.2/etc/init0000644000175000017500000000141613211621077014472 0ustar mehdimehdi# This will be sourced before launching a Singularity container. # Any variables prefixed with "SINGULARITYENV_" will be transposed # properly into the container. For example: # SINGULARITYENV_LD_LIBRARY_PATH -> LD_LIBRARY_PATH # Environment modules if set, cause errors in containers unset module unset ml # Bash env has been known to cause issues in containers unset BASH_ENV # Provide a sane path within the container if [ -z ${SINGULARITYENV_PATH+x} ]; then SINGULARITYENV_PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin" else SINGULARITYENV_PATH="$SINGULARITYENV_PATH:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin" fi # Don't save the shell's HISTFILE SINGULARITYENV_HISTFILE="" export SINGULARITYENV_PATH SINGULARITYENV_HISTFILE singularity-2.4.2/etc/singularity.conf.in0000644000175000017500000001571513211621077017441 0ustar mehdimehdi# SINGULARITY.CONF # This is the global configuration file for Singularity. This file controls # what the container is allowed to do on a particular host, and as a result # this file must be owned by root. # ALLOW SETUID: [BOOL] # DEFAULT: @ALLOW_SETUID_DEFAULT@ # Should we allow users to utilize the setuid program flow within Singularity? # note1: This is the default mode, and to utilize all features, this option # will need to be enabled. # note2: If this option is disabled, it will rely on the user namespace # exclusively which has not been integrated equally between the different # Linux distributions. @ALLOW_SETUID@ = @ALLOW_SETUID_DEFAULT@ # MAX LOOP DEVICES: [INT] # DEFAULT: @MAX_LOOP_DEVS_DEFAULT@ # Set the maximum number of loop devices that Singularity should ever attempt # to utilize. @MAX_LOOP_DEVS@ = @MAX_LOOP_DEVS_DEFAULT@ # ALLOW PID NS: [BOOL] # DEFAULT: @ALLOW_PID_NS_DEFAULT@ # Should we allow users to request the PID namespace? Note that for some HPC # resources, the PID namespace may confuse the resource manager and break how # some MPI implementations utilize shared memory. (note, on some older # systems, the PID namespace is always used) @ALLOW_PID_NS@ = @ALLOW_PID_NS_DEFAULT@ # CONFIG PASSWD: [BOOL] # DEFAULT: @CONFIG_PASSWD_DEFAULT@ # If /etc/passwd exists within the container, this will automatically append # an entry for the calling user. @CONFIG_PASSWD@ = @CONFIG_PASSWD_DEFAULT@ # CONFIG GROUP: [BOOL] # DEFAULT: @CONFIG_GROUP_DEFAULT@ # If /etc/group exists within the container, this will automatically append # group entries for the calling user. @CONFIG_GROUP@ = @CONFIG_GROUP_DEFAULT@ # CONFIG RESOLV_CONF: [BOOL] # DEFAULT: @CONFIG_RESOLV_CONF_DEFAULT@ # If there is a bind point within the container, use the host's # /etc/resolv.conf. @CONFIG_RESOLV_CONF@ = @CONFIG_RESOLV_CONF_DEFAULT@ # MOUNT PROC: [BOOL] # DEFAULT: @MOUNT_PROC_DEFAULT@ # Should we automatically bind mount /proc within the container? @MOUNT_PROC@ = @MOUNT_PROC_DEFAULT@ # MOUNT SYS: [BOOL] # DEFAULT: @MOUNT_SYS_DEFAULT@ # Should we automatically bind mount /sys within the container? @MOUNT_SYS@ = @MOUNT_SYS_DEFAULT@ # MOUNT DEV: [yes/no/minimal] # DEFAULT: @MOUNT_DEV_DEFAULT@ # Should we automatically bind mount /dev within the container? If 'minimal' # is chosen, then only 'null', 'zero', 'random', 'urandom', and 'shm' will # be included (the same effect as the --contain options) @MOUNT_DEV@ = @MOUNT_DEV_DEFAULT@ # MOUNT DEVPTS: [BOOL] # DEFAULT: @MOUNT_DEVPTS_DEFAULT@ # Should we mount a new instance of devpts if there is a 'minimal' # /dev, or -C is passed? Note, this requires that your kernel was # configured with CONFIG_DEVPTS_MULTIPLE_INSTANCES=y, or that you're # running kernel 4.7 or newer. @MOUNT_DEVPTS@ = @MOUNT_DEVPTS_DEFAULT@ # MOUNT HOME: [BOOL] # DEFAULT: @MOUNT_HOME_DEFAULT@ # Should we automatically determine the calling user's home directory and # attempt to mount it's base path into the container? If the --contain option # is used, the home directory will be created within the session directory or # can be overridden with the SINGULARITY_HOME or SINGULARITY_WORKDIR # environment variables (or their corresponding command line options). @MOUNT_HOME@ = @MOUNT_HOME_DEFAULT@ # MOUNT TMP: [BOOL] # DEFAULT: @MOUNT_TMP_DEFAULT@ # Should we automatically bind mount /tmp and /var/tmp into the container? If # the --contain option is used, both tmp locations will be created in the # session directory or can be specified via the SINGULARITY_WORKDIR # environment variable (or the --workingdir command line option). @MOUNT_TMP@ = @MOUNT_TMP_DEFAULT@ # MOUNT HOSTFS: [BOOL] # DEFAULT: @MOUNT_HOSTFS_DEFAULT@ # Probe for all mounted file systems that are mounted on the host, and bind # those into the container? @MOUNT_HOSTFS@ = @MOUNT_HOSTFS_DEFAULT@ # BIND PATH: [STRING] # DEFAULT: Undefined # Define a list of files/directories that should be made available from within # the container. The file or directory must exist within the container on # which to attach to. you can specify a different source and destination # path (respectively) with a colon; otherwise source and dest are the same. #bind path = /etc/singularity/default-nsswitch.conf:/etc/nsswitch.conf #bind path = /opt #bind path = /scratch bind path = /etc/localtime bind path = /etc/hosts # USER BIND CONTROL: [BOOL] # DEFAULT: @USER_BIND_CONTROL_DEFAULT@ # Allow users to influence and/or define bind points at runtime? This will allow # users to specify bind points, scratch and tmp locations. (note: User bind # control is only allowed if the host also supports PR_SET_NO_NEW_PRIVS) @USER_BIND_CONTROL@ = @USER_BIND_CONTROL_DEFAULT@ # ENABLE OVERLAY: [yes/no/try] # DEFAULT: @ENABLE_OVERLAY_DEFAULT@ # Enabling this option will make it possible to specify bind paths to locations # that do not currently exist within the container. If 'try' is chosen, # overlayfs will be tried but if it is unavailable it will be silently ignored. @ENABLE_OVERLAY@ = @ENABLE_OVERLAY_DEFAULT@ # MOUNT SLAVE: [BOOL] # DEFAULT: @MOUNT_SLAVE_DEFAULT@ # Should we automatically propagate file-system changes from the host? # This should be set to 'yes' when autofs mounts in the system should # show up in the container. @MOUNT_SLAVE@ = @MOUNT_SLAVE_DEFAULT@ # SESSIONDIR MAXSIZE: [STRING] # DEFAULT: @SESSIONDIR_MAXSIZE_DEFAULT@ # This specifies how large the default sessiondir should be (in MB) and it will # only affect users who use the "--contain" options and don't also specify a # location to do default read/writes to (e.g. "--workdir" or "--home"). @SESSIONDIR_MAXSIZE@ = @SESSIONDIR_MAXSIZE_DEFAULT@ # LIMIT CONTAINER OWNERS: [STRING] # DEFAULT: @LIMIT_CONTAINER_OWNERS_DEFAULT@ # Only allow containers to be used that are owned by a given user. If this # configuration is undefined (commented or set to NULL), all containers are # allowed to be used. This feature only applies when Singularity is running in # SUID mode and the user is non-root. #@LIMIT_CONTAINER_OWNERS@ = gmk, singularity, nobody # LIMIT CONTAINER PATHS: [STRING] # DEFAULT: @LIMIT_CONTAINER_PATHS_DEFAULT@ # Only allow containers to be used that are located within an allowed path # prefix. If this configuration is undefined (commented or set to NULL), # containers will be allowed to run from anywhere on the file system. This # feature only applies when Singularity is running in SUID mode and the user is # non-root. #@LIMIT_CONTAINER_PATHS@ = /scratch, /tmp, /global # ALLOW CONTAINER ${TYPE}: [BOOL] # DEFAULT: yes # This feature limits what kind of containers that Singularity will allow # users to use (note this does not apply for root). @ALLOW_CONTAINER_SQUASHFS@ = @ALLOW_CONTAINER_SQUASHFS_DEFAULT@ @ALLOW_CONTAINER_EXTFS@ = @ALLOW_CONTAINER_EXTFS_DEFAULT@ @ALLOW_CONTAINER_DIR@ = @ALLOW_CONTAINER_DIR_DEFAULT@ # AUTOFS BUG PATH: [STRING] # DEFAULT: Undefined # Define list of autofs directories which produces "Too many levels of symbolink links" # errors when accessed from container (typically bind mounts) #autofs bug path = /nfs #autofs bug path = /cifs-share singularity-2.4.2/.travis/0000755000175000017500000000000013211621077014415 5ustar mehdimehdisingularity-2.4.2/.travis/before_install0000755000175000017500000000143213211621077017333 0ustar mehdimehdi#!/bin/bash -ex wget -q -O - https://www.mongodb.org/static/pgp/server-3.2.pub | sudo apt-key add - if [ -z "$OS_TYPE" ]; then # default is ubuntu # install packages sudo apt-get update && sudo apt-get install -y flawfinder python-pip gcc # so that adjusted PATH propagates into sudo sudo sed -i -e 's/^Defaults\tsecure_path.*$//' /etc/sudoers # Install updated version of pylint sudo pip install pylint exit fi # run docker as shown at # https://djw8605.github.io/2016/05/03/building-centos-packages-on-travisci/ sudo apt-get update echo 'DOCKER_OPTS="-H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock -s devicemapper"' | \ sudo tee /etc/default/docker > /dev/null sudo service docker restart sleep 5 sudo docker pull ${OS_TYPE}:${OS_VERSION} singularity-2.4.2/.travis/centos_run0000755000175000017500000000205413211621077016523 0ustar mehdimehdi#!/bin/bash -ex # This script starts docker and (on el7) systemd and runs a test if [ "$OS_VERSION" = "6" ]; then sudo docker run --privileged --rm=true -v `pwd`:/build:rw centos:${OS_VERSION} /bin/bash -exc "cd /build;.travis/rpmbuild_test $OS_VERSION" exit fi # Mount /var/run/docker.sock and set --network=host so we can call docker from inside # cause some tests need it. Cannot mount to /var/run/docker.sock inside cause CentOS # /usr/sbin/init mounts another overlayfs on top of it docker run --privileged -d -ti -e "container=docker" -v /var/run/docker.sock:/docker.sock --network=host \ -v /sys/fs/cgroup:/sys/fs/cgroup -v `pwd`:/build:rw centos:${OS_VERSION} /usr/sbin/init DOCKER_CONTAINER_ID=$(docker ps | grep centos | awk '{print $1}') docker logs $DOCKER_CONTAINER_ID docker exec -ti $DOCKER_CONTAINER_ID /bin/bash -exc " export DOCKER_HOST=unix:///docker.sock chmod o+rw /docker.sock && cd /build && .travis/rpmbuild_test $OS_VERSION " docker ps -a docker stop $DOCKER_CONTAINER_ID docker rm -v $DOCKER_CONTAINER_ID singularity-2.4.2/.travis/rpmbuild_test0000755000175000017500000000156313211621077017225 0ustar mehdimehdi#!/bin/bash -ex # this script runs as root under docker OS_VERSION="$1" # build and install yum install -y libtool rpm-build make squashfs-tools ./autogen.sh ./configure make dist rpmbuild -ta *.tar.gz rpm -iv ~/rpmbuild/RPMS/*/*.rpm if [ "$OS_VERSION" = 6 ]; then echo "the test suite has not yet been ported to centos6 python, skipping." exit fi # Install python 3 and pylint yum install -y epel-release yum install -y python34 python34-pip python34-setuptools docker yum install -y pylint # run the test suite yum install -y libtool sudo which e2fsprogs sed -i 's,^prefix=.*,prefix=/usr,' test.sh sed -i 's,^sysconfdir=.*,sysconfdir=/etc,' test.sh sed -i 's,^localstatedir=.*,localstatedir=/var,' test.sh useradd testuser echo "Defaults:testuser env_keep=DOCKER_HOST" >>/etc/sudoers echo "testuser ALL=(ALL) NOPASSWD: ALL" >>/etc/sudoers su testuser -c "make test" singularity-2.4.2/.travis/script0000755000175000017500000000023213211621077015644 0ustar mehdimehdi#!/bin/bash -ex case $OS_TYPE in "") .travis/install_test ;; centos) .travis/centos_run ;; *) echo "Unknown OS_TYPE $OS_TYPE"; exit 1;; esac singularity-2.4.2/.travis/install_test0000755000175000017500000000013213211621077017044 0ustar mehdimehdi#!/bin/bash -ex ./autogen.sh ./configure --prefix=/usr/local sudo make install make test singularity-2.4.2/LICENSE-LBNL.md0000644000175000017500000000414713211621077015166 0ustar mehdimehdiCopyright (c) 2016-2017, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the University of California, Lawrence Berkeley National Laboratory, U.S. Dept. of Energy nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so. singularity-2.4.2/.travis.yml0000644000175000017500000000335513211621077015146 0ustar mehdimehdilanguage: cpp addons: apt: packages: - flawfinder - python-pip - squashfs-tools env: global: - secure: ZgH9zeVLyn7BvZo5IiVlBsmteLi1qsTwlCA6/60iIG+WJ1S1cbYRQeHyVvrat+wdjXqcJxRGyg/J6iGAFObIqngzSHPutyV48KgGV8tdPM0Rr+vtEJgW4b8pVcrlI2lsjgG1avqkW93SyOiaqWNE9cIQhlOKNEvIP2exqEuTra0muxl8rOWMpLTdKpA8NPsB0Xx+gRVZvwLtmSjqaSjNDrv/jt+HsnKazZQYKoewB+WO7hT/A4u0TcE+cggEep8LzjZt1xiU9O+OSHlibiKeYLDex+Vl6VtkNrlWYemFQR37nUJaleSXPFQ7H7DETmLCQtQ3NZSlrW/c+mwqFxJTfvgJ6xgK5fp7FlwMpqLBlgZrYAgyxSSC/N7o49zugYlJyoVKk9rDWGLRrxrmoX99hizsEE9bp6hyn9sXbBjsbzmQwhUOChiVAuEbGuLAYNyZIXbhfjbDUJVz748MQS0ymqK3jLrZOYTEs1/8IVk02FhRVKBdy6jHPbgD7egwls3P41jUgpmGxOLl9lvY/7lUo4ckrSso6jQELDIUcpyQiZxlOAhoic1sY6/vKqHB/RxUViufE/OdCDumwNcDfP8X8mpIeDUNn0ECogpx4ChpzdB/1c0YRRNT0O3E7qaVmyAApHCKcnZyskFWHr2FvCU2ToejgZxCqeP0YIdPMeROntA= - secure: "mOVQKTzAfpcemi9mLLts0h1aU2FhhNwurwkwiLBjiLYql+QgBX2wMWsCiVYx10trYil3jFCyEuPB4Zu84ukInSpb/QvKcBT7N1Cqpdkw2qrc2faPThPrmwyJ5gg7wb70sg0Fiz3eKY2fe1f6ab7ZNA+prvttskz4YKskkGIJoG6jH1JpD+T8jixozGSp3rgu4FkvMwkq4zzZ53R/8+eANfcleTA0evfPXfXv+n1nQjTMTCrsHeThzkXad/PZjqR4vMcCKdQ4g31B8uX17gOtoQvhe6jHCABFW5qzJ/Fgj6CE1LiEt6sLk0pwMjJ7vsbgdhgwveQker14CN5PumBNr08VFutpfJ/eEqVw2mIONUx6sfPExNa4V+xPTdXXVaLs55UiXOpi84FlNqn6AlmMMBQryDPMRcxudspXboHLReMP5fnikOjs1zBNJlUKoMFkcmR5A5jgZYwHE1WONVNevHM3wuhPaAjds4jxFPqckH6nTe3Zl5CTKRV4N+ieTd30y6dFcl1PXLFeE9kr2hH0PdCuxHK+W9zM/9nj8PZWhVyNe40OZl/Cvm4KNvTikSWGjzLlfaAlsgNXoxRxLpE7EniH+ZUgiEgbbxPETT4+z1W7uzIjJHPxjfJeImNAy2eMZ3CVeo4ZUuxt+f8rLGsyAkVhpj9ZQDPnICNoBV8H304=" sudo: true matrix: include: - os: linux - os: linux env: OS_TYPE=centos OS_VERSION=6 - os: linux env: OS_TYPE=centos OS_VERSION=7 services: - docker before_install: - .travis/before_install script: - .travis/script