mrb-0.1/0002755000175000017500000000000010430624173010312 5ustar ronronmrb-0.1/mrb.80000640000175000017500000001051110430624153011151 0ustar ronron.TH MRB 8 "May 9, 2006" .\" Please adjust this date whenever revising the manpage. .\" .\" Some roff macros, for reference: .\" .nh disable hyphenation .\" .hy enable hyphenation .\" .ad l left justify .\" .ad b justify to both left and right margins .\" .nf disable filling .\" .fi enable filling .\" .br insert line break .\" .sp insert n+1 empty lines .\" for manpage-specific macros, see man(7) .SH NAME mrb \- manage incremental snapshots with rsync/make. .SH SYNOPSIS .B mrb .I command .SH DESCRIPTION .B mrb is a simple aid to creating efficient incremental snapshots of a set, or sets, of directories whenever that may be required. It may be used as part of a regular automated backup regime, or for manually checkpointing changes at convenient points in time. .SH COMMANDS The following commands are recognised (where '\fIMODULE\fP' is the name of one of your snapshot definitions): .TP .BI new\- MODULE Create a skeleton definition for a new snapshot '\fIMODULE\fP'. .TP .BI dest\- MODULE Create the destination dir for '\fIMODULE\fP'. This directory must exist to create a snapshot. .TP .BI snap\- MODULE Create a snapshot of '\fIMODULE\fP'. .TP .B sync Create snapshots of all defined modules. If run as root this may be configured to include the modules of other users too (see MRB_SYNC_USERS in ~/.mrb/defaults). .TP .B help Show mrb's own help text. .SH CONFIGURATION FILES .SS Per-user configuration The following files may be used to specify global and local configuration options. .TP .I /etc/default/mrb system default configuration. .TP .I ~/.mrb/defaults per-user configuration. .SS Per-user options The following options control behaviour for all of a user's modules. .TP .B MRB_SNAPSHOT_LOG An optional file path where transfer details will be recorded. If unset these details will not be logged. .TP .B MRB_SYNC_USERS A space separated list of users whose modules should be included in a \fBsync\fP. This is mostly only useful for root, as mrb will assume the identity of each user before creating snapshots of their modules. If unset, only the invoking user's modules will be sync'ed. .TP .B MRB_CONFDIR An space separated list of the directories to search for module definition (*.mrc) files. They will be searched in the order given, with new modules added by default to the last one listed. There should be few reasons to change the default value. .SS Per-module configuration The default \fBMRB_CONFDIR\fP value will search for module definitions in: .TP .I /etc/mrb/*.mrc .TP .I ~/.mrb/*.mrc Those created by .BI new\- MODULE will be placed in this latter location by default. .SS Per-module options In each case \fImodule\fP below is the name of the particular module that the value set should apply to. These options should be defined in a file named \fImodule\fP.\fBmrc\fP. .TP .B module_SRC A space separated list of the files and (top level) directories to include in the snapshots for this module. .TP .B module_DEST The directory root where snapshots of \fImodule\fP should be stored. .TP .B module_INCLUDE An optional list of .BR rsync (1) include patterns. .TP .B module_EXCLUDE An optional list of .BR rsync (1) exclude patterns. .TP .B module_FILTER An optional list of .BR rsync (1) filter patterns. .TP .B module_FILTER_FILE An optional filename for .BR rsync (1) dir-merge filtering support. .TP .B module_RSYNC_OPTIONS Optional additional .BR rsync (1) options to pass verbatim when it is invoked. .TP .B module_PRECOMMAND An optional shell command to invoke just prior to creating a new snapshot. If the command does not return a successful exit status, then the snapshot creation will be aborted before it begins. It may be used to mount removable media or similar. .TP .B module_POSTCOMMAND An optional shell command to execute after making the snapshot. It will not be called if the snaphot creation failed at an earlier stage, and its return status may halt a sync operation if it fails with more modules still to process. It may be used, for example, to unmount removable media again. .TP .B module_USER An optional user name to check that mrb is running as before performing a snapshot. This can be used to ensure you have the correct permisson to access the files being mirrored before you get too far. .SH SEE ALSO .BR rsync (1), .BR make (1). .SH AUTHOR mrb was written by Ron . mrb-0.1/mrb0000750000175000017500000003576710430624153011031 0ustar ronron#!/usr/bin/make -f # # mrb make/rsync backup system # # Copyright 2006, Ron Lee. # This file is distributed under the terms of the GNU GPL version 2. # # WARNING: Misconfigured and/or buggy backup systems can easily # DESTROY YOUR PRECIOUS DATA # No warranty is, or can be, made against this happening # to you. Your only protection is your own care and # vigilance. This script is very simple, and should be # reasonably easy to audit for correctness according to # your particular needs. You are strongly encouraged to # carefully test any new configuration, or release of it, # before deploying it on your only copy of critical data. # # I may be a nice guy, and you may not even feel the need # to lock up your daughters or other valuables, but don't # trust me with your data. Please. It's too flimsy to # leave in the hands of someone whose attention is busy # enough worrying about their own. Your help to make sure # nobody will ever have to read this warning 'in anger' # will be greatly appreciated. So, on to the good bits ... # # $Id: mrb,v 1.12 2006-05-11 12:04:59 ron Exp $ # ------------------------------------------------------------------- # Configurable fundamentals. # A couple of simple axioms to kick things off. override THIS_MAKEFILE := $(lastword $(MAKEFILE_LIST)) GLOBAL_CONFDIR = /etc USER_CONFDIR = ~/.mrb USER_DEFAULTS = $(USER_CONFDIR)/defaults # Permit an expert user to reconfigure MRB_CONFDIR from an external file. # There probably isn't much else you'd want to tweak there, but read on # (carefully) if you really feel the need... -include $(GLOBAL_CONFDIR)/default/mrb -include $(USER_DEFAULTS) # The directories to search for module definitions. # A space separated list if multiple. They are included in the order # listed, but new modules will be added to only in the last one provided # by default. MRB_CONFDIR ?= $(GLOBAL_CONFDIR)/mrb $(USER_CONFDIR) -include $(addsuffix /*.mrc,$(MRB_CONFDIR)) # Some basic environment for the operations to come. USER ?= $(shell id -un) # No need for --delete if we are linking into a barren dir. RSYNC = rsync RSYNC_OPTIONS = --super -ahivS DATESTAMP := $(shell date +%Y%m%d-%H.%M) # ------------------------------------------------------------------- # The rules. # Default target explains how to press this button again properly. default: help # Convenience target for creating the skeleton of a new module. new-%: NEW_FILE = $(lastword $(MRB_CONFDIR))/$*.mrc new-%: @if [ x"$(MRB_CONFDIR)" = x ]; then \ echo; echo " Error: MRB_CONFDIR unset? Aborting."; echo; \ exit 1; \ fi @mkdir -p $(dir $(NEW_FILE)) @if [ -e $(NEW_FILE) ]; then \ echo; echo " Module '$*' already exists. Stopping."; echo;\ exit 1; \ fi @if [ ! -e $(USER_DEFAULTS) ]; then \ echo; \ echo " Creating $(USER_DEFAULTS)"; \ echo; \ echo "Edit this file to configure logging of transfers, or";\ echo "enable root 'sync' operations to create snapshots of";\ echo "other users' modules also."; \ \ echo "# This file configures mrb default settings for '$(USER)'" > $(USER_DEFAULTS);\ echo "# It was created on $(shell date) by $(THIS_MAKEFILE)" >> $(USER_DEFAULTS);\ echo >> $(USER_DEFAULTS); \ echo "# An optional file path to log transfer details to." >> $(USER_DEFAULTS);\ echo "MRB_SNAPSHOT_LOG = ~/.mrb/snapshot.log" >> $(USER_DEFAULTS);\ echo >> $(USER_DEFAULTS); \ echo "# A space separated list of users to include in a 'sync'." >> $(USER_DEFAULTS);\ echo "# This is typically only useful for the root user, as" >> $(USER_DEFAULTS);\ echo "# the identity of each user listed here is assumed when" >> $(USER_DEFAULTS);\ echo "# performing the sync of their modules." >> $(USER_DEFAULTS);\ echo "#MRB_SYNC_USERS = " >> $(USER_DEFAULTS); \ echo >> $(USER_DEFAULTS); \ fi @echo @echo " Creating skeleton $(NEW_FILE) ..." @echo "# This is the configuration file for the mrb snapshot module '$*'." > $(NEW_FILE) @echo "# It was created on $(shell date) for $(USER)" >> $(NEW_FILE) @echo "# by $(THIS_MAKEFILE)" >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# A space separated list of the source files and directories to be" >> $(NEW_FILE) @echo "# included in snapshots of this module. Specifying this is mandatory." >> $(NEW_FILE) @echo "#$*_SRC = " >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# The directory root where the snapshots should be stored." >> $(NEW_FILE) @echo "# Specifying this is mandatory." >> $(NEW_FILE) @echo "#$*_DEST = " >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# A pattern list for files under '$*_SRC' to explicitly include" >> $(NEW_FILE) @echo "# in the snapshot. Specifying this is optional." >> $(NEW_FILE) @echo "# See rsync(1) for syntax details." >> $(NEW_FILE) @echo "#$*_INCLUDE = *.o/ *.d/ core/ *.a/ *.dll/ *.mo/ *.lo/ *.la/ *.so/ \\" >> $(NEW_FILE) @echo "# *.init.d" >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# A pattern list for files under '$*_SRC' to explicitly exclude" >> $(NEW_FILE) @echo "# from the snapshot. Specifying this is optional." >> $(NEW_FILE) @echo "# See rsync(1) for syntax details." >> $(NEW_FILE) @echo "#$*_EXCLUDE = *.o *.d core .*.swp *.a *.dll *.gch *.mo *.lo *.la .libs/ \\" >> $(NEW_FILE) @echo "# *.so *.so.[0-9] *.so.[0-9].[0-9] *.so.[0-9].[0-9].[0-9]" >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# An rsync filter rule to apply for the snapshot." >> $(NEW_FILE) @echo "# Specifying this is optional. See rsync(1) for syntax details." >> $(NEW_FILE) @echo "#$*_FILTER = " >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# The file-name for an rsync per-directory filter to use if found." >> $(NEW_FILE) @echo "# Specifying this is optional. See rsync(1) for syntax details" >> $(NEW_FILE) @echo "# of 'dir-merge' filters. This is the name the file must match." >> $(NEW_FILE) @echo "# The default given here will scan all dirs from $*_SRC down." >> $(NEW_FILE) @echo "# Remove the leading '/' to search only the actual directory" >> $(NEW_FILE) @echo "# that is being transferred." >> $(NEW_FILE) @echo "#$*_FILTER_FILE = /.mrb-rsync-filter" >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# Optional additional rsync options to pass verbatim." >> $(NEW_FILE) @echo "#$*_RSYNC_OPTIONS = --prune-empty-dirs" >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# An optional shell command to execute before making the snapshot." >> $(NEW_FILE) @echo "# If the command does not return a successful exit status, then the" >> $(NEW_FILE) @echo "# snapshot creation will be aborted before it begins." >> $(NEW_FILE) @echo "# It may be used to mount $*_DEST on removable media or similar." >> $(NEW_FILE) @echo "#$*_PRECOMMAND = mount /mnt/backup" >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# An optional shell command to execute after making the snapshot." >> $(NEW_FILE) @echo "# It will not be called if the snaphot creation failed at an earlier" >> $(NEW_FILE) @echo "# stage. It's exit status will be passed to the user, but nothing" >> $(NEW_FILE) @echo "# remains to be done with it at this level." >> $(NEW_FILE) @echo "# It may be used, for example, to unmount removable media again." >> $(NEW_FILE) @echo "#$*_POSTCOMMAND = umount /mnt/backup" >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# An optional user name to check before performing a snapshot." >> $(NEW_FILE) @echo "# This can be used to ensure you have the correct permisson to" >> $(NEW_FILE) @echo "# access the files being mirrored before you get too far." >> $(NEW_FILE) @echo "$*_USER = $(USER)" >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo >> $(NEW_FILE) @echo "# Do not remove or edit this entry, it is used internally by mrb." >> $(NEW_FILE) @echo "$*_CONFIG = \$$(lastword \$$(MAKEFILE_LIST))" >> $(NEW_FILE) @echo @echo "Please edit this file now to define the details of the" @echo "snapshot which you would like it to create." @echo # Some basic checks shared by multiple operations. define BASIC_SANITY if [ x"$($*_CONFIG)" = x ]; then \ echo; \ echo " No module '$*'. Aborting backup run."; \ echo " You may create it with: \`$(THIS_MAKEFILE) new-$*\`";\ echo; \ exit 1; \ fi; \ if [ x"$($*_SRC)" = x ]; then \ echo; \ echo " Module $*_SRC not defined. Aborting backup run."; \ echo " Edit $($*_CONFIG) first, then try again."; \ echo; \ exit 1; \ fi; \ if [ x"$($*_DEST)" = x ]; then \ echo; \ echo " Module $*_DEST not defined. Aborting backup run."; \ echo " Edit $($*_CONFIG) first, then try again."; \ echo; \ exit 1; \ fi if [ x"$($*_USER)" != x ] && [ "$($*_USER)" != "$(USER)" ]; then\ echo; \ echo " Module requires user '$($*_USER)'. Aborting backup run.";\ echo " Edit $($*_CONFIG), or run as the required user."; \ echo; \ exit 1; \ fi endef # Convenience target for initialising a new module destination dir. # The snapshot target cowardly refuses to create new destinations # as a basic sanity trap. # Note: The pre- and post-commands are not run for this target. dest-%: @$(BASIC_SANITY) @echo; @echo " Creating '$($*_DEST)' dir" @echo; @mkdir -p $($*_DEST) # Syntax highlighting hack for the quote removal operation. DOUBLE_QUOTE := " #" # Helper target for issuing (more or less) arbitrary post-commands. do_postcommand-%: @[ x"$(subst $(DOUBLE_QUOTE),_,$($*_POSTCOMMAND))" = x ] || \ echo "Executing post-commands:" $($*_POSTCOMMAND) # The real worker target, creates a snapshot of some module. do_snap-%: LAST_SNAP = $(notdir $(lastword $(sort $(wildcard $($*_DEST)/$*-*.mrb)))) do_snap-%: THIS_SNAP = $($*_DEST)/$*-$(DATESTAMP).mrb do_snap-%: RSYNC_OPTIONS += $(addprefix --link-dest=,$(addprefix ../,$(LAST_SNAP))) do_snap-%: RSYNC_OPTIONS += $(addprefix --include=',$(addsuffix ',$($*_INCLUDE))) do_snap-%: RSYNC_OPTIONS += $(addprefix --exclude=',$(addsuffix ',$($*_EXCLUDE))) do_snap-%: RSYNC_OPTIONS += $(addprefix --filter=',$(addsuffix ',$($*_FILTER))) do_snap-%: RSYNC_OPTIONS += $(addprefix --filter=': ,$(addsuffix ',$($*_FILTER_FILE))) do_snap-%: RSYNC_OPTIONS += $($*_RSYNC_OPTIONS) do_snap-%: LOG_OUTPUT = $(addprefix 2>&1 | tee -a ,$(MRB_SNAPSHOT_LOG)) do_snap-%: # Begin with a little more sanity checking @for s in $($*_SRC); do \ if [ ! -r $$s ]; then \ echo; \ echo " Source '$$s' does not exist or is not readable.";\ echo " Aborting backup run."; \ echo; \ $(THIS_MAKEFILE) --no-print-directory do_postcommand-$*;\ exit 1; \ fi; \ done @if [ ! -d $($*_DEST) ]; then \ echo; \ echo " Destination dir '$($*_DEST)' does not exist."; \ echo " Aborting backup run."; \ echo " If this destination is correct, but new, you may create it now";\ echo " with the command: \`$(THIS_MAKEFILE) dest-$*\`"; \ echo; \ $(THIS_MAKEFILE) --no-print-directory do_postcommand-$*; \ exit 1; \ fi # Tell 'em what we're going to do. @echo @echo " Making $*-$(DATESTAMP) snapshot" @echo " from: $($*_SRC)" @echo " to : $($*_DEST)" @[ x"$($*_INCLUDE)" = x ] || echo " include : $($*_INCLUDE)" @[ x"$($*_EXCLUDE)" = x ] || echo " exclude : $($*_EXCLUDE)" @[ x"$($*_FILTER)" = x ] || echo " filter : $($*_FILTER)" @if [ x"$(LAST_SNAP)" = x ]; then \ echo " Performing full copy."; \ else \ echo " Incremental update based on '$(LAST_SNAP)'"; \ fi @echo # Do it. @mkdir $(THIS_SNAP) @for s in $($*_SRC); do \ echo "==> $(RSYNC) $(RSYNC_OPTIONS) $$s $(THIS_SNAP)" $(LOG_OUTPUT);\ $(RSYNC) $(RSYNC_OPTIONS) $$s $(THIS_SNAP) $(LOG_OUTPUT);\ done @[ x"$(MRB_SNAPSHOT_LOG)" = x ] || echo "Logged transfer to '$(MRB_SNAPSHOT_LOG)'" # Main user entry-point for creating new snapshots. # We do any PRECOMMAND's here, before forwarding to the main worker target, # as they may be needed to prepare the source and/or dest mount points # before the target specific variables are assigned their final values. snap-%: @$(BASIC_SANITY) @[ x"$(subst $(DOUBLE_QUOTE),_,$($*_PRECOMMAND))" = x ] || echo "Executing pre-commands:" $($*_PRECOMMAND) @$(THIS_MAKEFILE) --no-print-directory do_$@ @$(THIS_MAKEFILE) --no-print-directory do_postcommand-$* # Tell 'em its done. @echo; echo "All done. You may resume normal breathing patterns now." # Helper target for multiple user sync. user_sync-%: @echo; echo "Begin sync for '$*'" @su - $* -c '$(THIS_MAKEFILE) --no-print-directory sync' # Convenience target to create a snapshot of all defined modules. # If called by an ordinary user, it will find all the modules # available to that user. If MRB_SYNC_USERS contains a list of # users, it will try to sync for them as well. Typically this is # only useful for root, as we must assume their identity to # reliably perform that task. sync: $(addprefix snap-,$(notdir $(basename $(foreach d, \ $(MRB_CONFDIR),$(wildcard $(d)/*.mrc))))) \ $(addprefix user_sync-,$(MRB_SYNC_USERS)) @echo "Full sync for '$(USER)' done" # Convenience target to check the required config files are available. info: @echo @echo "$(THIS_MAKEFILE) is reading the following configuration files:" @for f in $(filter-out $(THIS_MAKEFILE),$(MAKEFILE_LIST)); do \ echo " $$f"; \ done @echo "as user '$(USER)'." @echo # If I have to explain this one, then I guess you are just reading this # 'for the articles' -- but I hope you'll have enjoyed it anyway... help: @echo @echo "Usage: $(THIS_MAKEFILE) " @echo @echo "mrb is a simple, on demand, backup system for taking a snapshot" @echo "of a set of directories. The following commands are recognised" @echo "(where 'MODULE' is the name of one of your snapshot definitions):" @echo @echo " new-MODULE - Create a skeleton definition for a new snapshot 'MODULE'." @echo " dest-MODULE - Create the destination dir for 'MODULE'." @echo " This directory must exist to create a snapshot." @echo @echo " snap-MODULE - Create a snapshot of 'MODULE'." @echo " sync - Create snapshots of all defined modules." @echo " If run as root this may be configured to include" @echo " the modules of other users too (see MRB_SYNC_USERS" @echo " in ~/.mrb/defaults)." @echo @echo " info - Report on the available config and module files." @echo " help - Display this text again." @echo .PHONY: default info help # TODO: # Support for remote stores. # How to link back to the last N snapshots if useful.. ? # (note you can presently override the value of LAST_SNAP, or # add additional --link-dest commands via module_RSYNC_OPTIONS) # Support for per-user precommand and postcommands instead # of executing the same ones repeatedly for each snapshot. # Report on file additions/removals since the last incremental. mrb-0.1/debian/0002755000175000017500000000000010430624173011534 5ustar ronronmrb-0.1/debian/changelog0000640000175000017500000000101610430022254013366 0ustar ronronmrb (0.1) unstable; urgency=low * Yes, the world is full of 'simple' backup systems already. And the bts is simply full of old unfixed bugs that point out the errors in their purported simplicity. So here is yet another one, hopefully so simple that it can't possibly ever have (m)any bugs of its own... Hats off especially to Mike Rubel and jw for wanting the right things and stirring this particular broth until my job was simple. -- Ron Lee Tue, 9 May 2006 14:10:19 +0930 mrb-0.1/debian/compat0000640000175000017500000000000210430017466012725 0ustar ronron5 mrb-0.1/debian/control0000640000175000017500000000146610430624153013136 0ustar ronronSource: mrb Section: admin Priority: optional Maintainer: Ron Lee Build-Depends: debhelper (>= 5.0.0) Standards-Version: 3.7.2.0 Package: mrb Architecture: all Depends: make, rsync Description: Manage incremental data snapshots with make/rsync mrb is a single, self-documenting, executable makefile, which aims to make trivial the task of maintaining a set of compact, incremental, rsync mirrors of your important (and sometimes rapidly changing) data. . It relies only on the time-hardened industry tools GNU make and rsync. Snapshots may be taken at any opportune interval. Multiple snapshot targets can be configured in a modular fashion, so fast changing data can be separated from static bulk data, with snapshots of each scheduled or triggered on demand, as may be appropriate for each. mrb-0.1/debian/copyright0000640000175000017500000000033010430022540013443 0ustar ronronThis package was Created by Ron on Tue, 9 May 2006 14:10:19 +0930. It is released under the terms of the GNU GPLv2. A copy of the GPL can be found in /usr/share/common-licenses on Debian systems. mrb-0.1/debian/rules0000750000175000017500000000203410430024245012576 0ustar ronron#!/usr/bin/make -f # -*- makefile -*- # Sample debian/rules that uses debhelper. # This file was originally written by Joey Hess and Craig Small. # As a special exception, when this file is copied by dh-make into a # dh-make output file, you may use that output file without restriction. # This special exception was added by Craig Small in version 0.37 of dh-make. # # Heavily pruned for mrb by Ron. # Uncomment this to turn on verbose mode. #export DH_VERBOSE=1 build: # nothing to build clean: dh_testdir dh_testroot dh_clean install: dh_testdir dh_testroot dh_clean -k dh_install mrb usr/bin dh_installman mrb.8 # Build architecture-independent files here. binary-indep: install # We have nothing to do by default. # Build architecture-dependent files here. binary-arch: install dh_testdir dh_testroot dh_installchangelogs dh_installdocs dh_compress dh_fixperms dh_installdeb dh_shlibdeps dh_gencontrol dh_md5sums dh_builddeb binary: binary-indep binary-arch .PHONY: build clean binary-indep binary-arch binary install