dobackup.pl
===========

Legal:

  Copyright 1997-2004 Webcon, Inc.

  This code is available for public use under the terms of the GNU Public
  License (GPL). See http://www.gnu.org/copyleft/gpl.html for license
  details. 

  This software is OSI Certified Open Source Software.
  OSI Certified is a certification mark of the Open Source Initiative.

  Webcon reserves the right to modify the licensing terms for any future
  version of this product.

Contact:

  website: http://www.webcon.ca/opensource/dobackup/
  email:   dobackup@webcon.ca

Quick overview:

  A flexible Perl script to handle unattended incremental backups of
  multiple servers. Handles multiple media sets with automatic media
  preparation and rotation, configurable 'what-to-backup', global exclusion
  patterns, user settable 'don't-backup-this-directory' metafiles. Design
  goal: zero-maintenance, nothing to do except change the media when told.

Getting started:

  Media preparation:
  ------------------

  Each unit of media used for dobackup must be uniquely identifiable. To
  this end, dobackup looks for a file named "MEDIAUNIT=<tag>" on the disk
  (in the mount-point really) before starting the backup. The reasons are
  two-fold: 1) to ensure that a media unit is really intended for backups,
  so you don't wipe out something important by accident, and 2) to verify
  that proper rotation of media units has occurred.

  To prepare each unit, format and mount it as usual, then:

  # touch MEDIAUNIT=<tag>
  # chmod 000 MEDIAUNIT=<tag>

  where <tag> is equal to the unit name, e.g. "JAZ01".

  If you have 4 JAZ cartridges, use names like JAZ01, JAZ02, JAZ03, JAZ04,
  then put each of these names in /etc/dobackup.conf, in the appropriate
  section. The names are really arbitrary. You could call the units "Larry",
  "Curly", and "Moe", if you wanted to, but they must be listed in the
  config file in the order in which they will be rotated. For example:

  [MediaSet=JazSetOne]
  Units= <<EOU
  JAZ01
  JAZ02
  JAZ03
  JAZ04
  EOU


  Scheduling:
  -----------

  Setup dobackup to run in root's crontab (/etc/crontab):

  Example 1: Backups to a permanent drive, daily incremental with a full
             backup on the 1st day of each month.

  #Daily Backups
  #(1:15am each day, full on the 1st of the month)
  15 1 1    * * root /usr/bin/perl /usr/sbin/dobackup.pl --fullbackup
  15 1 2-31 * * root /usr/bin/perl /usr/sbin/dobackup.pl

  Example 2: Backups to removable media, daily incremental with weekly media
             rotation. Full backup at the start of each week. Unmount device
             after each backup for extra security.

  #Daily Backups
  #(1:15am each day, eject Monday morning, ready for full backup Tuesday
  # morning)
  15 1 * * 0,3-6 root /usr/bin/perl /usr/sbin/dobackup.pl; umount /jaz
  15 1 * * 1     root /usr/bin/perl /usr/sbin/dobackup.pl --rotate
  15 1 * * 2     root /usr/bin/perl /usr/sbin/dobackup.pl --fullbackup; umount /jaz

Syntax:

  -h, --help       Prints this help message

  -d INT           Enable debugging mode (verbosity level = INT [1-65534])
  --debug INT      (If INT is omitted, a verbosity level of 1 is assumed)

  -V, --version    Prints version information

  --test           Tests to see if backup requirements have been satisfied.
                   Running in --test mode does not perform a backup.

  --config STRING  Use Configuration File named STRING.
                   (Default=/etc/dobackup.conf)

  --timestamp LONG Manually override all host timestamps with epoc+LONG.

  --fullbackup     Shortcut form of '--timestamp 0', forces a complete
                   non-incremental backup.

  --host STRING    Backup only host STRING. Host must exist in config file.

  --rotate         Eject current media and prepare config for next media
                   upon completion of this backup. Used with --test will
                   rotate media immediately without doing a backup.

  --setauto STRING When used in conjunction with --host, alter the automatic
                   backup flag for the given host according to STRING, which
                   can be one of 'on', 'off', or 'toggle', and then exit.

Description:

  General:
  --------

  This program is designed to be a flexible backup program. It will work
  with reliable removable media such as JAZ drives, potentially unreliable
  removable media like tape drives, dedicated drive partitions or simply a
  directory on an existing partition.

  We currently prefer to backup to dedicated drive partitions since they
  can be unmount'ed, offering greater data security and integrity and
  cannot be easily stolen like removable media (unless they swipe the
  whole server).

  The Program determines what to backup by comparing timestamps:
    1. If a Host's TimeStamp = 0, a full backup occurs.
    2. If a Host's TimeStamp exists and is not 0, then a backup of all
       files modified after the TimeStamp occurs.

    NEW: (v4.21)   A new "timewarp" mode has been added that acts much like
    the old incremental backups, but instead backs up everything that
    changed since the last FULL backup. This way only 2 files are ever
    required to perform a full restore to any given date. See the included
    sample config file for the config lines necessary to enable this mode.

  (TimeStamp is in seconds since Unix Epoch, 0 = 1970/01/01 00:00:00)


  Backup Exclusions:
  ------------------

  Rather than depending on filesystem-dependent flags or attributes, this
  program takes advantage of hidden "dot" (.) files for per-directory
  user-controllable exclusions.

  Any user can create a ".nobackup" file in any directory that they own. The
  contents of this file are read as one pattern per line to match files or
  directories that are not to be backed up under the current path.

  e.g. If the contents of /home/bob/.nobackup are:
	stuff
	private_thing

  then the items
	/home/bob/stuff
	/home/bob/private_things

  will be skipped during backup.

  More complex things can be excluded by using perl regex. For example, the
  following entries in /home/bob/.nobackup:

	\.mozilla/.*/Cache$
	.*\.(?:lha|zip|t?gz|Z|tar)$

  would prevent bob's Mozilla cache directory from being backed up, as well
  as any common archive formats.

  This makes it easy for users to exclude large files which don't need to be
  backed up without bothering the sysadmin.

  For backward-compatibility, by creating a zero-byte ".nobackup" file in
  any directory, that directory (and anything below it) are excluded from
  the backup.

  Note 1: Any pattern in a .nobackup file staring with "/" or ".." will be
          ignored.
  Note 2: The .nobackup file must have the same owner as the directory it
          resides in, or be owned by root.

  These regex patterns are also used in the "Excludes" per-host section of
  the config file for adding global exclusion patterns to your backups. See
  the sample config file for examples.


  Reliability:
  ------------

  The program is designed to be reliable. It makes use of many external
  tools, and since all paths are configurable and are potentially
  different on every system, all necessary external programs are verified
  prior to beginning the backup process, thereby ensuring the backup is
  able to complete successfully once started.

  By default, a backup summary report is e-mailed to the sysadmin (or any
  configured e-mail address) upon the completion of a successful backup, or
  upon encountering any internal failure.

  Verbose logs are stored by the program in a configurable log directory.
  Each log file is timestamped and host-stamped for easy tracking of
  individual backups. You can see exactly what files were backed up
  without having to dig out the actual backup media.

  Quite intentionally, the program does NOT check for enough remaining
  media space before starting the backup. In our opinion, it is better to
  have a partial backup written out, than none at all. Of course, your
  chosen archive tool (afio, by default) must be able to deal with
  partial files (afio does).

Preconditions:

  Program is intended to be run in root's crontab. All directories to be
  considered for backup should be listed one per line in the appropriate
  section of the config file. The system will recursively include any
  subdirectories under the ones listed.

Dependencies:

  On Redhat 9, dobackup uses tools from several packages. Not necessarily
  all of these will be required, depending on your specific configuration.

	afio (>= 2.5 for working large file support)
	eject
	fileutils
	grep
	iputils
	mount
	mt-st
	perl
	sendmail
	sh-utils
	textutils

  You'll also need these perl modules installed:

	Config::IniFiles

To-do:

  Please see the TODO file, part of this distribution.

Uncategorized notes that need to be expanded on:

  -Device used for backups should have an appropriate entry in /etc/fstab
   (Disk & Filesystem modes)

   Example:
	/dev/sda1 /jaz ext2 defaults,noauto 0 0

  -Tape mode had been unused for years, but has recently been put back into
   use by a few people. With a few minor updates to the code, tape mode seems
   to be back to working like a charm.

  -Multiple (remote) hosts can be backed up from one system. Appropriate NFS,
   SMB or other network filesystem mounts are left to the user. See sample
   config file.
