Documenting how to upgrade from atticmatic to borgmatic.

fixed typos in readme

.gitignore

@@ -0,0 +1,6 @@
+*.egg-info
+*.pyc
+*.swp
+.tox
+build
+dist

.hgignore

@@ -3,3 +3,5 @@ syntax: glob
 *.pyc
 *.swp
 .tox
+build
+dist

.hgtags

@@ -9,3 +9,20 @@ aa8a807f4ba28f0652764ed14713ffea2fd6922d 0.0.5
 569aef47a9b25c55b13753f94706f5d330219995 0.0.5
 a03495a8e8b471da63b5e2ae79d3ff9065839c2a 0.0.5
 7ea93ca83f426ec0a608a68580c72c0775b81f86 0.0.6
+cf4c7065f0711deda1cba878398bc05390e2c3f9 0.0.7
+38d72677343f0a5d6845f4ac50d6778397083d45 0.1.0
+ac5dfa01e9d14d09845f5e94c2c679e21c5eb2f9 0.1.1
+ac5dfa01e9d14d09845f5e94c2c679e21c5eb2f9 0.1.1
+7b6c87dca7ea312b2257ac1b46857b3f8c56b39c 0.1.1
+83067f995dd391e38544a7722dc3b254b59c5521 0.1.2
+acc7fb61566fe8028c179f43ecc735c851220b06 0.1.3
+6dda59c12de88f060eb7244e6d330173985a9639 0.1.4
+6dda59c12de88f060eb7244e6d330173985a9639 0.1.4
+e58246fc92bb22c2b2fd8b86a1227de69d2d0315 0.1.4
+0afff209b902698c2266986129d6dc9f5f913101 0.1.5
+4c63f3d90ec2bf6af1714a3acec84654a7c9edf3 0.1.6
+5a458ebef804be14e30d7375e3e9fbc26aedb80d 0.1.7
+977f19c2f6a515be6c5ef69cf17b0e0989532209 github/yaml_config_files
+0000000000000000000000000000000000000000 github/yaml_config_files
+28434dd0440cc8da44c2f3e9bd7e9402a59c3b40 github/master
+dbc96d3f83bd5570b6826537616d4160b3374836 0.1.8

AUTHORS

@@ -2,3 +2,4 @@ Dan Helfman <witten@torsion.org>: Main developer
 
 Alexander Görtz: Python 3 compatibility
 Henning Schroeder: Copy editing
+Robin `ypid` Schneider: Support additional options of Borg

NEWS

@@ -1,3 +1,55 @@
+1.0.0
+
+ * Attic is no longer supported, as there hasn't been any recent development on
+   it. Dropping Attic support will allow faster iteration on Borg-specific
+   features. If you're still using Attic, this is a good time to switch to Borg!
+ * Project renamed from atticmatic to borgmatic. See the borgmatic README for
+   information on upgrading.
+
+0.1.8
+
+ * Fix for handling of spaces in source_directories which resulted in backup up everything.
+ * Fix for broken links to Borg documentation.
+ * At verbosity zero, suppressing Borg check stderr spew about "Checking segments".
+ * Support for Borg --one-file-system.
+ * Support for Borg create --umask.
+ * Support for file globs in source_directories.
+
+0.1.7
+
+ * #11: Fixed parsing of punctuation in configuration file.
+ * Better error message when configuration file is missing.
+
+0.1.6
+
+ * #9: New configuration option for the encryption passphrase.
+ * #10: Support for Borg's new archive compression feature.
+
+0.1.5
+
+ * Changes to support release on PyPI. Now pip installable by name!
+
+0.1.4
+
+ * Adding test that setup.py version matches release version.
+
+0.1.3
+
+ * #1: Add support for "borg check --last N" to Borg backend.
+
+0.1.2
+
+ * As a convenience to new users, allow a missing default excludes file.
+ * New issue tracker, linked from documentation.
+
+0.1.1
+
+ * Adding borgmatic cron example, and updating documentation to refer to it.
+
+0.1.0
+
+ * New "borgmatic" command to support Borg backup software, a fork of Attic.
+
 0.0.7
 
  * Flag for multiple levels of verbosity: some, and lots.

README.md

@@ -1,77 +1,97 @@
-title: Atticmatic
-date:
-save_as: atticmatic/index.html
+title: Borgmatic
 
 ## Overview
 
-atticmatic is a simple Python wrapper script for the [Attic backup
-software](https://attic-backup.org/) that initiates a backup, prunes any old
-backups according to a retention policy, and validates backups for
-consistency. The script supports specifying your settings in a declarative
-configuration file rather than having to put them all on the command-line, and
-handles common errors.
+borgmatic (formerly atticmatic) is a simple Python wrapper script for the
+[Borg](https://borgbackup.readthedocs.org/en/stable/) backup software that
+initiates a backup, prunes any old backups according to a retention policy,
+and validates backups for consistency. The script supports specifying your
+settings in a declarative configuration file rather than having to put them
+all on the command-line, and handles common errors.
 
 Here's an example config file:
 
-    [location]
-    # Space-separated list of source directories to backup.
-    source_directories: /home /etc
+```INI
+[location]
+# Space-separated list of source directories to backup.
+# Globs are expanded.
+source_directories: /home /etc /var/log/syslog*
 
-    # Path to local or remote Attic repository.
-    repository: user@backupserver:sourcehostname.attic
+# Path to local or remote backup repository.
+repository: user@backupserver:sourcehostname.borg
 
-    [retention]
-    # Retention policy for how many backups to keep in each category.
-    keep_daily: 7
-    keep_weekly: 4
-    keep_monthly: 6
+[retention]
+# Retention policy for how many backups to keep in each category.
+keep_daily: 7
+keep_weekly: 4
+keep_monthly: 6
 
-    [consistency]
-    # Consistency checks to run, or "disabled" to prevent checks.
-    checks: repository archives
+[consistency]
+# Consistency checks to run, or "disabled" to prevent checks.
+checks: repository archives
+```
 
 Additionally, exclude patterns can be specified in a separate excludes config
 file, one pattern per line.
 
-atticmatic is hosted at <https://torsion.org/atticmatic> with [source code
-available](https://torsion.org/hg/atticmatic). It's also mirrored on
-[GitHub](https://github.com/witten/atticmatic) and
-[BitBucket](https://bitbucket.org/dhelfman/atticmatic) for convenience.
+borgmatic is hosted at <https://torsion.org/borgmatic> with [source code
+available](https://torsion.org/hg/borgmatic). It's also mirrored on
+[GitHub](https://github.com/witten/borgmatic) and
+[BitBucket](https://bitbucket.org/dhelfman/borgmatic) for convenience.
 
 
 ## Setup
 
-To get up and running with Attic, follow the [Attic Quick
-Start](https://attic-backup.org/quickstart.html) guide to create an Attic
-repository on a local or remote host. Note that if you plan to run atticmatic
-on a schedule with cron, and you encrypt your attic repository with a
-passphrase instead of a key file, you'll need to set the `ATTIC_PASSPHRASE`
-environment variable. See [attic's repository encryption
-documentation](https://attic-backup.org/quickstart.html#encrypted-repos) for
-more info.
+To get up and running, follow the [Borg Quick
+Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) to create
+a repository on a local or remote host. Note that if you plan to run
+borgmatic on a schedule with cron, and you encrypt your Borg repository with
+a passphrase instead of a key file, you'll need to set the borgmatic
+`encryption_passphrase` configuration variable. See the repository encryption
+section of the Quick Start for more info.
 
 If the repository is on a remote host, make sure that your local root user has
 key-based ssh access to the desired user account on the remote host.
 
-To install atticmatic, run the following command to download and install it:
+To install borgmatic, run the following command to download and install it:
 
-    sudo pip install --upgrade hg+https://torsion.org/hg/atticmatic
+    sudo pip install --upgrade borgmatic
 
-Then copy the following configuration files:
+Then, copy the following configuration files:
 
-    sudo cp sample/atticmatic.cron /etc/cron.d/atticmatic
-    sudo mkdir /etc/atticmatic/
-    sudo cp sample/config sample/excludes /etc/atticmatic/
+    sudo cp sample/borgmatic.cron /etc/cron.d/borgmatic
+    sudo mkdir /etc/borgmatic/
+    sudo cp sample/config sample/excludes /etc/borgmatic/
 
-Lastly, modify those files with your desired configuration.
+Lastly, modify the /etc files with your desired configuration.
 
 
+## Upgrading from atticmatic
+
+You can ignore this section if you're not an atticmatic user (the former name
+of borgmatic).
+
+borgmatic only supports Borg now and no longer supports Attic. So if you're
+an Attic user, consider switching to Borg. See the [Borg upgrade
+command](https://borgbackup.readthedocs.io/en/stable/usage.html#borg-upgrade)
+for more information. Then, follow the instructions above about setting up
+your borgmatic configuration files.
+
+If you were already using Borg with atticmatic, then you can easily upgrade
+from atticmatic to borgmatic. Simply run the following commands:
+
+    sudo pip uninstall atticmatic
+    sudo pip install borgmatic
+
+That's it! borgmatic will continue using your /etc/borgmatic configuration
+files.
+
 ## Usage
 
-You can run atticmatic and start a backup simply by invoking it without
+You can run borgmatic and start a backup simply by invoking it without
 arguments:
 
-    atticmatic
+    borgmatic
 
 This will also prune any old backups as per the configured retention policy,
 and check backups for consistency problems due to things like file damage.
@@ -80,15 +100,15 @@ By default, the backup will proceed silently except in the case of errors. But
 if you'd like to to get additional information about the progress of the
 backup as it proceeds, use the verbosity option:
 
-    atticmattic --verbosity 1
+    borgmatic --verbosity 1
 
 Or, for even more progress spew:
 
-    atticmattic --verbosity 2
+    borgmatic --verbosity 2
 
 If you'd like to see the available command-line arguments, view the help:
 
-    atticmattic --help
+    borgmatic --help
 
 
 ## Running tests
@@ -106,12 +126,12 @@ Then, to actually run tests, run:
 
 ### Broken pipe with remote repository
 
-When running atticmatic on a large remote repository, you may receive errors
-like the following, particularly while "attic check" is valiating backups for
+When running borgmatic on a large remote repository, you may receive errors
+like the following, particularly while "borg check" is validating backups for
 consistency:
 
     Write failed: Broken pipe
-    attic: Error: Connection closed by remote host
+    borg: Error: Connection closed by remote host
 
 This error can be caused by an ssh timeout, which you can rectify by adding
 the following to the ~/.ssh/config file on the client:
@@ -123,6 +143,11 @@ This should make the client keep the connection alive while validating
 backups.
 
 
-## Feedback
+## Issues and feedback
 
-Questions? Comments? Got a patch? Contact <mailto:witten@torsion.org>.
+Got an issue or an idea for a feature enhancement? Check out the [borgmatic
+issue tracker](https://tree.taiga.io/project/witten-borgmatic/issues). In
+order to create a new issue or comment on an issue, you'll need to [login
+first](https://tree.taiga.io/login).
+
+Other questions or comments? Contact <mailto:witten@torsion.org>.

atticmatic/command.py

@@ -1,51 +0,0 @@
-from __future__ import print_function
-from argparse import ArgumentParser
-from subprocess import CalledProcessError
-import sys
-
-from atticmatic.attic import check_archives, create_archive, prune_archives
-from atticmatic.config import parse_configuration
-
-
-DEFAULT_CONFIG_FILENAME = '/etc/atticmatic/config'
-DEFAULT_EXCLUDES_FILENAME = '/etc/atticmatic/excludes'
-
-
-def parse_arguments(*arguments):
-    '''
-    Parse the given command-line arguments and return them as an ArgumentParser instance.
-    '''
-    parser = ArgumentParser()
-    parser.add_argument(
-        '-c', '--config',
-        dest='config_filename',
-        default=DEFAULT_CONFIG_FILENAME,
-        help='Configuration filename',
-    )
-    parser.add_argument(
-        '--excludes',
-        dest='excludes_filename',
-        default=DEFAULT_EXCLUDES_FILENAME,
-        help='Excludes filename',
-    )
-    parser.add_argument(
-        '-v', '--verbosity',
-        type=int,
-        help='Display verbose progress (1 for some, 2 for lots)',
-    )
-
-    return parser.parse_args(arguments)
-
-
-def main():
-    try:
-        args = parse_arguments(*sys.argv[1:])
-        config = parse_configuration(args.config_filename)
-        repository = config.location['repository']
-
-        create_archive(args.excludes_filename, args.verbosity, **config.location)
-        prune_archives(args.verbosity, repository, config.retention)
-        check_archives(args.verbosity, repository, config.consistency)
-    except (ValueError, IOError, CalledProcessError) as error:
-        print(error, file=sys.stderr)
-        sys.exit(1)

borgmatic/borg.py

@@ -1,33 +1,60 @@
 from datetime import datetime
 import os
+import re
 import platform
 import subprocess
+from glob import glob
+from itertools import chain
 
-from atticmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
 
 
-def create_archive(excludes_filename, verbosity, source_directories, repository):
+# Integration with Borg for actually handling backups.
+
+
+COMMAND = 'borg'
+
+
+def initialize(storage_config, command=COMMAND):
+    passphrase = storage_config.get('encryption_passphrase')
+
+    if passphrase:
+        os.environ['{}_PASSPHRASE'.format(command.upper())] = passphrase
+
+
+def create_archive(
+    excludes_filename, verbosity, storage_config, source_directories, repository, command=COMMAND,
+    one_file_system=None
+):
     '''
-    Given an excludes filename, a vebosity flag, a space-separated list of source directories, and
-    a local or remote repository path, create an attic archive.
+    Given an excludes filename (or None), a vebosity flag, a storage config dict, a space-separated
+    list of source directories, a local or remote repository path, and a command to run, create an
+    attic archive.
     '''
-    sources = tuple(source_directories.split(' '))
+    sources = re.split('\s+', source_directories)
+    sources = tuple(chain.from_iterable(glob(x) or [x] for x in sources))
+    exclude_flags = ('--exclude-from', excludes_filename) if excludes_filename else ()
+    compression = storage_config.get('compression', None)
+    compression_flags = ('--compression', compression) if compression else ()
+    umask = storage_config.get('umask', None)
+    umask_flags = ('--umask', str(umask)) if umask else ()
+    one_file_system_flags = ('--one-file-system',) if one_file_system else ()
     verbosity_flags = {
         VERBOSITY_SOME: ('--stats',),
         VERBOSITY_LOTS: ('--verbose', '--stats'),
     }.get(verbosity, ())
 
-    command = (
-        'attic', 'create',
-        '--exclude-from', excludes_filename,
+    full_command = (
+        command, 'create',
         '{repo}::{hostname}-{timestamp}'.format(
             repo=repository,
             hostname=platform.node(),
             timestamp=datetime.now().isoformat(),
         ),
-    ) + sources + verbosity_flags
+    ) + sources + exclude_flags + compression_flags + one_file_system_flags + \
+        umask_flags + verbosity_flags
 
-    subprocess.check_call(command)
+    subprocess.check_call(full_command)
 
 
 def _make_prune_flags(retention_config):
@@ -52,18 +79,19 @@ def _make_prune_flags(retention_config):
     )
 
 
-def prune_archives(verbosity, repository, retention_config):
+def prune_archives(verbosity, repository, retention_config, command=COMMAND):
     '''
-    Given a verbosity flag, a local or remote repository path, and a retention config dict, prune
-    attic archives according the the retention policy specified in that configuration.
+    Given a verbosity flag, a local or remote repository path, a retention config dict, and a
+    command to run, prune attic archives according the the retention policy specified in that
+    configuration.
     '''
     verbosity_flags = {
         VERBOSITY_SOME: ('--stats',),
         VERBOSITY_LOTS: ('--verbose', '--stats'),
     }.get(verbosity, ())
 
-    command = (
-        'attic', 'prune',
+    full_command = (
+        command, 'prune',
         repository,
     ) + tuple(
         element
@@ -71,7 +99,7 @@ def prune_archives(verbosity, repository, retention_config):
         for element in pair
     ) + verbosity_flags
 
-    subprocess.check_call(command)
+    subprocess.check_call(full_command)
 
 
 DEFAULT_CHECKS = ('repository', 'archives')
@@ -103,7 +131,7 @@ def _parse_checks(consistency_config):
     )
 
 
-def _make_check_flags(checks):
+def _make_check_flags(checks, check_last=None):
     '''
     Given a parsed sequence of checks, transform it into tuple of command-line flags.
 
@@ -112,25 +140,30 @@ def _make_check_flags(checks):
         ('repository',)
 
     This will be returned as:
-    
+
         ('--repository-only',)
+
+    Additionally, if a check_last value is given, a "--last" flag will be added. Note that only
+    Borg supports this flag.
     '''
+    last_flag = ('--last', check_last) if check_last else ()
     if checks == DEFAULT_CHECKS:
-        return ()
+        return last_flag
 
     return tuple(
         '--{}-only'.format(check) for check in checks
-    )
+    ) + last_flag
 
 
-def check_archives(verbosity, repository, consistency_config):
+def check_archives(verbosity, repository, consistency_config, command=COMMAND):
     '''
-    Given a verbosity flag, a local or remote repository path, and a consistency config dict, check
-    the contained attic archives for consistency.
+    Given a verbosity flag, a local or remote repository path, a consistency config dict, and a
+    command to run, check the contained attic archives for consistency.
 
     If there are no consistency checks to run, skip running them.
     '''
     checks = _parse_checks(consistency_config)
+    check_last = consistency_config.get('check_last', None)
     if not checks:
         return
 
@@ -139,12 +172,12 @@ def check_archives(verbosity, repository, consistency_config):
         VERBOSITY_LOTS: ('--verbose',),
     }.get(verbosity, ())
 
-    command = (
-        'attic', 'check',
+    full_command = (
+        command, 'check',
         repository,
-    ) + _make_check_flags(checks) + verbosity_flags
+    ) + _make_check_flags(checks, check_last) + verbosity_flags
 
-    # Attic's check command spews to stdout even without the verbose flag. Suppress it.
+    # The check command spews to stdout/stderr even without the verbose flag. Suppress it.
     stdout = None if verbosity_flags else open(os.devnull, 'w')
 
-    subprocess.check_call(command, stdout=stdout)
+    subprocess.check_call(full_command, stdout=stdout, stderr=subprocess.STDOUT)

borgmatic/command.py

@@ -0,0 +1,57 @@
+from __future__ import print_function
+from argparse import ArgumentParser
+import os
+from subprocess import CalledProcessError
+import sys
+
+from borgmatic import borg
+from borgmatic.config import parse_configuration, CONFIG_FORMAT
+
+
+DEFAULT_CONFIG_FILENAME = '/etc/borgmatic/config'
+DEFAULT_EXCLUDES_FILENAME = '/etc/borgmatic/excludes'
+
+
+def parse_arguments(*arguments):
+    '''
+    Given the name of the command with which this script was invoked and command-line arguments,
+    parse the arguments and return them as an ArgumentParser instance. Use the command name to
+    determine the default configuration and excludes paths.
+    '''
+    parser = ArgumentParser()
+    parser.add_argument(
+        '-c', '--config',
+        dest='config_filename',
+        default=DEFAULT_CONFIG_FILENAME,
+        help='Configuration filename',
+    )
+    parser.add_argument(
+        '--excludes',
+        dest='excludes_filename',
+        default=DEFAULT_EXCLUDES_FILENAME if os.path.exists(DEFAULT_EXCLUDES_FILENAME) else None,
+        help='Excludes filename',
+    )
+    parser.add_argument(
+        '-v', '--verbosity',
+        type=int,
+        help='Display verbose progress (1 for some, 2 for lots)',
+    )
+
+    return parser.parse_args(arguments)
+
+
+def main():
+    try:
+        args = parse_arguments(*sys.argv[1:])
+        config = parse_configuration(args.config_filename, CONFIG_FORMAT)
+        repository = config.location['repository']
+
+        borg.initialize(config.storage)
+        borg.create_archive(
+            args.excludes_filename, args.verbosity, config.storage, **config.location
+        )
+        borg.prune_archives(args.verbosity, repository, config.retention)
+        borg.check_archives(args.verbosity, repository, config.consistency)
+    except (ValueError, IOError, CalledProcessError) as error:
+        print(error, file=sys.stderr)
+        sys.exit(1)

borgmatic/config.py

@@ -2,10 +2,10 @@ from collections import OrderedDict, namedtuple
 
 try:
     # Python 2
-    from ConfigParser import ConfigParser
+    from ConfigParser import RawConfigParser
 except ImportError:
     # Python 3
-    from configparser import ConfigParser
+    from configparser import RawConfigParser
 
 
 Section_format = namedtuple('Section_format', ('name', 'options'))
@@ -25,9 +25,18 @@ CONFIG_FORMAT = (
         'location',
         (
             option('source_directories'),
+            option('one_file_system', value_type=bool, required=False),
             option('repository'),
         ),
     ),
+    Section_format(
+        'storage',
+        (
+            option('encryption_passphrase', required=False),
+            option('compression', required=False),
+            option('umask', required=False),
+        ),
+    ),
     Section_format(
         'retention',
         (
@@ -44,6 +53,7 @@ CONFIG_FORMAT = (
         'consistency',
         (
             option('checks', required=False),
+            option('check_last', required=False),
         ),
     )
 )
@@ -51,7 +61,7 @@ CONFIG_FORMAT = (
 
 def validate_configuration_format(parser, config_format):
     '''
-    Given an open ConfigParser and an expected config file format, validate that the parsed
+    Given an open RawConfigParser and an expected config file format, validate that the parsed
     configuration file has the expected sections, that any required options are present in those
     sections, and that there aren't any unexpected options.
 
@@ -110,14 +120,9 @@ def validate_configuration_format(parser, config_format):
             )
 
 
-# Describes a parsed configuration, where each attribute is the name of a configuration file section 
-# and each value is a dict of that section's parsed options.
-Parsed_config = namedtuple('Config', (section_format.name for section_format in CONFIG_FORMAT))
-
-
 def parse_section_options(parser, section_format):
     '''
-    Given an open ConfigParser and an expected section format, return the option values from that
+    Given an open RawConfigParser and an expected section format, return the option values from that
     section as a dict mapping from option name to value. Omit those options that are not present in
     the parsed options.
 
@@ -126,6 +131,7 @@ def parse_section_options(parser, section_format):
     type_getter = {
         str: parser.get,
         int: parser.getint,
+        bool: parser.getboolean,
     }
 
     return OrderedDict(
@@ -135,21 +141,26 @@ def parse_section_options(parser, section_format):
     )
 
 
-def parse_configuration(config_filename):
+def parse_configuration(config_filename, config_format):
     '''
-    Given a config filename of the expected format, return the parsed configuration as Parsed_config
-    data structure.
+    Given a config filename and an expected config file format, return the parsed configuration
+    as a namedtuple with one attribute for each parsed section.
 
     Raise IOError if the file cannot be read, or ValueError if the format is not as expected.
     '''
-    parser = ConfigParser()
-    parser.read(config_filename)
+    parser = RawConfigParser()
+    if not parser.read(config_filename):
+        raise ValueError('Configuration file cannot be opened: {}'.format(config_filename))
 
-    validate_configuration_format(parser, CONFIG_FORMAT)
+    validate_configuration_format(parser, config_format)
+
+    # Describes a parsed configuration, where each attribute is the name of a configuration file
+    # section and each value is a dict of that section's parsed options.
+    Parsed_config = namedtuple('Parsed_config', (section_format.name for section_format in config_format))
 
     return Parsed_config(
         *(
             parse_section_options(parser, section_format)
-            for section_format in CONFIG_FORMAT
+            for section_format in config_format
         )
     )

borgmatic/tests/integration/test_command.py

@@ -1,11 +1,15 @@
+import os
 import sys
 
-from nose.tools import assert_raises
+from flexmock import flexmock
+import pytest
 
-from atticmatic import command as module
+from borgmatic import command as module
 
 
 def test_parse_arguments_with_no_arguments_uses_defaults():
+    flexmock(os.path).should_receive('exists').and_return(True)
+
     parser = module.parse_arguments()
 
     assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME
@@ -14,6 +18,8 @@ def test_parse_arguments_with_no_arguments_uses_defaults():
 
 
 def test_parse_arguments_with_filename_arguments_overrides_defaults():
+    flexmock(os.path).should_receive('exists').and_return(True)
+
     parser = module.parse_arguments('--config', 'myconfig', '--excludes', 'myexcludes')
 
     assert parser.config_filename == 'myconfig'
@@ -21,7 +27,29 @@ def test_parse_arguments_with_filename_arguments_overrides_defaults():
     assert parser.verbosity == None
 
 
+def test_parse_arguments_with_missing_default_excludes_file_sets_filename_to_none():
+    flexmock(os.path).should_receive('exists').and_return(False)
+
+    parser = module.parse_arguments()
+
+    assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME
+    assert parser.excludes_filename == None
+    assert parser.verbosity == None
+
+
+def test_parse_arguments_with_missing_overridden_excludes_file_retains_filename():
+    flexmock(os.path).should_receive('exists').and_return(False)
+
+    parser = module.parse_arguments('--excludes', 'myexcludes')
+
+    assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME
+    assert parser.excludes_filename == 'myexcludes'
+    assert parser.verbosity == None
+
+
 def test_parse_arguments_with_verbosity_flag_overrides_default():
+    flexmock(os.path).should_receive('exists').and_return(True)
+
     parser = module.parse_arguments('--verbosity', '1')
 
     assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME
@@ -30,11 +58,12 @@ def test_parse_arguments_with_verbosity_flag_overrides_default():
 
 
 def test_parse_arguments_with_invalid_arguments_exits():
+    flexmock(os.path).should_receive('exists').and_return(True)
     original_stderr = sys.stderr
     sys.stderr = sys.stdout
 
     try:
-        with assert_raises(SystemExit):
+        with pytest.raises(SystemExit):
             module.parse_arguments('--posix-me-harder')
     finally:
         sys.stderr = original_stderr

borgmatic/tests/integration/test_config.py

@@ -0,0 +1,29 @@
+try:
+    # Python 2
+    from cStringIO import StringIO
+except ImportError:
+    # Python 3
+    from io import StringIO
+
+from collections import OrderedDict
+import string
+
+from borgmatic import config as module
+
+
+def test_parse_section_options_with_punctuation_should_return_section_options():
+    parser = module.RawConfigParser()
+    parser.readfp(StringIO('[section]\nfoo: {}\n'.format(string.punctuation)))
+
+    section_format = module.Section_format(
+        'section',
+        (module.Config_option('foo', str, required=True),),
+    )
+
+    config = module.parse_section_options(parser, section_format)
+
+    assert config == OrderedDict(
+        (
+            ('foo', string.punctuation),
+        )
+    )

borgmatic/tests/integration/test_version.py

@@ -0,0 +1,8 @@
+import subprocess
+
+
+def test_setup_version_matches_news_version():
+    setup_version = subprocess.check_output(('python', 'setup.py', '--version')).decode('ascii')
+    news_version = open('NEWS').readline()
+
+    assert setup_version == news_version

borgmatic/tests/unit/test_borg.py

@@ -1,14 +1,38 @@
 from collections import OrderedDict
+from subprocess import STDOUT
+import os
 
 from flexmock import flexmock
 
-from atticmatic import attic as module
-from atticmatic.tests.builtins import builtins_mock
-from atticmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+from borgmatic import borg as module
+from borgmatic.tests.builtins import builtins_mock
+from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+
+
+def test_initialize_with_passphrase_should_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize({'encryption_passphrase': 'pass'}, command='attic')
+        assert os.environ.get('ATTIC_PASSPHRASE') == 'pass'
+    finally:
+        os.environ = orig_environ
+
+
+def test_initialize_without_passphrase_should_not_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize({}, command='attic')
+        assert os.environ.get('ATTIC_PASSPHRASE') == None
+    finally:
+        os.environ = orig_environ
 
 
 def insert_subprocess_mock(check_call_command, **kwargs):
-    subprocess = flexmock()
+    subprocess = flexmock(STDOUT=STDOUT)
     subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
     flexmock(module).subprocess = subprocess
 
@@ -29,7 +53,8 @@ def insert_datetime_mock():
     ).mock
 
 
-CREATE_COMMAND = ('attic', 'create', '--exclude-from', 'excludes', 'repo::host-now', 'foo', 'bar')
+CREATE_COMMAND_WITHOUT_EXCLUDES = ('attic', 'create', 'repo::host-now', 'foo', 'bar')
+CREATE_COMMAND = CREATE_COMMAND_WITHOUT_EXCLUDES + ('--exclude-from', 'excludes')
 
 
 def test_create_archive_should_call_attic_with_parameters():
@@ -40,8 +65,40 @@ def test_create_archive_should_call_attic_with_parameters():
     module.create_archive(
         excludes_filename='excludes',
         verbosity=None,
+        storage_config={},
         source_directories='foo bar',
         repository='repo',
+        command='attic',
+    )
+
+
+def test_create_archive_with_two_spaces_in_source_directories():
+    insert_subprocess_mock(CREATE_COMMAND)
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        excludes_filename='excludes',
+        verbosity=None,
+        storage_config={},
+        source_directories='foo  bar',
+        repository='repo',
+        command='attic',
+    )
+
+
+def test_create_archive_with_none_excludes_filename_should_call_attic_without_excludes():
+    insert_subprocess_mock(CREATE_COMMAND_WITHOUT_EXCLUDES)
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        excludes_filename=None,
+        verbosity=None,
+        storage_config={},
+        source_directories='foo bar',
+        repository='repo',
+        command='attic',
     )
 
 
@@ -53,8 +110,10 @@ def test_create_archive_with_verbosity_some_should_call_attic_with_stats_paramet
     module.create_archive(
         excludes_filename='excludes',
         verbosity=VERBOSITY_SOME,
+        storage_config={},
         source_directories='foo bar',
         repository='repo',
+        command='attic',
     )
 
 
@@ -66,8 +125,104 @@ def test_create_archive_with_verbosity_lots_should_call_attic_with_verbose_param
     module.create_archive(
         excludes_filename='excludes',
         verbosity=VERBOSITY_LOTS,
+        storage_config={},
         source_directories='foo bar',
         repository='repo',
+        command='attic',
+    )
+
+
+def test_create_archive_with_compression_should_call_attic_with_compression_parameters():
+    insert_subprocess_mock(CREATE_COMMAND + ('--compression', 'rle'))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        excludes_filename='excludes',
+        verbosity=None,
+        storage_config={'compression': 'rle'},
+        source_directories='foo bar',
+        repository='repo',
+        command='attic',
+    )
+
+
+def test_create_archive_with_one_file_system_should_call_attic_with_one_file_system_parameters():
+    insert_subprocess_mock(CREATE_COMMAND + ('--one-file-system',))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        excludes_filename='excludes',
+        verbosity=None,
+        storage_config={},
+        source_directories='foo bar',
+        repository='repo',
+        command='attic',
+        one_file_system=True,
+    )
+
+
+def test_create_archive_with_umask_should_call_attic_with_umask_parameters():
+    insert_subprocess_mock(CREATE_COMMAND + ('--umask', '740'))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        excludes_filename='excludes',
+        verbosity=None,
+        storage_config={'umask': 740},
+        source_directories='foo bar',
+        repository='repo',
+        command='attic',
+    )
+
+
+def test_create_archive_with_source_directories_glob_expands():
+    insert_subprocess_mock(('attic', 'create', 'repo::host-now', 'foo', 'food'))
+    insert_platform_mock()
+    insert_datetime_mock()
+    flexmock(module).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
+
+    module.create_archive(
+        excludes_filename=None,
+        verbosity=None,
+        storage_config={},
+        source_directories='foo*',
+        repository='repo',
+        command='attic',
+    )
+
+
+def test_create_archive_with_non_matching_source_directories_glob_passes_through():
+    insert_subprocess_mock(('attic', 'create', 'repo::host-now', 'foo*'))
+    insert_platform_mock()
+    insert_datetime_mock()
+    flexmock(module).should_receive('glob').with_args('foo*').and_return([])
+
+    module.create_archive(
+        excludes_filename=None,
+        verbosity=None,
+        storage_config={},
+        source_directories='foo*',
+        repository='repo',
+        command='attic',
+    )
+
+
+def test_create_archive_with_glob_should_call_attic_with_expanded_directories():
+    insert_subprocess_mock(('attic', 'create', 'repo::host-now', 'foo', 'food'))
+    insert_platform_mock()
+    insert_datetime_mock()
+    flexmock(module).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
+
+    module.create_archive(
+        excludes_filename=None,
+        verbosity=None,
+        storage_config={},
+        source_directories='foo*',
+        repository='repo',
+        command='attic',
     )
 
 
@@ -108,6 +263,7 @@ def test_prune_archives_should_call_attic_with_parameters():
         verbosity=None,
         repository='repo',
         retention_config=retention_config,
+        command='attic',
     )
 
 
@@ -122,6 +278,7 @@ def test_prune_archives_with_verbosity_some_should_call_attic_with_stats_paramet
         repository='repo',
         verbosity=VERBOSITY_SOME,
         retention_config=retention_config,
+        command='attic',
     )
 
 
@@ -136,6 +293,7 @@ def test_prune_archives_with_verbosity_lots_should_call_attic_with_verbose_param
         repository='repo',
         verbosity=VERBOSITY_LOTS,
         retention_config=retention_config,
+        command='attic',
     )
 
 
@@ -175,14 +333,28 @@ def test_make_check_flags_with_default_checks_returns_no_flags():
     assert flags == ()
 
 
+def test_make_check_flags_with_checks_and_last_returns_flags_including_last():
+    flags = module._make_check_flags(('foo', 'bar'), check_last=3)
+
+    assert flags == ('--foo-only', '--bar-only', '--last', 3)
+
+
+def test_make_check_flags_with_last_returns_last_flag():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS, check_last=3)
+
+    assert flags == ('--last', 3)
+
+
 def test_check_archives_should_call_attic_with_parameters():
-    consistency_config = flexmock()
-    flexmock(module).should_receive('_parse_checks').and_return(flexmock())
-    flexmock(module).should_receive('_make_check_flags').and_return(())
+    checks = flexmock()
+    check_last = flexmock()
+    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
     stdout = flexmock()
     insert_subprocess_mock(
         ('attic', 'check', 'repo'),
-        stdout=stdout,
+        stdout=stdout, stderr=STDOUT,
     )
     insert_platform_mock()
     insert_datetime_mock()
@@ -193,16 +365,17 @@ def test_check_archives_should_call_attic_with_parameters():
         verbosity=None,
         repository='repo',
         consistency_config=consistency_config,
+        command='attic',
     )
 
 
 def test_check_archives_with_verbosity_some_should_call_attic_with_verbose_parameter():
-    consistency_config = flexmock()
+    consistency_config = flexmock().should_receive('get').and_return(None).mock
     flexmock(module).should_receive('_parse_checks').and_return(flexmock())
     flexmock(module).should_receive('_make_check_flags').and_return(())
     insert_subprocess_mock(
         ('attic', 'check', 'repo', '--verbose'),
-        stdout=None,
+        stdout=None, stderr=STDOUT,
     )
     insert_platform_mock()
     insert_datetime_mock()
@@ -211,16 +384,17 @@ def test_check_archives_with_verbosity_some_should_call_attic_with_verbose_param
         verbosity=VERBOSITY_SOME,
         repository='repo',
         consistency_config=consistency_config,
+        command='attic',
     )
 
 
 def test_check_archives_with_verbosity_lots_should_call_attic_with_verbose_parameter():
-    consistency_config = flexmock()
+    consistency_config = flexmock().should_receive('get').and_return(None).mock
     flexmock(module).should_receive('_parse_checks').and_return(flexmock())
     flexmock(module).should_receive('_make_check_flags').and_return(())
     insert_subprocess_mock(
         ('attic', 'check', 'repo', '--verbose'),
-        stdout=None,
+        stdout=None, stderr=STDOUT,
     )
     insert_platform_mock()
     insert_datetime_mock()
@@ -229,11 +403,12 @@ def test_check_archives_with_verbosity_lots_should_call_attic_with_verbose_param
         verbosity=VERBOSITY_LOTS,
         repository='repo',
         consistency_config=consistency_config,
+        command='attic',
     )
 
 
 def test_check_archives_without_any_checks_should_bail():
-    consistency_config = flexmock()
+    consistency_config = flexmock().should_receive('get').and_return(None).mock
     flexmock(module).should_receive('_parse_checks').and_return(())
     insert_subprocess_never()
 
@@ -241,4 +416,5 @@ def test_check_archives_without_any_checks_should_bail():
         verbosity=None,
         repository='repo',
         consistency_config=consistency_config,
+        command='attic',
     )

borgmatic/tests/unit/test_config.py

@@ -1,9 +1,9 @@
 from collections import OrderedDict
 
 from flexmock import flexmock
-from nose.tools import assert_raises
+import pytest
 
-from atticmatic import config as module
+from borgmatic import config as module
 
 
 def test_option_should_create_config_option():
@@ -61,7 +61,7 @@ def test_validate_configuration_format_with_missing_required_section_should_rais
         ),
     )
 
-    with assert_raises(ValueError):
+    with pytest.raises(ValueError):
         module.validate_configuration_format(parser, config_format)
 
 
@@ -96,7 +96,7 @@ def test_validate_configuration_format_with_unknown_section_should_raise():
         module.Section_format('section', options=()),
     )
 
-    with assert_raises(ValueError):
+    with pytest.raises(ValueError):
         module.validate_configuration_format(parser, config_format)
 
 
@@ -114,7 +114,7 @@ def test_validate_configuration_format_with_missing_required_option_should_raise
         ),
     )
 
-    with assert_raises(ValueError):
+    with pytest.raises(ValueError):
         module.validate_configuration_format(parser, config_format)
 
 
@@ -146,7 +146,7 @@ def test_validate_configuration_format_with_extra_option_should_raise():
         ),
     )
 
-    with assert_raises(ValueError):
+    with pytest.raises(ValueError):
         module.validate_configuration_format(parser, config_format)
 
 
@@ -154,6 +154,7 @@ def test_parse_section_options_should_return_section_options():
     parser = flexmock()
     parser.should_receive('get').with_args('section', 'foo').and_return('value')
     parser.should_receive('getint').with_args('section', 'bar').and_return(1)
+    parser.should_receive('getboolean').never()
     parser.should_receive('has_option').with_args('section', 'foo').and_return(True)
     parser.should_receive('has_option').with_args('section', 'bar').and_return(True)
 
@@ -179,6 +180,7 @@ def test_parse_section_options_for_missing_section_should_return_empty_dict():
     parser = flexmock()
     parser.should_receive('get').never()
     parser.should_receive('getint').never()
+    parser.should_receive('getboolean').never()
     parser.should_receive('has_option').with_args('section', 'foo').and_return(False)
     parser.should_receive('has_option').with_args('section', 'bar').and_return(False)
 
@@ -197,25 +199,34 @@ def test_parse_section_options_for_missing_section_should_return_empty_dict():
 
 def insert_mock_parser():
     parser = flexmock()
-    parser.should_receive('read')
-    module.ConfigParser = lambda: parser
+    parser.should_receive('read').and_return([flexmock()])
+    module.RawConfigParser = lambda: parser
 
     return parser
 
 
 def test_parse_configuration_should_return_section_configs():
     parser = insert_mock_parser()
+    config_format = (flexmock(name='items'), flexmock(name='things'))
     mock_module = flexmock(module)
     mock_module.should_receive('validate_configuration_format').with_args(
-        parser, module.CONFIG_FORMAT,
+        parser, config_format,
     ).once()
-    mock_section_configs = (flexmock(),) * len(module.CONFIG_FORMAT)
+    mock_section_configs = (flexmock(), flexmock())
 
-    for section_format, section_config in zip(module.CONFIG_FORMAT, mock_section_configs):
+    for section_format, section_config in zip(config_format, mock_section_configs):
         mock_module.should_receive('parse_section_options').with_args(
             parser, section_format,
         ).and_return(section_config).once()
 
-    parsed_config = module.parse_configuration('filename')
+    parsed_config = module.parse_configuration('filename', config_format)
 
-    assert parsed_config == module.Parsed_config(*mock_section_configs)
+    assert parsed_config == type(parsed_config)(*mock_section_configs)
+
+
+def test_parse_configuration_with_file_open_error_should_raise():
+    parser = insert_mock_parser()
+    parser.should_receive('read').and_return([])
+
+    with pytest.raises(ValueError):
+        module.parse_configuration('filename', config_format=flexmock())

sample/atticmatic.cron

@@ -1,3 +0,0 @@
-# You can drop this file into /etc/cron.d/ to run atticmatic nightly.
-
-0 3 * * * root PATH=$PATH:/usr/local/bin /usr/local/bin/atticmatic

sample/borgmatic.cron

@@ -0,0 +1,3 @@
+# You can drop this file into /etc/cron.d/ to run borgmatic nightly.
+
+0 3 * * * root PATH=$PATH:/usr/local/bin /usr/local/bin/borgmatic

sample/config

@@ -1,14 +1,30 @@
 [location]
 # Space-separated list of source directories to backup.
-source_directories: /home /etc
+# Globs are expanded.
+source_directories: /home /etc /var/log/syslog*
 
-# Path to local or remote Attic repository.
-repository: user@backupserver:sourcehostname.attic
+# For Borg only, you can specify to stay in same file system (do not cross
+# mount points).
+#one_file_system: True
+
+# Path to local or remote repository.
+repository: user@backupserver:sourcehostname.borg
+
+[storage]
+# Passphrase to unlock the encryption key with. Only use on repositories that
+# were initialized with passphrase/repokey encryption.
+#encryption_passphrase: foo
+# Type of compression to use when creating archives. See
+# https://borgbackup.readthedocs.org/en/stable/usage.html#borg-create
+# for details. Defaults to no compression.
+#compression: lz4
+# Umask to be used for borg create.
+#umask: 0740
 
 [retention]
 # Retention policy for how many backups to keep in each category. See
-# https://attic-backup.org/usage.html#attic-prune for details.
-#keep_within: 3h
+# https://borgbackup.readthedocs.org/en/stable/usage.html#borg-prune for details.
+#keep_within: 3H
 #keep_hourly: 24
 keep_daily: 7
 keep_weekly: 4
@@ -19,5 +35,8 @@ keep_yearly: 1
 [consistency]
 # Space-separated list of consistency checks to run: "repository", "archives",
 # or both. Defaults to both. Set to "disabled" to disable all consistency
-# checks. See https://attic-backup.org/usage.html#attic-check for details.
+# checks. See https://borgbackup.readthedocs.org/en/stable/usage.html#borg-check
+# for details.
 checks: repository archives
+# Restrict the number of checked archives to the last n.
+#check_last: 3

setup.cfg

@@ -1,2 +1,2 @@
-[nosetests]
-detailed-errors=1
+[metadata]
+description-file=README.md

setup.py

@@ -1,15 +1,37 @@
 from setuptools import setup, find_packages
 
+
+VERSION = '1.0.0'
+
+
 setup(
-    name='atticmatic',
-    version='0.0.7',
-    description='A wrapper script for Attic backup software that creates and prunes backups',
+    name='borgmatic',
+    version=VERSION,
+    description='A wrapper script for Borg backup software that creates and prunes backups',
     author='Dan Helfman',
     author_email='witten@torsion.org',
+    url='https://torsion.org/borgmatic',
+    download_url='https://torsion.org/hg/borgmatic/archive/%s.tar.gz' % VERSION,
+    classifiers=(
+        'Development Status :: 5 - Production/Stable',
+        'Environment :: Console',
+        'Intended Audience :: System Administrators',
+        'License :: OSI Approved :: GNU General Public License v3 (GPLv3)',
+        'Programming Language :: Python',
+        'Topic :: Security :: Cryptography',
+        'Topic :: System :: Archiving :: Backup',
+    ),
     packages=find_packages(),
-    entry_points={'console_scripts': ['atticmatic = atticmatic.command:main']},
+    entry_points={
+        'console_scripts': [
+            'borgmatic = borgmatic.command:main',
+        ]
+    },
+    obsoletes=(
+        'atticmatic',
+    ),
     tests_require=(
         'flexmock',
-        'nose',
+        'pytest',
     )
 )

test_requirements.txt

@@ -1,2 +1,2 @@
-flexmock==0.9.7
-nose==1.3.4
+flexmock==0.10.2
+pytest==2.9.1

tox.ini

@@ -5,4 +5,4 @@ skipsdist=True
 [testenv]
 usedevelop=True
 deps=-rtest_requirements.txt
-commands = nosetests []
+commands = py.test []