#32: Fix for upgrade-borgmatic-config converting check_last option as a string instead of an integer.

fixed typos in readme

.gitignore

@@ -0,0 +1,6 @@
+*.egg-info
+*.pyc
+*.swp
+.tox
+build
+dist

.hgignore

@@ -2,4 +2,8 @@ syntax: glob
 *.egg-info
 *.pyc
 *.swp
+.cache
+.coverage
 .tox
+build
+dist

.hgtags

@@ -15,3 +15,19 @@ ac5dfa01e9d14d09845f5e94c2c679e21c5eb2f9 0.1.1
 ac5dfa01e9d14d09845f5e94c2c679e21c5eb2f9 0.1.1
 7b6c87dca7ea312b2257ac1b46857b3f8c56b39c 0.1.1
 83067f995dd391e38544a7722dc3b254b59c5521 0.1.2
+acc7fb61566fe8028c179f43ecc735c851220b06 0.1.3
+6dda59c12de88f060eb7244e6d330173985a9639 0.1.4
+6dda59c12de88f060eb7244e6d330173985a9639 0.1.4
+e58246fc92bb22c2b2fd8b86a1227de69d2d0315 0.1.4
+0afff209b902698c2266986129d6dc9f5f913101 0.1.5
+4c63f3d90ec2bf6af1714a3acec84654a7c9edf3 0.1.6
+5a458ebef804be14e30d7375e3e9fbc26aedb80d 0.1.7
+977f19c2f6a515be6c5ef69cf17b0e0989532209 github/yaml_config_files
+0000000000000000000000000000000000000000 github/yaml_config_files
+28434dd0440cc8da44c2f3e9bd7e9402a59c3b40 github/master
+dbc96d3f83bd5570b6826537616d4160b3374836 0.1.8
+0e1fbee9358de4f062fa9539e1355db83db70caa 1.0.0
+de2d7721cdec93a52d20222a9ddd579ed93c1017 1.0.1
+9603d13910b32d57a887765cab694ac5d0acc1f4 1.0.2
+32c6341dda9fad77a3982641bce8a3a45821842e 1.0.3
+5a003056a8ff4709c5bd4d6d33354199423f8a1d 1.1.0

AUTHORS

@@ -2,3 +2,4 @@ Dan Helfman <witten@torsion.org>: Main developer
 
 Alexander Görtz: Python 3 compatibility
 Henning Schroeder: Copy editing
+Robin `ypid` Schneider: Support additional options of Borg

MANIFEST.in

@@ -0,0 +1 @@
+include borgmatic/config/schema.yaml

NEWS

@@ -1,3 +1,69 @@
+1.1.1
+
+ * #32: Fix for upgrade-borgmatic-config converting check_last option as a string instead of an
+   integer.
+ * Fix for upgrade-borgmatic-config erroring when consistency checks option is not present.
+
+1.1.0
+
+ * Switched config file format to YAML. Run upgrade-borgmatic-config to upgrade.
+ * Added generate-borgmatic-config command for initial config creation.
+ * Dropped Python 2 support. Now Python 3 only.
+ * #18: Fix for README mention of sample files not included in package.
+ * #22: Sample files for triggering borgmatic from a systemd timer.
+ * Support for backing up to multiple repositories.
+ * To free up space, now pruning backups prior to creating a new backup.
+ * Enabled test coverage output during tox runs.
+ * Added logo.
+
+1.0.3
+
+ * #21: Fix for verbosity flag not actually causing verbose output.
+
+1.0.2
+
+ * #20: Fix for traceback when remote_path option is missing.
+
+1.0.1
+
+ * #19: Support for Borg's --remote-path option to use an alternate Borg
+   executable. See sample/config.
+
+1.0.0
+
+ * Attic is no longer supported, as there hasn't been any recent development on
+   it. Dropping Attic support will allow faster iteration on Borg-specific
+   features. If you're still using Attic, this is a good time to switch to Borg!
+ * Project renamed from atticmatic to borgmatic. See the borgmatic README for
+   information on upgrading.
+
+0.1.8
+
+ * Fix for handling of spaces in source_directories which resulted in backup up everything.
+ * Fix for broken links to Borg documentation.
+ * At verbosity zero, suppressing Borg check stderr spew about "Checking segments".
+ * Support for Borg --one-file-system.
+ * Support for Borg create --umask.
+ * Support for file globs in source_directories.
+
+0.1.7
+
+ * #11: Fixed parsing of punctuation in configuration file.
+ * Better error message when configuration file is missing.
+
+0.1.6
+
+ * #9: New configuration option for the encryption passphrase.
+ * #10: Support for Borg's new archive compression feature.
+
+0.1.5
+
+ * Changes to support release on PyPI. Now pip installable by name!
+
+0.1.4
+
+ * Adding test that setup.py version matches release version.
+
 0.1.3
 
  * #1: Add support for "borg check --last N" to Borg backend.

README.md

@@ -1,10 +1,11 @@
-title: Atticmatic
+title: borgmatic
+
+<img src="static/borgmatic.svg" alt="borgmatic logo" style="width: 8em; float: right; padding-left: 1em;" />
 
 ## Overview
 
-atticmatic is a simple Python wrapper script for the
-[Attic](https://attic-backup.org/) and
-[Borg](https://borgbackup.github.io/borgbackup/) backup software that
+borgmatic (formerly atticmatic) is a simple Python wrapper script for the
+[Borg](https://borgbackup.readthedocs.org/en/stable/) backup software that
 initiates a backup, prunes any old backups according to a retention policy,
 and validates backups for consistency. The script supports specifying your
 settings in a declarative configuration file rather than having to put them
@@ -12,75 +13,142 @@ all on the command-line, and handles common errors.
 
 Here's an example config file:
 
-    [location]
-    # Space-separated list of source directories to backup.
-    source_directories: /home /etc
+```yaml
+location:
+    # List of source directories to backup. Globs are expanded.
+    source_directories:
+        - /home
+        - /etc
+        - /var/log/syslog*
 
-    # Path to local or remote backup repository.
-    repository: user@backupserver:sourcehostname.attic
+    # Paths to local or remote repositories.
+    repositories:
+        - user@backupserver:sourcehostname.borg
 
-    [retention]
+    # Any paths matching these patterns are excluded from backups.
+    exclude_patterns:
+        - /home/*/.cache
+
+retention:
     # Retention policy for how many backups to keep in each category.
     keep_daily: 7
     keep_weekly: 4
     keep_monthly: 6
 
-    [consistency]
-    # Consistency checks to run, or "disabled" to prevent checks.
-    checks: repository archives
+consistency:
+    # List of consistency checks to run: "repository", "archives", or both.
+    checks:
+        - repository
+        - archives
+```
 
-Additionally, exclude patterns can be specified in a separate excludes config
-file, one pattern per line.
-
-atticmatic is hosted at <https://torsion.org/atticmatic> with [source code
-available](https://torsion.org/hg/atticmatic). It's also mirrored on
-[GitHub](https://github.com/witten/atticmatic) and
-[BitBucket](https://bitbucket.org/dhelfman/atticmatic) for convenience.
+borgmatic is hosted at <https://torsion.org/borgmatic> with [source code
+available](https://torsion.org/hg/borgmatic). It's also mirrored on
+[GitHub](https://github.com/witten/borgmatic) and
+[BitBucket](https://bitbucket.org/dhelfman/borgmatic) for convenience.
 
 
-## Setup
+## Installation
 
-To get up and running, follow the [Attic Quick
-Start](https://attic-backup.org/quickstart.html) or the [Borg Quick
-Start](https://borgbackup.github.io/borgbackup/quickstart.html) to create a
-repository on a local or remote host. Note that if you plan to run atticmatic
-on a schedule with cron, and you encrypt your attic repository with a
-passphrase instead of a key file, you'll need to set the `ATTIC_PASSPHRASE`
-environment variable. See the repository encryption section of the Quick Start
-for more info.
+To get up and running, follow the [Borg Quick
+Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) to create
+a repository on a local or remote host. Note that if you plan to run
+borgmatic on a schedule with cron, and you encrypt your Borg repository with
+a passphrase instead of a key file, you'll need to set the borgmatic
+`encryption_passphrase` configuration variable. See the repository encryption
+section of the Quick Start for more info.
 
 If the repository is on a remote host, make sure that your local root user has
 key-based ssh access to the desired user account on the remote host.
 
-To install atticmatic, run the following command to download and install it:
+To install borgmatic, run the following command to download and install it:
 
-    sudo pip install --upgrade hg+https://torsion.org/hg/atticmatic
+    sudo pip install --upgrade borgmatic
 
-If you are using Attic, copy the following configuration files:
+Make sure you're using Python 3, as borgmatic does not support Python 2. (You
+may have to use "pip3" or similar instead of "pip".)
 
-    sudo cp sample/atticmatic.cron /etc/cron.d/atticmatic
-    sudo mkdir /etc/atticmatic/
-    sudo cp sample/config sample/excludes /etc/atticmatic/
+Then, generate a sample configuration file:
 
-If you are using Borg, copy the files like this instead:
+    sudo generate-borgmatic-config
 
-    sudo cp sample/borgmatic.cron /etc/cron.d/borgmatic
-    sudo mkdir /etc/borgmatic/
-    sudo cp sample/config sample/excludes /etc/borgmatic/
+This generates a sample configuration file at /etc/borgmatic/config.yaml (by
+default). You should edit the file to suit your needs, as the values are just
+representative. All fields are optional except where indicated, so feel free
+to remove anything you don't need.
 
-Lastly, modify the /etc files with your desired configuration.
+
+## Upgrading
+
+In general, all you should need to do to upgrade borgmatic is run the
+following:
+
+    sudo pip install --upgrade borgmatic
+
+(You may have to use "pip3" or similar instead of "pip", so Python 3 gets
+used.)
+
+However, see below about special cases.
+
+
+### Upgrading from borgmatic 1.0.x
+
+borgmatic changed its configuration file format in version 1.1.0 from
+INI-style to YAML. This better supports validation, and has a more natural way
+to express lists of values. To upgrade your existing configuration, first
+upgrade to the new version of borgmatic:
+
+As of version 1.1.0, borgmatic no longer supports Python 2. If you were
+already running borgmatic with Python 3, then you can simply upgrade borgmatic
+in-place:
+
+    sudo pip install --upgrade borgmatic
+
+But if you were running borgmatic with Python 2, uninstall and reinstall instead:
+
+    sudo pip uninstall borgmatic
+    sudo pip3 install borgmatic
+
+The pip binary names for different versions of Python can differ, so the above
+commands may need some tweaking to work on your machine.
+
+
+Once borgmatic is upgraded, run:
+
+    sudo upgrade-borgmatic-config
+
+That will generate a new YAML configuration file at /etc/borgmatic/config.yaml
+(by default) using the values from both your existing configuration and
+excludes files. The new version of borgmatic will consume the YAML
+configuration file instead of the old one.
+
+
+### Upgrading from atticmatic
+
+You can ignore this section if you're not an atticmatic user (the former name
+of borgmatic).
+
+borgmatic only supports Borg now and no longer supports Attic. So if you're
+an Attic user, consider switching to Borg. See the [Borg upgrade
+command](https://borgbackup.readthedocs.io/en/stable/usage.html#borg-upgrade)
+for more information. Then, follow the instructions above about setting up
+your borgmatic configuration files.
+
+If you were already using Borg with atticmatic, then you can easily upgrade
+from atticmatic to borgmatic. Simply run the following commands:
+
+    sudo pip uninstall atticmatic
+    sudo pip install borgmatic
+
+That's it! borgmatic will continue using your /etc/borgmatic configuration
+files.
 
 
 ## Usage
 
-You can run atticmatic and start a backup simply by invoking it without
+You can run borgmatic and start a backup simply by invoking it without
 arguments:
 
-    atticmatic
-
-Or, if you're using Borg, use this command instead to make use of the Borg
-backend:
-
     borgmatic
 
 This will also prune any old backups as per the configured retention policy,
@@ -90,15 +158,48 @@ By default, the backup will proceed silently except in the case of errors. But
 if you'd like to to get additional information about the progress of the
 backup as it proceeds, use the verbosity option:
 
-    atticmattic --verbosity 1
+    borgmatic --verbosity 1
 
 Or, for even more progress spew:
 
-    atticmattic --verbosity 2
+    borgmatic --verbosity 2
 
 If you'd like to see the available command-line arguments, view the help:
 
-    atticmattic --help
+    borgmatic --help
+
+
+## Autopilot
+
+If you want to run borgmatic automatically, say once a day, the you can
+configure a job runner to invoke it periodically.
+
+### cron
+
+If you're using cron, download the [sample cron
+file](https://torsion.org/hg/borgmatic/raw-file/tip/sample/cron/borgmatic).
+Then, from the directory where you downloaded it:
+
+    sudo mv borgmatic /etc/cron.d/borgmatic
+    sudo chmod +x /etc/cron.d/borgmatic
+
+You can modify the cron file if you'd like to run borgmatic more or less frequently.
+
+### systemd
+
+If you're using systemd instead of cron to run jobs, download the [sample
+systemd service
+file](https://torsion.org/hg/borgmatic/raw-file/tip/sample/systemd/borgmatic.service)
+and the [sample systemd timer
+file](https://torsion.org/hg/borgmatic/raw-file/tip/sample/systemd/borgmatic.timer).
+Then, from the directory where you downloaded them:
+
+    sudo mv borgmatic.service borgmatic.timer /etc/systemd/system/
+    sudo systemctl enable borgmatic.timer
+    sudo systemctl start borgmatic.timer
+
+Feel free to modify the timer file based on how frequently you'd like
+borgmatic to run.
 
 
 ## Running tests
@@ -116,12 +217,12 @@ Then, to actually run tests, run:
 
 ### Broken pipe with remote repository
 
-When running atticmatic on a large remote repository, you may receive errors
-like the following, particularly while "attic check" is valiating backups for
+When running borgmatic on a large remote repository, you may receive errors
+like the following, particularly while "borg check" is validating backups for
 consistency:
 
     Write failed: Broken pipe
-    attic: Error: Connection closed by remote host
+    borg: Error: Connection closed by remote host
 
 This error can be caused by an ssh timeout, which you can rectify by adding
 the following to the ~/.ssh/config file on the client:
@@ -133,9 +234,22 @@ This should make the client keep the connection alive while validating
 backups.
 
 
+### libyaml compilation errors
+
+borgmatic depends on a Python YAML library (ruamel.yaml) that will optionally
+use a C YAML library (libyaml) if present. But if it's not installed, then
+when installing or upgrading borgmatic, you may see errors about compiling the
+YAML library. If so, not to worry. borgmatic should install and function
+correctly even without the C YAML library. And borgmatic won't be any faster
+with the C library present, so you don't need to go out of your way to install
+it.
+
+
 ## Issues and feedback
 
-Got an issue or an idea for a feature enhancement? Check out the [atticmatic
-issue tracker](https://tree.taiga.io/project/witten-atticmatic/issues).
+Got an issue or an idea for a feature enhancement? Check out the [borgmatic
+issue tracker](https://tree.taiga.io/project/witten-borgmatic/issues?page=1&status=399951,399952,399955). In
+order to create a new issue or comment on an issue, you'll need to [login
+first](https://tree.taiga.io/login).
 
 Other questions or comments? Contact <mailto:witten@torsion.org>.

atticmatic/backends/attic.py

@@ -1,12 +0,0 @@
-from functools import partial
-
-from atticmatic.backends import shared
-
-# An atticmatic backend that supports Attic for actually handling backups.
-
-COMMAND = 'attic'
-CONFIG_FORMAT = shared.CONFIG_FORMAT
-
-create_archive = partial(shared.create_archive, command=COMMAND)
-prune_archives = partial(shared.prune_archives, command=COMMAND)
-check_archives = partial(shared.check_archives, command=COMMAND)

atticmatic/backends/borg.py

@@ -1,24 +0,0 @@
-from functools import partial
-
-from atticmatic.config import Section_format, option
-from atticmatic.backends import shared
-
-# An atticmatic backend that supports Borg for actually handling backups.
-
-COMMAND = 'borg'
-CONFIG_FORMAT = (
-    shared.CONFIG_FORMAT[0],  # location
-    shared.CONFIG_FORMAT[1],  # retention
-    Section_format(
-        'consistency',
-        (
-            option('checks', required=False),
-            option('check_last', required=False),
-        ),
-    )
-)
-
-
-create_archive = partial(shared.create_archive, command=COMMAND)
-prune_archives = partial(shared.prune_archives, command=COMMAND)
-check_archives = partial(shared.check_archives, command=COMMAND)

atticmatic/command.py

@@ -1,72 +0,0 @@
-from __future__ import print_function
-from argparse import ArgumentParser
-from importlib import import_module
-import os
-from subprocess import CalledProcessError
-import sys
-
-from atticmatic.config import parse_configuration
-
-
-DEFAULT_CONFIG_FILENAME_PATTERN = '/etc/{}/config'
-DEFAULT_EXCLUDES_FILENAME_PATTERN = '/etc/{}/excludes'
-
-
-def parse_arguments(command_name, *arguments):
-    '''
-    Given the name of the command with which this script was invoked and command-line arguments,
-    parse the arguments and return them as an ArgumentParser instance. Use the command name to
-    determine the default configuration and excludes paths.
-    '''
-    config_filename_default = DEFAULT_CONFIG_FILENAME_PATTERN.format(command_name)
-    excludes_filename_default = DEFAULT_EXCLUDES_FILENAME_PATTERN.format(command_name)
-
-    parser = ArgumentParser()
-    parser.add_argument(
-        '-c', '--config',
-        dest='config_filename',
-        default=config_filename_default,
-        help='Configuration filename',
-    )
-    parser.add_argument(
-        '--excludes',
-        dest='excludes_filename',
-        default=excludes_filename_default if os.path.exists(excludes_filename_default) else None,
-        help='Excludes filename',
-    )
-    parser.add_argument(
-        '-v', '--verbosity',
-        type=int,
-        help='Display verbose progress (1 for some, 2 for lots)',
-    )
-
-    return parser.parse_args(arguments)
-
-
-def load_backend(command_name):
-    '''
-    Given the name of the command with which this script was invoked, return the corresponding
-    backend module responsible for actually dealing with backups.
-    '''
-    backend_name = {
-        'atticmatic': 'attic',
-        'borgmatic': 'borg',
-    }.get(command_name, 'attic')
-
-    return import_module('atticmatic.backends.{}'.format(backend_name))
-
-
-def main():
-    try:
-        command_name = os.path.basename(sys.argv[0])
-        args = parse_arguments(command_name, *sys.argv[1:])
-        backend = load_backend(command_name)
-        config = parse_configuration(args.config_filename, backend.CONFIG_FORMAT)
-        repository = config.location['repository']
-
-        backend.create_archive(args.excludes_filename, args.verbosity, **config.location)
-        backend.prune_archives(args.verbosity, repository, config.retention)
-        backend.check_archives(args.verbosity, repository, config.consistency)
-    except (ValueError, IOError, CalledProcessError) as error:
-        print(error, file=sys.stderr)
-        sys.exit(1)

atticmatic/tests/builtins.py

@@ -1,11 +0,0 @@
-from flexmock import flexmock
-import sys
-
-
-def builtins_mock():
-    try:
-        # Python 2
-        return flexmock(sys.modules['__builtin__'])
-    except KeyError:
-        # Python 3
-        return flexmock(sys.modules['builtins'])

atticmatic/tests/integration/test_command.py

@@ -1,72 +0,0 @@
-import os
-import sys
-
-from flexmock import flexmock
-from nose.tools import assert_raises
-
-from atticmatic import command as module
-
-
-COMMAND_NAME = 'foomatic'
-
-
-def test_parse_arguments_with_no_arguments_uses_defaults():
-    flexmock(os.path).should_receive('exists').and_return(True)
-
-    parser = module.parse_arguments(COMMAND_NAME)
-
-    assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME_PATTERN.format(COMMAND_NAME)
-    assert parser.excludes_filename == module.DEFAULT_EXCLUDES_FILENAME_PATTERN.format(COMMAND_NAME)
-    assert parser.verbosity == None
-
-
-def test_parse_arguments_with_filename_arguments_overrides_defaults():
-    flexmock(os.path).should_receive('exists').and_return(True)
-
-    parser = module.parse_arguments(COMMAND_NAME, '--config', 'myconfig', '--excludes', 'myexcludes')
-
-    assert parser.config_filename == 'myconfig'
-    assert parser.excludes_filename == 'myexcludes'
-    assert parser.verbosity == None
-
-
-def test_parse_arguments_with_missing_default_excludes_file_sets_filename_to_none():
-    flexmock(os.path).should_receive('exists').and_return(False)
-
-    parser = module.parse_arguments(COMMAND_NAME)
-
-    assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME_PATTERN.format(COMMAND_NAME)
-    assert parser.excludes_filename == None
-    assert parser.verbosity == None
-
-
-def test_parse_arguments_with_missing_overridden_excludes_file_retains_filename():
-    flexmock(os.path).should_receive('exists').and_return(False)
-
-    parser = module.parse_arguments(COMMAND_NAME, '--excludes', 'myexcludes')
-
-    assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME_PATTERN.format(COMMAND_NAME)
-    assert parser.excludes_filename == 'myexcludes'
-    assert parser.verbosity == None
-
-
-def test_parse_arguments_with_verbosity_flag_overrides_default():
-    flexmock(os.path).should_receive('exists').and_return(True)
-
-    parser = module.parse_arguments(COMMAND_NAME, '--verbosity', '1')
-
-    assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME_PATTERN.format(COMMAND_NAME)
-    assert parser.excludes_filename == module.DEFAULT_EXCLUDES_FILENAME_PATTERN.format(COMMAND_NAME)
-    assert parser.verbosity == 1
-
-
-def test_parse_arguments_with_invalid_arguments_exits():
-    flexmock(os.path).should_receive('exists').and_return(True)
-    original_stderr = sys.stderr
-    sys.stderr = sys.stdout
-
-    try:
-        with assert_raises(SystemExit):
-            module.parse_arguments(COMMAND_NAME, '--posix-me-harder')
-    finally:
-        sys.stderr = original_stderr

atticmatic/tests/unit/backends/test_shared.py

@@ -1,283 +0,0 @@
-from collections import OrderedDict
-
-from flexmock import flexmock
-
-from atticmatic.backends import shared as module
-from atticmatic.tests.builtins import builtins_mock
-from atticmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
-
-
-def insert_subprocess_mock(check_call_command, **kwargs):
-    subprocess = flexmock()
-    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
-    flexmock(module).subprocess = subprocess
-
-
-def insert_subprocess_never():
-    subprocess = flexmock()
-    subprocess.should_receive('check_call').never()
-    flexmock(module).subprocess = subprocess
-
-
-def insert_platform_mock():
-    flexmock(module.platform).should_receive('node').and_return('host')
-
-
-def insert_datetime_mock():
-    flexmock(module).datetime = flexmock().should_receive('now').and_return(
-        flexmock().should_receive('isoformat').and_return('now').mock
-    ).mock
-
-
-CREATE_COMMAND_WITHOUT_EXCLUDES = ('attic', 'create', 'repo::host-now', 'foo', 'bar')
-CREATE_COMMAND = CREATE_COMMAND_WITHOUT_EXCLUDES + ('--exclude-from', 'excludes')
-
-
-def test_create_archive_should_call_attic_with_parameters():
-    insert_subprocess_mock(CREATE_COMMAND)
-    insert_platform_mock()
-    insert_datetime_mock()
-
-    module.create_archive(
-        excludes_filename='excludes',
-        verbosity=None,
-        source_directories='foo bar',
-        repository='repo',
-        command='attic',
-    )
-
-
-def test_create_archive_with_none_excludes_filename_should_call_attic_without_excludes():
-    insert_subprocess_mock(CREATE_COMMAND_WITHOUT_EXCLUDES)
-    insert_platform_mock()
-    insert_datetime_mock()
-
-    module.create_archive(
-        excludes_filename=None,
-        verbosity=None,
-        source_directories='foo bar',
-        repository='repo',
-        command='attic',
-    )
-
-
-def test_create_archive_with_verbosity_some_should_call_attic_with_stats_parameter():
-    insert_subprocess_mock(CREATE_COMMAND + ('--stats',))
-    insert_platform_mock()
-    insert_datetime_mock()
-
-    module.create_archive(
-        excludes_filename='excludes',
-        verbosity=VERBOSITY_SOME,
-        source_directories='foo bar',
-        repository='repo',
-        command='attic',
-    )
-
-
-def test_create_archive_with_verbosity_lots_should_call_attic_with_verbose_parameter():
-    insert_subprocess_mock(CREATE_COMMAND + ('--verbose', '--stats'))
-    insert_platform_mock()
-    insert_datetime_mock()
-
-    module.create_archive(
-        excludes_filename='excludes',
-        verbosity=VERBOSITY_LOTS,
-        source_directories='foo bar',
-        repository='repo',
-        command='attic',
-    )
-
-
-BASE_PRUNE_FLAGS = (
-    ('--keep-daily', '1'),
-    ('--keep-weekly', '2'),
-    ('--keep-monthly', '3'),
-)
-
-
-def test_make_prune_flags_should_return_flags_from_config():
-    retention_config = OrderedDict(
-        (
-            ('keep_daily', 1),
-            ('keep_weekly', 2),
-            ('keep_monthly', 3),
-        )
-    )
-
-    result = module._make_prune_flags(retention_config)
-
-    assert tuple(result) == BASE_PRUNE_FLAGS
-
-
-PRUNE_COMMAND = (
-    'attic', 'prune', 'repo', '--keep-daily', '1', '--keep-weekly', '2', '--keep-monthly', '3',
-)
-
-
-def test_prune_archives_should_call_attic_with_parameters():
-    retention_config = flexmock()
-    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
-        BASE_PRUNE_FLAGS,
-    )
-    insert_subprocess_mock(PRUNE_COMMAND)
-
-    module.prune_archives(
-        verbosity=None,
-        repository='repo',
-        retention_config=retention_config,
-        command='attic',
-    )
-
-
-def test_prune_archives_with_verbosity_some_should_call_attic_with_stats_parameter():
-    retention_config = flexmock()
-    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
-        BASE_PRUNE_FLAGS,
-    )
-    insert_subprocess_mock(PRUNE_COMMAND + ('--stats',))
-
-    module.prune_archives(
-        repository='repo',
-        verbosity=VERBOSITY_SOME,
-        retention_config=retention_config,
-        command='attic',
-    )
-
-
-def test_prune_archives_with_verbosity_lots_should_call_attic_with_verbose_parameter():
-    retention_config = flexmock()
-    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
-        BASE_PRUNE_FLAGS,
-    )
-    insert_subprocess_mock(PRUNE_COMMAND + ('--verbose', '--stats',))
-
-    module.prune_archives(
-        repository='repo',
-        verbosity=VERBOSITY_LOTS,
-        retention_config=retention_config,
-        command='attic',
-    )
-
-
-def test_parse_checks_returns_them_as_tuple():
-    checks = module._parse_checks({'checks': 'foo disabled bar'})
-
-    assert checks == ('foo', 'bar')
-
-
-def test_parse_checks_with_missing_value_returns_defaults():
-    checks = module._parse_checks({})
-
-    assert checks == module.DEFAULT_CHECKS
-
-
-def test_parse_checks_with_blank_value_returns_defaults():
-    checks = module._parse_checks({'checks': ''})
-
-    assert checks == module.DEFAULT_CHECKS
-
-
-def test_parse_checks_with_disabled_returns_no_checks():
-    checks = module._parse_checks({'checks': 'disabled'})
-
-    assert checks == ()
-
-
-def test_make_check_flags_with_checks_returns_flags():
-    flags = module._make_check_flags(('foo', 'bar'))
-
-    assert flags == ('--foo-only', '--bar-only')
-
-
-def test_make_check_flags_with_default_checks_returns_no_flags():
-    flags = module._make_check_flags(module.DEFAULT_CHECKS)
-
-    assert flags == ()
-
-
-def test_make_check_flags_with_checks_and_last_returns_flags_including_last():
-    flags = module._make_check_flags(('foo', 'bar'), check_last=3)
-
-    assert flags == ('--foo-only', '--bar-only', '--last', 3)
-
-
-def test_make_check_flags_with_last_returns_last_flag():
-    flags = module._make_check_flags(module.DEFAULT_CHECKS, check_last=3)
-
-    assert flags == ('--last', 3)
-
-
-def test_check_archives_should_call_attic_with_parameters():
-    checks = flexmock()
-    check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
-    flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
-    stdout = flexmock()
-    insert_subprocess_mock(
-        ('attic', 'check', 'repo'),
-        stdout=stdout,
-    )
-    insert_platform_mock()
-    insert_datetime_mock()
-    builtins_mock().should_receive('open').and_return(stdout)
-    flexmock(module.os).should_receive('devnull')
-
-    module.check_archives(
-        verbosity=None,
-        repository='repo',
-        consistency_config=consistency_config,
-        command='attic',
-    )
-
-
-def test_check_archives_with_verbosity_some_should_call_attic_with_verbose_parameter():
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
-    flexmock(module).should_receive('_parse_checks').and_return(flexmock())
-    flexmock(module).should_receive('_make_check_flags').and_return(())
-    insert_subprocess_mock(
-        ('attic', 'check', 'repo', '--verbose'),
-        stdout=None,
-    )
-    insert_platform_mock()
-    insert_datetime_mock()
-
-    module.check_archives(
-        verbosity=VERBOSITY_SOME,
-        repository='repo',
-        consistency_config=consistency_config,
-        command='attic',
-    )
-
-
-def test_check_archives_with_verbosity_lots_should_call_attic_with_verbose_parameter():
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
-    flexmock(module).should_receive('_parse_checks').and_return(flexmock())
-    flexmock(module).should_receive('_make_check_flags').and_return(())
-    insert_subprocess_mock(
-        ('attic', 'check', 'repo', '--verbose'),
-        stdout=None,
-    )
-    insert_platform_mock()
-    insert_datetime_mock()
-
-    module.check_archives(
-        verbosity=VERBOSITY_LOTS,
-        repository='repo',
-        consistency_config=consistency_config,
-        command='attic',
-    )
-
-
-def test_check_archives_without_any_checks_should_bail():
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
-    flexmock(module).should_receive('_parse_checks').and_return(())
-    insert_subprocess_never()
-
-    module.check_archives(
-        verbosity=None,
-        repository='repo',
-        consistency_config=consistency_config,
-        command='attic',
-    )

atticmatic/tests/unit/test_command.py

@@ -1,33 +0,0 @@
-from flexmock import flexmock
-
-from atticmatic import command as module
-
-
-def test_load_backend_with_atticmatic_command_should_return_attic_backend():
-    backend = flexmock()
-    (
-        flexmock(module).should_receive('import_module').with_args('atticmatic.backends.attic')
-        .and_return(backend).once()
-    )
-
-    assert module.load_backend('atticmatic') == backend
-
-
-def test_load_backend_with_unknown_command_should_return_attic_backend():
-    backend = flexmock()
-    (
-        flexmock(module).should_receive('import_module').with_args('atticmatic.backends.attic')
-        .and_return(backend).once()
-    )
-
-    assert module.load_backend('unknownmatic') == backend
-
-
-def test_load_backend_with_borgmatic_command_should_return_borg_backend():
-    backend = flexmock()
-    (
-        flexmock(module).should_receive('import_module').with_args('atticmatic.backends.borg')
-        .and_return(backend).once()
-    )
-
-    assert module.load_backend('borgmatic') == backend

borgmatic/borg.py

@@ -1,56 +1,69 @@
 from datetime import datetime
+import glob
+import itertools
 import os
 import platform
+import re
 import subprocess
+import tempfile
 
-from atticmatic.config import Section_format, option
-from atticmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
 
 
-# Common backend functionality shared by Attic and Borg. As the two backup
-# commands diverge, these shared functions will likely need to be replaced
-# with non-shared functions within atticmatic.backends.attic and
-# atticmatic.backends.borg.
+# Integration with Borg for actually handling backups.
 
 
-CONFIG_FORMAT = (
-    Section_format(
-        'location',
-        (
-            option('source_directories'),
-            option('repository'),
-        ),
-    ),
-    Section_format(
-        'retention',
-        (
-            option('keep_within', required=False),
-            option('keep_hourly', int, required=False),
-            option('keep_daily', int, required=False),
-            option('keep_weekly', int, required=False),
-            option('keep_monthly', int, required=False),
-            option('keep_yearly', int, required=False),
-            option('prefix', required=False),
-        ),
-    ),
-    Section_format(
-        'consistency',
-        (
-            option('checks', required=False),
-        ),
+COMMAND = 'borg'
+
+
+def initialize(storage_config, command=COMMAND):
+    passphrase = storage_config.get('encryption_passphrase')
+
+    if passphrase:
+        os.environ['{}_PASSPHRASE'.format(command.upper())] = passphrase
+
+
+def _write_exclude_file(exclude_patterns=None):
+    '''
+    Given a sequence of exclude patterns, write them to a named temporary file and return it. Return
+    None if no patterns are provided.
+    '''
+    if not exclude_patterns:
+        return None
+
+    exclude_file = tempfile.NamedTemporaryFile('w')
+    exclude_file.write('\n'.join(exclude_patterns))
+    exclude_file.flush()
+
+    return exclude_file
+
+
+def create_archive(
+    verbosity, repository, location_config, storage_config, command=COMMAND,
+):
+    '''
+    Given a vebosity flag, a storage config dict, a list of source directories, a local or remote
+    repository path, a list of exclude patterns, and a command to run, create an attic archive.
+    '''
+    sources = tuple(
+        itertools.chain.from_iterable(
+            glob.glob(directory) or [directory]
+            for directory in location_config['source_directories']
+        )
     )
-)
 
-def create_archive(excludes_filename, verbosity, source_directories, repository, command):
-    '''
-    Given an excludes filename (or None), a vebosity flag, a space-separated list of source
-    directories, a local or remote repository path, and a command to run, create an attic archive.
-    '''
-    sources = tuple(source_directories.split(' '))
-    exclude_flags = ('--exclude-from', excludes_filename) if excludes_filename else ()
+    exclude_file = _write_exclude_file(location_config.get('exclude_patterns'))
+    exclude_flags = ('--exclude-from', exclude_file.name) if exclude_file else ()
+    compression = storage_config.get('compression', None)
+    compression_flags = ('--compression', compression) if compression else ()
+    umask = storage_config.get('umask', None)
+    umask_flags = ('--umask', str(umask)) if umask else ()
+    one_file_system_flags = ('--one-file-system',) if location_config.get('one_file_system') else ()
+    remote_path = location_config.get('remote_path')
+    remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
     verbosity_flags = {
-        VERBOSITY_SOME: ('--stats',),
-        VERBOSITY_LOTS: ('--verbose', '--stats'),
+        VERBOSITY_SOME: ('--info', '--stats',),
+        VERBOSITY_LOTS: ('--debug', '--list', '--stats'),
     }.get(verbosity, ())
 
     full_command = (
@@ -60,7 +73,8 @@ def create_archive(excludes_filename, verbosity, source_directories, repository,
             hostname=platform.node(),
             timestamp=datetime.now().isoformat(),
         ),
-    ) + sources + exclude_flags + verbosity_flags
+    ) + sources + exclude_flags + compression_flags + one_file_system_flags + \
+        remote_path_flags + umask_flags + verbosity_flags
 
     subprocess.check_call(full_command)
 
@@ -87,15 +101,16 @@ def _make_prune_flags(retention_config):
     )
 
 
-def prune_archives(verbosity, repository, retention_config, command):
+def prune_archives(verbosity, repository, retention_config, command=COMMAND, remote_path=None):
     '''
     Given a verbosity flag, a local or remote repository path, a retention config dict, and a
     command to run, prune attic archives according the the retention policy specified in that
     configuration.
     '''
+    remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
     verbosity_flags = {
-        VERBOSITY_SOME: ('--stats',),
-        VERBOSITY_LOTS: ('--verbose', '--stats'),
+        VERBOSITY_SOME: ('--info', '--stats',),
+        VERBOSITY_LOTS: ('--debug', '--stats'),
     }.get(verbosity, ())
 
     full_command = (
@@ -105,7 +120,7 @@ def prune_archives(verbosity, repository, retention_config, command):
         element
         for pair in _make_prune_flags(retention_config)
         for element in pair
-    ) + verbosity_flags
+    ) + remote_path_flags + verbosity_flags
 
     subprocess.check_call(full_command)
 
@@ -115,12 +130,11 @@ DEFAULT_CHECKS = ('repository', 'archives')
 
 def _parse_checks(consistency_config):
     '''
-    Given a consistency config with a space-separated "checks" option, transform it to a tuple of
-    named checks to run.
+    Given a consistency config with a "checks" list, transform it to a tuple of named checks to run.
 
     For example, given a retention config of:
 
-        {'checks': 'repository archives'}
+        {'checks': ['repository', 'archives']}
 
     This will be returned as:
 
@@ -129,14 +143,11 @@ def _parse_checks(consistency_config):
     If no "checks" option is present, return the DEFAULT_CHECKS. If the checks value is the string
     "disabled", return an empty tuple, meaning that no checks should be run.
     '''
-    checks = consistency_config.get('checks', '').strip()
-    if not checks:
-        return DEFAULT_CHECKS
+    checks = consistency_config.get('checks', [])
+    if checks == ['disabled']:
+        return ()
 
-    return tuple(
-        check for check in consistency_config['checks'].split(' ')
-        if check.lower() not in ('disabled', '')
-    )
+    return tuple(check for check in checks if check.lower() not in ('disabled', '')) or DEFAULT_CHECKS
 
 
 def _make_check_flags(checks, check_last=None):
@@ -148,7 +159,7 @@ def _make_check_flags(checks, check_last=None):
         ('repository',)
 
     This will be returned as:
-    
+
         ('--repository-only',)
 
     Additionally, if a check_last value is given, a "--last" flag will be added. Note that only
@@ -163,7 +174,7 @@ def _make_check_flags(checks, check_last=None):
     ) + last_flag
 
 
-def check_archives(verbosity, repository, consistency_config, command):
+def check_archives(verbosity, repository, consistency_config, command=COMMAND, remote_path=None):
     '''
     Given a verbosity flag, a local or remote repository path, a consistency config dict, and a
     command to run, check the contained attic archives for consistency.
@@ -175,17 +186,18 @@ def check_archives(verbosity, repository, consistency_config, command):
     if not checks:
         return
 
+    remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
     verbosity_flags = {
-        VERBOSITY_SOME: ('--verbose',),
-        VERBOSITY_LOTS: ('--verbose',),
+        VERBOSITY_SOME: ('--info',),
+        VERBOSITY_LOTS: ('--debug',),
     }.get(verbosity, ())
 
     full_command = (
         command, 'check',
         repository,
-    ) + _make_check_flags(checks, check_last) + verbosity_flags
+    ) + _make_check_flags(checks, check_last) + remote_path_flags + verbosity_flags
 
-    # The check command spews to stdout even without the verbose flag. Suppress it.
+    # The check command spews to stdout/stderr even without the verbose flag. Suppress it.
     stdout = None if verbosity_flags else open(os.devnull, 'w')
 
-    subprocess.check_call(full_command, stdout=stdout)
+    subprocess.check_call(full_command, stdout=stdout, stderr=subprocess.STDOUT)

borgmatic/commands/borgmatic.py

@@ -0,0 +1,66 @@
+from __future__ import print_function
+from argparse import ArgumentParser
+import os
+from subprocess import CalledProcessError
+import sys
+
+from borgmatic import borg
+from borgmatic.config import convert, validate
+
+
+LEGACY_CONFIG_FILENAME = '/etc/borgmatic/config'
+DEFAULT_CONFIG_FILENAME = '/etc/borgmatic/config.yaml'
+DEFAULT_EXCLUDES_FILENAME = '/etc/borgmatic/excludes'
+
+
+def parse_arguments(*arguments):
+    '''
+    Given command-line arguments with which this script was invoked, parse the arguments and return
+    them as an ArgumentParser instance.
+    '''
+    parser = ArgumentParser()
+    parser.add_argument(
+        '-c', '--config',
+        dest='config_filename',
+        default=DEFAULT_CONFIG_FILENAME,
+        help='Configuration filename',
+    )
+    parser.add_argument(
+        '--excludes',
+        dest='excludes_filename',
+        help='Excludes filename, deprecated in favor of exclude_patterns within configuration',
+    )
+    parser.add_argument(
+        '-v', '--verbosity',
+        type=int,
+        help='Display verbose progress (1 for some, 2 for lots)',
+    )
+
+    return parser.parse_args(arguments)
+
+
+def main():  # pragma: no cover
+    try:
+        args = parse_arguments(*sys.argv[1:])
+        convert.guard_configuration_upgraded(LEGACY_CONFIG_FILENAME, args.config_filename)
+        config = validate.parse_configuration(args.config_filename, validate.schema_filename())
+        (location, storage, retention, consistency) = (
+            config.get(section_name, {})
+            for section_name in ('location', 'storage', 'retention', 'consistency')
+        )
+        remote_path = location.get('remote_path')
+
+        borg.initialize(storage)
+
+        for repository in location['repositories']:
+            borg.prune_archives(args.verbosity, repository, retention, remote_path=remote_path)
+            borg.create_archive(
+                args.verbosity,
+                repository,
+                location,
+                storage,
+            )
+            borg.check_archives(args.verbosity, repository, consistency, remote_path=remote_path)
+    except (ValueError, OSError, CalledProcessError) as error:
+        print(error, file=sys.stderr)
+        sys.exit(1)

borgmatic/commands/convert_config.py

@@ -0,0 +1,99 @@
+from __future__ import print_function
+from argparse import ArgumentParser
+import os
+from subprocess import CalledProcessError
+import sys
+import textwrap
+
+from ruamel import yaml
+
+from borgmatic import borg
+from borgmatic.config import convert, generate, legacy, validate
+
+
+DEFAULT_SOURCE_CONFIG_FILENAME = '/etc/borgmatic/config'
+DEFAULT_SOURCE_EXCLUDES_FILENAME = '/etc/borgmatic/excludes'
+DEFAULT_DESTINATION_CONFIG_FILENAME = '/etc/borgmatic/config.yaml'
+
+
+def parse_arguments(*arguments):
+    '''
+    Given command-line arguments with which this script was invoked, parse the arguments and return
+    them as an ArgumentParser instance.
+    '''
+    parser = ArgumentParser(
+        description='''
+            Convert legacy INI-style borgmatic configuration and excludes files to a single YAML
+            configuration file. Note that this replaces any comments from the source files.
+        '''
+    )
+    parser.add_argument(
+        '-s', '--source-config',
+        dest='source_config_filename',
+        default=DEFAULT_SOURCE_CONFIG_FILENAME,
+        help='Source INI-style configuration filename. Default: {}'.format(DEFAULT_SOURCE_CONFIG_FILENAME),
+    )
+    parser.add_argument(
+        '-e', '--source-excludes',
+        dest='source_excludes_filename',
+        default=DEFAULT_SOURCE_EXCLUDES_FILENAME if os.path.exists(DEFAULT_SOURCE_EXCLUDES_FILENAME) else None,
+        help='Excludes filename',
+    )
+    parser.add_argument(
+        '-d', '--destination-config',
+        dest='destination_config_filename',
+        default=DEFAULT_DESTINATION_CONFIG_FILENAME,
+        help='Destination YAML configuration filename. Default: {}'.format(DEFAULT_DESTINATION_CONFIG_FILENAME),
+    )
+
+    return parser.parse_args(arguments)
+
+
+TEXT_WRAP_CHARACTERS = 80
+
+
+def display_result(args):  # pragma: no cover
+    result_lines = textwrap.wrap(
+        'Your borgmatic configuration has been upgraded. Please review the result in {}.'.format(
+            args.destination_config_filename
+        ),
+        TEXT_WRAP_CHARACTERS,
+    )
+    
+    delete_lines = textwrap.wrap(
+        'Once you are satisfied, you can safely delete {}{}.'.format(
+            args.source_config_filename,
+            ' and {}'.format(args.source_excludes_filename) if args.source_excludes_filename else '',
+        ),
+        TEXT_WRAP_CHARACTERS,
+    )
+
+    print('\n'.join(result_lines))
+    print()
+    print('\n'.join(delete_lines))
+
+
+def main():  # pragma: no cover
+    try:
+        args = parse_arguments(*sys.argv[1:])
+        schema = yaml.round_trip_load(open(validate.schema_filename()).read())
+        source_config = legacy.parse_configuration(args.source_config_filename, legacy.CONFIG_FORMAT)
+        source_config_file_mode = os.stat(args.source_config_filename).st_mode
+        source_excludes = (
+            open(args.source_excludes_filename).read().splitlines()
+            if args.source_excludes_filename
+            else []
+        )
+
+        destination_config = convert.convert_legacy_parsed_config(source_config, source_excludes, schema)
+
+        generate.write_configuration(
+            args.destination_config_filename,
+            destination_config,
+            mode=source_config_file_mode,
+        )
+
+        display_result(args)
+    except (ValueError, OSError) as error:
+        print(error, file=sys.stderr)
+        sys.exit(1)

borgmatic/commands/generate_config.py

@@ -0,0 +1,44 @@
+from __future__ import print_function
+from argparse import ArgumentParser
+import os
+from subprocess import CalledProcessError
+import sys
+
+from ruamel import yaml
+
+from borgmatic import borg
+from borgmatic.config import convert, generate, validate
+
+
+DEFAULT_DESTINATION_CONFIG_FILENAME = '/etc/borgmatic/config.yaml'
+
+
+def parse_arguments(*arguments):
+    '''
+    Given command-line arguments with which this script was invoked, parse the arguments and return
+    them as an ArgumentParser instance.
+    '''
+    parser = ArgumentParser(description='Generate a sample borgmatic YAML configuration file.')
+    parser.add_argument(
+        '-d', '--destination',
+        dest='destination_filename',
+        default=DEFAULT_DESTINATION_CONFIG_FILENAME,
+        help='Destination YAML configuration filename. Default: {}'.format(DEFAULT_DESTINATION_CONFIG_FILENAME),
+    )
+
+    return parser.parse_args(arguments)
+
+
+def main():  # pragma: no cover
+    try:
+        args = parse_arguments(*sys.argv[1:])
+
+        generate.generate_sample_configuration(args.destination_filename, validate.schema_filename())
+
+        print('Generated a sample configuration file at {}.'.format(args.destination_filename))
+        print()
+        print('Please edit the file to suit your needs. The values are just representative.')
+        print('All fields are optional except where indicated.')
+    except (ValueError, OSError) as error:
+        print(error, file=sys.stderr)
+        sys.exit(1)

borgmatic/config/convert.py

@@ -0,0 +1,106 @@
+import os
+
+from ruamel import yaml
+
+from borgmatic.config import generate
+
+
+def _convert_section(source_section_config, section_schema):
+    '''
+    Given a legacy Parsed_config instance for a single section, convert it to its corresponding
+    yaml.comments.CommentedMap representation in preparation for actual serialization to YAML.
+
+    Where integer types exist in the given section schema, convert their values to integers.
+    '''
+    destination_section_config = yaml.comments.CommentedMap([
+        (
+            option_name,
+            int(option_value)
+                if section_schema['map'].get(option_name, {}).get('type') == 'int' else option_value
+        )
+        for option_name, option_value in source_section_config.items()
+    ])
+
+    return destination_section_config
+
+
+def convert_legacy_parsed_config(source_config, source_excludes, schema):
+    '''
+    Given a legacy Parsed_config instance loaded from an INI-style config file and a list of exclude
+    patterns, convert them to a corresponding yaml.comments.CommentedMap representation in
+    preparation for serialization to a single YAML config file.
+
+    Additionally, use the given schema as a source of helpful comments to include within the
+    returned CommentedMap.
+    '''
+    destination_config = yaml.comments.CommentedMap([
+        (section_name, _convert_section(section_config, schema['map'][section_name]))
+        for section_name, section_config in source_config._asdict().items()
+    ])
+
+    # Split space-seperated values into actual lists, make "repository" into a list, and merge in
+    # excludes.
+    location = destination_config['location']
+    location['source_directories'] = source_config.location['source_directories'].split(' ')
+    location['repositories'] = [location.pop('repository')]
+    location['exclude_patterns'] = source_excludes
+
+    if source_config.consistency.get('checks'):
+        destination_config['consistency']['checks'] = source_config.consistency['checks'].split(' ')
+
+    # Add comments to each section, and then add comments to the fields in each section.
+    generate.add_comments_to_configuration(destination_config, schema)
+
+    for section_name, section_config in destination_config.items():
+        generate.add_comments_to_configuration(
+            section_config,
+            schema['map'][section_name],
+            indent=generate.INDENT,
+        )
+
+    return destination_config
+
+
+class LegacyConfigurationNotUpgraded(FileNotFoundError):
+    def __init__(self):
+        super(LegacyConfigurationNotUpgraded, self).__init__(
+            '''borgmatic changed its configuration file format in version 1.1.0 from INI-style
+to YAML. This better supports validation, and has a more natural way to express
+lists of values. To upgrade your existing configuration, run:
+
+    sudo upgrade-borgmatic-config
+
+That will generate a new YAML configuration file at /etc/borgmatic/config.yaml
+(by default) using the values from both your existing configuration and excludes
+files. The new version of borgmatic will consume the YAML configuration file
+instead of the old one.'''
+        )
+
+
+def guard_configuration_upgraded(source_config_filename, destination_config_filename):
+    '''
+    If legacy souce configuration exists but destination upgraded config doesn't, raise
+    LegacyConfigurationNotUpgraded.
+
+    The idea is that we want to alert the user about upgrading their config if they haven't already.
+    '''
+    if os.path.exists(source_config_filename) and not os.path.exists(destination_config_filename):
+        raise LegacyConfigurationNotUpgraded()
+
+
+class LegacyExcludesFilenamePresent(FileNotFoundError):
+    def __init__(self):
+        super(LegacyExcludesFilenamePresent, self).__init__(
+            '''borgmatic changed its configuration file format in version 1.1.0 from INI-style
+to YAML. This better supports validation, and has a more natural way to express
+lists of values. The new configuration file incorporates excludes, so you no
+longer need to provide an excludes filename on the command-line with an
+"--excludes" argument.
+
+Please remove the "--excludes" argument and run borgmatic again.'''
+        )
+
+
+def guard_excludes_filename_omitted(excludes_filename):
+    if excludes_filename != None:
+        raise LegacyExcludesFilenamePresent()

borgmatic/config/generate.py

@@ -0,0 +1,93 @@
+from collections import OrderedDict
+import os
+
+from ruamel import yaml
+
+
+INDENT = 4
+
+
+def _insert_newline_before_comment(config, field_name):
+    '''
+    Using some ruamel.yaml black magic, insert a blank line in the config right befor the given
+    field and its comments.
+    '''
+    config.ca.items[field_name][1].insert(
+        0,
+        yaml.tokens.CommentToken('\n', yaml.error.CommentMark(0), None),
+    )
+
+
+def _schema_to_sample_configuration(schema, level=0):
+    '''
+    Given a loaded configuration schema, generate and return sample config for it. Include comments
+    for each section based on the schema "desc" description.
+    '''
+    example = schema.get('example')
+    if example:
+        return example
+
+    config = yaml.comments.CommentedMap([
+        (
+            section_name,
+            _schema_to_sample_configuration(section_schema, level + 1),
+        )
+        for section_name, section_schema in schema['map'].items()
+    ])
+
+    add_comments_to_configuration(config, schema, indent=(level * INDENT))
+
+    return config
+
+
+def write_configuration(config_filename, config, mode=0o600):
+    '''
+    Given a target config filename and a config data structure of nested OrderedDicts, write out the
+    config to file as YAML. Create any containing directories as needed.
+    '''
+    if os.path.exists(config_filename):
+        raise FileExistsError('{} already exists. Aborting.'.format(config_filename))
+
+    try:
+        os.makedirs(os.path.dirname(config_filename), mode=0o700)
+    except (FileExistsError, FileNotFoundError):
+        pass
+
+    with open(config_filename, 'w') as config_file:
+        config_file.write(yaml.round_trip_dump(config, indent=INDENT, block_seq_indent=INDENT))
+
+    os.chmod(config_filename, mode)
+
+
+def add_comments_to_configuration(config, schema, indent=0):
+    '''
+    Using descriptions from a schema as a source, add those descriptions as comments to the given
+    config before each field. This function only adds comments for the top-most config map level.
+    Indent the comment the given number of characters.
+    '''
+    for index, field_name in enumerate(config.keys()):
+        field_schema = schema['map'].get(field_name, {})
+        description = field_schema.get('desc')
+
+        # No description to use? Skip it.
+        if not field_schema or not description:
+            continue
+
+        config.yaml_set_comment_before_after_key(
+            key=field_name,
+            before=description,
+            indent=indent,
+        )
+        if index > 0:
+            _insert_newline_before_comment(config, field_name)
+
+
+def generate_sample_configuration(config_filename, schema_filename):
+    '''
+    Given a target config filename and the path to a schema filename in pykwalify YAML schema
+    format, write out a sample configuration file based on that schema.
+    '''
+    schema = yaml.round_trip_load(open(schema_filename))
+    config = _schema_to_sample_configuration(schema)
+
+    write_configuration(config_filename, config)

borgmatic/config/legacy.py

@@ -1,11 +1,5 @@
 from collections import OrderedDict, namedtuple
-
-try:
-    # Python 2
-    from ConfigParser import ConfigParser
-except ImportError:
-    # Python 3
-    from configparser import ConfigParser
+from configparser import RawConfigParser
 
 
 Section_format = namedtuple('Section_format', ('name', 'options'))
@@ -20,9 +14,49 @@ def option(name, value_type=str, required=True):
     return Config_option(name, value_type, required)
 
 
+CONFIG_FORMAT = (
+    Section_format(
+        'location',
+        (
+            option('source_directories'),
+            option('one_file_system', value_type=bool, required=False),
+            option('remote_path', required=False),
+            option('repository'),
+        ),
+    ),
+    Section_format(
+        'storage',
+        (
+            option('encryption_passphrase', required=False),
+            option('compression', required=False),
+            option('umask', required=False),
+        ),
+    ),
+    Section_format(
+        'retention',
+        (
+            option('keep_within', required=False),
+            option('keep_hourly', int, required=False),
+            option('keep_daily', int, required=False),
+            option('keep_weekly', int, required=False),
+            option('keep_monthly', int, required=False),
+            option('keep_yearly', int, required=False),
+            option('prefix', required=False),
+        ),
+    ),
+    Section_format(
+        'consistency',
+        (
+            option('checks', required=False),
+            option('check_last', required=False),
+        ),
+    )
+)
+
+
 def validate_configuration_format(parser, config_format):
     '''
-    Given an open ConfigParser and an expected config file format, validate that the parsed
+    Given an open RawConfigParser and an expected config file format, validate that the parsed
     configuration file has the expected sections, that any required options are present in those
     sections, and that there aren't any unexpected options.
 
@@ -83,7 +117,7 @@ def validate_configuration_format(parser, config_format):
 
 def parse_section_options(parser, section_format):
     '''
-    Given an open ConfigParser and an expected section format, return the option values from that
+    Given an open RawConfigParser and an expected section format, return the option values from that
     section as a dict mapping from option name to value. Omit those options that are not present in
     the parsed options.
 
@@ -92,6 +126,7 @@ def parse_section_options(parser, section_format):
     type_getter = {
         str: parser.get,
         int: parser.getint,
+        bool: parser.getboolean,
     }
 
     return OrderedDict(
@@ -108,8 +143,9 @@ def parse_configuration(config_filename, config_format):
 
     Raise IOError if the file cannot be read, or ValueError if the format is not as expected.
     '''
-    parser = ConfigParser()
-    parser.read(config_filename)
+    parser = RawConfigParser()
+    if not parser.read(config_filename):
+        raise ValueError('Configuration file cannot be opened: {}'.format(config_filename))
 
     validate_configuration_format(parser, config_format)
 

borgmatic/config/schema.yaml

@@ -0,0 +1,126 @@
+name: Borgmatic configuration file schema
+version: 1
+map:
+    location:
+        desc: |
+            Where to look for files to backup, and where to store those backups. See
+            https://borgbackup.readthedocs.io/en/stable/quickstart.html and
+            https://borgbackup.readthedocs.io/en/stable/usage.html#borg-create for details.
+        required: True
+        map:
+            source_directories:
+                required: True
+                seq:
+                    - type: scalar
+                desc: List of source directories to backup (required). Globs are expanded.
+                example:
+                    - /home
+                    - /etc
+                    - /var/log/syslog*
+            one_file_system:
+                type: bool
+                desc: Stay in same file system (do not cross mount points).
+                example: yes
+            remote_path:
+                type: scalar
+                desc: Alternate Borg remote executable. Defaults to "borg".
+                example: borg1
+            repositories:
+                required: True
+                seq:
+                    - type: scalar
+                desc: |
+                    Paths to local or remote repositories (required). Multiple repositories are
+                    backed up to in sequence.
+                example:
+                    - user@backupserver:sourcehostname.borg
+            exclude_patterns:
+                seq:
+                    - type: scalar
+                desc: |
+                    Any paths matching these patterns are excluded from backups. Globs are expanded.
+                    See https://borgbackup.readthedocs.io/en/stable/usage.html#borg-help-patterns
+                    for details.
+                example:
+                    - '*.pyc'
+                    - /home/*/.cache
+                    - /etc/ssl
+    storage:
+        desc: |
+            Repository storage options. See
+            https://borgbackup.readthedocs.io/en/stable/usage.html#borg-create and
+            https://borgbackup.readthedocs.io/en/stable/usage.html#environment-variables for details.
+        map:
+            encryption_passphrase:
+                type: scalar
+                desc: |
+                    Passphrase to unlock the encryption key with. Only use on repositories that were
+                    initialized with passphrase/repokey encryption. Quote the value if it contains
+                    punctuation, so it parses correctly. And backslash any quote or backslash
+                    literals as well.
+                example: "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
+            compression:
+                type: scalar
+                desc: |
+                    Type of compression to use when creating archives. See
+                    https://borgbackup.readthedocs.org/en/stable/usage.html#borg-create for details.
+                    Defaults to no compression.
+                example: lz4
+            umask:
+                type: scalar
+                desc: Umask to be used for borg create.
+                example: 0077
+    retention:
+        desc: |
+            Retention policy for how many backups to keep in each category. See
+            https://borgbackup.readthedocs.org/en/stable/usage.html#borg-prune for details.
+        map:
+            keep_within:
+                type: scalar
+                desc: Keep all archives within this time interval.
+                example: 3H
+            keep_hourly:
+                type: int
+                desc: Number of hourly archives to keep.
+                example: 24
+            keep_daily:
+                type: int
+                desc: Number of daily archives to keep.
+                example: 7
+            keep_weekly:
+                type: int
+                desc: Number of weekly archives to keep.
+                example: 4
+            keep_monthly:
+                type: int
+                desc: Number of monthly archives to keep.
+                example: 6
+            keep_yearly:
+                type: int
+                desc: Number of yearly archives to keep.
+                example: 1
+            prefix:
+                type: scalar
+                desc: When pruning, only consider archive names starting with this prefix.
+                example: sourcehostname
+    consistency:
+        desc: |
+            Consistency checks to run after backups. See
+            https://borgbackup.readthedocs.org/en/stable/usage.html#borg-check for details.
+        map:
+            checks:
+                seq:
+                    - type: str
+                      enum: ['repository', 'archives', 'disabled']
+                      unique: True
+                desc: |
+                    List of consistency checks to run: "repository", "archives", or both. Defaults
+                    to both. Set to "disabled" to disable all consistency checks. See
+                    https://borgbackup.readthedocs.org/en/stable/usage.html#borg-check for details.
+                example:
+                    - repository
+                    - archives
+            check_last:
+                type: int
+                desc: Restrict the number of checked archives to the last n.
+                example: 3

borgmatic/config/validate.py

@@ -0,0 +1,74 @@
+import logging
+import sys
+import warnings
+
+import pkg_resources
+import pykwalify.core
+import pykwalify.errors
+from ruamel import yaml
+
+
+def schema_filename():
+    '''
+    Path to the installed YAML configuration schema file, used to validate and parse the
+    configuration.
+    '''
+    return pkg_resources.resource_filename('borgmatic', 'config/schema.yaml')
+
+
+class Validation_error(ValueError):
+    '''
+    A collection of error message strings generated when attempting to validate a particular
+    configurartion file.
+    '''
+    def __init__(self, config_filename, error_messages):
+        self.config_filename = config_filename
+        self.error_messages = error_messages
+
+
+def parse_configuration(config_filename, schema_filename):
+    '''
+    Given the path to a config filename in YAML format and the path to a schema filename in
+    pykwalify YAML schema format, return the parsed configuration as a data structure of nested
+    dicts and lists corresponding to the schema. Example return value:
+
+       {'location': {'source_directories': ['/home', '/etc'], 'repository': 'hostname.borg'},
+       'retention': {'keep_daily': 7}, 'consistency': {'checks': ['repository', 'archives']}}
+
+    Raise FileNotFoundError if the file does not exist, PermissionError if the user does not
+    have permissions to read the file, or Validation_error if the config does not match the schema.
+    '''
+    try:
+        config = yaml.round_trip_load(open(config_filename))
+        schema = yaml.round_trip_load(open(schema_filename))
+    except yaml.error.YAMLError as error:
+        raise Validation_error(config_filename, (str(error),))
+
+    # pykwalify gets angry if the example field is not a string. So rather than bend to its will,
+    # simply remove all examples before passing the schema to pykwalify.
+    for section_name, section_schema in schema['map'].items():
+        for field_name, field_schema in section_schema['map'].items():
+            field_schema.pop('example')
+
+    validator = pykwalify.core.Core(source_data=config, schema_data=schema)
+    parsed_result = validator.validate(raise_exception=False)
+
+    if validator.validation_errors:
+        raise Validation_error(config_filename, validator.validation_errors)
+
+    return parsed_result
+
+
+def display_validation_error(validation_error):
+    '''
+    Given a Validation_error, display its error messages to stderr.
+    '''
+    print(
+        'An error occurred while parsing a configuration file at {}:'.format(
+            validation_error.config_filename
+        ),
+        file=sys.stderr,
+    )
+
+    for error in validation_error.error_messages:
+        print(error, file=sys.stderr)

borgmatic/tests/integration/commands/test_borgmatic.py

@@ -0,0 +1,35 @@
+import os
+
+from flexmock import flexmock
+import pytest
+
+from borgmatic.commands import borgmatic as module
+
+
+def test_parse_arguments_with_no_arguments_uses_defaults():
+    parser = module.parse_arguments()
+
+    assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME
+    assert parser.excludes_filename == None
+    assert parser.verbosity is None
+
+
+def test_parse_arguments_with_filename_arguments_overrides_defaults():
+    parser = module.parse_arguments('--config', 'myconfig', '--excludes', 'myexcludes')
+
+    assert parser.config_filename == 'myconfig'
+    assert parser.excludes_filename == 'myexcludes'
+    assert parser.verbosity is None
+
+
+def test_parse_arguments_with_verbosity_flag_overrides_default():
+    parser = module.parse_arguments('--verbosity', '1')
+
+    assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME
+    assert parser.excludes_filename == None
+    assert parser.verbosity == 1
+
+
+def test_parse_arguments_with_invalid_arguments_exits():
+    with pytest.raises(SystemExit):
+        module.parse_arguments('--posix-me-harder')

borgmatic/tests/integration/commands/test_convert_config.py

@@ -0,0 +1,47 @@
+import os
+
+from flexmock import flexmock
+import pytest
+
+from borgmatic.commands import convert_config as module
+
+
+def test_parse_arguments_with_no_arguments_uses_defaults():
+    flexmock(os.path).should_receive('exists').and_return(True)
+
+    parser = module.parse_arguments()
+
+    assert parser.source_config_filename == module.DEFAULT_SOURCE_CONFIG_FILENAME
+    assert parser.source_excludes_filename == module.DEFAULT_SOURCE_EXCLUDES_FILENAME
+    assert parser.destination_config_filename == module.DEFAULT_DESTINATION_CONFIG_FILENAME
+
+
+def test_parse_arguments_with_filename_arguments_overrides_defaults():
+    flexmock(os.path).should_receive('exists').and_return(True)
+
+    parser = module.parse_arguments(
+        '--source-config', 'config',
+        '--source-excludes', 'excludes',
+        '--destination-config', 'config.yaml',
+    )
+
+    assert parser.source_config_filename == 'config'
+    assert parser.source_excludes_filename == 'excludes'
+    assert parser.destination_config_filename == 'config.yaml'
+
+
+def test_parse_arguments_with_missing_default_excludes_file_sets_filename_to_none():
+    flexmock(os.path).should_receive('exists').and_return(False)
+
+    parser = module.parse_arguments()
+
+    assert parser.source_config_filename == module.DEFAULT_SOURCE_CONFIG_FILENAME
+    assert parser.source_excludes_filename is None
+    assert parser.destination_config_filename == module.DEFAULT_DESTINATION_CONFIG_FILENAME
+
+
+def test_parse_arguments_with_invalid_arguments_exits():
+    flexmock(os.path).should_receive('exists').and_return(True)
+
+    with pytest.raises(SystemExit):
+        module.parse_arguments('--posix-me-harder')

borgmatic/tests/integration/commands/test_generate_config.py

@@ -0,0 +1,12 @@
+from borgmatic.commands import generate_config as module
+
+
+def test_parse_arguments_with_no_arguments_uses_defaults():
+    parser = module.parse_arguments()
+
+    assert parser.destination_filename == module.DEFAULT_DESTINATION_CONFIG_FILENAME
+
+def test_parse_arguments_with_filename_argument_overrides_defaults():
+    parser = module.parse_arguments('--destination', 'config.yaml')
+
+    assert parser.destination_filename == 'config.yaml'

borgmatic/tests/integration/config/test_generate.py

@@ -0,0 +1,65 @@
+from io import StringIO
+import os
+import sys
+
+from flexmock import flexmock
+import pytest
+
+from borgmatic.config import generate as module
+
+
+def test_insert_newline_before_comment_does_not_raise():
+    field_name = 'foo'
+    config = module.yaml.comments.CommentedMap([(field_name, 33)])
+    config.yaml_set_comment_before_after_key(key=field_name, before='Comment',)
+
+    module._insert_newline_before_comment(config, field_name)
+
+
+def test_write_configuration_does_not_raise():
+    flexmock(os.path).should_receive('exists').and_return(False)
+    flexmock(os).should_receive('makedirs')
+    builtins = flexmock(sys.modules['builtins'])
+    builtins.should_receive('open').and_return(StringIO())
+    flexmock(os).should_receive('chmod')
+
+    module.write_configuration('config.yaml', {})
+
+
+def test_write_configuration_with_already_existing_file_raises():
+    flexmock(os.path).should_receive('exists').and_return(True)
+
+    with pytest.raises(FileExistsError):
+        module.write_configuration('config.yaml', {})
+
+
+def test_write_configuration_with_already_existing_directory_does_not_raise():
+    flexmock(os.path).should_receive('exists').and_return(False)
+    flexmock(os).should_receive('makedirs').and_raise(FileExistsError)
+    builtins = flexmock(sys.modules['builtins'])
+    builtins.should_receive('open').and_return(StringIO())
+    flexmock(os).should_receive('chmod')
+
+    module.write_configuration('config.yaml', {})
+
+
+def test_add_comments_to_configuration_does_not_raise():
+    # Ensure that it can deal with fields both in the schema and missing from the schema.
+    config = module.yaml.comments.CommentedMap([('foo', 33), ('bar', 44), ('baz', 55)])
+    schema = {
+        'map': {
+            'foo': {'desc': 'Foo'},
+            'bar': {'desc': 'Bar'},
+        }
+    }
+
+    module.add_comments_to_configuration(config, schema)
+
+
+def test_generate_sample_configuration_does_not_raise():
+    builtins = flexmock(sys.modules['builtins'])
+    builtins.should_receive('open').with_args('schema.yaml').and_return('')
+    flexmock(module).should_receive('write_configuration')
+    flexmock(module).should_receive('_schema_to_sample_configuration')
+
+    module.generate_sample_configuration('config.yaml', 'schema.yaml')

borgmatic/tests/integration/config/test_legacy.py

@@ -0,0 +1,24 @@
+from io import StringIO
+
+from collections import OrderedDict
+import string
+
+from borgmatic.config import legacy as module
+
+
+def test_parse_section_options_with_punctuation_should_return_section_options():
+    parser = module.RawConfigParser()
+    parser.readfp(StringIO('[section]\nfoo: {}\n'.format(string.punctuation)))
+
+    section_format = module.Section_format(
+        'section',
+        (module.Config_option('foo', str, required=True),),
+    )
+
+    config = module.parse_section_options(parser, section_format)
+
+    assert config == OrderedDict(
+        (
+            ('foo', string.punctuation),
+        )
+    )

borgmatic/tests/integration/config/test_validate.py

@@ -0,0 +1,123 @@
+import io
+import string
+import sys
+import os
+
+from flexmock import flexmock
+import pytest
+
+from borgmatic.config import validate as module
+
+
+def test_schema_filename_returns_plausable_path():
+    schema_path = module.schema_filename()    
+
+    assert schema_path.endswith('/schema.yaml')
+
+
+def mock_config_and_schema(config_yaml):
+    '''
+    Set up mocks for the config config YAML string and the default schema so that the code under
+    test consumes them when parsing the configuration.
+    '''
+    config_stream = io.StringIO(config_yaml)
+    schema_stream = open(module.schema_filename())
+    builtins = flexmock(sys.modules['builtins'])
+    builtins.should_receive('open').with_args('config.yaml').and_return(config_stream)
+    builtins.should_receive('open').with_args('schema.yaml').and_return(schema_stream)
+
+
+def test_parse_configuration_transforms_file_into_mapping():
+    mock_config_and_schema(
+        '''
+        location:
+            source_directories:
+                - /home
+                - /etc
+
+            repositories:
+                - hostname.borg
+
+        retention:
+            keep_daily: 7
+
+        consistency:
+            checks:
+                - repository
+                - archives
+        '''
+    )
+
+    result = module.parse_configuration('config.yaml', 'schema.yaml')
+
+    assert result == {
+        'location': {'source_directories': ['/home', '/etc'], 'repositories': ['hostname.borg']},
+        'retention': {'keep_daily': 7},
+        'consistency': {'checks': ['repository', 'archives']},
+    }
+
+
+def test_parse_configuration_passes_through_quoted_punctuation():
+    escaped_punctuation = string.punctuation.replace('\\', r'\\').replace('"', r'\"')
+
+    mock_config_and_schema(
+        '''
+        location:
+            source_directories:
+                - /home
+
+            repositories:
+                - "{}.borg"
+        '''.format(escaped_punctuation)
+    )
+
+    result = module.parse_configuration('config.yaml', 'schema.yaml')
+
+    assert result == {
+        'location': {
+            'source_directories': ['/home'],
+            'repositories': ['{}.borg'.format(string.punctuation)],
+        },
+    }
+
+
+def test_parse_configuration_raises_for_missing_config_file():
+    with pytest.raises(FileNotFoundError):
+        module.parse_configuration('config.yaml', 'schema.yaml')
+
+
+def test_parse_configuration_raises_for_missing_schema_file():
+    mock_config_and_schema('')
+    builtins = flexmock(sys.modules['builtins'])
+    builtins.should_receive('open').with_args('schema.yaml').and_raise(FileNotFoundError)
+
+    with pytest.raises(FileNotFoundError):
+        module.parse_configuration('config.yaml', 'schema.yaml')
+
+
+def test_parse_configuration_raises_for_syntax_error():
+    mock_config_and_schema('foo:\nbar')
+
+    with pytest.raises(ValueError):
+        module.parse_configuration('config.yaml', 'schema.yaml')
+
+
+def test_parse_configuration_raises_for_validation_error():
+    mock_config_and_schema(
+        '''
+        location:
+            source_directories: yes
+            repositories:
+                - hostname.borg
+        '''
+    )
+
+    with pytest.raises(module.Validation_error):
+        module.parse_configuration('config.yaml', 'schema.yaml')
+
+
+def test_display_validation_error_does_not_raise():
+    flexmock(sys.modules['builtins']).should_receive('print')
+    error = module.Validation_error('config.yaml', ('oops', 'uh oh'))
+
+    module.display_validation_error(error)

borgmatic/tests/integration/test_version.py

@@ -0,0 +1,8 @@
+import subprocess
+
+
+def test_setup_version_matches_news_version():
+    setup_version = subprocess.check_output(('python', 'setup.py', '--version')).decode('ascii')
+    news_version = open('NEWS').readline()
+
+    assert setup_version == news_version

borgmatic/tests/unit/config/test_convert.py

@@ -0,0 +1,114 @@
+from collections import defaultdict, OrderedDict, namedtuple
+import os
+
+from flexmock import flexmock
+import pytest
+
+from borgmatic.config import convert as module
+
+
+Parsed_config = namedtuple('Parsed_config', ('location', 'storage', 'retention', 'consistency'))
+
+
+def test_convert_section_generates_integer_value_for_integer_type_in_schema():
+    flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
+    source_section_config = OrderedDict([('check_last', '3')])
+    section_schema = {'map': {'check_last': {'type': 'int'}}}
+
+    destination_config = module._convert_section(source_section_config, section_schema)
+
+    assert destination_config == OrderedDict([('check_last', 3)])
+
+
+def test_convert_legacy_parsed_config_transforms_source_config_to_mapping():
+    flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
+    source_config = Parsed_config(
+        location=OrderedDict([('source_directories', '/home'), ('repository', 'hostname.borg')]),
+        storage=OrderedDict([('encryption_passphrase', 'supersecret')]),
+        retention=OrderedDict([('keep_daily', 7)]),
+        consistency=OrderedDict([('checks', 'repository')]),
+    )
+    source_excludes = ['/var']
+    schema = {'map': defaultdict(lambda: {'map': {}})}
+
+    destination_config = module.convert_legacy_parsed_config(source_config, source_excludes, schema)
+
+    assert destination_config == OrderedDict([
+        (
+            'location',
+            OrderedDict([
+                ('source_directories', ['/home']),
+                ('repositories', ['hostname.borg']),
+                ('exclude_patterns', ['/var']),
+            ]),
+        ),
+        ('storage', OrderedDict([('encryption_passphrase', 'supersecret')])),
+        ('retention', OrderedDict([('keep_daily', 7)])),
+        ('consistency', OrderedDict([('checks', ['repository'])])),
+    ])
+
+
+def test_convert_legacy_parsed_config_splits_space_separated_values():
+    flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
+    source_config = Parsed_config(
+        location=OrderedDict([('source_directories', '/home /etc'), ('repository', 'hostname.borg')]),
+        storage=OrderedDict(),
+        retention=OrderedDict(),
+        consistency=OrderedDict([('checks', 'repository archives')]),
+    )
+    source_excludes = ['/var']
+    schema = {'map': defaultdict(lambda: {'map': {}})}
+
+    destination_config = module.convert_legacy_parsed_config(source_config, source_excludes, schema) 
+
+    assert destination_config == OrderedDict([
+        (
+            'location',
+            OrderedDict([
+                ('source_directories', ['/home', '/etc']),
+                ('repositories', ['hostname.borg']),
+                ('exclude_patterns', ['/var']),
+            ]),
+        ),
+        ('storage', OrderedDict()),
+        ('retention', OrderedDict()),
+        ('consistency', OrderedDict([('checks', ['repository', 'archives'])])),
+    ])
+
+
+def test_guard_configuration_upgraded_raises_when_only_source_config_present():
+    flexmock(os.path).should_receive('exists').with_args('config').and_return(True)
+    flexmock(os.path).should_receive('exists').with_args('config.yaml').and_return(False)
+
+    with pytest.raises(module.LegacyConfigurationNotUpgraded):
+        module.guard_configuration_upgraded('config', 'config.yaml')
+
+
+def test_guard_configuration_upgraded_does_not_raise_when_only_destination_config_present():
+    flexmock(os.path).should_receive('exists').with_args('config').and_return(False)
+    flexmock(os.path).should_receive('exists').with_args('config.yaml').and_return(True)
+
+    module.guard_configuration_upgraded('config', 'config.yaml')
+
+
+def test_guard_configuration_upgraded_does_not_raise_when_both_configs_present():
+    flexmock(os.path).should_receive('exists').with_args('config').and_return(True)
+    flexmock(os.path).should_receive('exists').with_args('config.yaml').and_return(True)
+
+    module.guard_configuration_upgraded('config', 'config.yaml')
+
+
+def test_guard_configuration_upgraded_does_not_raise_when_neither_config_present():
+    flexmock(os.path).should_receive('exists').with_args('config').and_return(False)
+    flexmock(os.path).should_receive('exists').with_args('config.yaml').and_return(False)
+
+    module.guard_configuration_upgraded('config', 'config.yaml')
+
+
+def test_guard_excludes_filename_omitted_raises_when_filename_provided():
+    with pytest.raises(module.LegacyExcludesFilenamePresent):
+        module.guard_excludes_filename_omitted(excludes_filename='/etc/borgmatic/excludes')
+
+
+def test_guard_excludes_filename_omitted_does_not_raise_when_filename_not_provided():
+    module.guard_excludes_filename_omitted(excludes_filename=None)

borgmatic/tests/unit/config/test_generate.py

@@ -0,0 +1,49 @@
+from collections import OrderedDict
+
+from flexmock import flexmock
+
+from borgmatic.config import generate as module
+
+
+def test_schema_to_sample_configuration_generates_config_with_examples():
+    flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
+    flexmock(module).should_receive('add_comments_to_configuration')
+    schema = {
+        'map': OrderedDict([
+            (
+                'section1', {
+                    'map': {
+                        'field1': OrderedDict([
+                            ('example', 'Example 1')
+                        ]),
+                    },
+                },
+            ),
+            (
+                'section2', {
+                    'map': OrderedDict([
+                        ('field2', {'example': 'Example 2'}),
+                        ('field3', {'example': 'Example 3'}),
+                    ]),
+                }
+            ),
+        ])
+    }
+
+    config = module._schema_to_sample_configuration(schema)
+
+    assert config == OrderedDict([
+        (
+            'section1',
+            OrderedDict([
+                ('field1', 'Example 1'),
+            ]),
+        ),
+        (
+            'section2',
+            OrderedDict([
+                ('field2', 'Example 2'),
+                ('field3', 'Example 3'),
+            ]),
+        )
+    ])

borgmatic/tests/unit/config/test_legacy.py

@@ -1,9 +1,9 @@
 from collections import OrderedDict
 
 from flexmock import flexmock
-from nose.tools import assert_raises
+import pytest
 
-from atticmatic import config as module
+from borgmatic.config import legacy as module
 
 
 def test_option_should_create_config_option():
@@ -61,7 +61,7 @@ def test_validate_configuration_format_with_missing_required_section_should_rais
         ),
     )
 
-    with assert_raises(ValueError):
+    with pytest.raises(ValueError):
         module.validate_configuration_format(parser, config_format)
 
 
@@ -96,7 +96,7 @@ def test_validate_configuration_format_with_unknown_section_should_raise():
         module.Section_format('section', options=()),
     )
 
-    with assert_raises(ValueError):
+    with pytest.raises(ValueError):
         module.validate_configuration_format(parser, config_format)
 
 
@@ -114,7 +114,7 @@ def test_validate_configuration_format_with_missing_required_option_should_raise
         ),
     )
 
-    with assert_raises(ValueError):
+    with pytest.raises(ValueError):
         module.validate_configuration_format(parser, config_format)
 
 
@@ -146,7 +146,7 @@ def test_validate_configuration_format_with_extra_option_should_raise():
         ),
     )
 
-    with assert_raises(ValueError):
+    with pytest.raises(ValueError):
         module.validate_configuration_format(parser, config_format)
 
 
@@ -154,6 +154,7 @@ def test_parse_section_options_should_return_section_options():
     parser = flexmock()
     parser.should_receive('get').with_args('section', 'foo').and_return('value')
     parser.should_receive('getint').with_args('section', 'bar').and_return(1)
+    parser.should_receive('getboolean').never()
     parser.should_receive('has_option').with_args('section', 'foo').and_return(True)
     parser.should_receive('has_option').with_args('section', 'bar').and_return(True)
 
@@ -179,6 +180,7 @@ def test_parse_section_options_for_missing_section_should_return_empty_dict():
     parser = flexmock()
     parser.should_receive('get').never()
     parser.should_receive('getint').never()
+    parser.should_receive('getboolean').never()
     parser.should_receive('has_option').with_args('section', 'foo').and_return(False)
     parser.should_receive('has_option').with_args('section', 'bar').and_return(False)
 
@@ -197,8 +199,8 @@ def test_parse_section_options_for_missing_section_should_return_empty_dict():
 
 def insert_mock_parser():
     parser = flexmock()
-    parser.should_receive('read')
-    module.ConfigParser = lambda: parser
+    parser.should_receive('read').and_return([flexmock()])
+    module.RawConfigParser = lambda: parser
 
     return parser
 
@@ -220,3 +222,11 @@ def test_parse_configuration_should_return_section_configs():
     parsed_config = module.parse_configuration('filename', config_format)
 
     assert parsed_config == type(parsed_config)(*mock_section_configs)
+
+
+def test_parse_configuration_with_file_open_error_should_raise():
+    parser = insert_mock_parser()
+    parser.should_receive('read').and_return([])
+
+    with pytest.raises(ValueError):
+        module.parse_configuration('filename', config_format=flexmock())

borgmatic/tests/unit/test_borg.py

@@ -0,0 +1,518 @@
+from collections import OrderedDict
+from subprocess import STDOUT
+import sys
+import os
+
+from flexmock import flexmock
+
+from borgmatic import borg as module
+from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+
+
+def test_initialize_with_passphrase_should_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize({'encryption_passphrase': 'pass'}, command='borg')
+        assert os.environ.get('BORG_PASSPHRASE') == 'pass'
+    finally:
+        os.environ = orig_environ
+
+
+def test_initialize_without_passphrase_should_not_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize({}, command='borg')
+        assert os.environ.get('BORG_PASSPHRASE') == None
+    finally:
+        os.environ = orig_environ
+
+def test_write_exclude_file_does_not_raise():
+    temporary_file = flexmock(
+        name='filename',
+        write=lambda mode: None,
+        flush=lambda: None,
+    )
+    flexmock(module.tempfile).should_receive('NamedTemporaryFile').and_return(temporary_file)
+
+    module._write_exclude_file(['exclude'])
+
+
+def test_write_exclude_file_with_empty_exclude_patterns_does_not_raise():
+    module._write_exclude_file([])
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(STDOUT=STDOUT)
+    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
+    flexmock(module).subprocess = subprocess
+
+
+def insert_subprocess_never():
+    subprocess = flexmock()
+    subprocess.should_receive('check_call').never()
+    flexmock(module).subprocess = subprocess
+
+
+def insert_platform_mock():
+    flexmock(module.platform).should_receive('node').and_return('host')
+
+
+def insert_datetime_mock():
+    flexmock(module).datetime = flexmock().should_receive('now').and_return(
+        flexmock().should_receive('isoformat').and_return('now').mock
+    ).mock
+
+
+CREATE_COMMAND = ('borg', 'create', 'repo::host-now', 'foo', 'bar')
+
+
+def test_create_archive_should_call_borg_with_parameters():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(CREATE_COMMAND)
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+def test_create_archive_with_exclude_patterns_should_call_borg_with_excludes():
+    flexmock(module).should_receive('_write_exclude_file').and_return(flexmock(name='excludes'))
+    insert_subprocess_mock(CREATE_COMMAND + ('--exclude-from', 'excludes'))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': ['exclude'],
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+def test_create_archive_with_verbosity_some_should_call_borg_with_info_parameter():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(CREATE_COMMAND + ('--info', '--stats',))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        verbosity=VERBOSITY_SOME,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+def test_create_archive_with_verbosity_lots_should_call_borg_with_debug_parameter():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(CREATE_COMMAND + ('--debug', '--list', '--stats'))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        verbosity=VERBOSITY_LOTS,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+def test_create_archive_with_compression_should_call_borg_with_compression_parameters():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(CREATE_COMMAND + ('--compression', 'rle'))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'compression': 'rle'},
+        command='borg',
+    )
+
+
+def test_create_archive_with_one_file_system_should_call_borg_with_one_file_system_parameters():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(CREATE_COMMAND + ('--one-file-system',))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'one_file_system': True,
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+def test_create_archive_with_remote_path_should_call_borg_with_remote_path_parameters():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(CREATE_COMMAND + ('--remote-path', 'borg1'))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'remote_path': 'borg1',
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+def test_create_archive_with_umask_should_call_borg_with_umask_parameters():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(CREATE_COMMAND + ('--umask', '740'))
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'umask': 740},
+        command='borg',
+    )
+
+
+def test_create_archive_with_source_directories_glob_expands():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(('borg', 'create', 'repo::host-now', 'foo', 'food'))
+    insert_platform_mock()
+    insert_datetime_mock()
+    flexmock(module.glob).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo*'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+def test_create_archive_with_non_matching_source_directories_glob_passes_through():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(('borg', 'create', 'repo::host-now', 'foo*'))
+    insert_platform_mock()
+    insert_datetime_mock()
+    flexmock(module.glob).should_receive('glob').with_args('foo*').and_return([])
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo*'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+def test_create_archive_with_glob_should_call_borg_with_expanded_directories():
+    flexmock(module).should_receive('_write_exclude_file')
+    insert_subprocess_mock(('borg', 'create', 'repo::host-now', 'foo', 'food'))
+    insert_platform_mock()
+    insert_datetime_mock()
+    flexmock(module.glob).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
+
+    module.create_archive(
+        verbosity=None,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo*'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        command='borg',
+    )
+
+
+BASE_PRUNE_FLAGS = (
+    ('--keep-daily', '1'),
+    ('--keep-weekly', '2'),
+    ('--keep-monthly', '3'),
+)
+
+
+def test_make_prune_flags_should_return_flags_from_config():
+    retention_config = OrderedDict(
+        (
+            ('keep_daily', 1),
+            ('keep_weekly', 2),
+            ('keep_monthly', 3),
+        )
+    )
+
+    result = module._make_prune_flags(retention_config)
+
+    assert tuple(result) == BASE_PRUNE_FLAGS
+
+
+PRUNE_COMMAND = (
+    'borg', 'prune', 'repo', '--keep-daily', '1', '--keep-weekly', '2', '--keep-monthly', '3',
+)
+
+
+def test_prune_archives_should_call_borg_with_parameters():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS,
+    )
+    insert_subprocess_mock(PRUNE_COMMAND)
+
+    module.prune_archives(
+        verbosity=None,
+        repository='repo',
+        retention_config=retention_config,
+        command='borg',
+    )
+
+
+def test_prune_archives_with_verbosity_some_should_call_borg_with_info_parameter():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS,
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--info', '--stats',))
+
+    module.prune_archives(
+        repository='repo',
+        verbosity=VERBOSITY_SOME,
+        retention_config=retention_config,
+        command='borg',
+    )
+
+
+def test_prune_archives_with_verbosity_lots_should_call_borg_with_debug_parameter():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS,
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--debug', '--stats',))
+
+    module.prune_archives(
+        repository='repo',
+        verbosity=VERBOSITY_LOTS,
+        retention_config=retention_config,
+        command='borg',
+    )
+
+def test_prune_archive_with_remote_path_should_call_borg_with_remote_path_parameters():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS,
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--remote-path', 'borg1'))
+
+    module.prune_archives(
+        verbosity=None,
+        repository='repo',
+        retention_config=retention_config,
+        command='borg',
+        remote_path='borg1',
+    )
+
+
+def test_parse_checks_returns_them_as_tuple():
+    checks = module._parse_checks({'checks': ['foo', 'disabled', 'bar']})
+
+    assert checks == ('foo', 'bar')
+
+
+def test_parse_checks_with_missing_value_returns_defaults():
+    checks = module._parse_checks({})
+
+    assert checks == module.DEFAULT_CHECKS
+
+
+def test_parse_checks_with_blank_value_returns_defaults():
+    checks = module._parse_checks({'checks': []})
+
+    assert checks == module.DEFAULT_CHECKS
+
+
+def test_parse_checks_with_disabled_returns_no_checks():
+    checks = module._parse_checks({'checks': ['disabled']})
+
+    assert checks == ()
+
+
+def test_make_check_flags_with_checks_returns_flags():
+    flags = module._make_check_flags(('foo', 'bar'))
+
+    assert flags == ('--foo-only', '--bar-only')
+
+
+def test_make_check_flags_with_default_checks_returns_no_flags():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS)
+
+    assert flags == ()
+
+
+def test_make_check_flags_with_checks_and_last_returns_flags_including_last():
+    flags = module._make_check_flags(('foo', 'bar'), check_last=3)
+
+    assert flags == ('--foo-only', '--bar-only', '--last', 3)
+
+
+def test_make_check_flags_with_last_returns_last_flag():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS, check_last=3)
+
+    assert flags == ('--last', 3)
+
+
+def test_check_archives_should_call_borg_with_parameters():
+    checks = flexmock()
+    check_last = flexmock()
+    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(
+        ('borg', 'check', 'repo'),
+        stdout=stdout, stderr=STDOUT,
+    )
+    insert_platform_mock()
+    insert_datetime_mock()
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        verbosity=None,
+        repository='repo',
+        consistency_config=consistency_config,
+        command='borg',
+    )
+
+
+def test_check_archives_with_verbosity_some_should_call_borg_with_info_parameter():
+    consistency_config = flexmock().should_receive('get').and_return(None).mock
+    flexmock(module).should_receive('_parse_checks').and_return(flexmock())
+    flexmock(module).should_receive('_make_check_flags').and_return(())
+    insert_subprocess_mock(
+        ('borg', 'check', 'repo', '--info'),
+        stdout=None, stderr=STDOUT,
+    )
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.check_archives(
+        verbosity=VERBOSITY_SOME,
+        repository='repo',
+        consistency_config=consistency_config,
+        command='borg',
+    )
+
+
+def test_check_archives_with_verbosity_lots_should_call_borg_with_debug_parameter():
+    consistency_config = flexmock().should_receive('get').and_return(None).mock
+    flexmock(module).should_receive('_parse_checks').and_return(flexmock())
+    flexmock(module).should_receive('_make_check_flags').and_return(())
+    insert_subprocess_mock(
+        ('borg', 'check', 'repo', '--debug'),
+        stdout=None, stderr=STDOUT,
+    )
+    insert_platform_mock()
+    insert_datetime_mock()
+
+    module.check_archives(
+        verbosity=VERBOSITY_LOTS,
+        repository='repo',
+        consistency_config=consistency_config,
+        command='borg',
+    )
+
+
+def test_check_archives_without_any_checks_should_bail():
+    consistency_config = flexmock().should_receive('get').and_return(None).mock
+    flexmock(module).should_receive('_parse_checks').and_return(())
+    insert_subprocess_never()
+
+    module.check_archives(
+        verbosity=None,
+        repository='repo',
+        consistency_config=consistency_config,
+        command='borg',
+    )
+
+
+def test_check_archives_with_remote_path_should_call_borg_with_remote_path_parameters():
+    checks = flexmock()
+    check_last = flexmock()
+    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(
+        ('borg', 'check', 'repo', '--remote-path', 'borg1'),
+        stdout=stdout, stderr=STDOUT,
+    )
+    insert_platform_mock()
+    insert_datetime_mock()
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        verbosity=None,
+        repository='repo',
+        consistency_config=consistency_config,
+        command='borg',
+        remote_path='borg1',
+    )

sample/atticmatic.cron

@@ -1,3 +0,0 @@
-# You can drop this file into /etc/cron.d/ to run atticmatic nightly.
-
-0 3 * * * root PATH=$PATH:/usr/local/bin /usr/local/bin/atticmatic

sample/config

@@ -1,27 +0,0 @@
-[location]
-# Space-separated list of source directories to backup.
-source_directories: /home /etc
-
-# Path to local or remote repository.
-repository: user@backupserver:sourcehostname.attic
-
-[retention]
-# Retention policy for how many backups to keep in each category. See
-# https://attic-backup.org/usage.html#attic-prune or
-# https://borgbackup.github.io/borgbackup/usage.html#borg-prune for details.
-#keep_within: 3H
-#keep_hourly: 24
-keep_daily: 7
-keep_weekly: 4
-keep_monthly: 6
-keep_yearly: 1
-#prefix: sourcehostname
-
-[consistency]
-# Space-separated list of consistency checks to run: "repository", "archives",
-# or both. Defaults to both. Set to "disabled" to disable all consistency
-# checks. See https://attic-backup.org/usage.html#attic-check or
-# https://borgbackup.github.io/borgbackup/usage.html#borg-check for details.
-checks: repository archives
-# For Borg only, you can restrict the number of checked archives to the last n.
-#check_last: 3

sample/excludes

@@ -1,3 +0,0 @@
-*.pyc
-/home/*/.cache
-/etc/ssl

sample/systemd/borgmatic.service

@@ -0,0 +1,6 @@
+[Unit]
+Description=borgmatic backup
+
+[Service]
+Type=oneshot
+ExecStart=/usr/local/bin/borgmatic

sample/systemd/borgmatic.timer

@@ -0,0 +1,8 @@
+[Unit]
+Description=Run borgmatic backup
+
+[Timer]
+OnCalendar=daily
+
+[Install]
+WantedBy=timers.target

setup.cfg

@@ -1,2 +1,2 @@
-[nosetests]
-detailed-errors=1
+[metadata]
+description-file=README.md

setup.py

@@ -1,20 +1,45 @@
 from setuptools import setup, find_packages
 
+
+VERSION = '1.1.1'
+
+
 setup(
-    name='atticmatic',
-    version='0.1.1',
-    description='A wrapper script for Attic/Borg backup software that creates and prunes backups',
+    name='borgmatic',
+    version=VERSION,
+    description='A wrapper script for Borg backup software that creates and prunes backups',
     author='Dan Helfman',
     author_email='witten@torsion.org',
+    url='https://torsion.org/borgmatic',
+    download_url='https://torsion.org/hg/borgmatic/archive/%s.tar.gz' % VERSION,
+    classifiers=(
+        'Development Status :: 5 - Production/Stable',
+        'Environment :: Console',
+        'Intended Audience :: System Administrators',
+        'License :: OSI Approved :: GNU General Public License v3 (GPLv3)',
+        'Programming Language :: Python',
+        'Topic :: Security :: Cryptography',
+        'Topic :: System :: Archiving :: Backup',
+    ),
     packages=find_packages(),
     entry_points={
         'console_scripts': [
-            'atticmatic = atticmatic.command:main',
-            'borgmatic = atticmatic.command:main',
+            'borgmatic = borgmatic.commands.borgmatic:main',
+            'upgrade-borgmatic-config = borgmatic.commands.convert_config:main',
+            'generate-borgmatic-config = borgmatic.commands.generate_config:main',
         ]
     },
+    obsoletes=[
+        'atticmatic',
+    ],
+    install_requires=(
+        'pykwalify',
+        'ruamel.yaml<=0.15',
+        'setuptools',
+    ),
     tests_require=(
         'flexmock',
-        'nose',
-    )
+        'pytest',
+    ),
+    include_package_data=True,
 )

static/borgmatic.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" x="0px" y="0px" viewBox="0 0 100 100" enable-background="new 0 0 100 100" xml:space="preserve"><g><circle cx="63.461" cy="59.615" r="5.626"></circle><path d="M81.432,48.015c0.779-10.849-1.758-19.894-7.407-26.293C68.447,15.403,59.915,11.923,50,11.923   c-6.729,0-12.834,1.645-17.81,4.724c-0.405-0.06-0.823-0.073-1.248,0.005C8.128,20.817,0,34.623,0,46.154   c0,6.155,1.856,11.005,5.519,14.417c2.082,1.941,4.461,3.223,6.659,3.921C13.041,74.256,18.345,81,23.077,81h1.438   C30.168,94,44.847,99.614,50,99.614c6.708,0,24.493-7.875,27.38-25.392c4.758-1.888,10.697-7.373,10.697-13.711   C88.077,54.639,85.395,49.855,81.432,48.015z M7.691,46.154c0-3.397,1.238-13.399,14.747-19.083   c-2.043,3.821-3.31,8.234-3.771,13.136c-3.06,2.521-5.577,7.792-6.366,15.873c-0.534-0.316-1.056-0.686-1.538-1.135   C8.725,53.045,7.691,50.088,7.691,46.154z M74.23,67.947c-1.806,0-3.309,1.389-3.451,3.188C69.608,85.928,53.914,92.575,50,92.575   c-3.275,0-14.607-4.474-19.03-14.325c1.001-0.709,2.168-1.25,3.646-1.25c3.809,0,3.847,0,3.847,0c2.126,0,7.692-4.023,7.692-19.494   C46.154,46.241,40.588,42,38.462,42c0,0-0.195,0-3.847,0c-3.65,0-5.566-4-7.692-4h-1.046c0.813-4,2.646-8.756,5.473-11.94   c4.321-4.87,10.771-7.329,18.65-7.329c8.016,0,14.528,2.636,18.835,7.515c4.821,5.462,6.697,13.762,5.424,23.95   c-0.123,0.984,0.183,1.99,0.84,2.734s1.602,1.179,2.595,1.179c1.671,0,3.46,2.562,3.46,6.371   C81.153,64.016,75.754,67.812,74.23,67.947z M30.432,59.615c0-3.377,2.734-6.107,6.106-6.107c3.373,0,6.107,2.73,6.107,6.107   s-2.734,6.107-6.107,6.107C33.166,65.723,30.432,62.992,30.432,59.615z"></path></g></svg>

test_requirements.txt

@@ -1,2 +1,5 @@
-flexmock==0.9.7
-nose==1.3.4
+flexmock==0.10.2
+pykwalify==1.6.0
+pytest==2.9.1
+pytest-cov==2.5.1
+ruamel.yaml==0.15.18

tox.ini

@@ -1,8 +1,8 @@
 [tox]
-envlist=py27,py34
+envlist=py34
 skipsdist=True
 
 [testenv]
 usedevelop=True
 deps=-rtest_requirements.txt
-commands = nosetests []
+commands = py.test --cov-report term-missing:skip-covered --cov=borgmatic borgmatic []