Revving version for release.

Thanks for taking the time to add this! Makes sense.

AUTHORS

@@ -8,3 +8,4 @@ newtonne: Read encryption password from external file
 Robin `ypid` Schneider: Support additional options of Borg
 Scott Squires: Custom archive names
 Thomas LÉVEIL: Support for a keep_minutely prune option
+Nick Whyte: Support prefix filtering for archive consistency checks

NEWS

@@ -1,3 +1,20 @@
+1.2.0
+ * #61: Support for Borg --list option via borgmatic command-line to list all archives.
+ * #61: Support for Borg --info option via borgmatic command-line to display summary information.
+ * #62: Update README to mention other ways of installing borgmatic.
+ * Support for Borg --prefix option for consistency checks via "prefix" option in borgmatic's
+   consistency configuration.
+ * Add introductory screencast link to documentation.
+ * #59: Ignore "check_last" and consistency "prefix" when "archives" not in consistency checks.
+ * #60: Add "Persistent" flag to systemd timer example.
+ * #63: Support for Borg --nobsdflags option to skip recording bsdflags (e.g. NODUMP, IMMUTABLE) in
+   archive.
+ * #69: Support for Borg prune --umask option using value of existing "umask" option in borgmatic's
+   storage configuration.
+ * Update tox.ini to only assume Python 3.x instead of Python 3.4 specifically.
+ * Add ~/.config/borgmatic/config.yaml to default configuration path probing.
+ * Document how to develop on and contribute to borgmatic.
+
 1.1.15
  * Support for Borg BORG_PASSCOMMAND environment variable to read a password from an external file.
  * Fix for Borg create error when using borgmatic's --dry-run and --verbosity options together.

README.md

@@ -44,6 +44,8 @@ borgmatic is hosted at <https://torsion.org/borgmatic> with [source code
 available](https://projects.torsion.org/witten/borgmatic). It's also mirrored
 on [GitHub](https://github.com/witten/borgmatic) for convenience.
 
+<a href="https://asciinema.org/a/164143" target="_blank"><img src="https://asciinema.org/a/164143.png" width="100%" /></a>
+
 
 ## Installation
 
@@ -75,11 +77,18 @@ sudo pip3 install --upgrade borgmatic
 Note that your pip binary may have a different name than "pip3". Make sure
 you're using Python 3, as borgmatic does not support Python 2.
 
-### Docker
+### Other ways to install
 
-If you would like to run borgmatic within Docker, please take a look at
-[b3vis/borgmatic](https://hub.docker.com/r/b3vis/borgmatic/) for more
-information.
+ * [A borgmatic Docker image](https://hub.docker.com/r/b3vis/borgmatic/) based
+   on Alpine Linux.
+ * [Another borgmatic Docker image](https://hub.docker.com/r/coaxial/borgmatic/) based
+   on Alpine Linux, updated automatically whenever the Alpine image updates.
+ * [A borgmatic package for
+   Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic).
+ * [A borgmatic package for Arch
+   Linux](https://aur.archlinux.org/packages/borgmatic/).
+ * [A borgmatic package for OpenBSD](http://ports.su/sysutils/borgmatic).
+<br><br>
 
 ## Configuration
 
@@ -89,6 +98,9 @@ After you install borgmatic, generate a sample configuration file:
 sudo generate-borgmatic-config
 ```
 
+If that command is not found, then it may be installed in a location that's
+not in your system `PATH`. Try looking in `/usr/local/bin/`.
+
 This generates a sample configuration file at /etc/borgmatic/config.yaml (by
 default). You should edit the file to suit your needs, as the values are just
 representative. All fields are optional except where indicated, so feel free
@@ -292,17 +304,75 @@ Feel free to modify the timer file based on how frequently you'd like
 borgmatic to run.
 
 
-## Running tests
+## Support and contributing
 
-First install tox, which is used for setting up testing environments:
+### Issues
 
-```bash
-pip3 install tox
+You've got issues? Or an idea for a feature enhancement? We've got an [issue
+tracker](https://projects.torsion.org/witten/borgmatic/issues). In order to
+create a new issue or comment on an issue, you'll need to [login
+first](https://projects.torsion.org/user/login). Note that you can login with
+an existing GitHub account if you prefer.
+
+Other questions or comments? Contact <mailto:witten@torsion.org>.
+
+
+### Contributing
+
+If you'd like to contribute to borgmatic development, please feel free to
+submit a [Pull Request](https://projects.torsion.org/witten/borgmatic/pulls)
+or open an [issue](https://projects.torsion.org/witten/borgmatic/issues) first
+to discuss your idea. We also accept Pull Requests on GitHub, if that's more
+your thing. In general, contributions are very welcome. We don't bite! 
+
+
+### Development
+
+To get set up to hack on borgmatic, first clone master via HTTPS or SSH:
+
+```
+git clone https://projects.torsion.org/witten/borgmatic.git
 ```
 
-Then, to actually run tests, run:
+Or:
 
 ```bash
+git clone ssh://git@projects.torsion.org:3022/witten/borgmatic.git
+```
+
+Then, install borgmatic
+"[editable](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs)"
+so that you can easily run borgmatic commands while you're hacking on them to
+make sure your changes work.
+
+```bash
+cd borgmatic/
+pip3 install --editable --user .
+```
+
+Note that this will typically install the borgmatic commands into
+`~/.local/bin`, which may or may not be on your PATH. There are other ways to
+install borgmatic editable as well, for instance into the system Python
+install (so without `--user`, as root), or even into a
+[virtualenv](https://virtualenv.pypa.io/en/stable/). How or where you install
+borgmatic is up to you, but generally an editable install makes development
+and testing easier.
+
+
+### Running tests
+
+Assuming you've cloned the borgmatic source code as described above, and
+you're in the `borgmatic/` working copy, install tox, which is used for
+setting up testing environments:
+
+```bash
+sudo pip3 install tox
+```
+
+Finally, to actually run tests, run:
+
+```bash
+cd borgmatic
 tox
 ```
 
@@ -341,14 +411,3 @@ YAML library. If so, not to worry. borgmatic should install and function
 correctly even without the C YAML library. And borgmatic won't be any faster
 with the C library present, so you don't need to go out of your way to install
 it.
-
-
-## Issues and feedback
-
-Got an issue or an idea for a feature enhancement? Check out the [borgmatic
-issue tracker](https://projects.torsion.org/witten/borgmatic/issues). In order
-to create a new issue or comment on an issue, you'll need to [login
-first](https://projects.torsion.org/user/login). Note that you can login with
-an existing GitHub account if you prefer.
-
-Other questions or comments? Contact <mailto:witten@torsion.org>.

borgmatic/borg/check.py

@@ -7,6 +7,7 @@ from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
 
 
 DEFAULT_CHECKS = ('repository', 'archives')
+DEFAULT_PREFIX = '{hostname}-'
 
 
 logger = logging.getLogger(__name__)
@@ -34,7 +35,7 @@ def _parse_checks(consistency_config):
     return tuple(check for check in checks if check.lower() not in ('disabled', '')) or DEFAULT_CHECKS
 
 
-def _make_check_flags(checks, check_last=None):
+def _make_check_flags(checks, check_last=None, prefix=None):
     '''
     Given a parsed sequence of checks, transform it into tuple of command-line flags.
 
@@ -47,17 +48,30 @@ def _make_check_flags(checks, check_last=None):
         ('--repository-only',)
 
     However, if both "repository" and "archives" are in checks, then omit them from the returned
-    flags because Borg does both checks by default. Additionally, if a check_last value is given,
-    a "--last" flag will be added.
+    flags because Borg does both checks by default.
+
+    Additionally, if a check_last value is given and "archives" is in checks, then include a
+    "--last" flag. And if a prefix value is given and "archives" is in checks, then include a
+    "--prefix" flag.
     '''
-    last_flag = ('--last', str(check_last)) if check_last else ()
+    if 'archives' in checks:
+        last_flags = ('--last', str(check_last)) if check_last else ()
+        prefix_flags = ('--prefix', prefix) if prefix else ('--prefix', DEFAULT_PREFIX)
+    else:
+        last_flags = ()
+        prefix_flags = ()
+        if check_last:
+            logger.warn('Ignoring check_last option, as "archives" is not in consistency checks.')
+        if prefix:
+            logger.warn('Ignoring consistency prefix option, as "archives" is not in consistency checks.')
+        
     if set(DEFAULT_CHECKS).issubset(set(checks)):
-        return last_flag
+        return last_flags + prefix_flags
 
     return tuple(
         '--{}-only'.format(check) for check in checks
         if check in DEFAULT_CHECKS
-    ) + last_flag
+    ) + last_flags + prefix_flags
 
 
 def check_archives(verbosity, repository, storage_config, consistency_config, local_path='borg',
@@ -81,11 +95,12 @@ def check_archives(verbosity, repository, storage_config, consistency_config, lo
             VERBOSITY_SOME: ('--info',),
             VERBOSITY_LOTS: ('--debug',),
         }.get(verbosity, ())
+        prefix = consistency_config.get('prefix')
 
         full_command = (
             local_path, 'check',
             repository,
-        ) + _make_check_flags(checks, check_last) + remote_path_flags + lock_wait_flags + verbosity_flags
+        ) + _make_check_flags(checks, check_last, prefix) + remote_path_flags + lock_wait_flags + verbosity_flags
 
         # The check command spews to stdout/stderr even without the verbose flag. Suppress it.
         stdout = None if verbosity_flags else open(os.devnull, 'w')

borgmatic/borg/create.py

@@ -114,44 +114,46 @@ def create_archive(
     sources = _expand_directories(location_config['source_directories'])
 
     pattern_file = _write_pattern_file(location_config.get('patterns'))
-    pattern_flags = _make_pattern_flags(
-        location_config,
-        pattern_file.name if pattern_file else None,
-    )
     exclude_file = _write_pattern_file(_expand_directories(location_config.get('exclude_patterns')))
-    exclude_flags = _make_exclude_flags(
-        location_config,
-        exclude_file.name if exclude_file else None,
-    )
     compression = storage_config.get('compression', None)
-    compression_flags = ('--compression', compression) if compression else ()
     remote_rate_limit = storage_config.get('remote_rate_limit', None)
-    remote_rate_limit_flags = ('--remote-ratelimit', str(remote_rate_limit)) if remote_rate_limit else ()
     umask = storage_config.get('umask', None)
-    umask_flags = ('--umask', str(umask)) if umask else ()
     lock_wait = storage_config.get('lock_wait', None)
-    lock_wait_flags = ('--lock-wait', str(lock_wait)) if lock_wait else ()
-    one_file_system_flags = ('--one-file-system',) if location_config.get('one_file_system') else ()
     files_cache = location_config.get('files_cache')
-    files_cache_flags = ('--files-cache', files_cache) if files_cache else ()
-    remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
-    verbosity_flags = {
-        VERBOSITY_SOME: ('--info',) if dry_run else ('--info', '--stats',),
-        VERBOSITY_LOTS: ('--debug', '--list',) if dry_run else ('--debug', '--list', '--stats',),
-    }.get(verbosity, ())
-    dry_run_flags = ('--dry-run',) if dry_run else ()
     default_archive_name_format = '{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}'
     archive_name_format = storage_config.get('archive_name_format', default_archive_name_format)
 
     full_command = (
-        local_path, 'create',
-        '{repository}::{archive_name_format}'.format(
-            repository=repository,
-            archive_name_format=archive_name_format,
-        ),
-    ) + sources + pattern_flags + exclude_flags + compression_flags + remote_rate_limit_flags + \
-        one_file_system_flags + files_cache_flags + remote_path_flags + umask_flags + \
-        lock_wait_flags + verbosity_flags + dry_run_flags
+        (
+            local_path, 'create',
+            '{repository}::{archive_name_format}'.format(
+                repository=repository,
+                archive_name_format=archive_name_format,
+            ),
+        )
+        + sources
+        + _make_pattern_flags(
+            location_config,
+            pattern_file.name if pattern_file else None,
+        )
+        + _make_exclude_flags(
+            location_config,
+            exclude_file.name if exclude_file else None,
+        )
+        + (('--compression', compression) if compression else ())
+        + (('--remote-ratelimit', str(remote_rate_limit)) if remote_rate_limit else ())
+        + (('--one-file-system',) if location_config.get('one_file_system') else ())
+        + (('--nobsdflags',) if location_config.get('bsd_flags') is False else ())
+        + (('--files-cache', files_cache) if files_cache else ())
+        + (('--remote-path', remote_path) if remote_path else ())
+        + (('--umask', str(umask)) if umask else ())
+        + (('--lock-wait', str(lock_wait)) if lock_wait else ())
+        + {
+            VERBOSITY_SOME: ('--info',) if dry_run else ('--info', '--stats',),
+            VERBOSITY_LOTS: ('--debug', '--list',) if dry_run else ('--debug', '--list', '--stats',),
+        }.get(verbosity, ())
+        + (('--dry-run',) if dry_run else ())
+    )
 
     logger.debug(' '.join(full_command))
     subprocess.check_call(full_command)

borgmatic/borg/info.py

@@ -0,0 +1,29 @@
+import logging
+import subprocess
+
+from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+
+
+logger = logging.getLogger(__name__)
+
+
+def display_archives_info(verbosity, repository, storage_config, local_path='borg',
+                          remote_path=None):
+    '''
+    Given a verbosity flag, a local or remote repository path, and a storage config dict,
+    display summary information for Borg archives in the repository.
+    '''
+    lock_wait = storage_config.get('lock_wait', None)
+
+    full_command = (
+        (local_path, 'info', repository)
+        + (('--remote-path', remote_path) if remote_path else ())
+        + (('--lock-wait', str(lock_wait)) if lock_wait else ())
+        + {
+            VERBOSITY_SOME: ('--info',),
+            VERBOSITY_LOTS: ('--debug',),
+        }.get(verbosity, ())
+    )
+
+    logger.debug(' '.join(full_command))
+    subprocess.check_call(full_command)

borgmatic/borg/list.py

@@ -0,0 +1,28 @@
+import logging
+import subprocess
+
+from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+
+
+logger = logging.getLogger(__name__)
+
+
+def list_archives(verbosity, repository, storage_config, local_path='borg', remote_path=None):
+    '''
+    Given a verbosity flag, a local or remote repository path, and a storage config dict,
+    list Borg archives in the repository.
+    '''
+    lock_wait = storage_config.get('lock_wait', None)
+
+    full_command = (
+        (local_path, 'list', repository)
+        + (('--remote-path', remote_path) if remote_path else ())
+        + (('--lock-wait', str(lock_wait)) if lock_wait else ())
+        + {
+            VERBOSITY_SOME: ('--info',),
+            VERBOSITY_LOTS: ('--debug',),
+        }.get(verbosity, ())
+    )
+
+    logger.debug(' '.join(full_command))
+    subprocess.check_call(full_command)

borgmatic/borg/prune.py

@@ -36,26 +36,30 @@ def prune_archives(verbosity, dry_run, repository, storage_config, retention_con
                    local_path='borg', remote_path=None):
     '''
     Given verbosity/dry-run flags, a local or remote repository path, a storage config dict, and a
-    retention config dict, prune Borg archives according the the retention policy specified in that
+    retention config dict, prune Borg archives according to the retention policy specified in that
     configuration.
     '''
-    remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
+    umask = storage_config.get('umask', None)
     lock_wait = storage_config.get('lock_wait', None)
-    lock_wait_flags = ('--lock-wait', str(lock_wait)) if lock_wait else ()
-    verbosity_flags = {
-        VERBOSITY_SOME: ('--info', '--stats',),
-        VERBOSITY_LOTS: ('--debug', '--stats', '--list'),
-    }.get(verbosity, ())
-    dry_run_flags = ('--dry-run',) if dry_run else ()
 
     full_command = (
-        local_path, 'prune',
-        repository,
-    ) + tuple(
-        element
-        for pair in _make_prune_flags(retention_config)
-        for element in pair
-    ) + remote_path_flags + lock_wait_flags + verbosity_flags + dry_run_flags
+        (
+            local_path, 'prune',
+            repository,
+        ) + tuple(
+            element
+            for pair in _make_prune_flags(retention_config)
+            for element in pair
+        )
+        + (('--remote-path', remote_path) if remote_path else ())
+        + (('--umask', str(umask)) if umask else ())
+        + (('--lock-wait', str(lock_wait)) if lock_wait else ())
+        + {
+            VERBOSITY_SOME: ('--info', '--stats',),
+            VERBOSITY_LOTS: ('--debug', '--stats', '--list'),
+        }.get(verbosity, ())
+        + (('--dry-run',) if dry_run else ())
+    )
 
     logger.debug(' '.join(full_command))
     subprocess.check_call(full_command)

borgmatic/commands/borgmatic.py

@@ -5,7 +5,8 @@ import os
 from subprocess import CalledProcessError
 import sys
 
-from borgmatic.borg import check, create, prune
+from borgmatic.borg import check as borg_check, create as borg_create, prune as borg_prune, \
+     list as borg_list, info as borg_info
 from borgmatic.commands import hook
 from borgmatic.config import collect, convert, validate
 from borgmatic.signals import configure_signals
@@ -61,11 +62,23 @@ def parse_arguments(*arguments):
         action='store_true',
         help='Check archives for consistency',
     )
+    parser.add_argument(
+        '-l', '--list',
+        dest='list',
+        action='store_true',
+        help='List archives',
+    )
+    parser.add_argument(
+        '-i', '--info',
+        dest='info',
+        action='store_true',
+        help='Display summary information on archives',
+    )
     parser.add_argument(
         '-n', '--dry-run',
         dest='dry_run',
         action='store_true',
-        help='Go through the motions, but do not actually write any changes to the repository',
+        help='Go through the motions, but do not actually write to any repositories',
     )
     parser.add_argument(
         '-v', '--verbosity',
@@ -75,14 +88,14 @@ def parse_arguments(*arguments):
 
     args = parser.parse_args(arguments)
 
-    # If any of the three action flags in the given parse arguments have been explicitly requested,
-    # leave them as-is. Otherwise, assume defaults: Mutate the given arguments to enable all the
-    # actions.
-    if not args.prune and not args.create and not args.check:
-        args.prune = True
-        args.create = True
-        args.check = True
+    # If any of the action flags are explicitly requested, leave them as-is. Otherwise, assume
+    # defaults: Mutate the given arguments to enable the default actions.
+    if args.prune or args.create or args.check or args.list or args.info:
+        return args
 
+    args.prune = True
+    args.create = True
+    args.check = True
     return args
 
 
@@ -101,7 +114,7 @@ def run_configuration(config_filename, args):  # pragma: no cover
     try:
         local_path = location.get('local_path', 'borg')
         remote_path = location.get('remote_path')
-        create.initialize_environment(storage)
+        borg_create.initialize_environment(storage)
         hook.execute_hook(hooks.get('before_backup'), config_filename, 'pre-backup')
 
         for unexpanded_repository in location['repositories']:
@@ -109,7 +122,7 @@ def run_configuration(config_filename, args):  # pragma: no cover
             dry_run_label = ' (dry run; not making any changes)' if args.dry_run else ''
             if args.prune:
                 logger.info('{}: Pruning archives{}'.format(repository, dry_run_label))
-                prune.prune_archives(
+                borg_prune.prune_archives(
                     args.verbosity,
                     args.dry_run,
                     repository,
@@ -120,7 +133,7 @@ def run_configuration(config_filename, args):  # pragma: no cover
                 )
             if args.create:
                 logger.info('{}: Creating archive{}'.format(repository, dry_run_label))
-                create.create_archive(
+                borg_create.create_archive(
                     args.verbosity,
                     args.dry_run,
                     repository,
@@ -131,14 +144,32 @@ def run_configuration(config_filename, args):  # pragma: no cover
                 )
             if args.check:
                 logger.info('{}: Running consistency checks'.format(repository))
-                check.check_archives(
-                    args.verbosity,
+                borg_check.check_archives(
+                   args.verbosity,
                     repository,
                     storage,
                     consistency,
                     local_path=local_path,
                     remote_path=remote_path,
                 )
+            if args.list:
+                logger.info('{}: Listing archives'.format(repository))
+                borg_list.list_archives(
+                    args.verbosity,
+                    repository,
+                    storage,
+                    local_path=local_path,
+                    remote_path=remote_path,
+                )
+            if args.info:
+                logger.info('{}: Displaying summary info for archives'.format(repository))
+                borg_info.display_archives_info(
+                    args.verbosity,
+                    repository,
+                    storage,
+                    local_path=local_path,
+                    remote_path=remote_path,
+                )
 
         hook.execute_hook(hooks.get('after_backup'), config_filename, 'post-backup')
     except (OSError, CalledProcessError):

borgmatic/config/collect.py

@@ -1,7 +1,11 @@
 import os
 
 
-DEFAULT_CONFIG_PATHS = ['/etc/borgmatic/config.yaml', '/etc/borgmatic.d']
+DEFAULT_CONFIG_PATHS = [
+    '/etc/borgmatic/config.yaml',
+    '/etc/borgmatic.d',
+    os.path.expanduser('~/.config/borgmatic/config.yaml'),
+]
 
 
 def collect_config_filenames(config_paths):

borgmatic/config/schema.yaml

@@ -22,6 +22,10 @@ map:
                 type: bool
                 desc: Stay in same file system (do not cross mount points).
                 example: true
+            bsd_flags:
+                type: bool
+                desc: Record bsdflags (e.g. NODUMP, IMMUTABLE) in archive. Defaults to true.
+                example: true
             files_cache:
                 type: scalar
                 desc: |
@@ -150,7 +154,8 @@ map:
                     "borg help placeholders" for details. Default is
                     "{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}". If you specify this option, you must
                     also specify a prefix in the retention section to avoid accidental pruning of
-                    archives with a different archive name format.
+                    archives with a different archive name format. And you should also specify a
+                    prefix in the consistency section as well.
                 example: "{hostname}-documents-{now}"
     retention:
         desc: |
@@ -218,6 +223,13 @@ map:
                 desc: Restrict the number of checked archives to the last n. Applies only to the
                       "archives" check.
                 example: 3
+            prefix:
+                type: scalar
+                desc: |
+                    When performing the "archives" check, only consider archive names starting with
+                    this prefix. Borg placeholders can be used. See the output of
+                    "borg help placeholders" for details. Default is "{hostname}-".
+                example: sourcehostname
     hooks:
         desc: |
             Shell commands or scripts to execute before and after a backup or if an error has occurred.

borgmatic/config/validate.py

@@ -8,6 +8,8 @@ import pykwalify.errors
 from ruamel import yaml
 
 
+logger = logging.getLogger(__name__)
+
 def schema_filename():
     '''
     Path to the installed YAML configuration schema file, used to validate and parse the
@@ -50,6 +52,11 @@ def apply_logical_validation(config_filename, parsed_configuration):
             )
         )
 
+    consistency_prefix = parsed_configuration.get('consistency', {}).get('prefix')
+    if archive_name_format and not consistency_prefix:
+        logger.warning('Since version 1.1.16, if you provide `archive_name_format`, you should also'
+                       ' specify `consistency.prefix`.')
+
 
 def parse_configuration(config_filename, schema_filename):
     '''

borgmatic/tests/unit/borg/test_check.py

@@ -42,40 +42,64 @@ def test_parse_checks_with_disabled_returns_no_checks():
     assert checks == ()
 
 
-def test_make_check_flags_with_checks_returns_flags():
+def test_make_check_flags_with_repository_check_returns_flag():
     flags = module._make_check_flags(('repository',))
 
     assert flags == ('--repository-only',)
 
 
-def test_make_check_flags_with_extract_check_does_not_make_extract_flag():
+def test_make_check_flags_with_extract_omits_extract_flag():
     flags = module._make_check_flags(('extract',))
 
     assert flags == ()
 
 
-def test_make_check_flags_with_default_checks_returns_no_flags():
+def test_make_check_flags_with_default_checks_returns_default_flags():
     flags = module._make_check_flags(module.DEFAULT_CHECKS)
 
-    assert flags == ()
+    assert flags == ('--prefix', module.DEFAULT_PREFIX)
 
 
-def test_make_check_flags_with_all_checks_returns_no_flags():
+def test_make_check_flags_with_all_checks_returns_default_flags():
     flags = module._make_check_flags(module.DEFAULT_CHECKS + ('extract',))
 
-    assert flags == ()
+    assert flags == ('--prefix', module.DEFAULT_PREFIX)
 
 
-def test_make_check_flags_with_checks_and_last_returns_flags_including_last():
+def test_make_check_flags_with_archives_check_and_last_includes_last_flag():
+    flags = module._make_check_flags(('archives',), check_last=3)
+
+    assert flags == ('--archives-only', '--last', '3', '--prefix', module.DEFAULT_PREFIX)
+
+
+def test_make_check_flags_with_repository_check_and_last_omits_last_flag():
     flags = module._make_check_flags(('repository',), check_last=3)
 
-    assert flags == ('--repository-only', '--last', '3')
+    assert flags == ('--repository-only',)
 
 
-def test_make_check_flags_with_default_checks_and_last_returns_last_flag():
+def test_make_check_flags_with_default_checks_and_last_includes_last_flag():
     flags = module._make_check_flags(module.DEFAULT_CHECKS, check_last=3)
 
-    assert flags == ('--last', '3')
+    assert flags == ('--last', '3', '--prefix', module.DEFAULT_PREFIX)
+
+
+def test_make_check_flags_with_archives_check_and_prefix_includes_prefix_flag():
+    flags = module._make_check_flags(('archives',), prefix='foo-')
+
+    assert flags == ('--archives-only', '--prefix', 'foo-')
+
+
+def test_make_check_flags_with_repository_check_and_prefix_omits_prefix_flag():
+    flags = module._make_check_flags(('repository',), prefix='foo-')
+
+    assert flags == ('--repository-only',)
+
+
+def test_make_check_flags_with_default_checks_and_prefix_includes_prefix_flag():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS, prefix='foo-')
+
+    assert flags == ('--prefix', 'foo-')
 
 
 @pytest.mark.parametrize(
@@ -89,9 +113,9 @@ def test_make_check_flags_with_default_checks_and_last_returns_last_flag():
 )
 def test_check_archives_calls_borg_with_parameters(checks):
     check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    consistency_config = {'check_last': check_last}
     flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, None).and_return(())
     stdout = flexmock()
     insert_subprocess_mock(
         ('borg', 'check', 'repo'),
@@ -111,7 +135,7 @@ def test_check_archives_calls_borg_with_parameters(checks):
 def test_check_archives_with_extract_check_calls_extract_only():
     checks = ('extract',)
     check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    consistency_config = {'check_last': check_last}
     flexmock(module).should_receive('_parse_checks').and_return(checks)
     flexmock(module).should_receive('_make_check_flags').never()
     flexmock(module.extract).should_receive('extract_last_archive_dry_run').once()
@@ -127,7 +151,7 @@ def test_check_archives_with_extract_check_calls_extract_only():
 
 def test_check_archives_with_verbosity_some_calls_borg_with_info_parameter():
     checks = ('repository',)
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
+    consistency_config = {'check_last': None}
     flexmock(module).should_receive('_parse_checks').and_return(checks)
     flexmock(module).should_receive('_make_check_flags').and_return(())
     insert_subprocess_mock(
@@ -145,7 +169,7 @@ def test_check_archives_with_verbosity_some_calls_borg_with_info_parameter():
 
 def test_check_archives_with_verbosity_lots_calls_borg_with_debug_parameter():
     checks = ('repository',)
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
+    consistency_config = {'check_last': None}
     flexmock(module).should_receive('_parse_checks').and_return(checks)
     flexmock(module).should_receive('_make_check_flags').and_return(())
     insert_subprocess_mock(
@@ -162,7 +186,7 @@ def test_check_archives_with_verbosity_lots_calls_borg_with_debug_parameter():
 
 
 def test_check_archives_without_any_checks_bails():
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
+    consistency_config = {'check_last': None}
     flexmock(module).should_receive('_parse_checks').and_return(())
     insert_subprocess_never()
 
@@ -177,9 +201,9 @@ def test_check_archives_without_any_checks_bails():
 def test_check_archives_with_local_path_calls_borg_via_local_path():
     checks = ('repository',)
     check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    consistency_config = {'check_last': check_last}
     flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, None).and_return(())
     stdout = flexmock()
     insert_subprocess_mock(
         ('borg1', 'check', 'repo'),
@@ -200,9 +224,9 @@ def test_check_archives_with_local_path_calls_borg_via_local_path():
 def test_check_archives_with_remote_path_calls_borg_with_remote_path_parameters():
     checks = ('repository',)
     check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    consistency_config = {'check_last': check_last}
     flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, None).and_return(())
     stdout = flexmock()
     insert_subprocess_mock(
         ('borg', 'check', 'repo', '--remote-path', 'borg1'),
@@ -223,9 +247,9 @@ def test_check_archives_with_remote_path_calls_borg_with_remote_path_parameters(
 def test_check_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
     checks = ('repository',)
     check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    consistency_config = {'check_last': check_last}
     flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, None).and_return(())
     stdout = flexmock()
     insert_subprocess_mock(
         ('borg', 'check', 'repo', '--lock-wait', '5'),
@@ -240,3 +264,27 @@ def test_check_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
         storage_config={'lock_wait': 5},
         consistency_config=consistency_config,
     )
+
+
+def test_check_archives_with_retention_prefix():
+    checks = ('repository',)
+    check_last = flexmock()
+    prefix = 'foo-'
+    consistency_config = {'check_last': check_last, 'prefix': prefix}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, prefix).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(
+        ('borg', 'check', 'repo'),
+        stdout=stdout, stderr=STDOUT,
+    )
+
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        verbosity=None,
+        repository='repo',
+        storage_config={},
+        consistency_config=consistency_config,
+    )

borgmatic/tests/unit/borg/test_create.py

@@ -435,6 +435,48 @@ def test_create_archive_with_one_file_system_calls_borg_with_one_file_system_par
     )
 
 
+def test_create_archive_with_bsd_flags_true_calls_borg_without_nobsdflags_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND)
+
+    module.create_archive(
+        verbosity=None,
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'bsd_flags': True,
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_bsd_flags_false_calls_borg_with_nobsdflags_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--nobsdflags',))
+
+    module.create_archive(
+        verbosity=None,
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'bsd_flags': False,
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
 def test_create_archive_with_files_cache_calls_borg_with_files_cache_parameters():
     flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
     flexmock(module).should_receive('_write_pattern_file').and_return(None)

borgmatic/tests/unit/borg/test_info.py

@@ -0,0 +1,77 @@
+from collections import OrderedDict
+
+from flexmock import flexmock
+
+from borgmatic.borg import info as module
+from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
+
+
+INFO_COMMAND = ('borg', 'info', 'repo')
+
+
+def test_display_archives_info_calls_borg_with_parameters():
+    insert_subprocess_mock(INFO_COMMAND)
+
+    module.display_archives_info(
+        verbosity=None,
+        repository='repo',
+        storage_config={},
+    )
+
+
+def test_display_archives_info_with_verbosity_some_calls_borg_with_info_parameter():
+    insert_subprocess_mock(INFO_COMMAND + ('--info',))
+
+    module.display_archives_info(
+        repository='repo',
+        storage_config={},
+        verbosity=VERBOSITY_SOME,
+    )
+
+
+def test_display_archives_info_with_verbosity_lots_calls_borg_with_debug_parameter():
+    insert_subprocess_mock(INFO_COMMAND + ('--debug',))
+
+    module.display_archives_info(
+        repository='repo',
+        storage_config={},
+        verbosity=VERBOSITY_LOTS,
+    )
+
+
+def test_display_archives_info_with_local_path_calls_borg_via_local_path():
+    insert_subprocess_mock(('borg1',) + INFO_COMMAND[1:])
+
+    module.display_archives_info(
+        verbosity=None,
+        repository='repo',
+        storage_config={},
+        local_path='borg1',
+    )
+
+
+def test_display_archives_info_with_remote_path_calls_borg_with_remote_path_parameters():
+    insert_subprocess_mock(INFO_COMMAND + ('--remote-path', 'borg1'))
+
+    module.display_archives_info(
+        verbosity=None,
+        repository='repo',
+        storage_config={},
+        remote_path='borg1',
+    )
+
+
+def test_display_archives_info_with_lock_wait_calls_borg_with_lock_wait_parameters():
+    storage_config = {'lock_wait': 5}
+    insert_subprocess_mock(INFO_COMMAND + ('--lock-wait', '5'))
+
+    module.display_archives_info(
+        verbosity=None,
+        repository='repo',
+        storage_config=storage_config,
+    )

borgmatic/tests/unit/borg/test_list.py

@@ -0,0 +1,77 @@
+from collections import OrderedDict
+
+from flexmock import flexmock
+
+from borgmatic.borg import list as module
+from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
+
+
+LIST_COMMAND = ('borg', 'list', 'repo')
+
+
+def test_list_archives_calls_borg_with_parameters():
+    insert_subprocess_mock(LIST_COMMAND)
+
+    module.list_archives(
+        verbosity=None,
+        repository='repo',
+        storage_config={},
+    )
+
+
+def test_list_archives_with_verbosity_some_calls_borg_with_info_parameter():
+    insert_subprocess_mock(LIST_COMMAND + ('--info',))
+
+    module.list_archives(
+        repository='repo',
+        storage_config={},
+        verbosity=VERBOSITY_SOME,
+    )
+
+
+def test_list_archives_with_verbosity_lots_calls_borg_with_debug_parameter():
+    insert_subprocess_mock(LIST_COMMAND + ('--debug',))
+
+    module.list_archives(
+        repository='repo',
+        storage_config={},
+        verbosity=VERBOSITY_LOTS,
+    )
+
+
+def test_list_archives_with_local_path_calls_borg_via_local_path():
+    insert_subprocess_mock(('borg1',) + LIST_COMMAND[1:])
+
+    module.list_archives(
+        verbosity=None,
+        repository='repo',
+        storage_config={},
+        local_path='borg1',
+    )
+
+
+def test_list_archives_with_remote_path_calls_borg_with_remote_path_parameters():
+    insert_subprocess_mock(LIST_COMMAND + ('--remote-path', 'borg1'))
+
+    module.list_archives(
+        verbosity=None,
+        repository='repo',
+        storage_config={},
+        remote_path='borg1',
+    )
+
+
+def test_list_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
+    storage_config = {'lock_wait': 5}
+    insert_subprocess_mock(LIST_COMMAND + ('--lock-wait', '5'))
+
+    module.list_archives(
+        verbosity=None,
+        repository='repo',
+        storage_config=storage_config,
+    )

borgmatic/tests/unit/borg/test_prune.py

@@ -153,6 +153,23 @@ def test_prune_archives_with_remote_path_calls_borg_with_remote_path_parameters(
     )
 
 
+def test_prune_archives_with_umask_calls_borg_with_umask_parameters():
+    storage_config = {'umask': '077'}
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS,
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--umask', '077'))
+
+    module.prune_archives(
+        verbosity=None,
+        dry_run=False,
+        repository='repo',
+        storage_config=storage_config,
+        retention_config=retention_config,
+    )
+
+
 def test_prune_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
     storage_config = {'lock_wait': 5}
     retention_config = flexmock()

borgmatic/tests/unit/config/test_validate.py

@@ -1,4 +1,5 @@
 import pytest
+from flexmock import flexmock
 
 from borgmatic.config import validate as module
 
@@ -23,13 +24,42 @@ def test_apply_logical_validation_raises_if_archive_name_format_present_without_
             },
         )
 
+def test_apply_logical_validation_raises_if_archive_name_format_present_without_retention_prefix():
+    with pytest.raises(module.Validation_error):
+        module.apply_logical_validation(
+            'config.yaml',
+            {
+                'storage': {'archive_name_format': '{hostname}-{now}'},
+                'retention': {'keep_daily': 7},
+                'consistency': {'prefix': '{hostname}-'}
+            },
+        )
+
+
+def test_apply_logical_validation_warns_if_archive_name_format_present_without_consistency_prefix():
+    logger = flexmock(module.logger)
+    logger.should_receive('warning').once()
 
-def test_apply_logical_validation_does_not_raise_if_archive_name_format_and_prefix_present():
     module.apply_logical_validation(
         'config.yaml',
         {
             'storage': {'archive_name_format': '{hostname}-{now}'},
             'retention': {'prefix': '{hostname}-'},
+            'consistency': {},
+        },
+    )
+
+
+def test_apply_logical_validation_does_not_raise_or_warn_if_archive_name_format_and_prefix_present():
+    logger = flexmock(module.logger)
+    logger.should_receive('warning').never()
+
+    module.apply_logical_validation(
+        'config.yaml',
+        {
+            'storage': {'archive_name_format': '{hostname}-{now}'},
+            'retention': {'prefix': '{hostname}-'},
+            'consistency': {'prefix': '{hostname}-'}
         },
     )
 

borgmatic/tests/unit/test_verbosity.py

@@ -7,5 +7,5 @@ def test_verbosity_to_log_level_maps_known_verbosity_to_log_level():
     assert module.verbosity_to_log_level(module.VERBOSITY_SOME) == logging.INFO
 
 
-def test_verbosity_to_log_level_maps_unknown_verbosity_to_error_level():
-    assert module.verbosity_to_log_level('my pants') == logging.ERROR
+def test_verbosity_to_log_level_maps_unknown_verbosity_to_warning_level():
+    assert module.verbosity_to_log_level('my pants') == logging.WARNING

borgmatic/verbosity.py

@@ -12,4 +12,4 @@ def verbosity_to_log_level(verbosity):
     return {
         VERBOSITY_SOME: logging.INFO,
         VERBOSITY_LOTS: logging.DEBUG,
-    }.get(verbosity, logging.ERROR)
+    }.get(verbosity, logging.WARNING)

sample/systemd/borgmatic.timer

@@ -3,6 +3,7 @@ Description=Run borgmatic backup
 
 [Timer]
 OnCalendar=daily
+Persistent=true
 
 [Install]
 WantedBy=timers.target

scripts/find-unsupported-borg-options

@@ -1,5 +1,7 @@
 #!/bin/bash
 
+set -o nounset
+
 # For each Borg sub-command that borgmatic uses, print out the Borg flags that borgmatic does not
 # appear to support yet. This script isn't terribly robust. It's intended as a basic tool to ferret
 # out unsupported Borg options so that they can be considered for addition to borgmatic.
@@ -11,10 +13,10 @@ generate-borgmatic-config --destination temp.yaml
 # from running borgmatic with the generated configuration. Then, collect the full set of available
 # Borg flags as reported by "borg --help" for that sub-command. Finally, compare the two lists of
 # flags to determine which Borg flags borgmatic doesn't yet support.
-for sub_command in prune create check; do
+for sub_command in prune create check list info; do
     echo "********** borg $sub_command **********"
 
-    for line in $(borgmatic --config temp.yaml --$sub_command -v 2 2>&1 | grep "borg $sub_command") ; do
+    for line in $(borgmatic --config temp.yaml --$sub_command -v 2 2>&1 | grep "borg\w* $sub_command") ; do
         echo "$line" | grep '^-' >> borgmatic_borg_flags
     done
     sort borgmatic_borg_flags > borgmatic_borg_flags.sorted

setup.py

@@ -1,7 +1,7 @@
 from setuptools import setup, find_packages
 
 
-VERSION = '1.1.15'
+VERSION = '1.2.0'
 
 
 setup(

tox.ini

@@ -1,5 +1,5 @@
 [tox]
-envlist=py34
+envlist=py3
 skipsdist=True
 
 [testenv]