Bumping version for release.

AUTHORS

@@ -2,7 +2,8 @@ Dan Helfman <witten@torsion.org>: Main developer
 
 Alexander Görtz: Python 3 compatibility
 Henning Schroeder: Copy editing
+Johannes Feichtner: Support for user hooks
 Michele Lazzeri: Custom archive names
 Robin `ypid` Schneider: Support additional options of Borg
 Scott Squires: Custom archive names
-Johannes Feichtner: Support for user hooks
+Thomas LÉVEIL: Support for a keep_minutely prune option

NEWS

@@ -1,15 +1,38 @@
+1.1.14
+ * #49: Fix for typo in --patterns-from option.
+ * #47: Support for Borg --dry-run option via borgmatic command-line.
+
+1.1.13
+ * #54: Fix for incorrect consistency check flags passed to Borg when all three checks ("repository",
+   "archives", and "extract") are specified in borgmatic configuration.
+ * #48: Add "local_path" to configuration for specifying an alternative Borg executable path.
+ * #49: Support for Borg experimental --patterns-from and --patterns options for specifying mixed
+   includes/excludes.
+ * Moved issue tracker from Taiga to integrated Gitea tracker at
+   https://projects.torsion.org/witten/borgmatic/issues
+
+1.1.12
+ * #46: Declare dependency on pykwalify 1.6 or above, as older versions yield "Unknown key: version"
+   rule errors.
+ * Support for Borg --keep-minutely prune option.
+
+1.1.11
+ * #26: Add "ssh_command" to configuration for specifying a custom SSH command or options.
+ * Fix for incorrect /etc/borgmatic.d/ configuration path probing on macOS. This problem manifested
+   as an error on startup: "[Errno 2] No such file or directory: '/etc/borgmatic.d'".
+
 1.1.10
  * Pass several Unix signals through to child processes like Borg. This means that Borg now properly
    shuts down if borgmatic is terminated (e.g. due to a system suspend).
- * #29: Support for using tilde in repository paths to reference home directory.
- * #42: Support for Borg --files-cache option for setting the files cache operation mode.
- * #44: Support for Borg --remote-ratelimit for limiting upload rate.
+ * #30: Support for using tilde in repository paths to reference home directory.
+ * #43: Support for Borg --files-cache option for setting the files cache operation mode.
+ * #45: Support for Borg --remote-ratelimit option for limiting upload rate.
  * Log invoked Borg commands when at highest verbosity level.
 
 1.1.9
- * #16, #38: Support for user-defined hooks before/after backup, or on error.
- * #33: Improve clarity of logging spew at high verbosity levels.
- * #29: Support for using tilde in source directory path to reference home directory.
+ * #17, #39: Support for user-defined hooks before/after backup, or on error.
+ * #34: Improve clarity of logging spew at high verbosity levels.
+ * #30: Support for using tilde in source directory path to reference home directory.
  * Require "prefix" in retention section when "archive_name_format" is set. This is to avoid
    accidental pruning of archives with a different archive name format. For similar reasons, default
    "prefix" to "{hostname}-" if not specified.
@@ -17,43 +40,43 @@
  * Update dead links to Borg documentation.
 
 1.1.8
- * #39: Fix to make /etc/borgmatic/config.yaml optional rather than required when using the default
+ * #40: Fix to make /etc/borgmatic/config.yaml optional rather than required when using the default
    config paths.
 
 1.1.7
 
- * #28: Add "archive_name_format" to configuration for customizing archive names.
+ * #29: Add "archive_name_format" to configuration for customizing archive names.
  * Fix for traceback when "exclude_from" value is empty in configuration file.
  * When pruning, make highest verbosity level list archives kept and pruned.
  * Clarification of Python 3 pip usage in documentation.
 
 1.1.6
 
- * #12, #35: Support for Borg --exclude-from, --exclude-caches, and --exclude-if-present options.
+ * #13, #36: Support for Borg --exclude-from, --exclude-caches, and --exclude-if-present options.
 
 1.1.5
 
- * #34: New "extract" consistency check that performs a dry-run extraction of the most recent
+ * #35: New "extract" consistency check that performs a dry-run extraction of the most recent
    archive.
 
 1.1.4
 
- * #17: Added command-line flags for performing a borgmatic run with only pruning, creating, or
+ * #18: Added command-line flags for performing a borgmatic run with only pruning, creating, or
    checking enabled. This supports use cases like running consistency checks from a different cron
    job with a different frequency, or running pruning with a different verbosity level.
 
 1.1.3
 
- * #14: Support for running multiple config files in /etc/borgmatic.d/ from a single borgmatic run.
+ * #15: Support for running multiple config files in /etc/borgmatic.d/ from a single borgmatic run.
  * Fix for generate-borgmatic-config writing config with invalid one_file_system value.
 
 1.1.2
 
- * #32: Fix for passing check_last as integer to subprocess when calling Borg.
+ * #33: Fix for passing check_last as integer to subprocess when calling Borg.
 
 1.1.1
 
- * Part of #32: Fix for upgrade-borgmatic-config converting check_last option as a string instead of
+ * Part of #33: Fix for upgrade-borgmatic-config converting check_last option as a string instead of
    an integer.
  * Fix for upgrade-borgmatic-config erroring when consistency checks option is not present.
 
@@ -62,8 +85,8 @@
  * Switched config file format to YAML. Run upgrade-borgmatic-config to upgrade.
  * Added generate-borgmatic-config command for initial config creation.
  * Dropped Python 2 support. Now Python 3 only.
- * #18: Fix for README mention of sample files not included in package.
- * #22: Sample files for triggering borgmatic from a systemd timer.
+ * #19: Fix for README mention of sample files not included in package.
+ * #23: Sample files for triggering borgmatic from a systemd timer.
  * Support for backing up to multiple repositories.
  * To free up space, now pruning backups prior to creating a new backup.
  * Enabled test coverage output during tox runs.
@@ -71,15 +94,15 @@
 
 1.0.3
 
- * #21: Fix for verbosity flag not actually causing verbose output.
+ * #22: Fix for verbosity flag not actually causing verbose output.
 
 1.0.2
 
- * #20: Fix for traceback when remote_path option is missing.
+ * #21: Fix for traceback when remote_path option is missing.
 
 1.0.1
 
- * #19: Support for Borg's --remote-path option to use an alternate Borg
+ * #20: Support for Borg's --remote-path option to use an alternate Borg
    executable. See sample/config.
 
 1.0.0
@@ -101,13 +124,13 @@
 
 0.1.7
 
- * #11: Fixed parsing of punctuation in configuration file.
+ * #12: Fixed parsing of punctuation in configuration file.
  * Better error message when configuration file is missing.
 
 0.1.6
 
- * #9: New configuration option for the encryption passphrase.
- * #10: Support for Borg's new archive compression feature.
+ * #10: New configuration option for the encryption passphrase.
+ * #11: Support for Borg's new archive compression feature.
 
 0.1.5
 
@@ -119,7 +142,7 @@
 
 0.1.3
 
- * #1: Add support for "borg check --last N" to Borg backend.
+ * #2: Add support for "borg check --last N" to Borg backend.
 
 0.1.2
 

README.md

@@ -1,4 +1,4 @@
-<img src="https://projects.torsion.org/witten/borgmatic/raw/master/static/borgmatic.png" width="150px" style="float: right; padding-left: 1em;">
+<img src="https://projects.torsion.org/witten/borgmatic/raw/branch/master/static/borgmatic.png" width="150px" style="float: right; padding-left: 1em;">
 
 ## Overview
 
@@ -49,11 +49,13 @@ on [GitHub](https://github.com/witten/borgmatic) for convenience.
 
 To get up and running, follow the [Borg Quick
 Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) to create
-a repository on a local or remote host. Note that if you plan to run
-borgmatic on a schedule with cron, and you encrypt your Borg repository with
-a passphrase instead of a key file, you'll need to set the borgmatic
-`encryption_passphrase` configuration variable. See the repository encryption
-section of the Quick Start for more info.
+a repository on a local or remote host. Note that if you plan to run borgmatic
+on a schedule with cron, and you encrypt your Borg repository with a
+passphrase instead of a key file, you'll either need to set the borgmatic
+`encryption_passphrase` configuration variable or set the `BORG_PASSPHRASE`
+environment variable. See the [repository encryption
+section](https://borgbackup.readthedocs.io/en/latest/quickstart.html#repository-encryption)
+of the Quick Start for more info.
 
 If the repository is on a remote host, make sure that your local root user has
 key-based ssh access to the desired user account on the remote host.
@@ -338,8 +340,9 @@ it.
 ## Issues and feedback
 
 Got an issue or an idea for a feature enhancement? Check out the [borgmatic
-issue tracker](https://tree.taiga.io/project/witten-borgmatic/issues?page=1&status=399951,399952,399955). In
-order to create a new issue or comment on an issue, you'll need to [login
-first](https://tree.taiga.io/login).
+issue tracker](https://projects.torsion.org/witten/borgmatic/issues). In order
+to create a new issue or comment on an issue, you'll need to [login
+first](https://projects.torsion.org/user/login). Note that you can login with
+an existing GitHub account if you prefer.
 
 Other questions or comments? Contact <mailto:witten@torsion.org>.

borgmatic/borg/check.py

@@ -46,10 +46,12 @@ def _make_check_flags(checks, check_last=None):
 
         ('--repository-only',)
 
-    Additionally, if a check_last value is given, a "--last" flag will be added.
+    However, if both "repository" and "archives" are in checks, then omit them from the returned
+    flags because Borg does both checks by default. Additionally, if a check_last value is given,
+    a "--last" flag will be added.
     '''
     last_flag = ('--last', str(check_last)) if check_last else ()
-    if checks == DEFAULT_CHECKS:
+    if set(DEFAULT_CHECKS).issubset(set(checks)):
         return last_flag
 
     return tuple(
@@ -58,10 +60,10 @@ def _make_check_flags(checks, check_last=None):
     ) + last_flag
 
 
-def check_archives(verbosity, repository, consistency_config, remote_path=None):
+def check_archives(verbosity, repository, consistency_config, local_path='borg', remote_path=None):
     '''
     Given a verbosity flag, a local or remote repository path, a consistency config dict, and a
-    command to run, check the contained Borg archives for consistency.
+    local/remote commands to run, check the contained Borg archives for consistency.
 
     If there are no consistency checks to run, skip running them.
     '''
@@ -76,7 +78,7 @@ def check_archives(verbosity, repository, consistency_config, remote_path=None):
         }.get(verbosity, ())
 
         full_command = (
-            'borg', 'check',
+            local_path, 'check',
             repository,
         ) + _make_check_flags(checks, check_last) + remote_path_flags + verbosity_flags
 
@@ -87,4 +89,4 @@ def check_archives(verbosity, repository, consistency_config, remote_path=None):
         subprocess.check_call(full_command, stdout=stdout, stderr=subprocess.STDOUT)
 
     if 'extract' in checks:
-        extract.extract_last_archive_dry_run(verbosity, repository, remote_path)
+        extract.extract_last_archive_dry_run(verbosity, repository, local_path, remote_path)

borgmatic/borg/create.py

@@ -11,12 +11,15 @@ from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
 logger = logging.getLogger(__name__)
 
 
-def initialize(storage_config):
+def initialize_environment(storage_config):
     passphrase = storage_config.get('encryption_passphrase')
-
     if passphrase:
         os.environ['BORG_PASSPHRASE'] = passphrase
 
+    ssh_command = storage_config.get('ssh_command')
+    if ssh_command:
+        os.environ['BORG_RSH'] = ssh_command
+
 
 def _expand_directory(directory):
     '''
@@ -28,28 +31,45 @@ def _expand_directory(directory):
     return glob.glob(expanded_directory) or [expanded_directory]
 
 
-def _write_exclude_file(exclude_patterns=None):
+def _write_pattern_file(patterns=None):
     '''
-    Given a sequence of exclude patterns, write them to a named temporary file and return it. Return
-    None if no patterns are provided.
+    Given a sequence of patterns, write them to a named temporary file and return it. Return None
+    if no patterns are provided.
     '''
-    if not exclude_patterns:
+    if not patterns:
         return None
 
-    exclude_file = tempfile.NamedTemporaryFile('w')
-    exclude_file.write('\n'.join(exclude_patterns))
-    exclude_file.flush()
+    pattern_file = tempfile.NamedTemporaryFile('w')
+    pattern_file.write('\n'.join(patterns))
+    pattern_file.flush()
 
-    return exclude_file
+    return pattern_file
 
 
-def _make_exclude_flags(location_config, exclude_patterns_filename=None):
+def _make_pattern_flags(location_config, pattern_filename=None):
+    '''
+    Given a location config dict with a potential pattern_from option, and a filename containing any
+    additional patterns, return the corresponding Borg flags for those files as a tuple.
+    '''
+    pattern_filenames = tuple(location_config.get('patterns_from') or ()) + (
+        (pattern_filename,) if pattern_filename else ()
+    )
+
+    return tuple(
+        itertools.chain.from_iterable(
+            ('--patterns-from', pattern_filename)
+            for pattern_filename in pattern_filenames
+        )
+    )
+
+
+def _make_exclude_flags(location_config, exclude_filename=None):
     '''
     Given a location config dict with various exclude options, and a filename containing any exclude
     patterns, return the corresponding Borg flags as a tuple.
     '''
     exclude_filenames = tuple(location_config.get('exclude_from') or ()) + (
-        (exclude_patterns_filename,) if exclude_patterns_filename else ()
+        (exclude_filename,) if exclude_filename else ()
     )
     exclude_from_flags = tuple(
         itertools.chain.from_iterable(
@@ -65,11 +85,11 @@ def _make_exclude_flags(location_config, exclude_patterns_filename=None):
 
 
 def create_archive(
-    verbosity, repository, location_config, storage_config,
+    verbosity, dry_run, repository, location_config, storage_config, local_path='borg', remote_path=None,
 ):
     '''
-    Given a vebosity flag, a local or remote repository path, a location config dict, and a storage
-    config dict, create a Borg archive.
+    Given vebosity/dry-run flags, a local or remote repository path, a location config dict, and a
+    storage config dict, create a Borg archive.
     '''
     sources = tuple(
         itertools.chain.from_iterable(
@@ -78,10 +98,15 @@ def create_archive(
         )
     )
 
-    exclude_patterns_file = _write_exclude_file(location_config.get('exclude_patterns'))
+    pattern_file = _write_pattern_file(location_config.get('patterns'))
+    pattern_flags = _make_pattern_flags(
+        location_config,
+        pattern_file.name if pattern_file else None,
+    )
+    exclude_file = _write_pattern_file(location_config.get('exclude_patterns'))
     exclude_flags = _make_exclude_flags(
         location_config,
-        exclude_patterns_file.name if exclude_patterns_file else None,
+        exclude_file.name if exclude_file else None,
     )
     compression = storage_config.get('compression', None)
     compression_flags = ('--compression', compression) if compression else ()
@@ -92,24 +117,24 @@ def create_archive(
     one_file_system_flags = ('--one-file-system',) if location_config.get('one_file_system') else ()
     files_cache = location_config.get('files_cache')
     files_cache_flags = ('--files-cache', files_cache) if files_cache else ()
-    remote_path = location_config.get('remote_path')
     remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
     verbosity_flags = {
         VERBOSITY_SOME: ('--info', '--stats',),
         VERBOSITY_LOTS: ('--debug', '--list', '--stats'),
     }.get(verbosity, ())
+    dry_run_flags = ('--dry-run',) if dry_run else ()
     default_archive_name_format = '{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}'
     archive_name_format = storage_config.get('archive_name_format', default_archive_name_format)
 
     full_command = (
-        'borg', 'create',
+        local_path, 'create',
         '{repository}::{archive_name_format}'.format(
             repository=repository,
             archive_name_format=archive_name_format,
         ),
-    ) + sources + exclude_flags + compression_flags + remote_rate_limit_flags + \
+    ) + sources + pattern_flags + exclude_flags + compression_flags + remote_rate_limit_flags + \
         one_file_system_flags + files_cache_flags + remote_path_flags + umask_flags + \
-        verbosity_flags
+        verbosity_flags + dry_run_flags
 
     logger.debug(' '.join(full_command))
     subprocess.check_call(full_command)

borgmatic/borg/extract.py

@@ -8,7 +8,7 @@ from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
 logger = logging.getLogger(__name__)
 
 
-def extract_last_archive_dry_run(verbosity, repository, remote_path=None):
+def extract_last_archive_dry_run(verbosity, repository, local_path='borg', remote_path=None):
     '''
     Perform an extraction dry-run of just the most recent archive. If there are no archives, skip
     the dry-run.
@@ -20,7 +20,7 @@ def extract_last_archive_dry_run(verbosity, repository, remote_path=None):
     }.get(verbosity, ())
 
     full_list_command = (
-        'borg', 'list',
+        local_path, 'list',
         '--short',
         repository,
     ) + remote_path_flags + verbosity_flags
@@ -33,7 +33,7 @@ def extract_last_archive_dry_run(verbosity, repository, remote_path=None):
 
     list_flag = ('--list',) if verbosity == VERBOSITY_LOTS else ()
     full_extract_command = (
-        'borg', 'extract',
+        local_path, 'extract',
         '--dry-run',
         '{repository}::{last_archive_name}'.format(
             repository=repository,

borgmatic/borg/prune.py

@@ -32,25 +32,26 @@ def _make_prune_flags(retention_config):
     )
 
 
-def prune_archives(verbosity, repository, retention_config, remote_path=None):
+def prune_archives(verbosity, dry_run, repository, retention_config, local_path='borg', remote_path=None):
     '''
-    Given a verbosity flag, a local or remote repository path, a retention config dict, prune Borg
-    archives according the the retention policy specified in that configuration.
+    Given verbosity/dry-run flags, a local or remote repository path, a retention config dict, prune
+    Borg archives according the the retention policy specified in that configuration.
     '''
     remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
     verbosity_flags = {
         VERBOSITY_SOME: ('--info', '--stats',),
         VERBOSITY_LOTS: ('--debug', '--stats', '--list'),
     }.get(verbosity, ())
+    dry_run_flags = ('--dry-run',) if dry_run else ()
 
     full_command = (
-        'borg', 'prune',
+        local_path, 'prune',
         repository,
     ) + tuple(
         element
         for pair in _make_prune_flags(retention_config)
         for element in pair
-    ) + remote_path_flags + verbosity_flags
+    ) + remote_path_flags + verbosity_flags + dry_run_flags
 
     logger.debug(' '.join(full_command))
     subprocess.check_call(full_command)

borgmatic/commands/borgmatic.py

@@ -61,6 +61,12 @@ def parse_arguments(*arguments):
         action='store_true',
         help='Check archives for consistency',
     )
+    parser.add_argument(
+        '-n', '--dry-run',
+        dest='dry_run',
+        action='store_true',
+        help='Go through the motions, but do not actually write any changes to the repository',
+    )
     parser.add_argument(
         '-v', '--verbosity',
         type=int,
@@ -93,26 +99,44 @@ def run_configuration(config_filename, args):  # pragma: no cover
     )
 
     try:
+        local_path = location.get('local_path', 'borg')
         remote_path = location.get('remote_path')
-        create.initialize(storage)
+        create.initialize_environment(storage)
         hook.execute_hook(hooks.get('before_backup'), config_filename, 'pre-backup')
 
         for unexpanded_repository in location['repositories']:
             repository = os.path.expanduser(unexpanded_repository)
+            dry_run_label = ' (dry run; not making any changes)' if args.dry_run else ''
             if args.prune:
-                logger.info('{}: Pruning archives'.format(repository))
-                prune.prune_archives(args.verbosity, repository, retention, remote_path=remote_path)
+                logger.info('{}: Pruning archives{}'.format(repository, dry_run_label))
+                prune.prune_archives(
+                    args.verbosity,
+                    args.dry_run,
+                    repository,
+                    retention,
+                    local_path=local_path,
+                    remote_path=remote_path,
+                )
             if args.create:
-                logger.info('{}: Creating archive'.format(repository))
+                logger.info('{}: Creating archive{}'.format(repository, dry_run_label))
                 create.create_archive(
                     args.verbosity,
+                    args.dry_run,
                     repository,
                     location,
                     storage,
+                    local_path=local_path,
+                    remote_path=remote_path,
                 )
             if args.check:
                 logger.info('{}: Running consistency checks'.format(repository))
-                check.check_archives(args.verbosity, repository, consistency, remote_path=remote_path)
+                check.check_archives(
+                    args.verbosity,
+                    repository,
+                    consistency,
+                    local_path=local_path,
+                    remote_path=remote_path,
+                )
 
         hook.execute_hook(hooks.get('after_backup'), config_filename, 'post-backup')
     except (OSError, CalledProcessError):

borgmatic/config/collect.py

@@ -11,13 +11,15 @@ def collect_config_filenames(config_paths):
     files. This is non-recursive, so any directories within the given directories are ignored.
 
     Return paths even if they don't exist on disk, so the user can find out about missing
-    configuration paths. However, skip /etc/borgmatic.d if it's missing, so the user doesn't have to
-    create it unless they need it.
+    configuration paths. However, skip a default config path if it's missing, so the user doesn't
+    have to create a default config path unless they need it.
     '''
+    real_default_config_paths = set(map(os.path.realpath, DEFAULT_CONFIG_PATHS))
+
     for path in config_paths:
         exists = os.path.exists(path)
 
-        if os.path.realpath(path) in DEFAULT_CONFIG_PATHS and not exists:
+        if os.path.realpath(path) in real_default_config_paths and not exists:
             continue
 
         if not os.path.isdir(path) or not exists:

borgmatic/config/schema.yaml

@@ -29,6 +29,10 @@ map:
                     https://borgbackup.readthedocs.io/en/stable/usage/create.html#description for
                     details.
                 example: ctime,size,inode
+            local_path:
+                type: scalar
+                desc: Alternate Borg local executable. Defaults to "borg".
+                example: borg1
             remote_path:
                 type: scalar
                 desc: Alternate Borg remote executable. Defaults to "borg".
@@ -42,6 +46,28 @@ map:
                     repositories are backed up to in sequence.
                 example:
                     - user@backupserver:sourcehostname.borg
+            patterns:
+                seq:
+                    - type: scalar
+                desc: |
+                    Any paths matching these patterns are included/excluded from backups. Globs are
+                    expanded. Note that Borg considers this option experimental. See the output of
+                    "borg help patterns" for more details. Quoting any value if it contains leading
+                    punctuation, so it parses correctly.
+                example:
+                    - 'R /'
+                    - '- /home/*/.cache'
+                    - '+ /home/susan'
+                    - '- /home/*'
+            patterns_from:
+                seq:
+                    - type: scalar
+                desc: |
+                    Read include/exclude patterns from one or more separate named files, one pattern
+                    per line. Note that Borg considers this option experimental. See the output of
+                    "borg help patterns" for more details.
+                example:
+                    - /etc/borgmatic/patterns
             exclude_patterns:
                 seq:
                     - type: scalar
@@ -57,7 +83,7 @@ map:
                     - type: scalar
                 desc: |
                     Read exclude patterns from one or more separate named files, one pattern per
-                    line.
+                    line. See the output of "borg help patterns" for more details.
                 example:
                     - /etc/borgmatic/excludes
             exclude_caches:
@@ -96,6 +122,10 @@ map:
                 type: int
                 desc: Remote network upload rate limit in kiBytes/second.
                 example: 100
+            ssh_command:
+                type: scalar
+                desc: Command to use instead of just "ssh". This can be used to specify ssh options.
+                example: ssh -i /path/to/private/key
             umask:
                 type: scalar
                 desc: Umask to be used for borg create.
@@ -118,6 +148,10 @@ map:
                 type: scalar
                 desc: Keep all archives within this time interval.
                 example: 3H
+            keep_minutely:
+                type: int
+                desc: Number of minutely archives to keep.
+                example: 60
             keep_hourly:
                 type: int
                 desc: Number of hourly archives to keep.

borgmatic/tests/integration/config/test_validate.py

@@ -44,6 +44,8 @@ def test_parse_configuration_transforms_file_into_mapping():
                 - hostname.borg
 
         retention:
+            keep_minutely: 60
+            keep_hourly: 24
             keep_daily: 7
 
         consistency:
@@ -57,7 +59,7 @@ def test_parse_configuration_transforms_file_into_mapping():
 
     assert result == {
         'location': {'source_directories': ['/home', '/etc'], 'repositories': ['hostname.borg']},
-        'retention': {'keep_daily': 7},
+        'retention': {'keep_daily': 7, 'keep_hourly': 24, 'keep_minutely': 60},
         'consistency': {'checks': ['repository', 'archives']},
     }
 

borgmatic/tests/unit/borg/test_check.py

@@ -60,6 +60,12 @@ def test_make_check_flags_with_default_checks_returns_no_flags():
     assert flags == ()
 
 
+def test_make_check_flags_with_all_checks_returns_no_flags():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS + ('extract',))
+
+    assert flags == ()
+
+
 def test_make_check_flags_with_checks_and_last_returns_flags_including_last():
     flags = module._make_check_flags(('repository',), check_last=3)
 
@@ -81,7 +87,7 @@ def test_make_check_flags_with_default_checks_and_last_returns_last_flag():
         ('repository', 'archives', 'other'),
     ),
 )
-def test_check_archives_should_call_borg_with_parameters(checks):
+def test_check_archives_calls_borg_with_parameters(checks):
     check_last = flexmock()
     consistency_config = flexmock().should_receive('get').and_return(check_last).mock
     flexmock(module).should_receive('_parse_checks').and_return(checks)
@@ -101,7 +107,7 @@ def test_check_archives_should_call_borg_with_parameters(checks):
     )
 
 
-def test_check_archives_with_extract_check_should_call_extract_only():
+def test_check_archives_with_extract_check_calls_extract_only():
     checks = ('extract',)
     check_last = flexmock()
     consistency_config = flexmock().should_receive('get').and_return(check_last).mock
@@ -117,7 +123,7 @@ def test_check_archives_with_extract_check_should_call_extract_only():
     )
 
 
-def test_check_archives_with_verbosity_some_should_call_borg_with_info_parameter():
+def test_check_archives_with_verbosity_some_calls_borg_with_info_parameter():
     checks = ('repository',)
     consistency_config = flexmock().should_receive('get').and_return(None).mock
     flexmock(module).should_receive('_parse_checks').and_return(checks)
@@ -134,7 +140,7 @@ def test_check_archives_with_verbosity_some_should_call_borg_with_info_parameter
     )
 
 
-def test_check_archives_with_verbosity_lots_should_call_borg_with_debug_parameter():
+def test_check_archives_with_verbosity_lots_calls_borg_with_debug_parameter():
     checks = ('repository',)
     consistency_config = flexmock().should_receive('get').and_return(None).mock
     flexmock(module).should_receive('_parse_checks').and_return(checks)
@@ -151,7 +157,7 @@ def test_check_archives_with_verbosity_lots_should_call_borg_with_debug_paramete
     )
 
 
-def test_check_archives_without_any_checks_should_bail():
+def test_check_archives_without_any_checks_bails():
     consistency_config = flexmock().should_receive('get').and_return(None).mock
     flexmock(module).should_receive('_parse_checks').and_return(())
     insert_subprocess_never()
@@ -163,7 +169,29 @@ def test_check_archives_without_any_checks_should_bail():
     )
 
 
-def test_check_archives_with_remote_path_should_call_borg_with_remote_path_parameters():
+def test_check_archives_with_local_path_calls_borg_via_local_path():
+    checks = ('repository',)
+    check_last = flexmock()
+    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(
+        ('borg1', 'check', 'repo'),
+        stdout=stdout, stderr=STDOUT,
+    )
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        verbosity=None,
+        repository='repo',
+        consistency_config=consistency_config,
+        local_path='borg1',
+    )
+
+
+def test_check_archives_with_remote_path_calls_borg_with_remote_path_parameters():
     checks = ('repository',)
     check_last = flexmock()
     consistency_config = flexmock().should_receive('get').and_return(check_last).mock

borgmatic/tests/unit/borg/test_create.py

@@ -6,24 +6,36 @@ from borgmatic.borg import create as module
 from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
 
 
-def test_initialize_with_passphrase_should_set_environment():
+def test_initialize_environment_with_passphrase_should_set_environment():
     orig_environ = os.environ
 
     try:
         os.environ = {}
-        module.initialize({'encryption_passphrase': 'pass'})
+        module.initialize_environment({'encryption_passphrase': 'pass'})
         assert os.environ.get('BORG_PASSPHRASE') == 'pass'
     finally:
         os.environ = orig_environ
 
 
-def test_initialize_without_passphrase_should_not_set_environment():
+def test_initialize_environment_with_ssh_command_should_set_environment():
     orig_environ = os.environ
 
     try:
         os.environ = {}
-        module.initialize({})
+        module.initialize_environment({'ssh_command': 'ssh -C'})
+        assert os.environ.get('BORG_RSH') == 'ssh -C'
+    finally:
+        os.environ = orig_environ
+
+
+def test_initialize_environment_without_configuration_should_not_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize_environment({})
         assert os.environ.get('BORG_PASSPHRASE') == None
+        assert os.environ.get('BORG_RSH') == None
     finally:
         os.environ = orig_environ
 
@@ -46,7 +58,7 @@ def test_expand_directory_with_glob_expands():
     assert paths == ['foo', 'food']
 
 
-def test_write_exclude_file_does_not_raise():
+def test_write_pattern_file_does_not_raise():
     temporary_file = flexmock(
         name='filename',
         write=lambda mode: None,
@@ -54,11 +66,11 @@ def test_write_exclude_file_does_not_raise():
     )
     flexmock(module.tempfile).should_receive('NamedTemporaryFile').and_return(temporary_file)
 
-    module._write_exclude_file(['exclude'])
+    module._write_pattern_file(['exclude'])
 
 
-def test_write_exclude_file_with_empty_exclude_patterns_does_not_raise():
-    module._write_exclude_file([])
+def test_write_pattern_file_with_empty_exclude_patterns_does_not_raise():
+    module._write_pattern_file([])
 
 
 def insert_subprocess_mock(check_call_command, **kwargs):
@@ -66,17 +78,50 @@ def insert_subprocess_mock(check_call_command, **kwargs):
     subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
 
 
+def test_make_pattern_flags_includes_pattern_filename_when_given():
+    pattern_flags = module._make_pattern_flags(
+        location_config={'patterns': ['R /', '- /var']},
+        pattern_filename='/tmp/patterns',
+    )
+
+    assert pattern_flags == ('--patterns-from', '/tmp/patterns')
+
+
+def test_make_pattern_flags_includes_patterns_from_filenames_when_in_config():
+    pattern_flags = module._make_pattern_flags(
+        location_config={'patterns_from': ['patterns', 'other']},
+    )
+
+    assert pattern_flags == ('--patterns-from', 'patterns', '--patterns-from', 'other')
+
+
+def test_make_pattern_flags_includes_both_filenames_when_patterns_given_and_patterns_from_in_config():
+    pattern_flags = module._make_pattern_flags(
+        location_config={'patterns_from': ['patterns']},
+        pattern_filename='/tmp/patterns',
+    )
+
+    assert pattern_flags == ('--patterns-from', 'patterns', '--patterns-from', '/tmp/patterns')
+
+
+def test_make_pattern_flags_considers_none_patterns_from_filenames_as_empty():
+    pattern_flags = module._make_pattern_flags(
+        location_config={'patterns_from': None},
+    )
+
+    assert pattern_flags == ()
+
+
 def test_make_exclude_flags_includes_exclude_patterns_filename_when_given():
     exclude_flags = module._make_exclude_flags(
         location_config={'exclude_patterns': ['*.pyc', '/var']},
-        exclude_patterns_filename='/tmp/excludes',
+        exclude_filename='/tmp/excludes',
     )
 
     assert exclude_flags == ('--exclude-from', '/tmp/excludes')
 
 
 def test_make_exclude_flags_includes_exclude_from_filenames_when_in_config():
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
 
     exclude_flags = module._make_exclude_flags(
         location_config={'exclude_from': ['excludes', 'other']},
@@ -86,19 +131,15 @@ def test_make_exclude_flags_includes_exclude_from_filenames_when_in_config():
 
 
 def test_make_exclude_flags_includes_both_filenames_when_patterns_given_and_exclude_from_in_config():
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-
     exclude_flags = module._make_exclude_flags(
         location_config={'exclude_from': ['excludes']},
-        exclude_patterns_filename='/tmp/excludes',
+        exclude_filename='/tmp/excludes',
     )
 
     assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', '/tmp/excludes')
 
 
 def test_make_exclude_flags_considers_none_exclude_from_filenames_as_empty():
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-
     exclude_flags = module._make_exclude_flags(
         location_config={'exclude_from': None},
     )
@@ -142,12 +183,14 @@ CREATE_COMMAND = ('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'fo
 
 def test_create_archive_calls_borg_with_parameters():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND)
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -158,15 +201,38 @@ def test_create_archive_calls_borg_with_parameters():
     )
 
 
+def test_create_archive_with_patterns_calls_borg_with_patterns():
+    pattern_flags = ('--patterns-from', 'patterns')
+    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
+    flexmock(module).should_receive('_write_pattern_file').and_return(flexmock(name='/tmp/patterns')).and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(pattern_flags)
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + pattern_flags)
+
+    module.create_archive(
+        verbosity=None,
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'patterns': ['pattern'],
+        },
+        storage_config={},
+    )
+
+
 def test_create_archive_with_exclude_patterns_calls_borg_with_excludes():
     exclude_flags = ('--exclude-from', 'excludes')
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(flexmock(name='/tmp/excludes'))
+    flexmock(module).should_receive('_write_pattern_file').and_return(None).and_return(flexmock(name='/tmp/excludes'))
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(exclude_flags)
     insert_subprocess_mock(CREATE_COMMAND + exclude_flags)
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -179,12 +245,15 @@ def test_create_archive_with_exclude_patterns_calls_borg_with_excludes():
 
 def test_create_archive_with_verbosity_some_calls_borg_with_info_parameter():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND + ('--info', '--stats',))
 
     module.create_archive(
         verbosity=VERBOSITY_SOME,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -197,12 +266,35 @@ def test_create_archive_with_verbosity_some_calls_borg_with_info_parameter():
 
 def test_create_archive_with_verbosity_lots_calls_borg_with_debug_parameter():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND + ('--debug', '--list', '--stats'))
 
     module.create_archive(
         verbosity=VERBOSITY_LOTS,
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_dry_run_calls_borg_with_dry_run_parameter():
+    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--dry-run',))
+
+    module.create_archive(
+        verbosity=None,
+        dry_run=True,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -215,12 +307,14 @@ def test_create_archive_with_verbosity_lots_calls_borg_with_debug_parameter():
 
 def test_create_archive_with_compression_calls_borg_with_compression_parameters():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND + ('--compression', 'rle'))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -233,12 +327,14 @@ def test_create_archive_with_compression_calls_borg_with_compression_parameters(
 
 def test_create_archive_with_remote_rate_limit_calls_borg_with_remote_ratelimit_parameters():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND + ('--remote-ratelimit', '100'))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -251,12 +347,14 @@ def test_create_archive_with_remote_rate_limit_calls_borg_with_remote_ratelimit_
 
 def test_create_archive_with_one_file_system_calls_borg_with_one_file_system_parameters():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND + ('--one-file-system',))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -270,12 +368,14 @@ def test_create_archive_with_one_file_system_calls_borg_with_one_file_system_par
 
 def test_create_archive_with_files_cache_calls_borg_with_files_cache_parameters():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND + ('--files-cache', 'ctime,size'))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -287,33 +387,58 @@ def test_create_archive_with_files_cache_calls_borg_with_files_cache_parameters(
     )
 
 
+def test_create_archive_with_local_path_calls_borg_via_local_path():
+    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(('borg1',) + CREATE_COMMAND[1:])
+
+    module.create_archive(
+        verbosity=None,
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        local_path='borg1',
+    )
+
+
 def test_create_archive_with_remote_path_calls_borg_with_remote_path_parameters():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND + ('--remote-path', 'borg1'))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
             'repositories': ['repo'],
-            'remote_path': 'borg1',
             'exclude_patterns': None,
         },
         storage_config={},
+        remote_path='borg1',
     )
 
 
 def test_create_archive_with_umask_calls_borg_with_umask_parameters():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(CREATE_COMMAND + ('--umask', '740'))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -326,13 +451,15 @@ def test_create_archive_with_umask_calls_borg_with_umask_parameters():
 
 def test_create_archive_with_source_directories_glob_expands():
     flexmock(module).should_receive('_expand_directory').and_return(['foo', 'food'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food'))
     flexmock(module.glob).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo*'],
@@ -345,13 +472,15 @@ def test_create_archive_with_source_directories_glob_expands():
 
 def test_create_archive_with_non_matching_source_directories_glob_passes_through():
     flexmock(module).should_receive('_expand_directory').and_return(['foo*'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo*'))
     flexmock(module.glob).should_receive('glob').with_args('foo*').and_return([])
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo*'],
@@ -364,12 +493,14 @@ def test_create_archive_with_non_matching_source_directories_glob_passes_through
 
 def test_create_archive_with_glob_calls_borg_with_expanded_directories():
     flexmock(module).should_receive('_expand_directory').and_return(['foo', 'food'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food'))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo*'],
@@ -382,12 +513,14 @@ def test_create_archive_with_glob_calls_borg_with_expanded_directories():
 
 def test_create_archive_with_archive_name_format_calls_borg_with_archive_name():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(('borg', 'create', 'repo::ARCHIVE_NAME', 'foo', 'bar'))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],
@@ -402,12 +535,14 @@ def test_create_archive_with_archive_name_format_calls_borg_with_archive_name():
 
 def test_create_archive_with_archive_name_format_accepts_borg_placeholders():
     flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
     flexmock(module).should_receive('_make_exclude_flags').and_return(())
     insert_subprocess_mock(('borg', 'create', 'repo::Documents_{hostname}-{now}', 'foo', 'bar'))
 
     module.create_archive(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         location_config={
             'source_directories': ['foo', 'bar'],

borgmatic/tests/unit/borg/test_extract.py

@@ -83,6 +83,23 @@ def test_extract_last_archive_dry_run_with_verbosity_lots_should_call_borg_with_
     )
 
 
+def test_extract_last_archive_dry_run_should_call_borg_via_local_path():
+    flexmock(sys.stdout).encoding = 'utf-8'
+    insert_subprocess_check_output_mock(
+        ('borg1', 'list', '--short', 'repo'),
+        result='archive1\narchive2\n'.encode('utf-8'),
+    )
+    insert_subprocess_mock(
+        ('borg1', 'extract', '--dry-run', 'repo::archive2'),
+    )
+
+    module.extract_last_archive_dry_run(
+        verbosity=None,
+        repository='repo',
+        local_path='borg1',
+    )
+
+
 def test_extract_last_archive_dry_run_should_call_borg_with_remote_path_parameters():
     flexmock(sys.stdout).encoding = 'utf-8'
     insert_subprocess_check_output_mock(

borgmatic/tests/unit/borg/test_prune.py

@@ -18,7 +18,7 @@ BASE_PRUNE_FLAGS = (
 )
 
 
-def test_make_prune_flags_should_return_flags_from_config_plus_default_prefix():
+def test_make_prune_flags_returns_flags_from_config_plus_default_prefix():
     retention_config = OrderedDict(
         (
             ('keep_daily', 1),
@@ -55,7 +55,7 @@ PRUNE_COMMAND = (
 )
 
 
-def test_prune_archives_should_call_borg_with_parameters():
+def test_prune_archives_calls_borg_with_parameters():
     retention_config = flexmock()
     flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
         BASE_PRUNE_FLAGS,
@@ -64,12 +64,13 @@ def test_prune_archives_should_call_borg_with_parameters():
 
     module.prune_archives(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         retention_config=retention_config,
     )
 
 
-def test_prune_archives_with_verbosity_some_should_call_borg_with_info_parameter():
+def test_prune_archives_with_verbosity_some_calls_borg_with_info_parameter():
     retention_config = flexmock()
     flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
         BASE_PRUNE_FLAGS,
@@ -79,11 +80,12 @@ def test_prune_archives_with_verbosity_some_should_call_borg_with_info_parameter
     module.prune_archives(
         repository='repo',
         verbosity=VERBOSITY_SOME,
+        dry_run=False,
         retention_config=retention_config,
     )
 
 
-def test_prune_archives_with_verbosity_lots_should_call_borg_with_debug_parameter():
+def test_prune_archives_with_verbosity_lots_calls_borg_with_debug_parameter():
     retention_config = flexmock()
     flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
         BASE_PRUNE_FLAGS,
@@ -93,11 +95,43 @@ def test_prune_archives_with_verbosity_lots_should_call_borg_with_debug_paramete
     module.prune_archives(
         repository='repo',
         verbosity=VERBOSITY_LOTS,
+        dry_run=False,
         retention_config=retention_config,
     )
 
 
-def test_prune_archives_with_remote_path_should_call_borg_with_remote_path_parameters():
+def test_prune_archives_with_dry_run_calls_borg_with_dry_run_parameter():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS,
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--dry-run',))
+
+    module.prune_archives(
+        repository='repo',
+        verbosity=None,
+        dry_run=True,
+        retention_config=retention_config,
+    )
+
+
+def test_prune_archives_with_local_path_calls_borg_via_local_path():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS,
+    )
+    insert_subprocess_mock(('borg1',) + PRUNE_COMMAND[1:])
+
+    module.prune_archives(
+        verbosity=None,
+        dry_run=False,
+        repository='repo',
+        retention_config=retention_config,
+        local_path='borg1',
+    )
+
+
+def test_prune_archives_with_remote_path_calls_borg_with_remote_path_parameters():
     retention_config = flexmock()
     flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
         BASE_PRUNE_FLAGS,
@@ -106,6 +140,7 @@ def test_prune_archives_with_remote_path_should_call_borg_with_remote_path_param
 
     module.prune_archives(
         verbosity=None,
+        dry_run=False,
         repository='repo',
         retention_config=retention_config,
         remote_path='borg1',

borgmatic/tests/unit/config/test_collect.py

@@ -58,6 +58,19 @@ def test_collect_config_filenames_skips_etc_borgmatic_dot_d_if_it_does_not_exist
     assert config_filenames == ('config.yaml',)
 
 
+def test_collect_config_filenames_skips_non_canonical_etc_borgmatic_dot_d_if_it_does_not_exist():
+    config_paths = ('config.yaml', '/etc/../etc/borgmatic.d')
+    mock_path = flexmock(module.os.path)
+    mock_path.should_receive('exists').with_args('config.yaml').and_return(True)
+    mock_path.should_receive('exists').with_args('/etc/../etc/borgmatic.d').and_return(False)
+    mock_path.should_receive('isdir').with_args('config.yaml').and_return(False)
+    mock_path.should_receive('isdir').with_args('/etc/../etc/borgmatic.d').and_return(True)
+
+    config_filenames = tuple(module.collect_config_filenames(config_paths))
+
+    assert config_filenames == ('config.yaml',)
+
+
 def test_collect_config_filenames_includes_other_directory_if_it_does_not_exist():
     config_paths = ('config.yaml', '/my/directory')
     mock_path = flexmock(module.os.path)

scripts/find-unsupported-borg-options

@@ -0,0 +1,47 @@
+#!/bin/bash
+
+# For each Borg sub-command that borgmatic uses, print out the Borg flags that borgmatic does not
+# appear to support yet. This script isn't terribly robust. It's intended as a basic tool to ferret
+# out unsupported Borg options so that they can be considered for addition to borgmatic.
+
+# Generate a sample borgmatic configuration with all options set.
+generate-borgmatic-config --destination temp.yaml
+
+# For each sub-command (prune, create, and check), collect the Borg command-line flags that result
+# from running borgmatic with the generated configuration. Then, collect the full set of available
+# Borg flags as reported by "borg --help" for that sub-command. Finally, compare the two lists of
+# flags to determine which Borg flags borgmatic doesn't yet support.
+for sub_command in prune create check; do
+    echo "********** borg $sub_command **********"
+
+    for line in $(borgmatic --config temp.yaml --$sub_command -v 2 2>&1 | grep "borg $sub_command") ; do
+        echo "$line" | grep '^-' >> borgmatic_borg_flags
+    done
+    sort borgmatic_borg_flags > borgmatic_borg_flags.sorted
+    mv borgmatic_borg_flags.sorted borgmatic_borg_flags
+
+    for line in $(borg $sub_command --help | awk -v RS= '/^usage:/') ; do
+        # Exclude a bunch of flags that borgmatic actually supports, but don't get exercised by the
+        # generated sample config, and also flags that don't make sense to support.
+        echo "$line" | grep -- -- | sed -r 's/(\[|\])//g' \
+            | grep -v '^-h$' \
+            | grep -v '^--archives-only$' \
+            | grep -v '^--repository-only$' \
+            | grep -v '^--stats$' \
+            | grep -v '^--list$' \
+            | grep -v '^--critical$' \
+            | grep -v '^--error$' \
+            | grep -v '^--warning$' \
+            | grep -v '^--info$' \
+            | grep -v '^--debug$' \
+            >> all_borg_flags
+    done
+    sort all_borg_flags > all_borg_flags.sorted
+    mv all_borg_flags.sorted all_borg_flags
+
+    comm -13 borgmatic_borg_flags all_borg_flags
+
+    rm ./*_borg_flags
+done
+
+rm temp.yaml

setup.py

@@ -1,7 +1,7 @@
 from setuptools import setup, find_packages
 
 
-VERSION = '1.1.10'
+VERSION = '1.1.14'
 
 
 setup(
@@ -32,7 +32,7 @@ setup(
         'atticmatic',
     ],
     install_requires=(
-        'pykwalify',
+        'pykwalify>=1.6.0',
         'ruamel.yaml<=0.15',
         'setuptools',
     ),