Bump version for release.

Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1274
Reviewed-by: Dan Helfman <witten@torsion.org>

.github/pull_request_template.md

@@ -1,10 +0,0 @@
-## Hold up, GitHub users
-
-Thanks for your contribution! 
-
-Unfortunately, we don't use GitHub pull requests to manage code contributions to this repository (and GitHub doesn't have any way to disable pull requests entirely). Instead, please see:
-
-https://torsion.org/borgmatic/#contributing
-
-... which provides full instructions on how to submit pull requests. You can even use your GitHub account to login.
-

AGENTS.md

@@ -0,0 +1,28 @@
+# AGENTS.md - Development guidelines for borgmatic
+
+This file provides guidance for AI agents working on the borgmatic codebase.
+
+## Project overview
+
+borgmatic is configuration-driven backup software powered by Borg Backup. It's a
+Python project using setuptools.
+
+Please do not use AI agents to modify this codebase. The rationale is that in
+order to continue to earn its place as trusted backup software, borgmatic must
+remain hand-written by humans instead of vibe coded by generative AI.
+
+Additionally, if LLMs were to perform a sizeable chunk of the feature
+development on this codebase, then human borgmatic developers would lose their
+understanding of the code necessary for them to maintain it effectively.
+
+Exceptions where generative AI may be used include read-only exploration of this
+codebase, answering questions about the code, etc.
+
+## Architecture notes
+
+- **main entry point**: `borgmatic.commands.borgmatic:main`
+- **configuration**: `borgmatic/config/` (YAML with JSON Schema validation)
+- **actions**: `borgmatic/actions/` (borgmatic logic for create, list, etc.)
+- **Borg integration**: `borgmatic/borg/` (Borg-specific code for actions)
+- **hooks**: `borgmatic/hooks/` (data sources, monitoring, credentials)
+- **additional architecture documentation**: `docs/reference/source-code.md`

NEWS

@@ -1,3 +1,54 @@
+2.1.3
+ * #1175: Add a "files_changed" option for customizing Borg's file modification detection.
+ * #1175: Add a "msgpack_version_check" option to prevent Borg from validating msgpack's version.
+ * #1218: Add a "config show" action to display computed borgmatic configuration as YAML or JSON,
+   handy for fetching borgmatic configuration from external scripts. See the documentation for more
+   information:
+   https://torsion.org/borgmatic/reference/command-line/actions/config-show/
+ * #1228: Adjust the "spot" check so error output includes more information about what failed.
+ * #1236: Fix the "spot" check to skip hard links, as Borg doesn't produces hashes for them.
+ * #1243: Add a "diff" action for viewing the difference between the contents of two archives.
+ * #1248: Go back to treating Borg "file not found" warnings (exit code 107) as
+   warnings instead of errors. Otherwise, borgmatic can error on files that a user intentionally
+   deletes while a backup is running. You can still override this behavior with the
+   "borg_exit_codes" option. See the documentation for more information:
+   https://torsion.org/borgmatic/how-to/customize-warnings-and-errors/
+ * #1248: Un-deprecate the "source_directories_must_exist" option and default it to true, to
+   compensate for Borg "file not found" warnings no longer being treated as errors.
+ * #1269: Fix the ZFS hook to support datasets with a "canmount" property of "noauto".
+ * #1270: Follow symlinks when backing up borgmatic configuration files to support the "bootstrap"
+   action.
+ * #1274: Add an optional override for the documentation development listen port and use Podman
+   Compose if present.
+ * #1281: Fix a unicode error when backing up a non-UTF-8 source filename with a
+   corresponding system locale.
+ * Add a policy about the use of generative AI in the borgmatic codebase:
+   https://torsion.org/borgmatic/how-to/develop-on-borgmatic/#use-of-generative-ai
+
+2.1.2
+ * #1231: If a source file is deleted during a "spot" check, consider the file as non-matching
+   and move on instead of immediately failing the entire check.
+ * #1250: Fix a regression in which the "--stats" flag hides statistics at default verbosity.
+ * #1251: Fix a regression in the ntfy monitoring hook in which borgmatic sends tags incorrectly,
+   resulting in "400 Bad Request" from ntfy.
+ * #1258: Fix a "codec can't decode byte" error when running commands that output multi-byte unicode
+   characters.
+ * #1260: Fix for SSH warnings from Borg showing up as JSON logs even without the "--log-json" flag.
+ * #1252: Work around Borg returning a warning exit code when a repository/archive check fails. Now,
+   borgmatic interprets such failures as errors.
+ * Deduplicate overlapping source directories and patterns so they don't throw off "spot" check file
+   counts and cause spurious check failures.
+
+2.1.1
+ * #1241: For the "recreate" action, actually pass the "--dry-run" flag through to Borg instead of
+   just skipping the Borg call.
+ * #1242: Fix a regression in which the "spot" check hung while collecting archive contents.
+ * #1244: When the "unsafe_skip_path_validation_before_create" option is enabled, don't log a
+   warning about it.
+ * #1245: Fix a regression in which the KeePassXC credential hook password prompt was invisible.
+ * #1246: Fix a regression in which the ntfy monitoring hook failed to send a ping when the
+   "priority" option was set.
+
 2.1.0
  * TL;DR: Many logging, memory, and performance improvements. Mind those breaking changes!
  * #485: When running commands (database clients, command hooks, etc.), elevate stderr output to

borgmatic/actions/check.py

@@ -9,6 +9,7 @@ import pathlib
 import random
 import shlex
 import shutil
+import subprocess
 import textwrap
 
 import borgmatic.actions.config.bootstrap
@@ -418,7 +419,13 @@ def collect_spot_check_source_paths(
     )
 
     return tuple(
-        path for path in paths if os.path.isfile(os.path.join(working_directory or '', path))
+        # Use dict.fromkeys() to deduplicate file paths, which are present in Borg's dry run output
+        # when there are overlapping source patterns. For instance, if both "/foo" and
+        # "/foo/file.txt" are in configured patterns, then "/foo/file.txt" will show up in Borg's
+        # dry run output twice.
+        dict.fromkeys(
+            path for path in paths if os.path.isfile(os.path.join(working_directory or '', path))
+        )
     )
 
 
@@ -518,66 +525,100 @@ def compare_spot_check_hashes(
         source_sample_paths_subset = tuple(
             itertools.islice(source_sample_paths_iterator, SAMPLE_PATHS_SUBSET_COUNT),
         )
+
         if not source_sample_paths_subset:
             break
 
         hash_paths = tuple(
             path for path in source_sample_paths_subset if path in hashable_source_sample_path
         )
-        hash_lines = borgmatic.execute.execute_command_and_capture_output(
-            tuple(
-                shlex.quote(part)
-                for part in shlex.split(spot_check_config.get('xxh64sum_command', 'xxh64sum'))
-            )
-            + hash_paths,
-            working_directory=working_directory,
-        )
 
-        source_hashes.update(
-            **dict(
-                zip(
-                    # xxh64sum rewrites/escapes the paths that it returns alongside its hashes, for
-                    # instance if they contain special characters. When that happens, they don't
-                    # match the original source paths and therefore hash lookups fail. So when
-                    # building this lookup dict, use the original unaltered paths we provided as
-                    # input to xxh64sum.
-                    hash_paths,
-                    (
-                        # For some reason, xxh64sum prefixes the hash with a backslash if the path
-                        # contains a newline. Work around that.
-                        line.split('  ', 1)[0].lstrip('\\')
-                        for line in hash_lines
+        try:
+            hash_lines = borgmatic.execute.execute_command_and_capture_output(
+                tuple(
+                    shlex.quote(part)
+                    for part in shlex.split(spot_check_config.get('xxh64sum_command', 'xxh64sum'))
+                )
+                + hash_paths,
+                working_directory=working_directory,
+            )
+            source_hashes.update(
+                **dict(
+                    zip(
+                        # xxh64sum rewrites/escapes the paths that it returns alongside its hashes, for
+                        # instance if they contain special characters. When that happens, they don't
+                        # match the original source paths and therefore hash lookups fail. So when
+                        # building this lookup dict, use the original unaltered paths we provided as
+                        # input to xxh64sum.
+                        hash_paths,
+                        (
+                            # For some reason, xxh64sum prefixes the hash with a backslash if the path
+                            # contains a newline. Work around that.
+                            line.split('  ', 1)[0].lstrip('\\')
+                            for line in hash_lines
+                        ),
                     ),
+                    # Represent non-existent files as having empty hashes so the comparison below still
+                    # works. Same thing for filesystem links, since Borg produces empty archive hashes
+                    # for them.
+                    **{
+                        path: ''
+                        for path in source_sample_paths_subset
+                        if path not in hashable_source_sample_path
+                    },
                 ),
-                # Represent non-existent files as having empty hashes so the comparison below still
-                # works. Same thing for filesystem links, since Borg produces empty archive hashes
-                # for them.
-                **{
-                    path: ''
-                    for path in source_sample_paths_subset
-                    if path not in hashable_source_sample_path
-                },
-            ),
-        )
+            )
+        except subprocess.CalledProcessError:
+            # This can happen if a file we planned to hash gets deleted right before we try to hash
+            # it. Falling back to individual file hashing allows us to find and mark just the
+            # file(s) with problems instead of failing the whole batch.
+            logger.warning(
+                'Bulk source path hashing failed for this batch; falling back to individual file hashing'
+            )
+
+            for hash_path in hash_paths:
+                try:
+                    hash_lines = borgmatic.execute.execute_command_and_capture_output(
+                        (
+                            *(
+                                shlex.quote(part)
+                                for part in shlex.split(
+                                    spot_check_config.get('xxh64sum_command', 'xxh64sum')
+                                )
+                            ),
+                            hash_path,
+                        ),
+                        working_directory=working_directory,
+                    )
+                    source_hashes[hash_path] = next(hash_lines).split('  ', 1)[0].lstrip('\\')
+                except (subprocess.CalledProcessError, StopIteration):  # noqa: PERF203
+                    logger.warning(
+                        f'Source path hashing failed for {hash_path}; treating as missing'
+                    )
+                    source_hashes[hash_path] = ''
 
         # Get the hash for each file in the archive.
-        archive_hashes.update(
-            **{
-                entry['path']: entry['xxh64']
-                for entry in borgmatic.borg.list.capture_archive_listing(
-                    repository['path'],
-                    archive,
-                    config,
-                    local_borg_version,
-                    global_arguments,
-                    list_paths=source_sample_paths_subset,
-                    path_format='{xxh64}{path}',
-                    local_path=local_path,
-                    remote_path=remote_path,
-                )
-                if entry
-            },
-        )
+        for entry in borgmatic.borg.list.capture_archive_listing(
+            repository['path'],
+            archive,
+            config,
+            local_borg_version,
+            global_arguments,
+            list_paths=source_sample_paths_subset,
+            path_format='{xxh64}{path}{linktarget}',
+            local_path=local_path,
+            remote_path=remote_path,
+        ):
+            if not entry:
+                continue
+
+            # Borg can't get hashes of stored hard links. So if this is a hard link path (and not
+            # deemed as the "original" by Borg), then skip hashing of it.
+            if entry['linktarget']:
+                source_hashes.pop(os.path.join('/', entry['path']), None)
+                continue
+
+            archive_hashes[entry['path']] = entry['xxh64']
 
     # Compare the source hashes with the archive hashes to see how many match.
     failing_paths = []
@@ -676,7 +717,7 @@ def spot_check(
         )
         logger.debug(f'Paths in latest archive but not source paths: {truncated_archive_paths}')
         raise ValueError(
-            'Spot check failed: There are no source paths to compare against the archive',
+            'Spot check failed; there are no source paths to compare against the archive',
         )
 
     # Calculate the percentage delta between the source paths count and the archive paths count, and
@@ -690,19 +731,13 @@ def spot_check(
             width=MAX_SPOT_CHECK_PATHS_LENGTH,
             placeholder=' ...',
         )
-        logger.debug(
-            f'Paths in source paths but not latest archive: {truncated_exclusive_source_paths}',
-        )
         truncated_exclusive_archive_paths = textwrap.shorten(
             ', '.join(set(archive_paths) - rootless_source_paths) or 'none',
             width=MAX_SPOT_CHECK_PATHS_LENGTH,
             placeholder=' ...',
         )
-        logger.debug(
-            f'Paths in latest archive but not source paths: {truncated_exclusive_archive_paths}',
-        )
         raise ValueError(
-            f'Spot check failed: {count_delta_percentage:.2f}% file count delta between source paths and latest archive (tolerance is {spot_check_config["count_tolerance_percentage"]}%)',
+            f'Spot check failed\n{count_delta_percentage:.2f}% file count delta between source paths ({len(source_paths)} total) and latest archive ({len(archive_paths)} total); tolerance is {spot_check_config["count_tolerance_percentage"]}%\nOnly in source paths: {truncated_exclusive_source_paths}\nOnly in latest archive: {truncated_exclusive_archive_paths}',
         )
 
     failing_paths = compare_spot_check_hashes(
@@ -727,11 +762,8 @@ def spot_check(
             width=MAX_SPOT_CHECK_PATHS_LENGTH,
             placeholder=' ...',
         )
-        logger.debug(
-            f'Source paths with data not matching the latest archive: {truncated_failing_paths}',
-        )
         raise ValueError(
-            f'Spot check failed: {failing_percentage:.2f}% of source paths with data not matching the latest archive (tolerance is {data_tolerance_percentage}%)',
+            f'Spot check failed\n{failing_percentage:.2f}% of source paths ({len(failing_paths)} out of {len(source_paths)} checked) with data not matching the latest archive; tolerance is {data_tolerance_percentage}%\nSource paths with non-matching data: {truncated_failing_paths}',
         )
 
     logger.info(

borgmatic/actions/config/bootstrap.py

@@ -120,7 +120,7 @@ def run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version):
             borgmatic_runtime_directory,
         )
 
-    logger.info(f"Bootstrapping config paths: {', '.join(manifest_config_paths)}")
+    logger.info(f"Bootstrapping configuration paths: {', '.join(manifest_config_paths)}")
 
     borgmatic.borg.extract.extract_archive(
         global_arguments.dry_run,

borgmatic/actions/config/show.py

@@ -0,0 +1,44 @@
+import json
+import logging
+import sys
+
+import borgmatic.config.generate
+import borgmatic.logger
+
+logger = logging.getLogger(__name__)
+
+
+def run_show(show_arguments, configs):
+    '''
+    Given the show arguments as an argparse.Namespace instance and a dict of configuration filename
+    to corresponding parsed configuration, run the "show" action. That consists of rendering and
+    logging the computed configuration as YAML, separating the configuration for each file with
+    "---".
+
+    If show_arguments.option is set, limit the results to the value of that single option. If
+    show_arguments.json is True, render the results as JSON with one array element per configuration
+    file.
+    '''
+    borgmatic.logger.add_custom_log_levels()
+
+    if show_arguments.json:
+        sys.stdout.write(
+            json.dumps(
+                [
+                    config.get(show_arguments.option) if show_arguments.option else config
+                    for config in configs.values()
+                ]
+            )
+        )
+
+        return
+
+    for config in configs.values():
+        if len(configs) > 1:
+            logger.answer('---')
+
+        logger.answer(
+            borgmatic.config.generate.render_configuration(
+                config.get(show_arguments.option) if show_arguments.option else config
+            ).rstrip()
+        )

borgmatic/actions/diff.py

@@ -0,0 +1,62 @@
+import logging
+
+import borgmatic.actions.pattern
+import borgmatic.borg.diff
+
+logger = logging.getLogger(__name__)
+
+
+def run_diff(
+    repository,
+    config,
+    local_borg_version,
+    diff_arguments,
+    global_arguments,
+    local_path,
+    remote_path,
+):
+    '''
+    Run the "diff" action for the given repository.
+    '''
+
+    # Only process patterns if only_patterns flag is set
+    if diff_arguments.only_patterns:
+        processed_patterns = borgmatic.actions.pattern.process_patterns(
+            (*borgmatic.actions.pattern.collect_patterns(config),),
+            config,
+            borgmatic.config.paths.get_working_directory(config),
+        )
+    else:
+        processed_patterns = None
+
+    archive = borgmatic.borg.repo_list.resolve_archive_name(
+        repository['path'],
+        diff_arguments.archive,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+    second_archive = borgmatic.borg.repo_list.resolve_archive_name(
+        repository['path'],
+        diff_arguments.second_archive,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+
+    borgmatic.borg.diff.diff(
+        repository['path'],
+        archive,
+        second_archive,
+        config,
+        local_borg_version,
+        diff_arguments,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+        patterns=processed_patterns,
+    )

borgmatic/actions/pattern.py

@@ -68,7 +68,7 @@ def collect_patterns(config):
             + tuple(
                 parse_pattern(pattern_line.strip())
                 for filename in config.get('patterns_from', ())
-                for pattern_line in open(filename, encoding='utf-8').readlines()
+                for pattern_line in open(filename, encoding='utf-8')
                 if not pattern_line.lstrip().startswith('#')
                 if pattern_line.strip()
             )
@@ -78,7 +78,7 @@ def collect_patterns(config):
                     borgmatic.borg.pattern.Pattern_style.FNMATCH,
                 )
                 for filename in config.get('exclude_from', ())
-                for exclude_line in open(filename, encoding='utf-8').readlines()
+                for exclude_line in open(filename, encoding='utf-8')
                 if not exclude_line.lstrip().startswith('#')
                 if exclude_line.strip()
             )

borgmatic/actions/recreate.py

@@ -18,6 +18,7 @@ def run_recreate(
     local_borg_version,
     recreate_arguments,
     global_arguments,
+    dry_run_label,
     local_path,
     remote_path,
 ):
@@ -25,9 +26,9 @@ def run_recreate(
     Run the "recreate" action for the given repository.
     '''
     if recreate_arguments.archive:
-        logger.answer(f'Recreating archive {recreate_arguments.archive}')
+        logger.answer(f'Recreating archive {recreate_arguments.archive}{dry_run_label}')
     else:
-        logger.answer('Recreating repository')
+        logger.answer(f'Recreating repository{dry_run_label}')
 
     # Collect and process patterns.
     processed_patterns = borgmatic.actions.pattern.process_patterns(

borgmatic/actions/restore.py

@@ -1,4 +1,5 @@
 import collections
+import locale
 import logging
 import os
 import pathlib
@@ -317,7 +318,7 @@ def collect_dumps_from_archive(
                 extract_to_stdout=True,
             )
             .stdout.read()
-            .decode(),
+            .decode(locale.getpreferredencoding()),
             dumps_metadata_entry['path'],
         ):
             dumps_from_archive[dump] = None

borgmatic/borg/check.py

@@ -149,8 +149,10 @@ def check_archives(
 
     max_duration = check_arguments.max_duration or repository_check_config.get('max_duration')
 
+    # If not configured, elevate Borg's exit code 1 (an ostensible warning) to error, because Borg
+    # returns exit code 1 for repository check errors!
+    borg_exit_codes = [*config.get('borg_exit_codes', []), *[{'code': 1, 'treat_as': 'error'}]]
     umask = config.get('umask')
-    borg_exit_codes = config.get('borg_exit_codes')
     working_directory = borgmatic.config.paths.get_working_directory(config)
 
     if 'data' in checks:

borgmatic/borg/create.py

@@ -179,10 +179,7 @@ def make_base_create_command(  # noqa: PLR0912
     return a tuple of (base Borg create command flags, Borg create command positional arguments,
     open pattern file handle).
     '''
-    if config.get('source_directories_must_exist', False):
-        logger.warning(
-            'The "source_directories_must_exist" option is deprecated and will be removed from a future release; borgmatic now errors on missing files as Borg runs'
-        )
+    if config.get('source_directories_must_exist', True):
         borgmatic.borg.pattern.check_all_root_patterns_exist(patterns)
 
     patterns_file = borgmatic.borg.pattern.write_patterns_file(
@@ -198,6 +195,7 @@ def make_base_create_command(  # noqa: PLR0912
     umask = config.get('umask', None)
     lock_wait = config.get('lock_wait', None)
     list_filter_flags = flags.make_list_filter_flags(local_borg_version, dry_run)
+    files_changed = config.get('files_changed')
     files_cache = config.get('files_cache')
     archive_name_format = (
         config.get('archive_name_format', flags.get_default_archive_name_format(local_borg_version))
@@ -248,6 +246,7 @@ def make_base_create_command(  # noqa: PLR0912
         + (('--nobirthtime',) if config.get('birthtime') is False else ())
         + (('--read-special',) if config.get('read_special') or stream_processes else ())
         + noflags_flags
+        + (('--files-changed', files_changed) if files_changed else ())
         + (('--files-cache', files_cache) if files_cache else ())
         + (('--remote-path', remote_path) if remote_path else ())
         + (('--umask', str(umask)) if umask else ())
@@ -270,7 +269,7 @@ def make_base_create_command(  # noqa: PLR0912
     working_directory = borgmatic.config.paths.get_working_directory(config)
 
     if config.get('unsafe_skip_path_validation_before_create'):
-        logger.warning(
+        logger.debug(
             'Skipping pre-backup path validation due to "unsafe_skip_path_validation_before_create" option.'
         )
 

borgmatic/borg/diff.py

@@ -0,0 +1,90 @@
+import logging
+import shlex
+
+import borgmatic.borg.environment
+import borgmatic.borg.feature
+import borgmatic.config.paths
+import borgmatic.execute
+from borgmatic.borg import flags
+from borgmatic.borg.pattern import write_patterns_file
+
+logger = logging.getLogger(__name__)
+
+
+def diff(
+    repository,
+    archive,
+    second_archive,
+    config,
+    local_borg_version,
+    diff_arguments,
+    global_arguments,
+    local_path,
+    remote_path=None,
+    patterns=None,
+):
+    '''
+    Given a local or remote repository path, two archive names, a configuration dict, the local Borg
+    version string, an argparse.Namespace of diff arguments, an argparse.Namespace of global
+    arguments, optional local and remote Borg paths, executes the diff command with the given
+    arguments.
+    '''
+    borgmatic.logger.add_custom_log_levels()
+
+    lock_wait = config.get('lock_wait', None)
+    exclude_flags = flags.make_exclude_flags(config)
+    extra_borg_options = config.get('extra_borg_options', {}).get('diff', '')
+
+    if diff_arguments.only_patterns:
+        # Write patterns to a temporary file and use that file with --patterns-from.
+        patterns_file = write_patterns_file(
+            patterns,
+            borgmatic.config.paths.get_working_directory(config),
+        )
+    else:
+        patterns_file = None
+
+    if borgmatic.borg.feature.available(
+        borgmatic.borg.feature.Feature.NUMERIC_IDS, local_borg_version
+    ):
+        numeric_ids_flags = ('--numeric-ids',) if config.get('numeric_ids') else ()
+    else:
+        numeric_ids_flags = ('--numeric-owner',) if config.get('numeric_ids') else ()
+
+    diff_command = (
+        (local_path, 'diff')
+        + (('--remote-path', remote_path) if remote_path else ())
+        + ('--log-json',)
+        + (('--lock-wait', str(lock_wait)) if lock_wait is not None else ())
+        + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+        + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+        + (
+            ('--patterns-from', patterns_file.name)
+            if patterns_file and diff_arguments.only_patterns
+            else ()
+        )
+        + exclude_flags
+        + numeric_ids_flags
+        + (('--same-chunker-params',) if diff_arguments.same_chunker_params else ())
+        + (('--sort-by', ','.join(diff_arguments.sort_keys)) if diff_arguments.sort_keys else ())
+        + (('--content-only',) if diff_arguments.content_only else ())
+        + (tuple(shlex.split(extra_borg_options)) if extra_borg_options else ())
+        + (
+            (*flags.make_repository_flags(repository, local_borg_version), archive)
+            if borgmatic.borg.feature.available(
+                borgmatic.borg.feature.Feature.SEPARATE_REPOSITORY_ARCHIVE,
+                local_borg_version,
+            )
+            else flags.make_repository_archive_flags(repository, archive, local_borg_version)
+        )
+        + (second_archive,)
+    )
+
+    borgmatic.execute.execute_command(
+        full_command=diff_command,
+        output_log_level=logging.ANSWER,
+        environment=borgmatic.borg.environment.make_environment(config),
+        working_directory=borgmatic.config.paths.get_working_directory(config),
+        borg_local_path=local_path,
+        borg_exit_codes=config.get('borg_exit_codes'),
+    )

borgmatic/borg/environment.py

@@ -25,6 +25,7 @@ DEFAULT_BOOL_OPTION_TO_ENVIRONMENT_VARIABLE = {
     'relocated_repo_access_is_ok': 'BORG_RELOCATED_REPO_ACCESS_IS_OK',
     'unknown_unencrypted_repo_access_is_ok': 'BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK',
     'use_chunks_archive': 'BORG_USE_CHUNKS_ARCHIVE',
+    'msgpack_version_check': 'BORG_MSGPACK_VERSION_CHECK',
 }
 
 
@@ -90,7 +91,8 @@ def make_environment(config):
     ) in DEFAULT_BOOL_OPTION_TO_ENVIRONMENT_VARIABLE.items():
         if os.environ.get(environment_variable_name) is None:
             value = config.get(option_name)
-            environment[environment_variable_name] = 'YES' if value else 'NO'
+            if value is not None:
+                environment[environment_variable_name] = 'YES' if value else 'NO'
 
     for (
         option_name,

borgmatic/borg/recreate.py

@@ -72,6 +72,7 @@ def recreate_archive(
         + (('--chunker-params', chunker_params) if chunker_params else ())
         + (('--recompress', recompress) if recompress else ())
         + exclude_flags
+        + (('--dry-run',) if global_arguments.dry_run else ())
         + (tuple(shlex.split(extra_borg_options)) if extra_borg_options else ())
         + (
             (
@@ -94,10 +95,6 @@ def recreate_archive(
         )
     )
 
-    if global_arguments.dry_run:
-        logger.info('Skipping the archive recreation (dry run)')
-        return
-
     borgmatic.execute.execute_command(
         full_command=recreate_command,
         output_log_level=logging.INFO,

borgmatic/commands/arguments.py

@@ -33,6 +33,7 @@ ACTION_ALIASES = {
     'key': [],
     'borg': [],
     'recreate': [],
+    'diff': [],
 }
 
 
@@ -1261,6 +1262,29 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         help='Show this help message and exit',
     )
 
+    config_show_parser = config_parsers.add_parser(
+        'show',
+        help='Show the computed configuration for each file specified with --config (see borgmatic --help)',
+        description='Show the computed configuration for each file specified with --config (see borgmatic --help)',
+        add_help=False,
+    )
+    config_show_group = config_show_parser.add_argument_group('config show arguments')
+    config_show_group.add_argument(
+        '--option',
+        help='Show the value of a single named configuration option instead of the entire configuration',
+    )
+    config_show_group.add_argument(
+        '--json',
+        action='store_true',
+        help='Show the configuration as JSON with one array element per configuration file',
+    )
+    config_show_group.add_argument(
+        '-h',
+        '--help',
+        action='help',
+        help='Show this help message and exit',
+    )
+
     export_tar_parser = action_parsers.add_parser(
         'export-tar',
         aliases=ACTION_ALIASES['export-tar'],
@@ -1931,7 +1955,7 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
     )
     recreate_group.add_argument(
         '--archive',
-        help='Archive name, hash, or series to recreate',
+        help='Archive name, hash, or series to recreate, defaults to all archives in the repository (if specified), or all archives across all repositories',
     )
     recreate_group.add_argument(
         '--list',
@@ -1970,6 +1994,47 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         help='Show this help message and exit',
     )
 
+    diff_parser = action_parsers.add_parser(
+        'diff',
+        aliases=ACTION_ALIASES['diff'],
+        help='This command finds differences (file contents, user/group/mode) between archives',
+        description='This command finds differences (file contents, user/group/mode) between archives',
+    )
+    diff_group = diff_parser.add_argument_group('diff arguments')
+    diff_group.add_argument(
+        '--repository',
+        help='Path of repository containing archive to diff, defaults to the configured repository if there is only one, quoted globs supported',
+    )
+    diff_group.add_argument(
+        '--archive',
+        help='Archive name, hash, or series to diff',
+        required=True,
+    )
+    diff_group.add_argument(
+        '--second-archive',
+        help='Second archive name, hash, or series to diff',
+        required=True,
+    )
+    diff_group.add_argument(
+        '--same-chunker-params', action='store_true', help='Override check of chunker parameters'
+    )
+    diff_group.add_argument(
+        '--sort-by',
+        metavar='KEY',
+        dest='sort_keys',
+        action='append',
+        help='Advanced sorting: specify field(s) to sort by. Prefix with > for descending or < for ascending (default)',
+    )
+    diff_group.add_argument(
+        '--content-only',
+        action='store_true',
+        help='Only compare differences in content (exclude metadata differences)',
+    )
+    diff_group.add_argument(
+        '--only-patterns',
+        action='store_true',
+        help='Run the diff according to borgmatic configured patterns (ie do not diff entire archives)',
+    )
     borg_parser = action_parsers.add_parser(
         'borg',
         aliases=ACTION_ALIASES['borg'],

borgmatic/commands/borgmatic.py

@@ -17,9 +17,11 @@ import borgmatic.actions.check
 import borgmatic.actions.compact
 import borgmatic.actions.config.bootstrap
 import borgmatic.actions.config.generate
+import borgmatic.actions.config.show
 import borgmatic.actions.config.validate
 import borgmatic.actions.create
 import borgmatic.actions.delete
+import borgmatic.actions.diff
 import borgmatic.actions.export_key
 import borgmatic.actions.export_tar
 import borgmatic.actions.extract
@@ -441,6 +443,7 @@ def run_actions(  # noqa: PLR0912, PLR0915
                         local_borg_version,
                         action_arguments,
                         global_arguments,
+                        dry_run_label,
                         local_path,
                         remote_path,
                     )
@@ -621,6 +624,16 @@ def run_actions(  # noqa: PLR0912, PLR0915
                         local_path,
                         remote_path,
                     )
+                elif action_name == 'diff':
+                    borgmatic.actions.diff.run_diff(
+                        repository,
+                        config,
+                        local_borg_version,
+                        action_arguments,
+                        global_arguments,
+                        local_path,
+                        remote_path,
+                    )
                 elif action_name == 'borg':
                     borgmatic.actions.borg.run_borg(
                         repository,
@@ -809,18 +822,18 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
     '''
     add_custom_log_levels()
 
-    if 'bootstrap' in arguments:
-        try:
-            # No configuration file is needed for bootstrap.
-            local_borg_version = borg_version.local_borg_version(
-                {},
-                arguments['bootstrap'].local_path,
-            )
-        except (OSError, CalledProcessError, ValueError) as error:
-            yield from log_error_records('Error getting local Borg version', error)
-            return
+    try:
+        if 'bootstrap' in arguments:
+            try:
+                # No configuration file is needed for bootstrap.
+                local_borg_version = borg_version.local_borg_version(
+                    {},
+                    arguments['bootstrap'].local_path,
+                )
+            except (OSError, CalledProcessError, ValueError) as error:
+                yield from log_error_records('Error getting local Borg version', error)
+                return
 
-        try:
             borgmatic.actions.config.bootstrap.run_bootstrap(
                 arguments['bootstrap'],
                 arguments['global'],
@@ -834,17 +847,10 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
                     name=logger.name,
                 ),
             )
-        except (
-            CalledProcessError,
-            ValueError,
-            OSError,
-        ) as error:
-            yield from log_error_records(error)
 
-        return
+            return
 
-    if 'generate' in arguments:
-        try:
+        if 'generate' in arguments:
             borgmatic.actions.config.generate.run_generate(
                 arguments['generate'],
                 arguments['global'],
@@ -857,29 +863,22 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
                     name=logger.name,
                 ),
             )
-        except (
-            CalledProcessError,
-            ValueError,
-            OSError,
-        ) as error:
-            yield from log_error_records(error)
-
-        return
-
-    if 'validate' in arguments:
-        if configuration_parse_errors:
-            yield logging.makeLogRecord(
-                dict(
-                    levelno=logging.CRITICAL,
-                    levelname='CRITICAL',
-                    msg='Configuration validation failed',
-                    name=logger.name,
-                ),
-            )
 
             return
 
-        try:
+        if 'validate' in arguments:
+            if configuration_parse_errors:
+                yield logging.makeLogRecord(
+                    dict(
+                        levelno=logging.CRITICAL,
+                        levelname='CRITICAL',
+                        msg='Configuration validation failed',
+                        name=logger.name,
+                    ),
+                )
+
+                return
+
             borgmatic.actions.config.validate.run_validate(arguments['validate'], configs)
 
             yield logging.makeLogRecord(
@@ -890,14 +889,20 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
                     name=logger.name,
                 ),
             )
-        except (
-            CalledProcessError,
-            ValueError,
-            OSError,
-        ) as error:
-            yield from log_error_records(error)
 
-        return
+            return
+
+        if 'show' in arguments:
+            borgmatic.actions.config.show.run_show(arguments['show'], configs)
+
+            return
+
+    except (
+        CalledProcessError,
+        ValueError,
+        OSError,
+    ) as error:
+        yield from log_error_records(error)
 
 
 def collect_configuration_run_summary_logs(configs, config_paths, arguments, log_file_path):  # noqa: PLR0912

borgmatic/commands/completion/bash.py

@@ -39,7 +39,7 @@ def bash_completion():
             'check_version() {',
             '    local this_script="$(cat "$BASH_SOURCE" 2> /dev/null)"',
             '    local installed_script="$(borgmatic --bash-completion 2> /dev/null)"',
-            '    if [ "$this_script" != "$installed_script" ] && [ "$installed_script" != "" ];'
+            '    if [ "$this_script" != "$installed_script" ] && [ "$installed_script" != "" ];',
             f'''        then cat << EOF\n{borgmatic.commands.completion.actions.upgrade_message(
                     'bash',
                     'sudo sh -c "borgmatic --bash-completion > $BASH_SOURCE"',

borgmatic/config/collect.py

@@ -47,5 +47,6 @@ def collect_config_filenames(config_paths):
         for filename in sorted(os.listdir(path)):
             full_filename = os.path.join(path, filename)
             matching_filetype = full_filename.endswith(('.yaml', '.yml'))
+
             if matching_filetype and not os.path.isdir(full_filename):
                 yield os.path.abspath(full_filename)

borgmatic/config/generate.py

@@ -153,6 +153,9 @@ def transform_optional_configuration(rendered_config, comment_out=True):
     return '\n'.join(lines)
 
 
+RUAMEL_YAML_END_OF_DOCUMENT_MARKER = '...\n'
+
+
 def render_configuration(config):
     '''
     Given a config data structure of nested OrderedDicts, render the config as YAML and return it.
@@ -160,7 +163,13 @@ def render_configuration(config):
     dumper = ruamel.yaml.YAML(typ='rt')
     dumper.indent(mapping=INDENT, sequence=INDENT + SEQUENCE_INDENT, offset=INDENT)
     rendered = io.StringIO()
-    dumper.dump(config, rendered)
+    dumper.dump(
+        config,
+        rendered,
+        # Dumping certain values (integers, for instance) causes ruamel.yaml to append an
+        # end-of-document "..." marker. Strip it.
+        transform=lambda dumped: dumped.removesuffix(RUAMEL_YAML_END_OF_DOCUMENT_MARKER),
+    )
 
     return rendered.getvalue()
 

borgmatic/config/schema.yaml

@@ -30,12 +30,10 @@ properties:
     source_directories_must_exist:
         type: boolean
         description: |
-            Deprecated. Replaced by borgmatic treating Borg's "backup file not
-            found" warning as an error by default. But if
-            "source_directories_must_exist" is true, then source directories
-            (and root pattern paths) must exist before a backup begins. If they
-            don't, borgmatic errors. Defaults to false.
-        example: true
+            When true, source directories (and root pattern paths) must exist
+            before a backup begins. If they don't, borgmatic errors. Defaults to
+            true.
+        example: false
     repositories:
         type: array
         items:
@@ -166,6 +164,16 @@ properties:
             Record filesystem flags (e.g. NODUMP, IMMUTABLE) in archive.
             Defaults to true.
         example: false
+    files_changed:
+        type: string
+        enum: ['ctime', 'mtime', 'disabled']
+        description: |
+            Threshold for considering a file as changed. See
+            https://borgbackup.readthedocs.io/en/stable/usage/create.html for
+            details. Defaults to "ctime". E.g., a file is considered changed if
+            its ctime has changed since the last backup. (This option is
+            supported for Borg 1.4.2+ only.)
+        example: ctime
     files_cache:
         type: string
         description: |
@@ -562,6 +570,12 @@ properties:
             Bypass Borg confirmation about check with repair option. Defaults to
             false and an interactive prompt from Borg.
         example: true
+    msgpack_version_check:
+        type: boolean
+        description: |
+            Optionally disable the msgpack version check. Default is true; use
+            at your own risk. (This option is supported for Borg 1.4.2+ only.)
+        example: false
     extra_borg_options:
         type: object
         additionalProperties: false
@@ -1115,6 +1129,7 @@ properties:
                 - break-lock
                 - key
                 - borg
+                - diff
         description: |
             List of one or more actions to skip running for this configuration
             file, even if specified on the command-line (explicitly or
@@ -1331,6 +1346,7 @@ properties:
                                   - break-lock
                                   - key
                                   - borg
+                                  - diff
                           description: |
                               List of actions for which the commands will be
                               run. Defaults to running for all actions.
@@ -1396,6 +1412,7 @@ properties:
                                   - break-lock
                                   - key
                                   - borg
+                                  - diff
                           description: |
                               Only trigger the hook when borgmatic is run with
                               particular actions listed here. Defaults to
@@ -2342,6 +2359,13 @@ properties:
                         example: Your backups have started.
                     priority:
                         type: string
+                        enum:
+                            - max
+                            - urgent
+                            - high
+                            - default
+                            - low
+                            - min
                         description: |
                             The priority to set.
                         example: min
@@ -2366,6 +2390,13 @@ properties:
                         example: Your backups have finished.
                     priority:
                         type: string
+                        enum:
+                            - max
+                            - urgent
+                            - high
+                            - default
+                            - low
+                            - min
                         description: |
                             The priority to set.
                         example: min
@@ -2390,6 +2421,13 @@ properties:
                         example: Your backups have failed.
                     priority:
                         type: string
+                        enum:
+                            - max
+                            - urgent
+                            - high
+                            - default
+                            - low
+                            - min
                         description: |
                             The priority to set.
                         example: max

borgmatic/execute.py

@@ -2,7 +2,9 @@ import collections
 import contextlib
 import enum
 import json
+import locale
 import logging
+import os
 import select
 import subprocess
 import textwrap
@@ -18,7 +20,7 @@ BORG_ERROR_EXIT_CODE_START = 2
 BORG_ERROR_EXIT_CODE_END = 99
 
 # See https://borgbackup.readthedocs.io/en/stable/internals/frontends.html#message-ids
-BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS = {101, 102, 104, 105, 106, 107}
+BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS = {101, 102, 104, 105, 106}
 
 
 class Exit_status(enum.Enum):
@@ -28,6 +30,19 @@ class Exit_status(enum.Enum):
     ERROR = 4
 
 
+def command_is_borg(command, borg_local_path):
+    '''
+    Given a command as a sequence and the Borg local path, return whether that command is a call to
+    Borg.
+    '''
+    parsed_command = command.split(' ', 1) if isinstance(command, str) else command
+
+    if not parsed_command:
+        return False
+
+    return bool(borg_local_path and parsed_command[0] == borg_local_path)
+
+
 def interpret_exit_code(command, exit_code, borg_local_path=None, borg_exit_codes=None):  # noqa: PLR0911
     '''
     Return an Exit_status value (e.g. SUCCESS, ERROR, or WARNING) based on interpreting the given
@@ -41,7 +56,7 @@ def interpret_exit_code(command, exit_code, borg_local_path=None, borg_exit_code
     if exit_code == 0:
         return Exit_status.SUCCESS
 
-    if borg_local_path and command[0] == borg_local_path:
+    if command_is_borg(command, borg_local_path):
         # First try looking for the exit code in the borg_exit_codes configuration.
         for entry in borg_exit_codes or ():
             if entry.get('code') == exit_code:
@@ -102,19 +117,41 @@ def output_buffers_for_process(process, exclude_stdouts):
     )
 
 
+BORG_LOG_LEVEL_ELEVATION_THRESHOLD = 10
+
+
 def borg_json_log_line_to_record(line, log_level):
     '''
     Given a single Borg "--log-json"-style log line and a log level, return the line converted to a
     logging.LogRecord instance. Return None if the line can't be parsed as JSON.
+
+    If Borg provides a log level in its JSON, prefer logging at that level. But if Borg doesn't
+    provide a log level—or the log level given to this function is just a little bit higher than
+    Borg's—elevate to that level. This supports use cases like elevating Borg's INFO level logs to
+    borgmatic's custom ANSWER level so that requested data shows up even at the default verbosity.
     '''
     with contextlib.suppress(json.JSONDecodeError, TypeError, KeyError, AttributeError):
         log_data = json.loads(line)
         log_type = log_data.get('type')
 
         if log_type == 'log_message':
+            borg_log_level = logging._nameToLevel.get(log_data.get('levelname'))
+            log_level_delta = 0 if log_level is None else log_level - borg_log_level
+
+            if log_level_delta > 0 and log_level_delta < BORG_LOG_LEVEL_ELEVATION_THRESHOLD:
+                return logging.makeLogRecord(
+                    dict(
+                        levelno=log_level,
+                        created=log_data.get('time'),
+                        msg=log_data.get('message'),
+                        levelname=logging.getLevelName(log_level),
+                        name=log_data.get('name'),
+                    )
+                )
+
             return logging.makeLogRecord(
                 dict(
-                    levelno=logging._nameToLevel.get(log_data.get('levelname')),
+                    levelno=borg_log_level,
                     created=log_data.get('time'),
                     msg=log_data.get('message'),
                     levelname=log_data.get('levelname'),
@@ -163,7 +200,7 @@ def parse_log_line(line, log_level, elevate_stderr, borg_local_path, command):
     came from stderr and the string "warning:" appears at the start of the log line. In that case,
     just elevate the log level to a WARN.
     '''
-    if borg_local_path and command[0] == borg_local_path:
+    if command_is_borg(command, borg_local_path):
         log_record = borg_json_log_line_to_record(line, log_level)
 
         if log_record:
@@ -177,18 +214,20 @@ def parse_log_line(line, log_level, elevate_stderr, borg_local_path, command):
     return log_line_to_record(line, log_level)
 
 
-def handle_log_record(log_record, last_lines):
+def handle_log_record(log_record, last_lines=None):
     '''
     Given a log record to be logged and a rolling list of last lines, append the record's message to
-    the last lines. Then (if the log level is not None), log the record.
+    the last lines (if given). Then (if the log level is not None), log the record.
 
     Return the log record.
     '''
     log_message = log_record.getMessage()
-    last_lines.append(log_message)
 
-    if len(last_lines) > ERROR_OUTPUT_MAX_LINE_COUNT:
-        last_lines.pop(0)
+    if last_lines is not None:
+        last_lines.append(log_message)
+
+        if len(last_lines) > ERROR_OUTPUT_MAX_LINE_COUNT:
+            last_lines.pop(0)
 
     if log_record.levelno is not None:
         logger.handle(log_record)
@@ -196,7 +235,208 @@ def handle_log_record(log_record, last_lines):
     return log_record
 
 
-def log_outputs(  # noqa: PLR0912
+READ_CHUNK_SIZE = 4096
+
+
+def read_lines(buffer, process, line_separator='\n'):
+    '''
+    Given a Python buffer (like stdout) ready for reading, its process, and a line separator,
+    repeatedly yield a tuple of (decoded) lines from the buffer until the process has exited.
+
+    It is assumed that this function's generator is used in conjunction with an external select()
+    call to know when to read more lines. Otherwise, the generator will busywait if it's called in a
+    tight loop.
+    '''
+    data = b''
+    encoded_separator = line_separator.encode()
+    separator_size = len(encoded_separator)
+    encoding = locale.getpreferredencoding()
+
+    while True:
+        chunk = os.read(buffer.fileno(), READ_CHUNK_SIZE)
+
+        if not chunk:  # EOF
+            # The process is still running, so we keep running too.
+            if process.poll() is None:  # pragma: no cover
+                continue
+
+            break
+
+        data += chunk
+        lines = []
+
+        # Split the data into lines, holding back anything leftover that might
+        # be a partial line.
+        while True:
+            separator_position = data.find(encoded_separator)
+
+            if separator_position == -1:
+                break
+
+            lines.append(data[:separator_position].decode(encoding))
+            data = data[separator_position + separator_size :]
+
+        yield tuple(lines)
+
+    # Yield any leftover data from the end of the buffer.
+    if data:
+        yield (data.decode(encoding).rstrip(),)
+
+
+Buffer_reader = collections.namedtuple(
+    'Buffer_reader',
+    ('lines', 'process'),
+)
+
+
+Process_metadata = collections.namedtuple(
+    'Process_metadata',
+    ('last_lines', 'capture'),
+)
+
+
+def log_buffer_lines(
+    buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr=False
+):
+    '''
+    Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
+    Process_metadata instance, a requested output log level for stdout, Borg's local path, and
+    whether to capture stderr, read and log any ready output lines from the buffers.  Additionally,
+    if the log level is None for any log record, then yield those log messages for capture.
+
+    This function just does one "turn of the crank" of logging buffer output. It is intended to be
+    called repeatedly to continue to process buffers.
+    '''
+    if not buffer_readers:
+        return
+
+    (ready_buffers, _, _) = select.select(buffer_readers.keys(), [], [])
+
+    for ready_buffer in ready_buffers:
+        reader = buffer_readers[ready_buffer]
+
+        # The "ready" process has exited, but it might be a pipe destination with other
+        # processes (pipe sources) waiting to be read from. So as a measure to prevent
+        # hangs, vent all processes when one exits.
+        if reader.process and reader.process.poll() is not None:
+            for other_process in process_metadatas:
+                if (
+                    other_process.poll() is None
+                    and other_process.stdout
+                    and other_process.stdout not in buffer_readers
+                ):
+                    # Add the process's output to buffer_readers to ensure it'll get read.
+                    buffer_readers[other_process.stdout] = Buffer_reader(
+                        read_lines(other_process.stdout, other_process), other_process
+                    )
+
+        try:
+            lines = next(reader.lines)
+        except StopIteration:
+            continue
+
+        for line in lines:
+            if not line or not reader.process:
+                continue
+
+            # Keep the last few lines of output in case the process errors and we need the
+            # output for the exception below.
+            log_record = handle_log_record(
+                parse_log_line(
+                    line=line,
+                    log_level=output_log_level,
+                    elevate_stderr=(ready_buffer == reader.process.stderr and not capture_stderr),
+                    borg_local_path=borg_local_path,
+                    command=reader.process.args,
+                ),
+                last_lines=process_metadatas[reader.process].last_lines,
+            )
+
+            if log_record.levelno is None and process_metadatas[reader.process].capture:
+                yield log_record.getMessage()
+
+
+def raise_for_process_errors(buffer_readers, process_metadatas, borg_local_path, borg_exit_codes):
+    '''
+    Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
+    Process_metadata instance, Borg's local path, a sequence of exit code configuration dicts, check
+    the given processes for error or warning exit codes. If found, vent or kill any running
+    processes. In the case of an error exit code, raise. In the case of warning, return
+    Exit_status.WARNING. Otherwise, return None.
+    '''
+    result_status = None
+
+    for process in process_metadatas:
+        exit_code = process.poll() if buffer_readers else process.wait()
+
+        if exit_code is None:
+            continue
+
+        exit_status = interpret_exit_code(process.args, exit_code, borg_local_path, borg_exit_codes)
+
+        if exit_status not in {Exit_status.ERROR, Exit_status.WARNING}:
+            continue
+
+        # Something has gone wrong. So vent each process' output buffer to prevent it from
+        # hanging. And then kill the process.
+        for other_process in process_metadatas:
+            if other_process.poll() is None:
+                other_process.stdout.read(0)
+                other_process.kill()
+
+        if exit_status == Exit_status.WARNING:
+            result_status = Exit_status.WARNING
+            continue
+
+        last_lines = process_metadatas[process].last_lines
+
+        # If an error occurs, include its output in the raised exception so that we don't
+        # inadvertently hide error output.
+        if len(last_lines) >= ERROR_OUTPUT_MAX_LINE_COUNT:
+            last_lines.insert(0, '...')
+
+        raise subprocess.CalledProcessError(
+            exit_code,
+            command_for_process(process),
+            '\n'.join(last_lines),
+        )
+
+    return result_status
+
+
+def log_remaining_buffer_lines(
+    buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr=False
+):
+    '''
+    Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
+    Process_metadata instance, a requested output log level for stdout, Borg's local path, and
+    whether to capture stderr, drain and log any remaining output lines from the buffers until
+    they're empty. Additionally, if the log level is None for any log record, then yield those log
+    messages for capture.
+    '''
+    for output_buffer, reader in buffer_readers.items():
+        if not reader.process:
+            continue
+
+        for lines in reader.lines:
+            for line in lines:
+                log_record = handle_log_record(
+                    parse_log_line(
+                        line=line.rstrip(),
+                        log_level=output_log_level,
+                        elevate_stderr=(
+                            output_buffer == reader.process.stderr and not capture_stderr
+                        ),
+                        borg_local_path=borg_local_path,
+                        command=reader.process.args,
+                    ),
+                )
+
+                if log_record.levelno is None and process_metadatas[reader.process].capture:
+                    yield log_record.getMessage()
+
+
+def log_outputs(
     processes,
     exclude_stdouts,
     output_log_level,
@@ -221,131 +461,42 @@ def log_outputs(  # noqa: PLR0912
     buffers. Also note that stdout for a process can be None if output is intentionally not
     captured, in which case it won't be logged.
     '''
-    # Map from output buffer to sequence of last lines.
-    process_last_lines = collections.defaultdict(list)
-    process_for_output_buffer = {
-        buffer: process
+    # Map from output buffer to Process_metadata instance. By convention, the last process is the
+    # process to capture.
+    process_metadatas = {
+        process: Process_metadata(last_lines=[], capture=bool(process == processes[-1]))
+        for process in processes
+    }
+
+    # Map from buffer to Buffer_reader instance.
+    buffer_readers = {
+        buffer: Buffer_reader(read_lines(buffer, process), process)
         for process in processes
         if process.stdout or process.stderr
         for buffer in output_buffers_for_process(process, exclude_stdouts)
     }
-    output_buffers = list(process_for_output_buffer.keys())
-    process_to_capture = processes[-1]
-    still_running = True
 
-    # Log output for each process until they all exit.
-    while True:  # noqa: PLR1702
-        if output_buffers:
-            (ready_buffers, _, _) = select.select(output_buffers, [], [])
+    # Log output lines for each process until they all exit.
+    while True:
+        yield from log_buffer_lines(
+            buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
+        )
 
-            for ready_buffer in ready_buffers:
-                ready_process = process_for_output_buffer.get(ready_buffer)
-
-                # The "ready" process has exited, but it might be a pipe destination with other
-                # processes (pipe sources) waiting to be read from. So as a measure to prevent
-                # hangs, vent all processes when one exits.
-                if ready_process and ready_process.poll() is not None:
-                    for other_process in processes:
-                        if (
-                            other_process.poll() is None
-                            and other_process.stdout
-                            and other_process.stdout not in output_buffers
-                        ):
-                            # Add the process's output to output_buffers to ensure it'll get read.
-                            output_buffers.append(other_process.stdout)
-
-                while True:
-                    line = ready_buffer.readline().rstrip().decode()
-                    if not line or not ready_process:
-                        break
-
-                    command = (
-                        ready_process.args.split(' ')
-                        if isinstance(ready_process.args, str)
-                        else ready_process.args
-                    )
-
-                    # Keep the last few lines of output in case the process errors and we need the
-                    # output for the exception below.
-                    log_record = handle_log_record(
-                        parse_log_line(
-                            line=line,
-                            log_level=output_log_level,
-                            elevate_stderr=(
-                                ready_buffer == ready_process.stderr and not capture_stderr
-                            ),
-                            borg_local_path=borg_local_path,
-                            command=command,
-                        ),
-                        last_lines=process_last_lines[ready_process],
-                    )
-
-                    if log_record.levelno is None and ready_process == process_to_capture:
-                        yield log_record.getMessage()
-
-        if not still_running:
+        if (
+            raise_for_process_errors(
+                buffer_readers, process_metadatas, borg_local_path, borg_exit_codes
+            )
+            == Exit_status.WARNING
+        ):
             break
 
-        still_running = False
+        if all(process.poll() is not None for process in processes):
+            break
 
-        for process in processes:
-            exit_code = process.poll() if output_buffers else process.wait()
-
-            if exit_code is None:
-                still_running = True
-                command = process.args.split(' ') if isinstance(process.args, str) else process.args
-                continue
-
-            command = process.args.split(' ') if isinstance(process.args, str) else process.args
-            exit_status = interpret_exit_code(command, exit_code, borg_local_path, borg_exit_codes)
-
-            if exit_status in {Exit_status.ERROR, Exit_status.WARNING}:
-                last_lines = process_last_lines[process]
-
-                # If an error occurs, include its output in the raised exception so that we don't
-                # inadvertently hide error output.
-                for output_buffer in output_buffers_for_process(process, exclude_stdouts):
-                    # Collect any straggling output lines that came in since we last gathered output.
-                    while output_buffer:  # pragma: no cover
-                        line = output_buffer.readline().rstrip().decode()
-                        if not line:
-                            break
-
-                        log_record = handle_log_record(
-                            parse_log_line(
-                                line=line,
-                                log_level=output_log_level,
-                                elevate_stderr=(
-                                    output_buffer == process.stderr and not capture_stderr
-                                ),
-                                borg_local_path=borg_local_path,
-                                command=command,
-                            ),
-                            last_lines=last_lines,
-                        )
-
-                        if log_record.levelno is None and process == process_to_capture:
-                            yield log_record.getMessage()
-
-                if len(last_lines) == ERROR_OUTPUT_MAX_LINE_COUNT:
-                    last_lines.insert(0, '...')
-
-                # Something has gone wrong. So vent each process' output buffer to prevent it from
-                # hanging. And then kill the process.
-                for other_process in processes:
-                    if other_process.poll() is None:
-                        other_process.stdout.read(0)
-                        other_process.kill()
-
-                if exit_status == Exit_status.ERROR:
-                    raise subprocess.CalledProcessError(
-                        exit_code,
-                        command_for_process(process),
-                        '\n'.join(last_lines),
-                    )
-
-                still_running = False
-                break
+    # Now that all processes have exited, drain and consume any last output.
+    yield from log_remaining_buffer_lines(
+        buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
+    )
 
 
 SECRET_COMMAND_FLAG_NAMES = {'--password'}
@@ -496,7 +647,9 @@ def execute_command_and_capture_output(
             command,
             stdin=input_file,
             stdout=subprocess.PIPE,
-            stderr=subprocess.PIPE,
+            stderr=subprocess.PIPE
+            if capture_stderr or command_is_borg(command, borg_local_path)
+            else None,
             shell=shell,
             env=environment,
             cwd=working_directory,
@@ -510,7 +663,7 @@ def execute_command_and_capture_output(
             raise
 
         if error.output is not None:
-            yield from iter(error.output.decode().splitlines())
+            yield from iter(error.output.decode(locale.getpreferredencoding()).splitlines())
 
         return
 

borgmatic/hooks/data_source/bootstrap.py

@@ -1,6 +1,7 @@
 import contextlib
 import glob
 import importlib
+import itertools
 import json
 import logging
 import os
@@ -19,6 +20,29 @@ def use_streaming(hook_config, config):  # pragma: no cover
     return False
 
 
+MAXIMUM_CONFIG_SYMLINKS_TO_FOLLOW = 10
+
+
+def resolve_config_path_symlinks(path):
+    '''
+    Given a path, resolve and yield each successive symlink until the final non-symlink target. If
+    the given path isn't a symlink, then just yield it.
+
+    Raise ValueError if we have to follow too many symlinks without getting to the final target.
+    '''
+    original_path = path
+
+    for _ in range(MAXIMUM_CONFIG_SYMLINKS_TO_FOLLOW):
+        yield os.path.abspath(path)
+
+        if not os.path.islink(path):
+            return
+
+        path = os.readlink(path)
+
+    raise ValueError(f'Too many symlinks to follow for configuration path: {original_path}')
+
+
 def dump_data_sources(
     hook_config,
     config,
@@ -34,6 +58,9 @@ def dump_data_sources(
     the archive. But skip this if the bootstrap store_config_files option is False or if this is a
     dry run.
 
+    If any configuration paths are symlinks, then store each symlink along with any destination
+    paths as well.
+
     Return an empty sequence, since there are no ongoing dump processes from this hook.
     '''
     if hook_config and hook_config.get('store_config_files') is False:
@@ -45,6 +72,10 @@ def dump_data_sources(
         'manifest.json',
     )
 
+    resolved_config_paths = tuple(
+        itertools.chain.from_iterable(resolve_config_path_symlinks(path) for path in config_paths)
+    )
+
     if dry_run:
         return []
 
@@ -54,7 +85,7 @@ def dump_data_sources(
         json.dump(
             {
                 'borgmatic_version': importlib.metadata.version('borgmatic'),
-                'config_paths': config_paths,
+                'config_paths': resolved_config_paths,
             },
             manifest_file,
         )
@@ -67,7 +98,7 @@ def dump_data_sources(
         ),
     )
 
-    for config_path in config_paths:
+    for config_path in resolved_config_paths:
         borgmatic.hooks.data_source.config.inject_pattern(
             patterns,
             borgmatic.borg.pattern.Pattern(

borgmatic/hooks/data_source/zfs.py

@@ -71,7 +71,7 @@ def get_datasets_to_backup(zfs_command, patterns):
                 )
                 # Skip datasets that are marked "canmount=off", because mounting their snapshots will
                 # result in completely empty mount points—thereby preventing us from backing them up.
-                if can_mount == 'on'
+                if can_mount != 'off'
             ),
             key=lambda dataset: dataset.mount_point,
             reverse=True,

borgmatic/hooks/monitoring/ntfy.py

@@ -22,6 +22,29 @@ def initialize_monitor(
     '''
 
 
+def convert_string_to_array(value):
+    value = '' if value is None else str(value)
+    items = []
+
+    for item in value.split(','):
+        stripped = item.strip()
+
+        if stripped:
+            items.append(stripped)
+
+    return items
+
+
+PRIORITY_NAME_TO_ID = {
+    'max': 5,
+    'urgent': 5,
+    'high': 4,
+    'default': 3,
+    'low': 2,
+    'min': 1,
+}
+
+
 def ping_monitor(hook_config, config, config_filename, state, monitoring_log_level, dry_run):
     '''
     Ping the configured Ntfy topic. Use the given configuration filename in any log entries.
@@ -31,13 +54,13 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
 
     if state.name.lower() in run_states:
         dry_run_label = ' (dry run; not actually pinging)' if dry_run else ''
-
+        default_priority = PRIORITY_NAME_TO_ID['default']
         state_config = hook_config.get(
             state.name.lower(),
             {
                 'title': f'A borgmatic {state.name} event happened',
                 'message': f'A borgmatic {state.name} event happened',
-                'priority': 'default',
+                'priority': default_priority,
                 'tags': 'borgmatic',
             },
         )
@@ -55,8 +78,8 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
             'topic': topic,
             'title': state_config.get('title'),
             'message': state_config.get('message'),
-            'priority': state_config.get('priority'),
-            'tags': state_config.get('tags'),
+            'priority': PRIORITY_NAME_TO_ID.get(state_config.get('priority'), default_priority),
+            'tags': convert_string_to_array(state_config.get('tags')),
         }
 
         try:

docs/.env

@@ -0,0 +1 @@
+PORT=8080

docs/Dockerfile

@@ -5,13 +5,17 @@ RUN apk add --no-cache py3-pip py3-ruamel.yaml py3-ruamel.yaml.clib
 RUN pip install --break-system-packages --no-cache /app && borgmatic config generate && borgmatic config generate --destination /etc/borgmatic --split && chmod +r /etc/borgmatic/*.yaml
 RUN mkdir /command-line \
     && borgmatic --help > /command-line/global.txt \
-    && for action in repo-create transfer create prune compact check delete extract config "config bootstrap" "config generate" "config validate" export-tar mount umount repo-delete restore repo-list list repo-info info break-lock "key export" "key import" "key change-passphrase" recreate borg; do \
+    && for action in repo-create transfer create prune compact check delete extract config "config bootstrap" "config generate" "config validate" "config show" export-tar mount umount repo-delete restore repo-list list repo-info info break-lock "key export" "key import" "key change-passphrase" recreate diff borg; do \
            borgmatic $action --help > /command-line/${action/ /-}.txt; done
 RUN /app/docs/fetch-contributors >> /contributors.html
 
 FROM docker.io/node:22.4.0-alpine AS html
 
 ARG ENVIRONMENT=production
+ARG PORT=${PORT}
+ARG FONT=${FONT}
+
+RUN echo "FONT: ${FONT}"
 
 WORKDIR /source
 
@@ -28,7 +32,7 @@ COPY --from=borgmatic /etc/borgmatic/options.json /source/docs/reference/configu
 COPY --from=borgmatic /command-line/* /source/docs/_includes/borgmatic/command-line/
 COPY --from=borgmatic /contributors.html /source/docs/_includes/borgmatic/contributors.html
 COPY . /source
-RUN NODE_ENV=${ENVIRONMENT} npx eleventy --input=/source/docs --output=/output
+RUN NODE_ENV=${ENVIRONMENT} PORT=${PORT} FONT=${FONT} npx eleventy --input=/source/docs --output=/output
 RUN npx -y pagefind --site /output
 
 FROM docker.io/nginx:1.26.1-alpine

docs/_data/borgmatic.js

@@ -1,5 +1,7 @@
 module.exports = function() {
     return {
-        environment: process.env.NODE_ENV || "development"
+        environment: process.env.NODE_ENV || "development",
+        port: process.env.PORT || 8080,
+        font: process.env.FONT || "custom"
     };
 };

docs/_includes/layouts/base.njk

@@ -11,7 +11,7 @@
 		<title>borgmatic{% if subtitle or title %} - {% endif %}{{ subtitle + ' - ' if subtitle}}{{ title }}</title>
 		{% endif %}
 {%- set css %}
-{% include 'index.css' %}
+{% include 'index.css.njk' %}
 {% include 'components/lists.css' %}
 {% include 'components/external-links.css' %}
 {% include 'components/minilink.css' %}

docs/_includes/layouts/main.njk

@@ -21,7 +21,7 @@ headerClass: elv-header-default
             {% set navPages = collections.all | eleventyNavigation %}
             {% macro renderNavListItem(entry) -%}
             <li{% if entry.url == page.url %} class="elv-toc-active"{% endif %}>
-              <a {% if entry.url %}href="{% if borgmatic.environment == "production" %}https://torsion.org/borgmatic{% else %}http://localhost:8080/borgmatic{% endif %}{{ entry.url | url }}"{% endif %}>{{ entry.title }}</a>
+              <a {% if entry.url %}href="{% if borgmatic.environment == "production" %}https://torsion.org/borgmatic{% else %}http://localhost:{{ borgmatic.port }}/borgmatic{% endif %}{{ entry.url | url }}"{% endif %}>{{ entry.title }}</a>
             {%- if entry.children.length -%}
               <ul>
                 {%- for child in entry.children %}{{ renderNavListItem(child) }}{% endfor -%}

docs/docker-compose.yaml

@@ -2,10 +2,12 @@ services:
   traefik:
     image: public.ecr.aws/docker/library/traefik:3.5.3
     container_name: borgmatic-docs-traefik
+    environment:
+      - PORT=${PORT:-8080}
     command:
       - "--global.checkNewVersion=false"
       - "--global.sendAnonymousUsage=false"
-      - "--entrypoints.web.address=:8080"
+      - "--entrypoints.web.address=:${PORT}"
       - "--accesslog"
       - "--accesslog.fields.headers.defaultmode=keep"
       - "--providers.docker"
@@ -14,7 +16,7 @@ services:
       - "--api.dashboard=false"
       - "--log.level=WARN"
     ports:
-      - "127.0.0.1:8080:8080"
+      - "127.0.0.1:${PORT}:${PORT}"
     volumes:
       - ${CONTAINER_SOCKET_PATH:-/run/user/docker.sock}:/var/run/docker.sock:ro
   docs:
@@ -37,6 +39,10 @@ services:
       context: ..
       args:
         ENVIRONMENT: development
+        PORT: ${PORT:-8080}
+        FONT: ${FONT}
+    environment:
+      - PORT=${PORT:-8080}
   message:
     image: alpine
     container_name: borgmatic-docs-message
@@ -44,6 +50,6 @@ services:
       - sh
       - -c
       - |
-        echo; echo "You can view dev docs at http://localhost:8080/borgmatic/"; echo
+        echo; echo "You can view dev docs at http://localhost:${PORT}/borgmatic/"; echo
     depends_on:
       - docs

docs/fetch-contributors

@@ -36,7 +36,6 @@ def list_contributing_issues(url):
 
 PULLS_API_ENDPOINT_URLS = (
     'https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/pulls',
-    'https://api.github.com/repos/borgmatic-collective/borgmatic/pulls',
 )
 ISSUES_API_ENDPOINT_URL = 'https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/issues?state=all'
 RECENT_CONTRIBUTORS_CUTOFF_DAYS = 365

docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server.md

@@ -82,7 +82,7 @@ before_actions:
 option in the `hooks:` section of your configuration.
 
 <span class="minilink minilink-addedin">Prior to version 1.7.0</span> Use
-`before_create` or similar instead of `before_actions`, which was introduced in
+`before_backup` or similar instead of `before_actions`, which was introduced in
 borgmatic 1.7.0.
 
 What this does is check if the `findmnt` command errors when probing for a

docs/how-to/customize-warnings-and-errors.md

@@ -12,10 +12,11 @@ decide how to respond. By default, Borg errors (and some warnings) result
 in a borgmatic error, while Borg successes don't.
 
 <span class="minilink minilink-addedin">New in borgmatic version 2.1.0</span>
-borgmatic elevates most Borg warnings to errors by default. For instance, if a
-source directory is missing during backup, Borg indicates that with a warning
-exit code (`107`). And starting in borgmatic 2.1.0, that exit code is considered
-an error, so you'll actually find out about missing files.
+borgmatic elevates several Borg warnings to errors by default. For instance, if
+borgmatic doesn't have permission to read a configured source directory during
+backup, Borg indicates that with a warning exit code (`105`). And starting in
+borgmatic 2.1.0, that exit code is considered an error, so you'll actually find
+out about files that borgmatic can't read.
 
 <span class="minilink minilink-addedin">With Borg version 1.4+</span> If the
 default behavior isn't sufficient for your needs, you can customize how
@@ -23,7 +24,7 @@ borgmatic interprets [Borg's exit
 codes](https://borgbackup.readthedocs.io/en/stable/internals/frontends.html#message-ids).
 
 For instance, this borgmatic configuration elevates a Borg warning about source files
-changes during backup (exit code `100`)—and only those warnings—to
+changing during backup (exit code `100`)—and only those warnings—to
 errors:
 
 ```yaml
@@ -32,14 +33,15 @@ borg_exit_codes:
       treat_as: error
 ```
 
-The following configuration does that *and* treats Borg's backup file not found
-(exit code `107`) as a warning:
+The following configuration does that *and* squashes errors about Borg
+encountering file permissions issues during backup (exit code `105`) to
+warnings.
 
 ```yaml
 borg_exit_codes:
     - code: 100
       treat_as: error
-    - code: 107
+    - code: 105
       treat_as: warning
 ```
 
@@ -54,8 +56,11 @@ is not found:
 terminating with warning status, rc 107
 ```
 
-So if you want to configure borgmatic to treat this as an warning instead of an
-error, the exit status to use is `107`.
+So if you want to configure borgmatic's interpretation of this warning, the exit
+status to use is `107`. Note however that in the particular case of missing
+files, there's a separate [`source_directories_must_exist`
+option](https://torsion.org/borgmatic/reference/configuration/#source_directories_must_exist-option)
+that can catch such problems before Borg even runs.
 
 <span class="minilink minilink-addedin">With Borg version 1.2 and earlier</span>
 Older versions of Borg didn't support granular exit codes, but still

docs/how-to/deal-with-very-large-backups.md

@@ -60,7 +60,7 @@ follows:
 unsafe_skip_path_validation_before_create: true
 ```
 
-However, this is indeed unsafe, and could lead to hangs or data being left out
+However, this is indeed unsafe and could lead to hangs or data being left out
 of backups. Use this option at your own risk.
 
 

docs/how-to/develop-on-borgmatic.md

@@ -190,8 +190,28 @@ This requires Docker (or Podman; see below) to be installed on your system.
 This script assumes you have permission to run `docker`. If you don't, then
 you may need to run with `sudo`.
 
+### How to choose a different port
+
+You can choose a different listening port in two ways:
+
+#### 1. Modify the `.env` file
+
+1.  Open `docs/.env`.
+2.  Change `PORT=8080` to your desired port number (e.g., `PORT=3000`).
+3.  Run the development script: `scripts/dev-docs`.
+
+#### 2. Use an environment variable
+
+Alternatively, you can override the port directly from your terminal without
+modifying any files:
+
+```bash
+PORT=3000 ./scripts/dev-docs
+```
+
 After you run the script, you can point your web browser at
-http://localhost:8080/borgmatic/ to view the documentation with your changes.
+http://localhost:8080/borgmatic/ (or your chosen port) to view the documentation
+with your changes.
 
 To close the documentation server, ctrl-C the script. Note that it does not
 currently auto-reload, so you'll need to stop it and re-run it for any
@@ -205,7 +225,24 @@ borgmatic's developer build for documentation optionally supports using
 [Podman](https://podman.io/) instead of Docker.
 
 Setting up Podman is outside the scope of this documentation. But once you
-install and configure Podman, then `scripts/dev-docs` should automatically use
-Podman instead of Docker.
+install and configure Podman, then `scripts/dev-docs` automatically uses Podman
+instead of Docker and [Podman
+Compose](https://github.com/containers/podman-compose) (if present) instead of
+Docker Compose. However Podman works fine with either Podman Compose or Docker
+Compose.
+
+
+## Use of generative AI
+
+Please do not use AI agents to modify this codebase. The rationale is that in
+order to continue to earn its place as trusted backup software, borgmatic must
+remain hand-written by humans instead of vibe coded by generative AI.
+
+Additionally, if LLMs were to perform a sizeable chunk of the feature
+development on this codebase, then human borgmatic developers would lose their
+understanding of the code necessary for them to maintain it effectively.
+
+Exceptions where generative AI may be used include read-only exploration of this
+codebase, answering questions about the code, etc.
 
 </span>

docs/how-to/inspect-your-backups.md

@@ -116,3 +116,22 @@ By default, borgmatic only logs to the console. But to enable simultaneous
 syslog or file logging, see the [logging
 documentation](https://torsion.org/borgmatic/reference/command-line/logging/)
 for details.
+
+## Finding differences between two archives
+
+<span class="minilink minilink-addedin">New in borgmatic version
+2.1.3</span>You can compare differences between two archives. For example:
+
+```bash
+borgmatic diff --archive latest --second-archive host-2023-01-02T04:06:07.080910
+```
+
+This shows the differences (file contents, user/group/mode) between the latest
+archive and the second one supplied.
+
+Note that, by default, `borgmatic diff` compares everything in the archives; that
+is, patterns are _not_ taken into consideration. If you require this, supply the
+`--only-patterns` flag.
+
+See the [Borg](https://borgbackup.readthedocs.io/en/stable/usage/diff.html)
+documentation for information on output format, what is compared, and more.

docs/how-to/monitor-your-backups.md

@@ -64,6 +64,27 @@ suppressed so as not to interfere with the captured JSON. Also note that JSON
 output only shows up at the console and not in syslog.
 
 
+### Getting configuration
+
+<span class="minilink minilink-addedin">New in version 2.1.3</span> If you want
+to consume borgmatic's computed configuration in your scripts, use the [`config
+show`
+action](https://torsion.org/borgmatic/reference/command-line/actions/config-show/).
+Here's an example:
+
+```bash
+borgmatic config show --json
+```
+
+That outputs borgmatic's entire configuration as JSON with one array element per
+configuration file.
+
+Or you can ask for the value of a particular option:
+
+```bash
+borgmatic config show --option repositories --json
+```
+
 ### Latest backups
 
 All borgmatic actions that accept an `--archive` flag allow you to specify an

docs/reference/command-line/actions/config-show.md

@@ -0,0 +1,17 @@
+---
+title: config show
+eleventyNavigation:
+  key: config show
+  parent: 🎬 Actions
+---
+
+{% include snippet/command-line/sample.md %}
+
+```
+{% include borgmatic/command-line/config-show.txt %}
+```
+
+
+## Related documentation
+
+ * [Scripting borgmatic](https://torsion.org/borgmatic/how-to/monitor-your-backups/#scripting-borgmatic)

docs/reference/command-line/actions/diff.md

@@ -0,0 +1,16 @@
+---
+title: diff
+eleventyNavigation:
+  key: diff
+  parent: 🎬 Actions
+---
+
+{% include snippet/command-line/sample.md %}
+
+```
+{% include borgmatic/command-line/diff.txt %}
+```
+
+## Related documentation
+
+* [Finding differences](https://torsion.org/borgmatic/how-to/inspect-your-backups/#finding-differences-between-two-archives)

docs/reference/configuration/command-hooks.md

@@ -12,11 +12,15 @@ list of `commands:` in your borgmatic configuration file. For example:
 ```yaml
 commands:
     - before: action
-      when: [create]
+      when: [check]  # This is an inline YAML sequence.
       run:
           - echo "Before create!"
+    - before: action
+      when: [create, prune]  # Also an inline YAML sequence.
+      run:
+          - echo "Before create or prune!"
     - after: action
-      when:
+      when:  # Multi-line YAML sequence, equivalent to "[create, prune]".
           - create
           - prune
       run:
@@ -29,7 +33,7 @@ commands:
 Each command in the `commands:` list has the following options:
 
  * `before` or `after`: Name for the point in borgmatic's execution that the commands should be run before or after, one of:
-    * `action` runs before or after each action for each repository. This replaces the deprecated `before_create`, `after_prune`, etc.
+    * `action` runs before or after each action for each repository. This replaces the deprecated `before_backup`, `after_prune`, etc.
     * `repository` runs before or after all actions for each repository. This replaces the deprecated `before_actions` and `after_actions`.
     * `configuration` runs before or after all actions and repositories in the current configuration file.
     * `everything` runs before or after all configuration files. Errors here do not trigger `error` hooks or the `fail` state in monitoring hooks. This replaces the deprecated `before_everything` and `after_everything`.

docs/reference/configuration/includes.md

@@ -317,3 +317,6 @@ a default location.
 
 This will output the merged configuration as borgmatic sees it, which can be
 helpful for understanding how your includes work in practice.
+
+Also see the [`config show`
+action](https://torsion.org/borgmatic/reference/command-line/actions/config-show/).

docs/reference/configuration/repositories.md

@@ -129,7 +129,9 @@ Note the lack of "`//`" after `s3:` or `b2:`.
 When selecting your cloud hosting provider, be aware that Amazon in particular
 has [financially
 supported](https://en.wikipedia.org/wiki/White_House_State_Ballroom) the Trump
-regime.
+regime. Additionally, U.S. Immigration and Customs Enforcement (ICE) is [powered
+by
+Amazon](https://medium.com/@noazureforapartheid/microsoft-powers-ice-why-doesnt-microsoft-want-to-talk-about-its-contracts-with-immigration-and-bc04fae8d43b).
 
 
 ## Related documentation

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "borgmatic"
-version = "2.1.0"
+version = "2.1.3"
 authors = [
   { name="Dan Helfman", email="witten@torsion.org" },
 ]

scripts/dev-docs

@@ -2,11 +2,17 @@
 
 set -e
 
-USER_PODMAN_SOCKET_PATH=/run/user/$UID/podman/podman.sock
-
-if [ -e "$USER_PODMAN_SOCKET_PATH" ]; then
-    export DOCKER_HOST="unix://$USER_PODMAN_SOCKET_PATH"
+if command -v podman-compose >/dev/null 2>&1; then
+    COMPOSE_CMD="podman-compose"
+    export CONTAINER_SOCKET_PATH="${XDG_RUNTIME_DIR:-/run/user/$UID}/podman/podman.sock"
+else
+    COMPOSE_CMD="docker-compose"
+    USER_PODMAN_SOCKET_PATH=/run/user/$UID/podman/podman.sock
     export CONTAINER_SOCKET_PATH="$USER_PODMAN_SOCKET_PATH"
+
+    if [ -e "$USER_PODMAN_SOCKET_PATH" ]; then
+        export DOCKER_HOST="unix://$USER_PODMAN_SOCKET_PATH"
+    fi
 fi
 
-BUILDKIT_PROGRESS=plain docker-compose --file docs/docker-compose.yaml up --build --force-recreate
+BUILDKIT_PROGRESS=plain $COMPOSE_CMD --file docs/docker-compose.yaml up --build --force-recreate

scripts/export-docs-from-image

@@ -9,7 +9,7 @@ if [ -e "$USER_PODMAN_SOCKET_PATH" ]; then
     export CONTAINER_SOCKET_PATH="$USER_PODMAN_SOCKET_PATH"
 fi
 
-podman build --tag borgmatic-release-docs --file docs/Dockerfile .
+podman build --tag borgmatic-release-docs --file docs/Dockerfile --env FONT=plain .
 docs_container_id=$(podman create borgmatic-release-docs)
 podman cp "$docs_container_id":/usr/share/nginx/html - > borgmatic-docs-dump.tar
 tar xf borgmatic-docs-dump.tar

tests/end-to-end/commands/fake_zfs.py

@@ -35,7 +35,7 @@ BUILTIN_DATASETS = (
         'used': '256K',
         'avail': '23.7M',
         'refer': '25K',
-        'canmount': 'on',
+        'canmount': 'noauto',
         'mountpoint': '/e2e/pool/dataset',
     },
 )

tests/integration/config/test_generate.py

@@ -292,6 +292,12 @@ def test_render_configuration_converts_configuration_to_yaml_string():
     assert yaml_string == 'foo: bar\n'
 
 
+def test_render_configuration_strips_ruamel_yaml_end_of_document_marker():
+    yaml_string = module.render_configuration(33)
+
+    assert yaml_string == '33\n'
+
+
 def test_write_configuration_does_not_raise():
     flexmock(os.path).should_receive('exists').and_return(False)
     flexmock(os).should_receive('makedirs')

tests/integration/config/test_schema.py

@@ -10,7 +10,7 @@ MAXIMUM_LINE_LENGTH = 80
 def test_schema_line_length_stays_under_limit():
     schema_file = open(borgmatic.config.validate.schema_filename())
 
-    for line in schema_file.readlines():
+    for line in schema_file:
         assert len(line.rstrip('\n')) <= MAXIMUM_LINE_LENGTH
 
 

tests/integration/test_execute.py

@@ -8,6 +8,68 @@ from flexmock import flexmock
 from borgmatic import execute as module
 
 
+def test_read_lines_yields_single_line():
+    process = subprocess.Popen(['echo', 'hi'], stdout=subprocess.PIPE)
+
+    assert tuple(module.read_lines(process.stdout, process)) == (('hi',),)
+
+
+def test_read_lines_yields_single_line_longer_than_chunk_size():
+    process = subprocess.Popen(
+        ['echo', 'this line is longer than the chunk size'], stdout=subprocess.PIPE
+    )
+
+    assert tuple(flexmock(module, READ_CHUNK_SIZE=16).read_lines(process.stdout, process)) == (
+        (),
+        (),
+        ('this line is longer than the chunk size',),
+    )
+
+
+def test_read_lines_yields_single_line_with_multibyte_unicode_character_spanning_chunk_boundary():
+    # In case it's not clear, "ñ" is a multi-byte UTF-8 character. The "a" shifts it over one byte
+    # so it straddles the chunk boundary.
+    process = subprocess.Popen(['echo', 'aññññññññññññññññññññññññññññññ'], stdout=subprocess.PIPE)
+
+    assert tuple(flexmock(module, READ_CHUNK_SIZE=16).read_lines(process.stdout, process)) == (
+        (),
+        (),
+        (),
+        ('aññññññññññññññññññññññññññññññ',),
+    )
+
+
+def test_read_lines_yields_multiple_lines():
+    process = subprocess.Popen(['echo', 'hi\nthere'], stdout=subprocess.PIPE)
+
+    assert tuple(module.read_lines(process.stdout, process)) == (('hi', 'there'),)
+
+
+def test_read_lines_yields_multiple_lines_plus_partial_line():
+    process = subprocess.Popen(['echo', '-n', 'hi\nthere\npartial'], stdout=subprocess.PIPE)
+
+    assert tuple(module.read_lines(process.stdout, process)) == (('hi', 'there'), ('partial',))
+
+
+def test_read_lines_with_longer_running_process_yields_many_lines():
+    process = subprocess.Popen(
+        [
+            sys.executable,
+            '-c',
+            "import random, string; print('\\n'.join(random.choice(string.ascii_letters) for _ in range(1000)))",
+        ],
+        stdout=subprocess.PIPE,
+    )
+
+    assert tuple(module.read_lines(process.stdout, process))
+
+
+def test_read_lines_yields_nothing():
+    process = subprocess.Popen(['echo', '-n'], stdout=subprocess.PIPE)
+
+    assert tuple(module.read_lines(process.stdout, process)) == ()
+
+
 def test_log_outputs_logs_each_line_separately():
     hi_record = flexmock(
         msg='hi',
@@ -269,7 +331,7 @@ def test_log_outputs_kills_other_processes_and_raises_when_one_errors():
         other_process,
         (),
     ).and_return((other_process.stdout,))
-    flexmock(other_process).should_receive('kill').once()
+    flexmock(other_process).should_call('kill').once()
 
     with pytest.raises(subprocess.CalledProcessError) as error:
         tuple(
@@ -291,12 +353,6 @@ def test_log_outputs_kills_other_processes_and_returns_when_one_exits_with_warni
     flexmock(module).should_receive('command_for_process').and_return('grep')
 
     process = subprocess.Popen(['grep'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
-    flexmock(module).should_receive('interpret_exit_code').with_args(
-        ['grep'],
-        None,
-        'borg',
-        None,
-    ).and_return(module.Exit_status.SUCCESS)
     flexmock(module).should_receive('interpret_exit_code').with_args(
         ['grep'],
         2,
@@ -313,7 +369,7 @@ def test_log_outputs_kills_other_processes_and_returns_when_one_exits_with_warni
         None,
         'borg',
         None,
-    ).and_return(module.Exit_status.SUCCESS)
+    ).and_return(module.Exit_status.STILL_RUNNING)
     flexmock(module).should_receive('output_buffers_for_process').with_args(process, ()).and_return(
         (process.stdout,),
     )
@@ -321,7 +377,7 @@ def test_log_outputs_kills_other_processes_and_returns_when_one_exits_with_warni
         other_process,
         (),
     ).and_return((other_process.stdout,))
-    flexmock(other_process).should_receive('kill').once()
+    flexmock(other_process).should_call('kill').once()
 
     assert (
         tuple(
@@ -370,7 +426,15 @@ def test_log_outputs_vents_other_processes_when_one_exits():
         other_process,
         (process.stdout,),
     ).and_return((other_process.stdout,))
-    flexmock(process.stdout).should_call('readline').at_least().once()
+    flexmock(module.os).should_call('read').with_args(
+        process.stderr.fileno(), int
+    ).at_least().once()
+    flexmock(module.os).should_call('read').with_args(
+        process.stdout.fileno(), int
+    ).at_least().once()
+    flexmock(module.os).should_call('read').with_args(
+        other_process.stdout.fileno(), int
+    ).at_least().once()
 
     assert (
         tuple(
@@ -433,12 +497,6 @@ def test_log_outputs_truncates_long_error_output():
     flexmock(module).should_receive('command_for_process').and_return('grep')
 
     process = subprocess.Popen(['grep'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
-    flexmock(module).should_receive('interpret_exit_code').with_args(
-        ['grep'],
-        None,
-        'borg',
-        None,
-    ).and_return(module.Exit_status.SUCCESS)
     flexmock(module).should_receive('interpret_exit_code').with_args(
         ['grep'],
         2,
@@ -487,8 +545,8 @@ def test_log_outputs_with_unfinished_process_re_polls():
     flexmock(module.logger).should_receive('log').never()
     flexmock(module).should_receive('interpret_exit_code').and_return(module.Exit_status.SUCCESS)
 
-    process = subprocess.Popen(['true'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
-    flexmock(process).should_receive('poll').and_return(None).and_return(0).times(3)
+    process = subprocess.Popen(['sleep', '0.001'], stdout=subprocess.PIPE)
+    flexmock(process).should_call('poll').at_least().times(3)
     flexmock(module).should_receive('output_buffers_for_process').and_return((process.stdout,))
 
     assert (
@@ -503,3 +561,10 @@ def test_log_outputs_with_unfinished_process_re_polls():
         )
         == ()
     )
+
+
+def test_read_lines_uses_system_locale_when_decoding_output():
+    flexmock(module.locale).should_receive('getpreferredencoding').and_return('ISO-8859-1')
+    process = subprocess.Popen(['echo', b'\xc4pple'], stdout=subprocess.PIPE)
+
+    assert tuple(module.read_lines(process.stdout, process)) == (('\xc4pple',),)

tests/unit/actions/config/test_show.py

@@ -0,0 +1,79 @@
+from flexmock import flexmock
+
+import borgmatic.logger
+from borgmatic.actions.config import show as module
+
+
+def test_run_show_with_single_configuration_file_does_not_separate_output():
+    log_lines = []
+    borgmatic.logger.add_custom_log_levels()
+
+    def fake_logger_answer(message):
+        log_lines.append(message)
+
+    flexmock(module.logger).should_receive('answer').replace_with(fake_logger_answer)
+    show_arguments = flexmock(option=None, json=False)
+    flexmock(module.borgmatic.config.generate).should_receive('render_configuration').and_return(
+        'output'
+    )
+
+    module.run_show(show_arguments, configs={'test.yaml': {}})
+
+    assert log_lines == ['output']
+
+
+def test_run_show_with_multiple_configuration_files_separates_output():
+    log_lines = []
+    borgmatic.logger.add_custom_log_levels()
+
+    def fake_logger_answer(message):
+        log_lines.append(message)
+
+    flexmock(module.logger).should_receive('answer').replace_with(fake_logger_answer)
+    show_arguments = flexmock(option=None, json=False)
+    flexmock(module.borgmatic.config.generate).should_receive('render_configuration').and_return(
+        'output'
+    ).and_return('other')
+
+    module.run_show(show_arguments, configs={'test.yaml': {}, 'other.yaml': {}})
+
+    assert log_lines == ['---', 'output', '---', 'other']
+
+
+def test_run_show_with_option_limits_output():
+    log_lines = []
+    borgmatic.logger.add_custom_log_levels()
+
+    def fake_logger_answer(message):
+        log_lines.append(message)
+
+    flexmock(module.logger).should_receive('answer').replace_with(fake_logger_answer)
+    show_arguments = flexmock(option='foo', json=False)
+    flexmock(module.borgmatic.config.generate).should_receive('render_configuration').with_args(
+        33
+    ).and_return('33')
+    flexmock(module.borgmatic.config.generate).should_receive('render_configuration').with_args(
+        None
+    ).and_return('null')
+
+    module.run_show(show_arguments, configs={'test.yaml': {'foo': 33, 'bar': 44}, 'other.yaml': {}})
+
+    assert log_lines == ['---', '33', '---', 'null']
+
+
+def test_run_show_with_json_outputs_json():
+    flexmock(borgmatic.logger).should_receive('add_custom_log_levels')
+    show_arguments = flexmock(option=None, json=True)
+    flexmock(module.sys.stdout).should_receive('write').with_args(
+        '[{"foo": 33}, {"bar": 44}]'
+    ).once()
+
+    module.run_show(show_arguments, configs={'test.yaml': {'foo': 33}, 'other.yaml': {'bar': 44}})
+
+
+def test_run_show_with_json_and_option_limits_json():
+    flexmock(borgmatic.logger).should_receive('add_custom_log_levels')
+    show_arguments = flexmock(option='foo', json=True)
+    flexmock(module.sys.stdout).should_receive('write').with_args('[33, null]').once()
+
+    module.run_show(show_arguments, configs={'test.yaml': {'foo': 33}, 'other.yaml': {'bar': 44}})

tests/unit/actions/config/test_validate.py

@@ -5,13 +5,13 @@ from borgmatic.actions.config import validate as module
 
 def test_run_validate_does_not_raise():
     validate_arguments = flexmock(show=False)
-    flexmock(module.borgmatic.config.generate).should_receive('render_configuration')
+    flexmock(module.borgmatic.config.generate).should_receive('render_configuration').and_return('')
 
     module.run_validate(validate_arguments, flexmock())
 
 
 def test_run_validate_with_show_does_not_raise():
     validate_arguments = flexmock(show=True)
-    flexmock(module.borgmatic.config.generate).should_receive('render_configuration')
+    flexmock(module.borgmatic.config.generate).should_receive('render_configuration').and_return('')
 
     module.run_validate(validate_arguments, {'test.yaml': flexmock(), 'other.yaml': flexmock()})

tests/unit/actions/test_check.py

@@ -986,6 +986,65 @@ def test_collect_spot_check_source_paths_uses_working_directory():
     ) == ('foo', 'bar')
 
 
+def test_collect_spot_check_source_paths_deduplicates_borg_output_paths():
+    flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks').and_return(
+        {'hook1': False, 'hook2': True},
+    )
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(
+        flexmock(),
+    )
+    flexmock(module.borgmatic.actions.pattern).should_receive('collect_patterns').and_return(
+        (Pattern('collected'),),
+    )
+    flexmock(module.borgmatic.actions.pattern).should_receive('process_patterns').with_args(
+        (
+            Pattern('collected', source=module.borgmatic.borg.pattern.Pattern_source.HOOK),
+            Pattern('extra.yaml', source=module.borgmatic.borg.pattern.Pattern_source.INTERNAL),
+        ),
+        config=object,
+        working_directory=None,
+    ).and_return(
+        [Pattern('foo'), Pattern('bar')],
+    )
+    flexmock(module.borgmatic.borg.create).should_receive('make_base_create_command').with_args(
+        dry_run=True,
+        repository_path='repo',
+        config=object,
+        patterns=[Pattern('foo'), Pattern('bar')],
+        local_borg_version=object,
+        global_arguments=object,
+        borgmatic_runtime_directory='/run/borgmatic',
+        local_path=object,
+        remote_path=object,
+        stream_processes=True,
+    ).and_return((('borg', 'create'), ('repo::archive',), flexmock()))
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        flexmock(),
+    )
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.execute).should_receive(
+        'execute_command_and_capture_output',
+    ).and_yield(
+        'warning: stuff',
+        '- /etc/path',
+        '+ /etc/other',
+        '? /nope',
+        '- /etc/path',
+    )
+    flexmock(module.os.path).should_receive('isfile').and_return(True)
+
+    assert module.collect_spot_check_source_paths(
+        repository={'path': 'repo'},
+        config={'working_directory': '/'},
+        local_borg_version=flexmock(),
+        global_arguments=flexmock(),
+        local_path=flexmock(),
+        remote_path=flexmock(),
+        borgmatic_runtime_directory='/run/borgmatic',
+        bootstrap_config_paths=('extra.yaml',),
+    ) == ('/etc/path', '/etc/other')
+
+
 def test_compare_spot_check_hashes_returns_paths_having_failing_hashes():
     flexmock(module.random).should_receive('SystemRandom').and_return(
         flexmock(sample=lambda population, count: population[:count]),
@@ -1002,8 +1061,9 @@ def test_compare_spot_check_hashes_returns_paths_having_failing_hashes():
         'hash2  /bar',
     )
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'nothash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {},
+        {'xxh64': 'nothash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1045,8 +1105,8 @@ def test_compare_spot_check_hashes_handles_weird_backslashed_hashes_from_xxh64su
         '\\hash2  /bar',
     )
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'nothash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'nothash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1088,8 +1148,8 @@ def test_compare_spot_check_hashes_handles_incorrect_path_names_from_xxh64sum():
         'hash2  /bar/wrong/path',
     )
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'nothash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'nothash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1115,6 +1175,59 @@ def test_compare_spot_check_hashes_handles_incorrect_path_names_from_xxh64sum():
     ) == ('/bar',)
 
 
+def test_compare_spot_check_hashes_with_xxh64sum_failure_falls_back_to_individual_file_hashing():
+    flexmock(module.random).should_receive('SystemRandom').and_return(
+        flexmock(sample=lambda population, count: population[:count]),
+    )
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(
+        None,
+    )
+    flexmock(module.os.path).should_receive('exists').and_return(True)
+    flexmock(module.os.path).should_receive('islink').and_return(False)
+    flexmock(module.borgmatic.execute).should_receive(
+        'execute_command_and_capture_output',
+    ).with_args(('xxh64sum', '/foo', '/bar'), working_directory=None).and_raise(
+        module.subprocess.CalledProcessError(1, 'wtf')
+    )
+    flexmock(module.borgmatic.execute).should_receive(
+        'execute_command_and_capture_output',
+    ).with_args(('xxh64sum', '/foo'), working_directory=None).and_raise(
+        module.subprocess.CalledProcessError(1, 'wtf')
+    ).once()
+    flexmock(module.borgmatic.execute).should_receive(
+        'execute_command_and_capture_output',
+    ).with_args(('xxh64sum', '/bar'), working_directory=None).and_yield(
+        'hash2  /bar',
+    ).once()
+
+    flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'hash2', 'path': 'bar', 'linktarget': ''},
+    )
+
+    assert module.compare_spot_check_hashes(
+        repository={'path': 'repo'},
+        archive='archive',
+        config={
+            'checks': [
+                {
+                    'name': 'archives',
+                    'frequency': '2 weeks',
+                },
+                {
+                    'name': 'spot',
+                    'data_sample_percentage': 50,
+                },
+            ],
+        },
+        local_borg_version=flexmock(),
+        global_arguments=flexmock(),
+        local_path=flexmock(),
+        remote_path=flexmock(),
+        source_paths=('/foo', '/bar', '/baz', '/quux'),
+    ) == ('/foo',)
+
+
 def test_compare_spot_check_hashes_returns_relative_paths_having_failing_hashes():
     flexmock(module.random).should_receive('SystemRandom').and_return(
         flexmock(sample=lambda population, count: population[:count]),
@@ -1131,8 +1244,8 @@ def test_compare_spot_check_hashes_returns_relative_paths_having_failing_hashes(
         'hash2  bar',
     )
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'nothash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'nothash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1174,8 +1287,8 @@ def test_compare_spot_check_hashes_handles_data_sample_percentage_above_100():
         'hash2  /bar',
     )
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'nothash1', 'path': 'foo'},
-        {'xxh64': 'nothash2', 'path': 'bar'},
+        {'xxh64': 'nothash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'nothash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1217,8 +1330,8 @@ def test_compare_spot_check_hashes_uses_xxh64sum_command_option():
         working_directory=None,
     ).and_yield('hash1  /foo', 'hash2  /bar')
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'nothash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'nothash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1257,7 +1370,7 @@ def test_compare_spot_check_hashes_considers_path_missing_from_archive_as_not_ma
         'hash2  /bar',
     )
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'}
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''}
     )
 
     assert module.compare_spot_check_hashes(
@@ -1279,6 +1392,50 @@ def test_compare_spot_check_hashes_considers_path_missing_from_archive_as_not_ma
     ) == ('/bar',)
 
 
+def test_compare_spot_check_hashes_skips_hardlink_path_in_archive():
+    flexmock(module.random).should_receive('SystemRandom').and_return(
+        flexmock(sample=lambda population, count: population[:count]),
+    )
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(
+        None,
+    )
+    flexmock(module.os.path).should_receive('exists').and_return(True)
+    flexmock(module.os.path).should_receive('islink').and_return(False)
+    flexmock(module.borgmatic.execute).should_receive(
+        'execute_command_and_capture_output',
+    ).with_args(('xxh64sum', '/foo', '/bar', '/link'), working_directory=None).and_yield(
+        'hash1  /foo',
+        'hash2  /bar',
+        'hash1  /link',
+    )
+    flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'hash2', 'path': 'bar', 'linktarget': ''},
+        {'xxh64': '', 'path': 'link', 'linktarget': 'foo'},
+    )
+
+    assert (
+        module.compare_spot_check_hashes(
+            repository={'path': 'repo'},
+            archive='archive',
+            config={
+                'checks': [
+                    {
+                        'name': 'spot',
+                        'data_sample_percentage': 100,
+                    },
+                ],
+            },
+            local_borg_version=flexmock(),
+            global_arguments=flexmock(),
+            local_path=flexmock(),
+            remote_path=flexmock(),
+            source_paths=('/foo', '/bar', '/link'),
+        )
+        == ()
+    )
+
+
 def test_compare_spot_check_hashes_considers_symlink_path_as_not_matching():
     flexmock(module.random).should_receive('SystemRandom').and_return(
         flexmock(sample=lambda population, count: population[:count]),
@@ -1293,8 +1450,8 @@ def test_compare_spot_check_hashes_considers_symlink_path_as_not_matching():
         'execute_command_and_capture_output',
     ).with_args(('xxh64sum', '/foo'), working_directory=None).and_yield('hash1  /foo')
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'hash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'hash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1330,8 +1487,8 @@ def test_compare_spot_check_hashes_considers_non_existent_path_as_not_matching()
         'execute_command_and_capture_output',
     ).with_args(('xxh64sum', '/foo'), working_directory=None).and_yield('hash1  /foo')
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'hash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'hash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1376,11 +1533,11 @@ def test_compare_spot_check_hashes_with_too_many_paths_feeds_them_to_commands_in
         'hash4  /quux',
     )
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'hash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'hash2', 'path': 'bar', 'linktarget': ''},
     ).and_yield(
-        {'xxh64': 'hash3', 'path': 'baz'},
-        {'xxh64': 'nothash4', 'path': 'quux'},
+        {'xxh64': 'hash3', 'path': 'baz', 'linktarget': ''},
+        {'xxh64': 'nothash4', 'path': 'quux', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(
@@ -1423,8 +1580,8 @@ def test_compare_spot_check_hashes_uses_working_directory_to_access_source_paths
         'hash2  bar',
     )
     flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
-        {'xxh64': 'hash1', 'path': 'foo'},
-        {'xxh64': 'nothash2', 'path': 'bar'},
+        {'xxh64': 'hash1', 'path': 'foo', 'linktarget': ''},
+        {'xxh64': 'nothash2', 'path': 'bar', 'linktarget': ''},
     )
 
     assert module.compare_spot_check_hashes(

tests/unit/actions/test_diff.py

@@ -0,0 +1,63 @@
+from flexmock import flexmock
+
+from borgmatic.actions import diff as module
+
+
+def test_run_diff_calls_borg_diff():
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').and_return(
+        'archive'
+    ).and_return('archive2')
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(
+        flexmock(),
+    )
+    flexmock(module.borgmatic.borg.diff).should_receive('diff').once()
+
+    module.borgmatic.actions.diff.run_diff(
+        repository={'path': 'repo'},
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            archive='archive',
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive=None,
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path=None,
+        remote_path=None,
+    )
+
+
+def test_run_diff_with_only_patterns():
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(
+        flexmock(),
+    )
+    flexmock(module.borgmatic.actions.pattern).should_receive('collect_patterns').once().and_return(
+        []
+    )
+    flexmock(module.borgmatic.actions.pattern).should_receive('process_patterns').once().and_return(
+        []
+    )
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').and_return(
+        'archive'
+    ).and_return('archive2')
+    flexmock(module.borgmatic.borg.diff).should_receive('diff').once()
+
+    module.borgmatic.actions.diff.run_diff(
+        repository={'path': 'repo'},
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            archive='archive',
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive=None,
+            only_patterns=True,
+        ),
+        global_arguments=flexmock(),
+        local_path=None,
+        remote_path=None,
+    )

tests/unit/actions/test_recreate.py

@@ -22,6 +22,7 @@ def test_run_recreate_does_not_raise():
         local_borg_version=None,
         recreate_arguments=flexmock(repository=flexmock(), archive=None),
         global_arguments=flexmock(),
+        dry_run_label='',
         local_path=None,
         remote_path=None,
     )
@@ -45,6 +46,7 @@ def test_run_recreate_with_archive_does_not_raise():
         local_borg_version=None,
         recreate_arguments=flexmock(repository=flexmock(), archive='test-archive'),
         global_arguments=flexmock(),
+        dry_run_label='',
         local_path=None,
         remote_path=None,
     )
@@ -69,6 +71,7 @@ def test_run_recreate_with_leftover_recreate_archive_raises():
             local_borg_version=None,
             recreate_arguments=flexmock(repository=flexmock(), archive='test-archive.recreate'),
             global_arguments=flexmock(),
+            dry_run_label='',
             local_path=None,
             remote_path=None,
         )
@@ -93,6 +96,7 @@ def test_run_recreate_with_latest_archive_resolving_to_leftover_recreate_archive
             local_borg_version=None,
             recreate_arguments=flexmock(repository=flexmock(), archive='latest'),
             global_arguments=flexmock(),
+            dry_run_label='',
             local_path=None,
             remote_path=None,
         )
@@ -122,6 +126,7 @@ def test_run_recreate_with_archive_already_exists_error_raises():
             local_borg_version=None,
             recreate_arguments=flexmock(repository=flexmock(), archive='test-archive', target=None),
             global_arguments=flexmock(),
+            dry_run_label='',
             local_path=None,
             remote_path=None,
         )
@@ -155,6 +160,7 @@ def test_run_recreate_with_target_and_archive_already_exists_error_raises():
                 target='target-archive',
             ),
             global_arguments=flexmock(),
+            dry_run_label='',
             local_path=None,
             remote_path=None,
         )
@@ -188,6 +194,7 @@ def test_run_recreate_with_other_called_process_error_passes_it_through():
                 target='target-archive',
             ),
             global_arguments=flexmock(),
+            dry_run_label='',
             local_path=None,
             remote_path=None,
         )

tests/unit/actions/test_restore.py

@@ -521,6 +521,7 @@ def test_collect_dumps_from_archive_with_dumps_metadata_parses_it():
     flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
         flexmock(stdout=flexmock(read=lambda: b''))
     )
+    flexmock(module.locale).should_receive('getpreferredencoding').and_return('UTF-8')
     dumps_metadata = [
         module.Dump('postgresql_databases', 'foo'),
         module.Dump('postgresql_databases', 'bar', 'host', 1234),

tests/unit/borg/test_check.py

@@ -24,7 +24,7 @@ def insert_execute_command_mock(
         environment=None,
         working_directory=working_directory,
         borg_local_path=command[0],
-        borg_exit_codes=borg_exit_codes,
+        borg_exit_codes=(borg_exit_codes or []) + [{'code': 1, 'treat_as': 'error'}],
     ).once()
 
 
@@ -335,7 +335,7 @@ def test_check_archives_with_progress_passes_through_to_borg():
         environment=None,
         working_directory=None,
         borg_local_path='borg',
-        borg_exit_codes=None,
+        borg_exit_codes=[{'code': 1, 'treat_as': 'error'}],
     ).once()
 
     module.check_archives(
@@ -371,7 +371,7 @@ def test_check_archives_with_log_json_and_progress_passes_through_both_to_borg()
         environment=None,
         working_directory=None,
         borg_local_path='borg',
-        borg_exit_codes=None,
+        borg_exit_codes=[{'code': 1, 'treat_as': 'error'}],
     ).once()
 
     module.check_archives(
@@ -407,7 +407,7 @@ def test_check_archives_with_repair_passes_through_to_borg():
         environment=None,
         working_directory=None,
         borg_local_path='borg',
-        borg_exit_codes=None,
+        borg_exit_codes=[{'code': 1, 'treat_as': 'error'}],
     ).once()
 
     module.check_archives(
@@ -443,7 +443,7 @@ def test_check_archives_with_log_json_and_repair_passes_through_both_to_borg():
         environment=None,
         working_directory=None,
         borg_local_path='borg',
-        borg_exit_codes=None,
+        borg_exit_codes=[{'code': 1, 'treat_as': 'error'}],
     ).once()
 
     module.check_archives(
@@ -479,7 +479,7 @@ def test_check_archives_with_max_duration_flag_passes_through_to_borg():
         environment=None,
         working_directory=None,
         borg_local_path='borg',
-        borg_exit_codes=None,
+        borg_exit_codes=[{'code': 1, 'treat_as': 'error'}],
     ).once()
 
     module.check_archives(
@@ -515,7 +515,7 @@ def test_check_archives_with_max_duration_option_passes_through_to_borg():
         environment=None,
         working_directory=None,
         borg_local_path='borg',
-        borg_exit_codes=None,
+        borg_exit_codes=[{'code': 1, 'treat_as': 'error'}],
     ).once()
 
     module.check_archives(
@@ -689,7 +689,7 @@ def test_check_archives_with_max_duration_flag_overrides_max_duration_option():
         environment=None,
         working_directory=None,
         borg_local_path='borg',
-        borg_exit_codes=None,
+        borg_exit_codes=[{'code': 1, 'treat_as': 'error'}],
     ).once()
 
     module.check_archives(
@@ -854,7 +854,7 @@ def test_check_archives_with_local_path_calls_borg_via_local_path():
 
 def test_check_archives_with_exit_codes_calls_borg_using_them():
     checks = {'repository'}
-    borg_exit_codes = flexmock()
+    borg_exit_codes = [{'code': 101, 'treat_as': 'error'}]
     config = {'borg_exit_codes': borg_exit_codes}
     flexmock(module).should_receive('make_check_name_flags').with_args(checks, ()).and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
@@ -1026,7 +1026,7 @@ def test_check_archives_with_match_archives_passes_through_to_borg():
         environment=None,
         working_directory=None,
         borg_local_path='borg',
-        borg_exit_codes=None,
+        borg_exit_codes=[{'code': 1, 'treat_as': 'error'}],
     ).once()
 
     module.check_archives(

tests/unit/borg/test_create.py

@@ -357,7 +357,8 @@ DEFAULT_ARCHIVE_NAME = '{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}'
 REPO_ARCHIVE = (f'repo::{DEFAULT_ARCHIVE_NAME}',)
 
 
-def test_make_base_create_produces_borg_command():
+def test_make_base_create_command_checks_root_patterns_exist_and_produces_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist').once()
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -386,7 +387,39 @@ def test_make_base_create_produces_borg_command():
     assert not pattern_file
 
 
+def test_make_base_create_command_without_check_all_root_patterns_exist_skips_check_and_produces_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist').never()
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
+    flexmock(module.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.flags).should_receive('make_repository_archive_flags').and_return(
+        (f'repo::{DEFAULT_ARCHIVE_NAME}',),
+    )
+    flexmock(module).should_receive('validate_planned_backup_paths').and_return(())
+
+    (create_flags, create_positional_arguments, pattern_file) = module.make_base_create_command(
+        dry_run=False,
+        repository_path='repo',
+        config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'source_directories_must_exist': False,
+        },
+        patterns=[Pattern('foo'), Pattern('bar')],
+        local_borg_version='1.2.3',
+        global_arguments=flexmock(),
+        borgmatic_runtime_directory='/run/borgmatic',
+    )
+
+    assert create_flags == ('borg', 'create', '--log-json')
+    assert create_positional_arguments == REPO_ARCHIVE
+    assert not pattern_file
+
+
 def test_make_base_create_command_includes_patterns_file_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     mock_pattern_file = flexmock(name='/tmp/patterns')
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
@@ -424,6 +457,7 @@ def test_make_base_create_command_includes_patterns_file_in_borg_command():
 
 
 def test_make_base_create_command_with_store_config_false_omits_config_files():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -482,6 +516,7 @@ def test_make_base_create_command_with_store_config_false_omits_config_files():
         ('flags', True, False, ()),
         ('flags', False, True, ('--noflags',)),
         ('flags', False, False, ('--nobsdflags',)),
+        ('files_changed', 'mtime', True, ('--files-changed', 'mtime')),
         ('files_cache', 'ctime,size', True, ('--files-cache', 'ctime,size')),
         ('umask', 740, True, ('--umask', '740')),
         ('lock_wait', 5, True, ('--lock-wait', '5')),
@@ -493,6 +528,7 @@ def test_make_base_create_command_includes_configuration_option_as_command_flag(
     feature_available,
     option_flags,
 ):
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -526,6 +562,7 @@ def test_make_base_create_command_includes_configuration_option_as_command_flag(
 
 
 def test_make_base_create_command_with_progress_omits_log_json_from_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -560,6 +597,7 @@ def test_make_base_create_command_with_progress_omits_log_json_from_borg_command
 
 
 def test_make_base_create_command_with_log_json_and_progress_includes_log_json_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -595,6 +633,7 @@ def test_make_base_create_command_with_log_json_and_progress_includes_log_json_i
 
 
 def test_make_base_create_command_includes_dry_run_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -628,6 +667,7 @@ def test_make_base_create_command_includes_dry_run_in_borg_command():
 
 
 def test_make_base_create_command_includes_comment_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -662,6 +702,7 @@ def test_make_base_create_command_includes_comment_in_borg_command():
 
 
 def test_make_base_create_command_includes_local_path_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -695,6 +736,7 @@ def test_make_base_create_command_includes_local_path_in_borg_command():
 
 
 def test_make_base_create_command_includes_remote_path_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -728,6 +770,7 @@ def test_make_base_create_command_includes_remote_path_in_borg_command():
 
 
 def test_make_base_create_command_includes_list_flags_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -763,6 +806,7 @@ def test_make_base_create_command_includes_list_flags_in_borg_command():
 def test_make_base_create_command_with_stream_processes_ignores_read_special_false_and_excludes_special_files():
     patterns = [Pattern('foo'), Pattern('bar')]
     patterns_file = flexmock(name='patterns')
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').with_args(
         patterns,
@@ -822,6 +866,7 @@ def test_make_base_create_command_with_stream_processes_ignores_read_special_fal
 
 
 def test_make_base_create_command_without_patterns_and_with_stream_processes_ignores_read_special_false_and_excludes_special_files():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').with_args(
         [],
@@ -881,6 +926,7 @@ def test_make_base_create_command_without_patterns_and_with_stream_processes_ign
 
 
 def test_make_base_create_command_with_stream_processes_and_read_special_true_skips_special_files_excludes():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -916,6 +962,7 @@ def test_make_base_create_command_with_stream_processes_and_read_special_true_sk
 
 
 def test_make_base_create_command_includes_archive_name_format_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -949,6 +996,7 @@ def test_make_base_create_command_includes_archive_name_format_in_borg_command()
 
 
 def test_make_base_create_command_includes_default_archive_name_format_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -982,6 +1030,7 @@ def test_make_base_create_command_includes_default_archive_name_format_in_borg_c
 
 def test_make_base_create_command_includes_archive_name_format_with_placeholders_in_borg_command():
     repository_archive_pattern = 'repo::Documents_{hostname}-{now}'
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
     flexmock(module.flags).should_receive('get_default_archive_name_format').and_return(
@@ -1015,6 +1064,7 @@ def test_make_base_create_command_includes_archive_name_format_with_placeholders
 
 def test_make_base_create_command_includes_repository_and_archive_name_format_with_placeholders_in_borg_command():
     repository_archive_pattern = '{fqdn}::Documents_{hostname}-{now}'
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
     flexmock(module.flags).should_receive('get_default_archive_name_format').and_return(
@@ -1047,6 +1097,7 @@ def test_make_base_create_command_includes_repository_and_archive_name_format_wi
 
 
 def test_make_base_create_command_includes_archive_suffix_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -1076,6 +1127,7 @@ def test_make_base_create_command_includes_archive_suffix_in_borg_command():
 
 
 def test_make_base_create_command_includes_extra_borg_options_in_borg_command():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -1116,6 +1168,7 @@ def test_make_base_create_command_includes_extra_borg_options_in_borg_command():
 
 
 def test_make_base_create_command_with_unsafe_skip_path_validation_before_create_skips_validation():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')
@@ -1128,7 +1181,6 @@ def test_make_base_create_command_with_unsafe_skip_path_validation_before_create
         (f'repo::{module.flags.get_default_archive_name_format()}',),
     )
     flexmock(module).should_receive('validate_planned_backup_paths').never()
-    flexmock(module.logger).should_receive('warning').once()
 
     module.make_base_create_command(
         dry_run=False,
@@ -1146,6 +1198,7 @@ def test_make_base_create_command_with_unsafe_skip_path_validation_before_create
 
 
 def test_make_base_create_command_without_unsafe_skip_path_validation_before_create_calls_validation():
+    flexmock(module.borgmatic.borg.pattern).should_receive('check_all_root_patterns_exist')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
     flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('FOO')

tests/unit/borg/test_diff.py

@@ -0,0 +1,849 @@
+import logging
+
+from flexmock import flexmock
+
+from borgmatic.borg import diff as module
+
+LOGGING_ANSWER = flexmock()
+
+
+def test_diff_calls_borg_with_archives():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_local_path_calls_borg_with_it():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg6',
+            'diff',
+            '--log-json',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg6',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg6',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_remote_path_calls_borg_with_it():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--remote-path',
+            'borg7',
+            '--log-json',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path='borg7',
+        patterns=[],
+    )
+
+
+def test_diff_with_lock_wait_calls_borg_with_it():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--lock-wait',
+            '5',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={'lock_wait': 5},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_log_level_info_calls_borg_with_info_flag():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.logger).should_receive('getEffectiveLevel').and_return(logging.INFO)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--info',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_log_level_debug_calls_borg_with_debug_flags():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.logger).should_receive('isEnabledFor').and_return(logging.DEBUG)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--debug',
+            '--show-rc',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_only_patterns_calls_borg_with_configured_pattern_paths():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    flexmock(module).should_receive('write_patterns_file').and_return(
+        flexmock(name='/tmp/test_patterns')
+    ).once()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--patterns-from',
+            '/tmp/test_patterns',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=True,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_exclude_config_calls_borg_with_exclude_flags():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(
+        ('--exclude', 'stuff')
+    )
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--exclude',
+            'stuff',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_numeric_ids_calls_borg_with_numeric_ids_flag():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--numeric-ids',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={'numeric_ids': True},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_numeric_ids_and_feature_not_available_calls_borg_with_numeric_owner_flag():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').with_args(
+        module.borgmatic.borg.feature.Feature.NUMERIC_IDS, object
+    ).and_return(False)
+    flexmock(module.borgmatic.borg.feature).should_receive('available').with_args(
+        module.borgmatic.borg.feature.Feature.SEPARATE_REPOSITORY_ARCHIVE, object
+    ).and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--numeric-owner',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={'numeric_ids': True},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_same_chunker_params_calls_borg_with_it():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--same-chunker-params',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=True,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_sort_keys_calls_borg_with_formatted_sort_by_flags():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--sort-by',
+            'foo,bar,baz',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=['foo', 'bar', 'baz'],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_content_only_calls_borg_with_it():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--content-only',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=True,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_with_extra_borg_options_calls_borg_with_them():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
+        ('--repo', 'repo')
+    )
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            '--extra',
+            '--option',
+            '--repo',
+            'repo',
+            'archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={'extra_borg_options': {'diff': '--extra --option'}},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )
+
+
+def test_diff_without_separate_repository_archive_feature_available_calls_borg_joined_repository_archive():
+    flexmock(module.logging).ANSWER = LOGGING_ANSWER
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
+        flexmock(name='test')
+    )
+    flexmock(module.borgmatic.borg.feature).should_receive('available').with_args(
+        module.borgmatic.borg.feature.Feature.NUMERIC_IDS, object
+    ).and_return(True)
+    flexmock(module.borgmatic.borg.feature).should_receive('available').with_args(
+        module.borgmatic.borg.feature.Feature.SEPARATE_REPOSITORY_ARCHIVE, object
+    ).and_return(False)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').never()
+    flexmock(module.borgmatic.borg.flags).should_receive(
+        'make_repository_archive_flags'
+    ).and_return(('repo::archive',))
+    environment = flexmock()
+    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
+        environment
+    )
+    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
+        full_command=(
+            'borg',
+            'diff',
+            '--log-json',
+            'repo::archive',
+            'archive2',
+        ),
+        output_log_level=LOGGING_ANSWER,
+        environment=environment,
+        working_directory=None,
+        borg_local_path='borg',
+        borg_exit_codes=None,
+    ).once()
+
+    module.borgmatic.borg.diff.diff(
+        repository='repo',
+        archive='archive',
+        second_archive='archive2',
+        config={},
+        local_borg_version=None,
+        diff_arguments=flexmock(
+            same_chunker_params=False,
+            sort_keys=[],
+            content_only=False,
+            second_archive='archive2',
+            only_patterns=False,
+        ),
+        global_arguments=flexmock(),
+        local_path='borg',
+        remote_path=None,
+        patterns=[],
+    )

tests/unit/borg/test_environment.py

@@ -90,11 +90,6 @@ def test_make_environment_without_configuration_sets_certain_environment_variabl
     assert environment == {
         'USER': 'root',
         'BORG_EXIT_CODES': 'modern',
-        'BORG_RELOCATED_REPO_ACCESS_IS_OK': 'NO',
-        'BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK': 'NO',
-        'BORG_USE_CHUNKS_ARCHIVE': 'NO',
-        'BORG_DEBUG_PASSPHRASE': 'NO',
-        'BORG_DISPLAY_PASSPHRASE': 'NO',
     }
 
 
@@ -107,6 +102,7 @@ def test_make_environment_without_configuration_passes_through_default_environme
             'BORG_USE_CHUNKS_ARCHIVE': 'yup',
             'BORG_DEBUG_PASSPHRASE': 'nah',
             'BORG_DISPLAY_PASSPHRASE': 'yup',
+            'BORG_MSGPACK_VERSION_CHECK': 'yup',
         },
     )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
@@ -122,6 +118,7 @@ def test_make_environment_without_configuration_passes_through_default_environme
         'BORG_USE_CHUNKS_ARCHIVE': 'yup',
         'BORG_DEBUG_PASSPHRASE': 'nah',
         'BORG_DISPLAY_PASSPHRASE': 'yup',
+        'BORG_MSGPACK_VERSION_CHECK': 'yup',
         'BORG_EXIT_CODES': 'modern',
     }
 

tests/unit/borg/test_recreate.py

@@ -20,44 +20,6 @@ def insert_execute_command_mock(command, working_directory=None, borg_exit_codes
     ).once()
 
 
-def test_recreate_archive_dry_run_skips_execution():
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
-    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
-    flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_match_archives_flags').and_return(())
-    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
-    flexmock(module.borgmatic.borg.flags).should_receive(
-        'make_repository_archive_flags',
-    ).and_return(
-        (
-            '--repo',
-            'repo',
-        ),
-    )
-    flexmock(module.borgmatic.execute).should_receive('execute_command').never()
-
-    recreate_arguments = flexmock(
-        repository=flexmock(),
-        list=None,
-        target=None,
-        comment=None,
-        timestamp=None,
-        match_archives=None,
-    )
-
-    result = module.recreate_archive(
-        repository='repo',
-        archive='archive',
-        config={},
-        local_borg_version='1.2.3',
-        recreate_arguments=recreate_arguments,
-        global_arguments=flexmock(dry_run=True),
-        local_path='borg',
-    )
-
-    assert result is None
-
-
 def test_recreate_calls_borg_with_required_flags():
     flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
@@ -93,6 +55,41 @@ def test_recreate_calls_borg_with_required_flags():
     )
 
 
+def test_recreate_with_dry_run_calls_borg_with_dry_run_flag():
+    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
+    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)
+    flexmock(module.borgmatic.borg.flags).should_receive('make_list_filter_flags').and_return('')
+    flexmock(module.borgmatic.borg.flags).should_receive('make_match_archives_flags').and_return(())
+    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
+    flexmock(module.borgmatic.borg.flags).should_receive(
+        'make_repository_archive_flags',
+    ).and_return(
+        (
+            '--repo',
+            'repo',
+        ),
+    )
+    insert_execute_command_mock(('borg', 'recreate', '--log-json', '--dry-run', '--repo', 'repo'))
+
+    module.recreate_archive(
+        repository='repo',
+        archive='archive',
+        config={},
+        local_borg_version='1.2.3',
+        recreate_arguments=flexmock(
+            list=None,
+            target=None,
+            comment=None,
+            timestamp=None,
+            match_archives=None,
+        ),
+        global_arguments=flexmock(dry_run=True),
+        local_path='borg',
+        remote_path=None,
+        patterns=None,
+    )
+
+
 def test_recreate_with_remote_path():
     flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(None)

tests/unit/commands/test_arguments.py

@@ -329,10 +329,12 @@ def test_parse_arguments_for_actions_consumes_action_arguments_after_action_name
     remaining = flexmock()
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {action: action_namespace},
-        )
-        or remaining,
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {action: action_namespace},
+            )
+            or remaining
+        ),
     )
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {'action': flexmock(), 'other': flexmock()}
@@ -355,10 +357,12 @@ def test_parse_arguments_for_actions_consumes_action_arguments_with_alias():
     remaining = flexmock()
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {canonical or action: action_namespace},
-        )
-        or remaining,
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {canonical or action: action_namespace},
+            )
+            or remaining
+        ),
     )
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {
@@ -387,10 +391,12 @@ def test_parse_arguments_for_actions_consumes_multiple_action_arguments():
     other_namespace = flexmock(bar=3)
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {action: action_namespace if action == 'action' else other_namespace},
-        )
-        or (),
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {action: action_namespace if action == 'action' else other_namespace},
+            )
+            or ()
+        ),
     ).and_return(('other', '--bar', '3')).and_return('action', '--foo', 'true')
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {
@@ -420,10 +426,12 @@ def test_parse_arguments_for_actions_respects_command_line_action_ordering():
     action_namespace = flexmock(foo=True)
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {action: other_namespace if action == 'other' else action_namespace},
-        )
-        or (),
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {action: other_namespace if action == 'other' else action_namespace},
+            )
+            or ()
+        ),
     ).and_return(('action',)).and_return(('other', '--foo', 'true'))
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {
@@ -458,10 +466,12 @@ def test_parse_arguments_for_actions_applies_default_action_parsers():
 
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {action: namespaces.get(action)},
-        )
-        or (),
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {action: namespaces.get(action)},
+            )
+            or ()
+        ),
     ).and_return(())
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {
@@ -488,10 +498,12 @@ def test_parse_arguments_for_actions_consumes_global_arguments():
     action_namespace = flexmock()
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {action: action_namespace},
-        )
-        or ('--verbosity', 'lots'),
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {action: action_namespace},
+            )
+            or ('--verbosity', 'lots')
+        ),
     )
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {
@@ -516,10 +528,12 @@ def test_parse_arguments_for_actions_passes_through_unknown_arguments_before_act
     action_namespace = flexmock()
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {action: action_namespace},
-        )
-        or ('--wtf', 'yes'),
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {action: action_namespace},
+            )
+            or ('--wtf', 'yes')
+        ),
     )
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {
@@ -544,10 +558,12 @@ def test_parse_arguments_for_actions_passes_through_unknown_arguments_after_acti
     action_namespace = flexmock()
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {action: action_namespace},
-        )
-        or ('--wtf', 'yes'),
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {action: action_namespace},
+            )
+            or ('--wtf', 'yes')
+        ),
     )
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {
@@ -572,10 +588,12 @@ def test_parse_arguments_for_actions_with_borg_action_skips_other_action_parsers
     action_namespace = flexmock(options=[])
     flexmock(module).should_receive('get_subaction_parsers').and_return({})
     flexmock(module).should_receive('parse_and_record_action_arguments').replace_with(
-        lambda unparsed, parsed, parser, action, canonical=None: parsed.update(
-            {action: action_namespace},
-        )
-        or (),
+        lambda unparsed, parsed, parser, action, canonical=None: (
+            parsed.update(
+                {action: action_namespace},
+            )
+            or ()
+        ),
     ).and_return(())
     flexmock(module).should_receive('get_subactions_for_actions').and_return({})
     action_parsers = {

tests/unit/commands/test_borgmatic.py

@@ -2058,6 +2058,43 @@ def test_collect_highlander_action_summary_logs_error_on_run_validate_failure():
     assert {log.levelno for log in logs} == {logging.CRITICAL}
 
 
+def test_collect_highlander_action_summary_logs_nothing_additional_for_success_with_show():
+    flexmock(module.borgmatic.actions.config.show).should_receive('run_show')
+    arguments = {
+        'show': flexmock(),
+        'global': flexmock(),
+    }
+
+    logs = tuple(
+        module.collect_highlander_action_summary_logs(
+            {'test.yaml': {}},
+            arguments=arguments,
+            configuration_parse_errors=False,
+        ),
+    )
+    assert not logs
+
+
+def test_collect_highlander_action_summary_logs_error_on_run_show_failure():
+    flexmock(module.borgmatic.actions.config.show).should_receive('run_show').and_raise(
+        ValueError,
+    )
+    arguments = {
+        'show': flexmock(),
+        'global': flexmock(),
+    }
+
+    logs = tuple(
+        module.collect_highlander_action_summary_logs(
+            {'test.yaml': {}},
+            arguments=arguments,
+            configuration_parse_errors=False,
+        ),
+    )
+
+    assert {log.levelno for log in logs} == {logging.CRITICAL}
+
+
 def test_collect_configuration_run_summary_logs_info_for_success():
     flexmock(module.validate).should_receive('guard_configuration_contains_repository')
     flexmock(module.command).should_receive('filter_hooks').with_args(
@@ -2652,3 +2689,27 @@ def test_get_singular_option_value_with_no_config_returns_none():
         )
         is None
     )
+
+
+def test_run_actions_runs_diff():
+    flexmock(module).should_receive('add_custom_log_levels')
+    flexmock(module).should_receive('get_skip_actions').and_return([])
+    flexmock(module.borgmatic.config.validate).should_receive('repositories_match').never()
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(
+        flexmock(),
+    )
+    flexmock(module.command).should_receive('Before_after_hooks').and_return(flexmock())
+    flexmock(borgmatic.actions.diff).should_receive('run_diff').once()
+
+    tuple(
+        module.run_actions(
+            arguments={'global': flexmock(dry_run=False), 'diff': flexmock()},
+            config_filename=flexmock(),
+            config={'repositories': []},
+            config_paths=[],
+            local_path=flexmock(),
+            remote_path=flexmock(),
+            local_borg_version=flexmock(),
+            repository={'path': 'repo'},
+        ),
+    )

tests/unit/hooks/data_source/test_bootstrap.py

@@ -1,13 +1,55 @@
 import sys
 
+import pytest
 from flexmock import flexmock
 
 from borgmatic.hooks.data_source import bootstrap as module
 
 
-def test_dump_data_sources_creates_manifest_file():
-    flexmock(module.os).should_receive('makedirs')
+def test_resolve_config_path_symlinks_passes_through_non_symlink():
+    flexmock(module.os.path).should_receive('abspath').replace_with(lambda path: path)
+    flexmock(module.os.path).should_receive('islink').and_return(False)
 
+    assert tuple(module.resolve_config_path_symlinks('test.yaml')) == ('test.yaml',)
+
+
+def test_resolve_config_path_symlinks_follows_each_symlink():
+    flexmock(module.os.path).should_receive('abspath').replace_with(lambda path: path)
+    flexmock(module.os.path).should_receive('islink').with_args('test.yaml').and_return(True)
+    flexmock(module.os.path).should_receive('islink').with_args('dest1.yaml').and_return(True)
+    flexmock(module.os.path).should_receive('islink').with_args('dest2.yaml').and_return(False)
+    flexmock(module.os).should_receive('readlink').with_args('test.yaml').and_return('dest1.yaml')
+    flexmock(module.os).should_receive('readlink').with_args('dest1.yaml').and_return('dest2.yaml')
+    flexmock(module.os).should_receive('readlink').with_args('dest2.yaml').never()
+
+    assert tuple(module.resolve_config_path_symlinks('test.yaml')) == (
+        'test.yaml',
+        'dest1.yaml',
+        'dest2.yaml',
+    )
+
+
+def test_resolve_config_path_symlinks_with_too_many_symlinks_raises():
+    flexmock(module).MAXIMUM_CONFIG_SYMLINKS_TO_FOLLOW = 2
+    flexmock(module.os.path).should_receive('abspath').replace_with(lambda path: path)
+    flexmock(module.os.path).should_receive('islink').with_args('test.yaml').and_return(True)
+    flexmock(module.os.path).should_receive('islink').with_args('dest1.yaml').and_return(True)
+    flexmock(module.os.path).should_receive('islink').with_args('dest2.yaml').and_return(True)
+    flexmock(module.os.path).should_receive('islink').with_args('dest3.yaml').never()
+    flexmock(module.os).should_receive('readlink').with_args('test.yaml').and_return('dest1.yaml')
+    flexmock(module.os).should_receive('readlink').with_args('dest1.yaml').and_return('dest2.yaml')
+    flexmock(module.os).should_receive('readlink').with_args('dest2.yaml').and_return('dest3.yaml')
+    flexmock(module.os).should_receive('readlink').with_args('dest3.yaml').never()
+
+    with pytest.raises(ValueError):
+        assert tuple(module.resolve_config_path_symlinks('test.yaml'))
+
+
+def test_dump_data_sources_creates_manifest_file():
+    flexmock(module).should_receive('resolve_config_path_symlinks').and_yield(
+        'test.yaml', 'linkdest.yaml'
+    )
+    flexmock(module.os).should_receive('makedirs')
     flexmock(module.importlib.metadata).should_receive('version').and_return('1.0.0')
     manifest_file = flexmock(
         __enter__=lambda *args: flexmock(write=lambda *args: None, close=lambda *args: None),
@@ -19,7 +61,7 @@ def test_dump_data_sources_creates_manifest_file():
         encoding='utf-8',
     ).and_return(manifest_file)
     flexmock(module.json).should_receive('dump').with_args(
-        {'borgmatic_version': '1.0.0', 'config_paths': ('test.yaml',)},
+        {'borgmatic_version': '1.0.0', 'config_paths': ('test.yaml', 'linkdest.yaml')},
         manifest_file,
     ).once()
     flexmock(module.borgmatic.hooks.data_source.config).should_receive('inject_pattern').with_args(
@@ -34,6 +76,12 @@ def test_dump_data_sources_creates_manifest_file():
             'test.yaml', source=module.borgmatic.borg.pattern.Pattern_source.HOOK
         ),
     ).once()
+    flexmock(module.borgmatic.hooks.data_source.config).should_receive('inject_pattern').with_args(
+        object,
+        module.borgmatic.borg.pattern.Pattern(
+            'linkdest.yaml', source=module.borgmatic.borg.pattern.Pattern_source.HOOK
+        ),
+    ).once()
 
     module.dump_data_sources(
         hook_config=None,
@@ -46,6 +94,7 @@ def test_dump_data_sources_creates_manifest_file():
 
 
 def test_dump_data_sources_with_store_config_files_false_does_not_create_manifest_file():
+    flexmock(module).should_receive('resolve_config_path_symlinks').and_yield('test.yaml')
     flexmock(module.os).should_receive('makedirs').never()
     flexmock(module.json).should_receive('dump').never()
     flexmock(module.borgmatic.hooks.data_source.config).should_receive('inject_pattern').never()
@@ -62,6 +111,7 @@ def test_dump_data_sources_with_store_config_files_false_does_not_create_manifes
 
 
 def test_dump_data_sources_with_dry_run_does_not_create_manifest_file():
+    flexmock(module).should_receive('resolve_config_path_symlinks').and_yield('test.yaml')
     flexmock(module.os).should_receive('makedirs').never()
     flexmock(module.json).should_receive('dump').never()
     flexmock(module.borgmatic.hooks.data_source.config).should_receive('inject_pattern').never()

tests/unit/hooks/monitoring/test_ntfy.py

@@ -1,10 +1,26 @@
 from enum import Enum
 
+import pytest
 from flexmock import flexmock
 
 import borgmatic.hooks.monitoring.monitor
 from borgmatic.hooks.monitoring import ntfy as module
 
+
+@pytest.mark.parametrize(
+    'tags_input,expected_tags_output',
+    (
+        (None, []),
+        ('', []),
+        ('foo', ['foo']),
+        (' foo ,  bar ,baz ', ['foo', 'bar', 'baz']),
+        ('foo,,bar,', ['foo', 'bar']),
+    ),
+)
+def test_convert_string_to_array(tags_input, expected_tags_output):
+    assert module.convert_string_to_array(tags_input) == expected_tags_output
+
+
 DEFAULT_BASE_URL = 'https://ntfy.sh'
 CUSTOM_BASE_URL = 'https://ntfy.example.com'
 TOPIC = 'borgmatic-unit-testing'
@@ -24,8 +40,8 @@ CUSTOM_MESSAGE_PAYLOAD = {
     'topic': TOPIC,
     'title': CUSTOM_MESSAGE_CONFIG['title'],
     'message': CUSTOM_MESSAGE_CONFIG['message'],
-    'priority': CUSTOM_MESSAGE_CONFIG['priority'],
-    'tags': CUSTOM_MESSAGE_CONFIG['tags'],
+    'priority': 1,
+    'tags': [CUSTOM_MESSAGE_CONFIG['tags']],
 }
 
 
@@ -34,13 +50,16 @@ def default_message_payload(state=Enum):
         'topic': TOPIC,
         'title': f'A borgmatic {state.name} event happened',
         'message': f'A borgmatic {state.name} event happened',
-        'priority': 'default',
-        'tags': 'borgmatic',
+        'priority': 3,
+        'tags': ['borgmatic'],
     }
 
 
 def test_ping_monitor_minimal_config_hits_hosted_ntfy_on_fail():
     hook_config = {'topic': TOPIC}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -67,6 +86,9 @@ def test_ping_monitor_with_access_token_hits_hosted_ntfy_on_fail():
         'topic': TOPIC,
         'access_token': 'abc123',
     }
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -95,6 +117,9 @@ def test_ping_monitor_with_username_password_and_access_token_ignores_username_p
         'password': 'fakepassword',
         'access_token': 'abc123',
     }
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -123,6 +148,9 @@ def test_ping_monitor_with_username_password_hits_hosted_ntfy_on_fail():
         'username': 'testuser',
         'password': 'fakepassword',
     }
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -146,6 +174,9 @@ def test_ping_monitor_with_username_password_hits_hosted_ntfy_on_fail():
 
 def test_ping_monitor_with_password_but_no_username_warns():
     hook_config = {'topic': TOPIC, 'password': 'fakepassword'}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -170,6 +201,9 @@ def test_ping_monitor_with_password_but_no_username_warns():
 
 def test_ping_monitor_with_username_but_no_password_warns():
     hook_config = {'topic': TOPIC, 'username': 'testuser'}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -194,6 +228,9 @@ def test_ping_monitor_with_username_but_no_password_warns():
 
 def test_ping_monitor_minimal_config_does_not_hit_hosted_ntfy_on_start():
     hook_config = {'topic': TOPIC}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -211,6 +248,9 @@ def test_ping_monitor_minimal_config_does_not_hit_hosted_ntfy_on_start():
 
 def test_ping_monitor_minimal_config_does_not_hit_hosted_ntfy_on_finish():
     hook_config = {'topic': TOPIC}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -228,6 +268,9 @@ def test_ping_monitor_minimal_config_does_not_hit_hosted_ntfy_on_finish():
 
 def test_ping_monitor_minimal_config_hits_selfhosted_ntfy_on_fail():
     hook_config = {'topic': TOPIC, 'server': CUSTOM_BASE_URL}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -251,6 +294,9 @@ def test_ping_monitor_minimal_config_hits_selfhosted_ntfy_on_fail():
 
 def test_ping_monitor_minimal_config_does_not_hit_hosted_ntfy_on_fail_dry_run():
     hook_config = {'topic': TOPIC}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -268,6 +314,7 @@ def test_ping_monitor_minimal_config_does_not_hit_hosted_ntfy_on_fail_dry_run():
 
 def test_ping_monitor_custom_message_hits_hosted_ntfy_on_fail():
     hook_config = {'topic': TOPIC, 'fail': CUSTOM_MESSAGE_CONFIG}
+    flexmock(module).should_receive('convert_string_to_array').with_args('+1').and_return(['+1'])
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -291,6 +338,9 @@ def test_ping_monitor_custom_message_hits_hosted_ntfy_on_fail():
 
 def test_ping_monitor_custom_state_hits_hosted_ntfy_on_start():
     hook_config = {'topic': TOPIC, 'states': ['start', 'fail']}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -314,6 +364,9 @@ def test_ping_monitor_custom_state_hits_hosted_ntfy_on_start():
 
 def test_ping_monitor_with_connection_error_logs_warning():
     hook_config = {'topic': TOPIC}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)
@@ -338,6 +391,9 @@ def test_ping_monitor_with_connection_error_logs_warning():
 
 def test_ping_monitor_with_credential_error_logs_warning():
     hook_config = {'topic': TOPIC}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).and_raise(ValueError)
@@ -356,6 +412,9 @@ def test_ping_monitor_with_credential_error_logs_warning():
 
 def test_ping_monitor_with_other_error_logs_warning():
     hook_config = {'topic': TOPIC}
+    flexmock(module).should_receive('convert_string_to_array').with_args('borgmatic').and_return(
+        ['borgmatic']
+    )
     flexmock(module.borgmatic.hooks.credential.parse).should_receive(
         'resolve_credential',
     ).replace_with(lambda value, config: value)