For the "bootstrap" action, stop using the default achive name format. Also add tests (#1310).

NEWS

@@ -1,3 +1,29 @@
+2.1.7.dev0
+ * #1309: Add support for the "--quick-stats" flag and the "quick_statistics" option to the "prune"
+   action. Borg >= 1.4.5 and < 2 only.
+ * #1310: For the "bootstrap" action, provide a more helpful error message when borgmatic can't
+   determine the latest archive.
+ * #1317: Add an "archive_hostname" option and a corresponding "--archive-hostname" flag for
+   overriding the hostname used for the "{hostname}" placeholder in the "archive_name_format"
+   option. Also add an "archive_username" option and corresponding "--archive-username" flag to
+   override the "{user}" plaecholder. Both options/flags are Borg 1.4.5+ only.
+ * #1319: Fix the ZFS hook's overzealous unmounting of snapshot paths when a source dataset is at
+   "/".
+ * #1322: Fix for the "restore" action sometimes failing to find a database dump that was dumped
+   with a default port.
+ * #1324: For the MariaDB and MySQL hooks, add "events", "routines", and "tablespaces" options for
+   disabling dumping of scheduled events, stored routines, and tablespaces, respectively.
+ * #1327: Fix an error from the "diff" action when exclude options are configured.
+ * #1331: Fix the "repo-create" action to more surgically suppress Borg "Repository does not exist"
+   logs and avoid inadvertently suppressing other error logs.
+ * #1333: Fix the "--progress" flag on the "compact" action to actually update the progress of
+   segment compaction.
+ * #1334: Add the "CAP_FOWNER" capability to "CapabilityBoundingSet" in the sample systemd service,
+   so that Borg can open source files without changing file access times.
+ * #1336: Fix for the "bootstrap" action ignoring the "--verbosity" flag when no configuration is
+   present.
+ * Fix a bug in which the "compact" action does not pass a compact threshold of zero to Borg.
+
 2.1.6
  * #1256: Fix a race condition in which borgmatic sometimes swallows Borg error output without
    logging it.
@@ -10,16 +36,16 @@
  * #1303: For the MariaDB hook, include only a subset of system data when dumping the "mysql" system
    database (or "all" databases), so the dump is actually restorable. See the documentation for more
    information: https://torsion.org/borgmatic/reference/configuration/data-sources/mariadb/
- * #1314: Add a minimal stand-alone borgmatic binary in addition to the standard one. The minimal
-   binary omits support for the Apprise monitoring hook.
- * Add an experimental "browse" action providing a console UI for browsing your backups. See the
-   documentation for more information:
-   https://torsion.org/borgmatic/how-to/inspect-your-backups/#browsing-backups
  * #1308: Update the Apprise monitoring hook's "url" option to support loading credentials with the
    "{credential ...}" syntax. See the documentation for more information:
    https://torsion.org/borgmatic/reference/configuration/credentials/
  * #1309: Add a "--quick-stats" flag and corresponding "quick_statistics" option for showing only
    abbreviated statistics for the "create" action. Borg 1.4.5+ only.
+ * #1314: Add a minimal stand-alone borgmatic binary in addition to the standard one. The minimal
+   binary omits support for the Apprise monitoring hook.
+ * Add an experimental "browse" action providing a console UI for browsing your backups. See the
+   documentation for more information:
+   https://torsion.org/borgmatic/how-to/inspect-your-backups/#browsing-backups
  * Update the KeePassXC credential hook to support KeePassXC's secret service integration. See the
    documentation for more information:
    https://torsion.org/borgmatic/reference/configuration/credentials/keepassxc/

README.md

@@ -106,16 +106,16 @@ reference guides</a>.
 ## Hosting providers
 
 Need somewhere to store your encrypted off-site backups? The following hosting
-providers include specific support for Borg/borgmatic—and fund borgmatic
+provider includes specific support for Borg/borgmatic—and funds borgmatic
-development and hosting when you use these referral links to sign up:
+development and hosting when you use this referral links to sign up:
 
 <ul>
  <li class="referral"><a href="https://www.borgbase.com/?utm_source=borgmatic">BorgBase</a>: Borg hosting service with support for monitoring, 2FA, and append-only repos</li>
- <li class="referral"><a href="https://hetzner.cloud/?ref=v9dOJ98Ic9I8">Hetzner</a>: A "storage box" that includes support for Borg</li>
 </ul>
 
-Additionally, rsync.net has a compatible storage offering, but does not fund
+Additionally, Hetzner and rsync\.net have compatible storage offerings, but do
-borgmatic development or hosting.
+not fund borgmatic development or hosting.
+
 
 ## Support and contributing
 

borgmatic/actions/config/bootstrap.py

@@ -1,8 +1,12 @@
+import argparse
+import datetime
+import itertools
 import json
 import logging
 import os
 
 import borgmatic.borg.extract
+import borgmatic.borg.info
 import borgmatic.borg.repo_list
 import borgmatic.config.paths
 
@@ -14,6 +18,9 @@ def make_bootstrap_config(bootstrap_arguments):
     Given the bootstrap arguments as an argparse.Namespace, return a corresponding config dict.
     '''
     return {
+        # Without this glob, borgmatic will use a default archive name format based on hostname,
+        # which might artificially limit the archives that borgmatic can bootstrap from.
+        'match_archives': '*',
         'borgmatic_source_directory': bootstrap_arguments.borgmatic_source_directory,
         'local_path': bootstrap_arguments.local_path,
         'remote_path': bootstrap_arguments.remote_path,
@@ -100,6 +107,45 @@ def run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version):
     Raise CalledProcessError or OSError if Borg could not be run.
     '''
     config = make_bootstrap_config(bootstrap_arguments)
+
+    if bootstrap_arguments.archive == 'latest':
+        try:
+            archives_data = json.loads(
+                borgmatic.borg.info.display_archives_info(
+                    bootstrap_arguments.repository,
+                    config,
+                    local_borg_version,
+                    info_arguments=argparse.Namespace(archive=None, prefix=None, json=True),
+                    global_arguments=global_arguments,
+                    local_path=bootstrap_arguments.local_path,
+                    remote_path=bootstrap_arguments.remote_path,
+                )
+            )['archives']
+
+            def get_repo_archive_format(archive_data):
+                return archive_data['command_line'][-1]
+
+            def get_archive_start(archive_data):
+                return datetime.datetime.fromisoformat(archive_data['start'])
+
+            latest_archives = {
+                repo_archive_format: max(archives_data, key=get_archive_start)['name']
+                for repo_archive_format, archives_data in itertools.groupby(
+                    sorted(archives_data, key=get_repo_archive_format),
+                    key=get_repo_archive_format,
+                )
+                if not repo_archive_format.endswith('checkpoint')
+            }
+        except (json.JSONDecodeError, KeyError, IndexError):
+            raise ValueError(
+                f'Cannot determine the latest archive for {bootstrap_arguments.repository}'
+            )
+
+        if len(latest_archives) > 1:
+            raise ValueError(
+                f'The repository appears to have multiple "latest" archives, each with a different archive name format: {", ".join(sorted(latest_archives.values()))}. Please select one with --archive.'
+            )
+
     archive_name = borgmatic.borg.repo_list.resolve_archive_name(
         bootstrap_arguments.repository,
         bootstrap_arguments.archive,

borgmatic/borg/borg.py

@@ -29,7 +29,6 @@ def run_arbitrary_borg(
     command.
     '''
     borgmatic.logger.add_custom_log_levels()
-    lock_wait = config.get('lock_wait', None)
 
     try:
         options = options[1:] if options[0] == '--' else options
@@ -53,7 +52,7 @@ def run_arbitrary_borg(
         + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
         + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
         + flags.make_flags('remote-path', remote_path)
-        + flags.make_flags('lock-wait', lock_wait)
+        + flags.make_flags('lock-wait', config.get('lock_wait'))
         + command_options
     )
 

borgmatic/borg/break_lock.py

@@ -21,8 +21,8 @@ def break_lock(
     argparse.Namespace of global arguments, and optional local and remote Borg paths, break any
     repository and cache locks leftover from Borg aborting.
     '''
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('break_lock', '')
 
     full_command = (

borgmatic/borg/change_passphrase.py

@@ -24,8 +24,8 @@ def change_passphrase(
     based on an interactive prompt.
     '''
     borgmatic.logger.add_custom_log_levels()
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('key_change_passphrase', '')
 
     full_command = (

borgmatic/borg/check.py

@@ -20,7 +20,7 @@ def make_archive_filter_flags(local_borg_version, config, checks, check_argument
     flag. And if "prefix" is set in configuration and "archives" is in checks, then include a
     "--match-archives" flag.
     '''
-    check_last = config.get('check_last', None)
+    check_last = config.get('check_last')
     prefix = config.get('prefix')
 
     if 'archives' in checks or 'data' in checks:

borgmatic/borg/compact.py

@@ -3,7 +3,7 @@ import shlex
 
 import borgmatic.config.paths
 from borgmatic.borg import environment, feature, flags
-from borgmatic.execute import execute_command
+from borgmatic.execute import DO_NOT_CAPTURE, execute_command
 
 logger = logging.getLogger(__name__)
 
@@ -22,8 +22,8 @@ def compact_segments(
     Given dry-run flag, a local or remote repository path, a configuration dict, and the local Borg
     version, compact the segments in a repository.
     '''
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('compact', '')
     threshold = config.get('compact_threshold')
 
@@ -35,7 +35,7 @@ def compact_segments(
         + (('--lock-wait', str(lock_wait)) if lock_wait else ())
         + (('--progress',) if config.get('progress') else ())
         + (('--cleanup-commits',) if cleanup_commits else ())
-        + (('--threshold', str(threshold)) if threshold else ())
+        + (('--threshold', str(threshold)) if threshold is not None else ())
         + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
         + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
         + (
@@ -54,6 +54,7 @@ def compact_segments(
     execute_command(
         full_command,
         output_log_level=logging.INFO,
+        output_file=DO_NOT_CAPTURE if config.get('progress') else None,
         environment=environment.make_environment(config),
         working_directory=borgmatic.config.paths.get_working_directory(config),
         borg_local_path=local_path,

borgmatic/borg/create.py

@@ -188,14 +188,14 @@ def make_base_create_command(  # noqa: PLR0912
         patterns,
         borgmatic_runtime_directory,
     )
-    checkpoint_interval = config.get('checkpoint_interval', None)
+    checkpoint_interval = config.get('checkpoint_interval')
-    checkpoint_volume = config.get('checkpoint_volume', None)
+    checkpoint_volume = config.get('checkpoint_volume')
-    chunker_params = config.get('chunker_params', None)
+    chunker_params = config.get('chunker_params')
-    compression = config.get('compression', None)
+    compression = config.get('compression')
-    upload_rate_limit = config.get('upload_rate_limit', None)
+    upload_rate_limit = config.get('upload_rate_limit')
-    upload_buffer_size = config.get('upload_buffer_size', None)
+    upload_buffer_size = config.get('upload_buffer_size')
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     list_filter_flags = flags.make_list_filter_flags(local_borg_version, dry_run)
     files_changed = config.get('files_changed')
     files_cache = config.get('files_cache')

borgmatic/borg/diff.py

@@ -31,8 +31,7 @@ def diff(
     '''
     borgmatic.logger.add_custom_log_levels()
 
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
-    exclude_flags = flags.make_exclude_flags(config)
     extra_borg_options = config.get('extra_borg_options', {}).get('diff', '')
 
     if diff_arguments.only_patterns:
@@ -63,7 +62,6 @@ def diff(
             if patterns_file and diff_arguments.only_patterns
             else ()
         )
-        + exclude_flags
         + numeric_ids_flags
         + (('--same-chunker-params',) if diff_arguments.same_chunker_params else ())
         + (('--sort-by', ','.join(diff_arguments.sort_keys)) if diff_arguments.sort_keys else ())

borgmatic/borg/environment.py

@@ -4,6 +4,8 @@ import borgmatic.borg.passcommand
 import borgmatic.hooks.credential.parse
 
 OPTION_TO_ENVIRONMENT_VARIABLE = {
+    'archive_hostname': 'BORG_HOSTNAME',
+    'archive_username': 'BORG_USERNAME',
     'borg_base_directory': 'BORG_BASE_DIR',
     'borg_config_directory': 'BORG_CONFIG_DIR',
     'borg_cache_directory': 'BORG_CACHE_DIR',

borgmatic/borg/export_key.py

@@ -29,8 +29,8 @@ def export_key(
     Raise FileExistsError if a path is given but it already exists on disk.
     '''
     borgmatic.logger.add_custom_log_levels()
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     working_directory = borgmatic.config.paths.get_working_directory(config)
     extra_borg_options = config.get('extra_borg_options', {}).get('key_export', '')
 

borgmatic/borg/export_tar.py

@@ -33,8 +33,8 @@ def export_tar_archive(
     If the destination path is "-", then stream the output to stdout instead of to a file.
     '''
     borgmatic.logger.add_custom_log_levels()
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('export_tar', '')
 
     full_command = (

borgmatic/borg/extract.py

@@ -104,8 +104,8 @@ def extract_archive(
     If extract to stdout is True, then start the extraction streaming to stdout, and return that
     extract process as an instance of subprocess.Popen.
     '''
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('extract', '')
 
     if feature.available(feature.Feature.NUMERIC_IDS, local_borg_version):

borgmatic/borg/flags.py

@@ -94,15 +94,19 @@ def make_match_archives_flags(  # noqa: PLR0911
     archive_name_format,
     local_borg_version,
     default_archive_name_format=None,
+    force_flags_even_for_globs=False,
 ):
     '''
     Return match archives flags based on the given match archives value, if any. If it isn't set,
     return match archives flags to match archives created with the given (or default) archive name
     format. This is done by replacing certain archive name format placeholders for ephemeral data
     (like "{now}") with globs.
+
+    If the given match archives value is just a glob (e.g. "*"), omit any flags—unless the force
+    flags even for globs value is True.
     '''
     if match_archives:
-        if match_archives in {'*', 're:.*', 'sh:*'}:
+        if not force_flags_even_for_globs and match_archives in {'*', 're:.*', 'sh:*'}:
             return ()
 
         if feature.available(feature.Feature.MATCH_ARCHIVES, local_borg_version):
@@ -125,7 +129,7 @@ def make_match_archives_flags(  # noqa: PLR0911
         or get_default_archive_name_format(local_borg_version),
     )
 
-    if derived_match_archives == '*':
+    if not force_flags_even_for_globs and derived_match_archives == '*':
         return ()
 
     if feature.available(feature.Feature.MATCH_ARCHIVES, local_borg_version):

borgmatic/borg/import_key.py

@@ -27,8 +27,8 @@ def import_key(
 
     Raise ValueError if the path is given and it does not exist.
     '''
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     working_directory = borgmatic.config.paths.get_working_directory(config)
     extra_borg_options = config.get('extra_borg_options', {}).get('key_import', '')
 

borgmatic/borg/info.py

@@ -54,6 +54,9 @@ def make_info_command(
                     info_arguments.archive or config.get('match_archives'),
                     config.get('archive_name_format'),
                     local_borg_version,
+                    # Borg helpfully omits archives from its JSON output unless "--match-archives"
+                    # is given. So force that flag to get set when JSON is enabled.
+                    force_flags_even_for_globs=info_arguments.json,
                 )
             )
         )

borgmatic/borg/mount.py

@@ -24,8 +24,8 @@ def mount_archive(
     dict, the local Borg version, global arguments as an argparse.Namespace instance, and optional
     local and remote Borg paths, mount the archive onto the mount point.
     '''
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('mount', '')
 
     full_command = (

borgmatic/borg/prune.py

@@ -65,8 +65,8 @@ def prune_archives(
     archives according to the retention policy specified in that configuration.
     '''
     borgmatic.logger.add_custom_log_levels()
-    umask = config.get('umask', None)
+    umask = config.get('umask')
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('prune', '')
 
     full_command = (
@@ -83,10 +83,23 @@ def prune_archives(
             and not feature.available(feature.Feature.NO_PRUNE_STATS, local_borg_version)
             else ()
         )
+        + (
+            ('--quick-stats',)
+            if config.get('quick_statistics')
+            and not dry_run
+            and not feature.available(feature.Feature.NO_PRUNE_STATS, local_borg_version)
+            else ()
+        )
         + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
         + flags.make_flags_from_arguments(
             prune_arguments,
-            excludes=('repository', 'match_archives', 'statistics', 'list_details'),
+            excludes=(
+                'repository',
+                'match_archives',
+                'statistics',
+                'quick_statistics',
+                'list_details',
+            ),
         )
         + (('--list',) if config.get('list_details') else ())
         + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
@@ -95,7 +108,7 @@ def prune_archives(
         + flags.make_repository_flags(repository_path, local_borg_version)
     )
 
-    if config.get('statistics') or config.get('list_details'):
+    if config.get('statistics') or config.get('quick_statistics') or config.get('list_details'):
         output_log_level = logging.ANSWER
     else:
         output_log_level = logging.INFO

borgmatic/borg/recreate.py

@@ -28,14 +28,14 @@ def recreate_archive(
     arguments, optional local and remote Borg paths, executes the recreate command with the given
     arguments.
     '''
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     exclude_flags = flags.make_exclude_flags(config)
-    compression = config.get('compression', None)
+    compression = config.get('compression')
-    chunker_params = config.get('chunker_params', None)
+    chunker_params = config.get('chunker_params')
     extra_borg_options = config.get('extra_borg_options', {}).get('recreate', '')
 
     # Available recompress MODES: "if-different", "always", "never" (default)
-    recompress = config.get('recompress', None)
+    recompress = config.get('recompress')
 
     # Write patterns to a temporary file and use that file with --patterns-from.
     patterns_file = write_patterns_file(

borgmatic/borg/repo_create.py

@@ -5,6 +5,7 @@ import shlex
 import subprocess
 
 import borgmatic.config.paths
+import borgmatic.logger
 from borgmatic.borg import environment, feature, flags, repo_info
 from borgmatic.execute import DO_NOT_CAPTURE, execute_command
 
@@ -40,17 +41,21 @@ def create_repository(
     Raise subprocess.CalledProcessError if "borg info" returns an error exit code.
     '''
     try:
-        info_data = json.loads(
+        # Suppress Borg's "repository does not exist" error log, so the user isn't confused by
-            repo_info.display_repository_info(
+        # seeing an error during successful repository creation.
-                repository_path,
+        with borgmatic.logger.Logs_suppressed(msgid='Repository.DoesNotExist'):
-                config,
+            info_data = json.loads(
-                local_borg_version,
+                repo_info.display_repository_info(
-                argparse.Namespace(json=True),
+                    repository_path,
-                global_arguments,
+                    config,
-                local_path,
+                    local_borg_version,
-                remote_path,
+                    argparse.Namespace(json=True),
-            ),
+                    global_arguments,
-        )
+                    local_path,
+                    remote_path,
+                ),
+            )
+
         repository_encryption_mode = info_data.get('encryption', {}).get('mode')
 
         if repository_encryption_mode != encryption_mode:

borgmatic/borg/repo_info.py

@@ -24,7 +24,7 @@ def display_repository_info(
     information for the Borg repository or return JSON summary information.
     '''
     borgmatic.logger.add_custom_log_levels()
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get(
         'repo_info' if feature.available(feature.Feature.REPO_INFO, local_borg_version) else 'info',
         '',
@@ -37,7 +37,6 @@ def display_repository_info(
             if feature.available(feature.Feature.REPO_INFO, local_borg_version)
             else ('info',)
         )
-        + (('--critical',) if repo_info_arguments.json else ())
         + (
             ('--info',)
             if logger.getEffectiveLevel() == logging.INFO and not repo_info_arguments.json

borgmatic/commands/arguments.py

@@ -805,6 +805,13 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         action='store_true',
         help='Display statistics of the pruned archive [Borg 1 only]',
     )
+    prune_group.add_argument(
+        '--quick-stats',
+        dest='quick_statistics',
+        default=None,
+        action='store_true',
+        help='Display statistics of the pruned archive, skipping repository-wide "All archives" and chunk index statistics [Borg >= 1.4.5 and < 2 only]',
+    )
     prune_group.add_argument(
         '--list',
         dest='list_details',

borgmatic/commands/borgmatic.py

@@ -665,6 +665,13 @@ def load_configurations(config_filenames, arguments, overrides=None, resolve_env
     config_paths = set()
     logs = []
 
+    # As a special case for the "bootstrap" action, parse configuration for a non-existent
+    # configuration file with None for a filename. This sets any command-line arguments into an
+    # empty configuration dict, so for instance a "--verbosity" flag gets used even if there is no
+    # configuration file yet.
+    if 'bootstrap' in arguments and not config_filenames:
+        config_filenames = (None,)
+
     # Parse and load each configuration file.
     for config_filename in config_filenames:
         logs.extend(

borgmatic/config/schema.yaml

@@ -521,6 +521,21 @@ properties:
             If match_archives is not specified, borgmatic defaults to deriving
             the match_archives value from archive_name_format.
         example: "sh:{hostname}-*"
+    archive_hostname:
+        type: string
+        description: |
+            Hostname to use for the "{hostname}" placeholder in
+            "archive_name_format", "match_archives", etc. Defaults to the system
+            hostname. (This option is supported for Borg 1.4.5+ only.)
+        example: example.org
+    archive_username:
+        type: string
+        description: |
+            Username to use for the "{user}" placeholder in
+            "archive_name_format", "match_archives", etc. Defaults to the
+            username of the user running borgmatic. (This option is supported
+            for Borg 1.4.5+ only.)
+        example: backup_user
     file_list_format:
         type: string
         description: |
@@ -1097,8 +1112,9 @@ properties:
             Display statistics for an archive when running supported actions,
             skipping the repository-wide "All archives" and chunk index
             statistics to save some time. Corresponds to the "--quick-stats"
-            flag on those actions. Defaults to false. (This option is supported 
+            flag on those actions. Defaults to false. (This option is supported
-            for Borg 1.4.5+ only.)
+            for Borg 1.4.5+ only for the "create" action and for Borg >= 1.4.5
+            and < Borg 2 for the "prune" action.)
         example: true
     list_details:
         type: boolean
@@ -1841,6 +1857,27 @@ properties:
                         client and restore server. The default varies based on
                         the MariaDB version.
                     example: false
+                events:
+                    type: boolean
+                    description: |
+                        Whether to include scheduled events within the dump.
+                        Disable if your database user doesn't have the
+                        permissions to dump events. Defaults to true.
+                    example: false
+                routines:
+                    type: boolean
+                    description: |
+                        Whether to include stored routines within the dump.
+                        Disable if your user database doesn't have the
+                        permissions to dump routines. Defaults to true.
+                    example: false
+                tablespaces:
+                    type: boolean
+                    description: |
+                        Whether to include tablespaces within the dump. Disable
+                        if your database user doesn't have the permissions to
+                        dump tablespaces. Defaults to true.
+                    example: false
                 mariadb_dump_command:
                     type: string
                     description: |
@@ -2049,6 +2086,27 @@ properties:
                         client and restore server. The default varies based on
                         the MySQL installation.
                     example: false
+                events:
+                    type: boolean
+                    description: |
+                        Whether to include scheduled events within the dump.
+                        Disable if your database user doesn't have the
+                        permissions to dump events. Defaults to true.
+                    example: false
+                routines:
+                    type: boolean
+                    description: |
+                        Whether to include stored routines within the dump.
+                        Disable if your database user doesn't have the
+                        permissions to dump routines. Defaults to true.
+                    example: false
+                tablespaces:
+                    type: boolean
+                    description: |
+                        Whether to include tablespaces within the dump. Disable
+                        if your database user doesn't have the permissions to
+                        dump tablespaces. Defaults to true.
+                    example: false
                 mysql_dump_command:
                     type: string
                     description: |

borgmatic/config/validate.py

@@ -108,7 +108,8 @@ def parse_configuration(
     rendition of JSON Schema format, arguments as dict from action name to argparse.Namespace, a
     sequence of configuration file override strings in the form of "option.suboption=value", and
     whether to resolve environment variables, return the parsed configuration as a data structure of
-    nested dicts and lists corresponding to the schema. Example return value.
+    nested dicts and lists corresponding to the schema. Apply the given arguments to the config,
+    modifying it based on the values of those arguments.
 
     Example return value:
 
@@ -122,13 +123,21 @@ def parse_configuration(
     Also return a set of loaded configuration paths and a sequence of logging.LogRecord instances
     containing any warnings about the configuration.
 
+    If the given config filename is None, then create a configuration dict from whole cloth,
+    applying the given arguments to it. This is useful for the "bootstrap" action, for which
+    configuration may not yet exist.
+
     Raise FileNotFoundError if the file does not exist, PermissionError if the user does not
     have permissions to read the file, or Validation_error if the config does not match the schema.
     '''
     config_paths = set()
 
     try:
-        config = load.load_configuration(config_filename, config_paths)
+        config = (
+            load.load_configuration(config_filename, config_paths)
+            if config_filename
+            else {'repositories': []}
+        )
         schema = load.load_configuration(schema_filename)
     except (ruamel.yaml.error.YAMLError, RecursionError) as error:
         raise Validation_error(config_filename, (str(error),))

borgmatic/execute.py

@@ -157,6 +157,7 @@ def borg_json_log_line_to_record(line, log_level):
                         levelno=log_level,
                         created=log_data.get('time'),
                         msg=log_data.get('message'),
+                        msgid=log_data.get('msgid'),
                         levelname=logging.getLevelName(log_level),
                         name=log_data.get('name'),
                     )
@@ -167,6 +168,7 @@ def borg_json_log_line_to_record(line, log_level):
                     levelno=borg_log_level,
                     created=log_data.get('time'),
                     msg=log_data.get('message'),
+                    msgid=log_data.get('msgid'),
                     levelname=log_data.get('levelname'),
                     name=log_data.get('name'),
                 )
@@ -178,6 +180,7 @@ def borg_json_log_line_to_record(line, log_level):
                     levelno=log_level,
                     created=time.time(),
                     msg=f'{log_data.get("status")} {log_data.get("path")}',
+                    msgid=log_data.get('msgid'),
                     levelname=logging.getLevelName(log_level),
                     name='borg.file_status',
                 )

borgmatic/hooks/data_source/mariadb.py

@@ -241,8 +241,11 @@ def execute_dump_command(
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if database.get('tls') is True else ())
         + (('--skip-ssl',) if database.get('tls') is False else ())
-        + ('--databases', '--events', '--routines', '--all-tablespaces')
+        + (('--events',) if database.get('events', True) else ())
+        + (('--routines',) if database.get('routines', True) else ())
+        + (('--all-tablespaces',) if database.get('tablespaces', True) else ())
         + (('--system=users,udfs,servers',) if SYSTEM_DATABASE_NAME in database_names else ())
+        + ('--databases',)
         + tuple(name for name in database_names if name != SYSTEM_DATABASE_NAME)
         + ('--result-file', dump_filename)
     )
@@ -422,7 +425,7 @@ def make_data_source_dump_patterns(
     port=None,
     container=None,
     label=None,
-):  # pragma: no cover
+):
     '''
     Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
     and a database name to match, return the corresponding glob patterns to match the database dump
@@ -431,24 +434,54 @@ def make_data_source_dump_patterns(
     borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
 
     return (
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path('borgmatic'), name, hostname, port, container, label
+            dump.make_data_source_dump_filename(
+                make_dump_path('borgmatic'), name, hostname, port, container, label
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_runtime_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_source_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
         ),
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path(borgmatic_runtime_directory),
+            (
-            name,
+                dump.make_data_source_dump_filename(
-            hostname,
+                    make_dump_path('borgmatic'),
-            port,
+                    name,
-            container,
+                    hostname,
-            label,
+                    port=None,
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port == get_default_port(databases, config)
+            else ()
         ),
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path(borgmatic_source_directory),
+            (
-            name,
+                dump.make_data_source_dump_filename(
-            hostname,
+                    make_dump_path('borgmatic'),
-            port,
+                    name,
-            container,
+                    hostname,
-            label,
+                    port=get_default_port(databases, config),
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port is None
+            else ()
         ),
     )
 

borgmatic/hooks/data_source/mongodb.py

@@ -216,7 +216,7 @@ def make_data_source_dump_patterns(
     port=None,
     container=None,
     label=None,
-):  # pragma: no cover
+):
     '''
     Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
     and a database name to match, return the corresponding glob patterns to match the database dump
@@ -225,24 +225,54 @@ def make_data_source_dump_patterns(
     borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
 
     return (
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path('borgmatic'), name, hostname, port, container, label
+            dump.make_data_source_dump_filename(
+                make_dump_path('borgmatic'), name, hostname, port, container, label
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_runtime_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_source_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
         ),
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path(borgmatic_runtime_directory),
+            (
-            name,
+                dump.make_data_source_dump_filename(
-            hostname,
+                    make_dump_path('borgmatic'),
-            port,
+                    name,
-            container,
+                    hostname,
-            label,
+                    port=None,
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port == get_default_port(databases, config)
+            else ()
         ),
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path(borgmatic_source_directory),
+            (
-            name,
+                dump.make_data_source_dump_filename(
-            hostname,
+                    make_dump_path('borgmatic'),
-            port,
+                    name,
-            container,
+                    hostname,
-            label,
+                    port=get_default_port(databases, config),
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port is None
+            else ()
         ),
     )
 

borgmatic/hooks/data_source/mysql.py

@@ -165,7 +165,10 @@ def execute_dump_command(
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if database.get('tls') is True else ())
         + (('--skip-ssl',) if database.get('tls') is False else ())
-        + ('--databases', '--events', '--routines', '--all-tablespaces')
+        + (('--events',) if database.get('events', True) else ())
+        + (('--routines',) if database.get('routines', True) else ())
+        + (('--all-tablespaces',) if database.get('tablespaces', True) else ())
+        + ('--databases',)
         + database_names
         + ('--result-file', dump_filename)
     )
@@ -343,7 +346,7 @@ def make_data_source_dump_patterns(
     port=None,
     container=None,
     label=None,
-):  # pragma: no cover
+):
     '''
     Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
     and a database name to match, return the corresponding glob patterns to match the database dump
@@ -352,24 +355,54 @@ def make_data_source_dump_patterns(
     borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
 
     return (
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path('borgmatic'), name, hostname, port, container, label
+            dump.make_data_source_dump_filename(
+                make_dump_path('borgmatic'), name, hostname, port, container, label
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_runtime_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_source_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
         ),
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path(borgmatic_runtime_directory),
+            (
-            name,
+                dump.make_data_source_dump_filename(
-            hostname,
+                    make_dump_path('borgmatic'),
-            port,
+                    name,
-            container,
+                    hostname,
-            label,
+                    port=None,
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port == get_default_port(databases, config)
+            else ()
         ),
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path(borgmatic_source_directory),
+            (
-            name,
+                dump.make_data_source_dump_filename(
-            hostname,
+                    make_dump_path('borgmatic'),
-            port,
+                    name,
-            container,
+                    hostname,
-            label,
+                    port=get_default_port(databases, config),
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port is None
+            else ()
         ),
     )
 

borgmatic/hooks/data_source/postgresql.py

@@ -310,7 +310,7 @@ def make_data_source_dump_patterns(
     port=None,
     container=None,
     label=None,
-):  # pragma: no cover
+):
     '''
     Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
     and a database name to match, return the corresponding glob patterns to match the database dump
@@ -319,24 +319,54 @@ def make_data_source_dump_patterns(
     borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
 
     return (
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path('borgmatic'), name, hostname, port, container, label
+            dump.make_data_source_dump_filename(
+                make_dump_path('borgmatic'), name, hostname, port, container, label
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_runtime_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_source_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
         ),
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path(borgmatic_runtime_directory),
+            (
-            name,
+                dump.make_data_source_dump_filename(
-            hostname,
+                    make_dump_path('borgmatic'),
-            port,
+                    name,
-            container,
+                    hostname,
-            label,
+                    port=None,
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port == get_default_port(databases, config)
+            else ()
         ),
-        dump.make_data_source_dump_filename(
+        *(
-            make_dump_path(borgmatic_source_directory),
+            (
-            name,
+                dump.make_data_source_dump_filename(
-            hostname,
+                    make_dump_path('borgmatic'),
-            port,
+                    name,
-            container,
+                    hostname,
-            label,
+                    port=get_default_port(databases, config),
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port is None
+            else ()
         ),
     )
 

borgmatic/hooks/data_source/zfs.py

@@ -403,41 +403,57 @@ def remove_data_source_dumps(hook_config, config, borgmatic_runtime_directory, p
     snapshot_dataset_names = {
         full_snapshot_name.split('@')[0] for full_snapshot_name in full_snapshot_names
     }
+    hash_to_dataset_mount_point = {}
+
+    # Make a map from mount point hash to the corresponding (dataset name, mount point) tuple.
+    for dataset_name, mount_point in dataset_name_to_mount_point.items():
+        mount_point_hash = hashlib.shake_256(mount_point.encode('utf-8')).hexdigest(
+            MOUNT_POINT_HASH_LENGTH
+        )
+        hash_to_dataset_mount_point[mount_point_hash] = (dataset_name, mount_point)
 
     for snapshots_directory in glob.glob(snapshots_glob):
         if not os.path.isdir(snapshots_directory):
             continue
 
-        for dataset_name, mount_point in dataset_name_to_mount_point.items():
+        # Get the dataset and mount point corresponding to the hash found in this snapshot directory
-            snapshot_mount_path = os.path.join(snapshots_directory, mount_point.lstrip(os.path.sep))
+        # path. If none is found, bail.
+        try:
+            (dataset_name, mount_point) = hash_to_dataset_mount_point[
+                os.path.basename(snapshots_directory)
+            ]
+        except KeyError:
+            continue
 
-            # If this dataset name does not correspond to a known snapshot, then this is probably
+        snapshot_mount_path = os.path.join(snapshots_directory, mount_point.lstrip(os.path.sep))
-            # just a "shadow" of a nested dataset and therefore there's nothing to unmount.
+
-            if not os.path.isdir(snapshot_mount_path) or dataset_name not in snapshot_dataset_names:
+        # If this dataset name doesn't correspond to a known snapshot, then this is probably
+        # just a "shadow" of a nested dataset and therefore there's nothing to unmount.
+        if not os.path.isdir(snapshot_mount_path) or dataset_name not in snapshot_dataset_names:
+            continue
+
+        # This might fail if the path is already mounted, but we swallow errors here since we'll
+        # do another recursive delete below. The point of doing it here is that we don't want to
+        # try to unmount a non-mounted directory (which *will* fail), and probing for whether a
+        # directory is mounted is tough to do in a cross-platform way.
+        if not dry_run:
+            shutil.rmtree(snapshot_mount_path, ignore_errors=True)
+
+            # If the delete was successful, that means there's nothing to unmount.
+            if not os.path.isdir(snapshot_mount_path):
                 continue
 
-            # This might fail if the path is already mounted, but we swallow errors here since we'll
+        logger.debug(f'Unmounting ZFS snapshot at {snapshot_mount_path}{dry_run_label}')
-            # do another recursive delete below. The point of doing it here is that we don't want to
-            # try to unmount a non-mounted directory (which *will* fail), and probing for whether a
-            # directory is mounted is tough to do in a cross-platform way.
-            if not dry_run:
-                shutil.rmtree(snapshot_mount_path, ignore_errors=True)
 
-                # If the delete was successful, that means there's nothing to unmount.
+        if not dry_run:
-                if not os.path.isdir(snapshot_mount_path):
+            try:
-                    continue
+                unmount_snapshot(umount_command, snapshot_mount_path)
-
+            except FileNotFoundError:
-            logger.debug(f'Unmounting ZFS snapshot at {snapshot_mount_path}{dry_run_label}')
+                logger.debug(f'Could not find "{umount_command}" command')
-
+                return
-            if not dry_run:
+            except subprocess.CalledProcessError as error:
-                try:
+                logger.debug(error)
-                    unmount_snapshot(umount_command, snapshot_mount_path)
+                continue
-                except FileNotFoundError:
-                    logger.debug(f'Could not find "{umount_command}" command')
-                    return
-                except subprocess.CalledProcessError as error:
-                    logger.debug(error)
-                    continue
 
         if not dry_run:
             shutil.rmtree(snapshot_mount_path, ignore_errors=True)

borgmatic/logger.py

@@ -154,6 +154,8 @@ def log_record_to_json(record):
     '''
     Given a logging.LogRecord, return it as a JSON-encoded string containing relevant attributes.
     '''
+    message_id = getattr(record, 'msgid', None)
+
     return json.dumps(
         dict(
             type='log_message',
@@ -162,6 +164,7 @@ def log_record_to_json(record):
             levelname=record.levelname,
             name=record.name,
         )
+        | ({'msgid': message_id} if message_id is not None else {})
     )
 
 
@@ -336,6 +339,91 @@ class Log_prefix:
         set_log_prefix(self.original_prefix)
 
 
+class Log_exclude_filter(logging.Filter):
+    '''
+    A Python log filter that omits log records matching given attributes.
+    '''
+
+    def __init__(self, name, filter_attributes):
+        '''
+        Given a unique name for this filter and a dict of attributes to filter on, set the filter
+        name and save the attributes for use below.
+        '''
+        self.filter_attributes = filter_attributes
+
+        super().__init__(name)
+
+    def filter(self, log_record):
+        '''
+        Given a log record, return False (indicating the record should be omitted) if the record's
+        attributes match any of the saved filter attributes. Return True (indicating do not omit)
+        otherwise.
+        '''
+        for attribute_name, value in self.filter_attributes.items():
+            if getattr(log_record, attribute_name, None) == value:
+                return False
+
+        return True
+
+
+def add_log_exclude_filter(name, filter_attributes):
+    '''
+    Given a unique filter name and a dict of attributes to filter on, create a log exclude filter
+    with them and add the filter to each log handler.
+    '''
+    for handler in logging.getLogger().handlers:
+        handler.addFilter(Log_exclude_filter(name, filter_attributes))
+
+
+def remove_log_exclude_filter(name):
+    '''
+    Given a unique filter name, remove matching filters from each log handler.
+    '''
+    for handler in logging.getLogger().handlers:
+        for exclude_filter in handler.filters:
+            if getattr(exclude_filter, 'name', None) == name:
+                handler.removeFilter(exclude_filter)
+
+
+class Logs_suppressed:
+    '''
+    A Python context manager for temporarily adding a log filter that suppresses requested log
+    records for the duration of the context manager.
+
+    Example use:
+
+
+       with borgmatic.logger.Logs_suppressed(msgid='Repository.DoesNotExist'):
+            do_something_that_logs()
+
+    For the scope of that "with" statement, any records logged with the given message ID are
+    filtered out of the log output. "msgid" is just an example; any logging.LogRecord attributes
+    (standard or custom) can be passed in to filter on.
+
+    Multiple instances of this context manager with different filter attributes can be in use at
+    once.
+    '''
+
+    def __init__(self, **filter_attributes):
+        '''
+        Given the desired log record filter attributes as keyword arguments, save them for use below.
+        '''
+        self.filter_attributes = filter_attributes
+
+    def __enter__(self):
+        '''
+        Create a log filter with the saved filter attributes and add the filter to every logging
+        handler, so that they filter out the desired log records.
+        '''
+        add_log_exclude_filter(name=str(id(self)), filter_attributes=self.filter_attributes)
+
+    def __exit__(self, exception_type, exception, traceback):
+        '''
+        Remove the previously added filter from every logging handler.
+        '''
+        remove_log_exclude_filter(name=str(id(self)))
+
+
 class Delayed_logging_handler(logging.handlers.BufferingHandler):
     '''
     A logging handler that buffers logs and doesn't flush them until explicitly flushed (after

docs/how-to/set-up-backups.md

@@ -13,16 +13,15 @@ already.
 ## Hosting providers
 
 Need somewhere to store your encrypted off-site backups? The following hosting
-providers include specific support for Borg/borgmatic—and fund borgmatic
+provider includes specific support for Borg/borgmatic—and funds borgmatic
-development and hosting when you use these referral links to sign up:
+development and hosting when you use this referral links to sign up:
 
 <ul>
  <li class="referral"><a href="https://www.borgbase.com/?utm_source=borgmatic">BorgBase</a>: Borg hosting service with support for monitoring, 2FA, and append-only repos</li>
- <li class="referral"><a href="https://hetzner.cloud/?ref=v9dOJ98Ic9I8">Hetzner</a>: A "storage box" that includes support for Borg</li>
 </ul>
 
-Additionally, rsync.net has a compatible storage offering, but does not fund
+Additionally, Hetzner and rsync\.net have compatible storage offerings, but do
-borgmatic development or hosting.
+not fund borgmatic development or hosting.
 
 
 ## Configuration

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "borgmatic"
-version = "2.1.6"
+version = "2.1.7.dev0"
 authors = [
   { name="Dan Helfman", email="witten@torsion.org" },
 ]

sample/systemd/borgmatic.service

@@ -60,7 +60,7 @@ ProtectSystem=full
 # includes, for instance, programs to snapshot filesystems (e.g. ZFS, LVM, and
 # Btrfs). But rather than commenting this out entirely, one workaround may be to
 # add "CAP_SYS_ADMIN".
-CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW
+CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_FOWNER CAP_NET_RAW
 
 # Lower CPU and I/O priority.
 Nice=19

tests/integration/config/test_validate.py

@@ -15,14 +15,17 @@ def test_schema_filename_returns_plausible_path():
     assert schema_path.endswith('/schema.yaml')
 
 
-def mock_config_and_schema(config_yaml, schema_yaml=None):
+def mock_config_and_schema(config_yaml=None, schema_yaml=None):
     '''
-    Set up mocks for the given config config YAML string and the schema YAML string, or the default
+    Set up mocks for the given config config YAML string (if provided) and the schema YAML string or
-    schema if no schema is provided. The idea is that that the code under test consumes these mocks
+    the default schema if no schema is provided. The idea is that that the code under test consumes
-    when parsing the configuration.
+    these mocks when parsing the configuration.
     '''
-    config_stream = io.StringIO(config_yaml)
+    if config_yaml is None:
-    config_stream.name = 'config.yaml'
+        config_stream = None
+    else:
+        config_stream = io.StringIO(config_yaml)
+        config_stream.name = 'config.yaml'
 
     if schema_yaml is None:
         schema_stream = open(module.schema_filename())
@@ -81,6 +84,24 @@ def test_parse_configuration_transforms_file_into_mapping():
     assert logs == []
 
 
+def test_parse_configuration_with_none_config_filename_creates_configuration_from_whole_cloth():
+    mock_config_and_schema()
+
+    config, config_paths, logs = module.parse_configuration(
+        None,
+        '/tmp/schema.yaml',
+        arguments={'global': flexmock(verbosity=2)},
+    )
+
+    assert config == {
+        'bootstrap': {},
+        'repositories': [],
+        'verbosity': 2,
+    }
+    assert config_paths == set()
+    assert logs == []
+
+
 def test_parse_configuration_passes_through_quoted_punctuation():
     escaped_punctuation = string.punctuation.replace('\\', r'\\').replace('"', r'\"')
 

tests/unit/actions/config/test_bootstrap.py

@@ -15,6 +15,7 @@ def test_make_bootstrap_config_uses_bootstrap_arguments():
         )
     )
 
+    assert config['match_archives'] == '*'
     assert config['borgmatic_source_directory'] == '/source'
     assert config['local_path'] == 'borg1'
     assert config['remote_path'] == 'borg2'
@@ -210,6 +211,393 @@ def test_run_bootstrap_does_not_raise():
     module.run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version)
 
 
+def test_run_bootstrap_with_latest_archive_and_invalid_info_json_raises():
+    flexmock(module).should_receive('make_bootstrap_config').and_return({})
+    flexmock(module.borgmatic.borg.info).should_receive('display_archives_info').and_return(
+        '{invalid'
+    )
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').never()
+
+    bootstrap_arguments = flexmock(
+        repository='repo',
+        archive='latest',
+        destination='dest',
+        strip_components=1,
+        user_runtime_directory='/borgmatic',
+        ssh_command=None,
+        local_path='borg7',
+        remote_path='borg8',
+        progress=None,
+    )
+    global_arguments = flexmock(
+        dry_run=False,
+    )
+    local_borg_version = flexmock()
+
+    with pytest.raises(ValueError):
+        module.run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version)
+
+
+def test_run_bootstrap_with_latest_archive_and_info_json_without_command_line_raises():
+    flexmock(module).should_receive('make_bootstrap_config').and_return({})
+    flexmock(module.borgmatic.borg.info).should_receive('display_archives_info').and_return(
+        '''
+        {
+            "archives": [
+                {
+                    "name": "host-archive1",
+                    "start": "2026-08-06T12:02:53.000000"
+                },
+                {
+                    "name": "host-archive2",
+                    "start": "2026-08-06T12:04:53.000000"
+                }
+            ],
+            "repository": {}
+        }
+        '''
+    )
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').never()
+
+    bootstrap_arguments = flexmock(
+        repository='repo',
+        archive='latest',
+        destination='dest',
+        strip_components=1,
+        user_runtime_directory='/borgmatic',
+        ssh_command=None,
+        local_path='borg7',
+        remote_path='borg8',
+        progress=None,
+    )
+    global_arguments = flexmock(
+        dry_run=False,
+    )
+    local_borg_version = flexmock()
+
+    with pytest.raises(ValueError):
+        module.run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version)
+
+
+def test_run_bootstrap_with_latest_archive_and_info_json_with_empty_command_line_raises():
+    flexmock(module).should_receive('make_bootstrap_config').and_return({})
+    flexmock(module.borgmatic.borg.info).should_receive('display_archives_info').and_return(
+        '''
+        {
+            "archives": [
+                {
+                    "command_line": [],
+                    "name": "host-archive1",
+                    "start": "2026-08-06T12:02:53.000000"
+                },
+                {
+                    "command_line": [],
+                    "name": "host-archive2",
+                    "start": "2026-08-06T12:04:53.000000"
+                }
+            ],
+            "repository": {}
+        }
+        '''
+    )
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').never()
+
+    bootstrap_arguments = flexmock(
+        repository='repo',
+        archive='latest',
+        destination='dest',
+        strip_components=1,
+        user_runtime_directory='/borgmatic',
+        ssh_command=None,
+        local_path='borg7',
+        remote_path='borg8',
+        progress=None,
+    )
+    global_arguments = flexmock(
+        dry_run=False,
+    )
+    local_borg_version = flexmock()
+
+    with pytest.raises(ValueError):
+        module.run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version)
+
+
+def test_run_bootstrap_with_latest_archive_and_info_json_without_start_raises():
+    flexmock(module).should_receive('make_bootstrap_config').and_return({})
+    flexmock(module.borgmatic.borg.info).should_receive('display_archives_info').and_return(
+        '''
+        {
+            "archives": [
+                {
+                    "command_line": [
+                        "borg",
+                        "create",
+                        "--patterns-from",
+                        "/tmp/patterns",
+                        "--log-json",
+                        "--stats",
+                        "--debug",
+                        "--show-rc",
+                        "test.borg::{hostname}-archive"
+                    ],
+                    "name": "host-archive1"
+                },
+                {
+                    "command_line": [
+                        "borg",
+                        "create",
+                        "--patterns-from",
+                        "/tmp/patterns",
+                        "--log-json",
+                        "--stats",
+                        "--debug",
+                        "--show-rc",
+                        "test.borg::{hostname}-archive"
+                    ],
+                    "name": "host-archive2"
+                }
+            ],
+            "repository": {}
+        }
+        '''
+    )
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').never()
+
+    bootstrap_arguments = flexmock(
+        repository='repo',
+        archive='latest',
+        destination='dest',
+        strip_components=1,
+        user_runtime_directory='/borgmatic',
+        ssh_command=None,
+        local_path='borg7',
+        remote_path='borg8',
+        progress=None,
+    )
+    global_arguments = flexmock(
+        dry_run=False,
+    )
+    local_borg_version = flexmock()
+
+    with pytest.raises(ValueError):
+        module.run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version)
+
+
+def test_run_bootstrap_with_latest_archive_and_info_json_without_name_raises():
+    flexmock(module).should_receive('make_bootstrap_config').and_return({})
+    flexmock(module.borgmatic.borg.info).should_receive('display_archives_info').and_return(
+        '''
+        {
+            "archives": [
+                {
+                    "command_line": [
+                        "borg",
+                        "create",
+                        "--patterns-from",
+                        "/tmp/patterns",
+                        "--log-json",
+                        "--stats",
+                        "--debug",
+                        "--show-rc",
+                        "test.borg::{hostname}-archive1"
+                    ],
+                    "start": "2026-08-06T12:02:53.000000",
+                    "name": "host-archive1"
+                },
+                {
+                    "command_line": [
+                        "borg",
+                        "create",
+                        "--patterns-from",
+                        "/tmp/patterns",
+                        "--log-json",
+                        "--stats",
+                        "--debug",
+                        "--show-rc",
+                        "test.borg::{hostname}-archive2"
+                    ],
+                    "start": "2026-08-06T12:04:53.000000",
+                    "name": "host-archive2"
+                }
+            ],
+            "repository": {}
+        }
+
+        '''
+    )
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').never()
+
+    bootstrap_arguments = flexmock(
+        repository='repo',
+        archive='latest',
+        destination='dest',
+        strip_components=1,
+        user_runtime_directory='/borgmatic',
+        ssh_command=None,
+        local_path='borg7',
+        remote_path='borg8',
+        progress=None,
+    )
+    global_arguments = flexmock(
+        dry_run=False,
+    )
+    local_borg_version = flexmock()
+
+    with pytest.raises(ValueError):
+        module.run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version)
+
+
+def test_run_bootstrap_with_latest_archive_and_multiple_latests_raises():
+    flexmock(module).should_receive('make_bootstrap_config').and_return({})
+    flexmock(module.borgmatic.borg.info).should_receive('display_archives_info').and_return(
+        '''
+        {
+            "archives": [
+                {
+                    "command_line": [
+                        "borg",
+                        "create",
+                        "--patterns-from",
+                        "/tmp/patterns",
+                        "--log-json",
+                        "--stats",
+                        "--debug",
+                        "--show-rc",
+                        "test.borg::{hostname}-archive1"
+                    ],
+                    "start": "2026-08-06T12:02:53.000000",
+                    "name": "host-archive1"
+                },
+                {
+                    "command_line": [
+                        "borg",
+                        "create",
+                        "--patterns-from",
+                        "/tmp/patterns",
+                        "--log-json",
+                        "--stats",
+                        "--debug",
+                        "--show-rc",
+                        "test.borg::{hostname}-archive2"
+                    ],
+                    "start": "2026-08-06T12:04:53.000000",
+                    "name": "host-archive2"
+                }
+            ],
+            "repository": {}
+        }
+
+        '''
+    )
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').never()
+
+    bootstrap_arguments = flexmock(
+        repository='repo',
+        archive='latest',
+        destination='dest',
+        strip_components=1,
+        user_runtime_directory='/borgmatic',
+        ssh_command=None,
+        local_path='borg7',
+        remote_path='borg8',
+        progress=None,
+    )
+    global_arguments = flexmock(
+        dry_run=False,
+    )
+    local_borg_version = flexmock()
+
+    with pytest.raises(ValueError) as error:
+        module.run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version)
+
+    assert 'multiple' in str(error)
+
+
+def test_run_bootstrap_with_latest_archive_with_one_latest_does_not_raise():
+    flexmock(module).should_receive('make_bootstrap_config').and_return({})
+    flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').and_return(
+        'archive',
+    )
+    flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
+        flexmock(),
+    )
+    flexmock(module).should_receive('load_config_paths_from_archive').and_return(
+        ['/borgmatic/config.yaml']
+    )
+    flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
+        flexmock(),
+    )
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'make_runtime_directory_glob',
+    ).replace_with(lambda path: path)
+    extract_process = flexmock(
+        stdout=flexmock(
+            read=lambda: '{"config_paths": ["borgmatic/config.yaml"]}',
+        ),
+    )
+    flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
+        extract_process,
+    ).once()
+    flexmock(module.borgmatic.borg.info).should_receive('display_archives_info').and_return(
+        '''
+        {
+            "archives": [
+                {
+                    "command_line": [
+                        "borg",
+                        "create",
+                        "--patterns-from",
+                        "/tmp/patterns",
+                        "--log-json",
+                        "--stats",
+                        "--debug",
+                        "--show-rc",
+                        "test.borg::{hostname}-archive"
+                    ],
+                    "start": "2026-08-06T12:02:53.000000",
+                    "name": "host-archive1"
+                },
+                {
+                    "command_line": [
+                        "borg",
+                        "create",
+                        "--patterns-from",
+                        "/tmp/patterns",
+                        "--log-json",
+                        "--stats",
+                        "--debug",
+                        "--show-rc",
+                        "test.borg::{hostname}-archive"
+                    ],
+                    "start": "2026-08-06T12:04:53.000000",
+                    "name": "host-archive2"
+                }
+            ],
+            "repository": {}
+        }
+
+        '''
+    )
+
+    bootstrap_arguments = flexmock(
+        repository='repo',
+        archive='latest',
+        destination='dest',
+        strip_components=1,
+        user_runtime_directory='/borgmatic',
+        ssh_command=None,
+        local_path='borg7',
+        remote_path='borg8',
+        progress=None,
+    )
+    global_arguments = flexmock(
+        dry_run=False,
+    )
+    local_borg_version = flexmock()
+
+    module.run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version)
+
+
 def test_run_bootstrap_translates_ssh_command_argument_to_config():
     config = {}
     flexmock(module).should_receive('make_bootstrap_config').and_return(config)

tests/unit/borg/test_borg.py

@@ -191,9 +191,12 @@ def test_run_arbitrary_borg_with_exit_codes_calls_borg_using_them():
 def test_run_arbitrary_borg_with_remote_path_calls_borg_with_remote_path_flags():
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
     flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
-    flexmock(module.flags).should_receive('make_flags').and_return(
+    flexmock(module.flags).should_receive('make_flags').and_return(())
+    flexmock(module.flags).should_receive('make_flags').with_args(
+        'remote-path', 'borg1'
+    ).and_return(
         ('--remote-path', 'borg1'),
-    ).and_return(())
+    )
     flexmock(module.environment).should_receive('make_environment')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module).should_receive('execute_command').with_args(
@@ -219,9 +222,12 @@ def test_run_arbitrary_borg_with_remote_path_calls_borg_with_remote_path_flags()
 def test_run_arbitrary_borg_with_remote_path_injection_attack_gets_escaped():
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
     flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
-    flexmock(module.flags).should_receive('make_flags').and_return(
+    flexmock(module.flags).should_receive('make_flags').and_return(())
+    flexmock(module.flags).should_receive('make_flags').with_args(
+        'remote-path', 'borg1; naughty-command'
+    ).and_return(
         ('--remote-path', 'borg1; naughty-command'),
-    ).and_return(())
+    )
     flexmock(module.environment).should_receive('make_environment')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module).should_receive('execute_command').with_args(
@@ -240,7 +246,7 @@ def test_run_arbitrary_borg_with_remote_path_injection_attack_gets_escaped():
         config={},
         local_borg_version='1.2.3',
         options=['break-lock', '::'],
-        remote_path='borg1',
+        remote_path='borg1; naughty-command',
     )
 
 

tests/unit/borg/test_compact.py

@@ -10,6 +10,7 @@ from ..test_verbosity import insert_logging_mock
 def insert_execute_command_mock(
     compact_command,
     output_log_level,
+    output_file=None,
     working_directory=None,
     borg_exit_codes=None,
 ):
@@ -20,6 +21,7 @@ def insert_execute_command_mock(
     flexmock(module).should_receive('execute_command').with_args(
         compact_command,
         output_log_level=output_log_level,
+        output_file=output_file,
         environment=None,
         working_directory=working_directory,
         borg_local_path=compact_command[0],
@@ -149,26 +151,11 @@ def test_compact_segments_with_exit_codes_calls_borg_using_them():
     )
 
 
-def test_compact_segments_with_remote_path_calls_borg_with_remote_path_flags():
-    flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
-    insert_execute_command_mock(
-        (*COMPACT_COMMAND, '--remote-path', 'borg1', '--log-json', 'repo'), logging.INFO
-    )
-    insert_logging_mock(logging.WARNING)
-
-    module.compact_segments(
-        dry_run=False,
-        repository_path='repo',
-        config={},
-        local_borg_version='1.2.3',
-        global_arguments=flexmock(),
-        remote_path='borg1',
-    )
-
-
 def test_compact_segments_with_progress_calls_borg_with_progress_flag():
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
-    insert_execute_command_mock((*COMPACT_COMMAND, '--progress', 'repo'), logging.INFO)
+    insert_execute_command_mock(
+        (*COMPACT_COMMAND, '--progress', 'repo'), logging.INFO, module.DO_NOT_CAPTURE
+    )
     insert_logging_mock(logging.WARNING)
 
     module.compact_segments(
@@ -183,7 +170,9 @@ def test_compact_segments_with_progress_calls_borg_with_progress_flag():
 def test_compact_segments_with_log_json_and_progress_calls_borg_with_both_flags():
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
     insert_execute_command_mock(
-        (*COMPACT_COMMAND, '--log-json', '--progress', 'repo'), logging.INFO
+        (*COMPACT_COMMAND, '--log-json', '--progress', 'repo'),
+        logging.INFO,
+        module.DO_NOT_CAPTURE,
     )
     insert_logging_mock(logging.WARNING)
 
@@ -229,6 +218,22 @@ def test_compact_segments_with_threshold_calls_borg_with_threshold_flag():
     )
 
 
+def test_compact_segments_with_zero_threshold_calls_borg_with_threshold_flag():
+    flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
+    insert_execute_command_mock(
+        (*COMPACT_COMMAND, '--log-json', '--threshold', '0', 'repo'), logging.INFO
+    )
+    insert_logging_mock(logging.WARNING)
+
+    module.compact_segments(
+        dry_run=False,
+        repository_path='repo',
+        config={'compact_threshold': 0},
+        local_borg_version='1.2.3',
+        global_arguments=flexmock(),
+    )
+
+
 def test_compact_segments_with_umask_calls_borg_with_umask_flags():
     config = {'umask': '077'}
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))

tests/unit/borg/test_diff.py

@@ -12,7 +12,6 @@ LOGGING_ANSWER = flexmock()
 def test_diff_calls_borg_with_archives():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -67,7 +66,6 @@ def test_diff_calls_borg_with_archives():
 def test_diff_with_local_path_calls_borg_with_it():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -122,7 +120,6 @@ def test_diff_with_local_path_calls_borg_with_it():
 def test_diff_with_remote_path_calls_borg_with_it():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -179,7 +176,6 @@ def test_diff_with_remote_path_calls_borg_with_it():
 def test_diff_with_lock_wait_calls_borg_with_it():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -236,7 +232,6 @@ def test_diff_with_lock_wait_calls_borg_with_it():
 def test_diff_with_log_level_info_calls_borg_with_info_flag():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -293,7 +288,6 @@ def test_diff_with_log_level_info_calls_borg_with_info_flag():
 def test_diff_with_log_level_debug_calls_borg_with_debug_flags():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -351,7 +345,6 @@ def test_diff_with_log_level_debug_calls_borg_with_debug_flags():
 def test_diff_with_only_patterns_calls_borg_with_configured_pattern_paths():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
     flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
@@ -406,69 +399,9 @@ def test_diff_with_only_patterns_calls_borg_with_configured_pattern_paths():
     )
 
 
-def test_diff_with_exclude_config_calls_borg_with_exclude_flags():
-    flexmock(module.logging).ANSWER = LOGGING_ANSWER
-    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(
-        ('--exclude', 'stuff')
-    )
-    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
-    flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
-        flexmock(name='test')
-    )
-    flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
-    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
-        ('--repo', 'repo')
-    )
-    flexmock(module.borgmatic.borg.flags).should_receive('make_repository_archive_flags').never()
-    environment = flexmock()
-    flexmock(module.borgmatic.borg.environment).should_receive('make_environment').and_return(
-        environment
-    )
-    flexmock(module.borgmatic.execute).should_receive('execute_command').with_args(
-        full_command=(
-            'borg',
-            'diff',
-            '--log-json',
-            '--exclude',
-            'stuff',
-            '--repo',
-            'repo',
-            'archive',
-            'archive2',
-        ),
-        output_log_level=LOGGING_ANSWER,
-        environment=environment,
-        working_directory=None,
-        borg_local_path='borg',
-        borg_exit_codes=None,
-    ).once()
-    insert_logging_mock(logging.WARNING)
-
-    module.borgmatic.borg.diff.diff(
-        repository='repo',
-        archive='archive',
-        second_archive='archive2',
-        config={},
-        local_borg_version=None,
-        diff_arguments=flexmock(
-            same_chunker_params=False,
-            sort_keys=[],
-            content_only=False,
-            second_archive='archive2',
-            only_patterns=False,
-        ),
-        global_arguments=flexmock(),
-        local_path='borg',
-        remote_path=None,
-        patterns=[],
-    )
-
-
 def test_diff_with_numeric_ids_calls_borg_with_numeric_ids_flag():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -524,7 +457,6 @@ def test_diff_with_numeric_ids_calls_borg_with_numeric_ids_flag():
 def test_diff_with_numeric_ids_and_feature_not_available_calls_borg_with_numeric_owner_flag():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -585,7 +517,6 @@ def test_diff_with_numeric_ids_and_feature_not_available_calls_borg_with_numeric
 def test_diff_with_same_chunker_params_calls_borg_with_it():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -641,7 +572,6 @@ def test_diff_with_same_chunker_params_calls_borg_with_it():
 def test_diff_with_sort_keys_calls_borg_with_formatted_sort_by_flags():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -698,7 +628,6 @@ def test_diff_with_sort_keys_calls_borg_with_formatted_sort_by_flags():
 def test_diff_with_content_only_calls_borg_with_it():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -754,7 +683,6 @@ def test_diff_with_content_only_calls_borg_with_it():
 def test_diff_with_extra_borg_options_calls_borg_with_them():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')
@@ -811,7 +739,6 @@ def test_diff_with_extra_borg_options_calls_borg_with_them():
 def test_diff_without_separate_repository_archive_feature_available_calls_borg_joined_repository_archive():
     flexmock(module.logging).ANSWER = LOGGING_ANSWER
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
-    flexmock(module.borgmatic.borg.flags).should_receive('make_exclude_flags').and_return(())
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module.borgmatic.borg.pattern).should_receive('write_patterns_file').and_return(
         flexmock(name='test')

tests/unit/borg/test_flags.py

@@ -108,80 +108,128 @@ def test_get_default_archive_name_format_without_archive_series_feature_uses_non
 
 
 @pytest.mark.parametrize(
-    'match_archives,archive_name_format,feature_available,expected_result',
+    'match_archives,archive_name_format,feature_available,force_flags_even_for_globs,expected_result',
     (
-        (None, None, True, ('--match-archives', 'sh:{hostname}-*')),
+        (None, None, True, False, ('--match-archives', 'sh:{hostname}-*')),
-        (None, '', True, ('--match-archives', 'sh:{hostname}-*')),
+        (None, None, True, True, ('--match-archives', 'sh:{hostname}-*')),
+        (None, '', True, False, ('--match-archives', 'sh:{hostname}-*')),
         (
             're:foo-.*',
             '{hostname}-{now}',
             True,
+            False,
             ('--match-archives', 're:foo-.*'),
         ),
         (
             'sh:foo-*',
             '{hostname}-{now}',
             False,
+            False,
             ('--glob-archives', 'foo-*'),
         ),
         (
             'foo-*',
             '{hostname}-{now}',
             False,
+            False,
             ('--glob-archives', 'foo-*'),
         ),
         (
             None,
             '{hostname}-docs-{now}',
             True,
+            False,
             ('--match-archives', 'sh:{hostname}-docs-*'),
         ),
         (
             None,
             '{utcnow}-docs-{user}',
             True,
+            False,
             ('--match-archives', 'sh:*-docs-{user}'),
         ),
-        (None, '{fqdn}-{pid}', True, ('--match-archives', 'sh:{fqdn}-*')),
+        (None, '{fqdn}-{pid}', True, False, ('--match-archives', 'sh:{fqdn}-*')),
         (
             None,
             'stuff-{now:%Y-%m-%dT%H:%M:%S.%f}',
             True,
+            False,
             ('--match-archives', 'sh:stuff-*'),
         ),
         (
             None,
             '{hostname}-docs-{now}',
             False,
+            False,
             ('--glob-archives', '{hostname}-docs-*'),
         ),
         (
             None,
             '{now}',
             False,
+            False,
+            (),
+        ),
+        (
+            None,
+            '{now}',
+            False,
+            True,
+            ('--glob-archives', '*'),
+        ),
+        (
+            None,
+            '{now}',
+            True,
+            False,
             (),
         ),
         (
             None,
             '{now}',
             True,
-            (),
+            True,
+            ('--match-archives', 'sh:*'),
         ),
         (
             None,
             '{utcnow}-docs-{user}',
             False,
+            False,
             ('--glob-archives', '*-docs-{user}'),
         ),
         (
             '*',
             '{now}',
             True,
+            False,
             (),
         ),
         (
             '*',
             '{now}',
+            True,
+            True,
+            ('--match-archives', '*'),
+        ),
+        (
+            '*',
+            '{now}',
+            False,
+            False,
+            (),
+        ),
+        (
+            '*',
+            '{now}',
+            False,
+            True,
+            ('--glob-archives', '*'),
+        ),
+        (
+            're:.*',
+            '{now}',
+            True,
             False,
             (),
         ),
@@ -189,24 +237,35 @@ def test_get_default_archive_name_format_without_archive_series_feature_uses_non
             're:.*',
             '{now}',
             True,
+            True,
+            ('--match-archives', 're:.*'),
+        ),
+        (
+            'sh:*',
+            '{now}',
+            True,
+            False,
             (),
         ),
         (
             'sh:*',
             '{now}',
             True,
-            (),
+            True,
+            ('--match-archives', 'sh:*'),
         ),
         (
             'abcdefabcdef',
             None,
             True,
+            False,
             ('--match-archives', 'aid:abcdefabcdef'),
         ),
         (
             'aid:abcdefabcdef',
             None,
             True,
+            False,
             ('--match-archives', 'aid:abcdefabcdef'),
         ),
     ),
@@ -215,6 +274,7 @@ def test_make_match_archives_flags_makes_flags_with_globs(
     match_archives,
     archive_name_format,
     feature_available,
+    force_flags_even_for_globs,
     expected_result,
 ):
     flexmock(module.feature).should_receive('available').and_return(feature_available)
@@ -227,6 +287,7 @@ def test_make_match_archives_flags_makes_flags_with_globs(
             match_archives,
             archive_name_format,
             local_borg_version=flexmock(),
+            force_flags_even_for_globs=force_flags_even_for_globs,
         )
         == expected_result
     )

tests/unit/borg/test_info.py

@@ -14,6 +14,7 @@ def test_make_info_command_constructs_borg_info_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -38,6 +39,7 @@ def test_make_info_command_with_log_info_passes_through_to_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -62,6 +64,7 @@ def test_make_info_command_with_log_info_and_json_omits_borg_logging_flags():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=True,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(('--json',))
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -86,6 +89,7 @@ def test_make_info_command_with_log_debug_passes_through_to_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -110,6 +114,7 @@ def test_make_info_command_with_log_debug_and_json_omits_borg_logging_flags():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=True,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(('--json',))
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -128,12 +133,13 @@ def test_make_info_command_with_log_debug_and_json_omits_borg_logging_flags():
     assert command == ('borg', 'info', '--log-json', '--json', '--repo', 'repo')
 
 
-def test_make_info_command_with_json_passes_through_to_command():
+def test_make_info_command_with_json_passes_through_to_command_with_forced_match_archives_flag():
     flexmock(module.flags).should_receive('make_flags').and_return(())
     flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=True,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(('--json',))
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -158,6 +164,7 @@ def test_make_info_command_with_archive_uses_match_archives_flags():
         'archive',
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(('--match-archives', 'archive'))
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -190,6 +197,7 @@ def test_make_info_command_with_local_path_passes_through_to_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -218,6 +226,7 @@ def test_make_info_command_with_remote_path_passes_through_to_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -244,6 +253,7 @@ def test_make_info_command_with_umask_passes_through_to_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -271,6 +281,7 @@ def test_make_info_command_with_lock_wait_passes_through_to_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -296,6 +307,7 @@ def test_make_info_command_with_extra_borg_options_passes_through_to_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -333,6 +345,7 @@ def test_make_info_command_transforms_prefix_into_match_archives_flags():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -369,6 +382,7 @@ def test_make_info_command_prefers_prefix_over_archive_name_format():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -401,6 +415,7 @@ def test_make_info_command_transforms_archive_name_format_into_match_archives_fl
         None,
         'bar-{now}',
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(('--match-archives', 'sh:bar-*'))
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -433,6 +448,7 @@ def test_make_info_command_with_match_archives_option_passes_through_to_command(
         'sh:foo-*',
         'bar-{now}',
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(('--match-archives', 'sh:foo-*'))
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -469,6 +485,7 @@ def test_make_info_command_with_match_archives_flag_passes_through_to_command():
         'sh:foo-*',
         'bar-{now}',
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(('--match-archives', 'sh:foo-*'))
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
@@ -504,6 +521,7 @@ def test_make_info_command_passes_arguments_through_to_command(argument_name):
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(
         (flag_name, 'value'),
@@ -537,6 +555,7 @@ def test_make_info_command_with_date_based_matching_passes_through_to_command():
         None,
         None,
         '2.3.4',
+        force_flags_even_for_globs=False,
     ).and_return(())
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(
         ('--newer', '1d', '--newest', '1y', '--older', '1m', '--oldest', '1w'),

tests/unit/borg/test_list.py

@@ -96,9 +96,10 @@ def test_make_list_command_includes_json():
 
 def test_make_list_command_includes_lock_wait():
     insert_logging_mock(logging.WARNING)
-    flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(()).and_return(
+    flexmock(module.flags).should_receive('make_flags').and_return(())
+    flexmock(module.flags).should_receive('make_flags').with_args('lock-wait', 5).and_return(
         ('--lock-wait', '5'),
-    ).and_return(())
+    )
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
 

tests/unit/borg/test_prune.py

@@ -425,6 +425,31 @@ def test_prune_archives_with_stats_config_calls_borg_with_stats_flag():
     )
 
 
+def test_prune_archives_with_quick_stats_config_calls_borg_with_quick_stats_flag():
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
+    flexmock(module).should_receive('make_prune_flags').and_return(BASE_PRUNE_FLAGS)
+    flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
+    flexmock(module.feature).should_receive('available').with_args(
+        module.feature.Feature.NO_PRUNE_STATS,
+        '1.2.3',
+    ).and_return(False)
+    insert_execute_command_mock(
+        (*PRUNE_COMMAND, '--quick-stats', 'repo'), module.borgmatic.logger.ANSWER
+    )
+    insert_logging_mock(logging.WARNING)
+
+    prune_arguments = flexmock(statistics=None, list_details=False)
+    module.prune_archives(
+        dry_run=False,
+        repository_path='repo',
+        config={'quick_statistics': True},
+        local_borg_version='1.2.3',
+        global_arguments=flexmock(),
+        prune_arguments=prune_arguments,
+    )
+
+
 def test_prune_archives_with_list_config_calls_borg_with_list_flag():
     flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
     flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
@@ -647,3 +672,26 @@ def test_prune_archives_calls_borg_without_stats_when_feature_is_not_available()
         global_arguments=flexmock(),
         prune_arguments=prune_arguments,
     )
+
+
+def test_prune_archives_calls_borg_without_quick_stats_when_feature_is_not_available():
+    flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
+    flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
+    flexmock(module).should_receive('make_prune_flags').and_return(BASE_PRUNE_FLAGS)
+    flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
+    flexmock(module.feature).should_receive('available').with_args(
+        module.feature.Feature.NO_PRUNE_STATS,
+        '2.0.0b10',
+    ).and_return(True)
+    insert_execute_command_mock((*PRUNE_COMMAND, 'repo'), logging.ANSWER)
+    insert_logging_mock(logging.WARNING)
+
+    prune_arguments = flexmock(quick_statistics=True, list_details=False)
+    module.prune_archives(
+        dry_run=False,
+        repository_path='repo',
+        config={'quick_statistics': True},
+        local_borg_version='2.0.0b10',
+        global_arguments=flexmock(),
+        prune_arguments=prune_arguments,
+    )

tests/unit/borg/test_repo_create.py

@@ -13,12 +13,14 @@ REPO_CREATE_COMMAND = ('borg', 'repo-create', '--encryption', 'repokey')
 
 
 def insert_repo_info_command_found_mock():
+    flexmock(module.borgmatic.logger).should_receive('Logs_suppressed').and_return(flexmock())
     flexmock(module.repo_info).should_receive('display_repository_info').and_return(
         '{"encryption": {"mode": "repokey"}}',
     )
 
 
 def insert_repo_info_command_not_found_mock():
+    flexmock(module.borgmatic.logger).should_receive('Logs_suppressed').and_return(flexmock())
     flexmock(module.repo_info).should_receive('display_repository_info').and_raise(
         subprocess.CalledProcessError(
             sorted(module.REPO_INFO_REPOSITORY_NOT_FOUND_EXIT_CODES)[0],
@@ -158,6 +160,7 @@ def test_create_repository_errors_when_repository_with_differing_encryption_mode
 
 
 def test_create_repository_raises_for_unknown_repo_info_command_error():
+    flexmock(module.borgmatic.logger).should_receive('Logs_suppressed').and_return(flexmock())
     flexmock(module.repo_info).should_receive('display_repository_info').and_raise(
         subprocess.CalledProcessError(REPO_INFO_SOME_UNKNOWN_EXIT_CODE, []),
     )

tests/unit/borg/test_repo_info.py

@@ -144,7 +144,7 @@ def test_display_repository_info_with_log_info_and_json_suppresses_most_borg_out
     flexmock(module.environment).should_receive('make_environment')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module).should_receive('execute_command_and_capture_output').with_args(
-        ('borg', 'repo-info', '--critical', '--log-json', '--json', '--repo', 'repo'),
+        ('borg', 'repo-info', '--log-json', '--json', '--repo', 'repo'),
         environment=None,
         working_directory=None,
         borg_local_path='borg',
@@ -222,7 +222,7 @@ def test_display_repository_info_with_log_debug_and_json_suppresses_most_borg_ou
     flexmock(module.environment).should_receive('make_environment')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module).should_receive('execute_command_and_capture_output').with_args(
-        ('borg', 'repo-info', '--critical', '--log-json', '--json', '--repo', 'repo'),
+        ('borg', 'repo-info', '--log-json', '--json', '--repo', 'repo'),
         environment=None,
         working_directory=None,
         borg_local_path='borg',
@@ -258,7 +258,7 @@ def test_display_repository_info_with_json_calls_borg_with_json_flag():
     flexmock(module.environment).should_receive('make_environment')
     flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
     flexmock(module).should_receive('execute_command_and_capture_output').with_args(
-        ('borg', 'repo-info', '--critical', '--log-json', '--json', '--repo', 'repo'),
+        ('borg', 'repo-info', '--log-json', '--json', '--repo', 'repo'),
         environment=None,
         working_directory=None,
         borg_local_path='borg',

tests/unit/borg/test_repo_list.py

@@ -948,10 +948,13 @@ def test_make_repo_list_command_includes_umask():
 
 
 def test_make_repo_list_command_transforms_prefix_into_match_archives():
-    flexmock(module.feature).should_receive('available').and_return(False)
+    flexmock(module.feature).should_receive('available').and_return(True)
-    flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(()).and_return(
+    flexmock(module.flags).should_receive('make_flags').and_return(())
+    flexmock(module.flags).should_receive('make_flags').with_args(
+        'match-archives', 'sh:foo*'
+    ).and_return(
         ('--match-archives', 'sh:foo*'),
-    ).and_return(())
+    )
     flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
         None,
         None,
@@ -971,14 +974,17 @@ def test_make_repo_list_command_transforms_prefix_into_match_archives():
         global_arguments=flexmock(),
     )
 
-    assert command == ('borg', 'list', '--log-json', '--match-archives', 'sh:foo*', 'repo')
+    assert command == ('borg', 'repo-list', '--log-json', '--match-archives', 'sh:foo*', 'repo')
 
 
 def test_make_repo_list_command_prefers_prefix_over_archive_name_format():
-    flexmock(module.feature).should_receive('available').and_return(False)
+    flexmock(module.feature).should_receive('available').and_return(True)
-    flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(()).and_return(
+    flexmock(module.flags).should_receive('make_flags').and_return(())
+    flexmock(module.flags).should_receive('make_flags').with_args(
+        'match-archives', 'sh:foo*'
+    ).and_return(
         ('--match-archives', 'sh:foo*'),
-    ).and_return(())
+    )
     flexmock(module.flags).should_receive('make_match_archives_flags').never()
     flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
     flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
@@ -994,7 +1000,7 @@ def test_make_repo_list_command_prefers_prefix_over_archive_name_format():
         global_arguments=flexmock(),
     )
 
-    assert command == ('borg', 'list', '--log-json', '--match-archives', 'sh:foo*', 'repo')
+    assert command == ('borg', 'repo-list', '--log-json', '--match-archives', 'sh:foo*', 'repo')
 
 
 def test_make_repo_list_command_transforms_archive_name_format_into_match_archives():
@@ -1029,7 +1035,8 @@ def test_make_repo_list_command_transforms_archive_name_format_into_match_archiv
 
 def test_make_repo_list_command_includes_format_from_command_line():
     flexmock(module.feature).should_receive('available').and_return(False)
-    flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(()).and_return(
+    flexmock(module.flags).should_receive('make_flags').and_return(())
+    flexmock(module.flags).should_receive('make_flags').with_args('format', 'stuff').and_return(
         ('--format', 'stuff')
     )
     flexmock(module.flags).should_receive('make_match_archives_flags').with_args(

tests/unit/commands/test_borgmatic.py

@@ -1801,6 +1801,50 @@ def test_load_configurations_logs_critical_for_parse_error():
     assert max(log.levelno for log in logs) == logging.CRITICAL
 
 
+def test_load_configurations_with_bootstrap_action_and_no_configuration_file_creates_configuration_from_whole_cloth():
+    configuration = flexmock()
+    test_expected_logs = [flexmock(), flexmock()]
+    flexmock(module.validate).should_receive('parse_configuration').and_return(
+        configuration,
+        [None],
+        test_expected_logs,
+    )
+
+    configs, config_paths, logs = tuple(
+        module.load_configurations(
+            (),
+            arguments={'bootstrap': flexmock()},
+            resolve_env=False,
+        ),
+    )
+
+    assert configs == {None: configuration}
+    assert config_paths == [None]
+    assert logs
+
+
+def test_load_configurations_with_bootstrap_action_and_existing_configuration_file_uses_it():
+    configuration = flexmock()
+    test_expected_logs = [flexmock(), flexmock()]
+    flexmock(module.validate).should_receive('parse_configuration').and_return(
+        configuration,
+        ['/tmp/test.yaml'],
+        test_expected_logs,
+    )
+
+    configs, config_paths, logs = tuple(
+        module.load_configurations(
+            ('test.yaml',),
+            arguments={'bootstrap': flexmock()},
+            resolve_env=False,
+        ),
+    )
+
+    assert configs == {'test.yaml': configuration}
+    assert config_paths == ['/tmp/test.yaml']
+    assert logs
+
+
 def test_log_record_does_not_raise():
     module.log_record(levelno=1, foo='bar', baz='quux')
 

tests/unit/hooks/data_source/test_mariadb.py

@@ -855,10 +855,10 @@ def test_execute_dump_command_runs_mariadb_dump():
             '--defaults-extra-file=/dev/fd/99',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -907,11 +907,11 @@ def test_execute_dump_command_substitutes_system_flag_for_system_database_name()
             '--defaults-extra-file=/dev/fd/99',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
             '--system=users,udfs,servers',
+            '--databases',
             '--result-file',
             'dump',
         ),
@@ -956,10 +956,10 @@ def test_execute_dump_command_with_environment_password_transport_skips_defaults
             '--single-transaction',
             '--user',
             'root',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1007,10 +1007,10 @@ def test_execute_dump_command_runs_mariadb_dump_without_add_drop_database():
             'mariadb-dump',
             '--defaults-extra-file=/dev/fd/99',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1067,10 +1067,10 @@ def test_execute_dump_command_runs_mariadb_dump_with_hostname_and_port():
             '5433',
             '--protocol',
             'tcp',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1120,10 +1120,10 @@ def test_execute_dump_command_runs_mariadb_dump_with_tls():
             '--add-drop-database',
             '--single-transaction',
             '--ssl',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1173,10 +1173,10 @@ def test_execute_dump_command_runs_mariadb_dump_without_tls():
             '--add-drop-database',
             '--single-transaction',
             '--skip-ssl',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1202,6 +1202,159 @@ def test_execute_dump_command_runs_mariadb_dump_without_tls():
     )
 
 
+def test_execute_dump_command_runs_mariadb_dump_without_events():
+    process = flexmock()
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return('dump')
+    flexmock(module.os.path).should_receive('exists').and_return(False)
+    flexmock(module.borgmatic.hooks.credential.parse).should_receive(
+        'resolve_credential',
+    ).replace_with(lambda value, config: value)
+    flexmock(module).should_receive('parse_extra_options').and_return((), None)
+    flexmock(module.database_config).should_receive('resolve_database_option').and_return(None)
+    flexmock(module).should_receive('make_defaults_file_options').with_args(
+        'root',
+        'trustsome1',
+        None,
+    ).and_return(('--defaults-extra-file=/dev/fd/99',))
+    flexmock(module.dump).should_receive('create_named_pipe_for_dump')
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+
+    flexmock(module).should_receive('execute_command').with_args(
+        (
+            'mariadb-dump',
+            '--defaults-extra-file=/dev/fd/99',
+            '--add-drop-database',
+            '--single-transaction',
+            '--routines',
+            '--all-tablespaces',
+            '--databases',
+            'foo',
+            '--result-file',
+            'dump',
+        ),
+        environment=None,
+        run_to_completion=False,
+        working_directory=None,
+    ).and_return(process).once()
+
+    assert (
+        module.execute_dump_command(
+            database={'name': 'foo', 'events': False},
+            config={},
+            username='root',
+            password='trustsome1',
+            dump_path=flexmock(),
+            database_names=('foo',),
+            environment=None,
+            dry_run=False,
+            dry_run_label='',
+        )
+        == process
+    )
+
+
+def test_execute_dump_command_runs_mariadb_dump_without_routines():
+    process = flexmock()
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return('dump')
+    flexmock(module.os.path).should_receive('exists').and_return(False)
+    flexmock(module.borgmatic.hooks.credential.parse).should_receive(
+        'resolve_credential',
+    ).replace_with(lambda value, config: value)
+    flexmock(module).should_receive('parse_extra_options').and_return((), None)
+    flexmock(module.database_config).should_receive('resolve_database_option').and_return(None)
+    flexmock(module).should_receive('make_defaults_file_options').with_args(
+        'root',
+        'trustsome1',
+        None,
+    ).and_return(('--defaults-extra-file=/dev/fd/99',))
+    flexmock(module.dump).should_receive('create_named_pipe_for_dump')
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+
+    flexmock(module).should_receive('execute_command').with_args(
+        (
+            'mariadb-dump',
+            '--defaults-extra-file=/dev/fd/99',
+            '--add-drop-database',
+            '--single-transaction',
+            '--events',
+            '--all-tablespaces',
+            '--databases',
+            'foo',
+            '--result-file',
+            'dump',
+        ),
+        environment=None,
+        run_to_completion=False,
+        working_directory=None,
+    ).and_return(process).once()
+
+    assert (
+        module.execute_dump_command(
+            database={'name': 'foo', 'routines': False},
+            config={},
+            username='root',
+            password='trustsome1',
+            dump_path=flexmock(),
+            database_names=('foo',),
+            environment=None,
+            dry_run=False,
+            dry_run_label='',
+        )
+        == process
+    )
+
+
+def test_execute_dump_command_runs_mariadb_dump_without_tablespaces():
+    process = flexmock()
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return('dump')
+    flexmock(module.os.path).should_receive('exists').and_return(False)
+    flexmock(module.borgmatic.hooks.credential.parse).should_receive(
+        'resolve_credential',
+    ).replace_with(lambda value, config: value)
+    flexmock(module).should_receive('parse_extra_options').and_return((), None)
+    flexmock(module.database_config).should_receive('resolve_database_option').and_return(None)
+    flexmock(module).should_receive('make_defaults_file_options').with_args(
+        'root',
+        'trustsome1',
+        None,
+    ).and_return(('--defaults-extra-file=/dev/fd/99',))
+    flexmock(module.dump).should_receive('create_named_pipe_for_dump')
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+
+    flexmock(module).should_receive('execute_command').with_args(
+        (
+            'mariadb-dump',
+            '--defaults-extra-file=/dev/fd/99',
+            '--add-drop-database',
+            '--single-transaction',
+            '--events',
+            '--routines',
+            '--databases',
+            'foo',
+            '--result-file',
+            'dump',
+        ),
+        environment=None,
+        run_to_completion=False,
+        working_directory=None,
+    ).and_return(process).once()
+
+    assert (
+        module.execute_dump_command(
+            database={'name': 'foo', 'tablespaces': False},
+            config={},
+            username='root',
+            password='trustsome1',
+            dump_path=flexmock(),
+            database_names=('foo',),
+            environment=None,
+            dry_run=False,
+            dry_run_label='',
+        )
+        == process
+    )
+
+
 def test_execute_dump_command_runs_mariadb_dump_with_username_and_password():
     process = flexmock()
     flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return('dump')
@@ -1225,10 +1378,10 @@ def test_execute_dump_command_runs_mariadb_dump_with_username_and_password():
             '--defaults-extra-file=/dev/fd/99',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1278,10 +1431,10 @@ def test_execute_dump_command_runs_mariadb_dump_with_options():
             '--stuff=such',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1331,10 +1484,10 @@ def test_execute_dump_command_runs_non_default_mariadb_dump_with_options():
             '--stuff=such',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1426,6 +1579,86 @@ def test_execute_dump_command_with_dry_run_skips_mariadb_dump():
     )
 
 
+def test_make_data_source_dump_patterns_with_no_port_adds_pattern_with_default_port():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=None,
+    ) == (
+        'borgmatic/host/db',
+        'run/host/db',
+        '.borgmatic/host/db',
+        'borgmatic/host:9999/db',
+    )
+
+
+def test_make_data_source_dump_patterns_with_default_port_adds_pattern_with_no_port():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=9999,
+    ) == (
+        'borgmatic/host:9999/db',
+        'run/host:9999/db',
+        '.borgmatic/host:9999/db',
+        'borgmatic/host/db',
+    )
+
+
+def test_make_data_source_dump_patterns_with_non_default_port_adds_no_extra_patterns():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=1234,
+    ) == (
+        'borgmatic/host:1234/db',
+        'run/host:1234/db',
+        '.borgmatic/host:1234/db',
+    )
+
+
 def test_restore_data_source_dump_runs_mariadb_to_restore():
     hook_config = [{'name': 'foo'}, {'name': 'bar'}]
     extract_process = flexmock(stdout=flexmock())

tests/unit/hooks/data_source/test_mongodb.py

@@ -378,6 +378,86 @@ def test_build_dump_command_with_username_injection_attack_gets_escaped():
     assert "'bob; naughty-command'" in command
 
 
+def test_make_data_source_dump_patterns_with_no_port_adds_pattern_with_default_port():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=None,
+    ) == (
+        'borgmatic/host/db',
+        'run/host/db',
+        '.borgmatic/host/db',
+        'borgmatic/host:9999/db',
+    )
+
+
+def test_make_data_source_dump_patterns_with_default_port_adds_pattern_with_no_port():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=9999,
+    ) == (
+        'borgmatic/host:9999/db',
+        'run/host:9999/db',
+        '.borgmatic/host:9999/db',
+        'borgmatic/host/db',
+    )
+
+
+def test_make_data_source_dump_patterns_with_non_default_port_adds_no_extra_patterns():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=1234,
+    ) == (
+        'borgmatic/host:1234/db',
+        'run/host:1234/db',
+        '.borgmatic/host:1234/db',
+    )
+
+
 def test_restore_data_source_dump_runs_mongorestore():
     hook_config = [{'name': 'foo', 'schemas': None}, {'name': 'bar'}]
     extract_process = flexmock(stdout=flexmock())

tests/unit/hooks/data_source/test_mysql.py

@@ -718,10 +718,10 @@ def test_execute_dump_command_runs_mysqldump():
             '--defaults-extra-file=/dev/fd/99',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -771,10 +771,10 @@ def test_execute_dump_command_with_environment_password_transport_skips_defaults
             '--single-transaction',
             '--user',
             'root',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -822,10 +822,10 @@ def test_execute_dump_command_runs_mysqldump_without_add_drop_database():
             'mysqldump',
             '--defaults-extra-file=/dev/fd/99',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -882,10 +882,10 @@ def test_execute_dump_command_runs_mysqldump_with_hostname_and_port():
             '5433',
             '--protocol',
             'tcp',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -935,10 +935,10 @@ def test_execute_dump_command_runs_mysqldump_with_tls():
             '--add-drop-database',
             '--single-transaction',
             '--ssl',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -988,10 +988,10 @@ def test_execute_dump_command_runs_mysqldump_without_tls():
             '--add-drop-database',
             '--single-transaction',
             '--skip-ssl',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1017,6 +1017,159 @@ def test_execute_dump_command_runs_mysqldump_without_tls():
     )
 
 
+def test_execute_dump_command_runs_mysqldump_without_events():
+    process = flexmock()
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return('dump')
+    flexmock(module.os.path).should_receive('exists').and_return(False)
+    flexmock(module.borgmatic.hooks.credential.parse).should_receive(
+        'resolve_credential',
+    ).replace_with(lambda value, config: value)
+    flexmock(module.borgmatic.hooks.data_source.mariadb).should_receive(
+        'parse_extra_options',
+    ).and_return((), None)
+    flexmock(module.database_config).should_receive('resolve_database_option').and_return(None)
+    flexmock(module.borgmatic.hooks.data_source.mariadb).should_receive(
+        'make_defaults_file_options',
+    ).with_args('root', 'trustsome1', None).and_return(('--defaults-extra-file=/dev/fd/99',))
+    flexmock(module.dump).should_receive('create_named_pipe_for_dump')
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+
+    flexmock(module).should_receive('execute_command').with_args(
+        (
+            'mysqldump',
+            '--defaults-extra-file=/dev/fd/99',
+            '--add-drop-database',
+            '--single-transaction',
+            '--routines',
+            '--all-tablespaces',
+            '--databases',
+            'foo',
+            '--result-file',
+            'dump',
+        ),
+        environment=None,
+        run_to_completion=False,
+        working_directory=None,
+    ).and_return(process).once()
+
+    assert (
+        module.execute_dump_command(
+            database={'name': 'foo', 'events': False},
+            config={},
+            username='root',
+            password='trustsome1',
+            dump_path=flexmock(),
+            database_names=('foo',),
+            environment=None,
+            dry_run=False,
+            dry_run_label='',
+        )
+        == process
+    )
+
+
+def test_execute_dump_command_runs_mysqldump_without_routines():
+    process = flexmock()
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return('dump')
+    flexmock(module.os.path).should_receive('exists').and_return(False)
+    flexmock(module.borgmatic.hooks.credential.parse).should_receive(
+        'resolve_credential',
+    ).replace_with(lambda value, config: value)
+    flexmock(module.borgmatic.hooks.data_source.mariadb).should_receive(
+        'parse_extra_options',
+    ).and_return((), None)
+    flexmock(module.database_config).should_receive('resolve_database_option').and_return(None)
+    flexmock(module.borgmatic.hooks.data_source.mariadb).should_receive(
+        'make_defaults_file_options',
+    ).with_args('root', 'trustsome1', None).and_return(('--defaults-extra-file=/dev/fd/99',))
+    flexmock(module.dump).should_receive('create_named_pipe_for_dump')
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+
+    flexmock(module).should_receive('execute_command').with_args(
+        (
+            'mysqldump',
+            '--defaults-extra-file=/dev/fd/99',
+            '--add-drop-database',
+            '--single-transaction',
+            '--events',
+            '--all-tablespaces',
+            '--databases',
+            'foo',
+            '--result-file',
+            'dump',
+        ),
+        environment=None,
+        run_to_completion=False,
+        working_directory=None,
+    ).and_return(process).once()
+
+    assert (
+        module.execute_dump_command(
+            database={'name': 'foo', 'routines': False},
+            config={},
+            username='root',
+            password='trustsome1',
+            dump_path=flexmock(),
+            database_names=('foo',),
+            environment=None,
+            dry_run=False,
+            dry_run_label='',
+        )
+        == process
+    )
+
+
+def test_execute_dump_command_runs_mysqldump_without_tablespaces():
+    process = flexmock()
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return('dump')
+    flexmock(module.os.path).should_receive('exists').and_return(False)
+    flexmock(module.borgmatic.hooks.credential.parse).should_receive(
+        'resolve_credential',
+    ).replace_with(lambda value, config: value)
+    flexmock(module.borgmatic.hooks.data_source.mariadb).should_receive(
+        'parse_extra_options',
+    ).and_return((), None)
+    flexmock(module.database_config).should_receive('resolve_database_option').and_return(None)
+    flexmock(module.borgmatic.hooks.data_source.mariadb).should_receive(
+        'make_defaults_file_options',
+    ).with_args('root', 'trustsome1', None).and_return(('--defaults-extra-file=/dev/fd/99',))
+    flexmock(module.dump).should_receive('create_named_pipe_for_dump')
+    flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
+
+    flexmock(module).should_receive('execute_command').with_args(
+        (
+            'mysqldump',
+            '--defaults-extra-file=/dev/fd/99',
+            '--add-drop-database',
+            '--single-transaction',
+            '--events',
+            '--routines',
+            '--databases',
+            'foo',
+            '--result-file',
+            'dump',
+        ),
+        environment=None,
+        run_to_completion=False,
+        working_directory=None,
+    ).and_return(process).once()
+
+    assert (
+        module.execute_dump_command(
+            database={'name': 'foo', 'tablespaces': False},
+            config={},
+            username='root',
+            password='trustsome1',
+            dump_path=flexmock(),
+            database_names=('foo',),
+            environment=None,
+            dry_run=False,
+            dry_run_label='',
+        )
+        == process
+    )
+
+
 def test_execute_dump_command_runs_mysqldump_with_username_and_password():
     process = flexmock()
     flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return('dump')
@@ -1040,10 +1193,10 @@ def test_execute_dump_command_runs_mysqldump_with_username_and_password():
             '--defaults-extra-file=/dev/fd/99',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1093,10 +1246,10 @@ def test_execute_dump_command_runs_mysqldump_with_options():
             '--stuff=such',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1145,10 +1298,10 @@ def test_execute_dump_command_runs_non_default_mysqldump():
             '--defaults-extra-file=/dev/fd/99',
             '--add-drop-database',
             '--single-transaction',
-            '--databases',
             '--events',
             '--routines',
             '--all-tablespaces',
+            '--databases',
             'foo',
             '--result-file',
             'dump',
@@ -1239,6 +1392,86 @@ def test_execute_dump_command_with_dry_run_skips_mysqldump():
     )
 
 
+def test_make_data_source_dump_patterns_with_no_port_adds_pattern_with_default_port():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=None,
+    ) == (
+        'borgmatic/host/db',
+        'run/host/db',
+        '.borgmatic/host/db',
+        'borgmatic/host:9999/db',
+    )
+
+
+def test_make_data_source_dump_patterns_with_default_port_adds_pattern_with_no_port():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=9999,
+    ) == (
+        'borgmatic/host:9999/db',
+        'run/host:9999/db',
+        '.borgmatic/host:9999/db',
+        'borgmatic/host/db',
+    )
+
+
+def test_make_data_source_dump_patterns_with_non_default_port_adds_no_extra_patterns():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=1234,
+    ) == (
+        'borgmatic/host:1234/db',
+        'run/host:1234/db',
+        '.borgmatic/host:1234/db',
+    )
+
+
 def test_restore_data_source_dump_runs_mysql_to_restore():
     hook_config = [{'name': 'foo'}, {'name': 'bar'}]
     extract_process = flexmock(stdout=flexmock())

tests/unit/hooks/data_source/test_postgresql.py

@@ -979,6 +979,86 @@ def test_dump_data_sources_runs_non_default_pg_dump():
     ) == [process]
 
 
+def test_make_data_source_dump_patterns_with_no_port_adds_pattern_with_default_port():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=None,
+    ) == (
+        'borgmatic/host/db',
+        'run/host/db',
+        '.borgmatic/host/db',
+        'borgmatic/host:9999/db',
+    )
+
+
+def test_make_data_source_dump_patterns_with_default_port_adds_pattern_with_no_port():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=9999,
+    ) == (
+        'borgmatic/host:9999/db',
+        'run/host:9999/db',
+        '.borgmatic/host:9999/db',
+        'borgmatic/host/db',
+    )
+
+
+def test_make_data_source_dump_patterns_with_non_default_port_adds_no_extra_patterns():
+    flexmock(module.borgmatic.config.paths).should_receive(
+        'get_borgmatic_source_directory'
+    ).and_return('.borgmatic')
+    flexmock(module).should_receive('make_dump_path').replace_with(lambda path: path)
+    flexmock(module.dump).should_receive('make_data_source_dump_filename').replace_with(
+        lambda dump_path, name, hostname, port, container, label: '/'.join(
+            (dump_path, f'{hostname}:{port}' if port else hostname, name)
+        )
+    )
+    flexmock(module).should_receive('get_default_port').and_return(9999)
+
+    assert module.make_data_source_dump_patterns(
+        databases=flexmock(),
+        config=flexmock(),
+        borgmatic_runtime_directory='run',
+        name='db',
+        hostname='host',
+        port=1234,
+    ) == (
+        'borgmatic/host:1234/db',
+        'run/host:1234/db',
+        '.borgmatic/host:1234/db',
+    )
+
+
 def test_restore_data_source_dump_runs_pg_restore():
     hook_config = [{'name': 'foo', 'schemas': None}, {'name': 'bar'}]
     extract_process = flexmock(stdout=flexmock())

tests/unit/hooks/data_source/test_zfs.py

@@ -528,6 +528,9 @@ def test_remove_data_source_dumps_unmounts_and_destroys_snapshots():
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
+    flexmock(module.hashlib).should_receive('shake_256').and_return(
+        flexmock(hexdigest=lambda length: 'b33f')
+    )
     flexmock(module.glob).should_receive('glob').replace_with(
         lambda path: [path.replace('*', 'b33f')],
     )
@@ -561,6 +564,9 @@ def test_remove_data_source_dumps_use_custom_commands():
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
+    flexmock(module.hashlib).should_receive('shake_256').and_return(
+        flexmock(hexdigest=lambda length: 'b33f')
+    )
     flexmock(module.glob).should_receive('glob').replace_with(
         lambda path: [path.replace('*', 'b33f')],
     )
@@ -644,6 +650,9 @@ def test_remove_data_source_dumps_bails_for_missing_umount_command():
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
+    flexmock(module.hashlib).should_receive('shake_256').and_return(
+        flexmock(hexdigest=lambda length: 'b33f')
+    )
     flexmock(module.glob).should_receive('glob').replace_with(
         lambda path: [path.replace('*', 'b33f')],
     )
@@ -675,6 +684,9 @@ def test_remove_data_source_dumps_swallows_umount_command_error():
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
+    flexmock(module.hashlib).should_receive('shake_256').and_return(
+        flexmock(hexdigest=lambda length: 'b33f')
+    )
     flexmock(module.glob).should_receive('glob').replace_with(
         lambda path: [path.replace('*', 'b33f')],
     )
@@ -709,6 +721,9 @@ def test_remove_data_source_dumps_skips_unmount_snapshot_directories_that_are_no
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
+    flexmock(module.hashlib).should_receive('shake_256').and_return(
+        flexmock(hexdigest=lambda length: 'b33f')
+    )
     flexmock(module.glob).should_receive('glob').replace_with(
         lambda path: [path.replace('*', 'b33f')],
     )
@@ -739,6 +754,9 @@ def test_remove_data_source_dumps_skips_unmount_snapshot_mount_paths_that_are_no
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
+    flexmock(module.hashlib).should_receive('shake_256').and_return(
+        flexmock(hexdigest=lambda length: 'b33f')
+    )
     flexmock(module.glob).should_receive('glob').replace_with(
         lambda path: [path.replace('*', 'b33f')],
     )
@@ -774,21 +792,37 @@ def test_remove_data_source_dumps_skips_unmount_snapshot_mount_paths_for_unknown
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
-    flexmock(module.glob).should_receive('glob').replace_with(
+    flexmock(module.hashlib).should_receive('shake_256').with_args(b'/mnt/dataset').and_return(
-        lambda path: [path.replace('*', 'b33f')],
+        flexmock(hexdigest=lambda length: 'd34d')
     )
+    flexmock(module.hashlib).should_receive('shake_256').with_args(
+        b'/mnt/dataset/shadow'
+    ).and_return(flexmock(hexdigest=lambda length: 'b33f'))
+    flexmock(module.glob).should_receive('glob').replace_with(
+        lambda path: [
+            path.replace('*', 'd34d'),
+            path.replace('*', 'b33f'),
+            path.replace('*', 'unknown'),
+        ],
+    )
+    flexmock(module.os.path).should_receive('isdir').with_args(
+        '/run/borgmatic/zfs_snapshots/d34d',
+    ).and_return(True)
     flexmock(module.os.path).should_receive('isdir').with_args(
         '/run/borgmatic/zfs_snapshots/b33f',
     ).and_return(True)
     flexmock(module.os.path).should_receive('isdir').with_args(
-        '/run/borgmatic/zfs_snapshots/b33f/mnt/dataset',
+        '/run/borgmatic/zfs_snapshots/d34d/mnt/dataset',
     ).and_return(True)
     flexmock(module.os.path).should_receive('isdir').with_args(
         '/run/borgmatic/zfs_snapshots/b33f/mnt/dataset/shadow',
     ).and_return(True)
+    flexmock(module.os.path).should_receive('isdir').with_args(
+        '/run/borgmatic/zfs_snapshots/unknown',
+    ).and_return(True)
     flexmock(module.shutil).should_receive('rmtree')
     flexmock(module).should_receive('unmount_snapshot').with_args(
-        'umount', '/run/borgmatic/zfs_snapshots/b33f/mnt/dataset'
+        'umount', '/run/borgmatic/zfs_snapshots/d34d/mnt/dataset'
     ).once()
     flexmock(module).should_receive('unmount_snapshot').with_args(
         'umount', '/run/borgmatic/zfs_snapshots/b33f/mnt/dataset/shadow'
@@ -817,6 +851,9 @@ def test_remove_data_source_dumps_skips_unmount_snapshot_mount_paths_after_rmtre
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
+    flexmock(module.hashlib).should_receive('shake_256').and_return(
+        flexmock(hexdigest=lambda length: 'b33f')
+    )
     flexmock(module.glob).should_receive('glob').replace_with(
         lambda path: [path.replace('*', 'b33f')],
     )
@@ -852,6 +889,9 @@ def test_remove_data_source_dumps_with_dry_run_skips_unmount_and_destroy():
     flexmock(module.borgmatic.config.paths).should_receive(
         'replace_temporary_subdirectory_with_glob',
     ).and_return('/run/borgmatic')
+    flexmock(module.hashlib).should_receive('shake_256').and_return(
+        flexmock(hexdigest=lambda length: 'b33f')
+    )
     flexmock(module.glob).should_receive('glob').replace_with(
         lambda path: [path.replace('*', 'b33f')],
     )

tests/unit/test_execute.py

@@ -124,6 +124,20 @@ def test_borg_json_log_line_to_record_parses_log_message_line():
     assert record.levelno == module.logging.INFO
     assert record.created == 12345
     assert record.msg == 'All done'
+    assert record.msgid is None
+    assert record.levelname == 'INFO'
+    assert record.name == 'borg.something'
+
+
+def test_borg_json_log_line_to_record_parses_log_message_line_with_message_id():
+    line = '{"type": "log_message", "levelname": "INFO", "time": 12345, "message": "All done", "msgid": "all.done", "name": "borg.something"}'
+
+    record = module.borg_json_log_line_to_record(line, module.logging.INFO)
+
+    assert record.levelno == module.logging.INFO
+    assert record.created == 12345
+    assert record.msg == 'All done'
+    assert record.msgid == 'all.done'
     assert record.levelname == 'INFO'
     assert record.name == 'borg.something'
 
@@ -136,6 +150,20 @@ def test_borg_json_log_line_to_record_elevates_log_message_info_level_to_small_j
     assert record.levelno == 25
     assert record.created == 12345
     assert record.msg == 'All done'
+    assert record.msgid is None
+    assert record.levelname in {'ANSWER', 'Level 25'}
+    assert record.name == 'borg.something'
+
+
+def test_borg_json_log_line_to_record_elevates_log_message_and_includes_message_id():
+    line = '{"type": "log_message", "levelname": "INFO", "time": 12345, "message": "All done", "msgid": "all.done", "name": "borg.something"}'
+
+    record = module.borg_json_log_line_to_record(line, 25)
+
+    assert record.levelno == 25
+    assert record.created == 12345
+    assert record.msg == 'All done'
+    assert record.msgid == 'all.done'
     assert record.levelname in {'ANSWER', 'Level 25'}
     assert record.name == 'borg.something'
 
@@ -148,6 +176,7 @@ def test_borg_json_log_line_to_record_does_not_elevate_log_message_info_level_to
     assert record.levelno == module.logging.INFO
     assert record.created == 12345
     assert record.msg == 'All done'
+    assert record.msgid is None
     assert record.levelname == 'INFO'
     assert record.name == 'borg.something'
 
@@ -160,6 +189,7 @@ def test_borg_json_log_line_with_none_log_level_parses_log_message_line():
     assert record.levelno == module.logging.INFO
     assert record.created == 12345
     assert record.msg == 'All done'
+    assert record.msgid is None
     assert record.levelname == 'INFO'
     assert record.name == 'borg.something'
 
@@ -173,6 +203,21 @@ def test_borg_json_log_line_to_record_parses_file_status_line():
     assert record.levelno == module.logging.INFO
     assert record.created == 12345
     assert record.msg == '- /foo/bar'
+    assert record.msgid is None
+    assert record.levelname == 'INFO'
+    assert record.name == 'borg.file_status'
+
+
+def test_borg_json_log_line_to_record_parses_file_status_line_with_message_id():
+    flexmock(module.time).should_receive('time').and_return(12345)
+    line = '{"type": "file_status", "status": "-", "path": "/foo/bar", "msgid": "hi.there"}'
+
+    record = module.borg_json_log_line_to_record(line, module.logging.INFO)
+
+    assert record.levelno == module.logging.INFO
+    assert record.created == 12345
+    assert record.msg == '- /foo/bar'
+    assert record.msgid == 'hi.there'
     assert record.levelname == 'INFO'
     assert record.name == 'borg.file_status'
 

tests/unit/test_logger.py

@@ -243,6 +243,23 @@ def test_log_record_to_json_formats_record_as_json():
     )
 
 
+def test_log_record_to_json_with_message_id_formats_record_as_json():
+    assert (
+        module.log_record_to_json(
+            flexmock(
+                created=12345,
+                levelno=module.logging.INFO,
+                levelname='INFO',
+                name='borg.something',
+                extra='ignored',
+                getMessage=lambda: 'All done',
+                msgid='all.done',
+            )
+        )
+        == '{"type": "log_message", "time": 12345, "message": "All done", "levelname": "INFO", "name": "borg.something", "msgid": "all.done"}'
+    )
+
+
 def test_console_color_formatter_format_includes_log_message():
     flexmock(module).should_receive('add_custom_log_levels')
     flexmock(module.logging).ANSWER = module.ANSWER
@@ -417,6 +434,117 @@ def test_log_prefix_sets_prefix_and_then_restores_original_prefix_after():
         pass
 
 
+def test_log_exclude_filter_filter_omits_log_matching_any_attributes():
+    exclude_filter = module.Log_exclude_filter('my filter', {'foo': 'bar', 'baz': 'quux'})
+
+    assert exclude_filter.filter(flexmock(foo='nope', baz='quux')) is False
+
+
+def test_log_exclude_filter_filter_includes_log_matching_no_attributes():
+    exclude_filter = module.Log_exclude_filter('my filter', {'foo': 'bar', 'baz': 'quux'})
+
+    assert exclude_filter.filter(flexmock(foo='nope', baz='uh uh')) is True
+
+
+def test_log_exclude_filter_filter_includes_log_matching_no_attributes_and_in_fact_missing_them_entirely():
+    exclude_filter = module.Log_exclude_filter('my filter', {'foo': 'bar', 'baz': 'quux'})
+
+    assert exclude_filter.filter(flexmock(other='nope', thing='uh uh')) is True
+
+
+def test_add_log_exclude_filter_adds_filter_to_each_handler():
+    flexmock(module).should_receive('Log_exclude_filter').and_return(flexmock())
+    handlers = [flexmock(), flexmock()]
+    handlers[0].should_receive('addFilter').once()
+    handlers[1].should_receive('addFilter').once()
+
+    flexmock(module.logging).should_receive('getLogger').and_return(
+        flexmock(handlers=handlers, removeHandler=lambda handler: None)
+    )
+
+    module.add_log_exclude_filter('my filter', {'foo': 'bar', 'baz': 'quux'})
+
+
+def test_remove_log_exclude_filter_removes_filter_from_each_handler():
+    handlers = [
+        flexmock(
+            filters=[
+                flexmock(name='my filter'),
+                flexmock(name='my filter'),
+            ],
+        ),
+        flexmock(
+            filters=[
+                flexmock(name='my filter'),
+            ],
+        ),
+    ]
+    handlers[0].should_receive('removeFilter').twice()
+    handlers[1].should_receive('removeFilter').once()
+
+    flexmock(module.logging).should_receive('getLogger').and_return(
+        flexmock(handlers=handlers, removeHandler=lambda handler: None),
+    )
+
+    module.remove_log_exclude_filter(name='my filter')
+
+
+def test_remove_log_exclude_filter_skips_remove_for_filter_without_matching_name():
+    handlers = [
+        flexmock(
+            filters=[
+                flexmock(name='other filter'),
+                flexmock(name='my filter'),
+            ],
+        ),
+        flexmock(
+            filters=[
+                flexmock(name='my filter'),
+            ],
+        ),
+    ]
+    handlers[0].should_receive('removeFilter').once()
+    handlers[1].should_receive('removeFilter').once()
+
+    flexmock(module.logging).should_receive('getLogger').and_return(
+        flexmock(handlers=handlers, removeHandler=lambda handler: None),
+    )
+
+    module.remove_log_exclude_filter(name='my filter')
+
+
+def test_remove_log_exclude_filter_skips_remove_for_filter_without_name_attribute():
+    handlers = [
+        flexmock(
+            filters=[
+                flexmock(),
+                flexmock(name='my filter'),
+            ],
+        ),
+        flexmock(
+            filters=[
+                flexmock(name='my filter'),
+            ],
+        ),
+    ]
+    handlers[0].should_receive('removeFilter').once()
+    handlers[1].should_receive('removeFilter').once()
+
+    flexmock(module.logging).should_receive('getLogger').and_return(
+        flexmock(handlers=handlers, removeHandler=lambda handler: None),
+    )
+
+    module.remove_log_exclude_filter(name='my filter')
+
+
+def test_logs_suppressed_adds_and_removes_log_exclude_filter():
+    flexmock(module).should_receive('add_log_exclude_filter').once()
+    flexmock(module).should_receive('remove_log_exclude_filter').once()
+
+    with module.Logs_suppressed(foo='bar', baz='quux'):
+        pass
+
+
 def test_delayed_logging_handler_should_flush_without_targets_returns_false():
     handler = module.Delayed_logging_handler()