Bump version for release.

Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1206

.gitea/workflows/build.yaml

@@ -29,5 +29,3 @@ jobs:
           PASSWORD: "${{ secrets.REGISTRY_PASSWORD }}"
       - run: podman build --tag "$IMAGE_NAME" --file docs/Dockerfile --storage-opt "overlay.mount_program=/usr/bin/fuse-overlayfs" .
       - run: podman push "$IMAGE_NAME"
-      - run: scripts/export-docs-from-image
-      - run: curl --user "${{ secrets.REGISTRY_USERNAME }}:${{ secrets.REGISTRY_PASSWORD }}" --upload-file borgmatic-docs.tar.gz https://projects.torsion.org/api/packages/borgmatic-collective/generic/borgmatic-docs/$(head --lines=1 NEWS)/borgmatic-docs.tar.gz

NEWS

@@ -1,5 +1,72 @@
+2.0.13
+ * #1054: Allow the Btrfs hook to create and delete snapshots even when running
+   as a non-root user. See the documentation for more information:
+   https://torsion.org/borgmatic/reference/configuration/data-sources/btrfs/#non-root-user
+ * #1122: To prevent the user from inadvertently excluding the "bootstrap" action's manifest, always
+   error and exit when the borgmatic runtime directory overlaps with the configured excludes.
+ * #1179: Add a "file_list_format" option for setting the "list" action's output format and an
+   "archive_list_format" option for setting the "repo-list" action's format.
+ * #1192: Fix for over-aggressive deduplication of source directories that contain the borgmatic
+   runtime directory, potentially resulting in data loss (data not getting backed up) when
+   snapshotting these source directories.
+ * #1192, #1163: Document potential interactions between security settings in borgmatic's sample
+   systemd service file and the ZFS, LVM, and Btrfs hooks.
+ * #1193: In the documentation for the MariaDB/MySQL database hooks, clarify how to set custom
+   command-line flags for database commands.
+ * #1193: For the MariaDB and MySQL database hooks, add a "socket_path" option for Unix socket
+   database connections.
+ * #1193: For the MariaDB and MySQL database hooks, creates a consistent snapshot by dumping all
+   tables in a single transaction.
+ * #1194: Fix for an incorrect diff command shown when running the "generate config" action with a
+   source configuration file. 
+ * #1195: Fix a regression in the ZFS, LVM, and Btrfs hooks in which snapshotted paths ignored
+   global excludes.
+ * #1201: Document a problematic interaction between borgmatic and systemd-tmpfiles:
+   https://torsion.org/borgmatic/reference/configuration/runtime-directory/#systemd-tmpfiles
+ * #1203: Fix that errors and exits when the borgmatic runtime directory is partially excluded by
+   configured excludes. Previously, borgmatic only errored when the runtime directory was completely
+   excluded.
+ * #1206: Adjust Btrfs snapshot paths so that Borg 1.x gets file cache hits when backing them up,
+   improving performance.
+ * Update the sample systemd timer with a shorter random delay when catching up on a missed run.
+
+2.0.12
+ * #1127: Fix for the database hooks not respecting the "working_directory" option.
+ * #1181: Add an "ask_for_password" option to the KeePassXC credential hook for disabling
+   KeePassXC's password prompt, e.g. if you're only using a key file to decrypt your database.
+ * #1184: Fix the fish shell completion's detection of version mismatches.
+ * #1186: Fix a regression in the Btrfs hook in which subvolume snapshots didn't get cleaned up
+   until the start of the next borgmatic run.
+ * In the SQLite database hook, run SQLite such that it exits upon encountering an error instead of,
+   you know, not doing that.
+ * Add documentation on repositories, including SSH, Rclone, S3, and B2:
+   https://torsion.org/borgmatic/reference/configuration/repositories/
+ * Improve documentation search results for individual configuration options.
+ * Add borgmatic release artifacts (wheel and tarball) to each release on the releases page:
+   https://projects.torsion.org/borgmatic-collective/borgmatic/releases
+ * Move the tarball of borgmatic's HTML documentation from the packages page to the releases page.
+
+2.0.11
+ * #957: Document borgmatic's limitations around parallelism—both its own and Borg's. See the
+   documentation for more information:
+   https://torsion.org/borgmatic/how-to/make-per-application-backups/#limitations
+ * #1165: Fix for when the systemd service directories (RuntimeDirectory and StateDirectory) each
+   contain multiple paths.
+ * #1168: Fix for the "list", "info", and "delete" options in "extra_borg_options" being ignored
+   when "--archive" is omitted with Borg 1.x.
+ * #1169: Fix for a regression in the ZFS, LVM, and Btrfs hooks in which partial excludes of
+   snapshot paths were ignored.
+ * #1170: Fix for an inconsistent log level for Borg's last output line before exiting.
+ * #1172: Add an "environment" option to the Sentry monitoring hook.
+ * #1176: Fix the "--repository" flag not applying to command hooks.
+ * Add a "rename" option to "extra_borg_options" to support passing arbitrary flags to "borg
+   rename".
+ * Add documentation on patterns and excludes:
+   https://torsion.org/borgmatic/reference/configuration/patterns-and-excludes/
+ * Drop support for Python 3.9, which has been end-of-lifed.
+
 2.0.10
- * #427: Expand the "borg_extra_options" option to support passing arbitrary Borg flags to every
+ * #427: Expand the "extra_borg_options" option to support passing arbitrary Borg flags to every
    Borg sub-command that borgmatic uses. As part of this, deprecate the "init" option under
    "borg_extra_options" in favor of "repo_create".
  * #942: Factor reference material out of the documentation how-to guides. This means there's now a

borgmatic/actions/borg.py

@@ -2,7 +2,6 @@ import logging
 
 import borgmatic.borg.borg
 import borgmatic.borg.repo_list
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -19,26 +18,22 @@ def run_borg(
     '''
     Run the "borg" action for the given repository.
     '''
-    if borg_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        borg_arguments.repository,
-    ):
-        logger.info('Running arbitrary Borg command')
-        archive_name = borgmatic.borg.repo_list.resolve_archive_name(
-            repository['path'],
-            borg_arguments.archive,
-            config,
-            local_borg_version,
-            global_arguments,
-            local_path,
-            remote_path,
-        )
-        borgmatic.borg.borg.run_arbitrary_borg(
-            repository['path'],
-            config,
-            local_borg_version,
-            options=borg_arguments.options,
-            archive=archive_name,
-            local_path=local_path,
-            remote_path=remote_path,
-        )
+    logger.info('Running arbitrary Borg command')
+    archive_name = borgmatic.borg.repo_list.resolve_archive_name(
+        repository['path'],
+        borg_arguments.archive,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+    borgmatic.borg.borg.run_arbitrary_borg(
+        repository['path'],
+        config,
+        local_borg_version,
+        options=borg_arguments.options,
+        archive=archive_name,
+        local_path=local_path,
+        remote_path=remote_path,
+    )

borgmatic/actions/break_lock.py

@@ -1,7 +1,6 @@
 import logging
 
 import borgmatic.borg.break_lock
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -18,16 +17,12 @@ def run_break_lock(
     '''
     Run the "break-lock" action for the given repository.
     '''
-    if break_lock_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        break_lock_arguments.repository,
-    ):
-        logger.info('Breaking repository and cache locks')
-        borgmatic.borg.break_lock.break_lock(
-            repository['path'],
-            config,
-            local_borg_version,
-            global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-        )
+    logger.info('Breaking repository and cache locks')
+    borgmatic.borg.break_lock.break_lock(
+        repository['path'],
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+    )

borgmatic/actions/change_passphrase.py

@@ -1,7 +1,6 @@
 import logging
 
 import borgmatic.borg.change_passphrase
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -18,20 +17,13 @@ def run_change_passphrase(
     '''
     Run the "key change-passphrase" action for the given repository.
     '''
-    if (
-        change_passphrase_arguments.repository is None
-        or borgmatic.config.validate.repositories_match(
-            repository,
-            change_passphrase_arguments.repository,
-        )
-    ):
-        logger.info('Changing repository passphrase')
-        borgmatic.borg.change_passphrase.change_passphrase(
-            repository['path'],
-            config,
-            local_borg_version,
-            change_passphrase_arguments,
-            global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-        )
+    logger.info('Changing repository passphrase')
+    borgmatic.borg.change_passphrase.change_passphrase(
+        repository['path'],
+        config,
+        local_borg_version,
+        change_passphrase_arguments,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+    )

borgmatic/actions/check.py

@@ -20,11 +20,8 @@ import borgmatic.borg.extract
 import borgmatic.borg.list
 import borgmatic.borg.pattern
 import borgmatic.borg.repo_list
-import borgmatic.borg.state
 import borgmatic.config.paths
-import borgmatic.config.validate
 import borgmatic.execute
-import borgmatic.hooks.command
 
 DEFAULT_CHECKS = (
     {'name': 'repository', 'frequency': '1 month'},
@@ -750,12 +747,6 @@ def run_check(
 
     Raise ValueError if the Borg repository ID cannot be determined.
     '''
-    if check_arguments.repository and not borgmatic.config.validate.repositories_match(
-        repository,
-        check_arguments.repository,
-    ):
-        return
-
     logger.info('Running consistency checks')
 
     repository_id = borgmatic.borg.check.get_repository_id(

borgmatic/actions/compact.py

@@ -2,8 +2,6 @@ import logging
 
 import borgmatic.borg.compact
 import borgmatic.borg.feature
-import borgmatic.config.validate
-import borgmatic.hooks.command
 
 logger = logging.getLogger(__name__)
 
@@ -22,12 +20,6 @@ def run_compact(
     '''
     Run the "compact" action for the given repository.
     '''
-    if compact_arguments.repository and not borgmatic.config.validate.repositories_match(
-        repository,
-        compact_arguments.repository,
-    ):
-        return
-
     if borgmatic.borg.feature.available(borgmatic.borg.feature.Feature.COMPACT, local_borg_version):
         logger.info(f'Compacting segments{dry_run_label}')
         borgmatic.borg.compact.compact_segments(

borgmatic/actions/config/bootstrap.py

@@ -5,8 +5,6 @@ import os
 import borgmatic.borg.extract
 import borgmatic.borg.repo_list
 import borgmatic.config.paths
-import borgmatic.config.validate
-import borgmatic.hooks.command
 
 logger = logging.getLogger(__name__)
 

borgmatic/actions/config/generate.py

@@ -36,12 +36,12 @@ def run_generate(generate_arguments, global_arguments):
     if generate_arguments.source_filename:
         logger.answer(
             f'''
-Merged in the contents of configuration file at: {generate_arguments.source_filename}'''
+Merged the contents of the source configuration file: {generate_arguments.source_filename}'''
         )
 
         if not generate_arguments.split:
             logger.answer(
-                '''To review the changes made, run:
+                f'''To review the changes made, run:
 
     diff --unified {generate_arguments.source_filename} {generate_arguments.destination_path}''',
             )

borgmatic/actions/create.py

@@ -6,7 +6,6 @@ import borgmatic.borg.feature
 import borgmatic.borg.rename
 import borgmatic.borg.repo_list
 import borgmatic.config.paths
-import borgmatic.config.validate
 import borgmatic.hooks.dispatch
 from borgmatic.actions import pattern
 
@@ -30,12 +29,6 @@ def run_create(
 
     If create_arguments.json is True, yield the JSON output from creating the archive.
     '''
-    if create_arguments.repository and not borgmatic.config.validate.repositories_match(
-        repository,
-        create_arguments.repository,
-    ):
-        return
-
     if config.get('list_details') and config.get('progress'):
         raise ValueError(
             'With the create action, only one of --list/--files/list_details and --progress/progress can be used.',
@@ -56,12 +49,14 @@ def run_create(
             working_directory,
             borgmatic_runtime_directory,
         )
+
+        original_patterns = list(patterns)
         borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
             'remove_data_source_dumps',
             config,
             borgmatic.hooks.dispatch.Hook_type.DATA_SOURCE,
             borgmatic_runtime_directory,
-            patterns,
+            original_patterns,
             global_arguments.dry_run,
         )
         active_dumps = borgmatic.hooks.dispatch.call_hooks(
@@ -136,12 +131,14 @@ def run_create(
 
             yield output
 
+        # Use the original patterns so as to disregard any modifications made by any data source
+        # hooks, e.g. via dump_data_sources() above.
         borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
             'remove_data_source_dumps',
             config,
             borgmatic.hooks.dispatch.Hook_type.DATA_SOURCE,
             borgmatic_runtime_directory,
-            patterns,
+            original_patterns,
             global_arguments.dry_run,
         )
 

borgmatic/actions/delete.py

@@ -2,7 +2,6 @@ import logging
 
 import borgmatic.actions.arguments
 import borgmatic.borg.delete
-import borgmatic.borg.repo_delete
 import borgmatic.borg.repo_list
 
 logger = logging.getLogger(__name__)
@@ -20,32 +19,28 @@ def run_delete(
     '''
     Run the "delete" action for the given repository and archive(s).
     '''
-    if delete_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        delete_arguments.repository,
-    ):
-        logger.answer('Deleting archives')
+    logger.answer('Deleting archives')
 
-        archive_name = (
-            borgmatic.borg.repo_list.resolve_archive_name(
-                repository['path'],
-                delete_arguments.archive,
-                config,
-                local_borg_version,
-                global_arguments,
-                local_path,
-                remote_path,
-            )
-            if delete_arguments.archive
-            else None
-        )
-
-        borgmatic.borg.delete.delete_archives(
-            repository,
+    archive_name = (
+        borgmatic.borg.repo_list.resolve_archive_name(
+            repository['path'],
+            delete_arguments.archive,
             config,
             local_borg_version,
-            borgmatic.actions.arguments.update_arguments(delete_arguments, archive=archive_name),
             global_arguments,
             local_path,
             remote_path,
         )
+        if delete_arguments.archive
+        else None
+    )
+
+    borgmatic.borg.delete.delete_archives(
+        repository,
+        config,
+        local_borg_version,
+        borgmatic.actions.arguments.update_arguments(delete_arguments, archive=archive_name),
+        global_arguments,
+        local_path,
+        remote_path,
+    )

borgmatic/actions/export_key.py

@@ -1,7 +1,6 @@
 import logging
 
 import borgmatic.borg.export_key
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -18,17 +17,13 @@ def run_export_key(
     '''
     Run the "key export" action for the given repository.
     '''
-    if export_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        export_arguments.repository,
-    ):
-        logger.info('Exporting repository key')
-        borgmatic.borg.export_key.export_key(
-            repository['path'],
-            config,
-            local_borg_version,
-            export_arguments,
-            global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-        )
+    logger.info('Exporting repository key')
+    borgmatic.borg.export_key.export_key(
+        repository['path'],
+        config,
+        local_borg_version,
+        export_arguments,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+    )

borgmatic/actions/export_tar.py

@@ -2,7 +2,6 @@ import logging
 
 import borgmatic.borg.export_tar
 import borgmatic.borg.repo_list
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -19,30 +18,26 @@ def run_export_tar(
     '''
     Run the "export-tar" action for the given repository.
     '''
-    if export_tar_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        export_tar_arguments.repository,
-    ):
-        logger.info(f'Exporting archive {export_tar_arguments.archive} as tar file')
-        borgmatic.borg.export_tar.export_tar_archive(
-            global_arguments.dry_run,
+    logger.info(f'Exporting archive {export_tar_arguments.archive} as tar file')
+    borgmatic.borg.export_tar.export_tar_archive(
+        global_arguments.dry_run,
+        repository['path'],
+        borgmatic.borg.repo_list.resolve_archive_name(
             repository['path'],
-            borgmatic.borg.repo_list.resolve_archive_name(
-                repository['path'],
-                export_tar_arguments.archive,
-                config,
-                local_borg_version,
-                global_arguments,
-                local_path,
-                remote_path,
-            ),
-            export_tar_arguments.paths,
-            export_tar_arguments.destination,
+            export_tar_arguments.archive,
             config,
             local_borg_version,
             global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-            tar_filter=export_tar_arguments.tar_filter,
-            strip_components=export_tar_arguments.strip_components,
-        )
+            local_path,
+            remote_path,
+        ),
+        export_tar_arguments.paths,
+        export_tar_arguments.destination,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+        tar_filter=export_tar_arguments.tar_filter,
+        strip_components=export_tar_arguments.strip_components,
+    )

borgmatic/actions/extract.py

@@ -2,8 +2,6 @@ import logging
 
 import borgmatic.borg.extract
 import borgmatic.borg.repo_list
-import borgmatic.config.validate
-import borgmatic.hooks.command
 
 logger = logging.getLogger(__name__)
 
@@ -21,29 +19,25 @@ def run_extract(
     '''
     Run the "extract" action for the given repository.
     '''
-    if extract_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        extract_arguments.repository,
-    ):
-        logger.info(f'Extracting archive {extract_arguments.archive}')
-        borgmatic.borg.extract.extract_archive(
-            global_arguments.dry_run,
+    logger.info(f'Extracting archive {extract_arguments.archive}')
+    borgmatic.borg.extract.extract_archive(
+        global_arguments.dry_run,
+        repository['path'],
+        borgmatic.borg.repo_list.resolve_archive_name(
             repository['path'],
-            borgmatic.borg.repo_list.resolve_archive_name(
-                repository['path'],
-                extract_arguments.archive,
-                config,
-                local_borg_version,
-                global_arguments,
-                local_path,
-                remote_path,
-            ),
-            extract_arguments.paths,
+            extract_arguments.archive,
             config,
             local_borg_version,
             global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-            destination_path=extract_arguments.destination,
-            strip_components=extract_arguments.strip_components,
-        )
+            local_path,
+            remote_path,
+        ),
+        extract_arguments.paths,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+        destination_path=extract_arguments.destination,
+        strip_components=extract_arguments.strip_components,
+    )

borgmatic/actions/import_key.py

@@ -1,7 +1,6 @@
 import logging
 
 import borgmatic.borg.import_key
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -18,17 +17,13 @@ def run_import_key(
     '''
     Run the "key import" action for the given repository.
     '''
-    if import_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        import_arguments.repository,
-    ):
-        logger.info('Importing repository key')
-        borgmatic.borg.import_key.import_key(
-            repository['path'],
-            config,
-            local_borg_version,
-            import_arguments,
-            global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-        )
+    logger.info('Importing repository key')
+    borgmatic.borg.import_key.import_key(
+        repository['path'],
+        config,
+        local_borg_version,
+        import_arguments,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+    )

borgmatic/actions/info.py

@@ -4,7 +4,6 @@ import borgmatic.actions.arguments
 import borgmatic.actions.json
 import borgmatic.borg.info
 import borgmatic.borg.repo_list
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -23,30 +22,26 @@ def run_info(
 
     If info_arguments.json is True, yield the JSON output from the info for the archive.
     '''
-    if info_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        info_arguments.repository,
-    ):
-        if not info_arguments.json:
-            logger.answer('Displaying archive summary information')
+    if not info_arguments.json:
+        logger.answer('Displaying archive summary information')
 
-        archive_name = borgmatic.borg.repo_list.resolve_archive_name(
-            repository['path'],
-            info_arguments.archive,
-            config,
-            local_borg_version,
-            global_arguments,
-            local_path,
-            remote_path,
-        )
-        json_output = borgmatic.borg.info.display_archives_info(
-            repository['path'],
-            config,
-            local_borg_version,
-            borgmatic.actions.arguments.update_arguments(info_arguments, archive=archive_name),
-            global_arguments,
-            local_path,
-            remote_path,
-        )
-        if json_output:
-            yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))
+    archive_name = borgmatic.borg.repo_list.resolve_archive_name(
+        repository['path'],
+        info_arguments.archive,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+    json_output = borgmatic.borg.info.display_archives_info(
+        repository['path'],
+        config,
+        local_borg_version,
+        borgmatic.actions.arguments.update_arguments(info_arguments, archive=archive_name),
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+    if json_output:
+        yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))

borgmatic/actions/list.py

@@ -3,7 +3,6 @@ import logging
 import borgmatic.actions.arguments
 import borgmatic.actions.json
 import borgmatic.borg.list
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -22,33 +21,29 @@ def run_list(
 
     If list_arguments.json is True, yield the JSON output from listing the archive.
     '''
-    if list_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        list_arguments.repository,
-    ):
-        if not list_arguments.json:
-            if list_arguments.find_paths:  # pragma: no cover
-                logger.answer('Searching archives')
-            elif not list_arguments.archive:  # pragma: no cover
-                logger.answer('Listing archives')
+    if not list_arguments.json:
+        if list_arguments.find_paths:  # pragma: no cover
+            logger.answer('Searching archives')
+        elif not list_arguments.archive:  # pragma: no cover
+            logger.answer('Listing archives')
 
-        archive_name = borgmatic.borg.repo_list.resolve_archive_name(
-            repository['path'],
-            list_arguments.archive,
-            config,
-            local_borg_version,
-            global_arguments,
-            local_path,
-            remote_path,
-        )
-        json_output = borgmatic.borg.list.list_archive(
-            repository['path'],
-            config,
-            local_borg_version,
-            borgmatic.actions.arguments.update_arguments(list_arguments, archive=archive_name),
-            global_arguments,
-            local_path,
-            remote_path,
-        )
-        if json_output:
-            yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))
+    archive_name = borgmatic.borg.repo_list.resolve_archive_name(
+        repository['path'],
+        list_arguments.archive,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+    json_output = borgmatic.borg.list.list_archive(
+        repository['path'],
+        config,
+        local_borg_version,
+        borgmatic.actions.arguments.update_arguments(list_arguments, archive=archive_name),
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+    if json_output:
+        yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))

borgmatic/actions/mount.py

@@ -2,7 +2,6 @@ import logging
 
 import borgmatic.borg.mount
 import borgmatic.borg.repo_list
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -19,30 +18,26 @@ def run_mount(
     '''
     Run the "mount" action for the given repository.
     '''
-    if mount_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        mount_arguments.repository,
-    ):
-        if mount_arguments.archive:
-            logger.info(f'Mounting archive {mount_arguments.archive}')
-        else:  # pragma: nocover
-            logger.info('Mounting repository')
+    if mount_arguments.archive:
+        logger.info(f'Mounting archive {mount_arguments.archive}')
+    else:  # pragma: nocover
+        logger.info('Mounting repository')
 
-        borgmatic.borg.mount.mount_archive(
+    borgmatic.borg.mount.mount_archive(
+        repository['path'],
+        borgmatic.borg.repo_list.resolve_archive_name(
             repository['path'],
-            borgmatic.borg.repo_list.resolve_archive_name(
-                repository['path'],
-                mount_arguments.archive,
-                config,
-                local_borg_version,
-                global_arguments,
-                local_path,
-                remote_path,
-            ),
-            mount_arguments,
+            mount_arguments.archive,
             config,
             local_borg_version,
             global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-        )
+            local_path,
+            remote_path,
+        ),
+        mount_arguments,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+    )

borgmatic/actions/pattern.py

@@ -259,31 +259,37 @@ def deduplicate_runtime_directory_patterns(patterns, config, borgmatic_runtime_d
         return patterns
 
     deduplicated = {}  # Use just the keys as an ordered set.
+    runtime_directory_parents = set(pathlib.PurePath(borgmatic_runtime_directory).parents).union(
+        {pathlib.PurePath(borgmatic_runtime_directory)}
+    )
 
     for pattern in patterns:
         if pattern.type != borgmatic.borg.pattern.Pattern_type.ROOT:
             deduplicated[pattern] = True
             continue
 
-        parents = pathlib.PurePath(pattern.path).parents
+        pattern_parents = pathlib.PurePath(pattern.path).parents
 
-        # If another directory in the given list is a parent of current directory (even n levels up)
-        # and both are on the same filesystem (or one_file_system is not set), then the current
-        # directory is a duplicate.
+        # If:
+        #
+        #   1. another pattern is a parent of the current pattern (even n levels up),
+        #   2. both patterns are parents of the runtime directory (even n levels up),
+        #   3. and both patterns are on the same filesystem (or one_file_system is not set)
+        #
+        # ... then consider the current pattern as a duplicate.
         for other_pattern in patterns:
             if other_pattern.type != borgmatic.borg.pattern.Pattern_type.ROOT:
                 continue
 
-            if any(
-                pathlib.PurePath(other_pattern.path) == parent
-                and pathlib.PurePosixPath(other_pattern.path)
-                in pathlib.PurePath(borgmatic_runtime_directory).parents
-                and pattern.device is not None
-                and (
-                    other_pattern.device == pattern.device
-                    or config.get('one_file_system') is not True
-                )
-                for parent in parents
+            device_matches = pattern.device is not None and (
+                other_pattern.device == pattern.device or config.get('one_file_system') is not True
+            )
+
+            if (
+                pathlib.PurePath(other_pattern.path) in pattern_parents
+                and pathlib.PurePosixPath(other_pattern.path) in runtime_directory_parents
+                and pathlib.PurePosixPath(pattern.path) in runtime_directory_parents
+                and device_matches
             ):
                 break
         else:

borgmatic/actions/prune.py

@@ -1,8 +1,6 @@
 import logging
 
 import borgmatic.borg.prune
-import borgmatic.config.validate
-import borgmatic.hooks.command
 
 logger = logging.getLogger(__name__)
 
@@ -21,12 +19,6 @@ def run_prune(
     '''
     Run the "prune" action for the given repository.
     '''
-    if prune_arguments.repository and not borgmatic.config.validate.repositories_match(
-        repository,
-        prune_arguments.repository,
-    ):
-        return
-
     logger.info(f'Pruning archives{dry_run_label}')
     borgmatic.borg.prune.prune_archives(
         global_arguments.dry_run,

borgmatic/actions/recreate.py

@@ -4,7 +4,6 @@ import subprocess
 import borgmatic.borg.info
 import borgmatic.borg.recreate
 import borgmatic.borg.repo_list
-import borgmatic.config.validate
 from borgmatic.actions.pattern import collect_patterns, process_patterns
 
 logger = logging.getLogger(__name__)
@@ -25,64 +24,60 @@ def run_recreate(
     '''
     Run the "recreate" action for the given repository.
     '''
-    if recreate_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        recreate_arguments.repository,
-    ):
-        if recreate_arguments.archive:
-            logger.answer(f'Recreating archive {recreate_arguments.archive}')
-        else:
-            logger.answer('Recreating repository')
+    if recreate_arguments.archive:
+        logger.answer(f'Recreating archive {recreate_arguments.archive}')
+    else:
+        logger.answer('Recreating repository')
 
-        # Collect and process patterns.
-        processed_patterns = process_patterns(
-            collect_patterns(config),
-            config,
-            borgmatic.config.paths.get_working_directory(config),
+    # Collect and process patterns.
+    processed_patterns = process_patterns(
+        collect_patterns(config),
+        config,
+        borgmatic.config.paths.get_working_directory(config),
+    )
+
+    archive = borgmatic.borg.repo_list.resolve_archive_name(
+        repository['path'],
+        recreate_arguments.archive,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+
+    if archive and archive.endswith('.recreate'):
+        if recreate_arguments.archive == 'latest':
+            raise ValueError(
+                f'The latest archive "{archive}" is leftover from a prior recreate. Delete it first or select a different archive.',
+            )
+
+        raise ValueError(
+            f'The archive "{recreate_arguments.archive}" is leftover from a prior recreate. Select a different archive.',
         )
 
-        archive = borgmatic.borg.repo_list.resolve_archive_name(
+    try:
+        borgmatic.borg.recreate.recreate_archive(
             repository['path'],
-            recreate_arguments.archive,
+            archive,
             config,
             local_borg_version,
+            recreate_arguments,
             global_arguments,
-            local_path,
-            remote_path,
+            local_path=local_path,
+            remote_path=remote_path,
+            patterns=processed_patterns,
         )
-
-        if archive and archive.endswith('.recreate'):
-            if recreate_arguments.archive == 'latest':
+    except subprocess.CalledProcessError as error:
+        if error.returncode == BORG_EXIT_CODE_ARCHIVE_ALREADY_EXISTS:
+            if recreate_arguments.target:
                 raise ValueError(
-                    f'The latest archive "{archive}" is leftover from a prior recreate. Delete it first or select a different archive.',
+                    f'The archive "{recreate_arguments.target}" already exists. Delete it first or set a different target archive name.',
                 )
 
-            raise ValueError(
-                f'The archive "{recreate_arguments.archive}" is leftover from a prior recreate. Select a different archive.',
-            )
+            if archive:
+                raise ValueError(
+                    f'The archive "{archive}.recreate" is leftover from a prior recreate. Delete it first or select a different archive.',
+                )
 
-        try:
-            borgmatic.borg.recreate.recreate_archive(
-                repository['path'],
-                archive,
-                config,
-                local_borg_version,
-                recreate_arguments,
-                global_arguments,
-                local_path=local_path,
-                remote_path=remote_path,
-                patterns=processed_patterns,
-            )
-        except subprocess.CalledProcessError as error:
-            if error.returncode == BORG_EXIT_CODE_ARCHIVE_ALREADY_EXISTS:
-                if recreate_arguments.target:
-                    raise ValueError(
-                        f'The archive "{recreate_arguments.target}" already exists. Delete it first or set a different target archive name.',
-                    )
-
-                if archive:
-                    raise ValueError(
-                        f'The archive "{archive}.recreate" is leftover from a prior recreate. Delete it first or select a different archive.',
-                    )
-
-            raise
+        raise

borgmatic/actions/repo_create.py

@@ -1,7 +1,6 @@
 import logging
 
 import borgmatic.borg.repo_create
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -18,12 +17,6 @@ def run_repo_create(
     '''
     Run the "repo-create" action for the given repository.
     '''
-    if repo_create_arguments.repository and not borgmatic.config.validate.repositories_match(
-        repository,
-        repo_create_arguments.repository,
-    ):
-        return
-
     logger.info('Creating repository')
 
     encryption_mode = repo_create_arguments.encryption_mode or repository.get('encryption')

borgmatic/actions/repo_delete.py

@@ -17,20 +17,16 @@ def run_repo_delete(
     '''
     Run the "repo-delete" action for the given repository.
     '''
-    if repo_delete_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        repo_delete_arguments.repository,
-    ):
-        logger.answer(
-            'Deleting repository' + (' cache' if repo_delete_arguments.cache_only else ''),
-        )
+    logger.answer(
+        'Deleting repository' + (' cache' if repo_delete_arguments.cache_only else ''),
+    )
 
-        borgmatic.borg.repo_delete.delete_repository(
-            repository,
-            config,
-            local_borg_version,
-            repo_delete_arguments,
-            global_arguments,
-            local_path,
-            remote_path,
-        )
+    borgmatic.borg.repo_delete.delete_repository(
+        repository,
+        config,
+        local_borg_version,
+        repo_delete_arguments,
+        global_arguments,
+        local_path,
+        remote_path,
+    )

borgmatic/actions/repo_info.py

@@ -2,7 +2,6 @@ import logging
 
 import borgmatic.actions.json
 import borgmatic.borg.repo_info
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -21,21 +20,17 @@ def run_repo_info(
 
     If repo_info_arguments.json is True, yield the JSON output from the info for the repository.
     '''
-    if repo_info_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        repo_info_arguments.repository,
-    ):
-        if not repo_info_arguments.json:
-            logger.answer('Displaying repository summary information')
+    if not repo_info_arguments.json:
+        logger.answer('Displaying repository summary information')
 
-        json_output = borgmatic.borg.repo_info.display_repository_info(
-            repository['path'],
-            config,
-            local_borg_version,
-            repo_info_arguments=repo_info_arguments,
-            global_arguments=global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-        )
-        if json_output:
-            yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))
+    json_output = borgmatic.borg.repo_info.display_repository_info(
+        repository['path'],
+        config,
+        local_borg_version,
+        repo_info_arguments=repo_info_arguments,
+        global_arguments=global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+    )
+    if json_output:
+        yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))

borgmatic/actions/repo_list.py

@@ -2,7 +2,6 @@ import logging
 
 import borgmatic.actions.json
 import borgmatic.borg.repo_list
-import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
 
@@ -21,21 +20,17 @@ def run_repo_list(
 
     If repo_list_arguments.json is True, yield the JSON output from listing the repository.
     '''
-    if repo_list_arguments.repository is None or borgmatic.config.validate.repositories_match(
-        repository,
-        repo_list_arguments.repository,
-    ):
-        if not repo_list_arguments.json:
-            logger.answer('Listing repository')
+    if not repo_list_arguments.json:
+        logger.answer('Listing repository')
 
-        json_output = borgmatic.borg.repo_list.list_repository(
-            repository['path'],
-            config,
-            local_borg_version,
-            repo_list_arguments=repo_list_arguments,
-            global_arguments=global_arguments,
-            local_path=local_path,
-            remote_path=remote_path,
-        )
-        if json_output:
-            yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))
+    json_output = borgmatic.borg.repo_list.list_repository(
+        repository['path'],
+        config,
+        local_borg_version,
+        repo_list_arguments=repo_list_arguments,
+        global_arguments=global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+    )
+    if json_output:
+        yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))

borgmatic/actions/restore.py

@@ -8,10 +8,8 @@ import tempfile
 import borgmatic.actions.pattern
 import borgmatic.borg.extract
 import borgmatic.borg.list
-import borgmatic.borg.mount
 import borgmatic.borg.repo_list
 import borgmatic.config.paths
-import borgmatic.config.validate
 import borgmatic.hooks.data_source.dump
 import borgmatic.hooks.dispatch
 
@@ -524,18 +522,11 @@ def run_restore(
     remote_path,
 ):
     '''
-    Run the "restore" action for the given repository, but only if the repository matches the
-    requested repository in restore arguments.
+    Run the "restore" action for the given repository.
 
     Raise ValueError if a configured data source could not be found to restore or there's no
     matching dump in the archive.
     '''
-    if restore_arguments.repository and not borgmatic.config.validate.repositories_match(
-        repository,
-        restore_arguments.repository,
-    ):
-        return
-
     logger.info(f'Restoring data sources from archive {restore_arguments.archive}')
     working_directory = borgmatic.config.paths.get_working_directory(config)
 

borgmatic/borg/create.py

@@ -96,32 +96,29 @@ def validate_planned_backup_paths(
         if path_line and path_line.startswith(('- ', '+ '))
     )
 
-    # These are the subset of output paths contained within the borgmatic runtime directory.
-    paths_inside_runtime_directory = {
-        path for path in paths if any_parent_directories(path, (borgmatic_runtime_directory,))
-    }
-
-    # If the runtime directory isn't present in the source patterns, then we shouldn't expect it to
-    # be in the paths output from the Borg dry run.
-    runtime_directory_present_in_patterns = any(
+    runtime_directory_root_patterns = tuple(
         pattern
         for pattern in patterns
         if any_parent_directories(pattern.path, (borgmatic_runtime_directory,))
         if pattern.type == borgmatic.borg.pattern.Pattern_type.ROOT
     )
 
-    # If no paths to backup are inside the runtime directory, it must've been excluded.
-    if (
-        not paths_inside_runtime_directory
-        and runtime_directory_present_in_patterns
-        and not dry_run
-        and os.path.exists(borgmatic_runtime_directory)
-    ):
-        raise ValueError(
-            f'The runtime directory {os.path.normpath(borgmatic_runtime_directory)} overlaps with the configured excludes or patterns with excludes. Please ensure the runtime directory is not excluded.',
-        )
+    if not dry_run and os.path.exists(borgmatic_runtime_directory):
+        # If there are any root patterns in the runtime directory that are missing from the paths
+        # Borg is planning to backup, then they must've gotten excluded, e.g. by user-configured
+        # excludes. Error accordingly.
+        for pattern in runtime_directory_root_patterns:
+            if not any(any_parent_directories(path, (pattern.path,)) for path in paths):
+                raise ValueError(
+                    f'The runtime directory {os.path.normpath(borgmatic_runtime_directory)} overlaps with the configured excludes or patterns with excludes. Please ensure the runtime directory is not excluded.',
+                )
 
-    return tuple(path for path in paths if path not in paths_inside_runtime_directory)
+    # Return the subset of output paths *not* contained within the borgmatic runtime directory. The
+    # intent is that any downstream checks using these paths should skip runtime paths that
+    # borgmatic uses for its own bookkeeping, instead focusing on user-configured paths.
+    return tuple(
+        path for path in paths if not any_parent_directories(path, (borgmatic_runtime_directory,))
+    )
 
 
 MAX_SPECIAL_FILE_PATHS_LENGTH = 1000

borgmatic/borg/import_key.py

@@ -3,7 +3,6 @@ import os
 import shlex
 
 import borgmatic.config.paths
-import borgmatic.logger
 from borgmatic.borg import environment, flags
 from borgmatic.execute import DO_NOT_CAPTURE, execute_command
 

borgmatic/borg/list.py

@@ -18,6 +18,7 @@ MAKE_FLAGS_EXCLUDES = (
     'archive',
     'paths',
     'find_paths',
+    'format',
     *ARCHIVE_FILTER_FLAGS_MOVED_TO_REPO_LIST,
 )
 
@@ -54,6 +55,7 @@ def make_list_command(
         + flags.make_flags('umask', config.get('umask'))
         + flags.make_flags('log-json', config.get('log_json'))
         + flags.make_flags('lock-wait', config.get('lock_wait'))
+        + flags.make_flags('format', list_arguments.format or config.get('file_list_format'))
         + flags.make_flags_from_arguments(list_arguments, excludes=MAKE_FLAGS_EXCLUDES)
         + (tuple(shlex.split(extra_borg_options)) if extra_borg_options else ())
         + (

borgmatic/borg/repo_delete.py

@@ -27,7 +27,14 @@ def make_repo_delete_command(
     arguments to the repo_delete action as an argparse.Namespace, and global arguments, return a command
     as a tuple to repo_delete the entire repository.
     '''
-    extra_borg_options = config.get('extra_borg_options', {}).get('repo_delete', '')
+    extra_borg_options = config.get('extra_borg_options', {}).get(
+        'repo_delete'
+        if borgmatic.borg.feature.available(
+            borgmatic.borg.feature.Feature.REPO_DELETE, local_borg_version
+        )
+        else 'delete',
+        '',
+    )
 
     return (
         (local_path,)

borgmatic/borg/repo_info.py

@@ -25,7 +25,10 @@ def display_repository_info(
     '''
     borgmatic.logger.add_custom_log_levels()
     lock_wait = config.get('lock_wait', None)
-    extra_borg_options = config.get('extra_borg_options', {}).get('repo_info', '')
+    extra_borg_options = config.get('extra_borg_options', {}).get(
+        'repo_info' if feature.available(feature.Feature.REPO_INFO, local_borg_version) else 'info',
+        '',
+    )
 
     full_command = (
         (local_path,)

borgmatic/borg/repo_list.py

@@ -63,7 +63,10 @@ def get_latest_archive(
 
     Raises ValueError if there are no archives in the repository.
     '''
-    extra_borg_options = config.get('extra_borg_options', {}).get('repo_list', '')
+    extra_borg_options = config.get('extra_borg_options', {}).get(
+        'repo_list' if feature.available(feature.Feature.REPO_LIST, local_borg_version) else 'list',
+        '',
+    )
 
     full_command = (
         local_path,
@@ -107,7 +110,7 @@ def get_latest_archive(
     return latest_archive
 
 
-MAKE_FLAGS_EXCLUDES = ('repository', 'prefix', 'match_archives')
+MAKE_FLAGS_EXCLUDES = ('repository', 'format', 'prefix', 'match_archives')
 
 
 def make_repo_list_command(
@@ -124,7 +127,10 @@ def make_repo_list_command(
     arguments to the repo_list action, global arguments as an argparse.Namespace instance, and local and
     remote Borg paths, return a command as a tuple to list archives with a repository.
     '''
-    extra_borg_options = config.get('extra_borg_options', {}).get('repo_list', '')
+    extra_borg_options = config.get('extra_borg_options', {}).get(
+        'repo_list' if feature.available(feature.Feature.REPO_LIST, local_borg_version) else 'list',
+        '',
+    )
 
     return (
         (
@@ -164,6 +170,9 @@ def make_repo_list_command(
                 )
             )
         )
+        + flags.make_flags(
+            'format', repo_list_arguments.format or config.get('archive_list_format')
+        )
         + flags.make_flags_from_arguments(repo_list_arguments, excludes=MAKE_FLAGS_EXCLUDES)
         + (tuple(shlex.split(extra_borg_options)) if extra_borg_options else ())
         + flags.make_repository_flags(repository_path, local_borg_version)

borgmatic/borg/state.py

@@ -1 +1 @@
-DEFAULT_BORGMATIC_SOURCE_DIRECTORY = '~/.borgmatic'
+DEFAULT_BORGMATIC_SOURCE_DIRECTORY = '~/.borgmatic'  # pragma: nocover

borgmatic/commands/arguments.py

@@ -297,7 +297,14 @@ def parse_arguments_for_actions(unparsed_arguments, action_parsers, global_parse
     )
 
 
-OMITTED_FLAG_NAMES = {'match-archives', 'progress', 'statistics', 'list-details'}
+OMITTED_FLAG_NAMES = {
+    'match-archives',
+    'progress',
+    'statistics',
+    'list-details',
+    'file-list-format',
+    'archive-list-format',
+}
 
 
 def make_argument_description(schema, flag_name):
@@ -568,8 +575,7 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
     ignoring actions, and the combined parser is handy for displaying help that includes everything:
     global flags, a list of actions, etc.
     '''
-    config_paths = collect.get_default_config_paths(expand_home=True)
-    unexpanded_config_paths = collect.get_default_config_paths(expand_home=False)
+    config_paths = collect.get_default_config_paths()
 
     # Using allow_abbrev=False here prevents the global parser from erroring about "ambiguous"
     # options like --encryption. Such options are intended for an action parser rather than the
@@ -582,7 +588,7 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         '--config',
         dest='config_paths',
         action='append',
-        help=f"Configuration filename or directory, can specify flag multiple times, defaults to: -c {' -c '.join(unexpanded_config_paths)}",
+        help='Configuration filename or directory, can specify flag multiple times, defaults to /etc/borgmatic/config.yaml, /etc/borgmatic.d, $XDG_CONFIG_HOME/borgmatic/config.yaml, and $XDG_CONFIG_HOME/borgmatic.d, where $XDG_CONFIG_HOME defaults to $HOME/.config',
     )
     global_group.add_argument(
         '-n',
@@ -1215,7 +1221,7 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         '--destination',
         dest='destination_path',
         default=config_paths[0],
-        help=f'Destination configuration file (or directory if using --split), default: {unexpanded_config_paths[0]}',
+        help='Destination configuration file (or directory if using --split), default: /etc/borgmatic/config.yaml',
     )
     config_generate_group.add_argument(
         '--overwrite',
@@ -1542,7 +1548,7 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         action='store_true',
         help='Output only archive names',
     )
-    repo_list_group.add_argument('--format', help='Format for archive listing')
+    repo_list_group.add_argument('--format', help='Borg format for the archive listing')
     repo_list_group.add_argument(
         '--json',
         default=False,
@@ -1645,7 +1651,7 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         action='store_true',
         help='Output only path names',
     )
-    list_group.add_argument('--format', help='Format for file listing')
+    list_group.add_argument('--format', help='Borg format for the file listing')
     list_group.add_argument(
         '--json',
         default=False,
@@ -2015,7 +2021,7 @@ def parse_arguments(schema, *unparsed_arguments):
     )
 
     if not arguments['global'].config_paths:
-        arguments['global'].config_paths = collect.get_default_config_paths(expand_home=True)
+        arguments['global'].config_paths = collect.get_default_config_paths()
 
     for action_name in ('bootstrap', 'generate', 'validate'):
         if action_name in arguments and len(arguments) > HIGHLANDER_ACTION_ARGUMENTS_COUNT:

borgmatic/commands/borgmatic.py

@@ -340,7 +340,7 @@ def run_actions(  # noqa: PLR0912, PLR0915
     '''
     Given parsed command-line arguments as an argparse.ArgumentParser instance, the configuration
     filename, a configuration dict, a sequence of loaded configuration paths, local and remote paths
-    to Borg, a local Borg version string, and a repository name, run all actions from the
+    to Borg, a local Borg version string, and a repository dict, run all actions from the
     command-line arguments on the given repository.
 
     Yield JSON output strings from executing any actions that produce JSON.
@@ -362,6 +362,22 @@ def run_actions(  # noqa: PLR0912, PLR0915
         'repository': repository_path,
     }
     skip_actions = set(get_skip_actions(config, arguments))
+    requested_repository = next(
+        (
+            repository
+            for action_arguments in arguments.values()
+            for repository in (getattr(action_arguments, 'repository', None),)
+            if repository is not None
+        ),
+        None,
+    )
+
+    if requested_repository and not borgmatic.config.validate.repositories_match(
+        repository,
+        requested_repository,
+    ):
+        logger.debug('Skipping actions because the requested --repository does not match')
+        return
 
     with borgmatic.hooks.command.Before_after_hooks(
         command_hooks=config.get('commands'),

borgmatic/config/collect.py

@@ -1,17 +1,14 @@
 import os
 
 
-def get_default_config_paths(expand_home=True):
+def get_default_config_paths():
     '''
-    Based on the value of the XDG_CONFIG_HOME and HOME environment variables, return a list of
-    default configuration paths. This includes both system-wide configuration and configuration in
-    the current user's home directory.
-
-    Don't expand the home directory ($HOME) if the expand home flag is False.
+    Return a list of default configuration paths. This includes both system-wide
+    configuration and configuration in the current user's home directory.
     '''
-    user_config_directory = os.getenv('XDG_CONFIG_HOME') or os.path.join('$HOME', '.config')
-    if expand_home:
-        user_config_directory = os.path.expandvars(user_config_directory)
+    user_config_directory = os.path.expandvars(
+        os.getenv('XDG_CONFIG_HOME') or os.path.join('$HOME', '.config')
+    )
 
     return [
         '/etc/borgmatic/config.yaml',

borgmatic/config/generate.py

@@ -1,6 +1,7 @@
 import collections
 import contextlib
 import io
+import json
 import os
 import re
 
@@ -316,6 +317,24 @@ def merge_source_configuration_into_destination(destination_config, source_confi
     return destination_config
 
 
+def get_configuration_subset(config, option_name):  # pragma: no cover
+    '''
+    Given configuration as a ruamel.yaml.CommentedMap and an option name found within it at the top
+    level, return a new CommentedMap containing a subset of the configuration with only the given
+    option and no other top-level options.
+
+    This is useful when generating the sample configuration for a single option instead of a whole
+    configuration file.
+    '''
+    option_config = ruamel.yaml.CommentedMap({option_name: config[option_name]})
+
+    # Due to a quirk of ruamel.yaml, the comment right before a top-level key is not on that key and
+    # needs to get copied separately.
+    option_config.ca.items[option_name] = config.ca.items[option_name]
+
+    return option_config
+
+
 def generate_sample_configuration(
     dry_run,
     source_filename,
@@ -359,16 +378,22 @@ def generate_sample_configuration(
 
         os.makedirs(destination_path, exist_ok=True)
 
-        for option_name, option_config in destination_config.items():
+        for option_name in destination_config:
             write_configuration(
                 os.path.join(destination_path, f'{option_name}.yaml'),
                 transform_optional_configuration(
-                    render_configuration({option_name: option_config}),
+                    render_configuration(get_configuration_subset(destination_config, option_name)),
                     comment_out=False,
-                ),
+                ).strip(),
                 overwrite=overwrite,
             )
 
+        # Also dump a manifest listing all the options we've written.
+        json.dump(
+            {'option_names': list(destination_config.keys())},
+            open(os.path.join(destination_path, 'options.json'), 'w', encoding='utf-8'),
+        )
+
         return
 
     if os.path.exists(destination_path) and not os.path.isfile(destination_path):

borgmatic/config/paths.py

@@ -2,10 +2,19 @@ import contextlib
 import logging
 import os
 import tempfile
+from enum import Enum
 
 logger = logging.getLogger(__name__)
 
 
+class Systemd_directories(Enum):
+    RUNTIME_DIRECTORY = 0
+    STATE_DIRECTORY = 1
+    CACHE_DIRECTORY = 2
+    LOGS_DIRECTORY = 3
+    CONFIGURATION_DIRECTORY = 4
+
+
 def expand_user_in_path(path):
     '''
     Given a directory path, expand any tildes in it.
@@ -16,6 +25,17 @@ def expand_user_in_path(path):
         return None
 
 
+def resolve_systemd_directory(directory):
+    '''
+    Given a systemd directory environment variable enum, read the value if set and return the first
+    configured directory.
+    '''
+    separator = ':'
+
+    paths = os.environ.get(directory.name)
+    return paths.split(separator)[0] if paths else None
+
+
 def get_working_directory(config):  # pragma: no cover
     '''
     Given a configuration dict, get the working directory from it, expanding any tildes.
@@ -96,7 +116,9 @@ class Runtime_directory:
         runtime_directory = (
             config.get('user_runtime_directory')
             or os.environ.get('XDG_RUNTIME_DIR')  # Set by PAM on Linux.
-            or os.environ.get('RUNTIME_DIRECTORY')  # Set by systemd if configured.
+            or resolve_systemd_directory(
+                Systemd_directories.RUNTIME_DIRECTORY
+            )  # Set by systemd if configured.
         )
 
         if runtime_directory:
@@ -174,7 +196,9 @@ def get_borgmatic_state_directory(config):
         os.path.join(
             config.get('user_state_directory')
             or os.environ.get('XDG_STATE_HOME')
-            or os.environ.get('STATE_DIRECTORY')  # Set by systemd if configured.
+            or resolve_systemd_directory(
+                Systemd_directories.STATE_DIRECTORY
+            )  # Set by systemd if configured.
             or '~/.local/state',
             'borgmatic',
         ),

borgmatic/config/schema.yaml

@@ -173,9 +173,13 @@ properties:
             type: string
         description: |
             Any paths matching these patterns are included/excluded from
-            backups. Globs are expanded. (Tildes are not.) See the output of
-            "borg help patterns" for more details. Quote any value if it
-            contains leading punctuation, so it parses correctly.
+            backups. Recursion root patterns ("R ...") are effectively the same
+            as "source_directories"; they tell Borg which paths to backup
+            (modulo any excludes). Globs are expanded. (Tildes are not.) See
+            the output of "borg help patterns" for more details. Quote any value
+            if it contains leading punctuation, so it parses correctly. Also use
+            leading slashes in absolute paths, or data source hooks may be
+            unable to rewrite patterns as needed.
         example:
             - 'R /'
             - '- /home/*/.cache'
@@ -259,7 +263,7 @@ properties:
             Path for storing temporary runtime data like streaming database
             dumps and bootstrap metadata. borgmatic automatically creates and
             uses a "borgmatic" subdirectory here. Defaults to $XDG_RUNTIME_DIR
-            or or $TMPDIR or $TEMP or /run/user/$UID.
+            or $TMPDIR or $TEMP or /run/user/$UID.
         example: /run/user/1001
     user_state_directory:
         type: string
@@ -490,6 +494,27 @@ properties:
             If match_archives is not specified, borgmatic defaults to deriving
             the match_archives value from archive_name_format.
         example: "sh:{hostname}-*"
+    file_list_format:
+        type: string
+        description: |
+            Borg format for the files listing of the "list" action. Corresponds
+            to the "--format" flag. Defaults to "{mode} {user:6} {group:6}
+            {size:8} {mtime} {path}{extra}{NL}". With "--json", the form of the
+            format is ignored, but the keys used in it are added to the JSON
+            output. See
+            https://borgbackup.readthedocs.io/en/stable/usage/list.html for
+            details.
+        example: "{path}{extra}{NL}"
+    archive_list_format:
+        type: string
+        description: |
+            Borg format for the archives listing of the "repo-list" action.
+            Corresponds to the "--format" flag. Defaults to "{archive:<36}
+            {time} [{id}]{NL}". With "--json", the form of the format is
+            ignored, but the keys used in it are added to the JSON output. See
+            https://borgbackup.readthedocs.io/en/stable/usage/list.html for
+            details.
+        example: "{archive}{NL}"
     relocated_repo_access_is_ok:
         type: boolean
         description: |
@@ -607,6 +632,11 @@ properties:
                 description: |
                   Extra command-line options to pass to "borg recreate".
                 example: "--extra-option"
+            rename:
+                type: string
+                description: |
+                  Extra command-line options to pass to "borg rename".
+                example: "--extra-option"
             repo_create:
                 type: string
                 description: |
@@ -991,6 +1021,8 @@ properties:
         type: string
         description: |
             Python format string used for log messages written to the log file.
+            See https://docs.python.org/3/library/logging.html (and specifically
+            the LogRecord attributes with "{}-formatting") for details.
         example: "[{asctime}] {levelname}: {prefix}{message}"
     monitoring_verbosity:
         type: integer
@@ -1687,6 +1719,20 @@ properties:
                     description: |
                         Port to restore to. Defaults to the "port" option.
                     example: 5433
+                socket_path:
+                    type: string
+                    description: |
+                        Path of a Unix socket to connect to instead of a remote
+                        hostname. Ignored when "hostname" is set and not
+                        "localhost". Defaults to "/run/mysqld/mysqld.sock".
+                    example: database.example.org
+                restore_socket_path:
+                    type: string
+                    description: |
+                        Path of a Unix socket to connect to instead of a remote
+                        hostname. Ignored when "hostname" is set and not
+                        "localhost". Defaults to the "socket_path" option.
+                    example: database.example.org
                 username:
                     type: string
                     description: |
@@ -1750,18 +1796,21 @@ properties:
                 mariadb_dump_command:
                     type: string
                     description: |
-                        Command to use instead of "mariadb-dump". This can be
-                        used to run a specific mariadb_dump version (e.g., one
-                        inside a running container). If you run it from within a
-                        container, make sure to mount the path in the
-                        "user_runtime_directory" option from the host into the
-                        container at the same location. Defaults to
-                        "mariadb-dump".
+                        Command to use instead of "mariadb-dump". But set any
+                        additional command-line flags in "options", not here.
+                        This command can be used to run a specific mariadb_dump
+                        version (e.g., one inside a running container). If you
+                        run it from within a container, make sure to mount the
+                        path in the "user_runtime_directory" option from the
+                        host into the container at the same location. Defaults
+                        to "mariadb-dump".
                     example: docker exec mariadb_container mariadb-dump
                 mariadb_command:
                     type: string
                     description: |
-                        Command to run instead of "mariadb". This can be used to
+                        Command to run instead of "mariadb". But set any
+                        additional command-line flags in "list_options" or
+                        "restore_options", not here. This command can be used to
                         run a specific mariadb version (e.g., one inside a
                         running container). Defaults to "mariadb".
                     example: docker exec mariadb_container mariadb           
@@ -1878,6 +1927,20 @@ properties:
                     description: |
                         Port to restore to. Defaults to the "port" option.
                     example: 5433
+                socket_path:
+                    type: string
+                    description: |
+                        Path of a Unix socket to connect to instead of a remote
+                        hostname. Ignored when "hostname" is set and not
+                        "localhost". Defaults to "/run/mysqld/mysqld.sock".
+                    example: database.example.org
+                restore_socket_path:
+                    type: string
+                    description: |
+                        Path of a Unix socket to connect to instead of a remote
+                        hostname. Ignored when "hostname" is set and not
+                        "localhost". Defaults to the "socket_path" option.
+                    example: database.example.org
                 username:
                     type: string
                     description: |
@@ -1941,17 +2004,21 @@ properties:
                 mysql_dump_command:
                     type: string
                     description: |
-                        Command to use instead of "mysqldump". This can be used
-                        to run a specific mysql_dump version (e.g., one inside a
-                        running container). If you run it from within a
-                        container, make sure to mount the path in the
-                        "user_runtime_directory" option from the host into the
-                        container at the same location. Defaults to "mysqldump".
+                        Command to use instead of "mysqldump". But set any
+                        additional command-line flags in "options", not here.
+                        This command can be used to run a specific mysql_dump
+                        version (e.g., one inside a running container). If you
+                        run it from within a container, make sure to mount the
+                        path in the "user_runtime_directory" option from the
+                        host into the container at the same location. Defaults
+                        to "mysqldump".
                     example: docker exec mysql_container mysqldump
                 mysql_command:
                     type: string
                     description: |
-                        Command to run instead of "mysql". This can be used to
+                        Command to run instead of "mysql". But set any
+                        additional command-line flags in "list_options" or
+                        "restore_options", not here. This command can be used to
                         run a specific mysql version (e.g., one inside a running
                         container). Defaults to "mysql".
                     example: docker exec mysql_container mysql
@@ -3015,6 +3082,12 @@ properties:
                     project monitor. Used along with the data source name URL to
                     construct a cron URL.
                 example: mymonitor
+            environment:
+                type: string
+                description: |
+                    Sentry monitor environment used in the call to Sentry. If
+                    not set, the Sentry default is used.
+                example: production
             states:
                 type: array
                 items:
@@ -3159,6 +3232,14 @@ properties:
                 description: |
                     Command to use instead of "keepassxc-cli".
                 example: /usr/local/bin/keepassxc-cli
+            ask_for_password:
+                type: boolean
+                description: |
+                    Whether keepassxc-cli should prompt the user for a password.
+                    Disabling this is only really useful if you're unlocking
+                    your KeePassXC database with a key file instead of a
+                    password. Defaults to true.
+                example: false
             key_file:
                 type: string
                 description: |

borgmatic/execute.py

@@ -158,8 +158,8 @@ def log_outputs(processes, exclude_stdouts, output_log_level, borg_local_path, b
                     if not line or not ready_process:
                         break
 
-                    # Keep the last few lines of output in case the process errors, and we need the output for
-                    # the exception below.
+                    # Keep the last few lines of output in case the process errors, and we need the
+                    # output for the exception below.
                     append_last_lines(
                         buffer_last_lines[ready_buffer],
                         captured_outputs[ready_process],
@@ -199,7 +199,7 @@ def log_outputs(processes, exclude_stdouts, output_log_level, borg_local_path, b
                         last_lines,
                         captured_outputs[process],
                         line,
-                        output_log_level=logging.ERROR,
+                        output_log_level,
                     )
 
                 if len(last_lines) == ERROR_OUTPUT_MAX_LINE_COUNT:

borgmatic/hooks/credential/keepassxc.py

@@ -29,6 +29,7 @@ def load_credential(hook_config, config, credential_parameters):
     command = (
         tuple(shlex.split((hook_config or {}).get('keepassxc_cli_command', 'keepassxc-cli')))
         + ('show', '--show-protected', '--attributes', 'Password')
+        + (('--no-password',) if not (hook_config or {}).get('ask_for_password', True) else ())
         + (
             ('--key-file', hook_config['key_file'])
             if hook_config and hook_config.get('key_file')

borgmatic/hooks/data_source/btrfs.py

@@ -23,33 +23,24 @@ def use_streaming(hook_config, config):  # pragma: no cover
     return False
 
 
+BTRFS_SUBVOLUME_INODE_NUMBER = 256
+
+
 @functools.cache
-def path_is_a_subvolume(btrfs_command, path):
+def path_is_a_subvolume(path):
     '''
-    Given a btrfs command and a path, return whether the path is a Btrfs subvolume. Return False if
-    the btrfs command errors, which probably indicates there isn't a containing Btrfs subvolume for
-    the given path.
+    Given a path, return whether it is a Btrfs subvolume. Return False if the path doesn't exist.
 
-    As a performance optimization, multiple calls to this function with the same arguments are
-    cached.
+    As a performance optimization, multiple calls to this function with the same path are cached.
     '''
+    if path == os.environ.get('BTRFS_TEST_SUBVOLUME_PATH'):  # pragma: no cover
+        return True
+
     try:
-        borgmatic.execute.execute_command(
-            (
-                *btrfs_command.split(' '),
-                'subvolume',
-                'show',
-                path,
-            ),
-            output_log_level=None,
-            close_fds=True,
-        )
-    # An error from the command (probably) indicates that the path is not actually a subvolume.
-    except subprocess.CalledProcessError:
+        return os.stat(path).st_ino == BTRFS_SUBVOLUME_INODE_NUMBER
+    except FileNotFoundError:
         return False
 
-    return True
-
 
 @functools.cache
 def get_subvolume_property(btrfs_command, subvolume_path, property_name):
@@ -99,7 +90,7 @@ def get_containing_subvolume_path(btrfs_command, path):
         path,
         *tuple(str(ancestor) for ancestor in pathlib.PurePath(path).parents),
     ):
-        if not path_is_a_subvolume(btrfs_command, candidate_path):
+        if not path_is_a_subvolume(candidate_path):
             continue
 
         try:
@@ -146,8 +137,8 @@ Subvolume = collections.namedtuple('Subvolume', ('path', 'contained_patterns'),
 def get_subvolumes(btrfs_command, patterns):
     '''
     Given a Btrfs command to run and a sequence of configured patterns, find the intersection
-    between the current Btrfs filesystem and subvolume paths and the paths of any patterns. The
-    idea is that these pattern paths represent the requested subvolumes to snapshot.
+    between the current Btrfs filesystem/subvolume paths and the paths of any patterns. The idea is
+    that these pattern paths represent the requested subvolumes to snapshot.
 
     Only include subvolumes that contain at least one root pattern sourced from borgmatic
     configuration (as opposed to generated elsewhere in borgmatic).
@@ -180,7 +171,7 @@ def get_subvolumes(btrfs_command, patterns):
     return tuple(sorted(subvolumes, key=lambda subvolume: subvolume.path))
 
 
-BORGMATIC_SNAPSHOT_PREFIX = '.borgmatic-snapshot-'
+BORGMATIC_SNAPSHOT_PREFIX = '.borgmatic-snapshot'
 
 
 def make_snapshot_path(subvolume_path):
@@ -189,7 +180,7 @@ def make_snapshot_path(subvolume_path):
     '''
     return os.path.join(
         subvolume_path,
-        f'{BORGMATIC_SNAPSHOT_PREFIX}{os.getpid()}',
+        f'{BORGMATIC_SNAPSHOT_PREFIX}',
         # Included so that the snapshot ends up in the Borg archive at the "original" subvolume path.
     ) + subvolume_path.rstrip(os.path.sep)
 
@@ -202,16 +193,16 @@ def make_snapshot_exclude_pattern(subvolume_path):  # pragma: no cover
     directory within the snapshot itself. For instance, if you have a Btrfs subvolume at /mnt and
     make a snapshot of it at:
 
-        /mnt/.borgmatic-snapshot-1234/mnt
+        /mnt/.borgmatic-snapshot/mnt
 
     ... then the snapshot itself will have an empty directory at:
 
-        /mnt/.borgmatic-snapshot-1234/mnt/.borgmatic-snapshot-1234
+        /mnt/.borgmatic-snapshot/mnt/.borgmatic-snapshot
 
     So to prevent that from ending up in the Borg archive, this function produces an exclude pattern
     to exclude that path.
     '''
-    snapshot_directory = f'{BORGMATIC_SNAPSHOT_PREFIX}{os.getpid()}'
+    snapshot_directory = f'{BORGMATIC_SNAPSHOT_PREFIX}'
 
     return borgmatic.borg.pattern.Pattern(
         os.path.join(
@@ -244,7 +235,7 @@ def make_borg_snapshot_pattern(subvolume_path, pattern):
 
     rewritten_path = initial_caret + os.path.join(
         subvolume_path,
-        f'{BORGMATIC_SNAPSHOT_PREFIX}{os.getpid()}',
+        f'{BORGMATIC_SNAPSHOT_PREFIX}',
         # Use the Borg 1.4+ "slashdot" hack to prevent the snapshot path prefix from getting
         # included in the archive—but only if there's not already a slashdot hack present in the
         # pattern.
@@ -274,7 +265,6 @@ def snapshot_subvolume(btrfs_command, subvolume_path, snapshot_path):  # pragma:
             *btrfs_command.split(' '),
             'subvolume',
             'snapshot',
-            '-r',  # Read-only.
             subvolume_path,
             snapshot_path,
         ),

borgmatic/hooks/data_source/config.py

@@ -105,18 +105,20 @@ def get_ip_from_container(container):
     )
 
 
-def inject_pattern(patterns, data_source_pattern):
+def inject_pattern(patterns, data_source_pattern, override_excludes=True):
     '''
     Given a list of borgmatic.borg.pattern.Pattern instances representing the configured patterns,
     insert the given data source pattern at the start of the list. The idea is that borgmatic is
     injecting its own custom pattern specific to a data source hook into the user's configured
     patterns so that the hook's data gets included in the backup.
 
-    As part of this injection, if the data source pattern is a root pattern, also insert an
-    "include" version of the given root pattern, in an attempt to preempt any of the user's
-    configured exclude patterns that may follow.
+    As part of this injection, if the data source pattern is a root pattern and override_excludes is
+    True, also insert an "include" version of the given root pattern, in an attempt to preempt any
+    of the user's configured exclude patterns that may follow. The is to support use cases like
+    borgmatic injecting its own patterns for things like database dumps or bootstrap metadata, where
+    we don't want them to get accidentally excluded.
     '''
-    if data_source_pattern.type == borgmatic.borg.pattern.Pattern_type.ROOT:
+    if data_source_pattern.type == borgmatic.borg.pattern.Pattern_type.ROOT and override_excludes:
         patterns.insert(
             0,
             borgmatic.borg.pattern.Pattern(
@@ -138,30 +140,14 @@ def replace_pattern(patterns, pattern_to_replace, data_source_pattern):
     replacing a configured pattern with its own modified pattern specific to a data source hook so
     that the hook's data gets included in the backup.
 
-    As part of this replacement, if the data source pattern is a root pattern, also insert an
-    "include" version of the given root pattern right after the replaced pattern, in an attempt to
-    preempt any of the user's configured exclude patterns that may follow.
-
     If the pattern to replace can't be found in the given patterns, then just inject the data source
     pattern at the start of the list.
     '''
     try:
         index = patterns.index(pattern_to_replace)
     except ValueError:
-        inject_pattern(patterns, data_source_pattern)
+        inject_pattern(patterns, data_source_pattern, override_excludes=False)
 
         return
 
     patterns[index] = data_source_pattern
-
-    if data_source_pattern.type == borgmatic.borg.pattern.Pattern_type.ROOT:
-        patterns.insert(
-            index + 1,
-            borgmatic.borg.pattern.Pattern(
-                path=data_source_pattern.path,
-                type=borgmatic.borg.pattern.Pattern_type.INCLUDE,
-                style=data_source_pattern.style,
-                device=data_source_pattern.device,
-                source=borgmatic.borg.pattern.Pattern_source.HOOK,
-            ),
-        )

borgmatic/hooks/data_source/mariadb.py

@@ -132,6 +132,8 @@ def database_names_to_dump(database, config, username, password, environment, dr
     extra_options, defaults_extra_filename = parse_extra_options(database.get('list_options'))
     password_transport = database.get('password_transport', 'pipe')
     hostname = database_config.resolve_database_option('hostname', database)
+    socket_path = database.get('socket_path')
+
     show_command = (
         mariadb_show_command
         + (
@@ -143,6 +145,7 @@ def database_names_to_dump(database, config, username, password, environment, dr
         + (('--host', hostname) if hostname else ())
         + (('--port', str(database['port'])) if 'port' in database else ())
         + (('--protocol', 'tcp') if hostname or 'port' in database else ())
+        + (('--socket', socket_path) if socket_path else ())
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if database.get('tls') is True else ())
         + (('--skip-ssl',) if database.get('tls') is False else ())
@@ -155,7 +158,11 @@ def database_names_to_dump(database, config, username, password, environment, dr
     if skip_names:
         logger.debug(f'Skipping database names: {", ".join(skip_names)}')
 
-    show_output = execute_command_and_capture_output(show_command, environment=environment)
+    show_output = execute_command_and_capture_output(
+        show_command,
+        environment=environment,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
+    )
 
     return tuple(
         show_name
@@ -209,6 +216,8 @@ def execute_dump_command(
     extra_options, defaults_extra_filename = parse_extra_options(database.get('options'))
     password_transport = database.get('password_transport', 'pipe')
     hostname = database_config.resolve_database_option('hostname', database)
+    socket_path = database.get('socket_path')
+
     dump_command = (
         mariadb_dump_command
         + (
@@ -218,9 +227,11 @@ def execute_dump_command(
         )
         + extra_options
         + (('--add-drop-database',) if database.get('add_drop_database', True) else ())
+        + ('--single-transaction',)
         + (('--host', hostname) if hostname else ())
         + (('--port', str(database['port'])) if 'port' in database else ())
         + (('--protocol', 'tcp') if hostname or 'port' in database else ())
+        + (('--socket', socket_path) if socket_path else ())
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if database.get('tls') is True else ())
         + (('--skip-ssl',) if database.get('tls') is False else ())
@@ -239,6 +250,7 @@ def execute_dump_command(
         dump_command,
         environment=environment,
         run_to_completion=False,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
     )
 
 
@@ -443,6 +455,7 @@ def restore_data_source_dump(
     port = database_config.resolve_database_option(
         'port', data_source, connection_params, restore=True
     )
+    socket_path = database_config.resolve_database_option('socket_path', data_source, restore=True)
     tls = database_config.resolve_database_option('tls', data_source, restore=True)
     username = borgmatic.hooks.credential.parse.resolve_credential(
         database_config.resolve_database_option(
@@ -462,6 +475,7 @@ def restore_data_source_dump(
     )
     extra_options, defaults_extra_filename = parse_extra_options(data_source.get('restore_options'))
     password_transport = data_source.get('password_transport', 'pipe')
+
     restore_command = (
         mariadb_restore_command
         + (
@@ -474,6 +488,7 @@ def restore_data_source_dump(
         + (('--host', hostname) if hostname else ())
         + (('--port', str(port)) if port else ())
         + (('--protocol', 'tcp') if hostname or port else ())
+        + (('--socket', socket_path) if socket_path else ())
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if tls is True else ())
         + (('--skip-ssl',) if tls is False else ())
@@ -495,4 +510,5 @@ def restore_data_source_dump(
         output_log_level=logging.DEBUG,
         input_file=extract_process.stdout,
         environment=environment,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
     )

borgmatic/hooks/data_source/mongodb.py

@@ -90,11 +90,20 @@ def dump_data_sources(
 
         if dump_format == 'directory':
             dump.create_parent_directory_for_dump(dump_filename)
-            execute_command(command, shell=True)  # noqa: S604
+            execute_command(  # noqa: S604
+                command,
+                shell=True,
+                working_directory=borgmatic.config.paths.get_working_directory(config),
+            )
         else:
             dump.create_named_pipe_for_dump(dump_filename)
             processes.append(
-                execute_command(command, shell=True, run_to_completion=False),  # noqa: S604
+                execute_command(  # noqa: S604
+                    command,
+                    shell=True,
+                    run_to_completion=False,
+                    working_directory=borgmatic.config.paths.get_working_directory(config),
+                ),
             )
 
     if not dry_run:
@@ -273,6 +282,7 @@ def restore_data_source_dump(
         [extract_process] if extract_process else [],
         output_log_level=logging.DEBUG,
         input_file=extract_process.stdout if extract_process else None,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
     )
 
 

borgmatic/hooks/data_source/mysql.py

@@ -57,6 +57,8 @@ def database_names_to_dump(database, config, username, password, environment, dr
     )
     password_transport = database.get('password_transport', 'pipe')
     hostname = database_config.resolve_database_option('hostname', database)
+    socket_path = database.get('socket_path')
+
     show_command = (
         mysql_show_command
         + (
@@ -72,6 +74,7 @@ def database_names_to_dump(database, config, username, password, environment, dr
         + (('--host', hostname) if hostname else ())
         + (('--port', str(database['port'])) if 'port' in database else ())
         + (('--protocol', 'tcp') if hostname or 'port' in database else ())
+        + (('--socket', socket_path) if socket_path else ())
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if database.get('tls') is True else ())
         + (('--skip-ssl',) if database.get('tls') is False else ())
@@ -84,7 +87,11 @@ def database_names_to_dump(database, config, username, password, environment, dr
     if skip_names:
         logger.debug(f'Skipping database names: {", ".join(skip_names)}')
 
-    show_output = execute_command_and_capture_output(show_command, environment=environment)
+    show_output = execute_command_and_capture_output(
+        show_command,
+        environment=environment,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
+    )
 
     return tuple(
         show_name
@@ -136,6 +143,8 @@ def execute_dump_command(
     )
     password_transport = database.get('password_transport', 'pipe')
     hostname = database_config.resolve_database_option('hostname', database)
+    socket_path = database.get('socket_path')
+
     dump_command = (
         mysql_dump_command
         + (
@@ -149,9 +158,11 @@ def execute_dump_command(
         )
         + extra_options
         + (('--add-drop-database',) if database.get('add_drop_database', True) else ())
+        + ('--single-transaction',)
         + (('--host', hostname) if hostname else ())
         + (('--port', str(database['port'])) if 'port' in database else ())
         + (('--protocol', 'tcp') if hostname or 'port' in database else ())
+        + (('--socket', socket_path) if socket_path else ())
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if database.get('tls') is True else ())
         + (('--skip-ssl',) if database.get('tls') is False else ())
@@ -170,6 +181,7 @@ def execute_dump_command(
         dump_command,
         environment=environment,
         run_to_completion=False,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
     )
 
 
@@ -374,6 +386,7 @@ def restore_data_source_dump(
     port = database_config.resolve_database_option(
         'port', data_source, connection_params, restore=True
     )
+    socket_path = database_config.resolve_database_option('socket_path', data_source, restore=True)
     tls = database_config.resolve_database_option('tls', data_source, restore=True)
     username = borgmatic.hooks.credential.parse.resolve_credential(
         database_config.resolve_database_option(
@@ -395,6 +408,7 @@ def restore_data_source_dump(
         borgmatic.hooks.data_source.mariadb.parse_extra_options(data_source.get('restore_options'))
     )
     password_transport = data_source.get('password_transport', 'pipe')
+
     restore_command = (
         mysql_restore_command
         + (
@@ -411,6 +425,7 @@ def restore_data_source_dump(
         + (('--host', hostname) if hostname else ())
         + (('--port', str(port)) if port else ())
         + (('--protocol', 'tcp') if hostname or port else ())
+        + (('--socket', socket_path) if socket_path else ())
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if tls is True else ())
         + (('--skip-ssl',) if tls is False else ())
@@ -432,4 +447,5 @@ def restore_data_source_dump(
         output_log_level=logging.DEBUG,
         input_file=extract_process.stdout,
         environment=environment,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
     )

borgmatic/hooks/data_source/postgresql.py

@@ -103,7 +103,11 @@ def database_names_to_dump(database, config, environment, dry_run):
         + (tuple(database['list_options'].split(' ')) if 'list_options' in database else ())
     )
     logger.debug('Querying for "all" PostgreSQL databases to dump')
-    list_output = execute_command_and_capture_output(list_command, environment=environment)
+    list_output = execute_command_and_capture_output(
+        list_command,
+        environment=environment,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
+    )
 
     return tuple(
         row[0]
@@ -245,6 +249,7 @@ def dump_data_sources(
                     command,
                     shell=True,
                     environment=environment,
+                    working_directory=borgmatic.config.paths.get_working_directory(config),
                 )
             else:
                 dump.create_named_pipe_for_dump(dump_filename)
@@ -254,6 +259,7 @@ def dump_data_sources(
                         shell=True,
                         environment=environment,
                         run_to_completion=False,
+                        working_directory=borgmatic.config.paths.get_working_directory(config),
                     ),
                 )
 
@@ -422,5 +428,10 @@ def restore_data_source_dump(
         output_log_level=logging.DEBUG,
         input_file=extract_process.stdout if extract_process else None,
         environment=environment,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
+    )
+    execute_command(
+        analyze_command,
+        environment=environment,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
     )
-    execute_command(analyze_command, environment=environment)

borgmatic/hooks/data_source/sqlite.py

@@ -86,6 +86,7 @@ def dump_data_sources(
         )
         command = (
             *sqlite_command,
+            '-bail',
             shlex.quote(database_path),
             '.dump',
             '>',
@@ -100,7 +101,12 @@ def dump_data_sources(
 
         dump.create_named_pipe_for_dump(dump_filename)
         processes.append(
-            execute_command(command, shell=True, run_to_completion=False),  # noqa: S604
+            execute_command(  # noqa: S604
+                command,
+                shell=True,
+                run_to_completion=False,
+                working_directory=borgmatic.config.paths.get_working_directory(config),
+            ),
         )
 
     if not dry_run:
@@ -196,7 +202,7 @@ def restore_data_source_dump(
         shlex.quote(part)
         for part in shlex.split(data_source.get('sqlite_restore_command') or 'sqlite3')
     )
-    restore_command = (*sqlite_restore_command, shlex.quote(database_path))
+    restore_command = (*sqlite_restore_command, '-bail', shlex.quote(database_path))
     # Don't give Borg local path so as to error on warnings, as "borg extract" only gives a warning
     # if the restore paths don't exist in the archive.
     execute_command_with_processes(
@@ -204,4 +210,5 @@ def restore_data_source_dump(
         [extract_process],
         output_log_level=logging.DEBUG,
         input_file=extract_process.stdout,
+        working_directory=borgmatic.config.paths.get_working_directory(config),
     )

borgmatic/hooks/dispatch.py

@@ -3,7 +3,6 @@ import importlib
 import logging
 import pkgutil
 
-import borgmatic.hooks.command
 import borgmatic.hooks.credential
 import borgmatic.hooks.data_source
 import borgmatic.hooks.monitoring

borgmatic/hooks/monitoring/sentry.py

@@ -1,5 +1,6 @@
 import logging
 import re
+import urllib
 
 import requests
 
@@ -40,6 +41,7 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
     dry_run_label = ' (dry run; not actually pinging)' if dry_run else ''
 
     data_source_name_url = hook_config.get('data_source_name_url')
+    environment = hook_config.get('environment')
     monitor_slug = hook_config.get('monitor_slug')
     match = DATA_SOURCE_NAME_URL_PATTERN.match(data_source_name_url)
 
@@ -65,10 +67,12 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
     if dry_run:
         return
 
+    environment_query = f'&environment={urllib.parse.quote(environment)}' if environment else ''
+
     logging.getLogger('urllib3').setLevel(logging.ERROR)
     try:
         response = requests.post(
-            f'{cron_url}?status={status}',
+            f'{cron_url}?status={status}{environment_query}',
             timeout=TIMEOUT_SECONDS,
             headers={'User-Agent': 'borgmatic'},
         )

docs/Dockerfile

@@ -24,6 +24,7 @@ RUN npm install @11ty/eleventy \
     markdown-it-anchor \
     markdown-it-replace-link
 COPY --from=borgmatic /etc/borgmatic/* /source/docs/_includes/borgmatic/
+COPY --from=borgmatic /etc/borgmatic/options.json /source/docs/reference/configuration/index.json
 COPY --from=borgmatic /command-line/* /source/docs/_includes/borgmatic/command-line/
 COPY --from=borgmatic /contributors.html /source/docs/_includes/borgmatic/contributors.html
 COPY . /source

docs/how-to/add-preparation-and-cleanup-steps-to-backups.md

@@ -1,5 +1,5 @@
 ---
-title: How to add preparation and cleanup steps to backups
+title: 🧹 How to add preparation and cleanup steps to backups
 eleventyNavigation:
   key: 🧹 Add preparation and cleanup steps
   parent: How-to guides

docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server.md

@@ -1,5 +1,5 @@
 ---
-title: How to backup to a removable drive or an intermittent server
+title: 💾 How to backup to a removable drive or an intermittent server
 eleventyNavigation:
   key: 💾 Backup to a removable drive/server
   parent: How-to guides

docs/how-to/backup-your-databases.md

@@ -1,5 +1,5 @@
 ---
-title: How to backup your databases
+title: 🗄️ How to backup your databases
 eleventyNavigation:
   key: 🗄️ Backup your databases
   parent: How-to guides
@@ -249,6 +249,10 @@ container separate from borgmatic.
 A similar approach can work with MySQL, using `mysql_dump_command` instead of
 `mariadb_dump_command` to run `mysqldump` in a container.
 
+Note: With MariaDB and MySQL, use `options`, `list_options`, and/or
+`restore_options` to override command-line flags rather than putting flags into
+`mariadb_dump_command`, `mysql_dump_command`, etc.
+
 
 #### Database client in a temporary container
 
@@ -544,12 +548,13 @@ Or from the configuration file:
 postgresql_databases:
     - name: users
         hostname: database1.example.org
-        restore_hostname: database1.example.org
+        restore_hostname: database2.example.org
         restore_port: 5433
         restore_username: postgres
         restore_password: trustsome1
 ```
 
+
 ### Manual restoration
 
 If you prefer to restore a database without the help of borgmatic, first
@@ -654,9 +659,14 @@ configuration. Here's an example with MariaDB:
 ```yaml
 mariadb_databases:
     - name: posts
-      options: "--single-transaction --quick"
+      options: "--single-transaction"
 ```
 
+<span class="minilink minilink-addedin">New in version 2.0.13</span> borgmatic
+passes `--single-transaction` to MariaDB/MySQL by default, and you no longer
+need to set that in `options:`.
+
+
 ### borgmatic hangs during backup
 
 See Limitations above about `read_special`. You may need to exclude certain

docs/how-to/customize-warnings-and-errors.md

@@ -1,5 +1,5 @@
 ---
-title: How to customize warnings and errors
+title: 💥 How to customize warnings and errors
 eleventyNavigation:
   key: 💥 Customize warnings/errors
   parent: How-to guides

docs/how-to/deal-with-very-large-backups.md

@@ -1,5 +1,5 @@
 ---
-title: How to deal with very large backups
+title: 📏 How to deal with very large backups
 eleventyNavigation:
   key: 📏 Deal with very large backups
   parent: How-to guides

docs/how-to/develop-on-borgmatic.md

@@ -1,10 +1,12 @@
 ---
-title: How to develop on borgmatic
+title: 🏗️ How to develop on borgmatic
 eleventyNavigation:
   key: 🏗️ Develop on borgmatic
   parent: How-to guides
   order: 15
 ---
+<span data-pagefind-weight="0.25">
+
 To get set up to develop on borgmatic, first [`install
 uv`](https://docs.astral.sh/uv/) to make managing your borgmatic environment
 easier without impacting other Python applications on your system.
@@ -205,3 +207,5 @@ borgmatic's developer build for documentation optionally supports using
 Setting up Podman is outside the scope of this documentation. But once you
 install and configure Podman, then `scripts/dev-docs` should automatically use
 Podman instead of Docker.
+
+</span>

docs/how-to/extract-a-backup.md

@@ -1,5 +1,5 @@
 ---
-title: How to extract a backup
+title: 📤 How to extract a backup
 eleventyNavigation:
   key: 📤 Extract a backup
   parent: How-to guides

docs/how-to/inspect-your-backups.md

@@ -1,5 +1,5 @@
 ---
-title: How to inspect your backups
+title: 🔎 How to inspect your backups
 eleventyNavigation:
   key: 🔎 Inspect your backups
   parent: How-to guides

docs/how-to/make-backups-redundant.md

@@ -1,5 +1,5 @@
 ---
-title: How to make backups redundant
+title: ☁️  How to make backups redundant
 eleventyNavigation:
   key: ☁️  Make backups redundant
   parent: How-to guides
@@ -40,10 +40,11 @@ Here's a way of visualizing what borgmatic does with the above configuration:
 This gives you redundancy of your data across repositories and even
 potentially across providers.
 
-See [Borg repository URLs
-documentation](https://borgbackup.readthedocs.io/en/stable/usage/general.html#repository-urls)
+See [repositories
+documentation](https://torsion.org/borgmatic/reference/configuration/repositories/)
 for more information on how to specify local and remote repository paths.
 
+
 ### Different options per repository
 
 What if you want borgmatic to backup to multiple repositories—while also

docs/how-to/make-per-application-backups.md

@@ -1,5 +1,5 @@
 ---
-title: How to make per-application backups
+title: 🗂️ How to make per-application backups
 eleventyNavigation:
   key: 🗂️ Make per-application backups
   parent: How-to guides
@@ -59,6 +59,20 @@ entry using borgmatic's `--config` flag instead of relying on
 `/etc/borgmatic.d`.
 
 
+## Limitations
+
+borgmatic does not currently support its own parallelism—being run multiple
+times on the same machine simultaneously. In particular, many of the [data
+source
+hooks](https://torsion.org/borgmatic/reference/configuration/data-sources/) rely
+on global borgmatic runtime files which can't be shared across processes, and
+therefore multiple borgmatic instances on the same machine would interfere with
+each other.
+
+A single borgmatic instance also doesn't currently support running multiple Borg
+instances in parallel on the same machine.
+
+
 <a id="archive-naming"></a>
 <a id="configuration-includes"></a>
 <a id="configuration-overrides"></a>

docs/how-to/monitor-your-backups.md

@@ -1,5 +1,5 @@
 ---
-title: How to monitor your backups
+title: 🚨 How to monitor your backups
 eleventyNavigation:
   key: 🚨 Monitor your backups
   parent: How-to guides

docs/how-to/provide-your-passwords.md

@@ -1,5 +1,5 @@
 ---
-title: How to provide your passwords
+title: 🔒 How to provide your passwords
 eleventyNavigation:
   key: 🔒 Provide your passwords
   parent: How-to guides

docs/how-to/run-arbitrary-borg-commands.md

@@ -1,5 +1,5 @@
 ---
-title: How to run arbitrary Borg commands
+title: 🔧 How to run arbitrary Borg commands
 eleventyNavigation:
   key: 🔧 Run arbitrary Borg commands
   parent: How-to guides

docs/how-to/set-up-backups.md

@@ -1,5 +1,5 @@
 ---
-title: How to set up backups
+title: 📥 How to set up backups
 eleventyNavigation:
   key: 📥 Set up backups
   parent: How-to guides
@@ -75,12 +75,16 @@ with `sudo`. If that doesn't work, you may need to update your [sudoers
 Besides the approaches described above, there are several other options for
 installing borgmatic:
 
- * [container image with scheduled backups](https://hub.docker.com/r/b3vis/borgmatic/) (+ Docker Compose files)
- * [container image with multi-arch and Docker CLI support](https://hub.docker.com/r/modem7/borgmatic-docker/)
+#### <span data-pagefind-weight="7.0">Docker / Podman</span>
+
+ * [container image with scheduled backups](https://github.com/borgmatic-collective/docker-borgmatic) (+ Docker Compose files)
+ * [container image with multi-arch and Docker CLI support](https://github.com/modem7/docker-borgmatic)
+
+#### Operating system packages
+
  * [Debian](https://tracker.debian.org/pkg/borgmatic)
  * [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
- * [Fedora official](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
- * [Fedora unofficial](https://copr.fedorainfracloud.org/coprs/heffer/borgmatic/)
+ * [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
  * [Gentoo](https://packages.gentoo.org/packages/app-backup/borgmatic)
  * [Arch Linux](https://archlinux.org/packages/extra/any/borgmatic/)
  * [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=borgmatic)
@@ -88,9 +92,11 @@ installing borgmatic:
  * [openSUSE](https://software.opensuse.org/package/borgmatic)
  * [macOS (via Homebrew)](https://formulae.brew.sh/formula/borgmatic)
  * [macOS (via MacPorts)](https://ports.macports.org/port/borgmatic/)
- * [NixOS](https://search.nixos.org/packages?show=borgmatic&sort=relevance&type=packages&query=borgmatic)
+ * [NixOS](https://search.nixos.org/packages?channel=unstable&show=borgmatic&query=borgmatic)
+
+#### Etc.
+
  * [Ansible role](https://github.com/borgbase/ansible-role-borgbackup)
- * [Unraid](https://unraid.net/community/apps?q=borgmatic#r)
  * [uv tool install](https://docs.astral.sh/uv/)
 
 
@@ -150,6 +156,9 @@ added new options since you originally created your configuration file. Also
 check out how to [upgrade your
 configuration](https://torsion.org/borgmatic/how-to/upgrade/#upgrading-your-configuration).
 
+And see the documentation on [patterns and
+excludes](https://torsion.org/borgmatic/reference/configuration/patterns-and-excludes/).
+
 
 ### Encryption
 
@@ -205,15 +214,20 @@ edits are valid.
 
 Before you can create backups with borgmatic, you first need to create a Borg
 repository so you have a destination for your backup archives. (But skip this
-step if you already have a Borg repository.) To create a repository, run a
-command like the following with Borg 1.x:
+step if you already have a Borg repository.) To create a repository, assuming
+it's already [specified in borgmatic's
+configuration](https://torsion.org/borgmatic/reference/configuration/repositories/),
+run a command like the following with Borg 1.x:
 
 ```bash
-sudo borgmatic init --encryption repokey
+sudo borgmatic repo-create --encryption repokey
 ```
 
-<span class="minilink minilink-addedin">New in borgmatic version 1.9.0</span>
-Or, with Borg 2.x:
+<span class="minilink minilink-addedin">Prior to borgmatic version 1.9.0</span>
+The `repo-create` action was called `init`.
+
+<span class="minilink minilink-addedin">With Borg version 2.x</span> Borg 2.x
+uses more specific encryption modes like `repokey-aes-ocb`. For example:
 
 ```bash
 sudo borgmatic repo-create --encryption repokey-aes-ocb
@@ -341,48 +355,18 @@ Access](https://projects.torsion.org/borgmatic-collective/borgmatic/issues/293).
 
 ## Niceties
 
+<a id="shell-completion"></a>
+<a id="bash"></a>
+<a id="fish"></a>
 
-### Shell completion
 
-borgmatic includes a shell completion script (currently only for Bash and Fish) to
-support tab-completing borgmatic command-line actions and flags. Depending on
-how you installed borgmatic, this may be enabled by default.
+### Command-line autocompletion
 
-#### Bash
+borgmatic includes autocompletion scripts for various command-line shells to
+support tab-completing command-line actions and flags. See the [autocompletion
+documentation](https://torsion.org/borgmatic/reference/command-line/autocompletion/)
+for details.
 
-If completions aren't enabled, start by installing the `bash-completion` Linux package or the
-[`bash-completion@2`](https://formulae.brew.sh/formula/bash-completion@2)
-macOS Homebrew formula. Then, install the shell completion script globally:
-
-```bash
-sudo su -c "borgmatic --bash-completion > $(pkg-config --variable=completionsdir bash-completion)/borgmatic"
-```
-
-If you don't have `pkg-config` installed, you can try the following path
-instead:
-
-```bash
-sudo su -c "borgmatic --bash-completion > /usr/share/bash-completion/completions/borgmatic"
-```
-
-Or, if you'd like to install the script for only the current user:
-
-```bash
-mkdir --parents ~/.local/share/bash-completion/completions
-borgmatic --bash-completion > ~/.local/share/bash-completion/completions/borgmatic
-```
-
-Finally, restart your shell (`exit` and open a new shell) so the completions
-take effect.
-
-#### fish
-
-To add completions for fish, install the completions file globally:
-
-```fish
-borgmatic --fish-completion | sudo tee /usr/share/fish/vendor_completions.d/borgmatic.fish
-source /usr/share/fish/vendor_completions.d/borgmatic.fish
-```
 
 ### Colored output
 

docs/how-to/snapshot-your-filesystems.md

@@ -1,5 +1,5 @@
 ---
-title: How to snapshot your filesystems
+title: 📸 How to snapshot your filesystems
 eleventyNavigation:
   key: 📸 Snapshot your filesystems
   parent: How-to guides

docs/how-to/upgrade.md

@@ -1,5 +1,5 @@
 ---
-title: How to upgrade borgmatic and Borg
+title: 📦 How to upgrade borgmatic and Borg
 eleventyNavigation:
   key: 📦 Upgrade borgmatic/Borg
   parent: How-to guides

docs/reference/command-line/actions/index.md

@@ -1,5 +1,5 @@
 ---
-title: Actions
+title: 🎬 Actions
 eleventyNavigation:
   key: 🎬 Actions
   parent: 💻 Command-line

docs/reference/command-line/autocompletion/bash.md

@@ -0,0 +1,36 @@
+---
+title: bash
+eleventyNavigation:
+  key: bash
+  parent: ➡️  Autocompletion
+---
+borgmatic includes an autocompletion script for
+[Bash](https://www.gnu.org/software/bash/) to support tab-completing
+command-line actions and flags. Depending on how you installed borgmatic, this
+may be enabled by default.
+
+If completions aren't enabled, start by installing the `bash-completion` Linux
+package or the
+[`bash-completion@2`](https://formulae.brew.sh/formula/bash-completion@2) macOS
+Homebrew formula. Then, install the shell completion script globally:
+
+```bash
+sudo su -c "borgmatic --bash-completion > $(pkg-config --variable=completionsdir bash-completion)/borgmatic"
+```
+
+If you don't have `pkg-config` installed, you can try the following path
+instead:
+
+```bash
+sudo su -c "borgmatic --bash-completion > /usr/share/bash-completion/completions/borgmatic"
+```
+
+Or, if you'd like to install the script for only the current user:
+
+```bash
+mkdir --parents ~/.local/share/bash-completion/completions
+borgmatic --bash-completion > ~/.local/share/bash-completion/completions/borgmatic
+```
+
+Finally, restart your shell (`exit` and open a new shell) so the completions
+take effect.

docs/reference/command-line/autocompletion/fish.md

@@ -0,0 +1,17 @@
+---
+title: fish
+eleventyNavigation:
+  key: fish
+  parent: ➡️  Autocompletion
+---
+borgmatic includes an autocompletion script for [fish
+shell](https://fishshell.com/) to support tab-completing command-line actions
+and flags. Depending on how you installed borgmatic, this may be enabled by
+default.
+
+If completions aren't enabled, install the completions file globally:
+
+```shell
+borgmatic --fish-completion | sudo tee /usr/share/fish/vendor_completions.d/borgmatic.fish
+source /usr/share/fish/vendor_completions.d/borgmatic.fish
+```

docs/reference/command-line/autocompletion/index.md

@@ -0,0 +1,13 @@
+---
+title: ➡️  Autocompletion
+eleventyNavigation:
+  key: ➡️  Autocompletion
+  parent: 💻 Command-line
+---
+
+borgmatic includes autocompletion scripts for various command-line shells to
+support tab-completing command-line actions and flags. Depending on how you
+installed borgmatic, this may be enabled by default. Here are the supported
+shells and how to configure borgmatic autocompletion for each:
+
+{{ collections.all | eleventyNavigation: "➡️  Autocompletion" | eleventyNavigationToHtml | replace: 'href="/reference/', 'href="/borgmatic/reference/' }}

docs/reference/command-line/index.md

@@ -1,5 +1,5 @@
 ---
-title: Command-line
+title: 💻 Command-line
 eleventyNavigation:
   key: 💻 Command-line
   parent: Reference guides

docs/reference/command-line/logging.md

@@ -1,5 +1,5 @@
 ---
-title: Logging
+title: 🪵 Logging
 eleventyNavigation:
   key: 🪵 Logging
   parent: 💻 Command-line

docs/reference/command-line/overrides.md

@@ -1,5 +1,5 @@
 ---
-title: Overrides
+title: 🔄 Overrides
 eleventyNavigation:
   key: 🔄 Overrides
   parent: 💻 Command-line

docs/reference/configuration/archive-name-format.md

@@ -1,5 +1,5 @@
 ---
-title: Archive name format
+title: 📛 Archive name format
 eleventyNavigation:
   key: 📛 Archive name format
   parent: ⚙️  Configuration

docs/reference/configuration/command-hooks.md

@@ -1,5 +1,5 @@
 ---
-title: Command hooks
+title: 🪝 Command hooks
 eleventyNavigation:
   key: 🪝 Command hooks
   parent: ⚙️  Configuration

docs/reference/configuration/consistency-checks.md

@@ -1,5 +1,5 @@
 ---
-title: Consistency checks
+title: ✅ Consistency checks
 eleventyNavigation:
   key: ✅ Consistency checks
   parent: ⚙️  Configuration

docs/reference/configuration/constants.md

@@ -1,5 +1,5 @@
 ---
-title: Constants
+title: 🟰 Constants
 eleventyNavigation:
   key: 🟰 Constants
   parent: ⚙️  Configuration

docs/reference/configuration/credentials/container.md

@@ -5,11 +5,14 @@ eleventyNavigation:
   parent: 🔒 Credentials
 ---
 <span class="minilink minilink-addedin">New in version 1.9.11</span> When
-running inside a container, borgmatic can read [Docker
-secrets](https://docs.docker.com/compose/how-tos/use-secrets/) and [Podman
+running inside a container, borgmatic can read [<span
+data-pagefind-weight="0.5">Docker</span>
+secrets](https://docs.docker.com/compose/how-tos/use-secrets/) and [<span
+data-pagefind-weight="0.5">Podman</span>
 secrets](https://www.redhat.com/en/blog/new-podman-secrets-command). Creating
 those secrets and passing them into your borgmatic container is outside the
-scope of this documentation, but here's a simple example of that with [Docker
+scope of this documentation, but here's a simple example of that with [<span
+data-pagefind-weight="0.5">Docker</span>
 Compose](https://docs.docker.com/compose/):
 
 ```yaml
@@ -25,9 +28,9 @@ secrets:
 ```
 
 This assumes there's a file on the host at `/etc/borgmatic/passphrase.txt`
-containing your passphrase. Docker or Podman mounts the contents of that file
-into a secret named `borgmatic_passphrase` in the borgmatic container at
-`/run/secrets/`.
+containing your passphrase. <span data-pagefind-weight="0.5">Docker or
+Podman</span> mounts the contents of that file into a secret named
+`borgmatic_passphrase` in the borgmatic container at `/run/secrets/`.
 
 Once your container secret is in place, you can consume it within your borgmatic
 configuration file:

docs/reference/configuration/credentials/index.md

@@ -1,5 +1,5 @@
 ---
-title: Credentials
+title: 🔒 Credentials
 eleventyNavigation:
   key: 🔒 Credentials
   parent: ⚙️  Configuration

docs/reference/configuration/credentials/keepassxc.md

@@ -42,6 +42,9 @@ postgresql_databases:
       password: "{credential keepassxc /etc/keys.kdbx database}"
 ```
 
+
+### Custom command
+
 You can also optionally override the `keepassxc-cli` command that borgmatic calls to load
 passwords:
 
@@ -49,3 +52,49 @@ passwords:
 keepassxc:
     keepassxc_cli_command: /usr/local/bin/keepassxc-cli
 ```
+
+Another example:
+
+```yaml
+keepassxc:
+    keepassxc_cli_command: docker exec keepassxc keepassxc-cli
+```
+
+### Key file
+
+<span class="minilink minilink-addedin">New in version 2.0.0</span>KeePassXC
+supports unlocking a database with a separate [key
+file](https://keepassxc.org/docs/#faq-keyfile-howto) instead of or in addition
+to a password. To configure borgmatic for that, use the `key_file` option:
+
+```yaml
+keepassxc:
+    key_file: /path/to/keyfile
+```
+
+<span class="minilink minilink-addedin">New in version 2.0.12</span>By default,
+keepassxc-cli prompts the user for the password to unlock a database. But if you
+only want to provide a key file to unlock your database and not a password, for
+instance to support unattended backups, use the `ask_for_password` option:
+
+```yaml
+keepassxc:
+    ask_for_password: false
+    key_file: /path/to/keyfile
+```
+
+### YubiKey
+
+<span class="minilink minilink-addedin">New in version 2.0.0</span>KeePassXC
+also supports unlocking a database with the help of [a
+YubiKey](https://keepassxc.org/docs/#faq-yubikey-2fa). To configure borgmatic
+for that, use the `yubikey` option:
+
+```yaml
+keepassxc:
+    yubikey: 1:7370001
+```
+
+The value here is the YubiKey slot number (e.g., `1` or `2`) and optional serial
+number (e.g., `7370001`) used to access the KeePassXC database. Join the two
+values with a colon, but omit the colon if you're leaving out the serial number.

docs/reference/configuration/data-sources/btrfs.md

@@ -60,7 +60,7 @@ Additionally, borgmatic rewrites the snapshot file paths so that they appear at
 their original subvolume locations in a Borg archive. For instance, if your
 subvolume path is `/var/subvolume`, then the snapshotted files will appear in an
 archive at `/var/subvolume` as well—even if borgmatic has to mount the snapshot
-somewhere in `/var/subvolume/.borgmatic-snapshot-1234/` to perform the backup.
+somewhere in `/var/subvolume/.borgmatic-snapshot/` to perform the backup.
 
 <span class="minilink minilink-addedin">With Borg version 1.2 and
 earlier</span>Snapshotted files are instead stored at a path dependent on the
@@ -70,18 +70,46 @@ temporary snapshot directory in use at the time the archive was created, as Borg
 
 ## Performance
 
-<span class="minilink minilink-addedin">With Borg version 1.x</span> Because of
-the way that Btrfs snapshot paths change from one borgmatic invocation to the
-next, the [Borg file
+<span class="minilink minilink-addedin">New in borgmatic version 2.0.13, with
+Borg version 1.x</span> borgmatic uses consistent snapshot paths between
+invocations, so Btrfs snapshots are cached correctly. No configuration is
+necessary.
+
+<span class="minilink minilink-addedin">Prior to borgmatic version 2.0.13, with
+Borg version 1.x</span> Because of the way that Btrfs snapshot paths change from
+one borgmatic invocation to the next, the [Borg file
 cache](https://borgbackup.readthedocs.io/en/stable/internals/data-structures.html#cache)
-will never get cache hits on snapshotted files. This makes backing up Btrfs
+never gets cache hits on snapshotted files. This makes backing up Btrfs
 snapshots a little slower than non-snapshotted files that have consistent paths.
+If this is an issue for you, upgrade to borgmatic to 2.0.13+.
 
 <span class="minilink minilink-addedin">With Borg version 2.x</span> Even
-snapshotted files should get cache hits, because Borg 2.x is smarter about how
-it looks up file paths in its cache—it constructs the cache key with the path
-*as it's seen in the archive* (which is consistent across runs) rather than the
-full absolute source path (which changes).
+snapshotted files get cache hits, because Borg 2.x is smarter about how it looks
+up file paths in its cache—it constructs the cache key with the path *as it's
+seen in the archive* (which is consistent across runs) rather than the full
+absolute source path (which changes).
+
+
+## systemd settings
+
+If you're using [systemd to run
+borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/#systemd), you
+may need to disable particular security settings like `CapabilityBoundingSet` to
+allow the Btrfs feature to work. See the comments in the sample systemd service
+file for details.
+
+
+## non-root user
+
+<span class="minilink minilink-addedin">New in version 2.0.13</span> If you'd
+like borgmatic to snapshot a Btrfs subvolume when running as a non-root user,
+make the following changes to the subvolume in question:
+
+1. `chown` the subvolume to be owned by your non-root user.
+2. Mount the subvolume with the `user_subvol_rm_allowed` mount option.
+
+These changes allow the non-root user to create and delete snapshots of the
+subvolume, which is necessary for the borgmatic Btrfs hook to work.
 
 
 ## Full configuration

docs/reference/configuration/data-sources/index.md

@@ -1,5 +1,5 @@
 ---
-title: Data sources
+title: 🗄️ Data sources
 eleventyNavigation:
   key: 🗄️ Data sources
   parent: ⚙️  Configuration

docs/reference/configuration/data-sources/lvm.md

@@ -114,6 +114,15 @@ the cache key with the path *as it's seen in the archive* (which is consistent
 across runs) rather than the full absolute source path (which can change).
 
 
+## systemd settings
+
+If you're using [systemd to run
+borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/#systemd), you
+may need to disable particular security settings like `ProtectKernelModules`,
+`CapabilityBoundingSet`, and/or `PrivateDevices` to allow the LVM feature to
+work. See the comments in the sample systemd service file for details.
+
+
 ## Full configuration
 
 ```yaml

docs/reference/configuration/data-sources/zfs.md

@@ -91,7 +91,7 @@ cache](https://borgbackup.readthedocs.io/en/stable/internals/data-structures.htm
 may not get cache hits on snapshotted files. This makes backing up ZFS snapshots
 a little slower than non-snapshotted files that have consistent paths. You can
 mitigate this by setting a fixed [runtime
-directory](https://torsion.org/borgmatic/reference/configuration/runtime-directory/))
+directory](https://torsion.org/borgmatic/reference/configuration/runtime-directory/)
 (that's not located in `/tmp`). This allows borgmatic to use a consistent
 snapshot path from one run to the next, thereby resulting in Borg files cache
 hits.
@@ -103,6 +103,15 @@ the cache key with the path *as it's seen in the archive* (which is consistent
 across runs) rather than the full absolute source path (which can change).
 
 
+## systemd settings
+
+If you're using [systemd to run
+borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/#systemd), you
+may need to disable particular security settings like `CapabilityBoundingSet`
+and/or `PrivateDevices` to allow the ZFS feature to work. See the comments in
+the sample systemd service file for details.
+
+
 ## Full configuration
 
 ```yaml

docs/reference/configuration/environment-variables.md

@@ -1,5 +1,5 @@
 ---
-title: Environment variables
+title: 💲 Environment variables
 eleventyNavigation:
   key: 💲 Environment variables
   parent: ⚙️  Configuration

docs/reference/configuration/includes.md

@@ -1,5 +1,5 @@
 ---
-title: Includes
+title: ❗ Includes
 eleventyNavigation:
   key: ❗ Includes
   parent: ⚙️  Configuration

docs/reference/configuration/index.md

@@ -1,14 +1,14 @@
 ---
-title: Configuration
+title: ⚙️  Configuration
 eleventyNavigation:
   key: ⚙️  Configuration
   parent: Reference guides
   order: 0
 ---
-Below is a sample borgmatic configuration file including all available options
-for the [most recent version of
+Below is a sample borgmatic configuration snippet for every available option in
+the [most recent version of
 borgmatic](https://projects.torsion.org/borgmatic-collective/borgmatic/releases).
-This file is also [available for
+A full example configuration file is also [available for
 download](https://torsion.org/borgmatic/reference/config.yaml).
 
 If you're using an older version of borgmatic, some of these options may not
@@ -16,6 +16,11 @@ work, and you should instead [generate a sample configuration file specific to
 your borgmatic
 version](https://torsion.org/borgmatic/how-to/set-up-backups/#configuration).
 
+<span data-pagefind-weight="6.0">
+{% for option_name in option_names %}
+### {{ option_name }} option
 ```yaml
-{% include borgmatic/config.yaml %}
+{% include borgmatic/{{ option_name }}.yaml %}
 ```
+{% endfor %}
+</span>

docs/reference/configuration/monitoring/index.md

@@ -1,5 +1,5 @@
 ---
-title: Monitoring
+title: 🚨 Monitoring
 eleventyNavigation:
   key: 🚨 Monitoring
   parent: ⚙️  Configuration

docs/reference/configuration/monitoring/sentry.md

@@ -26,6 +26,9 @@ sentry:
 The `monitor_slug` value comes from the "Monitor Slug" under "Cron Details" on
 the same Sentry monitor page.
 
+The `environment` value optionally specifies the environment that is used in
+Sentry.
+
 With this configuration, borgmatic pings Sentry whenever borgmatic starts,
 finishes, or fails, but only when any of the `create`, `prune`, `compact`, or
 `check` actions are run. You can optionally override the start/finish/fail
@@ -36,6 +39,7 @@ Sentry on failure:
 sentry:
     data_source_name_url: https://5f80ec@o294220.ingest.us.sentry.io/203069
     monitor_slug: mymonitor
+    environment: myenvironment
     states:
       - fail
 ```

docs/reference/configuration/patterns-and-excludes.md

@@ -0,0 +1,159 @@
+---
+title: ⛔ Patterns and excludes
+eleventyNavigation:
+  key: ⛔ Patterns and excludes
+  parent: ⚙️  Configuration
+---
+
+borgmatic's configuration has multiple options for specifying the source files
+to include in your backups. Which of these options you use depends on how
+complex your file matching needs are.
+
+## Source directories
+
+The `source_directories` option is the simplest way to specify the files and
+directories to include in your backups. Globs (`*`) allow you to match multiple
+paths at once, and tildes (`~`) get expanded to the current user's home
+directory. Here's an example:
+
+```yaml
+source_directories:
+    - /home
+    - /etc
+    - /var/log/syslog*
+    - /home/user/path with spaces
+    - ~/.config
+```
+
+## Excludes
+
+The `exclude_patterns` options lists particular paths to exclude from your
+backups, paths that would otherwise get included by `source_directories`. Globs
+and tildes are also supported here. Use quotes as needed. For example:
+
+```yaml
+exclude_patterns:
+    - '*.pyc'
+    - '/home/*/.cache'
+    - '*/.vim*.tmp'
+    - /etc/ssl
+    - /home/user/path with spaces
+```
+
+See the [Borg patterns
+documentation](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-patterns)
+for more details about the specific "fnmatch"-style syntax used by excludes.
+
+The `exclude_from` option is similar to `exclude_patterns`, but with your patterns
+listed in an external file instead of directly within borgmatic's configuration.
+Here's an example:
+
+```yaml
+exclude_from:
+    - /etc/borgmatic/excludes
+```
+
+Also see the [borgmatic configuration
+reference](https://torsion.org/borgmatic/reference/configuration/) for
+additional exclude-related options.
+
+
+## Patterns
+
+When you have more complex needs for including and excluding files to backup,
+the `patterns` option is available. The definitive documentation on patterns is
+the [Borg patterns
+documentation](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-patterns),
+but here are the basics.
+
+First, unless you have `source_directories`, you need root patterns. In fact,
+root patterns are just another way to specify the same paths as in
+`source_directories`; they tell Borg the starting point for recursing into
+directories to find files to backup. Root patterns are prefixed with "`R `". For
+instance:
+
+```yaml
+patterns:
+  - R /home
+  - R /etc
+```
+
+You can also add excludes and includes to your patterns. Excludes are prefixed
+with "`- `". If you're defining these directly in borgmatic's configuration
+file, use quotes around the pattern. For example:
+
+```yaml
+patterns:
+  - R /home
+  - '- /home/user/.cache'
+  - R /etc
+```
+
+How this works is that when Borg discovers a particular file path as it's
+recursing into root directories, it tries to match that path against any
+excludes and includes in your patterns *in order*, one at a time. If the first
+match is to an exclude, then Borg excludes the file from the backup. But if the
+first match is to an include, Borg includes it—even if there's a subsequent
+exclude.
+
+The respective order of root patterns vs. exclude and include patterns doesn't
+matter to Borg, so organize root patterns how you like.
+
+Here's an example of an include, which is prefixed with "`+ `":
+
+```yaml
+patterns:
+  - R /home
+  - '+ /home/user/.cache/keep-me'
+  - '- /home/user/.cache'
+  - R /etc
+```
+
+This example excludes all of the `.cache` directory—except for the `keep-me`
+subdirectory, which gets included since it's listed first.
+
+There's also a different kind of exclude pattern—a no-recurse exclude. That's
+prefixed with "`! `" and tells Borg to not only exclude any matching paths but
+also to ignore any subdirectories, saving file processing time. Here's an
+example:
+
+```yaml
+patterns:
+  - R /home
+  - '! /home/user/.cache'
+  - R /etc
+```
+
+The `patterns_from` option is similar to `patterns`, but with your patterns
+listed in an external file instead of directly within borgmatic's configuration.
+Here's an example:
+
+```yaml
+patterns_from:
+    - /etc/borgmatic/patterns
+```
+
+## Debugging
+
+Under the hood, borgmatic actually converts `source_directories`,
+`exclude_patterns`, and `exclude_from` values to Borg patterns and merges them
+with any `patterns` and `patterns_from` values you've configured—passing the
+resulting processed patterns to Borg.
+
+To see the combined patterns that borgmatic passes to Borg, run borgmatic with
+[`--verbosity 2`](https://torsion.org/borgmatic/reference/command-line/logging/)
+(and optionally `--dry-run`) and look for "`Writing patterns to ...`" in the
+output. For instance:
+
+```
+repo: Writing patterns to /tmp/borgmatic-xzwb6s07/borgmatic/tmp61shymp0:
+R /tmp/borgmatic-xzwb6s07/./borgmatic/sqlite_databases
++ /tmp/borgmatic-xzwb6s07/./borgmatic/sqlite_databases
+R /home
+R /etc
+! fm:/home/user/.cache
+```
+
+You'll notice that borgmatic prepends your patterns with its own to support
+use cases like streaming database dumps to Borg, creating filesystem snapshots,
+saving bootstrap metadata, and so on.

docs/reference/configuration/repositories.md

@@ -0,0 +1,138 @@
+---
+title: 🗃️ Repositories
+eleventyNavigation:
+  key: 🗃️ Repositories
+  parent: ⚙️  Configuration
+---
+
+Borg repositories are where your backups get stored. You can define them in
+borgmatic's configuration via the `repositories` option, something like:
+
+```yaml
+repositories:
+    - path: /path/to/first.borg
+      label: first
+    - path: /path/to/second.borg
+      label: second
+```
+
+Each repository has a `path` and an optional `label`. The [Borg repository URLs
+documentation](https://borgbackup.readthedocs.io/en/stable/usage/general.html#repository-urls)
+has examples of valid repositories paths, but see below for some
+borgmatic-specific examples.
+
+The `label` shows up in [logged
+messages](https://torsion.org/borgmatic/reference/command-line/logging/) about
+the repository and also serves as a way to refer to the repository via
+the `--repository` flag on the command-line for supported
+[actions](https://torsion.org/borgmatic/reference/command-line/actions/).
+
+When you run borgmatic's [`create`
+action](https://torsion.org/borgmatic/reference/command-line/actions/create/),
+it invokes Borg once for each configured repository in sequence. (So, not in
+parallel.) That means—in each repository—borgmatic creates a single new backup
+archive containing all of your [source
+directories](https://torsion.org/borgmatic/reference/configuration/patterns-and-excludes/).
+
+
+## SSH
+
+Backing up to a remote server via
+[SSH](https://en.wikipedia.org/wiki/Secure_Shell) looks like:
+
+```yaml
+repositories:
+    - path: ssh://user@host:port/./absolute/path/to/repo
+```
+
+Or relative to the remote user's home directory:
+
+```yaml
+repositories:
+    - path: ssh://user@host:port/~/relative/path/to/repo
+```
+
+This assumes that you've already configured SSH access (e.g. public keys, known
+hosts, authorized hosts, etc.) outside of borgmatic and that Borg is installed
+on the server.
+
+<span class="minilink minilink-addedin">With Borg version 2.x</span>The SSH
+syntax is a little different:
+
+```yaml
+repositories:
+    - path: ssh://user@host:port//absolute/path/to/repo
+```
+
+Or relative to the remote user's home directory:
+
+
+```yaml
+repositories:
+    - path: ssh://user@host:port/relative/path/to/repo
+```
+
+Also see the [`ssh_command` configuration
+option](https://torsion.org/borgmatic/reference/configuration/) for overriding
+the path to the SSH binary or passing it custom flags. For example:
+
+```yaml
+ssh_command: ssh -i /path/to/private/key
+```
+
+
+### SFTP
+
+[SFTP](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol) repositories
+work just like SSH repositories, but with `sftp://` substituted for `ssh://`.
+
+
+## Rclone
+
+<span class="minilink minilink-addedin">New in Borg version 2.x</span> If you're
+using Borg 2, you can backup to repositories via [Rclone](https://rclone.org/),
+which supports a large number of [cloud
+providers](https://rclone.org/#providers). This means that Borg, via Rclone,
+backs up directly to a cloud provider without having to create an intermediate
+repository.
+
+The borgmatic configuration for Rclone looks like:
+
+```yaml
+repositories:
+    - path: rclone:remote:path
+```
+
+Note the lack of "`//`" after `rclone:`.
+
+This configuration assumes that you've already [configured a corresponding
+Rclone remote](https://rclone.org/docs/).
+
+
+## S3 / B2
+
+<span class="minilink minilink-addedin">New in Borg version 2.x</span> Borg 2
+supports storing repositories directly on [Amazon
+S3](https://aws.amazon.com/s3/), [Backblaze
+B2](https://www.backblaze.com/cloud-storage), or an S3-alike service, even
+without the use of Rclone or an intermediate repository. The configuration for
+that might look like one of the following:
+
+```yaml
+repositories:
+    - path: s3:access_key_id:access_key_secret@/bucket/path 
+    - path: b2:access_key_id:access_key_secret@schema://hostname:port/bucket/path 
+```
+
+Note the lack of "`//`" after `s3:` or `b2:`.
+
+When selecting your cloud hosting provider, be aware that Amazon in particular
+has [financially
+supported](https://en.wikipedia.org/wiki/White_House_State_Ballroom) the Trump
+regime.
+
+
+## Related documentation
+
+ * [How to make backups redundant](https://torsion.org/borgmatic/how-to/make-backups-redundant/)
+ * [How to provide your passwords](https://torsion.org/borgmatic/how-to/provide-your-passwords/)

docs/reference/configuration/runtime-directory.md

@@ -1,5 +1,5 @@
 ---
-title: Runtime directory
+title: 📁 Runtime directory
 eleventyNavigation:
   key: 📁 Runtime directory
   parent: ⚙️  Configuration
@@ -39,3 +39,28 @@ system temporary directories.
 borgmatic created temporary streaming database dumps within the `~/.borgmatic`
 directory by default. At that time, the path was configurable by the
 `borgmatic_source_directory` configuration option (now deprecated).
+
+
+## systemd-tmpfiles
+
+If borgmatic's runtime directory is in `/tmp`, be aware that some systems may
+automatically delete `/tmp` files on a periodic basis, e.g. via
+[systemd-tmpfiles](https://www.freedesktop.org/software/systemd/man/251/systemd-tmpfiles.html).
+
+One sign that this is happening is borgmatic erroring during cleanup with "No
+such file or directory" on the runtime directory path, indicating that
+borgmatic's runtime directory is getting deleted out from under it.
+
+You can work around this by either excluding borgmatic's runtime directory from
+automatic systemd-tmpfiles management—or you can change borgmatic's runtime
+directory to not be in `/tmp` as described above.
+
+Here's what a systemd-tmpfiles exclude for borgmatic might look like, for
+instance in an `/etc/tmpfiles.d/borgmatic.conf` file:
+
+```
+x /tmp/borgmatic-*
+```
+
+That tells systemd-tmpfiles to ignore borgmatic's runtime directory when
+automatically deleting paths in `/tmp`.

docs/reference/source-code.md

@@ -1,5 +1,5 @@
 ---
-title: Source code
+title: 🐍 Source code
 eleventyNavigation:
   key: 🐍 Source code
   parent: Reference guides

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "borgmatic"
-version = "2.0.10"
+version = "2.0.13"
 authors = [
   { name="Dan Helfman", email="witten@torsion.org" },
 ]

sample/systemd/borgmatic.service

@@ -26,13 +26,16 @@ LockPersonality=true
 # But you can try setting it to "yes" for improved security if you don't use those features.
 MemoryDenyWriteExecute=no
 NoNewPrivileges=yes
-# Filesystem hooks like ZFS may not work unless PrivateDevices is disabled.
+# Filesystem hooks like ZFS and LVM may not work unless PrivateDevices is disabled.
 PrivateDevices=yes
 PrivateTmp=yes
 ProtectClock=yes
 ProtectControlGroups=yes
 ProtectHostname=yes
 ProtectKernelLogs=yes
+# You may need to disable this if you're using the LVM hook. But an alternative
+# might be to leave this enabled and preload the necessary kernel modules like
+# "dm_snapshot" at boot, so this service doesn't need to load them on demand.
 ProtectKernelModules=yes
 ProtectKernelTunables=yes
 RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK
@@ -54,7 +57,9 @@ ProtectSystem=full
 # BindReadOnlyPaths=-/root/.ssh
 
 # May interfere with running external programs within borgmatic hooks. This
-# includes, for instance, programs to snapshot filesystems (e.g. ZFS).
+# includes, for instance, programs to snapshot filesystems (e.g. ZFS, LVM, and
+# Btrfs). But rather than commenting this out entirely, one workaround may be to
+# add "CAP_SYS_ADMIN".
 CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW
 
 # Lower CPU and I/O priority.

sample/systemd/borgmatic.timer

@@ -4,7 +4,7 @@ Description=Run borgmatic backup
 [Timer]
 OnCalendar=daily
 Persistent=true
-RandomizedDelaySec=3h
+RandomizedDelaySec=10m
 
 [Install]
 WantedBy=timers.target

scripts/export-docs-from-image

@@ -2,10 +2,20 @@
 
 set -e
 
-docs_container_id=$(podman create "$IMAGE_NAME")
-podman cp $docs_container_id:/usr/share/nginx/html - > borgmatic-docs-dump.tar
+USER_PODMAN_SOCKET_PATH=/run/user/$UID/podman/podman.sock
+
+if [ -e "$USER_PODMAN_SOCKET_PATH" ]; then
+    export DOCKER_HOST="unix://$USER_PODMAN_SOCKET_PATH"
+    export CONTAINER_SOCKET_PATH="$USER_PODMAN_SOCKET_PATH"
+fi
+
+podman build --tag borgmatic-release-docs --file docs/Dockerfile .
+docs_container_id=$(podman create borgmatic-release-docs)
+podman cp "$docs_container_id":/usr/share/nginx/html - > borgmatic-docs-dump.tar
 tar xf borgmatic-docs-dump.tar
 rm borgmatic-docs-dump.tar
 mv html borgmatic-docs
-tar cfz borgmatic-docs.tar.gz borgmatic-docs
-podman rm --volumes $docs_container_id
+rm -f dist/borgmatic-docs.tar.gz
+tar cfz dist/borgmatic-docs.tar.gz borgmatic-docs
+rm -fr borgmatic-docs
+podman rm --volumes "$docs_container_id"

scripts/release

@@ -34,18 +34,40 @@ git push github $version
 # Build borgmatic and publish to pypi.
 rm -fr dist
 uv build
-twine upload -r pypi --username __token__ dist/borgmatic-*.tar.gz
-twine upload -r pypi --username __token__ dist/borgmatic-*-py3-none-any.whl
+tarball_path="dist/borgmatic-$version.tar.gz"
+wheel_path=$(ls dist/borgmatic-*-py3-none-any.whl)
+twine upload -r pypi --username __token__ "$tarball_path"
+twine upload -r pypi --username __token__ "$wheel_path"
+
+# Build docs and extract HTML.
+scripts/export-docs-from-image
+docs_path=dist/borgmatic-docs.tar.gz
 
 # Set release changelogs on projects.torsion.org and GitHub.
 release_changelog="$(cat NEWS | sed '/^$/q' | grep -v '^\S')"
 escaped_release_changelog="$(echo "$release_changelog" | sed -z 's/\n/\\n/g' | sed -z 's/\"/\\"/g')"
-curl --silent --request POST \
+release_id=$(curl --silent --request POST \
     "https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/releases" \
     --header "Authorization: token $projects_token" \
     --header "Accept: application/json" \
     --header "Content-Type: application/json" \
-    --data "{\"body\": \"$escaped_release_changelog\", \"draft\": false, \"name\": \"borgmatic $version\", \"prerelease\": false, \"tag_name\": \"$version\"}"
+    --data "{\"body\": \"$escaped_release_changelog\", \"draft\": false, \"name\": \"borgmatic $version\", \"prerelease\": false, \"tag_name\": \"$version\"}" \
+    | jq ".id")
+curl --silent --request POST \
+    "https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/releases/$release_id/assets?name=$(basename $wheel_path)" \
+    --header "Authorization: token $projects_token" \
+    --header "Accept: application/json" \
+    --form attachment=@"$wheel_path"
+curl --silent --request POST \
+    "https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/releases/$release_id/assets?name=$(basename $tarball_path)" \
+    --header "Authorization: token $projects_token" \
+    --header "Accept: application/json" \
+    --form attachment=@"$tarball_path"
+curl --silent --request POST \
+    "https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/releases/$release_id/assets?name=$(basename $docs_path)" \
+    --header "Authorization: token $projects_token" \
+    --header "Accept: application/json" \
+    --form attachment=@"$docs_path"
 
 github-release create --token="$github_token" --owner=witten --repo=borgmatic --tag="$version" --target_commit="main" \
     --name="borgmatic $version" --body="$release_changelog"