Fix schema example for "archive_username" (#1317).

Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1312

.forgejo/workflows/build.yaml

@@ -1,5 +1,5 @@
 name: build
-run-name: ${{ gitea.actor }} is building
+run-name: ${{ forgejo.actor }} is building
 on:
   push:
     branches: [main]
@@ -17,7 +17,7 @@ jobs:
   docs:
     needs: [test]
     runs-on: host
-    if: gitea.event_name == 'push'
+    if: forgejo.event_name == 'push'
     env:
       IMAGE_NAME: projects.torsion.org/borgmatic-collective/borgmatic:docs
 

.github/pull_request_template.md

@@ -1,10 +0,0 @@
-## Hold up, GitHub users
-
-Thanks for your contribution! 
-
-Unfortunately, we don't use GitHub pull requests to manage code contributions to this repository (and GitHub doesn't have any way to disable pull requests entirely). Instead, please see:
-
-https://torsion.org/borgmatic/#contributing
-
-... which provides full instructions on how to submit pull requests. You can even use your GitHub account to login.
-

AGENTS.md

@@ -0,0 +1,28 @@
+# AGENTS.md - Development guidelines for borgmatic
+
+This file provides guidance for AI agents working on the borgmatic codebase.
+
+## Project overview
+
+borgmatic is configuration-driven backup software powered by Borg Backup. It's a
+Python project using setuptools.
+
+Please do not use AI agents to modify this codebase. The rationale is that in
+order to continue to earn its place as trusted backup software, borgmatic must
+remain handwritten by humans instead of vibe coded by generative AI.
+
+Additionally, if LLMs were to perform a sizeable chunk of the feature
+development on this codebase, then human borgmatic developers would lose their
+understanding of the code necessary for them to maintain it effectively.
+
+Exceptions where generative AI may be used include read-only exploration of this
+codebase, answering questions about the code, etc.
+
+## Architecture notes
+
+- **main entry point**: `borgmatic.commands.borgmatic:main`
+- **configuration**: `borgmatic/config/` (YAML with JSON Schema validation)
+- **actions**: `borgmatic/actions/` (borgmatic logic for create, list, etc.)
+- **Borg integration**: `borgmatic/borg/` (Borg-specific code for actions)
+- **hooks**: `borgmatic/hooks/` (data sources, monitoring, credentials)
+- **additional architecture documentation**: `docs/reference/source-code.md`

MANIFEST.in

@@ -1,4 +1,5 @@
 # This file only applies to the source dist tarball, not the built wheel.
+include NEWS
 include borgmatic/config/schema.yaml
 graft docs
 graft sample

NEWS

@@ -1,3 +1,143 @@
+2.1.7.dev0
+ * #1309: Add support for the "--quick-stats" flag and the "quick_statistics" option to the "prune"
+   action. Borg >= 1.4.5 and < 2 only.
+ * #1317: Add an "archive_hostname" option and a corresponding "--archive-hostname" flag for
+   overriding the hostname used for the "{hostname}" placeholder in the "archive_name_format"
+   option. Also add an "archive_username" option and corresponding "--archive-username" flag to
+   override the "{user}" plaecholder. Both options/flags are Borg 1.4.5+ only.
+ * #1319: Fix the ZFS hook's overzealous unmounting of snapshot paths when a source dataset is at
+   "/".
+ * #1322: Fix for the "restore" action sometimes failing to find a database dump that was dumped
+   with a default port.
+ * #1324: For the MariaDB and MySQL hooks, add "events", "routines", and "tablespaces" options for
+   disabling dumping of scheduled events, stored routines, and tablespaces, respectively.
+ * #1327: Fix an error from the "diff" action when exclude options are configured.
+ * #1331: Fix the "repo-create" action to more surgically suppress Borg "Repository does not exist"
+   logs and avoid inadvertently suppressing other error logs.
+ * #1333: Fix the "--progress" flag on the "compact" action to actually update the progress of
+   segment compaction.
+ * #1334: Add the "CAP_FOWNER" capability to "CapabilityBoundingSet" in the sample systemd service,
+   so that Borg can open source files without changing file access times.
+ * Fix a bug in which the "compact" action does not pass a compact threshold of zero to Borg.
+
+2.1.6
+ * #1256: Fix a race condition in which borgmatic sometimes swallows Borg error output without
+   logging it.
+ * #1300: Fix the "source_directories_must_exist" option to support source directories relative to a
+   "working_directory".
+ * #1301: Expand the "patterns_from" and "exclude_from" options to support paths containing tildes
+   and globs.
+ * #1303: For the MariaDB and MySQL database hooks, include events, routines, and tablespaces when
+   dumping a database.
+ * #1303: For the MariaDB hook, include only a subset of system data when dumping the "mysql" system
+   database (or "all" databases), so the dump is actually restorable. See the documentation for more
+   information: https://torsion.org/borgmatic/reference/configuration/data-sources/mariadb/
+ * #1308: Update the Apprise monitoring hook's "url" option to support loading credentials with the
+   "{credential ...}" syntax. See the documentation for more information:
+   https://torsion.org/borgmatic/reference/configuration/credentials/
+ * #1309: Add a "--quick-stats" flag and corresponding "quick_statistics" option for showing only
+   abbreviated statistics for the "create" action. Borg 1.4.5+ only.
+ * #1314: Add a minimal stand-alone borgmatic binary in addition to the standard one. The minimal
+   binary omits support for the Apprise monitoring hook.
+ * Add an experimental "browse" action providing a console UI for browsing your backups. See the
+   documentation for more information:
+   https://torsion.org/borgmatic/how-to/inspect-your-backups/#browsing-backups
+ * Update the KeePassXC credential hook to support KeePassXC's secret service integration. See the
+   documentation for more information:
+   https://torsion.org/borgmatic/reference/configuration/credentials/keepassxc/
+ * Add the NEWS changelog file to release tarball (#1298).
+ * Enable reply by email on projects.torsion.org, so replies to notification emails get posted as
+   comments on tickets.
+ * For the MariaDB and MySQL database hooks, escape quotes in passwords when the
+   "password_transport" option is "pipe".
+ * Add a development script for upgrading pinned dependencies.
+ * Fix the PostgreSQL database hook to properly parse "*options" values containing quoted spaces.
+ * Update the documentation to use Pagefind's component-based search UI.
+
+2.1.5
+ * #1229: Document the permissions needed for the PostgreSQL database hook:
+   https://torsion.org/borgmatic/reference/configuration/data-sources/postgresql/
+ * #1289: Add mutual TLS support for the Loki monitoring hook. See the documentation for more
+   information: https://torsion.org/borgmatic/reference/configuration/monitoring/loki/
+ * #1292: Fix a "source directories do not exist" regression when configuration paths are relative
+   symlinks and the bootstrap data source hook is enabled.
+ * #1294: Fix a regression in which SSH warnings from remote repositories broke the "spot" check
+   and other actions as well.
+ * #1295: Fix the ZFS hook to properly unmount snapshots for empty datasets.
+
+2.1.4
+ * #1266: Add a stand-alone borgmatic Linux binary to the release downloads to serve as another way
+   to install borgmatic. Consider this binary a beta feature.
+ * #1286: Fix a regression in which running borgmatic with no arguments and no configuration files
+   doesn't error as expected.
+ * #1257: Fix for the Loki monitoring hook not respecting the monitoring verbosity.
+ * #1264: Improve performance of the "info" and "repo-list" actions by eliminating a second "borg
+   info" call that supports a "no matching archives" warning. The warning still occurs; it's just
+   done now without the extra "borg info" call.
+ * #1268: Fix the "spot" check, "extract" check, and all uses of the archive name "latest" to
+   respect the "match_archives" and "archives_name_format" options. This means that borgmatic now
+   uses the "latest" archive that also matches those options instead of the latest archive overall.
+ * When Borg exits with a warning exit code, show a description of it, so you don't have to look up
+   the code.
+ * Split out borgmatic installation documentation to its own page, so it's easier to find.
+ * Switch the default borgmatic installation method from pipx to uv, as uv is faster and used for
+   borgmatic development. If you'd like to switch, see the documentation for more information:
+   https://torsion.org/borgmatic/how-to/upgrade/
+ * Move the project tracker from Gitea to Forgejo.
+ * Fix a regression in which borgmatic didn't show an error message when run with no configuration.
+ * Fix a traceback in the "spot" check with Borg 2.
+
+2.1.3
+ * #1175: Add a "files_changed" option for customizing Borg's file modification detection.
+ * #1175: Add a "msgpack_version_check" option to prevent Borg from validating msgpack's version.
+ * #1218: Add a "config show" action to display computed borgmatic configuration as YAML or JSON,
+   handy for fetching borgmatic configuration from external scripts. See the documentation for more
+   information:
+   https://torsion.org/borgmatic/reference/command-line/actions/config-show/
+ * #1228: Adjust the "spot" check so error output includes more information about what failed.
+ * #1236: Fix the "spot" check to skip hard links, as Borg doesn't produces hashes for them.
+ * #1243: Add a "diff" action for viewing the difference between the contents of two archives.
+ * #1248: Go back to treating Borg "file not found" warnings (exit code 107) as
+   warnings instead of errors. Otherwise, borgmatic can error on files that a user intentionally
+   deletes while a backup is running. You can still override this behavior with the
+   "borg_exit_codes" option. See the documentation for more information:
+   https://torsion.org/borgmatic/how-to/customize-warnings-and-errors/
+ * #1248: Un-deprecate the "source_directories_must_exist" option and default it to true, to
+   compensate for Borg "file not found" warnings no longer being treated as errors.
+ * #1269: Fix the ZFS hook to support datasets with a "canmount" property of "noauto".
+ * #1270: Follow symlinks when backing up borgmatic configuration files to support the "bootstrap"
+   action.
+ * #1274: Add an optional override for the documentation development listen port and use Podman
+   Compose if present.
+ * #1281: Fix a unicode error when backing up a non-UTF-8 source filename with a
+   corresponding system locale.
+ * Add a policy about the use of generative AI in the borgmatic codebase:
+   https://torsion.org/borgmatic/how-to/develop-on-borgmatic/#use-of-generative-ai
+
+2.1.2
+ * #1231: If a source file is deleted during a "spot" check, consider the file as non-matching
+   and move on instead of immediately failing the entire check.
+ * #1250: Fix a regression in which the "--stats" flag hides statistics at default verbosity.
+ * #1251: Fix a regression in the ntfy monitoring hook in which borgmatic sends tags incorrectly,
+   resulting in "400 Bad Request" from ntfy.
+ * #1258: Fix a "codec can't decode byte" error when running commands that output multi-byte unicode
+   characters.
+ * #1260: Fix for SSH warnings from Borg showing up as JSON logs even without the "--log-json" flag.
+ * #1252: Work around Borg returning a warning exit code when a repository/archive check fails. Now,
+   borgmatic interprets such failures as errors.
+ * Deduplicate overlapping source directories and patterns so they don't throw off "spot" check file
+   counts and cause spurious check failures.
+
+2.1.1
+ * #1241: For the "recreate" action, actually pass the "--dry-run" flag through to Borg instead of
+   just skipping the Borg call.
+ * #1242: Fix a regression in which the "spot" check hung while collecting archive contents.
+ * #1244: When the "unsafe_skip_path_validation_before_create" option is enabled, don't log a
+   warning about it.
+ * #1245: Fix a regression in which the KeePassXC credential hook password prompt was invisible.
+ * #1246: Fix a regression in which the ntfy monitoring hook failed to send a ping when the
+   "priority" option was set.
+
 2.1.0
  * TL;DR: Many logging, memory, and performance improvements. Mind those breaking changes!
  * #485: When running commands (database clients, command hooks, etc.), elevate stderr output to

README.md

@@ -94,10 +94,11 @@ borgmatic is powered by [Borg Backup](https://www.borgbackup.org/).
 
 ## Getting started
 
-Your first step is to [install and configure
-borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/).
+Your first steps are to
+[install](https://torsion.org/borgmatic/how-to/install-borgmatic/) and
+[configure borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/).
 
-For additional documentation, check out the links above (left panel on wide screens)
+For additional documentation, check out the links on the top/left
 for <a href="https://torsion.org/borgmatic/#documentation">borgmatic how-to and
 reference guides</a>.
 
@@ -105,16 +106,16 @@ reference guides</a>.
 ## Hosting providers
 
 Need somewhere to store your encrypted off-site backups? The following hosting
-providers include specific support for Borg/borgmatic—and fund borgmatic
-development and hosting when you use these referral links to sign up:
+provider includes specific support for Borg/borgmatic—and funds borgmatic
+development and hosting when you use this referral links to sign up:
 
 <ul>
  <li class="referral"><a href="https://www.borgbase.com/?utm_source=borgmatic">BorgBase</a>: Borg hosting service with support for monitoring, 2FA, and append-only repos</li>
- <li class="referral"><a href="https://hetzner.cloud/?ref=v9dOJ98Ic9I8">Hetzner</a>: A "storage box" that includes support for Borg</li>
 </ul>
 
-Additionally, rsync.net has a compatible storage offering, but does not fund
-borgmatic development or hosting.
+Additionally, Hetzner and rsync\.net have compatible storage offerings, but do
+not fund borgmatic development or hosting.
+
 
 ## Support and contributing
 
@@ -183,4 +184,4 @@ Thanks to all borgmatic contributors! There are multiple ways to contribute to
 this project, so the following includes those who have fixed bugs, contributed
 features, *or* filed tickets.
 
-{% include borgmatic/contributors.html %}
+{% include borgmatic/contributors.html %}

binary_requirements.in

@@ -0,0 +1,19 @@
+.
+apprise
+attrs
+certifi
+charset-normalizer
+click
+idna
+jsonschema
+jsonschema-specifications
+markdown
+oauthlib
+packaging
+pyyaml
+referencing
+requests
+requests-oauthlib
+rpds-py
+ruamel-yaml
+urllib3

binary_requirements.txt

@@ -0,0 +1,21 @@
+# This file was autogenerated by uv via the following command:
+#    uv pip compile --annotation-style line binary_requirements.in -o binary_requirements.txt
+apprise==1.10.0           # via -r binary_requirements.in
+attrs==26.1.0             # via jsonschema, referencing, -r binary_requirements.in
+.                         # via -r binary_requirements.in
+certifi==2026.5.20        # via apprise, requests, -r binary_requirements.in
+charset-normalizer==3.4.7  # via requests, -r binary_requirements.in
+click==8.4.1              # via apprise, -r binary_requirements.in
+idna==3.16                # via requests, -r binary_requirements.in
+jsonschema==4.26.0        # via borgmatic, -r binary_requirements.in
+jsonschema-specifications==2025.9.1  # via jsonschema, -r binary_requirements.in
+markdown==3.10.2          # via apprise, -r binary_requirements.in
+oauthlib==3.3.1           # via requests-oauthlib, -r binary_requirements.in
+packaging==26.2           # via borgmatic, -r binary_requirements.in
+pyyaml==6.0.3             # via apprise, -r binary_requirements.in
+referencing==0.37.0       # via jsonschema, jsonschema-specifications, -r binary_requirements.in
+requests==2.34.2          # via apprise, borgmatic, requests-oauthlib, -r binary_requirements.in
+requests-oauthlib==2.0.0  # via apprise, -r binary_requirements.in
+rpds-py==0.30.0           # via jsonschema, referencing, -r binary_requirements.in
+ruamel-yaml==0.19.1       # via borgmatic, -r binary_requirements.in
+urllib3==2.7.0            # via requests, -r binary_requirements.in

borgmatic/actions/browse/app.py

@@ -0,0 +1,108 @@
+import signal
+
+import textual.app
+import textual.binding
+import textual.widgets
+
+import borgmatic.actions.browse.carousel
+import borgmatic.actions.browse.configuration_files_list
+import borgmatic.actions.browse.logs
+import borgmatic.actions.browse.repositories_list
+
+
+class Browse_app(textual.app.App):
+    '''
+    The main app / entry point for the browse action UI.
+    '''
+
+    BINDINGS = (
+        textual.binding.Binding(key='q', action='quit', description='quit'),
+        textual.binding.Binding(key='v', action='toggle_logs', description='view logs'),
+        textual.binding.Binding(
+            key='c', action='command_palette', description='commands', show=False
+        ),
+    )
+    COMMAND_PALETTE_BINDING = 'c'
+    CSS = '''
+        .panel {
+            border: round $primary;
+            border-title-color: $text-primary;
+            width: 100%;
+            height: 100%;
+        }
+
+        #logs {
+            width: 100%;
+            height: 50%;
+            display: none;
+        }
+    '''
+
+    def __init__(self, configs):
+        self.configs = configs
+
+        super().__init__()
+
+    def get_system_commands(self, screen):  # pragma: no cover
+        '''
+        Remove the screenshot system command because it produces broken screenshots. (Emoji
+        weirdness, etc.) Also remove minimize and maximize because they don't accomplish much with
+        out particular layout.
+        '''
+        yield from (
+            command
+            for command in super().get_system_commands(screen)
+            if command.title not in {'Screenshot', 'Minimize', 'Maximize'}
+        )
+
+    def compose(self):
+        '''
+        Compose a UI consisting of:
+
+          * a header with the application name
+          * a carousel container that contains the main UI panels
+          * a logs panel where Python logs show up (panel hidden by default)
+          * a footer with available keys listed
+        '''
+        yield textual.widgets.Header()
+        yield borgmatic.actions.browse.carousel.Carousel(
+            [
+                borgmatic.actions.browse.configuration_files_list.Configuration_files_list(
+                    self.configs
+                )
+            ]
+            if len(self.configs) > 1
+            else [
+                borgmatic.actions.browse.repositories_list.Repositories_list(
+                    next(iter(self.configs.values()))
+                )
+            ]
+        )
+
+        logs_panel = borgmatic.actions.browse.logs.Logs()
+        yield logs_panel
+        yield textual.widgets.Footer()
+
+        borgmatic.actions.browse.logs.log_to_widget(logs_panel)
+
+    def on_mount(self):
+        '''
+        Set the application title, which ends up in the header.
+        '''
+        self.title = 'borgmatic browse'
+
+    def action_toggle_logs(self):
+        '''
+        Toggle the show/hide status of the logs panel.
+        '''
+        logs_panel = self.query_one('#logs')
+        logs_panel.styles.display = 'none' if logs_panel.styles.display == 'block' else 'block'
+
+    def exit(self):  # pragma: no cover
+        '''
+        Exit the application. But first raise a SIGTERM (handled in borgmatic/signals.py) to
+        encourage a fast exit by killing any ongoing Borg subprocesses.
+        '''
+        signal.raise_signal(signal.SIGTERM)
+
+        super().exit()

borgmatic/actions/browse/archive.py

@@ -0,0 +1,143 @@
+import argparse
+import collections
+import enum
+import json
+import logging
+
+import binaryornot.helpers
+
+import borgmatic.borg.extract
+import borgmatic.borg.list
+import borgmatic.borg.repo_list
+import borgmatic.borg.version
+
+logger = logging.getLogger(__name__)
+
+
+class Path_type(enum.Enum):
+    DIRECTORY = 'd'
+    LINK = 'l'
+    PIPE = 'p'
+    FILE = '-'
+
+
+# A data structure capturing a path stored in a Borg archive.
+Archive_path = collections.namedtuple(
+    'Archive_path',
+    ('path_type', 'file_path', 'link_target'),
+)
+
+
+def get_repository_archives(config, repository):
+    '''
+    Given a configuration dict and a repository dict, return a list of the repository's archives,
+    one dict per archive.
+    '''
+    with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
+        logger.info('Listing repository')
+        repo_list_arguments = argparse.Namespace(
+            repository=repository['path'],
+            short=None,
+            format=None,
+            json=True,
+            prefix=None,
+            match_archives=None,
+            sort_by=None,
+            first=None,
+            last=None,
+        )
+        global_arguments = argparse.Namespace()
+        local_path = config.get('local_path', 'borg')
+        remote_path = config.get('remote_path')
+        local_borg_version = borgmatic.borg.version.local_borg_version(config, local_path)
+
+        return json.loads(
+            borgmatic.borg.repo_list.list_repository(
+                repository['path'],
+                config,
+                local_borg_version,
+                repo_list_arguments,
+                global_arguments,
+                local_path,
+                remote_path,
+            )
+        )
+
+
+def get_archive_paths(config, repository, archive_name):
+    '''
+    Given a configuration dict, a repository dict, and an archive name in that repository, get a
+    generator of files, directories, symlinks, etc. found in the archive, each as an Archive_path
+    instance.
+    '''
+    with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
+        logger.info(f'Listing archive {archive_name}')
+
+        global_arguments = argparse.Namespace()
+        local_path = config.get('local_path', 'borg')
+        remote_path = config.get('remote_path')
+        local_borg_version = borgmatic.borg.version.local_borg_version(config, local_path)
+
+        return (
+            Archive_path(
+                path_data['type'],
+                path_data['path'],
+                path_data.get('linktarget'),
+            )
+            for path_data in borgmatic.borg.list.capture_archive_listing(
+                repository['path'],
+                archive_name,
+                config,
+                local_borg_version,
+                global_arguments,
+                local_path=local_path,
+                remote_path=remote_path,
+            )
+        )
+
+
+READLINES_HINT_BYTES = 100000
+TRUNCATION_MESSAGE = '[... truncated for display ...]'
+
+
+def get_archive_file_content(config, repository, archive_name, file_path):
+    '''
+    Given a configuration dict, a repository dict, an archive name in that repository, and a file
+    path in that archive, return the file's contents or None if the file can't be loaded, e.g.
+    because it's binary or can't be decoded.
+
+    If the file is too large, then truncate the returned content.
+    '''
+    with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
+        logger.info(f'Getting archive content of file {file_path}')
+        local_path = config.get('local_path', 'borg')
+        remote_path = config.get('remote_path')
+
+        lines = borgmatic.borg.extract.extract_archive(
+            dry_run=False,
+            repository=repository['path'],
+            archive=archive_name,
+            paths=(file_path,),
+            config=config,
+            local_borg_version=borgmatic.borg.version.local_borg_version(config, local_path),
+            global_arguments=argparse.Namespace(),
+            local_path=local_path,
+            remote_path=remote_path,
+            destination_path=None,
+            strip_components=None,
+            extract_to_stdout=True,
+        ).stdout.readlines(READLINES_HINT_BYTES)
+
+        content = b''.join(lines)
+
+        if binaryornot.helpers.is_binary_string(content):
+            return None
+
+        try:
+            return (
+                content.decode()
+                if len(content) < READLINES_HINT_BYTES
+                else f'{content.decode()}\n{TRUNCATION_MESSAGE}'
+            )
+        except UnicodeDecodeError:
+            return None

borgmatic/actions/browse/archives_list.py

@@ -0,0 +1,89 @@
+import textual.widgets
+
+import borgmatic.actions.browse.bindings
+import borgmatic.actions.browse.loading
+import borgmatic.actions.browse.workers
+
+
+class Archives_list(textual.widgets.OptionList):
+    '''
+    A widget for selecting a single Borg archive from among the archives in a repository. The item
+    selection event is handled in a Carousel instance, the parent widget of an Archives_list.
+    '''
+
+    BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
+
+    def __init__(self, config, repository):
+        '''
+        Given a configuration dict and a repository dict, prepare to load the archives from the
+        repository for eventual display in this widget. Actual loading kicks off in on_mount()
+        below.
+        '''
+        self.config = config
+        self.repository = repository
+
+        super().__init__(classes='panel')
+        self.border_title = '📚 archives'
+        self.highlighted_option_changed = False
+        self.archive_loaded = borgmatic.actions.browse.workers.Archive_loaded(
+            self, 'archive loaded'
+        )
+
+        self.loading_timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
+
+    def on_mount(self):
+        '''
+        When this widget gets mounted in the DOM, subscribe to archive loaded events so that we can
+        find out about archives as they load. Also start loading archives from the repository.
+
+        Loading is started *after* subscribing to the archive loaded signal so that there's not a
+        gap where we might miss out on signal publishes.
+        '''
+        self.archive_loaded.subscribe(self, self.on_archive_loaded)
+
+        borgmatic.actions.browse.workers.add_repository_archives(
+            self.app,
+            archive_loaded=self.archive_loaded,
+            config=self.config,
+            repository=self.repository,
+        )
+
+    def on_archive_loaded(self, archive_name):
+        '''
+        When an archive loads, add it as an option to this archives list. But if we get a
+        signal that all path loading is complete, stop and remove our loading indicator.
+        '''
+        if archive_name is borgmatic.actions.browse.workers.LOADING_DONE:
+            self.loading_timer.stop()
+            self.remove_option('loading-indicator')
+            return
+
+        label_pieces = (
+            (archive_name, '[dim](latest)[/dim]') if len(self.options) == 1 else (archive_name,)
+        )
+        highlighted_option = self.highlighted_option
+
+        loading_indicator = self.get_option('loading-indicator')
+        self.remove_option('loading-indicator')
+        self.add_options(
+            (
+                textual.widgets.option_list.Option(' '.join(label_pieces), id=archive_name),
+                loading_indicator,
+            ),
+        )
+
+        # Retain the highlighted option position even as other options load around it.
+        self.highlighted = (
+            self.get_option_index(highlighted_option.id)
+            if highlighted_option and self.highlighted_option_changed
+            else 0
+        )
+
+    def on_option_list_option_highlighted(self, event):
+        '''
+        When the highlighted option changes, record that fact. This flag is consumed in
+        borgmatic.actions.browse.workers.add_repository_archives() in order to retain the
+        highlighted option even as other options load around it.
+        '''
+        if self.highlighted not in {None, 0}:
+            self.highlighted_option_changed = True

borgmatic/actions/browse/bindings.py

@@ -0,0 +1,22 @@
+import textual.binding
+import textual.widgets
+
+OPTION_LIST_BINDINGS = (
+    *textual.widgets.OptionList.BINDINGS,
+    textual.binding.Binding(
+        key='up,k', action='cursor_up', description='scroll up', show=True, priority=True
+    ),
+    textual.binding.Binding(
+        key='down,j', action='cursor_down', description='scroll down', show=True, priority=True
+    ),
+    textual.binding.Binding(
+        key='pageup', action='page_up', description='page up', show=True, priority=True
+    ),
+    textual.binding.Binding(
+        key='pagedown', action='page_down', description='page down', show=True, priority=True
+    ),
+    textual.binding.Binding(
+        key='enter', action='select', description='select', show=True, priority=True
+    ),
+    textual.binding.Binding(key='right,l', action='select', description='select', show=False),
+)

borgmatic/actions/browse/carousel.py

@@ -0,0 +1,153 @@
+import os
+
+import textual.binding
+import textual.containers
+
+import borgmatic.actions.browse.archive
+import borgmatic.actions.browse.archives_list
+import borgmatic.actions.browse.configuration_files_list
+import borgmatic.actions.browse.directory_list
+import borgmatic.actions.browse.file_preview
+import borgmatic.actions.browse.icons
+import borgmatic.actions.browse.repositories_list
+
+
+def make_next_panel(focused_panel, option_id):
+    '''
+    Given a focused panel widget and the selected option ID, return the next panel corresponding to
+    that selection. This is the mechanism by which the user can successively drill down from
+    configuration file to repository to archive to root directory to non-root directory or file.
+
+    If the particular option ID on the focused panel doesn't have a supported next panel, then
+    return None.
+    '''
+    if isinstance(
+        focused_panel, borgmatic.actions.browse.configuration_files_list.Configuration_files_list
+    ):
+        return borgmatic.actions.browse.repositories_list.Repositories_list(
+            config=focused_panel.configs[option_id]
+        )
+
+    if isinstance(focused_panel, borgmatic.actions.browse.repositories_list.Repositories_list):
+        return borgmatic.actions.browse.archives_list.Archives_list(
+            config=focused_panel.config, repository=focused_panel.repositories[option_id]
+        )
+
+    if isinstance(focused_panel, borgmatic.actions.browse.archives_list.Archives_list):
+        return borgmatic.actions.browse.directory_list.Directory_list(
+            config=focused_panel.config, repository=focused_panel.repository, archive_name=option_id
+        )
+
+    if isinstance(focused_panel, borgmatic.actions.browse.directory_list.Directory_list):
+        option = focused_panel.get_option(option_id)
+
+        if option.prompt.startswith(
+            borgmatic.actions.browse.icons.PATH_TYPE_ICONS[
+                borgmatic.actions.browse.archive.Path_type.DIRECTORY.value
+            ]
+        ):
+            return borgmatic.actions.browse.directory_list.Directory_list(
+                focused_panel.config,
+                focused_panel.repository,
+                focused_panel.archive_name,
+                path_loaded=focused_panel.path_loaded,
+                path_components=(*focused_panel.path_components, option_id),
+            )
+
+        if option.prompt.startswith(
+            borgmatic.actions.browse.icons.PATH_TYPE_ICONS[
+                borgmatic.actions.browse.archive.Path_type.FILE.value
+            ]
+        ):
+            return borgmatic.actions.browse.file_preview.File_preview(
+                focused_panel.config,
+                focused_panel.repository,
+                focused_panel.archive_name,
+                file_path=os.path.sep.join((*focused_panel.path_components, option_id)),
+            )
+
+    return None
+
+
+class Carousel(textual.containers.Horizontal):
+    BINDINGS = (
+        textual.binding.Binding(
+            key='left,h', action='previous', description='previous', priority=True
+        ),
+    )
+
+    def __init__(self, panels):
+        self.panels = panels
+        self.focused_panel = panels[0]
+
+        super().__init__()
+
+    def compose(self):
+        '''
+        Compose with each of the contained panels and focus the first one.
+        '''
+        yield from self.panels
+
+        self.focused_panel.focus()
+
+    def action_previous(self):
+        '''
+        Make the previous panel into the focused panel.
+        '''
+        previous_panel_index = self.panels.index(self.focused_panel) - 1
+
+        if previous_panel_index < 0:
+            return
+
+        self.focused_panel.styles.display = 'none'
+
+        self.focused_panel = self.panels[previous_panel_index]
+        self.focused_panel.styles.display = 'block'
+        self.focused_panel.focus()
+
+    def action_next(self, option_id):
+        '''
+        Hide the current focused panel and create the next one.
+        '''
+        next_panel_index = self.panels.index(self.focused_panel) + 1
+
+        if next_panel_index < len(self.panels):
+            next_panel = self.panels[next_panel_index]
+            next_panel.styles.display = 'block'
+        else:
+            next_panel = make_next_panel(self.focused_panel, option_id)
+
+            if next_panel is None:
+                self.notify('Cannot display this content', severity='warning')
+                return
+
+            self.panels.append(next_panel)
+            next_panel.highlighted = 0
+
+        self.focused_panel.styles.display = 'none'
+        self.focused_panel = next_panel
+        self.focused_panel.focus()
+        self.mount(self.focused_panel)
+
+    def on_option_list_option_highlighted(self, event):
+        '''
+        The highlighted option has changed, so truncate any next panels.
+        '''
+        next_panel_index = self.panels.index(self.focused_panel) + 1
+
+        del self.panels[next_panel_index:]
+
+    def on_option_list_option_selected(self, event):
+        '''
+        An option has been selected, so advance to the next panel—unless the option selected is
+        "..", in which case go to the previous panel.
+        '''
+        if (
+            event.option_list != self.focused_panel or event.option_id == 'loading-indicator'
+        ):  # pragma: no cover
+            return
+
+        if event.option_id == '..':
+            self.action_previous()
+        else:
+            self.action_next(event.option_id)

borgmatic/actions/browse/configuration_files_list.py

@@ -0,0 +1,33 @@
+import os
+
+import textual.widgets
+
+import borgmatic.actions.browse.bindings
+
+
+class Configuration_files_list(textual.widgets.OptionList):
+    '''
+    A widget for selecting a single borgmatic configuration file from among available configuration
+    files. The item selection event is handled in a Carousel instance, the parent widget of an
+    Configuration_files_list.
+    '''
+
+    BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
+
+    def __init__(self, configs):
+        '''
+        Given a dict mapping from configuration path to corresponding configuration dict, add each
+        configuration path as an option to this widget.
+        '''
+        self.configs = configs
+        home_directory = os.path.expanduser('~')
+
+        super().__init__(
+            *(
+                textual.widgets.option_list.Option(unexpanded_path, id=config_path)
+                for config_path in configs
+                for unexpanded_path in (config_path.replace(home_directory, '~'),)
+            ),
+            classes='panel',
+        )
+        self.border_title = '📄 configuration files'

borgmatic/actions/browse/directory_list.py

@@ -0,0 +1,211 @@
+import os
+
+import textual.widgets
+
+import borgmatic.actions.browse.bindings
+import borgmatic.actions.browse.icons
+import borgmatic.actions.browse.loading
+import borgmatic.actions.browse.workers
+
+
+def get_relative_archive_path_components(archive_path, current_directory_path_components):
+    '''
+    Given an Archive_path instance and a tuple of path components for the currently browsed
+    directory, get the path components as a tuple for the archive path relative to that directory.
+
+    For instance, given an archive path with a path of 'foo/bar/baz/quux.txt' and current
+    directory path components of ('foo', 'bar'), return ('baz', 'quux.txt').
+
+    If the archive path is not actually relative to the current directory, return None.
+    '''
+    archive_path_components = tuple(archive_path.file_path.split(os.path.sep))
+
+    if not current_directory_path_components:
+        return archive_path_components
+
+    # If the loaded path doesn't match this directory list's own path, then we don't care about
+    # it for purposes of displaying this particular directory.
+    if (
+        tuple(archive_path_components[: len(current_directory_path_components)])
+        != current_directory_path_components
+    ):
+        return None
+
+    # Strip off the portion of the archive path that matches the directory list's own path.
+    return archive_path_components[len(current_directory_path_components) :]
+
+
+def make_directory_list_option(archive_path, relative_path_components):
+    '''
+    Given an Archive_path instance and a tuple of relative path components for it, make a
+    textual.widgets.option_list.Option for the path. Use an the icon based on whether this looks
+    like a terminal filename or a directory.
+    '''
+    pieces = (
+        borgmatic.actions.browse.icons.PATH_TYPE_ICONS.get(
+            archive_path.path_type if len(relative_path_components) == 1 else 'd', '❓'
+        ),
+        relative_path_components[0],
+    ) + (('→', archive_path.link_target) if archive_path.link_target else ())
+
+    return textual.widgets.option_list.Option(
+        prompt=' '.join(pieces), id=relative_path_components[0]
+    )
+
+
+def add_archive_paths(
+    directory_list,
+    config,
+    repository,
+    archive_name,
+    archive_paths,
+):
+    '''
+    Given a DirectoryList instance, a configuration dict, a repository dict, an archive name, and a
+    sequence of ArchivePath instances, add the paths to the directory list as options, sorting and
+    deduplicating the resulting directory list's options.
+
+    After all of this reshuffling, make sure the original highlighted option remains highlighted.
+    '''
+    highlighted_option = directory_list.highlighted_option
+    original_options_count = len(directory_list.options)
+
+    sorted_options = sorted(
+        (
+            *directory_list.options,
+            *(
+                make_directory_list_option(archive_path, relative_path_components)
+                for archive_path in archive_paths
+                for relative_path_components in (
+                    get_relative_archive_path_components(
+                        archive_path,
+                        directory_list.path_components,
+                    ),
+                )
+                if relative_path_components
+                if relative_path_components[0] not in directory_list._id_to_option
+            ),
+        ),
+        # The loading indicator "option" always goes to the bottom.
+        key=lambda option: ((option.id == 'loading-indicator'), option.prompt),
+    )
+
+    # If there aren't actually any options to add (due to deduplication), bail.
+    if len(sorted_options) == original_options_count:
+        return
+
+    # Retain the highlighted option position even as other options load around it.
+    directory_list.set_options(sorted_options)
+    directory_list.highlighted = (
+        directory_list.get_option_index(highlighted_option.id)
+        if highlighted_option and directory_list.highlighted_option_changed
+        else 0
+    )
+
+
+class Directory_list(textual.widgets.OptionList):
+    '''
+    A widget for selecting a path from among the contents of a particular directory in a Borg
+    archive. The item selection event is handled in a Carousel instance, the parent widget of a
+    Directory_list.
+    '''
+
+    BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
+
+    def __init__(self, config, repository, archive_name, path_loaded=None, path_components=None):
+        '''
+        Given a configuration dict, a repository dict, an archive name, an optional
+        Archive_path_loaded instance for signalling new paths as they load, and an optional tuple of
+        path components indicating this directory's position in the backed up filesystem, prepare to
+        load paths from the archive for eventual display in this widget. Actual loading kicks off in
+        on_mount() below.
+        '''
+        self.config = config
+        self.repository = repository
+        self.archive_name = archive_name
+        self.path_components = path_components or ()
+        self.highlighted_option_changed = False
+
+        super().__init__(classes='panel')
+
+        self.border_title = ' '.join(
+            (
+                '📁',
+                os.path.sep.join(self.path_components)
+                if self.path_components
+                else f'{archive_name}',
+            )
+        )
+
+        if self.path_components:
+            self.add_option(
+                textual.widgets.option_list.Option(
+                    '📁 ..',
+                    id='..',
+                ),
+            )
+
+        self.path_loaded = path_loaded or borgmatic.actions.browse.workers.Archive_path_loaded(
+            self, 'archive path loaded'
+        )
+
+        if not self.path_loaded.complete:
+            self.timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
+
+    def on_mount(self):
+        '''
+        When this widget gets mounted in the DOM, subscribe to path loaded events so that we can
+        find out about relevant archive paths as they load. And if this is a root directory list,
+        start loading paths from the archive. If this is a non-root directory list, add any already
+        loaded archive paths to this widget as options.
+
+        Loading is started *after* subscribing to path loaded signals so that there's not a gap
+        where we might miss out on any paths.
+        '''
+        self.path_loaded.subscribe(self, self.on_archive_path_loaded)
+
+        if self.path_components:
+            add_archive_paths(
+                directory_list=self,
+                config=self.config,
+                repository=self.repository,
+                archive_name=self.archive_name,
+                archive_paths=borgmatic.actions.browse.workers.get_paths(
+                    self.path_loaded.path_hierarchy, self.path_components
+                ),
+            )
+        else:
+            borgmatic.actions.browse.workers.load_archive_paths(
+                self.app,
+                path_loaded=self.path_loaded,
+                config=self.config,
+                repository=self.repository,
+                archive_name=self.archive_name,
+            )
+
+    def on_archive_path_loaded(self, data):
+        '''
+        When an archive path loads, add it as an option to this directory list. But if we get a
+        signal that all path loading is complete, stop and remove our loading indicator.
+        '''
+        if data is borgmatic.actions.browse.workers.LOADING_DONE:
+            self.timer.stop()
+            self.remove_option('loading-indicator')
+            return
+
+        add_archive_paths(
+            directory_list=self,
+            config=self.config,
+            repository=self.repository,
+            archive_name=self.archive_name,
+            archive_paths=(data,),
+        )
+
+    def on_option_list_option_highlighted(self, event):
+        '''
+        When the highlighted option changes, record that fact. This flag is consumed in
+        add_archive_paths() in order to retain the highlighted option even as other options load
+        around it.
+        '''
+        if self.highlighted not in {None, 0}:
+            self.highlighted_option_changed = True

borgmatic/actions/browse/file_preview.py

@@ -0,0 +1,87 @@
+import logging
+
+import rich.syntax
+import textual.binding
+import textual.widgets
+
+import borgmatic.actions.browse.loading
+import borgmatic.actions.browse.workers
+
+logger = logging.getLogger('__name__')
+
+
+class File_preview(textual.widgets.RichLog):
+    '''
+    A widget for extracting and previewing the contents of a file stored in a Borg archive.
+    '''
+
+    BINDINGS = (
+        *textual.widgets.RichLog.BINDINGS,
+        textual.binding.Binding(
+            key='up,k', action='scroll_up', description='scroll up', show=True, priority=True
+        ),
+        textual.binding.Binding(
+            key='down,j', action='scroll_down', description='scroll down', show=True, priority=True
+        ),
+        textual.binding.Binding(
+            key='pageup', action='page_up', description='page up', show=True, priority=True
+        ),
+        textual.binding.Binding(
+            key='pagedown', action='page_down', description='page down', show=True, priority=True
+        ),
+    )
+
+    def __init__(self, config, repository, archive_name, file_path):
+        '''
+        Given a configuration dict, a repository dict, an archive name, and the path of a file in
+        the archive, prepare to load the file's contents for eventual display in this widget. Actual
+        loading kicks off in on_mount() below.
+        '''
+        self.config = config
+        self.repository = repository
+        self.archive_name = archive_name
+        self.file_path = file_path
+
+        super().__init__(classes='panel')
+        self.border_title = f'📄 {self.file_path} preview'
+        self.auto_scroll = False
+        self.file_preview_loaded = borgmatic.actions.browse.workers.File_preview_loaded(
+            self, 'file preview loaded'
+        )
+
+        self.loading_timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
+
+    def on_mount(self):
+        '''
+        When this widget gets mounted in the DOM, subscribe to archive loaded events so that we can
+        find out about archives as they load. Also start loading file contents from the archive.
+
+        Loading is started *after* subscribing to the file preview loaded signal so that there's not
+        a gap where we might miss out on signal publishes.
+        '''
+        self.file_preview_loaded.subscribe(self, self.on_file_preview_loaded)
+
+        borgmatic.actions.browse.workers.load_file_preview(
+            self.app,
+            file_preview_loaded=self.file_preview_loaded,
+            config=self.config,
+            repository=self.repository,
+            archive_name=self.archive_name,
+            file_path=self.file_path,
+        )
+
+    def on_file_preview_loaded(self, file_contents):
+        '''
+        When a file loads, write its contents (syntax highlighted) to this file preview widget.
+        '''
+        self.loading_timer.stop()
+        self.clear()
+
+        if file_contents is None:
+            self.write('Cannot display a preview for this file')
+        else:
+            # Only pass the file path and not its contents to guess_lexer(). Passing the contents is
+            # more accurate, but also much slower.
+            self.write(
+                rich.syntax.Syntax(file_contents, rich.syntax.Syntax.guess_lexer(self.file_path))
+            )

borgmatic/actions/browse/icons.py

@@ -0,0 +1,8 @@
+import borgmatic.actions.browse.archive
+
+PATH_TYPE_ICONS = {
+    borgmatic.actions.browse.archive.Path_type.DIRECTORY.value: '📁',
+    borgmatic.actions.browse.archive.Path_type.LINK.value: '🔗',
+    borgmatic.actions.browse.archive.Path_type.PIPE.value: '🚰',
+    borgmatic.actions.browse.archive.Path_type.FILE.value: '📄',
+}

borgmatic/actions/browse/loading.py

@@ -0,0 +1,54 @@
+import contextlib
+import functools
+import logging
+
+import textual.widgets
+import textual.widgets.option_list
+
+LOADING_DOT_INTERVAL_SECONDS = 0.3
+
+
+logger = logging.getLogger('__name__')
+
+
+def update_inline_loading_indicator(widget):
+    '''
+    Given a textual.widgets.OptionList or a textual.widgets.RichLog instance, animate the existing
+    loading indicator inside it.
+    '''
+    if isinstance(widget, textual.widgets.OptionList):
+        with contextlib.suppress(textual.widgets.option_list.OptionDoesNotExist):
+            widget.replace_option_prompt(
+                'loading-indicator',
+                (str(widget.get_option('loading-indicator').prompt) + '.').replace('....', ''),
+            )
+    elif isinstance(widget, textual.widgets.RichLog):
+        with contextlib.suppress(IndexError):
+            loading_message = str(widget.lines[0].text)
+            widget.clear()
+            widget.write((loading_message + '.').replace('....', ''))
+    else:
+        raise ValueError(f'Unsupported widget type: {type(widget)}')
+
+
+LOADING_MESSAGE = '⏳ loading...'
+
+
+def add_inline_loading_indicator(widget):
+    '''
+    Given a textual.widgets.OptionList or a textual.widgets.RichLog instance, add a loading
+    indicator to it.
+    '''
+    if isinstance(widget, textual.widgets.OptionList):
+        loading_option = textual.widgets.option_list.Option(LOADING_MESSAGE, id='loading-indicator')
+        widget.add_option(loading_option)
+        widget.highlighted = None
+    elif isinstance(widget, textual.widgets.RichLog):
+        widget.write(LOADING_MESSAGE)
+    else:
+        raise ValueError(f'Unsupported widget type: {type(widget)}')
+
+    return widget.set_interval(
+        LOADING_DOT_INTERVAL_SECONDS,
+        functools.partial(update_inline_loading_indicator, widget),
+    )

borgmatic/actions/browse/logs.py

@@ -0,0 +1,102 @@
+import contextlib
+import logging
+
+import textual._context
+import textual.widgets
+import textual.worker
+
+import borgmatic.logger
+
+
+class Rich_color_formatter(logging.Formatter):
+    '''
+    A Python logging formatter that formats log records with Rich-compatible color markup according
+    to their levels.
+    '''
+
+    def __init__(self, *args, **kwargs):
+        self.prefix = None
+        super().__init__(
+            '{prefix}{message}',
+            *args,
+            style='{',
+            **kwargs,
+        )
+
+    def format(self, record):
+        '''
+        Given a log record, format it with Rich-compatibe color markup corresponding to its log
+        level.
+        '''
+        borgmatic.logger.add_custom_log_levels()
+
+        color = {
+            logging.CRITICAL: 'bright_red',
+            logging.ERROR: 'bright_red',
+            logging.WARNING: 'bright_yellow',
+            logging.ANSWER: 'bright_magenta',
+            logging.INFO: 'bright_green',
+            logging.DEBUG: 'bright_cyan',
+        }.get(record.levelno)
+        record.prefix = f'{self.prefix}: ' if self.prefix else ''
+
+        return f'[{color}]{super().format(record)}[/{color}]'
+
+
+class Browse_log_handler(logging.Handler):
+    '''
+    A Python log handler that writes any log records to a logging widget.
+    '''
+
+    def __init__(self, logs_widget):
+        '''
+        Given a logs widget, save it for use below.
+        '''
+        self.logs_widget = logs_widget
+
+        super().__init__()
+
+    def emit(self, record):
+        '''
+        Given a log record, format it and log it to the logs widgets. This works whether or not the
+        logging is happening in the main thread.
+        '''
+        message = self.format(record)
+
+        try:
+            textual.worker.get_current_worker()
+            self.logs_widget.app.call_from_thread(self.logs_widget.write, message)
+        except (RuntimeError, textual.worker.NoActiveWorker):
+            with contextlib.suppress(textual._context.NoActiveAppError):
+                self.logs_widget.write(message)
+
+
+def log_to_widget(logs_widget):
+    '''
+    Given a Textual RichLog logs widget, add a log handler and formatter that logs to it. Also
+    remove the default borgmatic console log handler so it doesn't try to log all over our UI.
+    '''
+    handler = Browse_log_handler(logs_widget)
+    handler.setFormatter(Rich_color_formatter())
+    logger = logging.getLogger()
+    logger.setLevel(min(handler.level for handler in logger.handlers))
+    logger.addHandler(handler)
+
+    with contextlib.suppress(StopIteration):
+        console_handler = next(
+            handler
+            for handler in logging.getLogger().handlers
+            if isinstance(handler, borgmatic.logger.Multi_stream_handler)
+        )
+        logger.removeHandler(console_handler)
+
+
+class Logs(textual.widgets.RichLog):
+    '''
+    A widget for viewing borgmatic logs in realtime. The log level is determined by borgmatic's
+    current verbosity level.
+    '''
+
+    def __init__(self):
+        super().__init__(markup=True, id='logs', classes='panel')
+        self.border_title = '🪵 logs'

borgmatic/actions/browse/repositories_list.py

@@ -0,0 +1,30 @@
+import textual.widgets
+
+import borgmatic.actions.browse.bindings
+
+
+class Repositories_list(textual.widgets.OptionList):
+    '''
+    A widget for selecting a single Borg repository from among the repositories in a borgmatic
+    configuration file. The item selection event is handled in a Carousel instance, the parent
+    widget of a Repositories_list.
+    '''
+
+    BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
+
+    def __init__(self, config):
+        '''
+        Given a configuration dict, populate the repositories in this widget.
+        '''
+        self.config = config
+        self.repositories = config['repositories']
+
+        super().__init__(
+            *(
+                textual.widgets.option_list.Option(label, id=index)
+                for index, repository in enumerate(self.repositories)
+                for label in (repository.get('label', repository.get('path')),)
+            ),
+            classes='panel',
+        )
+        self.border_title = '📦 repositories'

borgmatic/actions/browse/run.py

@@ -0,0 +1,29 @@
+import logging
+
+
+def run_browse(
+    diff_arguments,
+    global_arguments,
+    configs,
+):
+    '''
+    Run the "browse" action for the given borgmatic configurations. This launches a console UI.
+
+    Raise ValueError if the Textual library (a prerequisite for this action) can't be imported.
+    '''
+    if not configs:
+        return
+
+    logging.getLogger('asyncio').setLevel(logging.WARNING)
+
+    try:
+        import textual  # noqa: F401, PLC0415
+    except ImportError:  # pragma: no cover
+        raise ValueError(
+            'Unable to import the Textual library for the browse action; try installing "borgmatic[browse]"'
+        )
+
+    import borgmatic.actions.browse.app  # noqa: PLC0415
+
+    app = borgmatic.actions.browse.app.Browse_app(configs)
+    app.run()

borgmatic/actions/browse/workers.py

@@ -0,0 +1,192 @@
+import logging
+import os
+
+import textual
+import textual.signal
+
+import borgmatic.actions.browse.archive
+
+logger = logging.getLogger('__name__')
+
+
+LOADING_DONE = object()
+
+
+class Archive_loaded(textual.signal.Signal):
+    '''
+    A signal that publishes when each subsequent archive is loaded from a repository, intended for
+    consumption in widgets that display archives as they are loaded. This signal also publishes
+    when loading is complete.
+
+    Each subscribed callback call includes the archive as an archive name string. Given the lack of
+    other identifying information (configuration file, repository), there should be a separate
+    Archive_loaded instance per repository.
+    '''
+
+
+@textual.work(thread=True)
+def add_repository_archives(browse_app, archive_loaded, config, repository):
+    '''
+    Given a running Browse_app instance, an Archive_loaded instance, a configuration dict, and a
+    repository dict, load a list of the archives from the repository and add them as options in the
+    archives list. Reverse the order so the most recent archive is first.
+
+    This function runs in a separate thread from the main UI. When loading is complete, publish a
+    loading done signal.
+    '''
+    archives_data = borgmatic.actions.browse.archive.get_repository_archives(config, repository)
+
+    # Reverse the archives, so the common case of accessing the latest archive is easy because it's
+    # at the top.
+    for archive in reversed(archives_data['archives']):
+        archive_loaded.publish(archive['archive'])
+
+    archive_loaded.publish(LOADING_DONE)
+
+
+def record_path(archive_path, hierarchy, path_components):
+    '''
+    Given an Archive_path instance, a dict capturing a filesystem hierarchy of paths, and a tuple of
+    path components for the archive path, set the archive path into the hierarchy data structure.
+
+    For instance, if given an archive path and path components representing "foo/bar/baz.txt",
+    produce a hierarchy that looks like:
+
+        {'foo': {'bar': {'baz.txt': Archive_path('-', 'foo/bar/baz.txt', '')}}}
+
+    Note that the hierarchy is modified in place, so any existing paths there are retained.
+    '''
+    if len(path_components) == 1:
+        hierarchy[path_components[0]] = {} if archive_path.path_type == 'd' else archive_path
+        return
+
+    record_path(archive_path, hierarchy.setdefault(path_components[0], {}), path_components[1:])
+
+
+def get_paths(hierarchy, path_components, full_path_components=None):
+    '''
+    Given a dict capturing a filesystem hierarchy of paths (or a subset thereof), a tuple of path
+    components for a directory path relative to the hierarchy root, and an optional tuple of
+    *absolute* path components for the same path (if different), return a generator of the
+    contained file and directory Archive_path instances from the hierarchy.
+
+    For instance, given the following hierarchy:
+
+        {'foo': {'bar': {'baz.txt': Archive_path('-', 'foo/bar/baz.txt', ''), 'quux': {}}}}
+
+    ... and path components of ('foo', 'bar'), return a generator with the following:
+
+        * Archive_path('-', 'foo/bar/baz.txt', '')
+        * Archive_path('d', 'foo/bar/quux', '')
+
+    The given absolute path components are use to construct directory paths like that last archive
+    path.
+    '''
+    if full_path_components is None:
+        full_path_components = path_components
+
+    if len(path_components) == 1:
+        try:
+            return (
+                archive_path
+                if isinstance(archive_path, borgmatic.actions.browse.archive.Archive_path)
+                else borgmatic.actions.browse.archive.Archive_path(
+                    'd', os.path.join(*full_path_components, component), ''
+                )
+                for component, archive_path in hierarchy[path_components[0]].items()
+            )
+        except KeyError:
+            raise ValueError(f'Unknown file or directory: {path_components[0]}')
+
+    try:
+        return get_paths(hierarchy[path_components[0]], path_components[1:], full_path_components)
+    except KeyError:
+        raise ValueError(f'Unknown directory: {path_components[0]}')
+
+
+class Archive_path_loaded(textual.signal.Signal):
+    '''
+    A signal that publishes when each subsequent path is loaded from an archive, intended for
+    consumption in widgets that display paths as they are loaded. This signal also tracks the
+    complete filesystem hierarchy seen thus far, so new widgets that get created after loading has
+    started can "catch up" with existing known paths. Lastly, this signal publishes and tracks when
+    loading is complete.
+
+    Each subscrided callback call includes the loaded path as an Archive_path instance. There is
+    intended to be a separate Archive_path_loaded instance per archive, but that instance should be
+    shared among several different widgets for the same archive for performance reasons.
+    '''
+
+    def __init__(self, owner, name):
+        self.path_hierarchy = {}
+        self.complete = False
+
+        super().__init__(owner, name)
+
+    def publish(self, archive_path):
+        '''
+        Publish the given archive path to subscribers and record its path locally. But if the
+        archive path is actually LOADING_DONE, then record loading as complete.
+        '''
+        super().publish(archive_path)
+
+        if archive_path is LOADING_DONE:
+            self.complete = True
+        else:
+            record_path(
+                archive_path, self.path_hierarchy, archive_path.file_path.split(os.path.sep)
+            )
+
+
+@textual.work(thread=True)
+def load_archive_paths(browse_app, path_loaded, config, repository, archive_name):
+    '''
+    Given a running Browse_app instance, an Archive_path_loaded instance, a configuration dict, a
+    repository dict, and an archive name, load the paths in this archive and publish each one via
+    the Archive_path_loaded signal, so interested widgets can subscribe. Also send a "loading done"
+    signal when loading completes.
+
+    This function runs in a separate thread from the main UI. When loading is complete, publish a
+    loading done signal.
+    '''
+    for archive_path in borgmatic.actions.browse.archive.get_archive_paths(
+        config, repository, archive_name
+    ):
+        path_loaded.publish(archive_path)
+
+    path_loaded.publish(LOADING_DONE)
+
+
+class File_preview_loaded(textual.signal.Signal):
+    '''
+    A signal that publishes when file contents are loaded from an archive, intended for consumption
+    in widgets that display loaded files. This signal also publishes when loading is complete.
+
+    Each published callback includes a the file's contents as a string. Given the lack of other
+    identifying information (configuration file, repository, archive), there should be a separate
+    Archive_loaded instance per previewed file.
+    '''
+
+
+@textual.work(thread=True)
+def load_file_preview(
+    browse_app,
+    file_preview_loaded,
+    config,
+    repository,
+    archive_name,
+    file_path,
+):
+    '''
+    Given a running Browse_app instance, a File_preview_loaded instance, a configuration dict, a
+    repository dict, an archive name, and the path of a file in that archive, load the contents of
+    the file and write it into the given file preview widget.
+
+    This function runs in a separate thread from the main UI. When loading is complete, publish a
+    loading done signal.
+    '''
+    file_contents = borgmatic.actions.browse.archive.get_archive_file_content(
+        config, repository, archive_name, file_path
+    )
+
+    file_preview_loaded.publish(file_contents)

borgmatic/actions/check.py

@@ -9,6 +9,7 @@ import pathlib
 import random
 import shlex
 import shutil
+import subprocess
 import textwrap
 
 import borgmatic.actions.config.bootstrap
@@ -382,7 +383,7 @@ def collect_spot_check_source_paths(
         # Omit "progress" because it interferes with "list_details".
         config=dict(config, progress=False, list_details=True),
         patterns=borgmatic.actions.pattern.process_patterns(
-            borgmatic.actions.pattern.collect_patterns(config)
+            borgmatic.actions.pattern.collect_patterns(config, working_directory)
             + tuple(
                 borgmatic.borg.pattern.Pattern(
                     config_path,
@@ -418,7 +419,13 @@ def collect_spot_check_source_paths(
     )
 
     return tuple(
-        path for path in paths if os.path.isfile(os.path.join(working_directory or '', path))
+        # Use dict.fromkeys() to deduplicate file paths, which are present in Borg's dry run output
+        # when there are overlapping source patterns. For instance, if both "/foo" and
+        # "/foo/file.txt" are in configured patterns, then "/foo/file.txt" will show up in Borg's
+        # dry run output twice.
+        dict.fromkeys(
+            path for path in paths if os.path.isfile(os.path.join(working_directory or '', path))
+        )
     )
 
 
@@ -518,66 +525,100 @@ def compare_spot_check_hashes(
         source_sample_paths_subset = tuple(
             itertools.islice(source_sample_paths_iterator, SAMPLE_PATHS_SUBSET_COUNT),
         )
+
         if not source_sample_paths_subset:
             break
 
         hash_paths = tuple(
             path for path in source_sample_paths_subset if path in hashable_source_sample_path
         )
-        hash_lines = borgmatic.execute.execute_command_and_capture_output(
-            tuple(
-                shlex.quote(part)
-                for part in shlex.split(spot_check_config.get('xxh64sum_command', 'xxh64sum'))
-            )
-            + hash_paths,
-            working_directory=working_directory,
-        )
 
-        source_hashes.update(
-            **dict(
-                zip(
-                    # xxh64sum rewrites/escapes the paths that it returns alongside its hashes, for
-                    # instance if they contain special characters. When that happens, they don't
-                    # match the original source paths and therefore hash lookups fail. So when
-                    # building this lookup dict, use the original unaltered paths we provided as
-                    # input to xxh64sum.
-                    hash_paths,
-                    (
-                        # For some reason, xxh64sum prefixes the hash with a backslash if the path
-                        # contains a newline. Work around that.
-                        line.split('  ', 1)[0].lstrip('\\')
-                        for line in hash_lines
+        try:
+            hash_lines = borgmatic.execute.execute_command_and_capture_output(
+                tuple(
+                    shlex.quote(part)
+                    for part in shlex.split(spot_check_config.get('xxh64sum_command', 'xxh64sum'))
+                )
+                + hash_paths,
+                working_directory=working_directory,
+            )
+            source_hashes.update(
+                **dict(
+                    zip(
+                        # xxh64sum rewrites/escapes the paths that it returns alongside its hashes, for
+                        # instance if they contain special characters. When that happens, they don't
+                        # match the original source paths and therefore hash lookups fail. So when
+                        # building this lookup dict, use the original unaltered paths we provided as
+                        # input to xxh64sum.
+                        hash_paths,
+                        (
+                            # For some reason, xxh64sum prefixes the hash with a backslash if the path
+                            # contains a newline. Work around that.
+                            line.split('  ', 1)[0].lstrip('\\')
+                            for line in hash_lines
+                        ),
                     ),
+                    # Represent non-existent files as having empty hashes so the comparison below still
+                    # works. Same thing for filesystem links, since Borg produces empty archive hashes
+                    # for them.
+                    **{
+                        path: ''
+                        for path in source_sample_paths_subset
+                        if path not in hashable_source_sample_path
+                    },
                 ),
-                # Represent non-existent files as having empty hashes so the comparison below still
-                # works. Same thing for filesystem links, since Borg produces empty archive hashes
-                # for them.
-                **{
-                    path: ''
-                    for path in source_sample_paths_subset
-                    if path not in hashable_source_sample_path
-                },
-            ),
-        )
+            )
+        except subprocess.CalledProcessError:
+            # This can happen if a file we planned to hash gets deleted right before we try to hash
+            # it. Falling back to individual file hashing allows us to find and mark just the
+            # file(s) with problems instead of failing the whole batch.
+            logger.warning(
+                'Bulk source path hashing failed for this batch; falling back to individual file hashing'
+            )
+
+            for hash_path in hash_paths:
+                try:
+                    hash_lines = borgmatic.execute.execute_command_and_capture_output(
+                        (
+                            *(
+                                shlex.quote(part)
+                                for part in shlex.split(
+                                    spot_check_config.get('xxh64sum_command', 'xxh64sum')
+                                )
+                            ),
+                            hash_path,
+                        ),
+                        working_directory=working_directory,
+                    )
+                    source_hashes[hash_path] = next(hash_lines).split('  ', 1)[0].lstrip('\\')
+                except (subprocess.CalledProcessError, StopIteration):  # noqa: PERF203
+                    logger.warning(
+                        f'Source path hashing failed for {hash_path}; treating as missing'
+                    )
+                    source_hashes[hash_path] = ''
 
         # Get the hash for each file in the archive.
-        archive_hashes.update(
-            **{
-                entry['path']: entry['xxh64']
-                for entry in borgmatic.borg.list.capture_archive_listing(
-                    repository['path'],
-                    archive,
-                    config,
-                    local_borg_version,
-                    global_arguments,
-                    list_paths=source_sample_paths_subset,
-                    path_format='{xxh64}{path}',
-                    local_path=local_path,
-                    remote_path=remote_path,
-                )
-                if entry
-            },
-        )
+        for entry in borgmatic.borg.list.capture_archive_listing(
+            repository['path'],
+            archive,
+            config,
+            local_borg_version,
+            global_arguments,
+            list_paths=source_sample_paths_subset,
+            path_format='{xxh64}{path}{linktarget}{target}',
+            local_path=local_path,
+            remote_path=remote_path,
+        ):
+            if not entry:
+                continue
+
+            # Borg can't get hashes of stored hard links. So if this is a hard link path (and not
+            # deemed as the "original" by Borg), then skip hashing of it.
+            if entry.get('linktarget') or entry.get('target'):
+                source_hashes.pop(os.path.join('/', entry['path']), None)
+                continue
+
+            archive_hashes[entry['path']] = entry['xxh64']
 
     # Compare the source hashes with the archive hashes to see how many match.
     failing_paths = []
@@ -676,7 +717,7 @@ def spot_check(
         )
         logger.debug(f'Paths in latest archive but not source paths: {truncated_archive_paths}')
         raise ValueError(
-            'Spot check failed: There are no source paths to compare against the archive',
+            'Spot check failed; there are no source paths to compare against the archive',
         )
 
     # Calculate the percentage delta between the source paths count and the archive paths count, and
@@ -690,19 +731,13 @@ def spot_check(
             width=MAX_SPOT_CHECK_PATHS_LENGTH,
             placeholder=' ...',
         )
-        logger.debug(
-            f'Paths in source paths but not latest archive: {truncated_exclusive_source_paths}',
-        )
         truncated_exclusive_archive_paths = textwrap.shorten(
             ', '.join(set(archive_paths) - rootless_source_paths) or 'none',
             width=MAX_SPOT_CHECK_PATHS_LENGTH,
             placeholder=' ...',
         )
-        logger.debug(
-            f'Paths in latest archive but not source paths: {truncated_exclusive_archive_paths}',
-        )
         raise ValueError(
-            f'Spot check failed: {count_delta_percentage:.2f}% file count delta between source paths and latest archive (tolerance is {spot_check_config["count_tolerance_percentage"]}%)',
+            f'Spot check failed\n{count_delta_percentage:.2f}% file count delta between source paths ({len(source_paths)} total) and latest archive ({len(archive_paths)} total); tolerance is {spot_check_config["count_tolerance_percentage"]}%\nOnly in source paths: {truncated_exclusive_source_paths}\nOnly in latest archive: {truncated_exclusive_archive_paths}',
         )
 
     failing_paths = compare_spot_check_hashes(
@@ -727,11 +762,8 @@ def spot_check(
             width=MAX_SPOT_CHECK_PATHS_LENGTH,
             placeholder=' ...',
         )
-        logger.debug(
-            f'Source paths with data not matching the latest archive: {truncated_failing_paths}',
-        )
         raise ValueError(
-            f'Spot check failed: {failing_percentage:.2f}% of source paths with data not matching the latest archive (tolerance is {data_tolerance_percentage}%)',
+            f'Spot check failed\n{failing_percentage:.2f}% of source paths ({len(failing_paths)} out of {len(source_paths)} checked) with data not matching the latest archive; tolerance is {data_tolerance_percentage}%\nSource paths with non-matching data: {truncated_failing_paths}',
         )
 
     logger.info(

borgmatic/actions/config/bootstrap.py

@@ -120,7 +120,7 @@ def run_bootstrap(bootstrap_arguments, global_arguments, local_borg_version):
             borgmatic_runtime_directory,
         )
 
-    logger.info(f"Bootstrapping config paths: {', '.join(manifest_config_paths)}")
+    logger.info(f"Bootstrapping configuration paths: {', '.join(manifest_config_paths)}")
 
     borgmatic.borg.extract.extract_archive(
         global_arguments.dry_run,

borgmatic/actions/config/show.py

@@ -0,0 +1,44 @@
+import json
+import logging
+import sys
+
+import borgmatic.config.generate
+import borgmatic.logger
+
+logger = logging.getLogger(__name__)
+
+
+def run_show(show_arguments, configs):
+    '''
+    Given the show arguments as an argparse.Namespace instance and a dict of configuration filename
+    to corresponding parsed configuration, run the "show" action. That consists of rendering and
+    logging the computed configuration as YAML, separating the configuration for each file with
+    "---".
+
+    If show_arguments.option is set, limit the results to the value of that single option. If
+    show_arguments.json is True, render the results as JSON with one array element per configuration
+    file.
+    '''
+    borgmatic.logger.add_custom_log_levels()
+
+    if show_arguments.json:
+        sys.stdout.write(
+            json.dumps(
+                [
+                    config.get(show_arguments.option) if show_arguments.option else config
+                    for config in configs.values()
+                ]
+            )
+        )
+
+        return
+
+    for config in configs.values():
+        if len(configs) > 1:
+            logger.answer('---')
+
+        logger.answer(
+            borgmatic.config.generate.render_configuration(
+                config.get(show_arguments.option) if show_arguments.option else config
+            ).rstrip()
+        )

borgmatic/actions/create.py

@@ -45,7 +45,7 @@ def run_create(
 
     with borgmatic.config.paths.Runtime_directory(config) as borgmatic_runtime_directory:
         patterns = pattern.process_patterns(
-            pattern.collect_patterns(config),
+            pattern.collect_patterns(config, working_directory),
             config,
             working_directory,
             borgmatic_runtime_directory,

borgmatic/actions/diff.py

@@ -0,0 +1,63 @@
+import logging
+
+import borgmatic.actions.pattern
+import borgmatic.borg.diff
+
+logger = logging.getLogger(__name__)
+
+
+def run_diff(
+    repository,
+    config,
+    local_borg_version,
+    diff_arguments,
+    global_arguments,
+    local_path,
+    remote_path,
+):
+    '''
+    Run the "diff" action for the given repository.
+    '''
+
+    # Only process patterns if only_patterns flag is set
+    if diff_arguments.only_patterns:
+        working_directory = borgmatic.config.paths.get_working_directory(config)
+        processed_patterns = borgmatic.actions.pattern.process_patterns(
+            (*borgmatic.actions.pattern.collect_patterns(config, working_directory),),
+            config,
+            working_directory,
+        )
+    else:
+        processed_patterns = None
+
+    archive = borgmatic.borg.repo_list.resolve_archive_name(
+        repository['path'],
+        diff_arguments.archive,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+    second_archive = borgmatic.borg.repo_list.resolve_archive_name(
+        repository['path'],
+        diff_arguments.second_archive,
+        config,
+        local_borg_version,
+        global_arguments,
+        local_path,
+        remote_path,
+    )
+
+    borgmatic.borg.diff.diff(
+        repository['path'],
+        archive,
+        second_archive,
+        config,
+        local_borg_version,
+        diff_arguments,
+        global_arguments,
+        local_path=local_path,
+        remote_path=remote_path,
+        patterns=processed_patterns,
+    )

borgmatic/actions/pattern.py

@@ -34,10 +34,11 @@ def parse_pattern(pattern_line, default_style=borgmatic.borg.pattern.Pattern_sty
     )
 
 
-def collect_patterns(config):
+def collect_patterns(config, working_directory):
     '''
-    Given a configuration dict, produce a single sequence of patterns comprised of the configured
-    source directories, patterns, excludes, pattern files, and exclude files.
+    Given a configuration dict and the working directory, produce a single sequence of patterns
+    comprised of the configured source directories, patterns, excludes, pattern files, and exclude
+    files.
 
     The idea is that Borg has all these different ways of specifying includes, excludes, source
     directories, etc., but we'd like to collapse them all down to one common format (patterns) for
@@ -68,7 +69,8 @@ def collect_patterns(config):
             + tuple(
                 parse_pattern(pattern_line.strip())
                 for filename in config.get('patterns_from', ())
-                for pattern_line in open(filename, encoding='utf-8').readlines()
+                for expanded_path in expand_directory(filename, working_directory)
+                for pattern_line in open(expanded_path, encoding='utf-8')
                 if not pattern_line.lstrip().startswith('#')
                 if pattern_line.strip()
             )
@@ -78,7 +80,8 @@ def collect_patterns(config):
                     borgmatic.borg.pattern.Pattern_style.FNMATCH,
                 )
                 for filename in config.get('exclude_from', ())
-                for exclude_line in open(filename, encoding='utf-8').readlines()
+                for expanded_path in expand_directory(filename, working_directory)
+                for exclude_line in open(expanded_path, encoding='utf-8')
                 if not exclude_line.lstrip().startswith('#')
                 if exclude_line.strip()
             )

borgmatic/actions/recreate.py

@@ -18,6 +18,7 @@ def run_recreate(
     local_borg_version,
     recreate_arguments,
     global_arguments,
+    dry_run_label,
     local_path,
     remote_path,
 ):
@@ -25,14 +26,16 @@ def run_recreate(
     Run the "recreate" action for the given repository.
     '''
     if recreate_arguments.archive:
-        logger.answer(f'Recreating archive {recreate_arguments.archive}')
+        logger.answer(f'Recreating archive {recreate_arguments.archive}{dry_run_label}')
     else:
-        logger.answer('Recreating repository')
+        logger.answer(f'Recreating repository{dry_run_label}')
+
+    working_directory = borgmatic.config.paths.get_working_directory(config)
 
     # Collect and process patterns.
     processed_patterns = borgmatic.actions.pattern.process_patterns(
         (
-            *borgmatic.actions.pattern.collect_patterns(config),
+            *borgmatic.actions.pattern.collect_patterns(config, working_directory),
             # Also add borgmatic-specific paths, so they don't get excluded from the recreated
             # archive. Note that this doesn't currently work for archives created with Borg 1.2 or
             # below.
@@ -41,7 +44,7 @@ def run_recreate(
             ),
         ),
         config,
-        borgmatic.config.paths.get_working_directory(config),
+        working_directory,
     )
 
     archive = borgmatic.borg.repo_list.resolve_archive_name(

borgmatic/actions/restore.py

@@ -1,4 +1,5 @@
 import collections
+import locale
 import logging
 import os
 import pathlib
@@ -317,7 +318,7 @@ def collect_dumps_from_archive(
                 extract_to_stdout=True,
             )
             .stdout.read()
-            .decode(),
+            .decode(locale.getpreferredencoding()),
             dumps_metadata_entry['path'],
         ):
             dumps_from_archive[dump] = None
@@ -538,7 +539,7 @@ def run_restore(
 
     with borgmatic.config.paths.Runtime_directory(config) as borgmatic_runtime_directory:
         patterns = borgmatic.actions.pattern.process_patterns(
-            borgmatic.actions.pattern.collect_patterns(config),
+            borgmatic.actions.pattern.collect_patterns(config, working_directory),
             config,
             working_directory,
         )

borgmatic/borg/borg.py

@@ -29,7 +29,6 @@ def run_arbitrary_borg(
     command.
     '''
     borgmatic.logger.add_custom_log_levels()
-    lock_wait = config.get('lock_wait', None)
 
     try:
         options = options[1:] if options[0] == '--' else options
@@ -53,7 +52,7 @@ def run_arbitrary_borg(
         + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
         + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
         + flags.make_flags('remote-path', remote_path)
-        + flags.make_flags('lock-wait', lock_wait)
+        + flags.make_flags('lock-wait', config.get('lock_wait'))
         + command_options
     )
 

borgmatic/borg/break_lock.py

@@ -21,8 +21,8 @@ def break_lock(
     argparse.Namespace of global arguments, and optional local and remote Borg paths, break any
     repository and cache locks leftover from Borg aborting.
     '''
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('break_lock', '')
 
     full_command = (

borgmatic/borg/change_passphrase.py

@@ -24,8 +24,8 @@ def change_passphrase(
     based on an interactive prompt.
     '''
     borgmatic.logger.add_custom_log_levels()
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('key_change_passphrase', '')
 
     full_command = (

borgmatic/borg/check.py

@@ -20,7 +20,7 @@ def make_archive_filter_flags(local_borg_version, config, checks, check_argument
     flag. And if "prefix" is set in configuration and "archives" is in checks, then include a
     "--match-archives" flag.
     '''
-    check_last = config.get('check_last', None)
+    check_last = config.get('check_last')
     prefix = config.get('prefix')
 
     if 'archives' in checks or 'data' in checks:
@@ -149,8 +149,10 @@ def check_archives(
 
     max_duration = check_arguments.max_duration or repository_check_config.get('max_duration')
 
+    # If not configured, elevate Borg's exit code 1 (an ostensible warning) to error, because Borg
+    # returns exit code 1 for repository check errors!
+    borg_exit_codes = [*config.get('borg_exit_codes', []), *[{'code': 1, 'treat_as': 'error'}]]
     umask = config.get('umask')
-    borg_exit_codes = config.get('borg_exit_codes')
     working_directory = borgmatic.config.paths.get_working_directory(config)
 
     if 'data' in checks:

borgmatic/borg/compact.py

@@ -3,7 +3,7 @@ import shlex
 
 import borgmatic.config.paths
 from borgmatic.borg import environment, feature, flags
-from borgmatic.execute import execute_command
+from borgmatic.execute import DO_NOT_CAPTURE, execute_command
 
 logger = logging.getLogger(__name__)
 
@@ -22,8 +22,8 @@ def compact_segments(
     Given dry-run flag, a local or remote repository path, a configuration dict, and the local Borg
     version, compact the segments in a repository.
     '''
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('compact', '')
     threshold = config.get('compact_threshold')
 
@@ -35,7 +35,7 @@ def compact_segments(
         + (('--lock-wait', str(lock_wait)) if lock_wait else ())
         + (('--progress',) if config.get('progress') else ())
         + (('--cleanup-commits',) if cleanup_commits else ())
-        + (('--threshold', str(threshold)) if threshold else ())
+        + (('--threshold', str(threshold)) if threshold is not None else ())
         + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
         + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
         + (
@@ -54,6 +54,7 @@ def compact_segments(
     execute_command(
         full_command,
         output_log_level=logging.INFO,
+        output_file=DO_NOT_CAPTURE if config.get('progress') else None,
         environment=environment.make_environment(config),
         working_directory=borgmatic.config.paths.get_working_directory(config),
         borg_local_path=local_path,

borgmatic/borg/create.py

@@ -179,25 +179,25 @@ def make_base_create_command(  # noqa: PLR0912
     return a tuple of (base Borg create command flags, Borg create command positional arguments,
     open pattern file handle).
     '''
-    if config.get('source_directories_must_exist', False):
-        logger.warning(
-            'The "source_directories_must_exist" option is deprecated and will be removed from a future release; borgmatic now errors on missing files as Borg runs'
+    if config.get('source_directories_must_exist', True):
+        borgmatic.borg.pattern.check_all_root_patterns_exist(
+            patterns, borgmatic.config.paths.get_working_directory(config)
         )
-        borgmatic.borg.pattern.check_all_root_patterns_exist(patterns)
 
     patterns_file = borgmatic.borg.pattern.write_patterns_file(
         patterns,
         borgmatic_runtime_directory,
     )
-    checkpoint_interval = config.get('checkpoint_interval', None)
-    checkpoint_volume = config.get('checkpoint_volume', None)
-    chunker_params = config.get('chunker_params', None)
-    compression = config.get('compression', None)
-    upload_rate_limit = config.get('upload_rate_limit', None)
-    upload_buffer_size = config.get('upload_buffer_size', None)
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    checkpoint_interval = config.get('checkpoint_interval')
+    checkpoint_volume = config.get('checkpoint_volume')
+    chunker_params = config.get('chunker_params')
+    compression = config.get('compression')
+    upload_rate_limit = config.get('upload_rate_limit')
+    upload_buffer_size = config.get('upload_buffer_size')
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     list_filter_flags = flags.make_list_filter_flags(local_borg_version, dry_run)
+    files_changed = config.get('files_changed')
     files_cache = config.get('files_cache')
     archive_name_format = (
         config.get('archive_name_format', flags.get_default_archive_name_format(local_borg_version))
@@ -248,6 +248,7 @@ def make_base_create_command(  # noqa: PLR0912
         + (('--nobirthtime',) if config.get('birthtime') is False else ())
         + (('--read-special',) if config.get('read_special') or stream_processes else ())
         + noflags_flags
+        + (('--files-changed', files_changed) if files_changed else ())
         + (('--files-cache', files_cache) if files_cache else ())
         + (('--remote-path', remote_path) if remote_path else ())
         + (('--umask', str(umask)) if umask else ())
@@ -270,7 +271,7 @@ def make_base_create_command(  # noqa: PLR0912
     working_directory = borgmatic.config.paths.get_working_directory(config)
 
     if config.get('unsafe_skip_path_validation_before_create'):
-        logger.warning(
+        logger.debug(
             'Skipping pre-backup path validation due to "unsafe_skip_path_validation_before_create" option.'
         )
 
@@ -373,7 +374,9 @@ def create_archive(
 
     if json:
         output_log_level = None
-    elif config.get('list_details') or (config.get('statistics') and not dry_run):
+    elif config.get('list_details') or (
+        (config.get('statistics') or config.get('quick_statistics')) and not dry_run
+    ):
         output_log_level = logging.ANSWER
     else:
         output_log_level = logging.INFO
@@ -385,6 +388,11 @@ def create_archive(
     create_flags += (
         (('--info',) if logger.getEffectiveLevel() == logging.INFO and not json else ())
         + (('--stats',) if config.get('statistics') and not json and not dry_run else ())
+        + (
+            ('--quick-stats',)
+            if config.get('quick_statistics') and not json and not dry_run
+            else ()
+        )
         + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) and not json else ())
         + (('--progress',) if config.get('progress') else ())
         + (('--json',) if json else ())
@@ -392,7 +400,7 @@ def create_archive(
     borg_exit_codes = config.get('borg_exit_codes')
 
     if stream_processes:
-        return '\n'.join(
+        output = '\n'.join(
             execute_command_with_processes(
                 create_flags + create_positional_arguments,
                 stream_processes,
@@ -404,9 +412,10 @@ def create_archive(
                 borg_exit_codes=borg_exit_codes,
             )
         )
+        return output if json else None
 
     if output_log_level is None:
-        return '\n'.join(
+        output = '\n'.join(
             execute_command_and_capture_output(
                 create_flags + create_positional_arguments,
                 working_directory=working_directory,
@@ -415,6 +424,7 @@ def create_archive(
                 borg_exit_codes=borg_exit_codes,
             )
         )
+        return output if json else None
 
     execute_command(
         create_flags + create_positional_arguments,

borgmatic/borg/diff.py

@@ -0,0 +1,88 @@
+import logging
+import shlex
+
+import borgmatic.borg.environment
+import borgmatic.borg.feature
+import borgmatic.config.paths
+import borgmatic.execute
+from borgmatic.borg import flags
+from borgmatic.borg.pattern import write_patterns_file
+
+logger = logging.getLogger(__name__)
+
+
+def diff(
+    repository,
+    archive,
+    second_archive,
+    config,
+    local_borg_version,
+    diff_arguments,
+    global_arguments,
+    local_path,
+    remote_path=None,
+    patterns=None,
+):
+    '''
+    Given a local or remote repository path, two archive names, a configuration dict, the local Borg
+    version string, an argparse.Namespace of diff arguments, an argparse.Namespace of global
+    arguments, optional local and remote Borg paths, executes the diff command with the given
+    arguments.
+    '''
+    borgmatic.logger.add_custom_log_levels()
+
+    lock_wait = config.get('lock_wait')
+    extra_borg_options = config.get('extra_borg_options', {}).get('diff', '')
+
+    if diff_arguments.only_patterns:
+        # Write patterns to a temporary file and use that file with --patterns-from.
+        patterns_file = write_patterns_file(
+            patterns,
+            borgmatic.config.paths.get_working_directory(config),
+        )
+    else:
+        patterns_file = None
+
+    if borgmatic.borg.feature.available(
+        borgmatic.borg.feature.Feature.NUMERIC_IDS, local_borg_version
+    ):
+        numeric_ids_flags = ('--numeric-ids',) if config.get('numeric_ids') else ()
+    else:
+        numeric_ids_flags = ('--numeric-owner',) if config.get('numeric_ids') else ()
+
+    diff_command = (
+        (local_path, 'diff')
+        + (('--remote-path', remote_path) if remote_path else ())
+        + ('--log-json',)
+        + (('--lock-wait', str(lock_wait)) if lock_wait is not None else ())
+        + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+        + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+        + (
+            ('--patterns-from', patterns_file.name)
+            if patterns_file and diff_arguments.only_patterns
+            else ()
+        )
+        + numeric_ids_flags
+        + (('--same-chunker-params',) if diff_arguments.same_chunker_params else ())
+        + (('--sort-by', ','.join(diff_arguments.sort_keys)) if diff_arguments.sort_keys else ())
+        + (('--content-only',) if diff_arguments.content_only else ())
+        + (tuple(shlex.split(extra_borg_options)) if extra_borg_options else ())
+        + (
+            (*flags.make_repository_flags(repository, local_borg_version), archive)
+            if borgmatic.borg.feature.available(
+                borgmatic.borg.feature.Feature.SEPARATE_REPOSITORY_ARCHIVE,
+                local_borg_version,
+            )
+            else flags.make_repository_archive_flags(repository, archive, local_borg_version)
+        )
+        + (second_archive,)
+    )
+
+    borgmatic.execute.execute_command(
+        full_command=diff_command,
+        output_log_level=logging.ANSWER,
+        environment=borgmatic.borg.environment.make_environment(config),
+        working_directory=borgmatic.config.paths.get_working_directory(config),
+        borg_local_path=local_path,
+        borg_exit_codes=config.get('borg_exit_codes'),
+    )

borgmatic/borg/environment.py

@@ -4,6 +4,8 @@ import borgmatic.borg.passcommand
 import borgmatic.hooks.credential.parse
 
 OPTION_TO_ENVIRONMENT_VARIABLE = {
+    'archive_hostname': 'BORG_HOSTNAME',
+    'archive_username': 'BORG_USERNAME',
     'borg_base_directory': 'BORG_BASE_DIR',
     'borg_config_directory': 'BORG_CONFIG_DIR',
     'borg_cache_directory': 'BORG_CACHE_DIR',
@@ -25,6 +27,7 @@ DEFAULT_BOOL_OPTION_TO_ENVIRONMENT_VARIABLE = {
     'relocated_repo_access_is_ok': 'BORG_RELOCATED_REPO_ACCESS_IS_OK',
     'unknown_unencrypted_repo_access_is_ok': 'BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK',
     'use_chunks_archive': 'BORG_USE_CHUNKS_ARCHIVE',
+    'msgpack_version_check': 'BORG_MSGPACK_VERSION_CHECK',
 }
 
 
@@ -90,7 +93,8 @@ def make_environment(config):
     ) in DEFAULT_BOOL_OPTION_TO_ENVIRONMENT_VARIABLE.items():
         if os.environ.get(environment_variable_name) is None:
             value = config.get(option_name)
-            environment[environment_variable_name] = 'YES' if value else 'NO'
+            if value is not None:
+                environment[environment_variable_name] = 'YES' if value else 'NO'
 
     for (
         option_name,

borgmatic/borg/export_key.py

@@ -29,8 +29,8 @@ def export_key(
     Raise FileExistsError if a path is given but it already exists on disk.
     '''
     borgmatic.logger.add_custom_log_levels()
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     working_directory = borgmatic.config.paths.get_working_directory(config)
     extra_borg_options = config.get('extra_borg_options', {}).get('key_export', '')
 

borgmatic/borg/export_tar.py

@@ -33,8 +33,8 @@ def export_tar_archive(
     If the destination path is "-", then stream the output to stdout instead of to a file.
     '''
     borgmatic.logger.add_custom_log_levels()
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('export_tar', '')
 
     full_command = (

borgmatic/borg/extract.py

@@ -104,8 +104,8 @@ def extract_archive(
     If extract to stdout is True, then start the extraction streaming to stdout, and return that
     extract process as an instance of subprocess.Popen.
     '''
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('extract', '')
 
     if feature.available(feature.Feature.NUMERIC_IDS, local_borg_version):

borgmatic/borg/flags.py

@@ -1,5 +1,4 @@
 import itertools
-import json
 import logging
 import re
 
@@ -135,27 +134,18 @@ def make_match_archives_flags(  # noqa: PLR0911
     return ('--glob-archives', f'{derived_match_archives}')
 
 
-def warn_for_aggressive_archive_flags(json_command, json_output):
+def warn_for_aggressive_archive_flags(command, output_lines):
     '''
-    Given a JSON archives command and the resulting JSON string output from running it, parse the
-    JSON and warn if the command used an archive flag but the output indicates zero archives were
-    found.
+    Given an archives command and the resulting output lines from running it, warn if the command
+    used an archive flag but the output indicates zero archives were found.
     '''
-    archive_flags_used = {'--glob-archives', '--match-archives'}.intersection(set(json_command))
-
-    if not archive_flags_used:
-        return
-
-    try:
-        if len(json.loads(json_output)['archives']) == 0:
-            logger.warning('An archive filter was applied, but no matching archives were found.')
-            logger.warning(
-                'Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration.',
-            )
-    except json.JSONDecodeError as error:
-        logger.debug(f'Cannot parse JSON output from archive command: {error}')
-    except (TypeError, KeyError):
-        logger.debug('Cannot parse JSON output from archive command: No "archives" key found')
+    if {'--glob-archives', '--match-archives'}.intersection(set(command)) and len(
+        tuple(line for line in output_lines if not line.startswith('terminating with '))
+    ) == 0:
+        logger.warning('An archive filter was applied, but no matching archives were found.')
+        logger.warning(
+            'Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration.',
+        )
 
 
 def omit_flag(arguments, flag):

borgmatic/borg/import_key.py

@@ -27,8 +27,8 @@ def import_key(
 
     Raise ValueError if the path is given and it does not exist.
     '''
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     working_directory = borgmatic.config.paths.get_working_directory(config)
     extra_borg_options = config.get('extra_borg_options', {}).get('key_import', '')
 

borgmatic/borg/info.py

@@ -5,7 +5,7 @@ import shlex
 import borgmatic.config.paths
 import borgmatic.logger
 from borgmatic.borg import environment, feature, flags
-from borgmatic.execute import execute_command, execute_command_and_capture_output
+from borgmatic.execute import execute_command_and_capture_output
 
 logger = logging.getLogger(__name__)
 
@@ -103,9 +103,21 @@ def display_archives_info(
     borg_exit_codes = config.get('borg_exit_codes')
     working_directory = borgmatic.config.paths.get_working_directory(config)
 
-    json_info = '\n'.join(
+    if info_arguments.json:
+        return '\n'.join(
+            execute_command_and_capture_output(
+                json_command,
+                environment=environment.make_environment(config),
+                working_directory=working_directory,
+                borg_local_path=local_path,
+                borg_exit_codes=borg_exit_codes,
+            )
+        )
+
+    output_lines = tuple(
         execute_command_and_capture_output(
-            json_command,
+            main_command,
+            output_log_level=logging.ANSWER,
             environment=environment.make_environment(config),
             working_directory=working_directory,
             borg_local_path=local_path,
@@ -113,18 +125,6 @@ def display_archives_info(
         )
     )
 
-    if info_arguments.json:
-        return json_info
-
-    flags.warn_for_aggressive_archive_flags(json_command, json_info)
-
-    execute_command(
-        main_command,
-        output_log_level=logging.ANSWER,
-        environment=environment.make_environment(config),
-        working_directory=working_directory,
-        borg_local_path=local_path,
-        borg_exit_codes=borg_exit_codes,
-    )
+    flags.warn_for_aggressive_archive_flags(main_command, output_lines)
 
     return None

borgmatic/borg/list.py

@@ -115,10 +115,10 @@ def capture_archive_listing(
     Given a local or remote repository path, an archive name, a configuration dict, the local Borg
     version, global arguments as an argparse.Namespace, the archive paths (or Borg patterns) in
     which to list files, the Borg path format indicating keys to include in the output, and local
-    and remote Borg paths, capture the output of listing that archive and return it as a sequence of
-    dicts, one per path.
+    and remote Borg paths, capture the output of listing that archive and return it as a generator
+    of dicts, one per path.
     '''
-    return tuple(
+    return (
         json.loads(entry)
         for entry in execute_command_and_capture_output(
             make_list_command(

borgmatic/borg/mount.py

@@ -24,8 +24,8 @@ def mount_archive(
     dict, the local Borg version, global arguments as an argparse.Namespace instance, and optional
     local and remote Borg paths, mount the archive onto the mount point.
     '''
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('mount', '')
 
     full_command = (

borgmatic/borg/pattern.py

@@ -88,16 +88,16 @@ def write_patterns_file(patterns, borgmatic_runtime_directory, patterns_file=Non
     return patterns_file
 
 
-def check_all_root_patterns_exist(patterns):
+def check_all_root_patterns_exist(patterns, working_directory):
     '''
-    Given a sequence of Pattern instances, check that all root pattern paths exist. If any don't,
-    raise an exception.
+    Given a sequence of Pattern instances and the current working directory, check that all root
+    pattern paths exist. If any don't, raise an exception.
     '''
     missing_paths = [
         pattern.path
         for pattern in patterns
         if pattern.type == Pattern_type.ROOT
-        if not os.path.exists(pattern.path)
+        if not os.path.exists(os.path.join(working_directory or '', pattern.path))
     ]
 
     if missing_paths:

borgmatic/borg/prune.py

@@ -65,8 +65,8 @@ def prune_archives(
     archives according to the retention policy specified in that configuration.
     '''
     borgmatic.logger.add_custom_log_levels()
-    umask = config.get('umask', None)
-    lock_wait = config.get('lock_wait', None)
+    umask = config.get('umask')
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get('prune', '')
 
     full_command = (
@@ -83,10 +83,23 @@ def prune_archives(
             and not feature.available(feature.Feature.NO_PRUNE_STATS, local_borg_version)
             else ()
         )
+        + (
+            ('--quick-stats',)
+            if config.get('quick_statistics')
+            and not dry_run
+            and not feature.available(feature.Feature.NO_PRUNE_STATS, local_borg_version)
+            else ()
+        )
         + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
         + flags.make_flags_from_arguments(
             prune_arguments,
-            excludes=('repository', 'match_archives', 'statistics', 'list_details'),
+            excludes=(
+                'repository',
+                'match_archives',
+                'statistics',
+                'quick_statistics',
+                'list_details',
+            ),
         )
         + (('--list',) if config.get('list_details') else ())
         + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
@@ -95,7 +108,7 @@ def prune_archives(
         + flags.make_repository_flags(repository_path, local_borg_version)
     )
 
-    if config.get('statistics') or config.get('list_details'):
+    if config.get('statistics') or config.get('quick_statistics') or config.get('list_details'):
         output_log_level = logging.ANSWER
     else:
         output_log_level = logging.INFO

borgmatic/borg/recreate.py

@@ -28,14 +28,14 @@ def recreate_archive(
     arguments, optional local and remote Borg paths, executes the recreate command with the given
     arguments.
     '''
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     exclude_flags = flags.make_exclude_flags(config)
-    compression = config.get('compression', None)
-    chunker_params = config.get('chunker_params', None)
+    compression = config.get('compression')
+    chunker_params = config.get('chunker_params')
     extra_borg_options = config.get('extra_borg_options', {}).get('recreate', '')
 
     # Available recompress MODES: "if-different", "always", "never" (default)
-    recompress = config.get('recompress', None)
+    recompress = config.get('recompress')
 
     # Write patterns to a temporary file and use that file with --patterns-from.
     patterns_file = write_patterns_file(
@@ -72,6 +72,7 @@ def recreate_archive(
         + (('--chunker-params', chunker_params) if chunker_params else ())
         + (('--recompress', recompress) if recompress else ())
         + exclude_flags
+        + (('--dry-run',) if global_arguments.dry_run else ())
         + (tuple(shlex.split(extra_borg_options)) if extra_borg_options else ())
         + (
             (
@@ -94,10 +95,6 @@ def recreate_archive(
         )
     )
 
-    if global_arguments.dry_run:
-        logger.info('Skipping the archive recreation (dry run)')
-        return
-
     borgmatic.execute.execute_command(
         full_command=recreate_command,
         output_log_level=logging.INFO,

borgmatic/borg/repo_create.py

@@ -5,6 +5,7 @@ import shlex
 import subprocess
 
 import borgmatic.config.paths
+import borgmatic.logger
 from borgmatic.borg import environment, feature, flags, repo_info
 from borgmatic.execute import DO_NOT_CAPTURE, execute_command
 
@@ -40,17 +41,21 @@ def create_repository(
     Raise subprocess.CalledProcessError if "borg info" returns an error exit code.
     '''
     try:
-        info_data = json.loads(
-            repo_info.display_repository_info(
-                repository_path,
-                config,
-                local_borg_version,
-                argparse.Namespace(json=True),
-                global_arguments,
-                local_path,
-                remote_path,
-            ),
-        )
+        # Suppress Borg's "repository does not exist" error log, so the user isn't confused by
+        # seeing an error during successful repository creation.
+        with borgmatic.logger.Logs_suppressed(msgid='Repository.DoesNotExist'):
+            info_data = json.loads(
+                repo_info.display_repository_info(
+                    repository_path,
+                    config,
+                    local_borg_version,
+                    argparse.Namespace(json=True),
+                    global_arguments,
+                    local_path,
+                    remote_path,
+                ),
+            )
+
         repository_encryption_mode = info_data.get('encryption', {}).get('mode')
 
         if repository_encryption_mode != encryption_mode:

borgmatic/borg/repo_info.py

@@ -24,7 +24,7 @@ def display_repository_info(
     information for the Borg repository or return JSON summary information.
     '''
     borgmatic.logger.add_custom_log_levels()
-    lock_wait = config.get('lock_wait', None)
+    lock_wait = config.get('lock_wait')
     extra_borg_options = config.get('extra_borg_options', {}).get(
         'repo_info' if feature.available(feature.Feature.REPO_INFO, local_borg_version) else 'info',
         '',
@@ -37,7 +37,6 @@ def display_repository_info(
             if feature.available(feature.Feature.REPO_INFO, local_borg_version)
             else ('info',)
         )
-        + (('--critical',) if repo_info_arguments.json else ())
         + (
             ('--info',)
             if logger.getEffectiveLevel() == logging.INFO and not repo_info_arguments.json

borgmatic/borg/repo_list.py

@@ -6,7 +6,7 @@ import shlex
 import borgmatic.config.paths
 import borgmatic.logger
 from borgmatic.borg import environment, feature, flags
-from borgmatic.execute import execute_command, execute_command_and_capture_output
+from borgmatic.execute import execute_command_and_capture_output
 
 logger = logging.getLogger(__name__)
 
@@ -79,6 +79,13 @@ def get_latest_archive(
         *flags.make_flags('umask', config.get('umask')),
         *('--log-json',),
         *flags.make_flags('lock-wait', config.get('lock_wait')),
+        *(
+            flags.make_match_archives_flags(
+                config.get('match_archives'),
+                config.get('archive_name_format'),
+                local_borg_version,
+            )
+        ),
         *(
             flags.make_flags('consider-checkpoints', consider_checkpoints)
             if not feature.available(feature.Feature.REPO_LIST, local_borg_version)
@@ -219,9 +226,21 @@ def list_repository(
     working_directory = borgmatic.config.paths.get_working_directory(config)
     borg_exit_codes = config.get('borg_exit_codes')
 
-    json_listing = '\n'.join(
+    if repo_list_arguments.json:
+        return '\n'.join(
+            execute_command_and_capture_output(
+                json_command,
+                environment=environment.make_environment(config),
+                working_directory=working_directory,
+                borg_local_path=local_path,
+                borg_exit_codes=borg_exit_codes,
+            )
+        )
+
+    output_lines = tuple(
         execute_command_and_capture_output(
-            json_command,
+            main_command,
+            output_log_level=logging.ANSWER,
             environment=environment.make_environment(config),
             working_directory=working_directory,
             borg_local_path=local_path,
@@ -229,18 +248,6 @@ def list_repository(
         )
     )
 
-    if repo_list_arguments.json:
-        return json_listing
-
-    flags.warn_for_aggressive_archive_flags(json_command, json_listing)
-
-    execute_command(
-        main_command,
-        output_log_level=logging.ANSWER,
-        environment=environment.make_environment(config),
-        working_directory=working_directory,
-        borg_local_path=local_path,
-        borg_exit_codes=borg_exit_codes,
-    )
+    flags.warn_for_aggressive_archive_flags(main_command, output_lines)
 
     return None

borgmatic/commands/arguments.py

@@ -31,8 +31,10 @@ ACTION_ALIASES = {
     'transfer': [],
     'break-lock': [],
     'key': [],
-    'borg': [],
     'recreate': [],
+    'diff': [],
+    'browse': [],
+    'borg': [],
 }
 
 
@@ -323,7 +325,7 @@ def make_argument_description(schema, flag_name):
             ' To specify a different list element, replace the "[0]" with another array index ("[1]", "[2]", etc.).',
         )
 
-    if example and schema_type in ('array', 'object'):  # noqa: PLR6201
+    if example and schema_type in ('array', 'object'):
         example_buffer = io.StringIO()
         yaml = ruamel.yaml.YAML(typ='safe')
         yaml.default_flow_style = True
@@ -803,6 +805,13 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         action='store_true',
         help='Display statistics of the pruned archive [Borg 1 only]',
     )
+    prune_group.add_argument(
+        '--quick-stats',
+        dest='quick_statistics',
+        default=None,
+        action='store_true',
+        help='Display statistics of the pruned archive, skipping repository-wide "All archives" and chunk index statistics [Borg >= 1.4.5 and < 2 only]',
+    )
     prune_group.add_argument(
         '--list',
         dest='list_details',
@@ -895,6 +904,13 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         action='store_true',
         help='Display statistics of archive',
     )
+    create_group.add_argument(
+        '--quick-stats',
+        dest='quick_statistics',
+        default=None,
+        action='store_true',
+        help='Display statistics of archive, skipping repository-wide "All archives" and chunk index statistics [Borg 1.4.5+ only]',
+    )
     create_group.add_argument(
         '--list',
         '--files',
@@ -1261,6 +1277,29 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         help='Show this help message and exit',
     )
 
+    config_show_parser = config_parsers.add_parser(
+        'show',
+        help='Show the computed configuration for each file specified with --config (see borgmatic --help)',
+        description='Show the computed configuration for each file specified with --config (see borgmatic --help)',
+        add_help=False,
+    )
+    config_show_group = config_show_parser.add_argument_group('config show arguments')
+    config_show_group.add_argument(
+        '--option',
+        help='Show the value of a single named configuration option instead of the entire configuration',
+    )
+    config_show_group.add_argument(
+        '--json',
+        action='store_true',
+        help='Show the configuration as JSON with one array element per configuration file',
+    )
+    config_show_group.add_argument(
+        '-h',
+        '--help',
+        action='help',
+        help='Show this help message and exit',
+    )
+
     export_tar_parser = action_parsers.add_parser(
         'export-tar',
         aliases=ACTION_ALIASES['export-tar'],
@@ -1931,7 +1970,7 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
     )
     recreate_group.add_argument(
         '--archive',
-        help='Archive name, hash, or series to recreate',
+        help='Archive name, hash, or series to recreate, defaults to all archives in the repository (if specified), or all archives across all repositories',
     )
     recreate_group.add_argument(
         '--list',
@@ -1970,6 +2009,60 @@ def make_parsers(schema, unparsed_arguments):  # noqa: PLR0915
         help='Show this help message and exit',
     )
 
+    diff_parser = action_parsers.add_parser(
+        'diff',
+        aliases=ACTION_ALIASES['diff'],
+        help='Find differences (file contents, user/group/mode) between archives',
+        description='Find differences (file contents, user/group/mode) between archives',
+        add_help=False,
+    )
+    diff_group = diff_parser.add_argument_group('diff arguments')
+    diff_group.add_argument(
+        '--repository',
+        help='Path of repository containing archive to diff, defaults to the configured repository if there is only one, quoted globs supported',
+    )
+    diff_group.add_argument(
+        '--archive',
+        help='Archive name, hash, or series to diff',
+        required=True,
+    )
+    diff_group.add_argument(
+        '--second-archive',
+        help='Second archive name, hash, or series to diff',
+        required=True,
+    )
+    diff_group.add_argument(
+        '--same-chunker-params', action='store_true', help='Override check of chunker parameters'
+    )
+    diff_group.add_argument(
+        '--sort-by',
+        metavar='KEY',
+        dest='sort_keys',
+        action='append',
+        help='Advanced sorting: specify field(s) to sort by. Prefix with > for descending or < for ascending (default)',
+    )
+    diff_group.add_argument(
+        '--content-only',
+        action='store_true',
+        help='Only compare differences in content (exclude metadata differences)',
+    )
+    diff_group.add_argument(
+        '--only-patterns',
+        action='store_true',
+        help='Run the diff according to borgmatic configured patterns (ie do not diff entire archives)',
+    )
+    diff_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
+
+    browse_parser = action_parsers.add_parser(
+        'browse',
+        aliases=ACTION_ALIASES['browse'],
+        help='Browse repositories, archives, and files in a console UI',
+        description='Browse repositories, archives, and files in a console UI',
+        add_help=False,
+    )
+    browse_group = browse_parser.add_argument_group('browse arguments')
+    browse_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
+
     borg_parser = action_parsers.add_parser(
         'borg',
         aliases=ACTION_ALIASES['borg'],
@@ -2042,7 +2135,7 @@ def parse_arguments(schema, *unparsed_arguments):
         )
 
     if (
-        ('list' in arguments and 'repo-info' in arguments and arguments['list'].json)  # noqa: PLR0916
+        ('list' in arguments and 'repo-info' in arguments and arguments['list'].json)
         or ('list' in arguments and 'info' in arguments and arguments['list'].json)
         or ('repo-info' in arguments and 'info' in arguments and arguments['repo-info'].json)
     ):
@@ -2060,7 +2153,7 @@ def parse_arguments(schema, *unparsed_arguments):
             'With the repo-list action, only one of --prefix or --match-archives flags can be used.',
         )
 
-    if 'info' in arguments and (  # noqa: PLR0916
+    if 'info' in arguments and (
         (arguments['info'].archive and arguments['info'].prefix)
         or (arguments['info'].archive and arguments['info'].match_archives)
         or (arguments['info'].prefix and arguments['info'].match_archives)

borgmatic/commands/borgmatic.py

@@ -12,14 +12,17 @@ import ruamel.yaml
 
 import borgmatic.actions.borg
 import borgmatic.actions.break_lock
+import borgmatic.actions.browse.run
 import borgmatic.actions.change_passphrase
 import borgmatic.actions.check
 import borgmatic.actions.compact
 import borgmatic.actions.config.bootstrap
 import borgmatic.actions.config.generate
+import borgmatic.actions.config.show
 import borgmatic.actions.config.validate
 import borgmatic.actions.create
 import borgmatic.actions.delete
+import borgmatic.actions.diff
 import borgmatic.actions.export_key
 import borgmatic.actions.export_tar
 import borgmatic.actions.extract
@@ -209,7 +212,7 @@ def run_configuration(config_filename, config, config_paths, arguments):  # noqa
             f"Skipping {'/'.join(skip_actions)} action{'s' if len(skip_actions) > 1 else ''} due to configured skip_actions",
         )
 
-    try:  # noqa: PLR1702
+    try:
         with (
             Monitoring_hooks(config_filename, config, arguments, global_arguments),
             borgmatic.hooks.command.Before_after_hooks(
@@ -441,6 +444,7 @@ def run_actions(  # noqa: PLR0912, PLR0915
                         local_borg_version,
                         action_arguments,
                         global_arguments,
+                        dry_run_label,
                         local_path,
                         remote_path,
                     )
@@ -621,6 +625,16 @@ def run_actions(  # noqa: PLR0912, PLR0915
                         local_path,
                         remote_path,
                     )
+                elif action_name == 'diff':
+                    borgmatic.actions.diff.run_diff(
+                        repository,
+                        config,
+                        local_borg_version,
+                        action_arguments,
+                        global_arguments,
+                        local_path,
+                        remote_path,
+                    )
                 elif action_name == 'borg':
                     borgmatic.actions.borg.run_borg(
                         repository,
@@ -809,18 +823,18 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
     '''
     add_custom_log_levels()
 
-    if 'bootstrap' in arguments:
-        try:
-            # No configuration file is needed for bootstrap.
-            local_borg_version = borg_version.local_borg_version(
-                {},
-                arguments['bootstrap'].local_path,
-            )
-        except (OSError, CalledProcessError, ValueError) as error:
-            yield from log_error_records('Error getting local Borg version', error)
-            return
+    try:
+        if 'bootstrap' in arguments:
+            try:
+                local_borg_version = borg_version.local_borg_version(
+                    # No configuration file is needed for bootstrap.
+                    {},
+                    arguments['bootstrap'].local_path,
+                )
+            except (OSError, CalledProcessError, ValueError) as error:
+                yield from log_error_records('Error getting local Borg version', error)
+                return
 
-        try:
             borgmatic.actions.config.bootstrap.run_bootstrap(
                 arguments['bootstrap'],
                 arguments['global'],
@@ -834,17 +848,10 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
                     name=logger.name,
                 ),
             )
-        except (
-            CalledProcessError,
-            ValueError,
-            OSError,
-        ) as error:
-            yield from log_error_records(error)
 
-        return
+            return
 
-    if 'generate' in arguments:
-        try:
+        if 'generate' in arguments:
             borgmatic.actions.config.generate.run_generate(
                 arguments['generate'],
                 arguments['global'],
@@ -857,29 +864,22 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
                     name=logger.name,
                 ),
             )
-        except (
-            CalledProcessError,
-            ValueError,
-            OSError,
-        ) as error:
-            yield from log_error_records(error)
-
-        return
-
-    if 'validate' in arguments:
-        if configuration_parse_errors:
-            yield logging.makeLogRecord(
-                dict(
-                    levelno=logging.CRITICAL,
-                    levelname='CRITICAL',
-                    msg='Configuration validation failed',
-                    name=logger.name,
-                ),
-            )
 
             return
 
-        try:
+        if 'validate' in arguments:
+            if configuration_parse_errors:
+                yield logging.makeLogRecord(
+                    dict(
+                        levelno=logging.CRITICAL,
+                        levelname='CRITICAL',
+                        msg='Configuration validation failed',
+                        name=logger.name,
+                    ),
+                )
+
+                return
+
             borgmatic.actions.config.validate.run_validate(arguments['validate'], configs)
 
             yield logging.makeLogRecord(
@@ -890,14 +890,27 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
                     name=logger.name,
                 ),
             )
-        except (
-            CalledProcessError,
-            ValueError,
-            OSError,
-        ) as error:
-            yield from log_error_records(error)
 
-        return
+            return
+
+        if 'show' in arguments:
+            borgmatic.actions.config.show.run_show(arguments['show'], configs)
+
+            return
+
+        if 'browse' in arguments:
+            borgmatic.actions.browse.run.run_browse(
+                arguments['browse'],
+                arguments['global'],
+                configs,
+            )
+
+    except (
+        CalledProcessError,
+        ValueError,
+        OSError,
+    ) as error:
+        yield from log_error_records(error)
 
 
 def collect_configuration_run_summary_logs(configs, config_paths, arguments, log_file_path):  # noqa: PLR0912
@@ -1039,16 +1052,17 @@ def exit_with_help_link():  # pragma: no cover
     sys.exit(1)
 
 
-def check_and_show_help_on_no_args(configs):
+def check_and_show_help_on_no_args(configs, schema):
     '''
-    Given a dict of configuration filename to corresponding parsed configuration, check if the
-    borgmatic command is run without any arguments. If the configuration option "default_actions" is
-    set to False, show the help message. Otherwise, trigger the default backup behavior.
+    Given a dict of configuration filename to corresponding parsed configuration and the
+    configuration schema as a dict, check if the borgmatic command was run without any arguments. If
+    the configuration option "default_actions" is set to False, then show the help message an exit.
     '''
-    if len(sys.argv) == 1:  # No arguments provided
+    if len(sys.argv) == 1:  # No arguments provided.
         default_actions = any(config.get('default_actions', True) for config in configs.values())
-        if not default_actions:
-            parse_arguments('--help')
+
+        if configs and not default_actions:
+            parse_arguments(schema, '--help')
             sys.exit(0)
 
 
@@ -1117,7 +1131,7 @@ def main(extra_summary_logs=()):  # pragma: no cover
         exit_with_help_link()
     except SystemExit as error:
         if error.code == 0:
-            raise error
+            raise
 
         configure_logging(logging.CRITICAL)
         logger.critical(f"Error parsing arguments: {' '.join(sys.argv)}")
@@ -1145,8 +1159,7 @@ def main(extra_summary_logs=()):  # pragma: no cover
         resolve_env=global_arguments.resolve_env and not arguments.get('validate'),
     )
 
-    # Use the helper function to check and show help on no arguments, passing the preloaded configs
-    check_and_show_help_on_no_args(configs)
+    check_and_show_help_on_no_args(configs, schema)
 
     configuration_parse_errors = (
         (max(log.levelno for log in parse_logs) >= logging.CRITICAL) if parse_logs else False
@@ -1197,3 +1210,7 @@ def main(extra_summary_logs=()):  # pragma: no cover
     )
 
     display_summary(summary_logs, log_json)
+
+
+if __name__ == '__main__':  # pragma: no cover
+    main()

borgmatic/commands/completion/bash.py

@@ -39,7 +39,7 @@ def bash_completion():
             'check_version() {',
             '    local this_script="$(cat "$BASH_SOURCE" 2> /dev/null)"',
             '    local installed_script="$(borgmatic --bash-completion 2> /dev/null)"',
-            '    if [ "$this_script" != "$installed_script" ] && [ "$installed_script" != "" ];'
+            '    if [ "$this_script" != "$installed_script" ] && [ "$installed_script" != "" ];',
             f'''        then cat << EOF\n{borgmatic.commands.completion.actions.upgrade_message(
                     'bash',
                     'sudo sh -c "borgmatic --bash-completion > $BASH_SOURCE"',

borgmatic/config/collect.py

@@ -47,5 +47,6 @@ def collect_config_filenames(config_paths):
         for filename in sorted(os.listdir(path)):
             full_filename = os.path.join(path, filename)
             matching_filetype = full_filename.endswith(('.yaml', '.yml'))
+
             if matching_filetype and not os.path.isdir(full_filename):
                 yield os.path.abspath(full_filename)

borgmatic/config/generate.py

@@ -153,6 +153,9 @@ def transform_optional_configuration(rendered_config, comment_out=True):
     return '\n'.join(lines)
 
 
+RUAMEL_YAML_END_OF_DOCUMENT_MARKER = '...\n'
+
+
 def render_configuration(config):
     '''
     Given a config data structure of nested OrderedDicts, render the config as YAML and return it.
@@ -160,7 +163,13 @@ def render_configuration(config):
     dumper = ruamel.yaml.YAML(typ='rt')
     dumper.indent(mapping=INDENT, sequence=INDENT + SEQUENCE_INDENT, offset=INDENT)
     rendered = io.StringIO()
-    dumper.dump(config, rendered)
+    dumper.dump(
+        config,
+        rendered,
+        # Dumping certain values (integers, for instance) causes ruamel.yaml to append an
+        # end-of-document "..." marker. Strip it.
+        transform=lambda dumped: dumped.removesuffix(RUAMEL_YAML_END_OF_DOCUMENT_MARKER),
+    )
 
     return rendered.getvalue()
 

borgmatic/config/schema.yaml

@@ -30,12 +30,10 @@ properties:
     source_directories_must_exist:
         type: boolean
         description: |
-            Deprecated. Replaced by borgmatic treating Borg's "backup file not
-            found" warning as an error by default. But if
-            "source_directories_must_exist" is true, then source directories
-            (and root pattern paths) must exist before a backup begins. If they
-            don't, borgmatic errors. Defaults to false.
-        example: true
+            When true, source directories (and root pattern paths) must exist
+            before a backup begins. If they don't, borgmatic errors. Defaults to
+            true.
+        example: false
     repositories:
         type: array
         items:
@@ -166,6 +164,16 @@ properties:
             Record filesystem flags (e.g. NODUMP, IMMUTABLE) in archive.
             Defaults to true.
         example: false
+    files_changed:
+        type: string
+        enum: ['ctime', 'mtime', 'disabled']
+        description: |
+            Threshold for considering a file as changed. See
+            https://borgbackup.readthedocs.io/en/stable/usage/create.html for
+            details. Defaults to "ctime". E.g., a file is considered changed if
+            its ctime has changed since the last backup. (This option is
+            supported for Borg 1.4.2+ only.)
+        example: ctime
     files_cache:
         type: string
         description: |
@@ -481,6 +489,9 @@ properties:
         description: |
             Umask used for when executing Borg or calling hooks. Defaults to
             0077 for Borg or the umask that borgmatic is run with for hooks.
+            Even though this value is a YAML integer, borgmatic interprets it as
+            octal. YAML's "0o"-prefixed octal notation is not currently
+            supported.
         example: 0077
     lock_wait:
         type: integer
@@ -510,6 +521,21 @@ properties:
             If match_archives is not specified, borgmatic defaults to deriving
             the match_archives value from archive_name_format.
         example: "sh:{hostname}-*"
+    archive_hostname:
+        type: string
+        description: |
+            Hostname to use for the "{hostname}" placeholder in
+            "archive_name_format", "match_archives", etc. Defaults to the system
+            hostname. (This option is supported for Borg 1.4.5+ only.)
+        example: example.org
+    archive_username:
+        type: string
+        description: |
+            Username to use for the "{user}" placeholder in
+            "archive_name_format", "match_archives", etc. Defaults to the
+            username of the user running borgmatic. (This option is supported
+            for Borg 1.4.5+ only.)
+        example: backup_user
     file_list_format:
         type: string
         description: |
@@ -562,6 +588,12 @@ properties:
             Bypass Borg confirmation about check with repair option. Defaults to
             false and an interactive prompt from Borg.
         example: true
+    msgpack_version_check:
+        type: boolean
+        description: |
+            Optionally disable the msgpack version check. Default is true; use
+            at your own risk. (This option is supported for Borg 1.4.2+ only.)
+        example: false
     extra_borg_options:
         type: object
         additionalProperties: false
@@ -1074,6 +1106,16 @@ properties:
             Corresponds to the "--stats" flag on those actions. Defaults to
             false.
         example: true
+    quick_statistics:
+        type: boolean
+        description: |
+            Display statistics for an archive when running supported actions,
+            skipping the repository-wide "All archives" and chunk index
+            statistics to save some time. Corresponds to the "--quick-stats"
+            flag on those actions. Defaults to false. (This option is supported
+            for Borg 1.4.5+ only for the "create" action and for Borg >= 1.4.5
+            and < Borg 2 for the "prune" action.)
+        example: true
     list_details:
         type: boolean
         description: |
@@ -1084,9 +1126,9 @@ properties:
     default_actions:
         type: boolean
         description: |
-            Whether to apply default actions (create, prune, compact and check)
-            when no arguments are supplied to the borgmatic command. If set to
-            false, borgmatic displays the help message instead.
+            Whether to run default actions (create, prune, compact, and check)
+            when no arguments are given on the command line. If set to false,
+            borgmatic displays the help message instead.
         example: true
     skip_actions:
         type: array
@@ -1114,6 +1156,8 @@ properties:
                 - info
                 - break-lock
                 - key
+                - diff
+                - browse
                 - borg
         description: |
             List of one or more actions to skip running for this configuration
@@ -1330,6 +1374,8 @@ properties:
                                   - info
                                   - break-lock
                                   - key
+                                  - diff
+                                  - browse
                                   - borg
                           description: |
                               List of actions for which the commands will be
@@ -1395,6 +1441,8 @@ properties:
                                   - info
                                   - break-lock
                                   - key
+                                  - diff
+                                  - browse
                                   - borg
                           description: |
                               Only trigger the hook when borgmatic is run with
@@ -1809,6 +1857,27 @@ properties:
                         client and restore server. The default varies based on
                         the MariaDB version.
                     example: false
+                events:
+                    type: boolean
+                    description: |
+                        Whether to include scheduled events within the dump.
+                        Disable if your database user doesn't have the
+                        permissions to dump events. Defaults to true.
+                    example: false
+                routines:
+                    type: boolean
+                    description: |
+                        Whether to include stored routines within the dump.
+                        Disable if your user database doesn't have the
+                        permissions to dump routines. Defaults to true.
+                    example: false
+                tablespaces:
+                    type: boolean
+                    description: |
+                        Whether to include tablespaces within the dump. Disable
+                        if your database user doesn't have the permissions to
+                        dump tablespaces. Defaults to true.
+                    example: false
                 mariadb_dump_command:
                     type: string
                     description: |
@@ -2017,6 +2086,27 @@ properties:
                         client and restore server. The default varies based on
                         the MySQL installation.
                     example: false
+                events:
+                    type: boolean
+                    description: |
+                        Whether to include scheduled events within the dump.
+                        Disable if your database user doesn't have the
+                        permissions to dump events. Defaults to true.
+                    example: false
+                routines:
+                    type: boolean
+                    description: |
+                        Whether to include stored routines within the dump.
+                        Disable if your database user doesn't have the
+                        permissions to dump routines. Defaults to true.
+                    example: false
+                tablespaces:
+                    type: boolean
+                    description: |
+                        Whether to include tablespaces within the dump. Disable
+                        if your database user doesn't have the permissions to
+                        dump tablespaces. Defaults to true.
+                    example: false
                 mysql_dump_command:
                     type: string
                     description: |
@@ -2342,6 +2432,13 @@ properties:
                         example: Your backups have started.
                     priority:
                         type: string
+                        enum:
+                            - max
+                            - urgent
+                            - high
+                            - default
+                            - low
+                            - min
                         description: |
                             The priority to set.
                         example: min
@@ -2366,6 +2463,13 @@ properties:
                         example: Your backups have finished.
                     priority:
                         type: string
+                        enum:
+                            - max
+                            - urgent
+                            - high
+                            - default
+                            - low
+                            - min
                         description: |
                             The priority to set.
                         example: min
@@ -2390,6 +2494,13 @@ properties:
                         example: Your backups have failed.
                     priority:
                         type: string
+                        enum:
+                            - max
+                            - urgent
+                            - high
+                            - default
+                            - low
+                            - min
                         description: |
                             The priority to set.
                         example: max
@@ -2784,7 +2895,9 @@ properties:
                     properties:
                         url:
                             type: string
-                            description: URL of this Apprise service.
+                            description: |
+                                URL of this Apprise service. Supports the
+                                "{credential ...}" syntax.
                             example: "gotify://hostname/token"
                         label:
                             type: string
@@ -3060,6 +3173,24 @@ properties:
                     Grafana Loki log URL to notify when a backup begins,
                     ends, or fails.
                 example: "http://localhost:3100/loki/api/v1/push"
+            tls:
+                type: object
+                additionalProperties: false
+                properties:
+                    cert_path:
+                        type: string
+                        description: |
+                            Path to a PEM client certificate file for mutual
+                            TLS authentication.
+                        example: /etc/borgmatic/loki-client.crt
+                    key_path:
+                        type: string
+                        description: |
+                            Path to a PEM private key file for the client
+                            certificate.
+                        example: /etc/borgmatic/loki-client.key
+                description: |
+                    TLS options for mutual TLS (mTLS) authentication with Loki.
             labels:
                 type: object
                 additionalProperties:
@@ -3253,24 +3384,32 @@ properties:
                 description: |
                     Command to use instead of "keepassxc-cli".
                 example: /usr/local/bin/keepassxc-cli
+            secret_tool_command:
+                type: string
+                description: |
+                    Command to use instead of "secret-tool".
+                example: /usr/local/bin/secret-tool
             ask_for_password:
                 type: boolean
                 description: |
                     Whether keepassxc-cli should prompt the user for a password.
                     Disabling this is only really useful if you're unlocking
                     your KeePassXC database with a key file instead of a
-                    password. Defaults to true.
+                    password. Ignored when using KeePassXC's secret service
+                    integration. Defaults to true.
                 example: false
             key_file:
                 type: string
                 description: |
                     Path to a key file for unlocking the KeePassXC database.
+                    Ignored when using KeePassXC's secret service integration.
                 example: /path/to/keyfile
             yubikey:
                 type: string
                 description: |
                     YubiKey slot and optional serial number used to access the
-                    KeePassXC database. The format is "<slot[:serial]>", where:
+                    KeePassXC database. Ignored when using KeePassXC's secret
+                    service integration. The format is "<slot[:serial]>", where:
                      * <slot> is the YubiKey slot number (e.g., `1` or `2`).
                      * <serial> (optional) is the YubiKey's serial number (e.g.,
                        `7370001`).

borgmatic/execute.py

@@ -2,7 +2,9 @@ import collections
 import contextlib
 import enum
 import json
+import locale
 import logging
+import os
 import select
 import subprocess
 import textwrap
@@ -18,7 +20,7 @@ BORG_ERROR_EXIT_CODE_START = 2
 BORG_ERROR_EXIT_CODE_END = 99
 
 # See https://borgbackup.readthedocs.io/en/stable/internals/frontends.html#message-ids
-BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS = {101, 102, 104, 105, 106, 107}
+BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS = {101, 102, 104, 105, 106}
 
 
 class Exit_status(enum.Enum):
@@ -28,6 +30,31 @@ class Exit_status(enum.Enum):
     ERROR = 4
 
 
+def command_is_borg(command, borg_local_path):
+    '''
+    Given a command as a sequence and the Borg local path, return whether that command is a call to
+    Borg.
+    '''
+    parsed_command = command.split(' ', 1) if isinstance(command, str) else command
+
+    if not parsed_command:
+        return False
+
+    return bool(borg_local_path and parsed_command[0] == borg_local_path)
+
+
+BORG_EXIT_CODE_TO_DESCRIPTION = {
+    100: 'File changed while we backed it up',
+    101: 'Include pattern never matched',
+    102: 'General backup issue',
+    103: 'File type or inode changed while we backed it up',
+    104: 'Backup OS issue',
+    105: 'Backup permission issue',
+    106: 'Backup IO issue',
+    107: 'Backup file not found',
+}
+
+
 def interpret_exit_code(command, exit_code, borg_local_path=None, borg_exit_codes=None):  # noqa: PLR0911
     '''
     Return an Exit_status value (e.g. SUCCESS, ERROR, or WARNING) based on interpreting the given
@@ -41,46 +68,47 @@ def interpret_exit_code(command, exit_code, borg_local_path=None, borg_exit_code
     if exit_code == 0:
         return Exit_status.SUCCESS
 
-    if borg_local_path and command[0] == borg_local_path:
-        # First try looking for the exit code in the borg_exit_codes configuration.
-        for entry in borg_exit_codes or ():
-            if entry.get('code') == exit_code:
-                treat_as = entry.get('treat_as')
+    if not command_is_borg(command, borg_local_path):
+        return Exit_status.ERROR
 
-                if treat_as == 'error':
-                    logger.error(
-                        f'Treating exit code {exit_code} as an error, as per configuration',
-                    )
-                    return Exit_status.ERROR
+    description = BORG_EXIT_CODE_TO_DESCRIPTION.get(exit_code)
+    description_parenthetical = f' ({description})' if description else ''
 
-                if treat_as == 'warning':
-                    logger.warning(
-                        f'Treating exit code {exit_code} as a warning, as per configuration',
-                    )
-                    return Exit_status.WARNING
+    # First try looking for the exit code in the borg_exit_codes configuration.
+    for entry in borg_exit_codes or ():
+        if entry.get('code') == exit_code:
+            treat_as = entry.get('treat_as')
 
-        # If the exit code doesn't have explicit configuration, then fall back to the default
-        # behavior of treating Borg errors as errors and some Borg warnings as errors.
-        if exit_code in BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS:
-            logger.error(
-                f'Treating exit code {exit_code} as an error, as per borgmatic defaults',
-            )
-
-            return Exit_status.ERROR
-
-        return (
-            Exit_status.ERROR
-            if (
-                exit_code < 0
-                or (
-                    exit_code >= BORG_ERROR_EXIT_CODE_START
-                    and exit_code <= BORG_ERROR_EXIT_CODE_END
+            if treat_as == 'error':
+                logger.error(
+                    f'Treating exit code {exit_code}{description_parenthetical} as an error, as per configuration',
                 )
-            )
-            else Exit_status.WARNING
+                return Exit_status.ERROR
+
+            if treat_as == 'warning':
+                logger.warning(
+                    f'Treating exit code {exit_code}{description_parenthetical} as a warning, as per configuration',
+                )
+                return Exit_status.WARNING
+
+    # If the exit code doesn't have explicit configuration, then fall back to the default
+    # behavior of treating Borg errors as errors and some Borg warnings as errors.
+    if exit_code in BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS:
+        logger.error(
+            f'Treating exit code {exit_code}{description_parenthetical} as an error, as per borgmatic defaults',
         )
 
-    return Exit_status.ERROR
+        return Exit_status.ERROR
+
+    if exit_code < 0 or (
+        exit_code >= BORG_ERROR_EXIT_CODE_START and exit_code <= BORG_ERROR_EXIT_CODE_END
+    ):
+        return Exit_status.ERROR
+
+    logger.warning(
+        f'Treating exit code {exit_code}{description_parenthetical} as a warning, as per borgmatic defaults',
+    )
+    return Exit_status.WARNING
 
 
 def command_for_process(process):
@@ -102,21 +130,45 @@ def output_buffers_for_process(process, exclude_stdouts):
     )
 
 
+BORG_LOG_LEVEL_ELEVATION_THRESHOLD = 10
+
+
 def borg_json_log_line_to_record(line, log_level):
     '''
     Given a single Borg "--log-json"-style log line and a log level, return the line converted to a
     logging.LogRecord instance. Return None if the line can't be parsed as JSON.
+
+    If Borg provides a log level in its JSON, prefer logging at that level. But if Borg doesn't
+    provide a log level—or the log level given to this function is just a little bit higher than
+    Borg's—elevate to that level. This supports use cases like elevating Borg's INFO level logs to
+    borgmatic's custom ANSWER level so that requested data shows up even at the default verbosity.
     '''
     with contextlib.suppress(json.JSONDecodeError, TypeError, KeyError, AttributeError):
         log_data = json.loads(line)
         log_type = log_data.get('type')
 
         if log_type == 'log_message':
+            borg_log_level = logging._nameToLevel.get(log_data.get('levelname'))
+            log_level_delta = 0 if log_level is None else log_level - borg_log_level
+
+            if log_level_delta > 0 and log_level_delta < BORG_LOG_LEVEL_ELEVATION_THRESHOLD:
+                return logging.makeLogRecord(
+                    dict(
+                        levelno=log_level,
+                        created=log_data.get('time'),
+                        msg=log_data.get('message'),
+                        msgid=log_data.get('msgid'),
+                        levelname=logging.getLevelName(log_level),
+                        name=log_data.get('name'),
+                    )
+                )
+
             return logging.makeLogRecord(
                 dict(
-                    levelno=logging._nameToLevel.get(log_data.get('levelname')),
+                    levelno=borg_log_level,
                     created=log_data.get('time'),
                     msg=log_data.get('message'),
+                    msgid=log_data.get('msgid'),
                     levelname=log_data.get('levelname'),
                     name=log_data.get('name'),
                 )
@@ -128,6 +180,7 @@ def borg_json_log_line_to_record(line, log_level):
                     levelno=log_level,
                     created=time.time(),
                     msg=f'{log_data.get("status")} {log_data.get("path")}',
+                    msgid=log_data.get('msgid'),
                     levelname=logging.getLevelName(log_level),
                     name='borg.file_status',
                 )
@@ -163,7 +216,7 @@ def parse_log_line(line, log_level, elevate_stderr, borg_local_path, command):
     came from stderr and the string "warning:" appears at the start of the log line. In that case,
     just elevate the log level to a WARN.
     '''
-    if borg_local_path and command[0] == borg_local_path:
+    if command_is_borg(command, borg_local_path):
         log_record = borg_json_log_line_to_record(line, log_level)
 
         if log_record:
@@ -177,18 +230,20 @@ def parse_log_line(line, log_level, elevate_stderr, borg_local_path, command):
     return log_line_to_record(line, log_level)
 
 
-def handle_log_record(log_record, last_lines):
+def handle_log_record(log_record, last_lines=None):
     '''
     Given a log record to be logged and a rolling list of last lines, append the record's message to
-    the last lines. Then (if the log level is not None), log the record.
+    the last lines (if given). Then (if the log level is not None), log the record.
 
     Return the log record.
     '''
     log_message = log_record.getMessage()
-    last_lines.append(log_message)
 
-    if len(last_lines) > ERROR_OUTPUT_MAX_LINE_COUNT:
-        last_lines.pop(0)
+    if last_lines is not None:
+        last_lines.append(log_message)
+
+        if len(last_lines) > ERROR_OUTPUT_MAX_LINE_COUNT:
+            last_lines.pop(0)
 
     if log_record.levelno is not None:
         logger.handle(log_record)
@@ -196,7 +251,232 @@ def handle_log_record(log_record, last_lines):
     return log_record
 
 
-def log_outputs(  # noqa: PLR0912
+READ_CHUNK_SIZE = 4096
+
+
+def read_lines(buffer, process, line_separator='\n'):
+    '''
+    Given a Python buffer (like stdout) ready for reading, its process, and a line separator,
+    repeatedly yield a tuple of (decoded) lines from the buffer until the process has exited.
+
+    It is assumed that this function's generator is used in conjunction with an external select()
+    call to know when to read more lines. Otherwise, the generator will busywait if it's called in a
+    tight loop.
+    '''
+    data = b''
+    encoded_separator = line_separator.encode()
+    separator_size = len(encoded_separator)
+    encoding = locale.getpreferredencoding()
+
+    while True:
+        chunk = os.read(buffer.fileno(), READ_CHUNK_SIZE)
+
+        if not chunk:  # EOF
+            # The process is still running, so we keep running too.
+            if process.poll() is None:  # pragma: no cover
+                continue
+
+            break
+
+        data += chunk
+        lines = []
+
+        # Split the data into lines, holding back anything leftover that might
+        # be a partial line.
+        while True:
+            separator_position = data.find(encoded_separator)
+
+            if separator_position == -1:
+                break
+
+            lines.append(data[:separator_position].decode(encoding))
+            data = data[separator_position + separator_size :]
+
+        yield tuple(lines)
+
+    # Yield any leftover data from the end of the buffer.
+    if data:
+        yield (data.decode(encoding).rstrip(),)
+
+
+Buffer_reader = collections.namedtuple(
+    'Buffer_reader',
+    ('lines', 'process'),
+)
+
+
+Process_metadata = collections.namedtuple(
+    'Process_metadata',
+    ('last_lines', 'capture'),
+)
+
+
+def log_buffer_lines(
+    buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr=False
+):
+    '''
+    Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
+    Process_metadata instance, a requested output log level for stdout, Borg's local path, and
+    whether to capture stderr, read and log any ready output lines from the buffers. Additionally,
+    for any log records with a log level the same as the output log level, yield those log messages
+    for capture.
+
+    This function just does one "turn of the crank" of logging buffer output. It is intended to be
+    called repeatedly to continue to process buffers.
+    '''
+    if not buffer_readers:
+        return
+
+    (ready_buffers, _, _) = select.select(buffer_readers.keys(), [], [])
+
+    for ready_buffer in ready_buffers:
+        reader = buffer_readers[ready_buffer]
+
+        # The "ready" process has exited, but it might be a pipe destination with other
+        # processes (pipe sources) waiting to be read from. So as a measure to prevent
+        # hangs, vent all processes when one exits.
+        if reader.process and reader.process.poll() is not None:
+            for other_process in process_metadatas:
+                if (
+                    other_process.poll() is None
+                    and other_process.stdout
+                    and other_process.stdout not in buffer_readers
+                ):
+                    # Add the process's output to buffer_readers to ensure it'll get read.
+                    buffer_readers[other_process.stdout] = Buffer_reader(
+                        read_lines(other_process.stdout, other_process), other_process
+                    )
+
+        try:
+            lines = next(reader.lines)
+        except StopIteration:
+            continue
+
+        for line in lines:
+            if not line or not reader.process:
+                continue
+
+            # Keep the last few lines of output in case the process errors and we need the
+            # output for the exception below.
+            log_record = handle_log_record(
+                parse_log_line(
+                    line=line,
+                    log_level=output_log_level,
+                    elevate_stderr=(ready_buffer == reader.process.stderr and not capture_stderr),
+                    borg_local_path=borg_local_path,
+                    command=reader.process.args,
+                ),
+                last_lines=process_metadatas[reader.process].last_lines,
+            )
+
+            if (
+                log_record.levelno is None or log_record.levelno == output_log_level
+            ) and process_metadatas[reader.process].capture:
+                yield log_record.getMessage()
+
+
+def raise_for_process_errors(
+    buffer_readers,
+    process_metadatas,
+    output_log_level,
+    borg_local_path,
+    borg_exit_codes,
+    capture_stderr=False,
+):
+    '''
+    Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
+    Process_metadata instance, a requested output log level for stdout, Borg's local path, a
+    sequence of exit code configuration dicts, and whether to capture stderr, check the given
+    processes for error or warning exit codes. If found, vent or kill any running processes and
+    drain any remaining buffer lines. In the case of an error exit code, raise.  In the case of
+    warning, return Exit_status.WARNING. Otherwise, return None.
+    '''
+    result_status = None
+
+    for process in process_metadatas:
+        exit_code = process.poll() if buffer_readers else process.wait()
+
+        if exit_code is None:
+            continue
+
+        exit_status = interpret_exit_code(process.args, exit_code, borg_local_path, borg_exit_codes)
+
+        if exit_status not in {Exit_status.ERROR, Exit_status.WARNING}:
+            continue
+
+        # Something has gone wrong. So vent each process' output buffer to prevent it from
+        # hanging. And then kill the process.
+        for other_process in process_metadatas:
+            if other_process.poll() is None:
+                other_process.stdout.read(0)
+                other_process.kill()
+
+        if exit_status == Exit_status.WARNING:
+            result_status = Exit_status.WARNING
+            continue
+
+        # Drain and log remaining buffer lines, so no output gets lost. But swallow captured output,
+        # since we're raising below instead of yielding captured lines.
+        tuple(
+            log_remaining_buffer_lines(
+                buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
+            )
+        )
+        last_lines = process_metadatas[process].last_lines
+
+        # If an error occurs, include its output in the raised exception so that we don't
+        # inadvertently hide error output.
+        if len(last_lines) >= ERROR_OUTPUT_MAX_LINE_COUNT:
+            last_lines.insert(0, '...')
+
+        raise subprocess.CalledProcessError(
+            exit_code,
+            command_for_process(process),
+            '\n'.join(last_lines),
+        )
+
+    return result_status
+
+
+def log_remaining_buffer_lines(
+    buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr=False
+):
+    '''
+    Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
+    Process_metadata instance, a requested output log level for stdout, Borg's local path, and
+    whether to capture stderr, drain and log any remaining output lines from the buffers until
+    they're empty. Additionally, for any log records with a log level the same as the output log
+    level, yield those log messages for capture.
+
+    The main difference between this function and log_buffer_lines() is that this one doesn't just
+    do one "turn of the crank" of logging buffer output; it completely drains any remaining output.
+    '''
+    for output_buffer, reader in buffer_readers.items():
+        if not reader.process:
+            continue
+
+        for lines in reader.lines:
+            for line in lines:
+                log_record = handle_log_record(
+                    parse_log_line(
+                        line=line.rstrip(),
+                        log_level=output_log_level,
+                        elevate_stderr=(
+                            output_buffer == reader.process.stderr and not capture_stderr
+                        ),
+                        borg_local_path=borg_local_path,
+                        command=reader.process.args,
+                    ),
+                    last_lines=process_metadatas[reader.process].last_lines,
+                )
+
+                if (
+                    log_record.levelno is None or log_record.levelno == output_log_level
+                ) and process_metadatas[reader.process].capture:
+                    yield log_record.getMessage()
+
+
+def log_outputs(
     processes,
     exclude_stdouts,
     output_log_level,
@@ -221,131 +501,47 @@ def log_outputs(  # noqa: PLR0912
     buffers. Also note that stdout for a process can be None if output is intentionally not
     captured, in which case it won't be logged.
     '''
-    # Map from output buffer to sequence of last lines.
-    process_last_lines = collections.defaultdict(list)
-    process_for_output_buffer = {
-        buffer: process
+    # Map from output buffer to Process_metadata instance. By convention, the last process is the
+    # process to capture.
+    process_metadatas = {
+        process: Process_metadata(last_lines=[], capture=bool(process == processes[-1]))
+        for process in processes
+    }
+
+    # Map from buffer to Buffer_reader instance.
+    buffer_readers = {
+        buffer: Buffer_reader(read_lines(buffer, process), process)
         for process in processes
         if process.stdout or process.stderr
         for buffer in output_buffers_for_process(process, exclude_stdouts)
     }
-    output_buffers = list(process_for_output_buffer.keys())
-    process_to_capture = processes[-1]
-    still_running = True
 
-    # Log output for each process until they all exit.
-    while True:  # noqa: PLR1702
-        if output_buffers:
-            (ready_buffers, _, _) = select.select(output_buffers, [], [])
+    # Log output lines for each process until they all exit or one errors.
+    while True:
+        yield from log_buffer_lines(
+            buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
+        )
 
-            for ready_buffer in ready_buffers:
-                ready_process = process_for_output_buffer.get(ready_buffer)
-
-                # The "ready" process has exited, but it might be a pipe destination with other
-                # processes (pipe sources) waiting to be read from. So as a measure to prevent
-                # hangs, vent all processes when one exits.
-                if ready_process and ready_process.poll() is not None:
-                    for other_process in processes:
-                        if (
-                            other_process.poll() is None
-                            and other_process.stdout
-                            and other_process.stdout not in output_buffers
-                        ):
-                            # Add the process's output to output_buffers to ensure it'll get read.
-                            output_buffers.append(other_process.stdout)
-
-                while True:
-                    line = ready_buffer.readline().rstrip().decode()
-                    if not line or not ready_process:
-                        break
-
-                    command = (
-                        ready_process.args.split(' ')
-                        if isinstance(ready_process.args, str)
-                        else ready_process.args
-                    )
-
-                    # Keep the last few lines of output in case the process errors and we need the
-                    # output for the exception below.
-                    log_record = handle_log_record(
-                        parse_log_line(
-                            line=line,
-                            log_level=output_log_level,
-                            elevate_stderr=(
-                                ready_buffer == ready_process.stderr and not capture_stderr
-                            ),
-                            borg_local_path=borg_local_path,
-                            command=command,
-                        ),
-                        last_lines=process_last_lines[ready_process],
-                    )
-
-                    if log_record.levelno is None and ready_process == process_to_capture:
-                        yield log_record.getMessage()
-
-        if not still_running:
+        if (
+            raise_for_process_errors(
+                buffer_readers,
+                process_metadatas,
+                output_log_level,
+                borg_local_path,
+                borg_exit_codes,
+                capture_stderr,
+            )
+            == Exit_status.WARNING
+        ):
             break
 
-        still_running = False
+        if all(process.poll() is not None for process in processes):
+            break
 
-        for process in processes:
-            exit_code = process.poll() if output_buffers else process.wait()
-
-            if exit_code is None:
-                still_running = True
-                command = process.args.split(' ') if isinstance(process.args, str) else process.args
-                continue
-
-            command = process.args.split(' ') if isinstance(process.args, str) else process.args
-            exit_status = interpret_exit_code(command, exit_code, borg_local_path, borg_exit_codes)
-
-            if exit_status in {Exit_status.ERROR, Exit_status.WARNING}:
-                last_lines = process_last_lines[process]
-
-                # If an error occurs, include its output in the raised exception so that we don't
-                # inadvertently hide error output.
-                for output_buffer in output_buffers_for_process(process, exclude_stdouts):
-                    # Collect any straggling output lines that came in since we last gathered output.
-                    while output_buffer:  # pragma: no cover
-                        line = output_buffer.readline().rstrip().decode()
-                        if not line:
-                            break
-
-                        log_record = handle_log_record(
-                            parse_log_line(
-                                line=line,
-                                log_level=output_log_level,
-                                elevate_stderr=(
-                                    output_buffer == process.stderr and not capture_stderr
-                                ),
-                                borg_local_path=borg_local_path,
-                                command=command,
-                            ),
-                            last_lines=last_lines,
-                        )
-
-                        if log_record.levelno is None and process == process_to_capture:
-                            yield log_record.getMessage()
-
-                if len(last_lines) == ERROR_OUTPUT_MAX_LINE_COUNT:
-                    last_lines.insert(0, '...')
-
-                # Something has gone wrong. So vent each process' output buffer to prevent it from
-                # hanging. And then kill the process.
-                for other_process in processes:
-                    if other_process.poll() is None:
-                        other_process.stdout.read(0)
-                        other_process.kill()
-
-                if exit_status == Exit_status.ERROR:
-                    raise subprocess.CalledProcessError(
-                        exit_code,
-                        command_for_process(process),
-                        '\n'.join(last_lines),
-                    )
-
-                still_running = False
-                break
+    # Now that all processes have exited, drain and consume any last output.
+    yield from log_remaining_buffer_lines(
+        buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
+    )
 
 
 SECRET_COMMAND_FLAG_NAMES = {'--password'}
@@ -464,6 +660,7 @@ def execute_command(
 
 def execute_command_and_capture_output(
     full_command,
+    output_log_level=None,
     input_file=None,
     capture_stderr=False,
     shell=False,
@@ -478,13 +675,15 @@ def execute_command_and_capture_output(
     output (stdout) as a generator that yields one line at a time. The generator must be consumed in
     order for the called command to execute.
 
-    If an input file descriptor is given, then pipe it to the command's stdin. If capture stderr is
-    True, then capture stderr in addition to stdout. If shell is True, execute the command within a
-    shell. If an environment variables dict is given, then pass it into the command. If a working
-    directory is given, use that as the present working directory when running the command. If a
-    Borg local path is given, and the command matches it (regardless of arguments), treat exit code
-    1 as a warning instead of an error. But if Borg exit codes are given as a sequence of exit code
-    configuration dicts, then use that configuration to decide what's an error and what's a warning.
+    If an output log level is given, then instead of suppressing log output, also output the
+    captured lines at the given log level.  If an input file descriptor is given, then pipe it to
+    the command's stdin. If capture stderr is True, then capture stderr in addition to stdout. If
+    shell is True, execute the command within a shell. If an environment variables dict is given,
+    then pass it into the command. If a working directory is given, use that as the present working
+    directory when running the command. If a Borg local path is given, and the command matches it
+    (regardless of arguments), treat exit code 1 as a warning instead of an error. But if Borg exit
+    codes are given as a sequence of exit code configuration dicts, then use that configuration to
+    decide what's an error and what's a warning.
 
     Raise subprocesses.CalledProcessError if an error occurs while running the command.
     '''
@@ -496,7 +695,9 @@ def execute_command_and_capture_output(
             command,
             stdin=input_file,
             stdout=subprocess.PIPE,
-            stderr=subprocess.PIPE,
+            stderr=subprocess.PIPE
+            if capture_stderr or command_is_borg(command, borg_local_path)
+            else None,
             shell=shell,
             env=environment,
             cwd=working_directory,
@@ -510,22 +711,20 @@ def execute_command_and_capture_output(
             raise
 
         if error.output is not None:
-            yield from iter(error.output.decode().splitlines())
+            yield from iter(error.output.decode(locale.getpreferredencoding()).splitlines())
 
         return
 
     with borgmatic.logger.Log_prefix(None):  # Log command output without any prefix.
-        captured_lines = log_outputs(
+        yield from log_outputs(
             (process,),
             (input_file,),
-            None,
+            output_log_level,
             borg_local_path,
             borg_exit_codes,
             capture_stderr=capture_stderr,
         )
 
-    yield from captured_lines
-
 
 def execute_command_with_processes(
     full_command,
@@ -588,12 +787,10 @@ def execute_command_with_processes(
         raise
 
     with borgmatic.logger.Log_prefix(None):  # Log command output without any prefix.
-        captured_lines = log_outputs(
+        yield from log_outputs(
             (*processes, command_process),
             (input_file, output_file),
             output_log_level,
             borg_local_path,
             borg_exit_codes,
         )
-
-    yield from captured_lines

borgmatic/hooks/credential/keepassxc.py

@@ -7,19 +7,34 @@ import borgmatic.execute
 logger = logging.getLogger(__name__)
 
 
+SECRET_SERVICE_DATABASE_PATH = 'secret-service'
+
+
 def load_credential(hook_config, config, credential_parameters):
     '''
     Given the hook configuration dict, the configuration dict, and a credential parameters tuple
     containing a KeePassXC database path and an attribute name to load, run keepassxc-cli to fetch
-    the corresponding KeePassXC credential and return it.
+    the corresponding KeePassXC credential and return it. Or use secret-tool if the database path
+    is "secret-service", indicating that KeePassXC's secret service integration should be used
+    instead.
 
-    Raise ValueError if keepassxc-cli can't retrieve the credential.
+    Raise ValueError if keepassxc-cli or secret-tool can't retrieve the credential.
     '''
     try:
         (database_path, attribute_name) = credential_parameters
     except ValueError:
         raise ValueError(f'Invalid KeePassXC credential parameters: {credential_parameters}')
 
+    if database_path == SECRET_SERVICE_DATABASE_PATH:
+        command = (
+            *shlex.split((hook_config or {}).get('secret_tool_command', 'secret-tool')),
+            *('lookup', 'Path', attribute_name),
+        )
+
+        return '\n'.join(borgmatic.execute.execute_command_and_capture_output(command)).rstrip(
+            os.linesep
+        )
+
     expanded_database_path = os.path.expanduser(database_path)
 
     if not os.path.exists(expanded_database_path):

borgmatic/hooks/data_source/bootstrap.py

@@ -1,6 +1,7 @@
 import contextlib
 import glob
 import importlib
+import itertools
 import json
 import logging
 import os
@@ -19,6 +20,32 @@ def use_streaming(hook_config, config):  # pragma: no cover
     return False
 
 
+MAXIMUM_CONFIG_SYMLINKS_TO_FOLLOW = 10
+
+
+def resolve_config_path_symlinks(path):
+    '''
+    Given a path, resolve and yield each successive symlink until the final non-symlink target. If
+    the given path isn't a symlink, then just yield it.
+
+    The purpose of this is to ensure that configuration files that are behind a symbolic link (or
+    several) actually get backed up.
+
+    Raise ValueError if we have to follow too many symlinks without getting to the final target.
+    '''
+    original_path = os.path.normpath(path)
+
+    for _ in range(MAXIMUM_CONFIG_SYMLINKS_TO_FOLLOW):
+        yield path
+
+        if not os.path.islink(path):
+            return
+
+        path = os.path.normpath(os.path.join(os.path.dirname(path), os.readlink(path)))
+
+    raise ValueError(f'Too many symlinks to follow for configuration path: {original_path}')
+
+
 def dump_data_sources(
     hook_config,
     config,
@@ -34,6 +61,9 @@ def dump_data_sources(
     the archive. But skip this if the bootstrap store_config_files option is False or if this is a
     dry run.
 
+    If any configuration paths are symlinks, then store each symlink along with any destination
+    paths as well.
+
     Return an empty sequence, since there are no ongoing dump processes from this hook.
     '''
     if hook_config and hook_config.get('store_config_files') is False:
@@ -45,6 +75,10 @@ def dump_data_sources(
         'manifest.json',
     )
 
+    resolved_config_paths = tuple(
+        itertools.chain.from_iterable(resolve_config_path_symlinks(path) for path in config_paths)
+    )
+
     if dry_run:
         return []
 
@@ -54,7 +88,7 @@ def dump_data_sources(
         json.dump(
             {
                 'borgmatic_version': importlib.metadata.version('borgmatic'),
-                'config_paths': config_paths,
+                'config_paths': resolved_config_paths,
             },
             manifest_file,
         )
@@ -67,7 +101,7 @@ def dump_data_sources(
         ),
     )
 
-    for config_path in config_paths:
+    for config_path in resolved_config_paths:
         borgmatic.hooks.data_source.config.inject_pattern(
             patterns,
             borgmatic.borg.pattern.Pattern(

borgmatic/hooks/data_source/mariadb.py

@@ -1,4 +1,3 @@
-import copy
 import logging
 import os
 import re
@@ -67,7 +66,9 @@ def make_defaults_file_options(username=None, password=None, defaults_extra_file
     Do not use the returned value for multiple different command invocations. That will not work
     because each pipe is "used up" once read.
     '''
-    escaped_password = None if password is None else password.replace('\\', '\\\\')
+    escaped_password = (
+        None if password is None else password.replace('\\', '\\\\').replace('"', '\\"')
+    )
 
     values = '\n'.join(
         (
@@ -106,6 +107,10 @@ def make_defaults_file_options(username=None, password=None, defaults_extra_file
     return (f'--defaults-extra-file=/dev/fd/{read_file_descriptor}',)
 
 
+EXCLUDED_SYSTEM_DATABASE_NAMES = ('information_schema', 'performance_schema', 'sys')
+SYSTEM_DATABASE_NAME = 'mysql'
+
+
 def database_names_to_dump(database, config, username, password, environment, dry_run):
     '''
     Given a requested database config, a configuration dict, a database username and password, an
@@ -164,17 +169,18 @@ def database_names_to_dump(database, config, username, password, environment, dr
         working_directory=borgmatic.config.paths.get_working_directory(config),
     )
 
+    # Dumping system databases directly doesn't work; too much gets dumped (including virtual tables
+    # that MariaDB recreates on startup) and so the dump isn't restorable. Therefore several system
+    # databases are excluded here. But also see below where select system tables from the "mysql"
+    # system table are dumped via the "--system" flag.
     return tuple(
         show_name.strip()
         for show_name in show_lines
-        if show_name not in SYSTEM_DATABASE_NAMES
+        if show_name not in EXCLUDED_SYSTEM_DATABASE_NAMES
         if not skip_names or show_name not in skip_names
     )
 
 
-SYSTEM_DATABASE_NAMES = ('information_schema', 'mysql', 'performance_schema', 'sys')
-
-
 def execute_dump_command(
     database,
     config,
@@ -235,8 +241,12 @@ def execute_dump_command(
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if database.get('tls') is True else ())
         + (('--skip-ssl',) if database.get('tls') is False else ())
+        + (('--events',) if database.get('events', True) else ())
+        + (('--routines',) if database.get('routines', True) else ())
+        + (('--all-tablespaces',) if database.get('tablespaces', True) else ())
+        + (('--system=users,udfs,servers',) if SYSTEM_DATABASE_NAME in database_names else ())
         + ('--databases',)
-        + database_names
+        + tuple(name for name in database_names if name != SYSTEM_DATABASE_NAME)
         + ('--result-file', dump_filename)
     )
 
@@ -323,6 +333,7 @@ def dump_data_sources(
 
             raise ValueError('Cannot find any MariaDB databases to dump.')
 
+        # Database dumps to individual files.
         if database['name'] == 'all' and database.get('format'):
             for database_name in dump_database_names:
                 dumps_metadata.append(
@@ -335,8 +346,7 @@ def dump_data_sources(
                         database.get('container'),
                     )
                 )
-                renamed_database = copy.copy(database)
-                renamed_database['name'] = database_name
+                renamed_database = dict(database, name=database_name)
                 processes.append(
                     execute_dump_command(
                         renamed_database,
@@ -350,6 +360,7 @@ def dump_data_sources(
                         dry_run_label,
                     ),
                 )
+        # Database dumps all to one file.
         else:
             dumps_metadata.append(
                 borgmatic.actions.restore.Dump(
@@ -414,7 +425,7 @@ def make_data_source_dump_patterns(
     port=None,
     container=None,
     label=None,
-):  # pragma: no cover
+):
     '''
     Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
     and a database name to match, return the corresponding glob patterns to match the database dump
@@ -423,24 +434,54 @@ def make_data_source_dump_patterns(
     borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
 
     return (
-        dump.make_data_source_dump_filename(
-            make_dump_path('borgmatic'), name, hostname, port, container, label
+        *(
+            dump.make_data_source_dump_filename(
+                make_dump_path('borgmatic'), name, hostname, port, container, label
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_runtime_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_source_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
         ),
-        dump.make_data_source_dump_filename(
-            make_dump_path(borgmatic_runtime_directory),
-            name,
-            hostname,
-            port,
-            container,
-            label,
+        *(
+            (
+                dump.make_data_source_dump_filename(
+                    make_dump_path('borgmatic'),
+                    name,
+                    hostname,
+                    port=None,
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port == get_default_port(databases, config)
+            else ()
         ),
-        dump.make_data_source_dump_filename(
-            make_dump_path(borgmatic_source_directory),
-            name,
-            hostname,
-            port,
-            container,
-            label,
+        *(
+            (
+                dump.make_data_source_dump_filename(
+                    make_dump_path('borgmatic'),
+                    name,
+                    hostname,
+                    port=get_default_port(databases, config),
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port is None
+            else ()
         ),
     )
 

borgmatic/hooks/data_source/mongodb.py

@@ -216,7 +216,7 @@ def make_data_source_dump_patterns(
     port=None,
     container=None,
     label=None,
-):  # pragma: no cover
+):
     '''
     Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
     and a database name to match, return the corresponding glob patterns to match the database dump
@@ -225,24 +225,54 @@ def make_data_source_dump_patterns(
     borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
 
     return (
-        dump.make_data_source_dump_filename(
-            make_dump_path('borgmatic'), name, hostname, port, container, label
+        *(
+            dump.make_data_source_dump_filename(
+                make_dump_path('borgmatic'), name, hostname, port, container, label
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_runtime_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_source_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
         ),
-        dump.make_data_source_dump_filename(
-            make_dump_path(borgmatic_runtime_directory),
-            name,
-            hostname,
-            port,
-            container,
-            label,
+        *(
+            (
+                dump.make_data_source_dump_filename(
+                    make_dump_path('borgmatic'),
+                    name,
+                    hostname,
+                    port=None,
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port == get_default_port(databases, config)
+            else ()
         ),
-        dump.make_data_source_dump_filename(
-            make_dump_path(borgmatic_source_directory),
-            name,
-            hostname,
-            port,
-            container,
-            label,
+        *(
+            (
+                dump.make_data_source_dump_filename(
+                    make_dump_path('borgmatic'),
+                    name,
+                    hostname,
+                    port=get_default_port(databases, config),
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port is None
+            else ()
         ),
     )
 

borgmatic/hooks/data_source/mysql.py

@@ -1,4 +1,3 @@
-import copy
 import logging
 import os
 import shlex
@@ -166,6 +165,9 @@ def execute_dump_command(
         + (('--user', username) if username and password_transport == 'environment' else ())
         + (('--ssl',) if database.get('tls') is True else ())
         + (('--skip-ssl',) if database.get('tls') is False else ())
+        + (('--events',) if database.get('events', True) else ())
+        + (('--routines',) if database.get('routines', True) else ())
+        + (('--all-tablespaces',) if database.get('tablespaces', True) else ())
         + ('--databases',)
         + database_names
         + ('--result-file', dump_filename)
@@ -266,8 +268,7 @@ def dump_data_sources(
                         database.get('container'),
                     )
                 )
-                renamed_database = copy.copy(database)
-                renamed_database['name'] = database_name
+                renamed_database = dict(database, name=database_name)
                 processes.append(
                     execute_dump_command(
                         renamed_database,
@@ -345,7 +346,7 @@ def make_data_source_dump_patterns(
     port=None,
     container=None,
     label=None,
-):  # pragma: no cover
+):
     '''
     Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
     and a database name to match, return the corresponding glob patterns to match the database dump
@@ -354,24 +355,54 @@ def make_data_source_dump_patterns(
     borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
 
     return (
-        dump.make_data_source_dump_filename(
-            make_dump_path('borgmatic'), name, hostname, port, container, label
+        *(
+            dump.make_data_source_dump_filename(
+                make_dump_path('borgmatic'), name, hostname, port, container, label
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_runtime_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_source_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
         ),
-        dump.make_data_source_dump_filename(
-            make_dump_path(borgmatic_runtime_directory),
-            name,
-            hostname,
-            port,
-            container,
-            label,
+        *(
+            (
+                dump.make_data_source_dump_filename(
+                    make_dump_path('borgmatic'),
+                    name,
+                    hostname,
+                    port=None,
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port == get_default_port(databases, config)
+            else ()
         ),
-        dump.make_data_source_dump_filename(
-            make_dump_path(borgmatic_source_directory),
-            name,
-            hostname,
-            port,
-            container,
-            label,
+        *(
+            (
+                dump.make_data_source_dump_filename(
+                    make_dump_path('borgmatic'),
+                    name,
+                    hostname,
+                    port=get_default_port(databases, config),
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port is None
+            else ()
         ),
     )
 

borgmatic/hooks/data_source/postgresql.py

@@ -100,7 +100,11 @@ def database_names_to_dump(database, config, environment, dry_run):
             if 'username' in database
             else ()
         )
-        + (tuple(database['list_options'].split(' ')) if 'list_options' in database else ())
+        + (
+            tuple(shlex.quote(part) for part in shlex.split(database['list_options']))
+            if 'list_options' in database
+            else ()
+        )
     )
     logger.debug('Querying for "all" PostgreSQL databases to dump')
     list_lines = execute_command_and_capture_output(
@@ -226,7 +230,7 @@ def dump_data_sources(
                 + (('--compress', shlex.quote(str(compression))) if compression is not None else ())
                 + (('--file', shlex.quote(dump_filename)) if dump_format == 'directory' else ())
                 + (
-                    tuple(shlex.quote(option) for option in database['options'].split(' '))
+                    tuple(shlex.quote(part) for part in shlex.split(database['options']))
                     if 'options' in database
                     else ()
                 )
@@ -306,7 +310,7 @@ def make_data_source_dump_patterns(
     port=None,
     container=None,
     label=None,
-):  # pragma: no cover
+):
     '''
     Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
     and a database name to match, return the corresponding glob patterns to match the database dump
@@ -315,24 +319,54 @@ def make_data_source_dump_patterns(
     borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
 
     return (
-        dump.make_data_source_dump_filename(
-            make_dump_path('borgmatic'), name, hostname, port, container, label
+        *(
+            dump.make_data_source_dump_filename(
+                make_dump_path('borgmatic'), name, hostname, port, container, label
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_runtime_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
+            dump.make_data_source_dump_filename(
+                make_dump_path(borgmatic_source_directory),
+                name,
+                hostname,
+                port,
+                container,
+                label,
+            ),
         ),
-        dump.make_data_source_dump_filename(
-            make_dump_path(borgmatic_runtime_directory),
-            name,
-            hostname,
-            port,
-            container,
-            label,
+        *(
+            (
+                dump.make_data_source_dump_filename(
+                    make_dump_path('borgmatic'),
+                    name,
+                    hostname,
+                    port=None,
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port == get_default_port(databases, config)
+            else ()
         ),
-        dump.make_data_source_dump_filename(
-            make_dump_path(borgmatic_source_directory),
-            name,
-            hostname,
-            port,
-            container,
-            label,
+        *(
+            (
+                dump.make_data_source_dump_filename(
+                    make_dump_path('borgmatic'),
+                    name,
+                    hostname,
+                    port=get_default_port(databases, config),
+                    container=container,
+                    label=label,
+                ),
+            )
+            if port is None
+            else ()
         ),
     )
 
@@ -393,7 +427,7 @@ def restore_data_source_dump(
         + (('--username', username) if username else ())
         + (('--dbname', data_source['name']) if not all_databases else ())
         + (
-            tuple(data_source['analyze_options'].split(' '))
+            tuple(shlex.quote(part) for part in shlex.split(data_source['analyze_options']))
             if 'analyze_options' in data_source
             else ()
         )
@@ -414,7 +448,7 @@ def restore_data_source_dump(
         + (('--username', username) if username else ())
         + (('--no-owner',) if data_source.get('no_owner', False) else ())
         + (
-            tuple(data_source['restore_options'].split(' '))
+            tuple(shlex.quote(part) for part in shlex.split(data_source['restore_options']))
             if 'restore_options' in data_source
             else ()
         )

borgmatic/hooks/data_source/zfs.py

@@ -2,6 +2,7 @@ import collections
 import glob
 import hashlib
 import logging
+import operator
 import os
 import shutil
 import subprocess
@@ -71,7 +72,7 @@ def get_datasets_to_backup(zfs_command, patterns):
                 )
                 # Skip datasets that are marked "canmount=off", because mounting their snapshots will
                 # result in completely empty mount points—thereby preventing us from backing them up.
-                if can_mount == 'on'
+                if can_mount != 'off'
             ),
             key=lambda dataset: dataset.mount_point,
             reverse=True,
@@ -123,7 +124,8 @@ def get_datasets_to_backup(zfs_command, patterns):
 
 def get_all_dataset_mount_points(zfs_command):
     '''
-    Given a ZFS command to run, return all ZFS datasets as a sequence of sorted mount points.
+    Given a ZFS command to run, return a dict from ZFS dataset name to mount point (reverse sorted
+    by mount point).
     '''
     list_lines = borgmatic.execute.execute_command_and_capture_output(
         (
@@ -133,19 +135,23 @@ def get_all_dataset_mount_points(zfs_command):
             '-t',
             'filesystem',
             '-o',
-            'mountpoint',
+            'name,mountpoint',
         ),
         close_fds=True,
     )
 
-    return tuple(
+    return dict(
         sorted(
-            {
-                mount_point
+            (
+                (dataset_name, mount_point)
                 for line in list_lines
-                for mount_point in (line.rstrip(),)
+                for (dataset_name, mount_point) in (line.rstrip().split('\t'),)
                 if mount_point != 'none'
-            },
+            ),
+            key=operator.itemgetter(1),
+            # Reversing the sorted datasets ensures that we unmount the longer mount point paths of
+            # child datasets before the shorter mount point paths of parent datasets.
+            reverse=True,
         ),
     )
 
@@ -376,7 +382,8 @@ def remove_data_source_dumps(hook_config, config, borgmatic_runtime_directory, p
     zfs_command = hook_config.get('zfs_command', 'zfs')
 
     try:
-        dataset_mount_points = get_all_dataset_mount_points(zfs_command)
+        dataset_name_to_mount_point = get_all_dataset_mount_points(zfs_command)
+        full_snapshot_names = get_all_snapshots(zfs_command)
     except FileNotFoundError:
         logger.debug(f'Could not find "{zfs_command}" command')
         return
@@ -393,50 +400,65 @@ def remove_data_source_dumps(hook_config, config, borgmatic_runtime_directory, p
     )
     logger.debug(f'Looking for snapshots to remove in {snapshots_glob}{dry_run_label}')
     umount_command = hook_config.get('umount_command', 'umount')
+    snapshot_dataset_names = {
+        full_snapshot_name.split('@')[0] for full_snapshot_name in full_snapshot_names
+    }
+    hash_to_dataset_mount_point = {}
+
+    # Make a map from mount point hash to the corresponding (dataset name, mount point) tuple.
+    for dataset_name, mount_point in dataset_name_to_mount_point.items():
+        mount_point_hash = hashlib.shake_256(mount_point.encode('utf-8')).hexdigest(
+            MOUNT_POINT_HASH_LENGTH
+        )
+        hash_to_dataset_mount_point[mount_point_hash] = (dataset_name, mount_point)
 
     for snapshots_directory in glob.glob(snapshots_glob):
         if not os.path.isdir(snapshots_directory):
             continue
 
-        # Reversing the sorted datasets ensures that we unmount the longer mount point paths of
-        # child datasets before the shorter mount point paths of parent datasets.
-        for mount_point in reversed(dataset_mount_points):
-            snapshot_mount_path = os.path.join(snapshots_directory, mount_point.lstrip(os.path.sep))
+        # Get the dataset and mount point corresponding to the hash found in this snapshot directory
+        # path. If none is found, bail.
+        try:
+            (dataset_name, mount_point) = hash_to_dataset_mount_point[
+                os.path.basename(snapshots_directory)
+            ]
+        except KeyError:
+            continue
 
-            # If the snapshot mount path is empty, this is probably just a "shadow" of a nested
-            # dataset and therefore there's nothing to unmount.
-            if not os.path.isdir(snapshot_mount_path) or not os.listdir(snapshot_mount_path):
+        snapshot_mount_path = os.path.join(snapshots_directory, mount_point.lstrip(os.path.sep))
+
+        # If this dataset name doesn't correspond to a known snapshot, then this is probably
+        # just a "shadow" of a nested dataset and therefore there's nothing to unmount.
+        if not os.path.isdir(snapshot_mount_path) or dataset_name not in snapshot_dataset_names:
+            continue
+
+        # This might fail if the path is already mounted, but we swallow errors here since we'll
+        # do another recursive delete below. The point of doing it here is that we don't want to
+        # try to unmount a non-mounted directory (which *will* fail), and probing for whether a
+        # directory is mounted is tough to do in a cross-platform way.
+        if not dry_run:
+            shutil.rmtree(snapshot_mount_path, ignore_errors=True)
+
+            # If the delete was successful, that means there's nothing to unmount.
+            if not os.path.isdir(snapshot_mount_path):
                 continue
 
-            # This might fail if the path is already mounted, but we swallow errors here since we'll
-            # do another recursive delete below. The point of doing it here is that we don't want to
-            # try to unmount a non-mounted directory (which *will* fail), and probing for whether a
-            # directory is mounted is tough to do in a cross-platform way.
-            if not dry_run:
-                shutil.rmtree(snapshot_mount_path, ignore_errors=True)
+        logger.debug(f'Unmounting ZFS snapshot at {snapshot_mount_path}{dry_run_label}')
 
-                # If the delete was successful, that means there's nothing to unmount.
-                if not os.path.isdir(snapshot_mount_path):
-                    continue
-
-            logger.debug(f'Unmounting ZFS snapshot at {snapshot_mount_path}{dry_run_label}')
-
-            if not dry_run:
-                try:
-                    unmount_snapshot(umount_command, snapshot_mount_path)
-                except FileNotFoundError:
-                    logger.debug(f'Could not find "{umount_command}" command')
-                    return
-                except subprocess.CalledProcessError as error:
-                    logger.debug(error)
-                    continue
+        if not dry_run:
+            try:
+                unmount_snapshot(umount_command, snapshot_mount_path)
+            except FileNotFoundError:
+                logger.debug(f'Could not find "{umount_command}" command')
+                return
+            except subprocess.CalledProcessError as error:
+                logger.debug(error)
+                continue
 
         if not dry_run:
             shutil.rmtree(snapshot_mount_path, ignore_errors=True)
 
     # Destroy snapshots.
-    full_snapshot_names = get_all_snapshots(zfs_command)
-
     for full_snapshot_name in full_snapshot_names:
         # Only destroy snapshots that borgmatic actually created!
         if not full_snapshot_name.split('@')[-1].startswith(BORGMATIC_SNAPSHOT_PREFIX):

borgmatic/hooks/monitoring/apprise.py

@@ -1,6 +1,7 @@
 import logging
 import operator
 
+import borgmatic.hooks.credential.parse
 import borgmatic.hooks.monitoring.logs
 import borgmatic.hooks.monitoring.monitor
 
@@ -44,7 +45,9 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
         import apprise  # noqa: PLC0415
         from apprise import NotifyFormat, NotifyType  # noqa: PLC0415
     except ImportError:  # pragma: no cover
-        logger.warning('Unable to import Apprise in monitoring hook')
+        logger.warning(
+            'Unable to import Apprise in its monitoring hook; try installing "borgmatic[Apprise]"'
+        )
         return
 
     state_to_notify_type = {
@@ -76,7 +79,12 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
     logger.info(f'Pinging Apprise services: {labels_string}{dry_run_string}')
 
     apprise_object = apprise.Apprise()
-    apprise_object.add(list(map(operator.itemgetter('url'), hook_config.get('services'))))
+    apprise_object.add(
+        [
+            borgmatic.hooks.credential.parse.resolve_credential(service['url'], config)
+            for service in hook_config.get('services')
+        ]
+    )
 
     if dry_run:
         return

borgmatic/hooks/monitoring/loki.py

@@ -27,9 +27,16 @@ class Loki_log_buffer:
     adding labels to the log stream and takes care of communication with Loki.
     '''
 
-    def __init__(self, url, dry_run):
+    def __init__(self, url, dry_run, tls_cert_path=None, tls_key_path=None):
+        '''
+        Given a Loki URL, a dry run flag, and optional TLS certificate and key paths for mTLS authentication,
+        create an instance of Loki_log_buffer.
+        '''
+
         self.url = url
         self.dry_run = dry_run
+        self.tls_cert_path = tls_cert_path
+        self.tls_key_path = tls_key_path
         self.root = {'streams': [{'stream': {}, 'values': []}]}
 
     def add_value(self, value):
@@ -77,6 +84,7 @@ class Loki_log_buffer:
                     'Content-Type': 'application/json',
                     'User-Agent': 'borgmatic',
                 },
+                cert=(self.tls_cert_path, self.tls_key_path) if self.tls_cert_path else None,
             )
             result.raise_for_status()
         except requests.RequestException:
@@ -88,16 +96,19 @@ class Loki_log_handler(logging.Handler):
     A log handler that sends logs to Loki.
     '''
 
-    def __init__(self, url, send_logs, dry_run):
+    def __init__(self, url, send_logs, log_level, dry_run, tls_cert_path=None, tls_key_path=None):
         '''
         Given a URL to send logs to, whether all borgmatic logs should be sent (or just explicitly
-        added messages from this hook), and whether this is a dry run, create an instance of
-        Loki_log_buffer.
+        added messages from this hook), the log level to use (influencing which logs get sent), and
+        whether this is a dry run, create an instance of Loki_log_buffer.
         '''
         super().__init__()
 
-        self.buffer = Loki_log_buffer(url, dry_run)
+        self.buffer = Loki_log_buffer(
+            url, dry_run, tls_cert_path=tls_cert_path, tls_key_path=tls_key_path
+        )
         self.send_logs = send_logs
+        self.setLevel(log_level)
 
     def emit(self, record):
         '''
@@ -138,7 +149,21 @@ def initialize_monitor(hook_config, config, config_filename, monitoring_log_leve
     Add a handler to the root logger to regularly send the logs to Loki.
     '''
     url = hook_config.get('url')
-    loki = Loki_log_handler(url, hook_config.get('send_logs', False), dry_run)
+    tls = hook_config.get('tls', {})
+
+    if bool(tls.get('cert_path')) != bool(tls.get('key_path')):
+        raise ValueError(
+            'Invalid Loki TLS configuration: cert_path and key_path must both be set or both be unset'
+        )
+
+    loki = Loki_log_handler(
+        url,
+        hook_config.get('send_logs', False),
+        monitoring_log_level,
+        dry_run,
+        tls_cert_path=tls.get('cert_path'),
+        tls_key_path=tls.get('key_path'),
+    )
 
     for key, value in hook_config.get('labels').items():
         if value == '__hostname':
@@ -150,7 +175,9 @@ def initialize_monitor(hook_config, config, config_filename, monitoring_log_leve
         else:
             loki.add_label(key, value)
 
-    logging.getLogger().addHandler(loki)
+    global_logger = logging.getLogger()
+    global_logger.addHandler(loki)
+    global_logger.setLevel(min(handler.level for handler in global_logger.handlers))
 
 
 def ping_monitor(hook_config, config, config_filename, state, monitoring_log_level, dry_run):
@@ -166,9 +193,11 @@ def destroy_monitor(hook_config, config, monitoring_log_level, dry_run):
     '''
     Remove the monitor handler that was added to the root logger.
     '''
-    logger = logging.getLogger()
+    global_logger = logging.getLogger()
 
-    for handler in tuple(logger.handlers):
+    for handler in tuple(global_logger.handlers):
         if isinstance(handler, Loki_log_handler):
             handler.flush()
-            logger.removeHandler(handler)
+            global_logger.removeHandler(handler)
+
+    global_logger.setLevel(min(handler.level for handler in global_logger.handlers))

borgmatic/hooks/monitoring/ntfy.py

@@ -22,6 +22,29 @@ def initialize_monitor(
     '''
 
 
+def convert_string_to_array(value):
+    value = '' if value is None else str(value)
+    items = []
+
+    for item in value.split(','):
+        stripped = item.strip()
+
+        if stripped:
+            items.append(stripped)
+
+    return items
+
+
+PRIORITY_NAME_TO_ID = {
+    'max': 5,
+    'urgent': 5,
+    'high': 4,
+    'default': 3,
+    'low': 2,
+    'min': 1,
+}
+
+
 def ping_monitor(hook_config, config, config_filename, state, monitoring_log_level, dry_run):
     '''
     Ping the configured Ntfy topic. Use the given configuration filename in any log entries.
@@ -31,13 +54,13 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
 
     if state.name.lower() in run_states:
         dry_run_label = ' (dry run; not actually pinging)' if dry_run else ''
-
+        default_priority = PRIORITY_NAME_TO_ID['default']
         state_config = hook_config.get(
             state.name.lower(),
             {
                 'title': f'A borgmatic {state.name} event happened',
                 'message': f'A borgmatic {state.name} event happened',
-                'priority': 'default',
+                'priority': default_priority,
                 'tags': 'borgmatic',
             },
         )
@@ -55,8 +78,8 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
             'topic': topic,
             'title': state_config.get('title'),
             'message': state_config.get('message'),
-            'priority': state_config.get('priority'),
-            'tags': state_config.get('tags'),
+            'priority': PRIORITY_NAME_TO_ID.get(state_config.get('priority'), default_priority),
+            'tags': convert_string_to_array(state_config.get('tags')),
         }
 
         try:

borgmatic/logger.py

@@ -154,6 +154,8 @@ def log_record_to_json(record):
     '''
     Given a logging.LogRecord, return it as a JSON-encoded string containing relevant attributes.
     '''
+    message_id = getattr(record, 'msgid', None)
+
     return json.dumps(
         dict(
             type='log_message',
@@ -162,6 +164,7 @@ def log_record_to_json(record):
             levelname=record.levelname,
             name=record.name,
         )
+        | ({'msgid': message_id} if message_id is not None else {})
     )
 
 
@@ -169,7 +172,7 @@ class Json_formatter(logging.Formatter):
     def __init__(self, fmt='{message}', *args, style='{', **kwargs):
         super().__init__(*args, fmt=fmt, style=style, **kwargs)
 
-    def format(self, record):  # noqa: PLR6301
+    def format(self, record):
         return log_record_to_json(record)
 
 
@@ -336,6 +339,91 @@ class Log_prefix:
         set_log_prefix(self.original_prefix)
 
 
+class Log_exclude_filter(logging.Filter):
+    '''
+    A Python log filter that omits log records matching given attributes.
+    '''
+
+    def __init__(self, name, filter_attributes):
+        '''
+        Given a unique name for this filter and a dict of attributes to filter on, set the filter
+        name and save the attributes for use below.
+        '''
+        self.filter_attributes = filter_attributes
+
+        super().__init__(name)
+
+    def filter(self, log_record):
+        '''
+        Given a log record, return False (indicating the record should be omitted) if the record's
+        attributes match any of the saved filter attributes. Return True (indicating do not omit)
+        otherwise.
+        '''
+        for attribute_name, value in self.filter_attributes.items():
+            if getattr(log_record, attribute_name, None) == value:
+                return False
+
+        return True
+
+
+def add_log_exclude_filter(name, filter_attributes):
+    '''
+    Given a unique filter name and a dict of attributes to filter on, create a log exclude filter
+    with them and add the filter to each log handler.
+    '''
+    for handler in logging.getLogger().handlers:
+        handler.addFilter(Log_exclude_filter(name, filter_attributes))
+
+
+def remove_log_exclude_filter(name):
+    '''
+    Given a unique filter name, remove matching filters from each log handler.
+    '''
+    for handler in logging.getLogger().handlers:
+        for exclude_filter in handler.filters:
+            if getattr(exclude_filter, 'name', None) == name:
+                handler.removeFilter(exclude_filter)
+
+
+class Logs_suppressed:
+    '''
+    A Python context manager for temporarily adding a log filter that suppresses requested log
+    records for the duration of the context manager.
+
+    Example use:
+
+
+       with borgmatic.logger.Logs_suppressed(msgid='Repository.DoesNotExist'):
+            do_something_that_logs()
+
+    For the scope of that "with" statement, any records logged with the given message ID are
+    filtered out of the log output. "msgid" is just an example; any logging.LogRecord attributes
+    (standard or custom) can be passed in to filter on.
+
+    Multiple instances of this context manager with different filter attributes can be in use at
+    once.
+    '''
+
+    def __init__(self, **filter_attributes):
+        '''
+        Given the desired log record filter attributes as keyword arguments, save them for use below.
+        '''
+        self.filter_attributes = filter_attributes
+
+    def __enter__(self):
+        '''
+        Create a log filter with the saved filter attributes and add the filter to every logging
+        handler, so that they filter out the desired log records.
+        '''
+        add_log_exclude_filter(name=str(id(self)), filter_attributes=self.filter_attributes)
+
+    def __exit__(self, exception_type, exception, traceback):
+        '''
+        Remove the previously added filter from every logging handler.
+        '''
+        remove_log_exclude_filter(name=str(id(self)))
+
+
 class Delayed_logging_handler(logging.handlers.BufferingHandler):
     '''
     A logging handler that buffers logs and doesn't flush them until explicitly flushed (after

docs/.env

@@ -0,0 +1 @@
+PORT=8080

docs/Dockerfile

@@ -5,13 +5,17 @@ RUN apk add --no-cache py3-pip py3-ruamel.yaml py3-ruamel.yaml.clib
 RUN pip install --break-system-packages --no-cache /app && borgmatic config generate && borgmatic config generate --destination /etc/borgmatic --split && chmod +r /etc/borgmatic/*.yaml
 RUN mkdir /command-line \
     && borgmatic --help > /command-line/global.txt \
-    && for action in repo-create transfer create prune compact check delete extract config "config bootstrap" "config generate" "config validate" export-tar mount umount repo-delete restore repo-list list repo-info info break-lock "key export" "key import" "key change-passphrase" recreate borg; do \
+    && for action in repo-create transfer create prune compact check delete extract config "config bootstrap" "config generate" "config validate" "config show" export-tar mount umount repo-delete restore repo-list list repo-info info break-lock "key export" "key import" "key change-passphrase" recreate diff browse borg; do \
            borgmatic $action --help > /command-line/${action/ /-}.txt; done
 RUN /app/docs/fetch-contributors >> /contributors.html
 
 FROM docker.io/node:22.4.0-alpine AS html
 
 ARG ENVIRONMENT=production
+ARG PORT=${PORT}
+ARG FONT=${FONT}
+
+RUN echo "FONT: ${FONT}"
 
 WORKDIR /source
 
@@ -28,7 +32,7 @@ COPY --from=borgmatic /etc/borgmatic/options.json /source/docs/reference/configu
 COPY --from=borgmatic /command-line/* /source/docs/_includes/borgmatic/command-line/
 COPY --from=borgmatic /contributors.html /source/docs/_includes/borgmatic/contributors.html
 COPY . /source
-RUN NODE_ENV=${ENVIRONMENT} npx eleventy --input=/source/docs --output=/output
+RUN NODE_ENV=${ENVIRONMENT} PORT=${PORT} FONT=${FONT} npx eleventy --input=/source/docs --output=/output
 RUN npx -y pagefind --site /output
 
 FROM docker.io/nginx:1.26.1-alpine

docs/_data/borgmatic.js

@@ -1,5 +1,7 @@
 module.exports = function() {
     return {
-        environment: process.env.NODE_ENV || "development"
+        environment: process.env.NODE_ENV || "development",
+        port: process.env.PORT || 8080,
+        font: process.env.FONT || "custom"
     };
 };

docs/_includes/layouts/base.njk

@@ -11,7 +11,7 @@
 		<title>borgmatic{% if subtitle or title %} - {% endif %}{{ subtitle + ' - ' if subtitle}}{{ title }}</title>
 		{% endif %}
 {%- set css %}
-{% include 'index.css' %}
+{% include 'index.css.njk' %}
 {% include 'components/lists.css' %}
 {% include 'components/external-links.css' %}
 {% include 'components/minilink.css' %}
@@ -23,6 +23,7 @@
 {% if feedTitle and feedUrl %}
 		<link rel="alternate" href="{{ feedUrl }}" title="{{ feedTitle }}" type="application/atom+xml">
 {% endif %}
+    {{ head_additions | safe }}
 	</head>
 	<body>
 

docs/_includes/layouts/main.njk

@@ -21,7 +21,7 @@ headerClass: elv-header-default
             {% set navPages = collections.all | eleventyNavigation %}
             {% macro renderNavListItem(entry) -%}
             <li{% if entry.url == page.url %} class="elv-toc-active"{% endif %}>
-              <a {% if entry.url %}href="{% if borgmatic.environment == "production" %}https://torsion.org/borgmatic{% else %}http://localhost:8080/borgmatic{% endif %}{{ entry.url | url }}"{% endif %}>{{ entry.title }}</a>
+              <a {% if entry.url %}href="{% if borgmatic.environment == "production" %}https://torsion.org/borgmatic{% else %}http://localhost:{{ borgmatic.port }}/borgmatic{% endif %}{{ entry.url | url }}"{% endif %}>{{ entry.title }}</a>
             {%- if entry.children.length -%}
               <ul>
                 {%- for child in entry.children %}{{ renderNavListItem(child) }}{% endfor -%}

docs/_includes/snippet/command-line/sample.md

@@ -1,5 +1,5 @@
 Here's the command-line help for this action in the [most recent version of
 borgmatic](https://projects.torsion.org/borgmatic-collective/borgmatic/releases).
-If you're using an older version, some of these flags may not work, and you
-should instead run the action with `--help` to see the flags specific to your
-borgmatic version.
+If you're using an older version, some of these flags may not work (or the action
+may be missing entirely). You should instead run the action with `--help` to see
+the flags and actions specific to your borgmatic version.

docs/docker-compose.yaml

@@ -2,10 +2,12 @@ services:
   traefik:
     image: public.ecr.aws/docker/library/traefik:3.5.3
     container_name: borgmatic-docs-traefik
+    environment:
+      - PORT=${PORT:-8080}
     command:
       - "--global.checkNewVersion=false"
       - "--global.sendAnonymousUsage=false"
-      - "--entrypoints.web.address=:8080"
+      - "--entrypoints.web.address=:${PORT}"
       - "--accesslog"
       - "--accesslog.fields.headers.defaultmode=keep"
       - "--providers.docker"
@@ -14,7 +16,7 @@ services:
       - "--api.dashboard=false"
       - "--log.level=WARN"
     ports:
-      - "127.0.0.1:8080:8080"
+      - "127.0.0.1:${PORT}:${PORT}"
     volumes:
       - ${CONTAINER_SOCKET_PATH:-/run/user/docker.sock}:/var/run/docker.sock:ro
   docs:
@@ -37,6 +39,10 @@ services:
       context: ..
       args:
         ENVIRONMENT: development
+        PORT: ${PORT:-8080}
+        FONT: ${FONT}
+    environment:
+      - PORT=${PORT:-8080}
   message:
     image: alpine
     container_name: borgmatic-docs-message
@@ -44,6 +50,6 @@ services:
       - sh
       - -c
       - |
-        echo; echo "You can view dev docs at http://localhost:8080/borgmatic/"; echo
+        echo; echo "You can view dev docs at http://localhost:${PORT}/borgmatic/"; echo
     depends_on:
       - docs

docs/fetch-contributors

@@ -14,7 +14,7 @@ import requests
 
 def list_merged_pulls(url):
     '''
-    Given a Gitea or GitHub API endpoint URL for pull requests, fetch and return the corresponding
+    Given a Forgejo or GitHub API endpoint URL for pull requests, fetch and return the corresponding
     JSON for all such merged pull requests.
     '''
     response = requests.get(f'{url}?state=closed', headers={'Accept': 'application/json', 'Content-Type': 'application/json'})
@@ -36,7 +36,6 @@ def list_contributing_issues(url):
 
 PULLS_API_ENDPOINT_URLS = (
     'https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/pulls',
-    'https://api.github.com/repos/borgmatic-collective/borgmatic/pulls',
 )
 ISSUES_API_ENDPOINT_URL = 'https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/issues?state=all'
 RECENT_CONTRIBUTORS_CUTOFF_DAYS = 365

docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server.md

@@ -82,7 +82,7 @@ before_actions:
 option in the `hooks:` section of your configuration.
 
 <span class="minilink minilink-addedin">Prior to version 1.7.0</span> Use
-`before_create` or similar instead of `before_actions`, which was introduced in
+`before_backup` or similar instead of `before_actions`, which was introduced in
 borgmatic 1.7.0.
 
 What this does is check if the `findmnt` command errors when probing for a

docs/how-to/customize-warnings-and-errors.md

@@ -12,10 +12,11 @@ decide how to respond. By default, Borg errors (and some warnings) result
 in a borgmatic error, while Borg successes don't.
 
 <span class="minilink minilink-addedin">New in borgmatic version 2.1.0</span>
-borgmatic elevates most Borg warnings to errors by default. For instance, if a
-source directory is missing during backup, Borg indicates that with a warning
-exit code (`107`). And starting in borgmatic 2.1.0, that exit code is considered
-an error, so you'll actually find out about missing files.
+borgmatic elevates several Borg warnings to errors by default. For instance, if
+borgmatic doesn't have permission to read a configured source directory during
+backup, Borg indicates that with a warning exit code (`105`). And starting in
+borgmatic 2.1.0, that exit code is considered an error, so you'll actually find
+out about files that borgmatic can't read.
 
 <span class="minilink minilink-addedin">With Borg version 1.4+</span> If the
 default behavior isn't sufficient for your needs, you can customize how
@@ -23,7 +24,7 @@ borgmatic interprets [Borg's exit
 codes](https://borgbackup.readthedocs.io/en/stable/internals/frontends.html#message-ids).
 
 For instance, this borgmatic configuration elevates a Borg warning about source files
-changes during backup (exit code `100`)—and only those warnings—to
+changing during backup (exit code `100`)—and only those warnings—to
 errors:
 
 ```yaml
@@ -32,14 +33,15 @@ borg_exit_codes:
       treat_as: error
 ```
 
-The following configuration does that *and* treats Borg's backup file not found
-(exit code `107`) as a warning:
+The following configuration does that *and* squashes errors about Borg
+encountering file permissions issues during backup (exit code `105`) to
+warnings.
 
 ```yaml
 borg_exit_codes:
     - code: 100
       treat_as: error
-    - code: 107
+    - code: 105
       treat_as: warning
 ```
 
@@ -54,8 +56,11 @@ is not found:
 terminating with warning status, rc 107
 ```
 
-So if you want to configure borgmatic to treat this as an warning instead of an
-error, the exit status to use is `107`.
+So if you want to configure borgmatic's interpretation of this warning, the exit
+status to use is `107`. Note however that in the particular case of missing
+files, there's a separate [`source_directories_must_exist`
+option](https://torsion.org/borgmatic/reference/configuration/#source_directories_must_exist-option)
+that can catch such problems before Borg even runs.
 
 <span class="minilink minilink-addedin">With Borg version 1.2 and earlier</span>
 Older versions of Borg didn't support granular exit codes, but still

docs/how-to/deal-with-very-large-backups.md

@@ -60,7 +60,7 @@ follows:
 unsafe_skip_path_validation_before_create: true
 ```
 
-However, this is indeed unsafe, and could lead to hangs or data being left out
+However, this is indeed unsafe and could lead to hangs or data being left out
 of backups. Use this option at your own risk.
 
 

docs/how-to/develop-on-borgmatic.md

@@ -31,7 +31,7 @@ changes work:
 ```bash
 cd borgmatic
 uv tool update-shell
-uv tool install --editable .
+uv tool install --editable .[dev]
 ```
 
 Or to work on the [Apprise
@@ -39,7 +39,14 @@ hook](https://torsion.org/borgmatic/reference/configuration/monitoring/apprise/)
 change that last line to:
 
 ```bash
-uv tool install --editable .[Apprise]
+uv tool install --editable .[dev,Apprise]
+```
+
+Or to work on the [browse
+action](https://torsion.org/borgmatic/reference/command-line/actions/browse/):
+
+```bash
+uv tool install --editable .[dev,browse]
 ```
 
 To get oriented with the borgmatic source code, have a look at the [source
@@ -190,8 +197,28 @@ This requires Docker (or Podman; see below) to be installed on your system.
 This script assumes you have permission to run `docker`. If you don't, then
 you may need to run with `sudo`.
 
+### How to choose a different port
+
+You can choose a different listening port in two ways:
+
+#### 1. Modify the `.env` file
+
+1.  Open `docs/.env`.
+2.  Change `PORT=8080` to your desired port number (e.g., `PORT=3000`).
+3.  Run the development script: `scripts/dev-docs`.
+
+#### 2. Use an environment variable
+
+Alternatively, you can override the port directly from your terminal without
+modifying any files:
+
+```bash
+PORT=3000 ./scripts/dev-docs
+```
+
 After you run the script, you can point your web browser at
-http://localhost:8080/borgmatic/ to view the documentation with your changes.
+http://localhost:8080/borgmatic/ (or your chosen port) to view the documentation
+with your changes.
 
 To close the documentation server, ctrl-C the script. Note that it does not
 currently auto-reload, so you'll need to stop it and re-run it for any
@@ -205,7 +232,24 @@ borgmatic's developer build for documentation optionally supports using
 [Podman](https://podman.io/) instead of Docker.
 
 Setting up Podman is outside the scope of this documentation. But once you
-install and configure Podman, then `scripts/dev-docs` should automatically use
-Podman instead of Docker.
+install and configure Podman, then `scripts/dev-docs` automatically uses Podman
+instead of Docker and [Podman
+Compose](https://github.com/containers/podman-compose) (if present) instead of
+Docker Compose. However Podman works fine with either Podman Compose or Docker
+Compose.
+
+
+## Use of generative AI
+
+Please do not use AI agents to modify this codebase. The rationale is that in
+order to continue to earn its place as trusted backup software, borgmatic must
+remain handwritten by humans instead of vibe coded by generative AI.
+
+Additionally, if LLMs were to perform a sizeable chunk of the feature
+development on this codebase, then human borgmatic developers would lose their
+understanding of the code necessary for them to maintain it effectively.
+
+Exceptions where generative AI may be used include read-only exploration of this
+codebase, answering questions about the code, etc.
 
 </span>

docs/how-to/inspect-your-backups.md

@@ -116,3 +116,71 @@ By default, borgmatic only logs to the console. But to enable simultaneous
 syslog or file logging, see the [logging
 documentation](https://torsion.org/borgmatic/reference/command-line/logging/)
 for details.
+
+## Finding differences between two archives
+
+<span class="minilink minilink-addedin">New in borgmatic version
+2.1.3</span>You can compare differences between two archives. For example:
+
+```bash
+borgmatic diff --archive latest --second-archive host-2023-01-02T04:06:07.080910
+```
+
+This shows the differences (file contents, user/group/mode) between the latest
+archive and the second one supplied.
+
+Note that, by default, `borgmatic diff` compares everything in the archives; that
+is, patterns are _not_ taken into consideration. If you require this, supply the
+`--only-patterns` flag.
+
+See the [Borg](https://borgbackup.readthedocs.io/en/stable/usage/diff.html)
+documentation for information on output format, what is compared, and more.
+
+
+## Browsing backups
+
+<span class="minilink minilink-addedin">New in version 2.1.6</span> <span
+class="minilink minilink-addedin">Experimental feature</span> borgmatic has an
+experimental console UI for browsing your repositories, archives, and files.
+Here's what it looks like:
+
+<img src="https://torsion.org/borgmatic/static/browse.png" alt="borgmatic browse screenshot" style="width: 100%">
+
+This feature is not intended to be a general-purpose Borg UI with every
+borgmatic feature, but rather it's for use cases like quickly looking at the
+contents of your backups when you're feeling too lazy to type out a full
+borgmatic command-line.
+
+Depending on how you installed borgmatic, it may not have come with the
+necessary Python libraries to support the browse action. (borgmatic's
+[stand-alone
+binary](https://projects.torsion.org/borgmatic-collective/borgmatic/releases)
+does not currently include them.) If you originally [installed borgmatic with
+uv](https://torsion.org/borgmatic/how-to/install-borgmatic/), run the following
+to install the libraries needed for the browse action:
+
+```bash
+sudo uv tool install borgmatic[browse]
+```
+
+Omit `sudo` if borgmatic is installed as a non-root user.
+
+Once the libraries are installed, run the following to access the browse action:
+
+```bash
+borgmatic browse
+```
+
+This launches a console UI where you can select a borgmatic configuration file
+(if there's more than one), select a Borg repository, select an archive in that
+repository, and even browse the backed up files in that archive.
+
+Use the keyboard or the mouse to navigate the UI. The footer at the bottom of
+the screen shows some of the available keys. Logs show up directly in the UI at
+the [selected
+verbosity](https://torsion.org/borgmatic/reference/command-line/logging/),
+although logs are hidden by default.
+
+Please [provide
+feedback](https://torsion.org/borgmatic/#support-and-contributing) if you find
+this feature useful—or even if you don't, but would like it to become useful.

docs/how-to/install-borgmatic.md

@@ -0,0 +1,108 @@
+---
+title: 📥 How to install borgmatic
+eleventyNavigation:
+  key: 📥 Install borgmatic
+  parent: How-to guides
+  order: -1
+---
+
+
+### Prerequisites
+
+Before installing borgmatic, first [install
+Borg](https://borgbackup.readthedocs.io/en/stable/installation.html), at least
+version 1.1. (borgmatic does not install Borg automatically so as to avoid
+conflicts with existing Borg installations.)
+
+Then, [install uv](https://docs.astral.sh/uv/getting-started/installation/) as
+the root user (with `sudo`) to make installing borgmatic easier without
+impacting other Python applications on your system. For Debian, there is a
+[third-party package for
+uv](https://dario.griffo.io/posts/how-to-install-uv-debian/). On Ubuntu, there
+is a [snap package](https://snapcraft.io/install/astral-uv/ubuntu). On Arch, you
+can just install the `python-uv` package.
+
+
+### Root install
+
+If you want borgmatic to run with privileged access so it can backup your system
+files, then install borgmatic as the root user by running the following
+commands:
+
+```bash
+sudo uv tool update-shell
+sudo uv tool install borgmatic
+```
+
+Check whether this worked with:
+
+```bash
+sudo su -
+borgmatic --version
+```
+
+If borgmatic is properly installed, that should output your borgmatic version.
+And if you'd also like `sudo borgmatic` to work as well, keep reading!
+
+
+### Non-root install
+
+If you only want to run borgmatic as a non-root user (without privileged file
+access) *or* you want to make `sudo borgmatic` work so borgmatic runs as root,
+then install borgmatic as a non-root user by running the following commands as
+that user:
+
+```bash
+uv tool update-shell
+uv tool install borgmatic
+```
+
+This should work even if you've also installed borgmatic as the root user.
+
+Check whether this worked with:
+
+```bash
+borgmatic --version
+```
+
+If borgmatic is properly installed, that should output your borgmatic version.
+You can also try `sudo borgmatic --version` if you intend to run borgmatic
+with `sudo`. If that doesn't work, you may need to update your [sudoers
+`secure_path` option](https://wiki.archlinux.org/title/Sudo).
+
+
+### Other ways to install
+
+Besides the approaches described above, there are several other options for
+installing borgmatic:
+
+#### <span data-pagefind-weight="7.0">Docker / Podman</span>
+
+ * [container image with scheduled backups](https://github.com/borgmatic-collective/docker-borgmatic) (+ Docker Compose files)
+ * [container image with multi-arch and Docker CLI support](https://github.com/modem7/docker-borgmatic)
+ * [Borgmatic Director UI](https://github.com/SpeedbitsInfinityTools/borgmatic-ui-community)
+
+#### Operating system packages
+
+ * [Debian](https://tracker.debian.org/pkg/borgmatic)
+ * [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
+ * [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
+ * [Gentoo](https://packages.gentoo.org/packages/app-backup/borgmatic)
+ * [Arch Linux](https://archlinux.org/packages/extra/any/borgmatic/)
+ * [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=borgmatic)
+ * [OpenBSD](https://openports.pl/path/sysutils/borgmatic)
+ * [openSUSE](https://software.opensuse.org/package/borgmatic)
+ * [macOS (via Homebrew)](https://formulae.brew.sh/formula/borgmatic)
+ * [macOS (via MacPorts)](https://ports.macports.org/port/borgmatic/)
+ * [NixOS](https://search.nixos.org/packages?channel=unstable&show=borgmatic&query=borgmatic)
+
+#### Etc.
+
+ * [stand-alone Linux binary](https://projects.torsion.org/borgmatic-collective/borgmatic/releases) (This is a beta feature! The minimal binary omits [Apprise support](https://torsion.org/borgmatic/reference/configuration/monitoring/apprise/).)
+ * [Ansible role](https://github.com/borgbase/ansible-role-borgbackup)
+ * [pipx](https://pipx.pypa.io/stable/)
+
+
+## Next steps
+
+* [Set up backups](https://torsion.org/borgmatic/how-to/set-up-backups/)

docs/how-to/monitor-your-backups.md

@@ -64,6 +64,27 @@ suppressed so as not to interfere with the captured JSON. Also note that JSON
 output only shows up at the console and not in syslog.
 
 
+### Getting configuration
+
+<span class="minilink minilink-addedin">New in version 2.1.3</span> If you want
+to consume borgmatic's computed configuration in your scripts, use the [`config
+show`
+action](https://torsion.org/borgmatic/reference/command-line/actions/config-show/).
+Here's an example:
+
+```bash
+borgmatic config show --json
+```
+
+That outputs borgmatic's entire configuration as JSON with one array element per
+configuration file.
+
+Or you can ask for the value of a particular option:
+
+```bash
+borgmatic config show --option repositories --json
+```
+
 ### Latest backups
 
 All borgmatic actions that accept an `--archive` flag allow you to specify an

docs/how-to/set-up-backups.md

@@ -1,118 +1,27 @@
 ---
-title: 📥 How to set up backups
+title: 📋 How to set up backups
 eleventyNavigation:
-  key: 📥 Set up backups
+  key: 📋 Set up backups
   parent: How-to guides
   order: 0
 ---
 
-To install borgmatic, first [install
-Borg](https://borgbackup.readthedocs.io/en/stable/installation.html), at least
-version 1.1. (borgmatic does not install Borg automatically so as to avoid
-conflicts with existing Borg installations.)
-
-Then, [install pipx](https://pypa.github.io/pipx/installation/) as the root
-user (with `sudo`) to make installing borgmatic easier without impacting other
-Python applications on your system. If you have trouble installing pipx with
-pip, then you can install a system package instead. E.g. on Ubuntu or Debian,
-run:
-
-```bash
-sudo apt update
-sudo apt install pipx
-```
-
-### Root install
-
-If you want to run borgmatic on a schedule with privileged access to your
-files, then you should install borgmatic as the root user by running the
-following commands:
-
-```bash
-sudo pipx ensurepath
-sudo pipx install borgmatic
-```
-
-Check whether this worked with:
-
-```bash
-sudo su -
-borgmatic --version
-```
-
-If borgmatic is properly installed, that should output your borgmatic version.
-And if you'd also like `sudo borgmatic` to work, keep reading!
-
-
-### Non-root install
-
-If you only want to run borgmatic as a non-root user (without privileged file
-access) *or* you want to make `sudo borgmatic` work so borgmatic runs as root,
-then install borgmatic as a non-root user by running the following commands as
-that user:
-
-```bash
-pipx ensurepath
-pipx install borgmatic
-```
-
-This should work even if you've also installed borgmatic as the root user.
-
-Check whether this worked with:
-
-```bash
-borgmatic --version
-```
-
-If borgmatic is properly installed, that should output your borgmatic version.
-You can also try `sudo borgmatic --version` if you intend to run borgmatic
-with `sudo`. If that doesn't work, you may need to update your [sudoers
-`secure_path` option](https://wiki.archlinux.org/title/Sudo).
-
-
-### Other ways to install
-
-Besides the approaches described above, there are several other options for
-installing borgmatic:
-
-#### <span data-pagefind-weight="7.0">Docker / Podman</span>
-
- * [container image with scheduled backups](https://github.com/borgmatic-collective/docker-borgmatic) (+ Docker Compose files)
- * [container image with multi-arch and Docker CLI support](https://github.com/modem7/docker-borgmatic)
-
-#### Operating system packages
-
- * [Debian](https://tracker.debian.org/pkg/borgmatic)
- * [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
- * [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
- * [Gentoo](https://packages.gentoo.org/packages/app-backup/borgmatic)
- * [Arch Linux](https://archlinux.org/packages/extra/any/borgmatic/)
- * [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=borgmatic)
- * [OpenBSD](https://openports.pl/path/sysutils/borgmatic)
- * [openSUSE](https://software.opensuse.org/package/borgmatic)
- * [macOS (via Homebrew)](https://formulae.brew.sh/formula/borgmatic)
- * [macOS (via MacPorts)](https://ports.macports.org/port/borgmatic/)
- * [NixOS](https://search.nixos.org/packages?channel=unstable&show=borgmatic&query=borgmatic)
-
-#### Etc.
-
- * [Ansible role](https://github.com/borgbase/ansible-role-borgbackup)
- * [uv tool install](https://docs.astral.sh/uv/)
-
+Start by [installing
+borgmatic](https://torsion.org/borgmatic/how-to/install-borgmatic/) if you haven't
+already.
 
 ## Hosting providers
 
 Need somewhere to store your encrypted off-site backups? The following hosting
-providers include specific support for Borg/borgmatic—and fund borgmatic
-development and hosting when you use these referral links to sign up:
+provider includes specific support for Borg/borgmatic—and funds borgmatic
+development and hosting when you use this referral links to sign up:
 
 <ul>
  <li class="referral"><a href="https://www.borgbase.com/?utm_source=borgmatic">BorgBase</a>: Borg hosting service with support for monitoring, 2FA, and append-only repos</li>
- <li class="referral"><a href="https://hetzner.cloud/?ref=v9dOJ98Ic9I8">Hetzner</a>: A "storage box" that includes support for Borg</li>
 </ul>
 
-Additionally, rsync.net has a compatible storage offering, but does not fund
-borgmatic development or hosting.
+Additionally, Hetzner and rsync\.net have compatible storage offerings, but do
+not fund borgmatic development or hosting.
 
 
 ## Configuration
@@ -322,7 +231,7 @@ If you're using systemd instead of cron to run jobs, you can still configure
 borgmatic to run automatically.
 
 (If you installed borgmatic from [Other ways to
-install](https://torsion.org/borgmatic/how-to/set-up-backups/#other-ways-to-install),
+install](https://torsion.org/borgmatic/how-to/install-borgmatic/#other-ways-to-install),
 you may already have borgmatic systemd service and timer files. If so, you may
 be able to skip some of the steps below.)
 

docs/how-to/upgrade.md

@@ -5,10 +5,25 @@ eleventyNavigation:
   parent: How-to guides
   order: 14
 ---
-In general, all you should need to do to upgrade borgmatic if you've
-[installed it with
-pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation)
-is to run the following:
+In general, all you should need to do to upgrade borgmatic if you've [installed
+it with uv](https://docs.astral.sh/uv/) is to run the following:
+
+```bash
+sudo uv tool upgrade borgmatic
+```
+
+Omit `sudo` if you installed borgmatic as a non-root user. And if you
+installed borgmatic *both* as root and as a non-root user, you'll need to
+upgrade each installation independently.
+
+
+### Upgrading from other installation methods
+
+#### pipx
+
+If you have borgmatic installed with
+[pipx](https://pipx.pypa.io/latest/installation/), and you'd like to continue
+using pipx, then you can upgrade borgmatic with:
 
 ```bash
 sudo pipx upgrade borgmatic
@@ -18,16 +33,30 @@ Omit `sudo` if you installed borgmatic as a non-root user. And if you
 installed borgmatic *both* as root and as a non-root user, you'll need to
 upgrade each installation independently.
 
+But if you'd like to switch your borgmatic install from pipx to
+[uv](https://docs.astral.sh/uv/), uninstall borgmatic with pipx (`sudo pipx
+uninstall borgmatic`) and then [install borgmatic with
+uv](https://torsion.org/borgmatic/how-to/install-borgmatic/).
+
+Either one should work just fine. uv is just faster than pipx and also used for
+borgmatic
+[development](https://torsion.org/borgmatic/how-to/develop-on-borgmatic/).
+
+
+#### pip install
+
 If you originally installed borgmatic with `sudo pip3 install --user`, you can
 uninstall it first with `sudo pip3 uninstall borgmatic` and then [install it
 again with
-pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation),
+uv](https://torsion.org/borgmatic/how-to/install-borgmatic/),
 which should better isolate borgmatic from your other Python applications.
 
-But if you [installed borgmatic without pipx or
-pip3](https://torsion.org/borgmatic/how-to/set-up-backups/#other-ways-to-install),
-then your upgrade method may be different.
 
+#### Etc.
+
+If you installed borgmatic [some other
+way](https://torsion.org/borgmatic/how-to/install-borgmatic/#other-ways-to-install),
+then your upgrade method may be different.
 
 
 ### Upgrading your configuration

docs/reference/command-line/actions/browse.md

@@ -0,0 +1,17 @@
+---
+title: browse
+eleventyNavigation:
+  key: browse
+  parent: 🎬 Actions
+---
+
+<span class="minilink minilink-addedin">Experimental feature</span> {% include snippet/command-line/sample.md %}
+
+```
+{% include borgmatic/command-line/browse.txt %}
+```
+
+
+## Related documentation
+
+ * [Inspect your backups](https://torsion.org/borgmatic/how-to/inspect-your-backups/)

docs/reference/command-line/actions/config-show.md

@@ -0,0 +1,17 @@
+---
+title: config show
+eleventyNavigation:
+  key: config show
+  parent: 🎬 Actions
+---
+
+{% include snippet/command-line/sample.md %}
+
+```
+{% include borgmatic/command-line/config-show.txt %}
+```
+
+
+## Related documentation
+
+ * [Scripting borgmatic](https://torsion.org/borgmatic/how-to/monitor-your-backups/#scripting-borgmatic)

docs/reference/command-line/actions/diff.md

@@ -0,0 +1,16 @@
+---
+title: diff
+eleventyNavigation:
+  key: diff
+  parent: 🎬 Actions
+---
+
+{% include snippet/command-line/sample.md %}
+
+```
+{% include borgmatic/command-line/diff.txt %}
+```
+
+## Related documentation
+
+* [Finding differences](https://torsion.org/borgmatic/how-to/inspect-your-backups/#finding-differences-between-two-archives)

docs/reference/configuration/archive-name-format.md

@@ -97,4 +97,3 @@ option for limiting the archives used for the `check` action was a separate
 `prefix` in the `consistency` section. Both of these options are deprecated in
 favor of the auto-matching behavior (or `match_archives`/`--match-archives`)
 in newer versions of borgmatic.
-

docs/reference/configuration/command-hooks.md

@@ -12,11 +12,15 @@ list of `commands:` in your borgmatic configuration file. For example:
 ```yaml
 commands:
     - before: action
-      when: [create]
+      when: [check]  # This is an inline YAML sequence.
       run:
           - echo "Before create!"
+    - before: action
+      when: [create, prune]  # Also an inline YAML sequence.
+      run:
+          - echo "Before create or prune!"
     - after: action
-      when:
+      when:  # Multi-line YAML sequence, equivalent to "[create, prune]".
           - create
           - prune
       run:
@@ -29,7 +33,7 @@ commands:
 Each command in the `commands:` list has the following options:
 
  * `before` or `after`: Name for the point in borgmatic's execution that the commands should be run before or after, one of:
-    * `action` runs before or after each action for each repository. This replaces the deprecated `before_create`, `after_prune`, etc.
+    * `action` runs before or after each action for each repository. This replaces the deprecated `before_backup`, `after_prune`, etc.
     * `repository` runs before or after all actions for each repository. This replaces the deprecated `before_actions` and `after_actions`.
     * `configuration` runs before or after all actions and repositories in the current configuration file.
     * `everything` runs before or after all configuration files. Errors here do not trigger `error` hooks or the `fail` state in monitoring hooks. This replaces the deprecated `before_everything` and `after_everything`.

docs/reference/configuration/credentials/keepassxc.md

@@ -43,14 +43,15 @@ postgresql_databases:
 ```
 
 
-### Custom command
+### Custom commands
 
-You can also optionally override the `keepassxc-cli` command that borgmatic calls to load
-passwords:
+You can also optionally override the `keepassxc-cli` or `secret-tool` commands
+that borgmatic call to load passwords:
 
 ```yaml
 keepassxc:
     keepassxc_cli_command: /usr/local/bin/keepassxc-cli
+    secret_tool_command: /usr/local/bin/secret-tool
 ```
 
 Another example:
@@ -98,3 +99,30 @@ keepassxc:
 The value here is the YubiKey slot number (e.g., `1` or `2`) and optional serial
 number (e.g., `7370001`) used to access the KeePassXC database. Join the two
 values with a colon, but omit the colon if you're leaving out the serial number.
+
+
+### Secret service integration
+
+<span class="minilink minilink-addedin">New in version 2.1.6</span> borgmatic
+supports [KeePassXC's secret service
+integration](https://keepassxc.org/docs/KeePassXC_UserGuide#_secret_service_integration)
+that integrates with the [freedesktop secret service
+API](https://specifications.freedesktop.org/secret-service/latest/) and allows
+clients like borgmatic to access your passwords.
+
+To use this feature from borgmatic, specify `secret-service` instead of a
+KeePassXC database path when calling this credential hook. For instance:
+
+```yaml
+encryption_passphrase: "{credential keepassxc secret-service borgmatic}"
+```
+
+With this in place, borgmatic runs libsecret's `secret-tool` command to retrieve
+the password titled "borgmatic" on demand. KeePassXC may then prompt you to
+approve the password access, depending on how you've configured it.
+
+One benefit of using the KeePassXC's secret service integration like this is
+that you don't have to type a KeePassXC database passhprase (or use a keyfile)
+whenever borgmatic accesses your passwords. Instead, you can configure
+KeePassXC to prompt you to approve or deny each access. Or you can even
+pre-approve password access to support automated borgmatic runs.

docs/reference/configuration/data-sources/mariadb.md

@@ -16,10 +16,51 @@ mariadb_databases:
 ```
 
 
-### Full configuration
+## System databases
+
+<span class="minilink minilink-addedin">New in version 2.1.6</span> When dumping
+["all"
+databases](https://torsion.org/borgmatic/how-to/backup-your-databases/#all-databases),
+borgmatic excludes most data coming from [MariaDB system
+databases](https://mariadb.com/docs/server/reference/system-tables),
+because much of it is populated on MariaDB startup and thus not restorable (or
+just unnecessary to backup).
+
+The system data that borgmatic does include in these dumps are: users, roles,
+grants, user-defined functions, and remote servers—all from the `mysql` system
+database. This omits all other data from the `mysql` database, which includes
+index and table statistics, time zones, and installed server plugins. It also
+excludes the separate `information_schema`, `performance_schema`, and `sys`
+system databases. This is not currently configurable.
+
+Within a Borg archive, you can find this data stored in a dump named `mysql`—the
+name of the system table this data comes from. And if you'd like to dump this
+data without having to dump "all" databases, then you can configure a database
+named `mysql` in your borgmatic configuration. For example:
+
+```yaml
+mariadb_databases:
+    - name: mysql
+```
+
+Even in this case though, only the subset of system data described above is
+included in the dump.
+
+<span class="minilink minilink-addedin">Prior to version 2.1.6</span> Dumps of
+"all" databases excluded system databases and all of their data. Additionally,
+explicitly dumping `mysql` was treated like any other database—and thus wasn't
+easily restorable.
+
+
+## Full configuration
 
 {% include snippet/configuration/sample.md %}
 
 ```yaml
 {% include borgmatic/mariadb_databases.yaml %}
 ```
+
+
+## Related documentation
+
+ * [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)