diff --git a/docs/_includes/index.css b/docs/_includes/index.css
index c798f2e9..a81293da 100644
--- a/docs/_includes/index.css
+++ b/docs/_includes/index.css
@@ -112,7 +112,7 @@ h5 {
}
h1 {
font-size: 2.666666666667em; /* 48px /18 */
- margin: 0 0 .5em;
+ margin: 0;
}
main .elv-toc + h1 {
margin-top: 1em;
@@ -229,7 +229,7 @@ pre + .note {
/* Layout */
.elv-layout {
- padding: 1rem;
+ padding: 0 1rem 1rem;
margin: 0 auto;
max-width: 42rem;
clear: both;
diff --git a/docs/_includes/layouts/base.njk b/docs/_includes/layouts/base.njk
index 0807f5f0..994598a0 100644
--- a/docs/_includes/layouts/base.njk
+++ b/docs/_includes/layouts/base.njk
@@ -4,8 +4,8 @@
-
- {{ subtitle + ' - ' if subtitle}}{{ title }}
+
+ borgmatic - {{ subtitle + ' - ' if subtitle}}{{ title }}
{%- set css %}
{% include 'index.css' %}
{% include 'components/lists.css' %}
@@ -14,7 +14,6 @@
{% include 'components/toc.css' %}
{% include 'components/info-blocks.css' %}
{% include 'prism-theme.css' %}
-{% include 'asciinema.css' %}
{% endset %}
{% if feedTitle and feedUrl %}
diff --git a/docs/_includes/layouts/main.njk b/docs/_includes/layouts/main.njk
index 81300bae..f9e06dd9 100644
--- a/docs/_includes/layouts/main.njk
+++ b/docs/_includes/layouts/main.njk
@@ -11,7 +11,7 @@ headerClass: elv-header-default
{% set navPages = collections.all | eleventyNavigation %}
{% macro renderNavListItem(entry) -%}
- {{ entry.title }}
+ {{ entry.title }}
{%- if entry.children.length -%}
{%- for child in entry.children %}{{ renderNavListItem(child) }}{% endfor -%}
diff --git a/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md b/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md
index e2dd7361..00f04e3f 100644
--- a/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md
+++ b/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 10
---
-## Preparation and cleanup hooks
-
If you find yourself performing preparation tasks before your backup runs or
doing cleanup work afterwards, borgmatic command hooks may be of interest. These
are custom shell commands you can configure borgmatic to execute at various
@@ -14,7 +12,7 @@ points as it runs.
(But if you're looking to backup a database, it's probably easier to use the
[database backup
-feature](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/)
+feature](https://torsion.org/borgmatic/how-to/backup-your-databases/)
instead.)
New in version 2.0.0 Command
@@ -40,7 +38,7 @@ commands:
If you're coming from an older version of borgmatic, there is tooling to help
you [upgrade your
-configuration](https://torsion.org/borgmatic/docs/how-to/upgrade/#upgrading-your-configuration)
+configuration](https://torsion.org/borgmatic/how-to/upgrade/#upgrading-your-configuration)
to this new command hook format.
Note that if a `run:` command contains a special YAML character such as a colon,
@@ -315,14 +313,14 @@ commands:
```
Note that you can also interpolate [arbitrary environment
-variables](https://torsion.org/borgmatic/docs/how-to/provide-your-passwords/).
+variables](https://torsion.org/borgmatic/how-to/provide-your-passwords/).
## Hook output
Any output produced by your hooks shows up both at the console and in syslog
(when enabled). For more information, read about inspecting
+href="https://torsion.org/borgmatic/how-to/inspect-your-backups/">inspecting
your backups.
diff --git a/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server.md b/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server.md
index 4517ee8b..8c601b25 100644
--- a/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server.md
+++ b/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 11
---
-## Occasional backups
-
A common situation is backing up to a repository that's only sometimes online.
For instance, you might send most of your backups to the cloud, but
occasionally you want to plug in an external hard drive or backup to your
@@ -28,7 +26,7 @@ concept of "soft failure" come in.
## Soft failure command hooks
This feature leverages [borgmatic command
-hooks](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/),
+hooks](https://torsion.org/borgmatic/how-to/add-preparation-and-cleanup-steps-to-backups/),
so familiarize yourself with them first. The idea is that you write a simple
test in the form of a borgmatic command hook to see if backups should proceed or
not.
@@ -42,7 +40,7 @@ If you return any status besides 75, then it's a standard success or error.
So for instance, if you have an external drive that's only sometimes mounted,
declare its repository in its own [separate configuration
-file](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/),
+file](https://torsion.org/borgmatic/how-to/make-per-application-backups/),
say at `/etc/borgmatic.d/removable.yaml`:
```yaml
@@ -137,7 +135,7 @@ There are some caveats you should be aware of with this feature.
* If you're writing a soft failure script that you want to vary based on the
current repository, for instance so you can have multiple repositories in a
single configuration file, have a look at [command hook variable
- interpolation](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/#variable-interpolation).
+ interpolation](https://torsion.org/borgmatic/how-to/add-preparation-and-cleanup-steps-to-backups/#variable-interpolation).
And there's always still the option of putting anything that you don't want
soft-failed (like always-online cloud backups) in separate configuration
files from your soft-failing repositories.
diff --git a/docs/how-to/backup-your-databases.md b/docs/how-to/backup-your-databases.md
index 1256a05b..0caf775f 100644
--- a/docs/how-to/backup-your-databases.md
+++ b/docs/how-to/backup-your-databases.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 8
---
-## Database dump hooks
-
If you want to backup a database, it's best practice with most database
systems to backup an exported database dump, rather than backing up your
database's internal file storage. That's because the internal storage can
@@ -110,7 +108,7 @@ sqlite_databases:
```
See your [borgmatic configuration
-file](https://torsion.org/borgmatic/docs/reference/configuration/) for
+file](https://torsion.org/borgmatic/reference/configuration/) for
additional customization of the options passed to database commands (when
listing databases, restoring databases, etc.).
@@ -328,7 +326,7 @@ password to the temporary `pg_dump` container.
Similar command override options are available for (some of) the other
supported database types as well. See the [configuration
-reference](https://torsion.org/borgmatic/docs/reference/configuration/) for
+reference](https://torsion.org/borgmatic/reference/configuration/) for
details.
@@ -356,7 +354,7 @@ hooks:
If you don't want to keep your database passwords in your borgmatic
configuration file, you can instead pass them in [from external credential
-sources](https://torsion.org/borgmatic/docs/how-to/provide-your-passwords/).
+sources](https://torsion.org/borgmatic/how-to/provide-your-passwords/).
### Configuration backups
@@ -370,7 +368,7 @@ bring back any missing configuration files in order to restore a database.
New in version 1.7.15 borgmatic
automatically includes configuration files in your backup. See [the
documentation on the `config bootstrap`
-action](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/#extract-the-configuration-files-used-to-create-an-archive)
+action](https://torsion.org/borgmatic/how-to/extract-a-backup/#extract-the-configuration-files-used-to-create-an-archive)
for more information.
@@ -431,6 +429,20 @@ restoring dumps from the selected archive. So be very careful when and where
you run it.
+### Configuration file selection
+
+If you have a multi-configuration-file setup and you want to restore a database
+from a single configuration file, use the `--config` flag with the path of the
+configuration file to use for the restore. For example:
+
+```bash
+borgmatic restore --config /path/to/config.yaml --archive latest
+```
+
+Without `--config`, borgmatic tries to run the `restore` action once for each
+configuration file it finds.
+
+
### Repository selection
If you have a single repository in your borgmatic configuration file(s), no
@@ -590,7 +602,7 @@ postgresql_databases:
### Manual restoration
If you prefer to restore a database without the help of borgmatic, first
-[extract](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/) an
+[extract](https://torsion.org/borgmatic/how-to/extract-a-backup/) an
archive containing a database dump.
borgmatic extracts the dump file into the `borgmatic/` directory within the
@@ -607,7 +619,7 @@ After extraction, you can manually restore the dump file using native database
commands like `pg_restore`, `mysql`, `mongorestore`, `sqlite`, or similar.
Also see the documentation on [listing database
-dumps](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/#listing-database-dumps).
+dumps](https://torsion.org/borgmatic/how-to/inspect-your-backups/#listing-database-dumps).
## Limitations
@@ -657,7 +669,7 @@ automatically.
If this database integration is too limited for needs, borgmatic also supports
general-purpose [preparation and cleanup
-hooks](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/).
+hooks](https://torsion.org/borgmatic/how-to/add-preparation-and-cleanup-steps-to-backups/).
These hooks allows you to trigger arbitrary commands or scripts before and
after backups. So if necessary, you can use these hooks to create database
dumps with any database system.
diff --git a/docs/how-to/customize-warnings-and-errors.md b/docs/how-to/customize-warnings-and-errors.md
index d20c3469..6f917f23 100644
--- a/docs/how-to/customize-warnings-and-errors.md
+++ b/docs/how-to/customize-warnings-and-errors.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 13
---
-## When things go wrong
-
After Borg runs, it indicates whether it succeeded via its exit code, a
numeric ID indicating success, warning, or error. borgmatic consumes this exit
code to decide how to respond. Normally, a Borg error results in a borgmatic
diff --git a/docs/how-to/deal-with-very-large-backups.md b/docs/how-to/deal-with-very-large-backups.md
index cf4490c3..2a90356a 100644
--- a/docs/how-to/deal-with-very-large-backups.md
+++ b/docs/how-to/deal-with-very-large-backups.md
@@ -5,7 +5,7 @@ eleventyNavigation:
parent: How-to guides
order: 4
---
-## Biggish data
+
Borg itself is great for efficiently de-duplicating data across successive
backup archives, even when dealing with very large repositories. But you may
@@ -14,367 +14,29 @@ and `check` works well on small repositories, it's not so great on larger
ones. That's because running the default pruning, compact, and consistency
checks take a long time on large repositories.
-Prior to version 1.7.9 The
-default action ordering was `prune`, `compact`, `create`, and `check`.
+See the [actions
+documentation](https://torsion.org/borgmatic/reference/command-line/actions/)
+for details on customizing the actions that borgmatic runs.
-### A la carte actions
-If you find yourself wanting to customize the actions, you have some options.
-First, you can run borgmatic's `create`, `prune`, `compact`, or `check`
-actions separately. For instance, the following optional actions are
-available (among others):
-
-```bash
-borgmatic create
-borgmatic prune
-borgmatic compact
-borgmatic check
-```
-
-You can run borgmatic with only one of these actions provided, or you can mix
-and match any number of them in a single borgmatic run. This supports
-approaches like skipping certain actions while running others. For instance,
-this skips `prune` and `compact` and only runs `create` and `check`:
-
-```bash
-borgmatic create check
-```
-
-New in version 1.7.9 borgmatic
-now respects your specified command-line action order, running actions in the
-order you specify. In previous versions, borgmatic ran your specified actions
-in a fixed ordering regardless of the order they appeared on the command-line.
-
-But instead of running actions together, another option is to run backups with
-`create` on a frequent schedule (e.g. with `borgmatic create` called from one
-cron job), while only running expensive consistency checks with `check` on a
-much less frequent basis (e.g. with `borgmatic check` called from a separate
-cron job).
-
-New in version 1.8.5 Instead of
-(or in addition to) specifying actions on the command-line, you can configure
-borgmatic to [skip particular
-actions](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#skipping-actions).
+
+
+
+
+
### Consistency check configuration
-Another option is to customize your consistency checks. By default, if you
-omit consistency checks from configuration, borgmatic runs full-repository
-checks (`repository`) and per-archive checks (`archives`) within each
-repository, running the checks on a monthly basis. (See below about setting
-your own check frequency.)
-
-But if you find that archive checks are too slow, for example, you can
-configure borgmatic to run repository checks only. Configure this in the
-`consistency` section of borgmatic configuration:
-
-```yaml
-checks:
- - name: repository
-```
-
-Prior to version 1.8.0 Put
-this option in the `consistency:` section of your configuration.
-
-Prior to version 1.6.2 The
-`checks` option was a plain list of strings without the `name:` part, and
-borgmatic ran each configured check every time checks were run. For example:
-
-```yaml
-checks:
- - repository
-```
-
-
-Here are the available checks, roughly from fastest to slowest:
-
- * `archives`: Checks all of the archives' metadata in the repository.
- * `repository`: Checks the consistency of the whole repository. The checks run
- on the server and do not cause significant network traffic.
- * `extract`: Performs an extraction dry-run of the latest archive.
- * `data`: Verifies the data integrity of all archives contents, decrypting
- and decompressing all data.
- * `spot`: Compares file counts and contents between your source files and the
- latest archive.
-
-Note that the `data` check is a more thorough version of the `archives` check,
-so enabling the `data` check implicitly enables the `archives` check as well.
-
-See [Borg's check
-documentation](https://borgbackup.readthedocs.io/en/stable/usage/check.html)
-for more information.
-
-
-### Spot check
-
-The various consistency checks all have trade-offs around speed and
-thoroughness, but most of them don't even look at your original source
-files—arguably one important way to ensure your backups contain the files
-you'll want to restore in the case of catastrophe (or an accidentally deleted
-file). Because if something goes wrong with your source files, most
-consistency checks will still pass with flying colors and you won't discover
-there's a problem until you go to restore.
-
-New in version 1.8.10 That's
-where the spot check comes in. This check actually compares your source file
-counts and data against those in the latest archive, potentially catching
-problems like incorrect excludes, inadvertent deletes, files changed by
-malware, etc.
-
-But because an exhaustive comparison of all source files against the latest
-archive might be too slow, the spot check supports *sampling* a percentage of
-your source files for the comparison, ensuring they fall within configured
-tolerances.
-
-Here's how it works. Start by installing the `xxhash` OS package if you don't
-already have it, so the spot check can run the `xxh64sum` command and
-efficiently hash files for comparison. Then add something like the following
-to your borgmatic configuration:
-
-```yaml
-checks:
- - name: spot
- count_tolerance_percentage: 10
- data_sample_percentage: 1
- data_tolerance_percentage: 0.5
-```
-
-The `count_tolerance_percentage` is the percentage delta between the source
-directories file count and the latest backup archive file count that is
-allowed before the entire consistency check fails. For instance, if the spot
-check runs and finds 100 source files on disk and 105 files in the latest
-archive, that would be within the configured 10% count tolerance and the check
-would succeed. But if there were 100 source files and 200 archive files, the
-check would fail. (100 source files and only 50 archive files would also
-fail.)
-
-The `data_sample_percentage` is the percentage of total files in the source
-directories to randomly sample and compare to their corresponding files in the
-latest backup archive. A higher value allows a more accurate check—and a
-slower one. The comparison is performed by hashing the selected source files
-and counting hashes that don't match the latest archive. For instance, if you
-have 1,000 source files and your sample percentage is 1%, then only 10 source
-files will be compared against the latest archive. These sampled files are
-selected randomly each time, so in effect the spot check is probabilistic.
-
-The `data_tolerance_percentage` is the percentage of total files in the source
-directories that can fail a spot check data comparison without failing the
-entire consistency check. The value must be lower than or equal to the
-`data_sample_percentage`, because `data_tolerance_percentage` only looks at
-at the sampled files as determined by `data_sample_percentage`.
-
-All three options are required when using the spot check. And because the
-check relies on these configured tolerances, it may not be a
-set-it-and-forget-it type of consistency check, at least until you get the
-tolerances dialed in so there are minimal false positives or negatives. It is
-recommended you run `borgmatic check` several times after configuring the spot
-check, tweaking your tolerances as needed. For certain workloads where your
-source files experience wild swings of file contents or counts, the spot check
-may not suitable at all.
-
-What if you add, delete, or change a bunch of your source files and you don't
-want the spot check to fail the next time it's run? Run `borgmatic create` to
-create a new backup, thereby allowing the next spot check to run against an
-archive that contains your recent changes.
-
-Because the spot check only looks at the most recent archive, you may not want
-to run it immediately after a `create` action (borgmatic's default behavior).
-Instead, it may make more sense to run the spot check on a separate schedule
-from `create`.
-
-
-### Check frequency
-
-New in version 1.6.2 You can
-optionally configure checks to run on a periodic basis rather than every time
-borgmatic runs checks. For instance:
-
-```yaml
-checks:
- - name: repository
- frequency: 2 weeks
- - name: archives
- frequency: 1 month
-```
-
-Prior to version 1.8.0 Put
-this option in the `consistency:` section of your configuration.
-
-This tells borgmatic to run the `repository` consistency check at most once
-every two weeks for a given repository and the `archives` check at most once a
-month. The `frequency` value is a number followed by a unit of time, e.g. `3
-days`, `1 week`, `2 months`, etc. The set of possible time units is as
-follows (singular or plural):
-
- * `second`
- * `minute`
- * `hour`
- * `day`
- * `week` (7 days)
- * `month` (30 days)
- * `year` (365 days)
-
-The `frequency` defaults to `always` for a check configured without a
-`frequency`, which means run this check every time checks run. But if you omit
-consistency checks from configuration entirely, borgmatic runs full-repository
-checks (`repository`) and per-archive checks (`archives`) within each
-repository, at most once a month.
-
-Unlike a real scheduler like cron, borgmatic only makes a best effort to run
-checks on the configured frequency. It compares that frequency with how long
-it's been since the last check for a given repository If it hasn't been long
-enough, the check is skipped. And you still have to run `borgmatic check` (or
-`borgmatic` without actions) in order for checks to run, even when a
-`frequency` is configured!
-
-This also applies *across* configuration files that have the same repository
-configured. Make sure you have the same check frequency configured in each
-though—or the most frequently configured check will apply.
-
-New in version 1.9.0To support
-this frequency logic, borgmatic records check timestamps within the
-`~/.local/state/borgmatic/checks` directory. To override the `~/.local/state`
-portion of this path, set the `user_state_directory` configuration option.
-Alternatively, set the `XDG_STATE_HOME` environment variable.
-
-New in version 1.9.2The
-`STATE_DIRECTORY` environment variable also works for this purpose. It's set
-by systemd if `StateDirectory=borgmatic` is added to borgmatic's systemd
-service file.
-
-Prior to version 1.9.0
-borgmatic recorded check timestamps within the `~/.borgmatic` directory. At
-that time, the path was configurable by the `borgmatic_source_directory`
-configuration option (now deprecated).
-
-If you want to temporarily ignore your configured frequencies, you can invoke
-`borgmatic check --force` to run checks unconditionally.
-
-New in version 1.8.6 `borgmatic
-check --force` runs `check` even if it's specified in the `skip_actions`
-option.
-
-
-### Check days
-
-New in version 1.8.13 You can
-optionally configure checks to only run on particular days of the week. For
-instance:
-
-```yaml
-checks:
- - name: repository
- only_run_on:
- - Saturday
- - Sunday
- - name: archives
- only_run_on:
- - weekday
- - name: spot
- only_run_on:
- - Friday
- - weekend
-```
-
-Each day of the week is specified in the current locale (system
-language/country settings). `weekend` and `weekday` are also accepted.
-
-As with `frequency`, borgmatic only makes a best effort to run checks on the
-given day of the week. For instance, if you run `borgmatic check` daily, then
-every day borgmatic will have an opportunity to determine whether your checks
-are configured to run on that day. If they are, then the checks run. If not,
-they are skipped.
-
-For instance, with the above configuration, if borgmatic is run on a Saturday,
-the `repository` check will run. But on a Monday? The repository check will
-get skipped. And if borgmatic is never run on a Saturday or a Sunday, that
-check will never get a chance to run.
-
-Also, the day of the week configuration applies *after* any configured
-`frequency` for a check. So for instance, imagine the following configuration:
-
-```yaml
-checks:
- - name: repository
- frequency: 2 weeks
- only_run_on:
- - Monday
-```
-
-If you run borgmatic daily with that configuration, then borgmatic will first
-wait two weeks after the previous check before running the check again—on the
-first Monday after the `frequency` duration elapses.
-
-
-### Running only checks
-
-New in version 1.7.1 If you
-would like to only run consistency checks without creating backups (for
-instance with the `check` action on the command-line), you can omit
-the `source_directories` option entirely.
-
-Prior to version 1.7.1 In older
-versions of borgmatic, instead specify an empty `source_directories` value, as
-it is a mandatory option there:
-
-```yaml
-location:
- source_directories: []
-```
-
-
-### Disabling checks
-
-If that's still too slow, you can disable consistency checks entirely,
-either for a single repository or for all repositories.
-
-New in version 1.8.5 Disabling
-all consistency checks looks like this:
-
-```yaml
-skip_actions:
- - check
-```
-
-Prior to version 1.8.5 Use this
-configuration instead:
-
-```yaml
-checks:
- - name: disabled
-```
-
-Prior to version 1.8.0 Put
-`checks:` in the `consistency:` section of your configuration.
-
-Prior to version 1.6.2
-`checks:` was a plain list of strings without the `name:` part. For instance:
-
-```yaml
-checks:
- - disabled
-```
-
-If you have multiple repositories in your borgmatic configuration file,
-you can keep running consistency checks, but only against a subset of the
-repositories:
-
-```yaml
-check_repositories:
- - path/of/repository_to_check.borg
-```
-
-Finally, you can override your configuration file's consistency checks and
-run particular checks via the command-line. For instance:
-
-```bash
-borgmatic check --only data --only extract
-```
-
-This is useful for running slow consistency checks on an infrequent basis,
-separate from your regular checks. It is still subject to any configured
-check frequencies unless the `--force` flag is used.
+Another way of dealing with large backups is to customize your consistency
+checks. By default, if you omit consistency checks from configuration, borgmatic
+runs full-repository checks and per-archive checks within each repository on a
+monthly basis.
+
+But if you find that archive checks are too slow and/or you'd like to customize
+the check frequency, see the [consistency checks
+documentation](https://torsion.org/borgmatic/reference/configuration/consistency-checks/)
+for details.
## Troubleshooting
diff --git a/docs/how-to/develop-on-borgmatic.md b/docs/how-to/develop-on-borgmatic.md
index 084be6cd..34fc841f 100644
--- a/docs/how-to/develop-on-borgmatic.md
+++ b/docs/how-to/develop-on-borgmatic.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 15
---
-## Source code
-
To get set up to develop on borgmatic, first [`install
uv`](https://docs.astral.sh/uv/) to make managing your borgmatic environment
easier without impacting other Python applications on your system.
@@ -35,7 +33,7 @@ uv tool install --editable .
```
Or to work on the [Apprise
-hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#apprise-hook),
+hook](https://torsion.org/borgmatic/how-to/monitor-your-backups/#apprise-hook),
change that last line to:
```bash
@@ -43,7 +41,7 @@ uv tool install --editable .[Apprise]
```
To get oriented with the borgmatic source code, have a look at the [source
-code reference](https://torsion.org/borgmatic/docs/reference/source-code/).
+code reference](https://torsion.org/borgmatic/reference/source-code/).
### Source packages
diff --git a/docs/how-to/extract-a-backup.md b/docs/how-to/extract-a-backup.md
index 7f5bb662..dbd02db9 100644
--- a/docs/how-to/extract-a-backup.md
+++ b/docs/how-to/extract-a-backup.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 7
---
-## Extract
-
When the worst happens—or you want to test your backups—the first step is
to figure out which archive to extract. A good way to do that is to use the
`repo-list` action:
@@ -79,7 +77,7 @@ run the `extract` command above, borgmatic will extract `/var/path/1` and
If you're not sure which archive contains the files you're looking for, you
can [search across
-archives](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/#searching-for-a-file).
+archives](https://torsion.org/borgmatic/how-to/inspect-your-backups/#searching-for-a-file).
## Extract to a particular destination
@@ -100,7 +98,7 @@ files with extracted files unless that is your intent.
The `borgmatic extract` command only extracts files. To restore a database,
please see the [documentation on database backups and
-restores](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/).
+restores](https://torsion.org/borgmatic/how-to/backup-your-databases/).
borgmatic does not perform database restoration as part of `borgmatic extract`
so that you can extract files from your archive without impacting your live
databases.
@@ -167,7 +165,7 @@ To support this, borgmatic creates a manifest file that records the paths of
all the borgmatic configuration files stored within an archive. The file gets
written to borgmatic's runtime directory on disk and then stored within the
archive. See the [runtime directory
-documentation](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory)
+documentation](https://torsion.org/borgmatic/how-to/backup-your-databases/#runtime-directory)
for how and where that happens.
To extract the configuration files from an archive, use the `config bootstrap`
diff --git a/docs/how-to/inspect-your-backups.md b/docs/how-to/inspect-your-backups.md
index e6d316bc..802e21d2 100644
--- a/docs/how-to/inspect-your-backups.md
+++ b/docs/how-to/inspect-your-backups.md
@@ -5,42 +5,18 @@ eleventyNavigation:
parent: How-to guides
order: 5
---
-## Backup progress
-
By default, borgmatic runs proceed silently except in the case of warnings or
errors. But if you'd like to to get additional information about the progress of
-the backup as it proceeds, use the verbosity option:
-
-```bash
-borgmatic --verbosity 1
-```
-
-This lists the files that borgmatic is archiving, which are those that are new
-or changed since the last backup.
-
-Or, for even more progress and debug spew:
-
-```bash
-borgmatic --verbosity 2
-```
-
-The full set of verbosity levels are:
-
- * `-2`: disable output entirely New in borgmatic 1.7.14
- * `-1`: only show errors
- * `0`: default output
- * `1`: some additional output (informational level)
- * `2`: lots of additional output (debug level)
-
-New in version 2.0.0Set the
-verbosity in your borgmatic configuration via the `verbosity` option.
+the backup as it proceeds, see the [logging
+documentation](https://torsion.org/borgmatic/reference/command-line/logging/)
+for details.
## Backup summary
-If you're less concerned with progress during a backup, and you only want to
-see the summary of archive statistics at the end, you can use the stats
-option when performing a backup:
+If you're less concerned with progress during a backup, and you only want to see
+the summary of archive statistics at the end, use the stats option when
+performing a backup:
```bash
borgmatic --stats
@@ -63,8 +39,6 @@ with `--format`. Refer to the [borg list --format
documentation](https://borgbackup.readthedocs.io/en/stable/usage/list.html#the-format-specifier-syntax)
for available values.
-(No borgmatic `list` or `info` actions? Upgrade borgmatic!)
-
New in version 1.9.0 There are
also `repo-list` and `repo-info` actions for displaying repository information
with Borg 2.x:
@@ -75,7 +49,7 @@ borgmatic repo-info
```
See the [borgmatic command-line
-reference](https://torsion.org/borgmatic/docs/reference/command-line/) for
+reference](https://torsion.org/borgmatic/reference/command-line/) for
more information.
@@ -90,9 +64,8 @@ purpose. For instance, if you're looking for a `foo.txt`:
borgmatic list --find foo.txt
```
-This will list your archives and indicate those with files matching
-`*foo.txt*` anywhere in the archive. The `--find` parameter can alternatively
-be a [Borg
+This lists your archives and indicate those with files matching `*foo.txt*`
+anywhere in the archive. The `--find` parameter can alternatively be a [Borg
pattern](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-patterns).
To limit the archives searched, use the standard `list` parameters for
@@ -105,8 +78,8 @@ borgmatic list --find foo.txt --last 5
## Listing database dumps
-If you have enabled borgmatic's [database
-hooks](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/), you
+If you've enabled borgmatic's [database
+hooks](https://torsion.org/borgmatic/how-to/backup-your-databases/), you
can list backed up database dumps via borgmatic. For example:
```bash
@@ -123,7 +96,7 @@ doesn't store the leading `/`.)
With Borg version 1.2 and
earlierDatabase dump files are stored at a path dependent on the [runtime
-directory](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory)
+directory](https://torsion.org/borgmatic/how-to/backup-your-databases/#runtime-directory)
in use at the time the archive was created, as Borg 1.2 and earlier do not
support path rewriting.
@@ -133,103 +106,13 @@ the backup archive (where `~` was expanded to the home directory of the user
who performed the backup). This applied with all versions of Borg.
+
+
+
+
## Logging
-By default, borgmatic logs to the console. You can enable simultaneous syslog
-logging and customize its log level with the `--syslog-verbosity` flag, which
-is independent from the console logging `--verbosity` flag described above.
-For instance, to enable syslog logging, run:
-
-```bash
-borgmatic --syslog-verbosity 1
-```
-
-To increase syslog logging further to include debugging information, run:
-
-```bash
-borgmatic --syslog-verbosity 2
-```
-
-See above for further details about the verbosity levels.
-
-New in version 2.0.0Set the
-syslog verbosity in your borgmatic configuration via the `syslog_verbosity`
-option.
-
-Where these logs show up depends on your particular system. If you're using
-systemd, try running `journalctl -xe`. Otherwise, try viewing
-`/var/log/syslog` or similar.
-
-Prior to version 1.8.3borgmatic
-logged to syslog by default whenever run at a non-interactive console.
-
-### Rate limiting
-
-If you are using rsyslog or systemd's journal, be aware that by default they
-both throttle the rate at which logging occurs. So you may need to change
-either [the global rate
-limit](https://www.rootusers.com/how-to-change-log-rate-limiting-in-linux/) or
-[the per-service rate
-limit](https://www.freedesktop.org/software/systemd/man/journald.conf.html#RateLimitIntervalSec=)
-if you're finding that borgmatic logs are missing.
-
-Note that the [sample borgmatic systemd service
-file](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#systemd)
-already has this rate limit disabled for systemd's journal.
-
-
-### Logging to file
-
-If you don't want to use syslog, and you'd rather borgmatic log to a plain
-file, use the `--log-file` flag:
-
-```bash
-borgmatic --log-file /path/to/file.log
-```
-
-Note that if you use the `--log-file` flag, you are responsible for rotating
-the log file so it doesn't grow too large, for example with
-[logrotate](https://wiki.archlinux.org/index.php/Logrotate).
-
-You can use the `--log-file-verbosity` flag to customize the log file's log level:
-
-```bash
-borgmatic --log-file /path/to/file.log --log-file-verbosity 2
-```
-
-New in version 2.0.0Set the log
-file verbosity in your borgmatic configuration via the `log_file_verbosity`
-option.
-
-New in version 1.7.11 Use the
-`--log-file-format` flag to override the default log message format. This
-format string can contain a series of named placeholders wrapped in curly
-brackets. For instance, the default log format is: `[{asctime}] {levelname}:
-{message}`. This means each log message is recorded as the log time (in square
-brackets), a logging level name, a colon, and the actual log message.
-
-So if you only want each log message to get logged *without* a timestamp or a
-logging level name:
-
-```bash
-borgmatic --log-file /path/to/file.log --log-file-format "{message}"
-```
-
-Here is a list of available placeholders:
-
- * `{asctime}`: time the log message was created
- * `{levelname}`: level of the log message (`INFO`, `DEBUG`, etc.)
- * `{lineno}`: line number in the source file where the log message originated
- * `{message}`: actual log message
- * `{pathname}`: path of the source file where the log message originated
-
-See the [Python logging
-documentation](https://docs.python.org/3/library/logging.html#logrecord-attributes)
-for additional placeholders.
-
-Note that this `--log-file-format` flag only applies to the specified
-`--log-file` and not to syslog or other logging.
-
-New in version 2.0.0Set the
-defaults for these flags in your borgmatic configuration via the `log_file` and
-`log_file_format` options.
+By default, borgmatic only logs to the console. But to enable simultaneous
+syslog or file logging, see the [logging
+documentation](https://torsion.org/borgmatic/reference/command-line/logging/)
+for details.
diff --git a/docs/how-to/make-backups-redundant.md b/docs/how-to/make-backups-redundant.md
index 6f4d868b..347739e1 100644
--- a/docs/how-to/make-backups-redundant.md
+++ b/docs/how-to/make-backups-redundant.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 3
---
-## Multiple repositories
-
If you really care about your data, you probably want more than one backup of
it. borgmatic supports this in its configuration by specifying multiple backup
repositories. Here's an example:
@@ -51,7 +49,7 @@ for more information on how to specify local and remote repository paths.
What if you want borgmatic to backup to multiple repositories—while also
setting different options for each one? In that case, you'll need to use
[a separate borgmatic configuration file for each
-repository](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/)
+repository](https://torsion.org/borgmatic/how-to/make-per-application-backups/)
instead of the multiple repositories in one configuration file as described
above. That's because all of the repositories in a particular configuration
file get the same options applied.
diff --git a/docs/how-to/make-per-application-backups.md b/docs/how-to/make-per-application-backups.md
index d7a847c4..954a4ebf 100644
--- a/docs/how-to/make-per-application-backups.md
+++ b/docs/how-to/make-per-application-backups.md
@@ -1,12 +1,10 @@
---
title: How to make per-application backups
eleventyNavigation:
- key: 🔀 Make per-application backups
+ key: 🗂️ Make per-application backups
parent: How-to guides
order: 1
---
-## Multiple backup configurations
-
You may find yourself wanting to create different backup policies for
different applications on your system or even for different backup
repositories. For instance, you might want one backup configuration for your
@@ -54,634 +52,29 @@ paths on the command-line with borgmatic's `--config` flag. (See `borgmatic
--help` for more information.) For instance, if you want to schedule your
various borgmatic backups to run at different times, you'll need multiple
entries in your [scheduling software of
-choice](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#autopilot),
+choice](https://torsion.org/borgmatic/how-to/set-up-backups/#autopilot),
each entry using borgmatic's `--config` flag instead of relying on
`/etc/borgmatic.d`.
-## Archive naming
-
-If you've got multiple borgmatic configuration files, you might want to create
-archives with different naming schemes for each one. This is especially handy
-if each configuration file is backing up to the same Borg repository but you
-still want to be able to distinguish backup archives for one application from
-another.
-
-borgmatic supports this use case with an `archive_name_format` option. The
-idea is that you define a string format containing a number of [Borg
-placeholders](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-placeholders),
-and borgmatic uses that format to name any new archive it creates. For
-instance:
-
-```yaml
-archive_name_format: home-directories-{now}
-```
-
-Prior to version 1.8.0 Put
-this option in the `storage:` section of your configuration.
-
-This example means that when borgmatic creates an archive, its name will start
-with the string `home-directories-` and end with a timestamp for its creation
-time. If `archive_name_format` is unspecified, the default with Borg 1 is
-`{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}`, meaning your system hostname plus a
-timestamp in a particular format.
-
-New in borgmatic version 1.9.0 with
-Borg version 2.xThe default is just `{hostname}`, as Borg 2 does not
-require unique archive names; identical archive names form a common "series"
-that can be targeted together.
-
-
-### Archive filtering
-
-New in version 1.7.11 borgmatic
-uses the `archive_name_format` option to automatically limit which archives
-get used for actions operating on multiple archives. This prevents, for
-instance, duplicate archives from showing up in `repo-list` or `info`
-results—even if the same repository appears in multiple borgmatic
-configuration files. To take advantage of this feature, use a different
-`archive_name_format` in each configuration file.
-
-Under the hood, borgmatic accomplishes this by substituting globs for certain
-ephemeral data placeholders in your `archive_name_format`—and using the result
-to filter archives when running supported actions.
-
-For instance, let's say that you have this in your configuration:
-
-```yaml
-archive_name_format: {hostname}-user-data-{now}
-```
-
-Prior to version 1.8.0 Put
-this option in the `storage:` section of your configuration.
-
-borgmatic considers `{now}` an emphemeral data placeholder that will probably
-change per archive, while `{hostname}` won't. So it turns the example value
-into `{hostname}-user-data-*` and applies it to filter down the set of
-archives used for actions like `repo-list`, `info`, `prune`, `check`, etc.
-
-The end result is that when borgmatic runs the actions for a particular
-application-specific configuration file, it only operates on the archives
-created for that application. But this doesn't apply to actions like `compact`
-that operate on an entire repository.
-
-If this behavior isn't quite smart enough for your needs, you can use the
-`match_archives` option to override the pattern that borgmatic uses for
-filtering archives. For example:
-
-```yaml
-archive_name_format: {hostname}-user-data-{now}
-match_archives: sh:myhost-user-data-*
-```
-
-With Borg version 1.xUse a shell
-pattern for the `match_archives` value and see the [Borg patterns
-documentation](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-patterns)
-for more information.
-
-With Borg version 2.xSee the
-[match archives
-documentation](https://borgbackup.readthedocs.io/en/2.0.0b16/usage/help.html#borg-help-match-archives).
-
-Some borgmatic command-line actions also have a `--match-archives` flag that
-overrides both the auto-matching behavior and the `match_archives`
-configuration option.
-
-Prior to version 1.7.11 The way
-to limit the archives used for the `prune` action was a `prefix` option in the
-`retention` section for matching against the start of archive names. And the
-option for limiting the archives used for the `check` action was a separate
-`prefix` in the `consistency` section. Both of these options are deprecated in
-favor of the auto-matching behavior (or `match_archives`/`--match-archives`)
-in newer versions of borgmatic.
-
-
-## Configuration includes
-
-Once you have multiple different configuration files, you might want to share
-common configuration options across these files without having to copy and paste
-them. To achieve this, you can put fragments of common configuration options
-into a file and then include or inline that file into one or more borgmatic
-configuration files.
-
-Let's say that you want to include common consistency check configuration across all
-of your configuration files. You could do that in each configuration file with
-the following:
-
-```yaml
-repositories:
- - path: repo.borg
-
-checks:
- !include /etc/borgmatic/common_checks.yaml
-```
-
-Prior to version 1.8.0 These
-options were organized into sections like `location:` and `consistency:`.
-
-The contents of `common_checks.yaml` could be:
-
-```yaml
-- name: repository
- frequency: 3 weeks
-- name: archives
- frequency: 2 weeks
-```
-
-To prevent borgmatic from trying to load these configuration fragments by
-themselves and complaining that they are not valid configuration files, you
-should put them in a directory other than `/etc/borgmatic.d/`. (A subdirectory
-is fine.)
-
-When a configuration include is a relative path, borgmatic loads it from either
-the current working directory or from the directory containing the file doing
-the including.
-
-Note that this form of include must be a value rather than an option name. For
-example, this will not work:
-
-```yaml
-repositories:
- - path: repo.borg
-
-# Don't do this. It won't work!
-!include /etc/borgmatic/common_checks.yaml
-```
-
-But if you do want to merge in a option name *and* its values, keep reading!
-
-
-## Include merging
-
-If you need to get even fancier and merge in common configuration options, you
-can perform a YAML merge of included configuration using the YAML `<<` key.
-For instance, here's an example of a main configuration file that pulls in
-retention and consistency checks options via a single include:
-
-```yaml
-repositories:
- - path: repo.borg
-
-<<: !include /etc/borgmatic/common.yaml
-```
-
-This is what `common.yaml` might look like:
-
-```yaml
-keep_hourly: 24
-keep_daily: 7
-
-checks:
- - name: repository
- frequency: 3 weeks
- - name: archives
- frequency: 2 weeks
-```
-
-Prior to version 1.8.0 These
-options were organized into sections like `retention:` and `consistency:`.
-
-Once this include gets merged in, the resulting configuration has all of the
-options from the original configuration file *and* the options from the
-include.
-
-Note that this `<<` include merging syntax is only for merging in mappings
-(configuration options and their values). If you'd like to include a single
-value directly, please see above about standard includes.
-
-
-### Multiple merge includes
-
-borgmatic has a limitation preventing multiple `<<` include merges per file or
-option value. This means you can do a single `<<` merge at the global level,
-another `<<` within each nested option value, etc. (This is a YAML
-limitation.) For instance:
-
-```yaml
-repositories:
- - path: repo.borg
-
-# This won't work! You can't do multiple merges like this at the same level.
-<<: !include common1.yaml
-<<: !include common2.yaml
-```
-
-But read on for a way around this.
-
-New in version 1.8.1 You can
-include and merge multiple configuration files all at once. For instance:
-
-```yaml
-repositories:
- - path: repo.borg
-
-<<: !include [common1.yaml, common2.yaml, common3.yaml]
-```
-
-This merges in each included configuration file in turn, such that later files
-replace the options in earlier ones.
-
-Here's another way to do the same thing:
-
-```yaml
-repositories:
- - path: repo.borg
-
-<<: !include
- - common1.yaml
- - common2.yaml
- - common3.yaml
-```
-
-
-### Deep merge
-
-New in version 1.6.0 borgmatic
-performs a deep merge of merged include files, meaning that values are merged
-at all levels in the two configuration files. This allows you to include
-common configuration—up to full borgmatic configuration files—while overriding
-only the parts you want to customize.
-
-For instance, here's an example of a main configuration file that pulls in
-options via an include and then overrides one of them locally:
-
-```yaml
-<<: !include /etc/borgmatic/common.yaml
-
-constants:
- base_directory: /opt
-
-repositories:
- - path: repo.borg
-```
-
-This is what `common.yaml` might look like:
-
-```yaml
-constants:
- app_name: myapp
- base_directory: /var/lib
-```
-
-Once this include gets merged in, the resulting configuration would have an
-`app_name` value of `myapp` and an overridden `base_directory` value of
-`/opt`.
-
-When there's an option collision between the local file and the merged
-include, the local file's option takes precedence.
-
-
-#### List merge
-
-New in version 1.6.1 Colliding
-list values are appended together.
-
-New in version 1.7.12 If there
-is a list value from an include that you *don't* want in your local
-configuration file, you can omit it with an `!omit` tag. For instance:
-
-```yaml
-<<: !include /etc/borgmatic/common.yaml
-
-source_directories:
- - !omit /home
- - /var
-```
-
-And `common.yaml` like this:
-
-```yaml
-source_directories:
- - /home
- - /etc
-```
-
-Prior to version 1.8.0 Put the
-`source_directories` option in the `location:` section of your configuration.
-
-Once this include gets merged in, the resulting configuration will have a
-`source_directories` value of `/etc` and `/var`—with `/home` omitted.
-
-This feature currently only works on scalar (e.g. string or number) list items
-and will not work elsewhere in a configuration file. Be sure to put the
-`!omit` tag *before* the list item (after the dash). Putting `!omit` after the
-list item will not work, as it gets interpreted as part of the string. Here's
-an example of some things not to do:
-
-```yaml
-<<: !include /etc/borgmatic/common.yaml
-
-source_directories:
- # Do not do this! It will not work. "!omit" belongs before "/home".
- - /home !omit
-
-# Do not do this either! "!omit" only works on scalar list items.
-repositories: !omit
- # Also do not do this for the same reason! This is a list item, but it's
- # not a scalar.
- - !omit path: repo.borg
-```
-
-Additionally, the `!omit` tag only works in a configuration file that also
-performs a merge include with `<<: !include`. It doesn't make sense within,
-for instance, an included configuration file itself (unless it in turn
-performs its own merge include). That's because `!omit` only applies to the
-file doing the include; it doesn't work in reverse or propagate through
-includes.
-
-
-### Shallow merge
-
-Even though deep merging is generally pretty handy for included files,
-sometimes you want specific options in the local file to take precedence over
-included options—without any merging occurring for them.
-
-New in version 1.7.12 That's
-where the `!retain` tag comes in. Whenever you're merging an included file
-into your configuration file, you can optionally add the `!retain` tag to
-particular local mappings or lists to retain the local values and ignore
-included values.
-
-For instance, start with this configuration file containing the `!retain` tag
-on the `retention` mapping:
-
-```yaml
-<<: !include /etc/borgmatic/common.yaml
-
-repositories:
- - path: repo.borg
-
-checks: !retain
- - name: repository
-```
-
-And `common.yaml` like this:
-
-```yaml
-repositories:
- - path: common.borg
-
-checks:
- - name: archives
-```
-
-Prior to version 1.8.0 These
-options were organized into sections like `location:` and `consistency:`.
-
-Once this include gets merged in, the resulting configuration will have a
-`checks` value with a name of `repository` and no other values. That's because
-the `!retain` tag says to retain the local version of `checks` and ignore any
-values coming in from the include. But because the `repositories` list doesn't
-have a `!retain` tag, it still gets merged together to contain both
-`common.borg` and `repo.borg`.
-
-The `!retain` tag can only be placed on mappings (keys/values) and lists, and
-it goes right after the name of the option (and its colon) on the same line.
-The effects of `!retain` are recursive, meaning that if you place a `!retain`
-tag on a top-level mapping, even deeply nested values within it will not be
-merged.
-
-Additionally, the `!retain` tag only works in a configuration file that also
-performs a merge include with `<<: !include`. It doesn't make sense within,
-for instance, an included configuration file itself (unless it in turn
-performs its own merge include). That's because `!retain` only applies to the
-file doing the include; it doesn't work in reverse or propagate through
-includes.
-
-
-## Debugging includes
-
-New in version 1.7.15 If you'd
-like to see what the loaded configuration looks like after includes get merged
-in, run the `validate` action on your configuration file:
-
-```bash
-sudo borgmatic config validate --show
-```
-
-In version 1.7.12 through
-1.7.14 Use this command instead:
-
-```bash
-sudo validate-borgmatic-config --show
-```
-
-You'll need to specify your configuration file with `--config` if it's not in
-a default location.
-
-This will output the merged configuration as borgmatic sees it, which can be
-helpful for understanding how your includes work in practice.
-
-
-## Configuration overrides
-
-In more complex multi-application setups, you may want to override particular
-borgmatic configuration file options at the time you run borgmatic. For
-instance, you could reuse a common configuration file for multiple
-applications, but then set the repository for each application at runtime. Or
-you might want to try a variant of an option for testing purposes without
-actually touching your configuration file.
-
-New in version 2.0.0
-Whatever the reason, you can override borgmatic configuration options at the
-command-line, as there's a command-line flag corresponding to every
-configuration option (with its underscores converted to dashes).
-
-For instance, to override the `compression` configuration option, use the
-corresponding `--compression` flag on the command-line:
-
-```bash
-borgmatic create --compression zstd
-```
-
-What this does is load your given configuration files and for each one, disregard
-the configured value for the `compression` option and use the value given on the
-command-line instead—but just for the duration of the borgmatic run.
-
-You can override nested configuration options too by separating such option
-names with a period. For instance:
-
-```bash
-borgmatic create --bootstrap.store-config-files false
-```
-
-You can even set complex option data structures by using inline YAML syntax. For
-example, set the `repositories` option with a YAML list of key/value pairs:
-
-```bash
-borgmatic create --repositories "[{path: /mnt/backup, label: local}]"
-```
-
-If your override value contains characters like colons or spaces, then you'll
-need to use quotes for it to parse correctly.
-
-You can also set individual nested options within existing list elements:
-
-```bash
-borgmatic create --repositories[0].path /mnt/backup
-```
-
-This updates the `path` option for the first repository in `repositories`.
-Change the `[0]` index as needed to address different list elements. And note
-that this only works for elements already set in configuration; you can't append
-new list elements from the command-line.
-
-See the [command-line reference
-documentation](https://torsion.org/borgmatic/docs/reference/command-line/) for
-the full set of available arguments, including examples of each for the complex
-values.
-
-There are a handful of configuration options that don't have corresponding
-command-line flags at the global scope, but instead have flags within individual
-borgmatic actions. For instance, the `list_details` option can be overridden by
-the `--list` flag that's only present on particular actions. Similarly with
-`progress` and `--progress`, `statistics` and `--stats`, and `match_archives`
-and `--match-archives`.
-
-Also note that if you want to pass a command-line flag itself as a value to one
-of these override flags, that may not work. For instance, specifying
-`--extra-borg-options.create --no-cache-sync` results in an error, because
-`--no-cache-sync` gets interpreted as a borgmatic option (which in this case
-doesn't exist) rather than a Borg option.
-
-An alternate to command-line overrides is passing in your values via
-[environment
-variables](https://torsion.org/borgmatic/docs/how-to/provide-your-passwords/).
-
-
-### Deprecated overrides
-
-Prior to version 2.0.0
-Configuration overrides were performed with an `--override` flag. You can still
-use `--override` with borgmatic 2.0.0+, but it's deprecated in favor of the new
-command-line flags described above.
-
-Here's an example of `--override`:
-
-```bash
-borgmatic create --override remote_path=/usr/local/bin/borg1
-```
-
-What this does is load your given configuration files and for each one, disregard
-the configured value for the `remote_path` option and use the value given on the
-command-line instead—but just for the duration of the borgmatic run.
-
-You can even override nested values or multiple values at once. For instance:
-
-```bash
-borgmatic create --override parent_option.option1=value1 --override parent_option.option2=value2
-```
-
-Prior to version 1.8.0 Don't
-forget to specify the section that an option is in. That looks like a prefix
-on the option name, e.g. `location.repositories`.
-
-Note that each value is parsed as an actual YAML string, so you can set list
-values by using brackets. For instance:
-
-```bash
-borgmatic create --override repositories=[test1.borg,test2.borg]
-```
-
-Or a single list element:
-
-```bash
-borgmatic create --override repositories=[/root/test.borg]
-```
-
-Or a single list element that is a key/value pair:
-
-```bash
-borgmatic create --override repositories="[{path: test.borg, label: test}]"
-```
-
-If your override value contains characters like colons or spaces, then you'll
-need to use quotes for it to parse correctly.
-
-Another example:
-
-```bash
-borgmatic create --override repositories="['user@server:test.borg']"
-```
-
-There is not currently a way to override a single element of a list without
-replacing the whole list.
-
-Using the `[ ]` list syntax is required when overriding an option of the list
-type (like `location.repositories`). See the [configuration
-reference](https://torsion.org/borgmatic/docs/reference/configuration/) for
-which options are list types. (YAML list values look like `- this` with an
-indentation and a leading dash.)
-
-
-## Constant interpolation
-
-New in version 1.7.10 Another
-tool is borgmatic's support for defining custom constants. This is similar to
-the [variable interpolation
-feature](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/#variable-interpolation)
-for command hooks, but the constants feature lets you substitute your own
-custom values into any option values in the entire configuration file.
-
-Here's an example usage:
-
-```yaml
-constants:
- user: foo
- archive_prefix: bar
-
-source_directories:
- - /home/{user}/.config
- - /home/{user}/.ssh
-
-...
-
-archive_name_format: '{archive_prefix}-{now}'
-```
-
-Prior to version 1.8.0 Don't
-forget to specify the section (like `location:` or `storage:`) that any option
-is in.
-
-In this example, when borgmatic runs, all instances of `{user}` get replaced
-with `foo` and all instances of `{archive_prefix}` get replaced with `bar`.
-And `{now}` doesn't get replaced with anything, but gets passed directly to
-Borg, which has its own
-[placeholders](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-placeholders)
-using the same syntax as borgmatic constants. So borgmatic options like
-`archive_name_format` that get passed directly to Borg can use either Borg
-placeholders or borgmatic constants or both!
-
-After substitution, the logical result looks something like this:
-
-```yaml
-source_directories:
- - /home/foo/.config
- - /home/foo/.ssh
-
-...
-
-archive_name_format: 'bar-{now}'
-```
-
-Note that if you'd like to interpolate a constant into the beginning of a
-value, you'll need to quote it. For instance, this won't work:
-
-```yaml
-source_directories:
- - {my_home_directory}/.config # This will error!
-```
-
-Instead, do this:
-
-```yaml
-source_directories:
- - "{my_home_directory}/.config"
-```
-
-New in version 1.8.5 Constants
-work across includes, meaning you can define a constant and then include a
-separate configuration file that uses that constant.
-
-An alternate to constants is passing in your values via [environment
-variables](https://torsion.org/borgmatic/docs/how-to/provide-your-passwords/).
+
+
+
+
+
+## Related features
+
+Once you've got multiple configuration files, there are a few other borgmatic
+features that you might find handy:
+
+ * Use different archive naming schemes in each configuration file with the
+ [archive name
+ format](https://torsion.org/borgmatic/reference/configuration/archive-name-format/)
+ feature.
+ * Share common options across configuration files with
+ [includes](https://torsion.org/borgmatic/reference/configuration/includes/).
+ * Override configuration file options from the command-line with
+ [overrides](https://torsion.org/borgmatic/reference/command-line/overrides/).
+ * Also check out the
+ [constants](https://torsion.org/borgmatic/reference/configuration/constants/)
+ feature for defining custom per-configuration-file constants.
diff --git a/docs/how-to/monitor-your-backups.md b/docs/how-to/monitor-your-backups.md
index 287f4c0e..94ad8e7a 100644
--- a/docs/how-to/monitor-your-backups.md
+++ b/docs/how-to/monitor-your-backups.md
@@ -5,9 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 6
---
-
-## Monitoring and alerting
-
Having backups is great, but they won't do you a lot of good unless you have
confidence that they're running on a regular basis. That's where monitoring
and alerting comes in.
@@ -18,7 +15,7 @@ your particular infrastructure:
* **Job runner alerts**: The easiest place to start is with failure alerts from
the [scheduled job
- runner](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#autopilot)
+ runner](https://torsion.org/borgmatic/how-to/set-up-backups/#autopilot)
(cron, systemd, etc.) that's running borgmatic. But note that if the job
doesn't even get scheduled (e.g. due to the job runner not running), you
probably won't get an alert at all! Still, this is a decent first line of
@@ -28,23 +25,13 @@ your particular infrastructure:
you'll receive an alert when something goes wrong or when the service doesn't
hear from borgmatic for a configured interval (if supported). While these
services and libraries offer different features, you probably only need to
- use one of them at most. See these documentation links for configuration
- information:
- * [Apprise](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#apprise-hook)
- * [Cronhub](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#cronhub-hook)
- * [Cronitor](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#cronitor-hook)
- * [Grafana Loki](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#loki-hook)
- * [Healthchecks](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#healthchecks-hook)
- * [ntfy](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#ntfy-hook)
- * [PagerDuty](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#pagerduty-hook)
- * [Pushover](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#pushover-hook)
- * [Sentry](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#sentry-hook)
- * [Uptime Kuma](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#uptime-kuma-hook)
- * [Zabbix](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#zabbix-hook)
+ use one of them at most. See the [monitoring configuration
+ documentation](https://torsion.org/borgmatic/reference/configuration/monitoring/)
+ for details.
* **Third-party monitoring software:** You can use traditional monitoring
software to consume borgmatic JSON output and track when the last successful
backup occurred. See [scripting
- borgmatic](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#scripting-borgmatic)
+ borgmatic](https://torsion.org/borgmatic/how-to/monitor-your-backups/#scripting-borgmatic)
below for how to configure this.
* **Borg hosting providers:** Some [Borg hosting
providers](https://torsion.org/borgmatic/#hosting-providers) include
@@ -54,639 +41,17 @@ your particular infrastructure:
* **Consistency checks:** While not strictly part of monitoring, if you want
confidence that your backups are not only running but are restorable as well,
you can configure particular [consistency
- checks](https://torsion.org/borgmatic/docs/how-to/deal-with-very-large-backups/#consistency-check-configuration)
+ checks](https://torsion.org/borgmatic/how-to/deal-with-very-large-backups/#consistency-check-configuration)
or even script full [extract
- tests](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/).
+ tests](https://torsion.org/borgmatic/how-to/extract-a-backup/).
* **Commands run on error:** borgmatic's command hooks support running
arbitrary commands or scripts when borgmatic itself encounters an error
running your backups. So for instance, you can run a script to send yourself
a text message alert. But note that if borgmatic doesn't actually run, this
- alert won't fire. See the [documentation on command hooks](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/)
+ alert won't fire. See the [documentation on command hooks](https://torsion.org/borgmatic/how-to/add-preparation-and-cleanup-steps-to-backups/)
for details.
-## Healthchecks hook
-
-[Healthchecks](https://healthchecks.io/) is a service that provides "instant
-alerts when your cron jobs fail silently," and borgmatic has built-in
-integration with it. Once you create a Healthchecks account and project on
-their site, all you need to do is configure borgmatic with the unique "Ping
-URL" for your project. Here's an example:
-
-
-```yaml
-healthchecks:
- ping_url: https://hc-ping.com/addffa72-da17-40ae-be9c-ff591afb942a
-```
-
-Prior to version 1.8.0 Put
-this option in the `hooks:` section of your configuration.
-
-With this configuration, borgmatic pings your Healthchecks project when a
-backup begins, ends, or errors, but only when any of the `create`, `prune`,
-`compact`, or `check` actions are run.
-
-Then, if the actions complete successfully, borgmatic notifies Healthchecks of
-the success and includes borgmatic logs in the payload data sent to
-Healthchecks. This means that borgmatic logs show up in the Healthchecks UI,
-although be aware that Healthchecks currently has a 100-kilobyte limit for the
-logs in each ping.
-
-If an error occurs during any action or hook, borgmatic notifies Healthchecks,
-also tacking on logs including the error itself. But the logs are only
-included for errors that occur when a `create`, `prune`, `compact`, or `check`
-action is run.
-
-You can customize the verbosity of the logs that are sent to Healthchecks with
-borgmatic's `--monitoring-verbosity` flag. The `--list` and `--stats` flags
-may also be of use. See `borgmatic create --help` for more information.
-Additionally, see the [borgmatic configuration
-file](https://torsion.org/borgmatic/docs/reference/configuration/) for
-additional Healthchecks options.
-
-New in version 2.0.0Set the
-defaults for these flags in your borgmatic configuration via the
-`monitoring_verbosity`, `list`, and `statistics` options.
-
-You can configure Healthchecks to notify you by a [variety of
-mechanisms](https://healthchecks.io/#welcome-integrations) when backups fail
-or it doesn't hear from borgmatic for a certain period of time.
-
-
-## Cronitor hook
-
-[Cronitor](https://cronitor.io/) provides "Cron monitoring and uptime healthchecks
-for websites, services and APIs," and borgmatic has built-in
-integration with it. Once you create a Cronitor account and cron job monitor on
-their site, all you need to do is configure borgmatic with the unique "Ping
-API URL" for your monitor. Here's an example:
-
-
-```yaml
-cronitor:
- ping_url: https://cronitor.link/d3x0c1
-```
-
-Prior to version 1.8.0 Put
-this option in the `hooks:` section of your configuration.
-
-With this configuration, borgmatic pings your Cronitor monitor when a backup
-begins, ends, or errors, but only when any of the `create`, `prune`,
-`compact`, or `check` actions are run. Then, if the actions complete
-successfully or errors, borgmatic notifies Cronitor accordingly.
-
-You can configure Cronitor to notify you by a [variety of
-mechanisms](https://cronitor.io/docs/cron-job-notifications) when backups fail
-or it doesn't hear from borgmatic for a certain period of time.
-
-
-## Cronhub hook
-
-[Cronhub](https://cronhub.io/) provides "instant alerts when any of your
-background jobs fail silently or run longer than expected," and borgmatic has
-built-in integration with it. Once you create a Cronhub account and monitor on
-their site, all you need to do is configure borgmatic with the unique "Ping
-URL" for your monitor. Here's an example:
-
-
-```yaml
-cronhub:
- ping_url: https://cronhub.io/start/1f5e3410-254c-11e8-b61d-55875966d031
-```
-
-Prior to version 1.8.0 Put
-this option in the `hooks:` section of your configuration.
-
-With this configuration, borgmatic pings your Cronhub monitor when a backup
-begins, ends, or errors, but only when any of the `create`, `prune`,
-`compact`, or `check` actions are run. Then, if the actions complete
-successfully or errors, borgmatic notifies Cronhub accordingly.
-
-Note that even though you configure borgmatic with the "start" variant of the
-ping URL, borgmatic substitutes the correct state into the URL when pinging
-Cronhub ("start", "finish", or "fail").
-
-You can configure Cronhub to notify you by a [variety of
-mechanisms](https://docs.cronhub.io/integrations.html) when backups fail
-or it doesn't hear from borgmatic for a certain period of time.
-
-
-## PagerDuty hook
-
-In case you're new here: [borgmatic](https://torsion.org/borgmatic/) is
-simple, configuration-driven backup software for servers and workstations,
-powered by [Borg Backup](https://www.borgbackup.org/).
-
-[PagerDuty](https://www.pagerduty.com/) provides incident monitoring and
-alerting. borgmatic has built-in integration that can notify you via PagerDuty
-as soon as a backup fails, so you can make sure your backups keep working.
-
-First, create a PagerDuty account and service
-on their site. On the service, add an integration and set the Integration Type
-to "borgmatic".
-
-Then, configure borgmatic with the unique "Integration Key" for your service.
-Here's an example:
-
-
-```yaml
-pagerduty:
- integration_key: a177cad45bd374409f78906a810a3074
-```
-
-Prior to version 1.8.0 Put
-this option in the `hooks:` section of your configuration.
-
-With this configuration, borgmatic creates a PagerDuty event for your service
-whenever backups fail, but only when any of the `create`, `prune`, `compact`,
-or `check` actions are run. Note that borgmatic does not contact PagerDuty
-when a backup starts or when it ends without error.
-
-You can configure PagerDuty to notify you by a [variety of
-mechanisms](https://support.pagerduty.com/docs/notifications) when backups
-fail.
-
-If you have any issues with the integration, [please contact
-us](https://torsion.org/borgmatic/#support-and-contributing).
-
-
-### Sending logs
-
-New in version 1.9.14 borgmatic
-logs are included in the payload data sent to PagerDuty. This means that
-(truncated) borgmatic logs, including error messages, show up in the PagerDuty
-incident UI and corresponding notification emails.
-
-You can customize the verbosity of the logs that are sent with borgmatic's
-`--monitoring-verbosity` flag. The `--list` and `--stats` flags may also be of
-use. See `borgmatic create --help` for more information.
-
-New in version 2.0.0Set the
-defaults for these flags in your borgmatic configuration via the
-`monitoring_verbosity`, `list`, and `statistics` options.
-
-If you don't want any logs sent, you can disable log sending by setting
-`send_logs` to `false`:
-
-```yaml
-pagerduty:
- integration_key: a177cad45bd374409f78906a810a3074
- send_logs: false
-```
-
-
-## Pushover hook
-
-New in version 1.9.2
-[Pushover](https://pushover.net) makes it easy to get real-time notifications
-on your Android, iPhone, iPad, and Desktop (Android Wear and Apple Watch,
-too!).
-
-First, create a Pushover account and login on your mobile device. Create an
-Application in your Pushover dashboard.
-
-Then, configure borgmatic with your user's unique "User Key" found in your
-Pushover dashboard and the unique "API Token" from the created Application.
-
-Here's a basic example:
-
-
-```yaml
-pushover:
- token: 7ms6TXHpTokTou2P6x4SodDeentHRa
- user: hwRwoWsXMBWwgrSecfa9EfPey55WSN
-```
-
-
-With this configuration, borgmatic creates a Pushover event for your service
-whenever borgmatic fails, but only when any of the `create`, `prune`, `compact`,
-or `check` actions are run. Note that borgmatic does not contact Pushover
-when a backup starts or when it ends without error by default.
-
-You can configure Pushover to have custom parameters declared for borgmatic's
-`start`, `fail` and `finish` hooks states.
-
-Here's a more advanced example:
-
-
-```yaml
-pushover:
- token: 7ms6TXHpTokTou2P6x4SodDeentHRa
- user: hwRwoWsXMBWwgrSecfa9EfPey55WSN
- start:
- message: "Backup Started"
- priority: -2
- title: "Backup Started"
- html: True
- ttl: 10 # Message will be deleted after 10 seconds.
- fail:
- message: "Backup Failed"
- priority: 2 # Requests acknowledgement for messages.
- expire: 600 # Used only for priority 2. Default is 600 seconds.
- retry: 30 # Used only for priority 2. Default is 30 seconds.
- device: "pixel8"
- title: "Backup Failed"
- html: True
- sound: "siren"
- url: "https://ticketing-system.example.com/login"
- url_title: "Login to ticketing system"
- finish:
- message: "Backup Finished"
- priority: 0
- title: "Backup Finished"
- html: True
- ttl: 60
- url: "https://ticketing-system.example.com/login"
- url_title: "Login to ticketing system"
- states:
- - start
- - finish
- - fail
-```
-
-
-## Sentry hook
-
-New in version 1.9.7
-[Sentry](https://sentry.io/) is an application monitoring service that
-includes cron-style monitoring (either cloud-hosted or
-[self-hosted](https://develop.sentry.dev/self-hosted/)).
-
-To get started, create a [Sentry cron
-monitor](https://docs.sentry.io/product/crons/) in the Sentry UI. Under
-"Instrument your monitor," select "Sentry CLI" and copy the URL value for the
-displayed
-[`SENTRY_DSN`](https://docs.sentry.io/concepts/key-terms/dsn-explainer/)
-environment variable into borgmatic's Sentry `data_source_name_url`
-configuration option. For example:
-
-```
-sentry:
- data_source_name_url: https://5f80ec@o294220.ingest.us.sentry.io/203069
- monitor_slug: mymonitor
-```
-
-The `monitor_slug` value comes from the "Monitor Slug" under "Cron Details" on
-the same Sentry monitor page.
-
-With this configuration, borgmatic pings Sentry whenever borgmatic starts,
-finishes, or fails, but only when any of the `create`, `prune`, `compact`, or
-`check` actions are run. You can optionally override the start/finish/fail
-behavior with the `states` configuration option. For instance, to only ping
-Sentry on failure:
-
-```
-sentry:
- data_source_name_url: https://5f80ec@o294220.ingest.us.sentry.io/203069
- monitor_slug: mymonitor
- states:
- - fail
-```
-
-
-## ntfy hook
-
-New in version 1.6.3
-[ntfy](https://ntfy.sh) is a free, simple, service (either cloud-hosted or
-self-hosted) which offers simple pub/sub push notifications to multiple
-platforms including [web](https://ntfy.sh/stats),
-[Android](https://play.google.com/store/apps/details?id=io.heckel.ntfy) and
-[iOS](https://apps.apple.com/us/app/ntfy/id1625396347).
-
-Since push notifications for regular events might soon become quite annoying,
-this hook only fires on any errors by default in order to instantly alert you
-to issues. The `states` list can override this. Each state can have its own
-custom messages, priorities and tags or, if none are provided, will use the
-default.
-
-An example configuration is shown here with all the available options,
-including [priorities](https://ntfy.sh/docs/publish/#message-priority) and
-[tags](https://ntfy.sh/docs/publish/#tags-emojis):
-
-```yaml
-ntfy:
- topic: my-unique-topic
- server: https://ntfy.my-domain.com
- username: myuser
- password: secret
-
- start:
- title: A borgmatic backup started
- message: Watch this space...
- tags: borgmatic
- priority: min
- finish:
- title: A borgmatic backup completed successfully
- message: Nice!
- tags: borgmatic,+1
- priority: min
- fail:
- title: A borgmatic backup failed
- message: You should probably fix it
- tags: borgmatic,-1,skull
- priority: max
- states:
- - start
- - finish
- - fail
-```
-
-Prior to version 1.8.0 Put
-the `ntfy:` option in the `hooks:` section of your configuration.
-
-New in version 1.8.9 Instead of
-`username`/`password`, you can specify an [ntfy access
-token](https://docs.ntfy.sh/config/#access-tokens):
-
-```yaml
-ntfy:
- topic: my-unique-topic
- server: https://ntfy.my-domain.com
- access_token: tk_AgQdq7mVBoFD37zQVN29RhuMzNIz2
-````
-
-## Loki hook
-
-New in version 1.8.3 [Grafana
-Loki](https://grafana.com/oss/loki/) is a "horizontally scalable, highly
-available, multi-tenant log aggregation system inspired by Prometheus."
-borgmatic has built-in integration with Loki, sending both backup status and
-borgmatic logs.
-
-You can configure borgmatic to use either a [self-hosted Loki
-instance](https://grafana.com/docs/loki/latest/installation/) or [a Grafana
-Cloud account](https://grafana.com/auth/sign-up/create-user). Start by setting
-your Loki API push URL. Here's an example:
-
-```yaml
-loki:
- url: http://localhost:3100/loki/api/v1/push
-
- labels:
- app: borgmatic
- hostname: example.org
-```
-
-With this configuration, borgmatic sends its logs to your Loki instance as any
-of the `create`, `prune`, `compact`, or `check` actions are run. Then, after
-the actions complete, borgmatic notifies Loki of success or failure.
-
-This hook supports sending arbitrary labels to Loki. At least one label is
-required.
-
-There are also a few placeholders you can optionally use as label values:
-
- * `__config`: name of the borgmatic configuration file
- * `__config_path`: full path of the borgmatic configuration file
- * `__hostname`: the local machine hostname
-
-These placeholders are only substituted for the whole label value, not
-interpolated into a larger string. For instance:
-
-```yaml
-loki:
- url: http://localhost:3100/loki/api/v1/push
-
- labels:
- app: borgmatic
- config: __config
- hostname: __hostname
-```
-
-Also check out this [Loki dashboard for
-borgmatic](https://grafana.com/grafana/dashboards/20736-borgmatic-logs/) if
-you'd like to see your backup logs and statistics in one place.
-
-
-## Apprise hook
-
-New in version 1.8.4
-[Apprise](https://github.com/caronc/apprise/wiki) is a local notification library
-that "allows you to send a notification to almost all of the most popular
-[notification services](https://github.com/caronc/apprise/wiki) available to
-us today such as: Telegram, Discord, Slack, Amazon SNS, Gotify, etc."
-
-Depending on how you installed borgmatic, it may not have come with Apprise.
-For instance, if you originally [installed borgmatic with
-pipx](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#installation),
-run the following to install Apprise so borgmatic can use it:
-
-```bash
-sudo pipx uninstall borgmatic
-sudo pipx install borgmatic[Apprise]
-```
-
-Omit `sudo` if borgmatic is installed as a non-root user.
-
-Once Apprise is installed, configure borgmatic to notify one or more [Apprise
-services](https://github.com/caronc/apprise/wiki). For example:
-
-```yaml
-apprise:
- services:
- - url: gotify://hostname/token
- label: gotify
- - url: mastodons://access_key@hostname/@user
- label: mastodon
- states:
- - start
- - finish
- - fail
-```
-
-With this configuration, borgmatic pings each of the configured Apprise
-services when a backup begins, ends, or errors, but only when any of the
-`create`, `prune`, `compact`, or `check` actions are run. (By default, if
-`states` is not specified, Apprise services are only pinged on error.)
-
-You can optionally customize the contents of the default messages sent to
-these services:
-
-```yaml
-apprise:
- services:
- - url: gotify://hostname/token
- label: gotify
- start:
- title: Ping!
- body: Starting backup process.
- finish:
- title: Ping!
- body: Backups successfully made.
- fail:
- title: Ping!
- body: Your backups have failed.
- states:
- - start
- - finish
- - fail
-```
-
-New in version 1.8.9 borgmatic
-logs are automatically included in the body data sent to your Apprise services
-when a backup finishes or fails.
-
-You can customize the verbosity of the logs that are sent with borgmatic's
-`--monitoring-verbosity` flag. The `--list` and `--stats` flags may also be of
-use. See `borgmatic create --help` for more information.
-
-New in version 2.0.0Set the
-defaults for these flags in your borgmatic configuration via the
-`monitoring_verbosity`, `list`, and `statistics` options.
-
-If you don't want any logs sent, you can disable log sending by setting
-`send_logs` to `false`:
-
-```yaml
-apprise:
- services:
- - url: gotify://hostname/token
- label: gotify
- send_logs: false
-```
-
-Or to limit the size of logs sent to Apprise services:
-
-```yaml
-apprise:
- services:
- - url: gotify://hostname/token
- label: gotify
- logs_size_limit: 500
-```
-
-This may be necessary for some services that reject large requests.
-
-See the [configuration
-reference](https://torsion.org/borgmatic/docs/reference/configuration/) for
-details.
-
-## Uptime Kuma hook
-
-New in version 1.8.13 [Uptime
-Kuma](https://uptime.kuma.pet) is a self-hosted monitoring tool and can
-provide a Push monitor type to accept HTTP `GET` requests from a service
-instead of contacting it directly.
-
-Uptime Kuma allows you to see a history of monitor states and can in turn
-alert via ntfy, Gotify, Matrix, Apprise, Email, and many more.
-
-An example configuration is shown here with all the available options:
-
-```yaml
-uptime_kuma:
- push_url: https://kuma.my-domain.com/api/push/abcd1234
- states:
- - start
- - finish
- - fail
-```
-
-The `push_url` is provided to your from your Uptime Kuma service and
-originally includes a query string—the text including and after the question
-mark (`?`). But please do not include the query string in the `push_url`
-configuration; borgmatic will add this automatically depending on the state of
-your backup.
-
-Using `start`, `finish` and `fail` states means you will get two "up beats" in
-Uptime Kuma for successful backups and the ability to see failures if and when
-the backup started (was there a `start` beat?).
-
-A reasonable base-level configuration for an Uptime Kuma Monitor for a backup
-is below:
-
-```ini
-# These are to be entered into Uptime Kuma and not into your borgmatic
-# configuration.
-
-# Push monitors wait for the client to contact Uptime Kuma instead of Uptime
-# Kuma contacting the client. This is perfect for backup monitoring.
-Monitor Type = Push
-
-Heartbeat Interval = 90000 # = 25 hours = 1 day + 1 hour
-
-# Wait 6 times the Heartbeat Retry (below) before logging a heartbeat missed.
-Retries = 6
-
-# Multiplied by Retries this gives a grace period within which the monitor
-# goes into the "Pending" state.
-Heartbeat Retry = 360 # = 10 minutes
-
-# For each Heartbeat Interval if the backup fails repeatedly, a notification
-# is sent each time.
-Resend Notification every X times = 1
-```
-
-## Zabbix hook
-
-New in version 1.9.0
-[Zabbix](https://www.zabbix.com/) is an open-source monitoring tool used for
-tracking and managing the performance and availability of networks, servers,
-and applications in real-time.
-
-This hook does not do any notifications on its own. Instead, it relies on your
-Zabbix instance to notify and perform escalations based on the Zabbix
-configuration. The `states` defined in the configuration determine which
-states will trigger the hook. The value defined in the configuration of each
-state is used to populate the data of the configured Zabbix item. If none are
-provided, it defaults to a lower-case string of the state.
-
-An example configuration is shown here with all the available options.
-
-```yaml
-zabbix:
- server: http://cloud.zabbix.com/zabbix/api_jsonrpc.php
-
- username: myuser
- password: secret
- api_key: b2ecba64d8beb47fc161ae48b164cfd7104a79e8e48e6074ef5b141d8a0aeeca
-
- host: "borg-server"
- key: borg.status
- itemid: 55105
-
- start:
- value: "STARTED"
- finish:
- value: "OK"
- fail:
- value: "ERROR"
- states:
- - start
- - finish
- - fail
-```
-
-This hook requires the Zabbix server be running version 7.0.
-
-New in version 1.9.3 Zabbix 7.2+
-is supported as well.
-
-
-### Authentication methods
-
-Authentication can be accomplished via `api_key` or both `username` and
-`password`. If all three are declared, only `api_key` is used.
-
-
-### Items
-
-borgmatic writes its monitoring updates to a particular Zabbix item, which
-you'll need to create in advance. In the Zabbix web UI, [make a new item with a
-Type of "Zabbix
-trapper"](https://www.zabbix.com/documentation/current/en/manual/config/items/itemtypes/trapper)
-and a named Key. The "Type of information" for the item should be "Text", and
-"History" designates how much data you want to retain.
-
-When configuring borgmatic with this item to be updated, you can either declare
-the `itemid` or both `host` and `key`. If all three are declared, only `itemid`
-is used.
-
-Keep in mind that `host` refers to the "Host name" on the Zabbix server and not
-the "Visual name".
-
-
## Scripting borgmatic
To consume the output of borgmatic in other software, you can include an
diff --git a/docs/how-to/provide-your-passwords.md b/docs/how-to/provide-your-passwords.md
index 2350a1b2..84e29703 100644
--- a/docs/how-to/provide-your-passwords.md
+++ b/docs/how-to/provide-your-passwords.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 2
---
-## Providing passwords and secrets to borgmatic
-
If you want to use a Borg repository passphrase or database passwords with
borgmatic, you can set them directly in your borgmatic configuration file,
treating those secrets like any other option value. For instance, you can
@@ -40,352 +38,27 @@ should only ever get prompted for your password manager's passphrase at most
once per borgmatic run.
-### systemd service credentials
+
+
+
+
-borgmatic supports reading encrypted [systemd
-credentials](https://systemd.io/CREDENTIALS/). To use this feature, start by
-saving your password as an encrypted credential to
-`/etc/credstore.encrypted/borgmatic.pw`, e.g.,
-```bash
-systemd-ask-password -n | systemd-creds encrypt - /etc/credstore.encrypted/borgmatic.pw
-```
+### Using external credentials
-Then use the following in your configuration file:
+New in version 1.9.10 Several
+borgmatic options support reading their values directly from an external
+credential store or service. See the [credentials
+documentation](https://torsion.org/borgmatic/reference/configuration/credentials/)
+for details.
-```yaml
-encryption_passphrase: "{credential systemd borgmatic.pw}"
-```
-Prior to version 1.9.10 You can
-accomplish the same thing with this configuration:
+
-```yaml
-encryption_passcommand: cat ${CREDENTIALS_DIRECTORY}/borgmatic.pw
-```
-Note that the name `borgmatic.pw` is hardcoded in the systemd service file.
+### Using environment variables
-The `{credential ...}` syntax works for several different options in a borgmatic
-configuration file besides just `encryption_passphrase`. For instance, the
-username, password, and API token options within database and monitoring hooks
-support `{credential ...}`:
-
-```yaml
-postgresql_databases:
- - name: invoices
- username: postgres
- password: "{credential systemd borgmatic_db1}"
-```
-
-For specifics about which options are supported, see the
-[configuration
-reference](https://torsion.org/borgmatic/docs/reference/configuration/).
-
-To use these credentials, you'll need to modify the borgmatic systemd service
-file to support loading multiple credentials (assuming you need to load more
-than one or anything not named `borgmatic.pw`).
-
-Start by saving each encrypted credentials to
-`/etc/credstore.encrypted/borgmatic/`. E.g.,
-
-```bash
-mkdir /etc/credstore.encrypted/borgmatic
-systemd-ask-password -n | systemd-creds encrypt --name=borgmatic_backupserver1 - /etc/credstore.encrypted/borgmatic/backupserver1
-systemd-ask-password -n | systemd-creds encrypt --name=borgmatic_pw2 - /etc/credstore.encrypted/borgmatic/pw2
-...
-```
-
-Ensure that the file names, (e.g. `backupserver1`) match the corresponding part
-of the `--name` option *after* the underscore (_), and that the part *before*
-the underscore matches the directory name (e.g. `borgmatic`).
-
-Then, uncomment the appropriate line in the systemd service file:
-
-```
-systemctl edit borgmatic.service
-...
-# Load multiple encrypted credentials.
-LoadCredentialEncrypted=borgmatic:/etc/credstore.encrypted/borgmatic/
-```
-
-Finally, use something like the following in your borgmatic configuration file
-for each option value you'd like to load from systemd:
-
-```yaml
-encryption_passphrase: "{credential systemd borgmatic_backupserver1}"
-```
-
-Prior to version 1.9.10 Use the
-following instead, but only for the `encryption_passcommand` option and
-not other options:
-
-```yaml
-encryption_passcommand: cat ${CREDENTIALS_DIRECTORY}/borgmatic_backupserver1
-```
-
-Adjust `borgmatic_backupserver1` according to the name of the credential and the
-directory set in the service file.
-
-New in version 2.0.9 When using
-the systemd `{credential ...}` feature, borgmatic loads systemd credentials even
-when run outside of a systemd service. This works by falling back to calling
-`systemd-creds decrypt` instead of reading credentials directly. To customize
-this behavior, you can override the `systemd-creds` command and/or the
-credential store directory it uses:
-
-```yaml
-systemd:
- systemd_creds_command: /usr/local/bin/systemd-creds
- encrypted_credentials_directory: /path/to/credstore.encrypted
-```
-
-Prior to version 2.0.9 The
-systemd `{credential ...}` feature did not work when run outside of a systemd
-service. But depending on the borgmatic action invoked and the configuration
-option where `{credential ...}` was used, you could sometimes get away without
-working systemd credentials for certain actions. For instance, `borgmatic list`
-doesn't connect to any databases or monitoring services, and `borgmatic config
-validate` doesn't use credentials as all.
-
-
-### Container secrets
-
-New in version 1.9.11 When
-running inside a container, borgmatic can read [Docker
-secrets](https://docs.docker.com/compose/how-tos/use-secrets/) and [Podman
-secrets](https://www.redhat.com/en/blog/new-podman-secrets-command). Creating
-those secrets and passing them into your borgmatic container is outside the
-scope of this documentation, but here's a simple example of that with [Docker
-Compose](https://docs.docker.com/compose/):
-
-```yaml
-services:
- borgmatic:
- # Use the actual image name of your borgmatic container here.
- image: borgmatic:latest
- secrets:
- - borgmatic_passphrase
-secrets:
- borgmatic_passphrase:
- file: /etc/borgmatic/passphrase.txt
-```
-
-This assumes there's a file on the host at `/etc/borgmatic/passphrase.txt`
-containing your passphrase. Docker or Podman mounts the contents of that file
-into a secret named `borgmatic_passphrase` in the borgmatic container at
-`/run/secrets/`.
-
-Once your container secret is in place, you can consume it within your borgmatic
-configuration file:
-
-```yaml
-encryption_passphrase: "{credential container borgmatic_passphrase}"
-```
-
-This reads the secret securely from a file mounted at
-`/run/secrets/borgmatic_passphrase` within the borgmatic container.
-
-The `{credential ...}` syntax works for several different options in a borgmatic
-configuration file besides just `encryption_passphrase`. For instance, the
-username, password, and API token options within database and monitoring hooks
-support `{credential ...}`:
-
-```yaml
-postgresql_databases:
- - name: invoices
- username: postgres
- password: "{credential container borgmatic_db1}"
-```
-
-For specifics about which options are supported, see the
-[configuration
-reference](https://torsion.org/borgmatic/docs/reference/configuration/).
-
-You can also optionally override the `/run/secrets` directory that borgmatic reads secrets from
-inside a container:
-
-```yaml
-container:
- secrets_directory: /path/to/secrets
-```
-
-But you should only need to do this for development or testing purposes.
-
-
-### KeePassXC passwords
-
-New in version 1.9.11 borgmatic
-supports reading passwords from the [KeePassXC](https://keepassxc.org/) password
-manager. To use this feature, start by creating an entry in your KeePassXC
-database, putting your password into the "Password" field of that entry and
-making sure it's saved.
-
-Then, you can consume that password in your borgmatic configuration file. For
-instance, if the entry's title is "borgmatic" and your KeePassXC database is
-located at `/etc/keys.kdbx`, do this:
-
-```yaml
-encryption_passphrase: "{credential keepassxc /etc/keys.kdbx borgmatic}"
-```
-
-But if the entry's title is multiple words like `borg pw`, you'll
-need to quote it:
-
-```yaml
-encryption_passphrase: "{credential keepassxc /etc/keys.kdbx 'borg pw'}"
-```
-
-With this in place, borgmatic runs the `keepassxc-cli` command to retrieve the
-passphrase on demand. But note that `keepassxc-cli` will prompt for its own
-passphrase in order to unlock its database, so be prepared to enter it when
-running borgmatic.
-
-The `{credential ...}` syntax works for several different options in a borgmatic
-configuration file besides just `encryption_passphrase`. For instance, the
-username, password, and API token options within database and monitoring hooks
-support `{credential ...}`:
-
-```yaml
-postgresql_databases:
- - name: invoices
- username: postgres
- password: "{credential keepassxc /etc/keys.kdbx database}"
-```
-
-For specifics about which options are supported, see the
-[configuration
-reference](https://torsion.org/borgmatic/docs/reference/configuration/).
-
-You can also optionally override the `keepassxc-cli` command that borgmatic calls to load
-passwords:
-
-```yaml
-keepassxc:
- keepassxc_cli_command: /usr/local/bin/keepassxc-cli
-```
-
-
-### File-based credentials
-
-New in version 1.9.11 borgmatic
-supports reading credentials from arbitrary file paths. To use this feature,
-start by writing your credential into a file that borgmatic has permission to
-read. Take care not to include anything in the file other than your credential.
-(borgmatic is smart enough to strip off a trailing newline though.)
-
-You can consume that credential file in your borgmatic configuration. For
-instance, if your credential file is at `/credentials/borgmatic.txt`, do this:
-
-```yaml
-encryption_passphrase: "{credential file /credentials/borgmatic.txt}"
-```
-
-With this in place, borgmatic reads the credential from the file path.
-
-The `{credential ...}` syntax works for several different options in a borgmatic
-configuration file besides just `encryption_passphrase`. For instance, the
-username, password, and API token options within database and monitoring hooks
-support `{credential ...}`:
-
-```yaml
-postgresql_databases:
- - name: invoices
- username: postgres
- password: "{credential file /credentials/database.txt}"
-```
-
-For specifics about which options are supported, see the
-[configuration
-reference](https://torsion.org/borgmatic/docs/reference/configuration/).
-
-
-### Environment variable interpolation
-
-New in version 1.6.4 borgmatic
-supports interpolating arbitrary environment variables directly into option
-values in your configuration file. That means you can instruct borgmatic to
-pull your repository passphrase, your database passwords, or any other option
-values from environment variables.
-
-Be aware though that environment variables may be less secure than some of the
-other approaches above for getting credentials into borgmatic. That's because
-environment variables may be visible from within child processes and/or OS-level
-process metadata.
-
-Here's an example of using an environment variable from borgmatic's
-configuration file:
-
-```yaml
-encryption_passphrase: ${YOUR_PASSPHRASE}
-```
-
-Prior to version 1.8.0 Put
-this option in the `storage:` section of your configuration.
-
-This uses the `YOUR_PASSPHRASE` environment variable as your encryption
-passphrase. Note that the `{` `}` brackets are required. `$YOUR_PASSPHRASE` by
-itself will not work.
-
-In the case of `encryption_passphrase` in particular, an alternate approach
-is to use Borg's `BORG_PASSPHRASE` environment variable, which doesn't even
-require setting an explicit `encryption_passphrase` value in borgmatic's
-configuration file.
-
-For [database
-configuration](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/),
-the same approach applies. For example:
-
-```yaml
-postgresql_databases:
- - name: users
- password: ${YOUR_DATABASE_PASSWORD}
-```
-
-Prior to version 1.8.0 Put
-this option in the `hooks:` section of your configuration.
-
-This uses the `YOUR_DATABASE_PASSWORD` environment variable as your database
-password.
-
-
-#### Interpolation defaults
-
-If you'd like to set a default for your environment variables, you can do so
-with the following syntax:
-
-```yaml
-encryption_passphrase: ${YOUR_PASSPHRASE:-defaultpass}
-```
-
-Here, "`defaultpass`" is the default passphrase if the `YOUR_PASSPHRASE`
-environment variable is not set. Without a default, if the environment
-variable doesn't exist, borgmatic will error.
-
-
-#### Disabling interpolation
-
-To disable this environment variable interpolation feature entirely, you can
-pass the `--no-environment-interpolation` flag on the command-line.
-
-Or if you'd like to disable interpolation within a single option value, you
-can escape it with a backslash. For instance, if your password is literally
-`${A}@!`:
-
-```yaml
-encryption_passphrase: \${A}@!
-```
-
-
-## Related features
-
-Another way to override particular options within a borgmatic configuration
-file is to use a [configuration
-override](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#configuration-overrides)
-on the command-line. But please be aware of the security implications of
-specifying secrets on the command-line.
-
-Additionally, borgmatic action hooks support their own [variable
-interpolation](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/#variable-interpolation),
-although in that case it's for particular borgmatic runtime values rather than
-(only) environment variables.
+Another way to get passwords into your configuration file is by [interpolating
+arbitrary environment
+variables](https://torsion.org/borgmatic/reference/configuration/environment-variables/)
+directly into option values.
diff --git a/docs/how-to/restore-a-backup.md b/docs/how-to/restore-a-backup.md
index 30ab4cf9..1913ba65 100644
--- a/docs/how-to/restore-a-backup.md
+++ b/docs/how-to/restore-a-backup.md
@@ -1,3 +1,3 @@
-
+
diff --git a/docs/how-to/run-arbitrary-borg-commands.md b/docs/how-to/run-arbitrary-borg-commands.md
index b9601e1b..22b05619 100644
--- a/docs/how-to/run-arbitrary-borg-commands.md
+++ b/docs/how-to/run-arbitrary-borg-commands.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 12
---
-## Running Borg with borgmatic
-
Borg has several commands and options that borgmatic does not currently
support. Sometimes though, as a borgmatic user, you may find yourself wanting
to take advantage of these off-the-beaten-path Borg features. You could of
diff --git a/docs/how-to/run-preparation-steps-before-backups.md b/docs/how-to/run-preparation-steps-before-backups.md
index f6c83097..9ccf02bc 100644
--- a/docs/how-to/run-preparation-steps-before-backups.md
+++ b/docs/how-to/run-preparation-steps-before-backups.md
@@ -1,3 +1,3 @@
-
+
diff --git a/docs/how-to/set-up-backups.md b/docs/how-to/set-up-backups.md
index 56d7b7a6..e4be976c 100644
--- a/docs/how-to/set-up-backups.md
+++ b/docs/how-to/set-up-backups.md
@@ -5,14 +5,10 @@ eleventyNavigation:
parent: How-to guides
order: 0
---
-## Installation
-
-### Prerequisites
-
-First, [install
+To install borgmatic, first [install
Borg](https://borgbackup.readthedocs.io/en/stable/installation.html), at least
-version 1.1. borgmatic does not install Borg automatically so as to avoid
-conflicts with existing Borg installations.
+version 1.1. (borgmatic does not install Borg automatically so as to avoid
+conflicts with existing Borg installations.)
Then, [install pipx](https://pypa.github.io/pipx/installation/) as the root
user (with `sudo`) to make installing borgmatic easier without impacting other
@@ -147,11 +143,11 @@ care that if you uncomment a particular option, also uncomment its containing
section name—or else borgmatic won't recognize the option.
You can get the same sample configuration file from the [configuration
-reference](https://torsion.org/borgmatic/docs/reference/configuration/), the
+reference](https://torsion.org/borgmatic/reference/configuration/), the
authoritative set of all configuration options. This is handy if borgmatic has
added new options since you originally created your configuration file. Also
check out how to [upgrade your
-configuration](https://torsion.org/borgmatic/docs/how-to/upgrade/#upgrading-your-configuration).
+configuration](https://torsion.org/borgmatic/how-to/upgrade/#upgrading-your-configuration).
### Encryption
@@ -174,7 +170,7 @@ for more info.
If you'd like to configure your backups to go to multiple different
repositories, see the documentation on how to [make backups
-redundant](https://torsion.org/borgmatic/docs/how-to/make-backups-redundant/).
+redundant](https://torsion.org/borgmatic/how-to/make-backups-redundant/).
### Validation
@@ -271,45 +267,13 @@ If you'd like to specify an alternate configuration file path, use the
See `borgmatic --help` and `borgmatic create --help` for more information.
-## Default actions
+
+
+
-If you omit `create` and other actions, borgmatic runs through a set of
-default actions: `prune` any old backups as per the configured retention
-policy, `compact` segments to free up space (with Borg 1.2+, borgmatic
-1.5.23+), `create` a backup, *and* `check` backups for consistency problems
-due to things like file damage. For instance:
-
-```bash
-sudo borgmatic --verbosity 1 --list --stats
-```
-
-### Skipping actions
-
-New in version 1.8.5 You can
-configure borgmatic to skip running certain actions (default or otherwise).
-For instance, to always skip the `compact` action when using [Borg's
-append-only
-mode](https://borgbackup.readthedocs.io/en/stable/usage/notes.html#append-only-mode-forbid-compaction),
-set the `skip_actions` option:
-
-```
-skip_actions:
- - compact
-```
-
-### Disabling default actions
-
-By default, running `borgmatic` without any arguments will perform the default
-backup actions (create, prune, compact and check). If you want to disable this
-behavior and require explicit actions to be specified, add the following to
-your configuration:
-
-```yaml
-default_actions: false
-```
-
-With this setting, running `borgmatic` without arguments will show the help
-message instead of performing any actions.
+And check out the [actions
+documentation](https://torsion.org/borgmatic/reference/command-line/actions/)
+for details on customizing the actions that borgmatic runs.
## Autopilot
@@ -339,7 +303,7 @@ If you're using systemd instead of cron to run jobs, you can still configure
borgmatic to run automatically.
(If you installed borgmatic from [Other ways to
-install](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#other-ways-to-install),
+install](https://torsion.org/borgmatic/how-to/set-up-backups/#other-ways-to-install),
you may already have borgmatic systemd service and timer files. If so, you may
be able to skip some of the steps below.)
diff --git a/docs/how-to/snapshot-your-filesystems.md b/docs/how-to/snapshot-your-filesystems.md
index a2fa8297..9191679b 100644
--- a/docs/how-to/snapshot-your-filesystems.md
+++ b/docs/how-to/snapshot-your-filesystems.md
@@ -5,8 +5,6 @@ eleventyNavigation:
parent: How-to guides
order: 9
---
-## Filesystem hooks
-
Many filesystems support taking snapshots—point-in-time, read-only "copies" of
your data, ideal for backing up files that may change during the backup. These
snapshots initially don't use any additional storage space and can be made
@@ -76,7 +74,7 @@ canmount datasetname` to see the `canmount` value for a dataset.
During a backup, borgmatic automatically snapshots these discovered datasets,
temporarily mounts the snapshots within its [runtime
-directory](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory),
+directory](https://torsion.org/borgmatic/how-to/backup-your-databases/#runtime-directory),
and includes the snapshotted files in the paths sent to Borg. borgmatic is also
responsible for cleaning up (destroying) these snapshots after a backup
completes.
@@ -113,7 +111,7 @@ match `/var` to a pattern like `+ fm:/v*/lib/data`.
With Borg version 1.2 and
earlierSnapshotted files are instead stored at a path dependent on the
[runtime
-directory](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory)
+directory](https://torsion.org/borgmatic/how-to/backup-your-databases/#runtime-directory)
in use at the time the archive was created, as Borg 1.2 and earlier do not
support path rewriting.
@@ -122,7 +120,7 @@ support path rewriting.
Filesystem snapshots are stored in a Borg archive as normal files, so
you can use the standard
-[extract action](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/) to
+[extract action](https://torsion.org/borgmatic/how-to/extract-a-backup/) to
extract them.
@@ -135,7 +133,7 @@ cache](https://borgbackup.readthedocs.io/en/stable/internals/data-structures.htm
may not get cache hits on snapshotted files. This makes backing up ZFS snapshots
a little slower than non-snapshotted files that have consistent paths. You can
mitigate this by setting a fixed [runtime
-directory](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory)
+directory](https://torsion.org/borgmatic/how-to/backup-your-databases/#runtime-directory)
(that's not located in `/tmp`). This allows borgmatic to use a consistent
snapshot path from one run to the next, thereby resulting in Borg files cache
hits.
@@ -236,7 +234,7 @@ temporary snapshot directory in use at the time the archive was created, as Borg
Subvolume snapshots are stored in a Borg archive as normal files, so you can use
the standard [extract
-action](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/) to extract
+action](https://torsion.org/borgmatic/how-to/extract-a-backup/) to extract
them.
@@ -335,7 +333,7 @@ options.
During a backup, borgmatic automatically snapshots these discovered logical volumes
(non-recursively), temporarily mounts the snapshots within its [runtime
-directory](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory), and
+directory](https://torsion.org/borgmatic/how-to/backup-your-databases/#runtime-directory), and
includes the snapshotted files in the paths sent to Borg. borgmatic is also responsible for cleaning
up (deleting) these snapshots after a backup completes.
@@ -369,7 +367,7 @@ perform the backup.
With Borg version 1.2 and
earlierSnapshotted files are instead stored at a path dependent on the
[runtime
-directory](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory)
+directory](https://torsion.org/borgmatic/how-to/backup-your-databases/#runtime-directory)
in use at the time the archive was created, as Borg 1.2 and earlier do not
support path rewriting.
@@ -378,7 +376,7 @@ support path rewriting.
Logical volume snapshots are stored in a Borg archive as normal files, so
you can use the standard
-[extract action](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/) to
+[extract action](https://torsion.org/borgmatic/how-to/extract-a-backup/) to
extract them.
@@ -391,7 +389,7 @@ cache](https://borgbackup.readthedocs.io/en/stable/internals/data-structures.htm
may not get cache hits on snapshotted files. This makes backing up LVM snapshots
a little slower than non-snapshotted files that have consistent paths. You can
mitigate this by setting a fixed [runtime
-directory](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory)
+directory](https://torsion.org/borgmatic/how-to/backup-your-databases/#runtime-directory)
(that's not located in `/tmp`). This allows borgmatic to use a consistent
snapshot path from one run to the next, thereby resulting in Borg files cache
hits.
diff --git a/docs/how-to/upgrade.md b/docs/how-to/upgrade.md
index 50f13136..103e1759 100644
--- a/docs/how-to/upgrade.md
+++ b/docs/how-to/upgrade.md
@@ -5,11 +5,9 @@ eleventyNavigation:
parent: How-to guides
order: 14
---
-## Upgrading borgmatic
-
In general, all you should need to do to upgrade borgmatic if you've
[installed it with
-pipx](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#installation)
+pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation)
is to run the following:
```bash
@@ -23,11 +21,11 @@ upgrade each installation independently.
If you originally installed borgmatic with `sudo pip3 install --user`, you can
uninstall it first with `sudo pip3 uninstall borgmatic` and then [install it
again with
-pipx](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#installation),
+pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation),
which should better isolate borgmatic from your other Python applications.
But if you [installed borgmatic without pipx or
-pip3](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#other-ways-to-install),
+pip3](https://torsion.org/borgmatic/how-to/set-up-backups/#other-ways-to-install),
then your upgrade method may be different.
diff --git a/docs/reference/command-line/actions.md b/docs/reference/command-line/actions.md
new file mode 100644
index 00000000..6553c92e
--- /dev/null
+++ b/docs/reference/command-line/actions.md
@@ -0,0 +1,89 @@
+---
+title: Actions
+eleventyNavigation:
+ key: 🎬 Actions
+ parent: 💻 Command-line
+---
+An action in borgmatic is like a subcommand in Borg. The `create` action creates
+a backup, the `list` action shows the files in an archive, and so on.
+
+
+## Default actions
+
+If you omit `create` and other actions from the command-line, borgmatic runs
+through a set of default actions:
+
+1. `prune` any old backups as per the configured retention policy
+2. `compact` segments to free up space (with Borg 1.2+ and borgmatic 1.5.23+)
+3. `create` a backup
+4. `check` backups for consistency problems due to things like file damage
+
+Prior to version 1.7.9 The
+default action ordering was `prune`, `compact`, `create`, and `check`.
+
+### Disabling default actions
+
+If you want to disable this default action behavior and require explicit actions
+to be specified, add the following to your configuration:
+
+```yaml
+default_actions: false
+```
+
+With this setting, running `borgmatic` without arguments will show the help
+message instead of performing any actions.
+
+
+## A la carte actions
+
+If you find yourself wanting to customize the actions, you have some options.
+First, you can run borgmatic's `create`, `prune`, `compact`, or `check`
+actions separately. For instance, the following optional actions are
+available (among others):
+
+```bash
+borgmatic create
+borgmatic prune
+borgmatic compact
+borgmatic check
+```
+
+You can run borgmatic with only one of these actions provided, or you can mix
+and match any number of them in a single borgmatic run. This supports
+approaches like skipping certain actions while running others. For instance,
+this skips `prune` and `compact` and only runs `create` and `check`:
+
+```bash
+borgmatic create check
+```
+
+New in version 1.7.9 borgmatic
+now respects your specified command-line action order, running actions in the
+order you specify. In previous versions, borgmatic ran your specified actions
+in a fixed ordering regardless of the order they appeared on the command-line.
+
+But instead of running actions together, another option is to run backups with
+`create` on a frequent schedule (e.g. with `borgmatic create` called from one
+cron job), while only running expensive consistency checks with `check` on a
+much less frequent basis (e.g. with `borgmatic check` called from a separate
+cron job).
+
+New in version 1.8.5 Instead of
+(or in addition to) specifying actions on the command-line, you can configure
+borgmatic to [skip particular
+actions](https://torsion.org/borgmatic/how-to/set-up-backups/#skipping-actions).
+
+
+### Skipping actions
+
+New in version 1.8.5 You can
+configure borgmatic to skip running certain actions (default or otherwise).
+For instance, to always skip the `compact` action (e.g., when using [Borg's
+append-only
+mode](https://borgbackup.readthedocs.io/en/stable/usage/notes.html#append-only-mode-forbid-compaction)),
+set the `skip_actions` option in your configuration:
+
+```yaml
+skip_actions:
+ - compact
+```
diff --git a/docs/reference/command-line.md b/docs/reference/command-line/index.md
similarity index 81%
rename from docs/reference/command-line.md
rename to docs/reference/command-line/index.md
index 78f8e7f3..3f438700 100644
--- a/docs/reference/command-line.md
+++ b/docs/reference/command-line/index.md
@@ -1,18 +1,16 @@
---
-title: Command-line reference
+title: Command-line
eleventyNavigation:
- key: ⌨️ Command-line reference
+ key: 💻 Command-line
parent: Reference guides
order: 1
---
-## borgmatic options
-
Here are all of the available borgmatic command-line flags for the [most
recent version of
borgmatic](https://projects.torsion.org/borgmatic-collective/borgmatic/releases),
including the separate flags for each action (sub-command). Most of the flags
listed here do not have equivalents in borgmatic's [configuration
-file](https://torsion.org/borgmatic/docs/reference/configuration/).
+file](https://torsion.org/borgmatic/reference/configuration/).
If you're using an older version of borgmatic, some of these flags may not be
present in that version and you should instead use `borgmatic --help` or
diff --git a/docs/reference/command-line/logging.md b/docs/reference/command-line/logging.md
new file mode 100644
index 00000000..b4063367
--- /dev/null
+++ b/docs/reference/command-line/logging.md
@@ -0,0 +1,138 @@
+---
+title: Logging
+eleventyNavigation:
+ key: 🪵 Logging
+ parent: 💻 Command-line
+---
+By default, borgmatic runs proceed silently except in the case of warnings or
+errors. But if you'd like to to get additional information about the progress of
+the backup as it proceeds, use the verbosity option:
+
+```bash
+borgmatic --verbosity 1
+```
+
+Or, for even more progress and debug spew:
+
+```bash
+borgmatic --verbosity 2
+```
+
+The full set of verbosity levels are:
+
+ * `-2`: disable output entirely New in borgmatic 1.7.14
+ * `-1`: only show errors
+ * `0`: default output
+ * `1`: some additional output (informational level)
+ * `2`: lots of additional output (debug level)
+
+New in version 2.0.0Set the
+verbosity in your borgmatic configuration via the `verbosity` option.
+
+Additionally, for the `create` action in particular, you can include the
+`--list` flag to list the files that borgmatic is archiving—those files
+that are new or changed since the last backup.
+
+
+## Logging to syslog
+
+By default, borgmatic only logs its output to the console. You can enable
+simultaneous syslog logging and customize its log level with the
+`--syslog-verbosity` flag, which is independent from the console logging
+`--verbosity` flag described above. For instance, to enable syslog logging,
+run:
+
+```bash
+borgmatic --syslog-verbosity 1
+```
+
+To increase syslog logging further to include debugging information, run:
+
+```bash
+borgmatic --syslog-verbosity 2
+```
+
+See above for further details about the verbosity levels.
+
+New in version 2.0.0Set the
+syslog verbosity in your borgmatic configuration via the `syslog_verbosity`
+option.
+
+Where these logs show up depends on your particular system. If you're using
+systemd, try running `journalctl -xe`. Otherwise, try viewing
+`/var/log/syslog` or similar.
+
+Prior to version 1.8.3borgmatic
+logged to syslog by default whenever run at a non-interactive console.
+
+
+### Rate limiting
+
+If you are using rsyslog or systemd's journal, be aware that by default they
+both throttle the rate at which logging occurs. So you may need to change
+either [the global rate
+limit](https://www.rootusers.com/how-to-change-log-rate-limiting-in-linux/) or
+[the per-service rate
+limit](https://www.freedesktop.org/software/systemd/man/journald.conf.html#RateLimitIntervalSec=)
+if you're finding that borgmatic logs are missing.
+
+Note that the [sample borgmatic systemd service
+file](https://torsion.org/borgmatic/how-to/set-up-backups/#systemd)
+already has this rate limit disabled for systemd's journal.
+
+
+## Logging to file
+
+If you don't want to use syslog, and you'd rather borgmatic log to a plain
+file, use the `--log-file` flag:
+
+```bash
+borgmatic --log-file /path/to/file.log
+```
+
+Note that if you use the `--log-file` flag, you are responsible for rotating
+the log file so it doesn't grow too large, for example with
+[logrotate](https://wiki.archlinux.org/index.php/Logrotate).
+
+You can use the `--log-file-verbosity` flag to customize the log file's log level:
+
+```bash
+borgmatic --log-file /path/to/file.log --log-file-verbosity 2
+```
+
+New in version 2.0.0Set the log
+file verbosity in your borgmatic configuration via the `log_file_verbosity`
+option.
+
+New in version 1.7.11 Use the
+`--log-file-format` flag to override the default log message format. This
+format string can contain a series of named placeholders wrapped in curly
+brackets. For instance, the default log format is: `[{asctime}] {levelname}:
+{message}`. This means each log message is recorded as the log time (in square
+brackets), a logging level name, a colon, and the actual log message.
+
+So if you only want each log message to get logged *without* a timestamp or a
+logging level name:
+
+```bash
+borgmatic --log-file /path/to/file.log --log-file-format "{message}"
+```
+
+Here is a list of available placeholders:
+
+ * `{asctime}`: time the log message was created
+ * `{levelname}`: level of the log message (`INFO`, `DEBUG`, etc.)
+ * `{lineno}`: line number in the source file where the log message originated
+ * `{message}`: actual log message
+ * `{pathname}`: path of the source file where the log message originated
+
+See the [Python logging
+documentation](https://docs.python.org/3/library/logging.html#logrecord-attributes)
+for additional placeholders.
+
+Note that this `--log-file-format` flag only applies to the specified
+`--log-file` and not to syslog or other logging.
+
+New in version 2.0.0Set the
+defaults for these flags in your borgmatic configuration via the `log_file` and
+`log_file_format` options.
diff --git a/docs/reference/command-line/overrides.md b/docs/reference/command-line/overrides.md
new file mode 100644
index 00000000..e5826bb8
--- /dev/null
+++ b/docs/reference/command-line/overrides.md
@@ -0,0 +1,145 @@
+---
+title: Overrides
+eleventyNavigation:
+ key: 🔄 Overrides
+ parent: 💻 Command-line
+---
+In more complex multi-application setups, you may want to override particular
+borgmatic configuration file options at the time you run borgmatic. For
+instance, you could reuse a common configuration file for multiple
+applications, but then set the repository for each application at runtime. Or
+you might want to try a variant of an option for testing purposes without
+actually touching your configuration file.
+
+New in version 2.0.0
+Whatever the reason, you can override borgmatic configuration options at the
+command-line, as there's a command-line flag corresponding to every
+configuration option (with its underscores converted to dashes).
+
+For instance, to override the `compression` configuration option, use the
+corresponding `--compression` flag on the command-line:
+
+```bash
+borgmatic create --compression zstd
+```
+
+What this does is load your given configuration files and for each one, disregard
+the configured value for the `compression` option and use the value given on the
+command-line instead—but just for the duration of the borgmatic run.
+
+You can override nested configuration options too by separating such option
+names with a period. For instance:
+
+```bash
+borgmatic create --bootstrap.store-config-files false
+```
+
+You can even set complex option data structures by using inline YAML syntax. For
+example, set the `repositories` option with a YAML list of key/value pairs:
+
+```bash
+borgmatic create --repositories "[{path: /mnt/backup, label: local}]"
+```
+
+If your override value contains characters like colons or spaces, then you'll
+need to use quotes for it to parse correctly.
+
+You can also set individual nested options within existing list elements:
+
+```bash
+borgmatic create --repositories[0].path /mnt/backup
+```
+
+This updates the `path` option for the first repository in `repositories`.
+Change the `[0]` index as needed to address different list elements. And note
+that this only works for elements already set in configuration; you can't append
+new list elements from the command-line.
+
+See the [command-line reference
+documentation](https://torsion.org/borgmatic/reference/command-line/) for
+the full set of available arguments, including examples of each for the complex
+values.
+
+There are a handful of configuration options that don't have corresponding
+command-line flags at the global scope, but instead have flags within individual
+borgmatic actions. For instance, the `list_details` option can be overridden by
+the `--list` flag that's only present on particular actions. Similarly with
+`progress` and `--progress`, `statistics` and `--stats`, and `match_archives`
+and `--match-archives`.
+
+Also note that if you want to pass a command-line flag itself as a value to one
+of these override flags, that may not work. For instance, specifying
+`--extra-borg-options.create --no-cache-sync` results in an error, because
+`--no-cache-sync` gets interpreted as a borgmatic option (which in this case
+doesn't exist) rather than a Borg option.
+
+An alternate to command-line overrides is passing in your values via
+[environment
+variables](https://torsion.org/borgmatic/how-to/provide-your-passwords/).
+
+
+### Deprecated overrides
+
+Prior to version 2.0.0
+Configuration overrides were performed with an `--override` flag. You can still
+use `--override` with borgmatic 2.0.0+, but it's deprecated in favor of the new
+command-line flags described above.
+
+Here's an example of `--override`:
+
+```bash
+borgmatic create --override remote_path=/usr/local/bin/borg1
+```
+
+What this does is load your given configuration files and for each one, disregard
+the configured value for the `remote_path` option and use the value given on the
+command-line instead—but just for the duration of the borgmatic run.
+
+You can even override nested values or multiple values at once. For instance:
+
+```bash
+borgmatic create --override parent_option.option1=value1 --override parent_option.option2=value2
+```
+
+Prior to version 1.8.0 Don't
+forget to specify the section that an option is in. That looks like a prefix
+on the option name, e.g. `location.repositories`.
+
+Note that each value is parsed as an actual YAML string, so you can set list
+values by using brackets. For instance:
+
+```bash
+borgmatic create --override repositories=[test1.borg,test2.borg]
+```
+
+Or a single list element:
+
+```bash
+borgmatic create --override repositories=[/root/test.borg]
+```
+
+Or a single list element that is a key/value pair:
+
+```bash
+borgmatic create --override repositories="[{path: test.borg, label: test}]"
+```
+
+If your override value contains characters like colons or spaces, then you'll
+need to use quotes for it to parse correctly.
+
+Another example:
+
+```bash
+borgmatic create --override repositories="['user@server:test.borg']"
+```
+
+There is not currently a way to override a single element of a list without
+replacing the whole list.
+
+Using the `[ ]` list syntax is required when overriding an option of the list
+type (like `location.repositories`). See the [configuration
+reference](https://torsion.org/borgmatic/reference/configuration/) for
+which options are list types. (YAML list values look like `- this` with an
+indentation and a leading dash.)
+
+
diff --git a/docs/reference/configuration/archive-name-format.md b/docs/reference/configuration/archive-name-format.md
new file mode 100644
index 00000000..2ea2bbce
--- /dev/null
+++ b/docs/reference/configuration/archive-name-format.md
@@ -0,0 +1,100 @@
+---
+title: Archive name format
+eleventyNavigation:
+ key: 📛 Archive name format
+ parent: ⚙️ Configuration
+---
+If you've got multiple borgmatic configuration files, you might want to create
+archives with different naming schemes for each one. This is especially handy
+if each configuration file is backing up to the same Borg repository but you
+still want to be able to distinguish backup archives for one application from
+another.
+
+borgmatic supports this use case with an `archive_name_format` option. The
+idea is that you define a string format containing a number of [Borg
+placeholders](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-placeholders),
+and borgmatic uses that format to name any new archive it creates. For
+instance:
+
+```yaml
+archive_name_format: home-directories-{now}
+```
+
+Prior to version 1.8.0 Put
+this option in the `storage:` section of your configuration.
+
+This example means that when borgmatic creates an archive, its name will start
+with the string `home-directories-` and end with a timestamp for its creation
+time. If `archive_name_format` is unspecified, the default with Borg 1 is
+`{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}`, meaning your system hostname plus a
+timestamp in a particular format.
+
+New in borgmatic version 1.9.0 with
+Borg version 2.xThe default is just `{hostname}`, as Borg 2 does not
+require unique archive names; identical archive names form a common "series"
+that can be targeted together.
+
+
+### Archive filtering
+
+New in version 1.7.11 borgmatic
+uses the `archive_name_format` option to automatically limit which archives
+get used for actions operating on multiple archives. This prevents, for
+instance, duplicate archives from showing up in `repo-list` or `info`
+results—even if the same repository appears in multiple borgmatic
+configuration files. To take advantage of this feature, use a different
+`archive_name_format` in each configuration file.
+
+Under the hood, borgmatic accomplishes this by substituting globs for certain
+ephemeral data placeholders in your `archive_name_format`—and using the result
+to filter archives when running supported actions.
+
+For instance, let's say that you have this in your configuration:
+
+```yaml
+archive_name_format: {hostname}-user-data-{now}
+```
+
+Prior to version 1.8.0 Put
+this option in the `storage:` section of your configuration.
+
+borgmatic considers `{now}` an emphemeral data placeholder that will probably
+change per archive, while `{hostname}` won't. So it turns the example value
+into `{hostname}-user-data-*` and applies it to filter down the set of
+archives used for actions like `repo-list`, `info`, `prune`, `check`, etc.
+
+The end result is that when borgmatic runs the actions for a particular
+application-specific configuration file, it only operates on the archives
+created for that application. But this doesn't apply to actions like `compact`
+that operate on an entire repository.
+
+If this behavior isn't quite smart enough for your needs, you can use the
+`match_archives` option to override the pattern that borgmatic uses for
+filtering archives. For example:
+
+```yaml
+archive_name_format: {hostname}-user-data-{now}
+match_archives: sh:myhost-user-data-*
+```
+
+With Borg version 1.xUse a shell
+pattern for the `match_archives` value and see the [Borg patterns
+documentation](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-patterns)
+for more information.
+
+With Borg version 2.xSee the
+[match archives
+documentation](https://borgbackup.readthedocs.io/en/2.0.0b16/usage/help.html#borg-help-match-archives).
+
+Some borgmatic command-line actions also have a `--match-archives` flag that
+overrides both the auto-matching behavior and the `match_archives`
+configuration option.
+
+Prior to version 1.7.11 The way
+to limit the archives used for the `prune` action was a `prefix` option in the
+`retention` section for matching against the start of archive names. And the
+option for limiting the archives used for the `check` action was a separate
+`prefix` in the `consistency` section. Both of these options are deprecated in
+favor of the auto-matching behavior (or `match_archives`/`--match-archives`)
+in newer versions of borgmatic.
+
diff --git a/docs/reference/configuration/consistency-checks.md b/docs/reference/configuration/consistency-checks.md
new file mode 100644
index 00000000..7dd7dde1
--- /dev/null
+++ b/docs/reference/configuration/consistency-checks.md
@@ -0,0 +1,322 @@
+---
+title: Consistency checks
+eleventyNavigation:
+ key: ✅ Consistency checks
+ parent: ⚙️ Configuration
+---
+Consistency checks, run via the `check` action, serve to validate the integrity
+of your backups. borgmatic has several different named checks that perform
+different levels of this validation.
+
+By default, borgmatic runs full-repository checks (`repository`) and per-archive
+checks (`archives`) within each repository on a monthly basis. (See below about
+setting your own check frequency.)
+
+To customize the checks that borgmatic run, set the `checks` option in your
+configuration. For instance, to make borgmatic only run the `repository` check:
+
+```yaml
+checks:
+ - name: repository
+```
+
+Prior to version 1.8.0 Put
+this option in the `consistency:` section of your configuration.
+
+Prior to version 1.6.2 The
+`checks` option was a plain list of strings without the `name:` part, and
+borgmatic ran each configured check every time checks were run. For example:
+
+```yaml
+checks:
+ - repository
+```
+
+Here are the available checks, roughly from fastest to slowest:
+
+ * `archives`: Checks all of the archives' metadata in the repository.
+ * `repository`: Checks the consistency of the whole repository. The checks run
+ on the server and do not cause significant network traffic.
+ * `extract`: Performs an extraction dry-run of the latest archive.
+ * `data`: Verifies the data integrity of all archives contents, decrypting
+ and decompressing all data.
+ * `spot`: Compares file counts and contents between your source files and the
+ latest archive.
+
+Note that the `data` check is a more thorough version of the `archives` check,
+so enabling the `data` check implicitly enables the `archives` check as well.
+
+See [Borg's check
+documentation](https://borgbackup.readthedocs.io/en/stable/usage/check.html)
+for more information.
+
+
+## Spot check
+
+The various consistency checks all have trade-offs around speed and
+thoroughness, but most of them don't even look at your original source
+files—arguably one important way to ensure your backups contain the files
+you'll want to restore in the case of catastrophe (or an accidentally deleted
+file). Because if something goes wrong with your source files, most
+consistency checks will still pass with flying colors and you won't discover
+there's a problem until you go to restore.
+
+New in version 1.8.10 That's
+where the spot check comes in. This check actually compares your source file
+counts and data against those in the latest archive, potentially catching
+problems like incorrect excludes, inadvertent deletes, files changed by
+malware, etc.
+
+But because an exhaustive comparison of all source files against the latest
+archive might be too slow, the spot check supports *sampling* a percentage of
+your source files for the comparison, ensuring they fall within configured
+tolerances.
+
+Here's how it works. Start by installing the `xxhash` OS package if you don't
+already have it, so the spot check can run the `xxh64sum` command and
+efficiently hash files for comparison. Then add something like the following
+to your borgmatic configuration:
+
+```yaml
+checks:
+ - name: spot
+ count_tolerance_percentage: 10
+ data_sample_percentage: 1
+ data_tolerance_percentage: 0.5
+```
+
+The `count_tolerance_percentage` is the percentage delta between the source
+directories file count and the latest backup archive file count that is
+allowed before the entire consistency check fails. For instance, if the spot
+check runs and finds 100 source files on disk and 105 files in the latest
+archive, that would be within the configured 10% count tolerance and the check
+would succeed. But if there were 100 source files and 200 archive files, the
+check would fail. (100 source files and only 50 archive files would also
+fail.)
+
+The `data_sample_percentage` is the percentage of total files in the source
+directories to randomly sample and compare to their corresponding files in the
+latest backup archive. A higher value allows a more accurate check—and a
+slower one. The comparison is performed by hashing the selected source files
+and counting hashes that don't match the latest archive. For instance, if you
+have 1,000 source files and your sample percentage is 1%, then only 10 source
+files will be compared against the latest archive. These sampled files are
+selected randomly each time, so in effect the spot check is probabilistic.
+
+The `data_tolerance_percentage` is the percentage of total files in the source
+directories that can fail a spot check data comparison without failing the
+entire consistency check. The value must be lower than or equal to the
+`data_sample_percentage`, because `data_tolerance_percentage` only looks at
+at the sampled files as determined by `data_sample_percentage`.
+
+All three options are required when using the spot check. And because the
+check relies on these configured tolerances, it may not be a
+set-it-and-forget-it type of consistency check, at least until you get the
+tolerances dialed in so there are minimal false positives or negatives. It is
+recommended you run `borgmatic check` several times after configuring the spot
+check, tweaking your tolerances as needed. For certain workloads where your
+source files experience wild swings of file contents or counts, the spot check
+may not suitable at all.
+
+What if you add, delete, or change a bunch of your source files and you don't
+want the spot check to fail the next time it's run? Run `borgmatic create` to
+create a new backup, thereby allowing the next spot check to run against an
+archive that contains your recent changes.
+
+Because the spot check only looks at the most recent archive, you may not want
+to run it immediately after a `create` action (borgmatic's default behavior).
+Instead, it may make more sense to run the spot check on a separate schedule
+from `create`.
+
+
+## Check frequency
+
+New in version 1.6.2 You can
+optionally configure checks to run on a periodic basis rather than every time
+borgmatic runs checks. For instance:
+
+```yaml
+checks:
+ - name: repository
+ frequency: 2 weeks
+ - name: archives
+ frequency: 1 month
+```
+
+Prior to version 1.8.0 Put
+this option in the `consistency:` section of your configuration.
+
+This tells borgmatic to run the `repository` consistency check at most once
+every two weeks for a given repository and the `archives` check at most once a
+month. The `frequency` value is a number followed by a unit of time, e.g. `3
+days`, `1 week`, `2 months`, etc. The set of possible time units is as
+follows (singular or plural):
+
+ * `second`
+ * `minute`
+ * `hour`
+ * `day`
+ * `week` (7 days)
+ * `month` (30 days)
+ * `year` (365 days)
+
+The `frequency` defaults to `always` for a check configured without a
+`frequency`, which means run this check every time checks run. But if you omit
+consistency checks from configuration entirely, borgmatic runs full-repository
+checks (`repository`) and per-archive checks (`archives`) within each
+repository, at most once a month.
+
+Unlike a real scheduler like cron, borgmatic only makes a best effort to run
+checks on the configured frequency. It compares that frequency with how long
+it's been since the last check for a given repository If it hasn't been long
+enough, the check is skipped. And you still have to run `borgmatic check` (or
+`borgmatic` without actions) in order for checks to run, even when a
+`frequency` is configured!
+
+This also applies *across* configuration files that have the same repository
+configured. Make sure you have the same check frequency configured in each
+though—or the most frequently configured check will apply.
+
+New in version 1.9.0To support
+this frequency logic, borgmatic records check timestamps within the
+`~/.local/state/borgmatic/checks` directory. To override the `~/.local/state`
+portion of this path, set the `user_state_directory` configuration option.
+Alternatively, set the `XDG_STATE_HOME` environment variable.
+
+New in version 1.9.2The
+`STATE_DIRECTORY` environment variable also works for this purpose. It's set
+by systemd if `StateDirectory=borgmatic` is added to borgmatic's systemd
+service file.
+
+Prior to version 1.9.0
+borgmatic recorded check timestamps within the `~/.borgmatic` directory. At
+that time, the path was configurable by the `borgmatic_source_directory`
+configuration option (now deprecated).
+
+If you want to temporarily ignore your configured frequencies, you can invoke
+`borgmatic check --force` to run checks unconditionally.
+
+New in version 1.8.6 `borgmatic
+check --force` runs `check` even if it's specified in the `skip_actions`
+option.
+
+
+## Check days
+
+New in version 1.8.13 You can
+optionally configure checks to only run on particular days of the week. For
+instance:
+
+```yaml
+checks:
+ - name: repository
+ only_run_on:
+ - Saturday
+ - Sunday
+ - name: archives
+ only_run_on:
+ - weekday
+ - name: spot
+ only_run_on:
+ - Friday
+ - weekend
+```
+
+Each day of the week is specified in the current locale (system
+language/country settings). `weekend` and `weekday` are also accepted.
+
+As with `frequency`, borgmatic only makes a best effort to run checks on the
+given day of the week. For instance, if you run `borgmatic check` daily, then
+every day borgmatic will have an opportunity to determine whether your checks
+are configured to run on that day. If they are, then the checks run. If not,
+they are skipped.
+
+For instance, with the above configuration, if borgmatic is run on a Saturday,
+the `repository` check will run. But on a Monday? The repository check will
+get skipped. And if borgmatic is never run on a Saturday or a Sunday, that
+check will never get a chance to run.
+
+Also, the day of the week configuration applies *after* any configured
+`frequency` for a check. So for instance, imagine the following configuration:
+
+```yaml
+checks:
+ - name: repository
+ frequency: 2 weeks
+ only_run_on:
+ - Monday
+```
+
+If you run borgmatic daily with that configuration, then borgmatic will first
+wait two weeks after the previous check before running the check again—on the
+first Monday after the `frequency` duration elapses.
+
+
+## Running only checks
+
+New in version 1.7.1 If you
+would like to only run consistency checks without creating backups (for
+instance with the `check` action on the command-line), you can omit
+the `source_directories` option entirely.
+
+Prior to version 1.7.1 In older
+versions of borgmatic, instead specify an empty `source_directories` value, as
+it is a mandatory option there:
+
+```yaml
+location:
+ source_directories: []
+```
+
+
+## Disabling checks
+
+If that's still too slow, you can disable consistency checks entirely, either
+for a single repository or for all repositories.
+
+New in version 1.8.5 Disabling
+all consistency checks looks like this:
+
+```yaml
+skip_actions:
+ - check
+```
+
+Prior to version 1.8.5 Use this
+configuration instead:
+
+```yaml
+checks:
+ - name: disabled
+```
+
+Prior to version 1.8.0 Put
+`checks:` in the `consistency:` section of your configuration.
+
+Prior to version 1.6.2
+`checks:` was a plain list of strings without the `name:` part. For instance:
+
+```yaml
+checks:
+ - disabled
+```
+
+If you have multiple repositories in your borgmatic configuration file,
+you can keep running consistency checks, but only against a subset of the
+repositories:
+
+```yaml
+check_repositories:
+ - path/of/repository_to_check.borg
+```
+
+Finally, you can override your configuration file's consistency checks and
+run particular checks via the command-line. For instance:
+
+```bash
+borgmatic check --only data --only extract
+```
+
+This is useful for running slow consistency checks on an infrequent basis,
+separate from your regular checks. It is still subject to any configured
+check frequencies unless the `--force` flag is used.
diff --git a/docs/reference/configuration/constants.md b/docs/reference/configuration/constants.md
new file mode 100644
index 00000000..e01f6fd3
--- /dev/null
+++ b/docs/reference/configuration/constants.md
@@ -0,0 +1,77 @@
+---
+title: Constants
+eleventyNavigation:
+ key: 🟰 Constants
+ parent: ⚙️ Configuration
+---
+New in version 1.7.10 borgmatic
+supports defining custom configuration constants. This is similar to the
+[variable interpolation
+feature](https://torsion.org/borgmatic/how-to/add-preparation-and-cleanup-steps-to-backups/#variable-interpolation)
+for command hooks, but the constants feature lets you substitute your own custom
+values into any option values in the entire configuration file.
+
+Here's an example usage:
+
+```yaml
+constants:
+ user: foo
+ archive_prefix: bar
+
+source_directories:
+ - /home/{user}/.config
+ - /home/{user}/.ssh
+
+...
+
+archive_name_format: '{archive_prefix}-{now}'
+```
+
+Prior to version 1.8.0 Don't
+forget to specify the section (like `location:` or `storage:`) that any option
+is in.
+
+In this example, when borgmatic runs, all instances of `{user}` get replaced
+with `foo` and all instances of `{archive_prefix}` get replaced with `bar`.
+And `{now}` doesn't get replaced with anything, but gets passed directly to
+Borg, which has its own
+[placeholders](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-placeholders)
+using the same syntax as borgmatic constants. So borgmatic options like
+`archive_name_format` that get passed directly to Borg can use either Borg
+placeholders or borgmatic constants or both!
+
+After substitution, the logical result looks something like this:
+
+```yaml
+source_directories:
+ - /home/foo/.config
+ - /home/foo/.ssh
+
+...
+
+archive_name_format: 'bar-{now}'
+```
+
+Note that if you'd like to interpolate a constant into the beginning of a
+value, you'll need to quote it. For instance, this won't work:
+
+```yaml
+source_directories:
+ - {my_home_directory}/.config # This will error!
+```
+
+Instead, do this:
+
+```yaml
+source_directories:
+ - "{my_home_directory}/.config"
+```
+
+New in version 1.8.5 Constants
+work across
+[includes](https://torsion.org/borgmatic/reference/configuration/includes/),
+meaning you can define a constant and then include a separate configuration file
+that uses that constant.
+
+An alternate to constants is passing in your values via [environment
+variables](https://torsion.org/borgmatic/how-to/provide-your-passwords/).
diff --git a/docs/reference/configuration/credentials/container.md b/docs/reference/configuration/credentials/container.md
new file mode 100644
index 00000000..516bbb3d
--- /dev/null
+++ b/docs/reference/configuration/credentials/container.md
@@ -0,0 +1,62 @@
+---
+title: Container secerts
+eleventyNavigation:
+ key: • Container
+ parent: 🔒 Credentials
+---
+New in version 1.9.11 When
+running inside a container, borgmatic can read [Docker
+secrets](https://docs.docker.com/compose/how-tos/use-secrets/) and [Podman
+secrets](https://www.redhat.com/en/blog/new-podman-secrets-command). Creating
+those secrets and passing them into your borgmatic container is outside the
+scope of this documentation, but here's a simple example of that with [Docker
+Compose](https://docs.docker.com/compose/):
+
+```yaml
+services:
+ borgmatic:
+ # Use the actual image name of your borgmatic container here.
+ image: borgmatic:latest
+ secrets:
+ - borgmatic_passphrase
+secrets:
+ borgmatic_passphrase:
+ file: /etc/borgmatic/passphrase.txt
+```
+
+This assumes there's a file on the host at `/etc/borgmatic/passphrase.txt`
+containing your passphrase. Docker or Podman mounts the contents of that file
+into a secret named `borgmatic_passphrase` in the borgmatic container at
+`/run/secrets/`.
+
+Once your container secret is in place, you can consume it within your borgmatic
+configuration file:
+
+```yaml
+encryption_passphrase: "{credential container borgmatic_passphrase}"
+```
+
+This reads the secret securely from a file mounted at
+`/run/secrets/borgmatic_passphrase` within the borgmatic container.
+
+The `{credential ...}` syntax works for several different options in a borgmatic
+configuration file besides just `encryption_passphrase`. For instance, the
+username, password, and API token options within database and monitoring hooks
+support `{credential ...}`:
+
+```yaml
+postgresql_databases:
+ - name: invoices
+ username: postgres
+ password: "{credential container borgmatic_db1}"
+```
+
+You can also optionally override the `/run/secrets` directory that borgmatic reads secrets from
+inside a container:
+
+```yaml
+container:
+ secrets_directory: /path/to/secrets
+```
+
+But you should only need to do this for development or testing purposes.
diff --git a/docs/reference/configuration/credentials/file.md b/docs/reference/configuration/credentials/file.md
new file mode 100644
index 00000000..a44107a5
--- /dev/null
+++ b/docs/reference/configuration/credentials/file.md
@@ -0,0 +1,32 @@
+---
+title: File-based credentials
+eleventyNavigation:
+ key: • File
+ parent: 🔒 Credentials
+---
+New in version 1.9.11 borgmatic
+supports reading credentials from arbitrary file paths. To use this feature,
+start by writing your credential into a file that borgmatic has permission to
+read. Take care not to include anything in the file other than your credential.
+(borgmatic is smart enough to strip off a trailing newline though.)
+
+You can consume that credential file in your borgmatic configuration. For
+instance, if your credential file is at `/credentials/borgmatic.txt`, do this:
+
+```yaml
+encryption_passphrase: "{credential file /credentials/borgmatic.txt}"
+```
+
+With this in place, borgmatic reads the credential from the file path.
+
+The `{credential ...}` syntax works for several different options in a borgmatic
+configuration file besides just `encryption_passphrase`. For instance, the
+username, password, and API token options within database and monitoring hooks
+support `{credential ...}`:
+
+```yaml
+postgresql_databases:
+ - name: invoices
+ username: postgres
+ password: "{credential file /credentials/database.txt}"
+```
diff --git a/docs/reference/configuration/credentials/index.md b/docs/reference/configuration/credentials/index.md
new file mode 100644
index 00000000..0bba0c54
--- /dev/null
+++ b/docs/reference/configuration/credentials/index.md
@@ -0,0 +1,48 @@
+---
+title: Credentials
+eleventyNavigation:
+ key: 🔒 Credentials
+ parent: ⚙️ Configuration
+---
+New in version 1.9.10 Several
+borgmatic options support reading their values directly from an external
+credential store or service. To take advantage of this feature, use `{credential
+...}` syntax wherever you'd like borgmatic to read in a credential (for
+supported options). In borgmatic's configuration, this looks like:
+
+```yaml
+option: "{credential type ...}"
+```
+
+... where:
+
+ * `option` is the name of the configuration option being set
+ * `type` is the source of the credential, one of:
+ * `container`: [Container secrets](https://torsion.org/borgmatic/reference/configuration/credentials/container/)
+ * `file`: [File-based credentials](https://torsion.org/borgmatic/reference/configuration/credentials/file/)
+ * `keepassxc`: [KeePassXC passwords](https://torsion.org/borgmatic/reference/configuration/credentials/keepassxc/)
+ * `systemd`: [systemd service credentials](https://torsion.org/borgmatic/reference/configuration/credentials/systemd/)
+ * "`...`" provides additional arguments specific to the selected credential
+ type
+
+For example:
+
+```yaml
+encryption_passphrase: "{credential systemd borgmatic.pw}"
+```
+
+The `{credential ...}` syntax works for several different options in a borgmatic
+configuration file besides just `encryption_passphrase`. For instance, the
+username, password, and API token options within database and monitoring hooks
+support `{credential ...}`:
+
+```yaml
+postgresql_databases:
+ - name: invoices
+ username: postgres
+ password: "{credential systemd borgmatic_db1}"
+```
+
+For details about which options support the use of `{credential ...}` syntax,
+see the [configuration
+reference](https://torsion.org/borgmatic/reference/configuration/).
diff --git a/docs/reference/configuration/credentials/keepassxc.md b/docs/reference/configuration/credentials/keepassxc.md
new file mode 100644
index 00000000..f6442c27
--- /dev/null
+++ b/docs/reference/configuration/credentials/keepassxc.md
@@ -0,0 +1,51 @@
+---
+title: KeePassXC passwords
+eleventyNavigation:
+ key: • KeePassXC
+ parent: 🔒 Credentials
+---
+New in version 1.9.11 borgmatic
+supports reading passwords from the [KeePassXC](https://keepassxc.org/) password
+manager. To use this feature, start by creating an entry in your KeePassXC
+database, putting your password into the "Password" field of that entry and
+making sure it's saved.
+
+Then, you can consume that password in your borgmatic configuration file. For
+instance, if the entry's title is "borgmatic" and your KeePassXC database is
+located at `/etc/keys.kdbx`, do this:
+
+```yaml
+encryption_passphrase: "{credential keepassxc /etc/keys.kdbx borgmatic}"
+```
+
+But if the entry's title is multiple words like `borg pw`, you'll
+need to quote it:
+
+```yaml
+encryption_passphrase: "{credential keepassxc /etc/keys.kdbx 'borg pw'}"
+```
+
+With this in place, borgmatic runs the `keepassxc-cli` command to retrieve the
+passphrase on demand. But note that `keepassxc-cli` will prompt for its own
+passphrase in order to unlock its database, so be prepared to enter it when
+running borgmatic.
+
+The `{credential ...}` syntax works for several different options in a borgmatic
+configuration file besides just `encryption_passphrase`. For instance, the
+username, password, and API token options within database and monitoring hooks
+support `{credential ...}`:
+
+```yaml
+postgresql_databases:
+ - name: invoices
+ username: postgres
+ password: "{credential keepassxc /etc/keys.kdbx database}"
+```
+
+You can also optionally override the `keepassxc-cli` command that borgmatic calls to load
+passwords:
+
+```yaml
+keepassxc:
+ keepassxc_cli_command: /usr/local/bin/keepassxc-cli
+```
diff --git a/docs/reference/configuration/credentials/systemd.md b/docs/reference/configuration/credentials/systemd.md
new file mode 100644
index 00000000..f0974d2d
--- /dev/null
+++ b/docs/reference/configuration/credentials/systemd.md
@@ -0,0 +1,108 @@
+---
+title: systemd service credentials
+eleventyNavigation:
+ key: • systemd
+ parent: 🔒 Credentials
+---
+New in version 1.9.10 borgmatic
+supports reading encrypted [systemd
+credentials](https://systemd.io/CREDENTIALS/). To use this feature, start by
+saving your password as an encrypted credential to
+`/etc/credstore.encrypted/borgmatic.pw`, e.g.,
+
+```bash
+systemd-ask-password -n | systemd-creds encrypt - /etc/credstore.encrypted/borgmatic.pw
+```
+
+Then use the following in your configuration file:
+
+```yaml
+encryption_passphrase: "{credential systemd borgmatic.pw}"
+```
+
+Prior to version 1.9.10 You can
+accomplish the same thing with this configuration:
+
+```yaml
+encryption_passcommand: cat ${CREDENTIALS_DIRECTORY}/borgmatic.pw
+```
+
+Note that the name `borgmatic.pw` is hardcoded in the systemd service file.
+
+The `{credential ...}` syntax works for several different options in a borgmatic
+configuration file besides just `encryption_passphrase`. For instance, the
+username, password, and API token options within database and monitoring hooks
+support `{credential ...}`:
+
+```yaml
+postgresql_databases:
+ - name: invoices
+ username: postgres
+ password: "{credential systemd borgmatic_db1}"
+```
+
+To use these credentials, you'll need to modify the borgmatic systemd service
+file to support loading multiple credentials (assuming you need to load more
+than one or anything not named `borgmatic.pw`).
+
+Start by saving each encrypted credentials to
+`/etc/credstore.encrypted/borgmatic/`. E.g.,
+
+```bash
+mkdir /etc/credstore.encrypted/borgmatic
+systemd-ask-password -n | systemd-creds encrypt --name=borgmatic_backupserver1 - /etc/credstore.encrypted/borgmatic/backupserver1
+systemd-ask-password -n | systemd-creds encrypt --name=borgmatic_pw2 - /etc/credstore.encrypted/borgmatic/pw2
+...
+```
+
+Ensure that the file names, (e.g. `backupserver1`) match the corresponding part
+of the `--name` option *after* the underscore (_), and that the part *before*
+the underscore matches the directory name (e.g. `borgmatic`).
+
+Then, uncomment the appropriate line in the systemd service file:
+
+```
+systemctl edit borgmatic.service
+...
+# Load multiple encrypted credentials.
+LoadCredentialEncrypted=borgmatic:/etc/credstore.encrypted/borgmatic/
+```
+
+Finally, use something like the following in your borgmatic configuration file
+for each option value you'd like to load from systemd:
+
+```yaml
+encryption_passphrase: "{credential systemd borgmatic_backupserver1}"
+```
+
+Prior to version 1.9.10 Use the
+following instead, but only for the `encryption_passcommand` option and
+not other options:
+
+```yaml
+encryption_passcommand: cat ${CREDENTIALS_DIRECTORY}/borgmatic_backupserver1
+```
+
+Adjust `borgmatic_backupserver1` according to the name of the credential and the
+directory set in the service file.
+
+New in version 2.0.9 When using
+the systemd `{credential ...}` feature, borgmatic loads systemd credentials even
+when run outside of a systemd service. This works by falling back to calling
+`systemd-creds decrypt` instead of reading credentials directly. To customize
+this behavior, you can override the `systemd-creds` command and/or the
+credential store directory it uses:
+
+```yaml
+systemd:
+ systemd_creds_command: /usr/local/bin/systemd-creds
+ encrypted_credentials_directory: /path/to/credstore.encrypted
+```
+
+Prior to version 2.0.9 The
+systemd `{credential ...}` feature did not work when run outside of a systemd
+service. But depending on the borgmatic action invoked and the configuration
+option where `{credential ...}` was used, you could sometimes get away without
+working systemd credentials for certain actions. For instance, `borgmatic list`
+doesn't connect to any databases or monitoring services, and `borgmatic config
+validate` doesn't use credentials as all.
diff --git a/docs/reference/configuration/environment-variables.md b/docs/reference/configuration/environment-variables.md
new file mode 100644
index 00000000..363940c3
--- /dev/null
+++ b/docs/reference/configuration/environment-variables.md
@@ -0,0 +1,93 @@
+---
+title: Environment variables
+eleventyNavigation:
+ key: 💲 Environment variables
+ parent: ⚙️ Configuration
+---
+New in version 1.6.4 borgmatic
+supports interpolating arbitrary environment variables directly into option
+values in your configuration file. That means you can instruct borgmatic to
+pull your repository passphrase, your database passwords, or any other option
+values from environment variables.
+
+Be aware though that environment variables may be less secure than some of the
+other approaches above for getting credentials into borgmatic. That's because
+environment variables may be visible from within child processes and/or OS-level
+process metadata.
+
+Here's an example of using an environment variable from borgmatic's
+configuration file:
+
+```yaml
+encryption_passphrase: ${YOUR_PASSPHRASE}
+```
+
+Prior to version 1.8.0 Put
+this option in the `storage:` section of your configuration.
+
+This uses the `YOUR_PASSPHRASE` environment variable as your encryption
+passphrase. Note that the `{` `}` brackets are required. `$YOUR_PASSPHRASE` by
+itself will not work.
+
+In the case of `encryption_passphrase` in particular, an alternate approach
+is to use Borg's `BORG_PASSPHRASE` environment variable, which doesn't even
+require setting an explicit `encryption_passphrase` value in borgmatic's
+configuration file.
+
+For [database
+configuration](https://torsion.org/borgmatic/how-to/backup-your-databases/),
+the same approach applies. For example:
+
+```yaml
+postgresql_databases:
+ - name: users
+ password: ${YOUR_DATABASE_PASSWORD}
+```
+
+Prior to version 1.8.0 Put
+this option in the `hooks:` section of your configuration.
+
+This uses the `YOUR_DATABASE_PASSWORD` environment variable as your database
+password.
+
+
+### Interpolation defaults
+
+If you'd like to set a default for your environment variables, you can do so
+with the following syntax:
+
+```yaml
+encryption_passphrase: ${YOUR_PASSPHRASE:-defaultpass}
+```
+
+Here, "`defaultpass`" is the default passphrase if the `YOUR_PASSPHRASE`
+environment variable is not set. Without a default, if the environment
+variable doesn't exist, borgmatic will error.
+
+
+### Disabling interpolation
+
+To disable this environment variable interpolation feature entirely, you can
+pass the `--no-environment-interpolation` flag on the command-line.
+
+Or if you'd like to disable interpolation within a single option value, you
+can escape it with a backslash. For instance, if your password is literally
+`${A}@!`:
+
+```yaml
+encryption_passphrase: \${A}@!
+```
+
+
+## Related features
+
+Another way to override particular options within a borgmatic configuration
+file is to use a [configuration
+override](https://torsion.org/borgmatic/reference/configuration/overrides/)
+on the command-line. But please be aware of the security implications of
+specifying secrets on the command-line.
+
+Additionally, borgmatic action hooks support their own [variable
+interpolation](https://torsion.org/borgmatic/how-to/add-preparation-and-cleanup-steps-to-backups/#variable-interpolation),
+although in that case it's for particular borgmatic runtime values rather than
+(only) environment variables.
diff --git a/docs/reference/configuration/includes.md b/docs/reference/configuration/includes.md
new file mode 100644
index 00000000..e11bd6ea
--- /dev/null
+++ b/docs/reference/configuration/includes.md
@@ -0,0 +1,319 @@
+---
+title: Includes
+eleventyNavigation:
+ key: ❗ Includes
+ parent: ⚙️ Configuration
+---
+Once you have multiple different configuration files, you might want to share
+common configuration options across these files without having to copy and paste
+them. To achieve this, you can put fragments of common configuration options
+into a file and then include or inline that file into one or more borgmatic
+configuration files.
+
+Let's say that you want to include common consistency check configuration across all
+of your configuration files. You could do that in each configuration file with
+the following:
+
+```yaml
+repositories:
+ - path: repo.borg
+
+checks:
+ !include /etc/borgmatic/common_checks.yaml
+```
+
+Prior to version 1.8.0 These
+options were organized into sections like `location:` and `consistency:`.
+
+The contents of `common_checks.yaml` could be:
+
+```yaml
+- name: repository
+ frequency: 3 weeks
+- name: archives
+ frequency: 2 weeks
+```
+
+To prevent borgmatic from trying to load these configuration fragments by
+themselves and complaining that they are not valid configuration files, you
+should put them in a directory other than `/etc/borgmatic.d/`. (A subdirectory
+is fine.)
+
+When a configuration include is a relative path, borgmatic loads it from either
+the current working directory or from the directory containing the file doing
+the including.
+
+Note that this form of include must be a value rather than an option name. For
+example, this will not work:
+
+```yaml
+repositories:
+ - path: repo.borg
+
+# Don't do this. It won't work!
+!include /etc/borgmatic/common_checks.yaml
+```
+
+But if you do want to merge in a option name *and* its values, keep reading!
+
+
+## Include merging
+
+If you need to get even fancier and merge in common configuration options, you
+can perform a YAML merge of included configuration using the YAML `<<` key.
+For instance, here's an example of a main configuration file that pulls in
+retention and consistency checks options via a single include:
+
+```yaml
+repositories:
+ - path: repo.borg
+
+<<: !include /etc/borgmatic/common.yaml
+```
+
+This is what `common.yaml` might look like:
+
+```yaml
+keep_hourly: 24
+keep_daily: 7
+
+checks:
+ - name: repository
+ frequency: 3 weeks
+ - name: archives
+ frequency: 2 weeks
+```
+
+Prior to version 1.8.0 These
+options were organized into sections like `retention:` and `consistency:`.
+
+Once this include gets merged in, the resulting configuration has all of the
+options from the original configuration file *and* the options from the
+include.
+
+Note that this `<<` include merging syntax is only for merging in mappings
+(configuration options and their values). If you'd like to include a single
+value directly, please see above about standard includes.
+
+
+### Multiple merge includes
+
+borgmatic has a limitation preventing multiple `<<` include merges per file or
+option value. This means you can do a single `<<` merge at the global level,
+another `<<` within each nested option value, etc. (This is a YAML
+limitation.) For instance:
+
+```yaml
+repositories:
+ - path: repo.borg
+
+# This won't work! You can't do multiple merges like this at the same level.
+<<: !include common1.yaml
+<<: !include common2.yaml
+```
+
+But read on for a way around this.
+
+New in version 1.8.1 You can
+include and merge multiple configuration files all at once. For instance:
+
+```yaml
+repositories:
+ - path: repo.borg
+
+<<: !include [common1.yaml, common2.yaml, common3.yaml]
+```
+
+This merges in each included configuration file in turn, such that later files
+replace the options in earlier ones.
+
+Here's another way to do the same thing:
+
+```yaml
+repositories:
+ - path: repo.borg
+
+<<: !include
+ - common1.yaml
+ - common2.yaml
+ - common3.yaml
+```
+
+
+### Deep merge
+
+New in version 1.6.0 borgmatic
+performs a deep merge of merged include files, meaning that values are merged
+at all levels in the two configuration files. This allows you to include
+common configuration—up to full borgmatic configuration files—while overriding
+only the parts you want to customize.
+
+For instance, here's an example of a main configuration file that pulls in
+options via an include and then overrides one of them locally:
+
+```yaml
+<<: !include /etc/borgmatic/common.yaml
+
+constants:
+ base_directory: /opt
+
+repositories:
+ - path: repo.borg
+```
+
+This is what `common.yaml` might look like:
+
+```yaml
+constants:
+ app_name: myapp
+ base_directory: /var/lib
+```
+
+Once this include gets merged in, the resulting configuration would have an
+`app_name` value of `myapp` and an overridden `base_directory` value of
+`/opt`.
+
+When there's an option collision between the local file and the merged
+include, the local file's option takes precedence.
+
+
+#### List merge
+
+New in version 1.6.1 Colliding
+list values are appended together.
+
+New in version 1.7.12 If there
+is a list value from an include that you *don't* want in your local
+configuration file, you can omit it with an `!omit` tag. For instance:
+
+```yaml
+<<: !include /etc/borgmatic/common.yaml
+
+source_directories:
+ - !omit /home
+ - /var
+```
+
+And `common.yaml` like this:
+
+```yaml
+source_directories:
+ - /home
+ - /etc
+```
+
+Prior to version 1.8.0 Put the
+`source_directories` option in the `location:` section of your configuration.
+
+Once this include gets merged in, the resulting configuration will have a
+`source_directories` value of `/etc` and `/var`—with `/home` omitted.
+
+This feature currently only works on scalar (e.g. string or number) list items
+and will not work elsewhere in a configuration file. Be sure to put the
+`!omit` tag *before* the list item (after the dash). Putting `!omit` after the
+list item will not work, as it gets interpreted as part of the string. Here's
+an example of some things not to do:
+
+```yaml
+<<: !include /etc/borgmatic/common.yaml
+
+source_directories:
+ # Do not do this! It will not work. "!omit" belongs before "/home".
+ - /home !omit
+
+# Do not do this either! "!omit" only works on scalar list items.
+repositories: !omit
+ # Also do not do this for the same reason! This is a list item, but it's
+ # not a scalar.
+ - !omit path: repo.borg
+```
+
+Additionally, the `!omit` tag only works in a configuration file that also
+performs a merge include with `<<: !include`. It doesn't make sense within,
+for instance, an included configuration file itself (unless it in turn
+performs its own merge include). That's because `!omit` only applies to the
+file doing the include; it doesn't work in reverse or propagate through
+includes.
+
+
+### Shallow merge
+
+Even though deep merging is generally pretty handy for included files,
+sometimes you want specific options in the local file to take precedence over
+included options—without any merging occurring for them.
+
+New in version 1.7.12 That's
+where the `!retain` tag comes in. Whenever you're merging an included file
+into your configuration file, you can optionally add the `!retain` tag to
+particular local mappings or lists to retain the local values and ignore
+included values.
+
+For instance, start with this configuration file containing the `!retain` tag
+on the `retention` mapping:
+
+```yaml
+<<: !include /etc/borgmatic/common.yaml
+
+repositories:
+ - path: repo.borg
+
+checks: !retain
+ - name: repository
+```
+
+And `common.yaml` like this:
+
+```yaml
+repositories:
+ - path: common.borg
+
+checks:
+ - name: archives
+```
+
+Prior to version 1.8.0 These
+options were organized into sections like `location:` and `consistency:`.
+
+Once this include gets merged in, the resulting configuration will have a
+`checks` value with a name of `repository` and no other values. That's because
+the `!retain` tag says to retain the local version of `checks` and ignore any
+values coming in from the include. But because the `repositories` list doesn't
+have a `!retain` tag, it still gets merged together to contain both
+`common.borg` and `repo.borg`.
+
+The `!retain` tag can only be placed on mappings (keys/values) and lists, and
+it goes right after the name of the option (and its colon) on the same line.
+The effects of `!retain` are recursive, meaning that if you place a `!retain`
+tag on a top-level mapping, even deeply nested values within it will not be
+merged.
+
+Additionally, the `!retain` tag only works in a configuration file that also
+performs a merge include with `<<: !include`. It doesn't make sense within,
+for instance, an included configuration file itself (unless it in turn
+performs its own merge include). That's because `!retain` only applies to the
+file doing the include; it doesn't work in reverse or propagate through
+includes.
+
+
+## Debugging includes
+
+New in version 1.7.15 If you'd
+like to see what the loaded configuration looks like after includes get merged
+in, run the `validate` action on your configuration file:
+
+```bash
+sudo borgmatic config validate --show
+```
+
+In version 1.7.12 through
+1.7.14 Use this command instead:
+
+```bash
+sudo validate-borgmatic-config --show
+```
+
+You'll need to specify your configuration file with `--config` if it's not in
+a default location.
+
+This will output the merged configuration as borgmatic sees it, which can be
+helpful for understanding how your includes work in practice.
diff --git a/docs/reference/configuration.md b/docs/reference/configuration/index.md
similarity index 67%
rename from docs/reference/configuration.md
rename to docs/reference/configuration/index.md
index 0f8742bf..905ec15d 100644
--- a/docs/reference/configuration.md
+++ b/docs/reference/configuration/index.md
@@ -1,22 +1,20 @@
---
-title: Configuration reference
+title: Configuration
eleventyNavigation:
- key: ⚙️ Configuration reference
+ key: ⚙️ Configuration
parent: Reference guides
order: 0
---
-## Configuration file
-
Below is a sample borgmatic configuration file including all available options
for the [most recent version of
borgmatic](https://projects.torsion.org/borgmatic-collective/borgmatic/releases).
This file is also [available for
-download](https://torsion.org/borgmatic/docs/reference/config.yaml).
+download](https://torsion.org/borgmatic/reference/config.yaml).
If you're using an older version of borgmatic, some of these options may not
work, and you should instead [generate a sample configuration file specific to
your borgmatic
-version](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#configuration).
+version](https://torsion.org/borgmatic/how-to/set-up-backups/#configuration).
```yaml
{% include borgmatic/config.yaml %}
diff --git a/docs/reference/configuration/monitoring/apprise.md b/docs/reference/configuration/monitoring/apprise.md
new file mode 100644
index 00000000..dee0b6ed
--- /dev/null
+++ b/docs/reference/configuration/monitoring/apprise.md
@@ -0,0 +1,106 @@
+---
+title: Apprise
+eleventyNavigation:
+ key: • Apprise
+ parent: 🚨 Monitoring
+---
+New in version 1.8.4
+[Apprise](https://github.com/caronc/apprise/wiki) is a local notification library
+that "allows you to send a notification to almost all of the most popular
+[notification services](https://github.com/caronc/apprise/wiki) available to
+us today such as: Telegram, Discord, Slack, Amazon SNS, Gotify, etc."
+
+Depending on how you installed borgmatic, it may not have come with Apprise.
+For instance, if you originally [installed borgmatic with
+pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation),
+run the following to install Apprise so borgmatic can use it:
+
+```bash
+sudo pipx uninstall borgmatic
+sudo pipx install borgmatic[Apprise]
+```
+
+Omit `sudo` if borgmatic is installed as a non-root user.
+
+Once Apprise is installed, configure borgmatic to notify one or more [Apprise
+services](https://github.com/caronc/apprise/wiki). For example:
+
+```yaml
+apprise:
+ services:
+ - url: gotify://hostname/token
+ label: gotify
+ - url: mastodons://access_key@hostname/@user
+ label: mastodon
+ states:
+ - start
+ - finish
+ - fail
+```
+
+With this configuration, borgmatic pings each of the configured Apprise
+services when a backup begins, ends, or errors, but only when any of the
+`create`, `prune`, `compact`, or `check` actions are run. (By default, if
+`states` is not specified, Apprise services are only pinged on error.)
+
+You can optionally customize the contents of the default messages sent to
+these services:
+
+```yaml
+apprise:
+ services:
+ - url: gotify://hostname/token
+ label: gotify
+ start:
+ title: Ping!
+ body: Starting backup process.
+ finish:
+ title: Ping!
+ body: Backups successfully made.
+ fail:
+ title: Ping!
+ body: Your backups have failed.
+ states:
+ - start
+ - finish
+ - fail
+```
+
+New in version 1.8.9 borgmatic
+logs are automatically included in the body data sent to your Apprise services
+when a backup finishes or fails.
+
+You can customize the verbosity of the logs that are sent with borgmatic's
+`--monitoring-verbosity` flag. The `--list` and `--stats` flags may also be of
+use. See `borgmatic create --help` for more information.
+
+New in version 2.0.0Set the
+defaults for these flags in your borgmatic configuration via the
+`monitoring_verbosity`, `list`, and `statistics` options.
+
+If you don't want any logs sent, you can disable log sending by setting
+`send_logs` to `false`:
+
+```yaml
+apprise:
+ services:
+ - url: gotify://hostname/token
+ label: gotify
+ send_logs: false
+```
+
+Or to limit the size of logs sent to Apprise services:
+
+```yaml
+apprise:
+ services:
+ - url: gotify://hostname/token
+ label: gotify
+ logs_size_limit: 500
+```
+
+This may be necessary for some services that reject large requests.
+
+See the [configuration
+reference](https://torsion.org/borgmatic/reference/configuration/) for
+details.
diff --git a/docs/reference/configuration/monitoring/cronhub.md b/docs/reference/configuration/monitoring/cronhub.md
new file mode 100644
index 00000000..1794e428
--- /dev/null
+++ b/docs/reference/configuration/monitoring/cronhub.md
@@ -0,0 +1,33 @@
+---
+title: Cronhub
+eleventyNavigation:
+ key: • Cronhub
+ parent: 🚨 Monitoring
+---
+[Cronhub](https://cronhub.io/) provides "instant alerts when any of your
+background jobs fail silently or run longer than expected," and borgmatic has
+built-in integration with it. Once you create a Cronhub account and monitor on
+their site, all you need to do is configure borgmatic with the unique "Ping
+URL" for your monitor. Here's an example:
+
+
+```yaml
+cronhub:
+ ping_url: https://cronhub.io/start/1f5e3410-254c-11e8-b61d-55875966d031
+```
+
+Prior to version 1.8.0 Put
+this option in the `hooks:` section of your configuration.
+
+With this configuration, borgmatic pings your Cronhub monitor when a backup
+begins, ends, or errors, but only when any of the `create`, `prune`,
+`compact`, or `check` actions are run. Then, if the actions complete
+successfully or errors, borgmatic notifies Cronhub accordingly.
+
+Note that even though you configure borgmatic with the "start" variant of the
+ping URL, borgmatic substitutes the correct state into the URL when pinging
+Cronhub ("start", "finish", or "fail").
+
+You can configure Cronhub to notify you by a [variety of
+mechanisms](https://docs.cronhub.io/integrations.html) when backups fail
+or it doesn't hear from borgmatic for a certain period of time.
diff --git a/docs/reference/configuration/monitoring/cronitor.md b/docs/reference/configuration/monitoring/cronitor.md
new file mode 100644
index 00000000..b4c7b148
--- /dev/null
+++ b/docs/reference/configuration/monitoring/cronitor.md
@@ -0,0 +1,29 @@
+---
+title: Cronitor
+eleventyNavigation:
+ key: • Cronitor
+ parent: 🚨 Monitoring
+---
+[Cronitor](https://cronitor.io/) provides "Cron monitoring and uptime healthchecks
+for websites, services and APIs," and borgmatic has built-in
+integration with it. Once you create a Cronitor account and cron job monitor on
+their site, all you need to do is configure borgmatic with the unique "Ping
+API URL" for your monitor. Here's an example:
+
+
+```yaml
+cronitor:
+ ping_url: https://cronitor.link/d3x0c1
+```
+
+Prior to version 1.8.0 Put
+this option in the `hooks:` section of your configuration.
+
+With this configuration, borgmatic pings your Cronitor monitor when a backup
+begins, ends, or errors, but only when any of the `create`, `prune`,
+`compact`, or `check` actions are run. Then, if the actions complete
+successfully or errors, borgmatic notifies Cronitor accordingly.
+
+You can configure Cronitor to notify you by a [variety of
+mechanisms](https://cronitor.io/docs/cron-job-notifications) when backups fail
+or it doesn't hear from borgmatic for a certain period of time.
diff --git a/docs/reference/configuration/monitoring/healthchecks.md b/docs/reference/configuration/monitoring/healthchecks.md
new file mode 100644
index 00000000..68fd63e5
--- /dev/null
+++ b/docs/reference/configuration/monitoring/healthchecks.md
@@ -0,0 +1,50 @@
+---
+title: Healthchecks
+eleventyNavigation:
+ key: • Healthchecks
+ parent: 🚨 Monitoring
+---
+[Healthchecks](https://healthchecks.io/) is a service that provides "instant
+alerts when your cron jobs fail silently," and borgmatic has built-in
+integration with it. Once you create a Healthchecks account and project on
+their site, all you need to do is configure borgmatic with the unique "Ping
+URL" for your project. Here's an example:
+
+
+```yaml
+healthchecks:
+ ping_url: https://hc-ping.com/addffa72-da17-40ae-be9c-ff591afb942a
+```
+
+Prior to version 1.8.0 Put
+this option in the `hooks:` section of your configuration.
+
+With this configuration, borgmatic pings your Healthchecks project when a
+backup begins, ends, or errors, but only when any of the `create`, `prune`,
+`compact`, or `check` actions are run.
+
+Then, if the actions complete successfully, borgmatic notifies Healthchecks of
+the success and includes borgmatic logs in the payload data sent to
+Healthchecks. This means that borgmatic logs show up in the Healthchecks UI,
+although be aware that Healthchecks currently has a 100-kilobyte limit for the
+logs in each ping.
+
+If an error occurs during any action or hook, borgmatic notifies Healthchecks,
+also tacking on logs including the error itself. But the logs are only
+included for errors that occur when a `create`, `prune`, `compact`, or `check`
+action is run.
+
+You can customize the verbosity of the logs that are sent to Healthchecks with
+borgmatic's `--monitoring-verbosity` flag. The `--list` and `--stats` flags
+may also be of use. See `borgmatic create --help` for more information.
+Additionally, see the [borgmatic configuration
+file](https://torsion.org/borgmatic/reference/configuration/) for
+additional Healthchecks options.
+
+New in version 2.0.0Set the
+defaults for these flags in your borgmatic configuration via the
+`monitoring_verbosity`, `list`, and `statistics` options.
+
+You can configure Healthchecks to notify you by a [variety of
+mechanisms](https://healthchecks.io/#welcome-integrations) when backups fail
+or it doesn't hear from borgmatic for a certain period of time.
diff --git a/docs/reference/configuration/monitoring/index.md b/docs/reference/configuration/monitoring/index.md
new file mode 100644
index 00000000..dcf63d88
--- /dev/null
+++ b/docs/reference/configuration/monitoring/index.md
@@ -0,0 +1,25 @@
+---
+title: Monitoring
+eleventyNavigation:
+ key: 🚨 Monitoring
+ parent: ⚙️ Configuration
+---
+borgmatic integrates with third-party monitoring services and libraries, pinging
+them as backups happen. The idea is that you'll receive an alert when something
+goes wrong or when the service doesn't hear from borgmatic for a configured
+interval (if supported). While these services and libraries offer different
+features, you probably only need to use one of them at most. See their
+documentation for configuration information:
+
+ * [Apprise](https://torsion.org/borgmatic/reference/configuration/monitoring/apprise/)
+ * [Cronhub](https://torsion.org/borgmatic/reference/configuration/monitoring/cronhub/)
+ * [Cronitor](https://torsion.org/borgmatic/reference/configuration/monitoring/cronitor/)
+ * [Grafana Loki](https://torsion.org/borgmatic/reference/configuration/monitoring/loki/)
+ * [Healthchecks](https://torsion.org/borgmatic/reference/configuration/monitoring/healthchecks/)
+ * [ntfy](https://torsion.org/borgmatic/reference/configuration/monitoring/ntfy/)
+ * [PagerDuty](https://torsion.org/borgmatic/reference/configuration/monitoring/pagerduty/)
+ * [Pushover](https://torsion.org/borgmatic/reference/configuration/monitoring/pushover/)
+ * [Sentry](https://torsion.org/borgmatic/reference/configuration/monitoring/sentry/)
+ * [Uptime Kuma](https://torsion.org/borgmatic/reference/configuration/monitoring/uptime-kuma/)
+ * [Zabbix](https://torsion.org/borgmatic/reference/configuration/monitoring/zabbix/)
+
diff --git a/docs/reference/configuration/monitoring/loki.md b/docs/reference/configuration/monitoring/loki.md
new file mode 100644
index 00000000..62db0c80
--- /dev/null
+++ b/docs/reference/configuration/monitoring/loki.md
@@ -0,0 +1,55 @@
+---
+title: Loki
+eleventyNavigation:
+ key: • Loki
+ parent: 🚨 Monitoring
+---
+New in version 1.8.3 [Grafana
+Loki](https://grafana.com/oss/loki/) is a "horizontally scalable, highly
+available, multi-tenant log aggregation system inspired by Prometheus."
+borgmatic has built-in integration with Loki, sending both backup status and
+borgmatic logs.
+
+You can configure borgmatic to use either a [self-hosted Loki
+instance](https://grafana.com/docs/loki/latest/installation/) or [a Grafana
+Cloud account](https://grafana.com/auth/sign-up/create-user). Start by setting
+your Loki API push URL. Here's an example:
+
+```yaml
+loki:
+ url: http://localhost:3100/loki/api/v1/push
+
+ labels:
+ app: borgmatic
+ hostname: example.org
+```
+
+With this configuration, borgmatic sends its logs to your Loki instance as any
+of the `create`, `prune`, `compact`, or `check` actions are run. Then, after
+the actions complete, borgmatic notifies Loki of success or failure.
+
+This hook supports sending arbitrary labels to Loki. At least one label is
+required.
+
+There are also a few placeholders you can optionally use as label values:
+
+ * `__config`: name of the borgmatic configuration file
+ * `__config_path`: full path of the borgmatic configuration file
+ * `__hostname`: the local machine hostname
+
+These placeholders are only substituted for the whole label value, not
+interpolated into a larger string. For instance:
+
+```yaml
+loki:
+ url: http://localhost:3100/loki/api/v1/push
+
+ labels:
+ app: borgmatic
+ config: __config
+ hostname: __hostname
+```
+
+Also check out this [Loki dashboard for
+borgmatic](https://grafana.com/grafana/dashboards/20736-borgmatic-logs/) if
+you'd like to see your backup logs and statistics in one place.
diff --git a/docs/reference/configuration/monitoring/ntfy.md b/docs/reference/configuration/monitoring/ntfy.md
new file mode 100644
index 00000000..b114b5b4
--- /dev/null
+++ b/docs/reference/configuration/monitoring/ntfy.md
@@ -0,0 +1,64 @@
+---
+title: ntfy
+eleventyNavigation:
+ key: • ntfy
+ parent: 🚨 Monitoring
+---
+New in version 1.6.3
+[ntfy](https://ntfy.sh) is a free, simple, service (either cloud-hosted or
+self-hosted) which offers simple pub/sub push notifications to multiple
+platforms including [web](https://ntfy.sh/stats),
+[Android](https://play.google.com/store/apps/details?id=io.heckel.ntfy) and
+[iOS](https://apps.apple.com/us/app/ntfy/id1625396347).
+
+Since push notifications for regular events might soon become quite annoying,
+this hook only fires on any errors by default in order to instantly alert you
+to issues. The `states` list can override this. Each state can have its own
+custom messages, priorities and tags or, if none are provided, will use the
+default.
+
+An example configuration is shown here with all the available options,
+including [priorities](https://ntfy.sh/docs/publish/#message-priority) and
+[tags](https://ntfy.sh/docs/publish/#tags-emojis):
+
+```yaml
+ntfy:
+ topic: my-unique-topic
+ server: https://ntfy.my-domain.com
+ username: myuser
+ password: secret
+
+ start:
+ title: A borgmatic backup started
+ message: Watch this space...
+ tags: borgmatic
+ priority: min
+ finish:
+ title: A borgmatic backup completed successfully
+ message: Nice!
+ tags: borgmatic,+1
+ priority: min
+ fail:
+ title: A borgmatic backup failed
+ message: You should probably fix it
+ tags: borgmatic,-1,skull
+ priority: max
+ states:
+ - start
+ - finish
+ - fail
+```
+
+Prior to version 1.8.0 Put
+the `ntfy:` option in the `hooks:` section of your configuration.
+
+New in version 1.8.9 Instead of
+`username`/`password`, you can specify an [ntfy access
+token](https://docs.ntfy.sh/config/#access-tokens):
+
+```yaml
+ntfy:
+ topic: my-unique-topic
+ server: https://ntfy.my-domain.com
+ access_token: tk_AgQdq7mVBoFD37zQVN29RhuMzNIz2
+````
diff --git a/docs/reference/configuration/monitoring/pagerduty.md b/docs/reference/configuration/monitoring/pagerduty.md
new file mode 100644
index 00000000..01579120
--- /dev/null
+++ b/docs/reference/configuration/monitoring/pagerduty.md
@@ -0,0 +1,69 @@
+---
+title: PagerDuty
+eleventyNavigation:
+ key: • PagerDuty
+ parent: 🚨 Monitoring
+---
+In case you're new here: [borgmatic](https://torsion.org/borgmatic/) is
+simple, configuration-driven backup software for servers and workstations,
+powered by [Borg Backup](https://www.borgbackup.org/).
+
+[PagerDuty](https://www.pagerduty.com/) provides incident monitoring and
+alerting. borgmatic has built-in integration that can notify you via PagerDuty
+as soon as a backup fails, so you can make sure your backups keep working.
+
+First, create a PagerDuty account and service
+on their site. On the service, add an integration and set the Integration Type
+to "borgmatic".
+
+Then, configure borgmatic with the unique "Integration Key" for your service.
+Here's an example:
+
+
+```yaml
+pagerduty:
+ integration_key: a177cad45bd374409f78906a810a3074
+```
+
+Prior to version 1.8.0 Put
+this option in the `hooks:` section of your configuration.
+
+With this configuration, borgmatic creates a PagerDuty event for your service
+whenever backups fail, but only when any of the `create`, `prune`, `compact`,
+or `check` actions are run. Note that borgmatic does not contact PagerDuty
+when a backup starts or when it ends without error.
+
+You can configure PagerDuty to notify you by a [variety of
+mechanisms](https://support.pagerduty.com/docs/notifications) when backups
+fail.
+
+If you have any issues with the integration, [please contact
+us](https://torsion.org/borgmatic/#support-and-contributing).
+
+
+### Sending logs
+
+New in version 1.9.14 borgmatic
+logs are included in the payload data sent to PagerDuty. This means that
+(truncated) borgmatic logs, including error messages, show up in the PagerDuty
+incident UI and corresponding notification emails.
+
+You can customize the verbosity of the logs that are sent with borgmatic's
+`--monitoring-verbosity` flag. The `--list` and `--stats` flags may also be of
+use. See `borgmatic create --help` for more information.
+
+New in version 2.0.0Set the
+defaults for these flags in your borgmatic configuration via the
+`monitoring_verbosity`, `list`, and `statistics` options.
+
+If you don't want any logs sent, you can disable log sending by setting
+`send_logs` to `false`:
+
+```yaml
+pagerduty:
+ integration_key: a177cad45bd374409f78906a810a3074
+ send_logs: false
+```
+
+
diff --git a/docs/reference/configuration/monitoring/pushover.md b/docs/reference/configuration/monitoring/pushover.md
new file mode 100644
index 00000000..bcb1b2c9
--- /dev/null
+++ b/docs/reference/configuration/monitoring/pushover.md
@@ -0,0 +1,54 @@
+---
+title: Pushover
+eleventyNavigation:
+ key: • Pushover
+ parent: 🚨 Monitoring
+---
+New in version 1.9.2
+[Pushover](https://pushover.net) makes it easy to get real-time notifications
+on your Android, iPhone, iPad, and Desktop (Android Wear and Apple Watch,
+too!).
+
+First, create a Pushover account and login on your mobile device. Create an
+Application in your Pushover dashboard.
+
+Then, configure borgmatic with your user's unique "User Key" found in your
+Pushover dashboard and the unique "API Token" from the created Application.
+
+Here's a basic example:
+
+
+```yaml
+pushover:
+ token: 7ms6TXHpTokTou2P6x4SodDeentHRa
+ user: hwRwoWsXMBWwgrSecfa9EfPey55WSN
+```
+
+
+With this configuration, borgmatic creates a Pushover event for your service
+whenever borgmatic fails, but only when any of the `create`, `prune`, `compact`,
+or `check` actions are run. Note that borgmatic does not contact Pushover
+when a backup starts or when it ends without error by default.
+
+You can configure Pushover to have custom parameters declared for borgmatic's
+`start`, `fail` and `finish` hooks states.
+
+Here's a more advanced example:
+
+
+```yaml
+pushover:
+ token: 7ms6TXHpTokTou2P6x4SodDeentHRa
+ user: hwRwoWsXMBWwgrSecfa9EfPey55WSN
+ start:
+ message: "Backup Started"
+ priority: -2
+ title: "Backup Started"
+ html: True
+ ttl: 10 # Message will be deleted after 10 seconds.
+ fail:
+ message: "Backup Failed"
+ priority: 2 # Requests acknowledgement for messages.
+ expire: 600 # Used only for priority 2. Default is 600 seconds.
+ retry: 30 # Used only for priority 2. Default is 30 seconds.
+ device: "pixel8"
diff --git a/docs/reference/configuration/monitoring/sentry.md b/docs/reference/configuration/monitoring/sentry.md
new file mode 100644
index 00000000..ca9e5d42
--- /dev/null
+++ b/docs/reference/configuration/monitoring/sentry.md
@@ -0,0 +1,41 @@
+---
+title: Sentry
+eleventyNavigation:
+ key: • Sentry
+ parent: 🚨 Monitoring
+---
+New in version 1.9.7
+[Sentry](https://sentry.io/) is an application monitoring service that
+includes cron-style monitoring (either cloud-hosted or
+[self-hosted](https://develop.sentry.dev/self-hosted/)).
+
+To get started, create a [Sentry cron
+monitor](https://docs.sentry.io/product/crons/) in the Sentry UI. Under
+"Instrument your monitor," select "Sentry CLI" and copy the URL value for the
+displayed
+[`SENTRY_DSN`](https://docs.sentry.io/concepts/key-terms/dsn-explainer/)
+environment variable into borgmatic's Sentry `data_source_name_url`
+configuration option. For example:
+
+```
+sentry:
+ data_source_name_url: https://5f80ec@o294220.ingest.us.sentry.io/203069
+ monitor_slug: mymonitor
+```
+
+The `monitor_slug` value comes from the "Monitor Slug" under "Cron Details" on
+the same Sentry monitor page.
+
+With this configuration, borgmatic pings Sentry whenever borgmatic starts,
+finishes, or fails, but only when any of the `create`, `prune`, `compact`, or
+`check` actions are run. You can optionally override the start/finish/fail
+behavior with the `states` configuration option. For instance, to only ping
+Sentry on failure:
+
+```
+sentry:
+ data_source_name_url: https://5f80ec@o294220.ingest.us.sentry.io/203069
+ monitor_slug: mymonitor
+ states:
+ - fail
+```
diff --git a/docs/reference/configuration/monitoring/uptime-kuma.md b/docs/reference/configuration/monitoring/uptime-kuma.md
new file mode 100644
index 00000000..e40f3872
--- /dev/null
+++ b/docs/reference/configuration/monitoring/uptime-kuma.md
@@ -0,0 +1,59 @@
+---
+title: Uptime Kuma
+eleventyNavigation:
+ key: • Uptime Kuma
+ parent: 🚨 Monitoring
+---
+New in version 1.8.13 [Uptime
+Kuma](https://uptime.kuma.pet) is a self-hosted monitoring tool and can
+provide a Push monitor type to accept HTTP `GET` requests from a service
+instead of contacting it directly.
+
+Uptime Kuma allows you to see a history of monitor states and can in turn
+alert via ntfy, Gotify, Matrix, Apprise, Email, and many more.
+
+An example configuration is shown here with all the available options:
+
+```yaml
+uptime_kuma:
+ push_url: https://kuma.my-domain.com/api/push/abcd1234
+ states:
+ - start
+ - finish
+ - fail
+```
+
+The `push_url` is provided to your from your Uptime Kuma service and
+originally includes a query string—the text including and after the question
+mark (`?`). But please do not include the query string in the `push_url`
+configuration; borgmatic will add this automatically depending on the state of
+your backup.
+
+Using `start`, `finish` and `fail` states means you will get two "up beats" in
+Uptime Kuma for successful backups and the ability to see failures if and when
+the backup started (was there a `start` beat?).
+
+A reasonable base-level configuration for an Uptime Kuma Monitor for a backup
+is below:
+
+```ini
+# These are to be entered into Uptime Kuma and not into your borgmatic
+# configuration.
+
+# Push monitors wait for the client to contact Uptime Kuma instead of Uptime
+# Kuma contacting the client. This is perfect for backup monitoring.
+Monitor Type = Push
+
+Heartbeat Interval = 90000 # = 25 hours = 1 day + 1 hour
+
+# Wait 6 times the Heartbeat Retry (below) before logging a heartbeat missed.
+Retries = 6
+
+# Multiplied by Retries this gives a grace period within which the monitor
+# goes into the "Pending" state.
+Heartbeat Retry = 360 # = 10 minutes
+
+# For each Heartbeat Interval if the backup fails repeatedly, a notification
+# is sent each time.
+Resend Notification every X times = 1
+```
diff --git a/docs/reference/configuration/monitoring/zabbix.md b/docs/reference/configuration/monitoring/zabbix.md
new file mode 100644
index 00000000..46606298
--- /dev/null
+++ b/docs/reference/configuration/monitoring/zabbix.md
@@ -0,0 +1,71 @@
+---
+title: Zabbix
+eleventyNavigation:
+ key: • Zabbix
+ parent: 🚨 Monitoring
+---
+New in version 1.9.0
+[Zabbix](https://www.zabbix.com/) is an open-source monitoring tool used for
+tracking and managing the performance and availability of networks, servers,
+and applications in real-time.
+
+This hook does not do any notifications on its own. Instead, it relies on your
+Zabbix instance to notify and perform escalations based on the Zabbix
+configuration. The `states` defined in the configuration determine which
+states will trigger the hook. The value defined in the configuration of each
+state is used to populate the data of the configured Zabbix item. If none are
+provided, it defaults to a lower-case string of the state.
+
+An example configuration is shown here with all the available options.
+
+```yaml
+zabbix:
+ server: http://cloud.zabbix.com/zabbix/api_jsonrpc.php
+
+ username: myuser
+ password: secret
+ api_key: b2ecba64d8beb47fc161ae48b164cfd7104a79e8e48e6074ef5b141d8a0aeeca
+
+ host: "borg-server"
+ key: borg.status
+ itemid: 55105
+
+ start:
+ value: "STARTED"
+ finish:
+ value: "OK"
+ fail:
+ value: "ERROR"
+ states:
+ - start
+ - finish
+ - fail
+```
+
+This hook requires the Zabbix server be running version 7.0.
+
+New in version 1.9.3 Zabbix 7.2+
+is supported as well.
+
+
+### Authentication methods
+
+Authentication can be accomplished via `api_key` or both `username` and
+`password`. If all three are declared, only `api_key` is used.
+
+
+### Items
+
+borgmatic writes its monitoring updates to a particular Zabbix item, which
+you'll need to create in advance. In the Zabbix web UI, [make a new item with a
+Type of "Zabbix
+trapper"](https://www.zabbix.com/documentation/current/en/manual/config/items/itemtypes/trapper)
+and a named Key. The "Type of information" for the item should be "Text", and
+"History" designates how much data you want to retain.
+
+When configuring borgmatic with this item to be updated, you can either declare
+the `itemid` or both `host` and `key`. If all three are declared, only `itemid`
+is used.
+
+Keep in mind that `host` refers to the "Host name" on the Zabbix server and not
+the "Visual name".
diff --git a/docs/reference/source-code.md b/docs/reference/source-code.md
index e15482e6..b4312e59 100644
--- a/docs/reference/source-code.md
+++ b/docs/reference/source-code.md
@@ -1,16 +1,14 @@
---
-title: Source code reference
+title: Source code
eleventyNavigation:
- key: 🐍 Source code reference
+ key: 🐍 Source code
parent: Reference guides
order: 3
---
-## getting oriented
-
-If case you're interested in [developing on
-borgmatic](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic/),
-here's an abridged primer on how its Python source code is organized to help
-you get started. Starting at the top level, we have:
+If you're interested in [developing on
+borgmatic](https://torsion.org/borgmatic/how-to/develop-on-borgmatic/), here's
+an abridged primer on how its Python source code is organized to help you get
+started. Starting at the top level, we have:
* [borgmatic](https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/main/borgmatic): The main borgmatic source module. Most of the code is here. Within that:
* [actions](https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/main/borgmatic/actions): borgmatic-specific logic for running each action (create, list, check, etc.).
@@ -26,4 +24,5 @@ you get started. Starting at the top level, we have:
* [scripts](https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/main/scripts): Dev-facing scripts for things like building documentation and running end-to-end tests.
* [tests](https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/main/tests): Automated tests organized by: end-to-end, integration, and unit.
-So, broadly speaking, the control flow goes: `commands` → `config` followed by `commands` → `actions` → `borg` and `hooks`.
+So, broadly speaking, the control flow goes: `commands` → `config` followed by
+`commands` → `actions` → `borg` and `hooks`.
diff --git a/scripts/run-end-to-end-tests b/scripts/run-end-to-end-tests
index 41d9726c..f9167a90 100755
--- a/scripts/run-end-to-end-tests
+++ b/scripts/run-end-to-end-tests
@@ -6,7 +6,7 @@
# Run this script from the root directory of the borgmatic source.
#
# For more information, see:
-# https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic/
+# https://torsion.org/borgmatic/how-to/develop-on-borgmatic/
set -e
diff --git a/scripts/run-full-tests b/scripts/run-full-tests
index 913e55df..7bf172a7 100755
--- a/scripts/run-full-tests
+++ b/scripts/run-full-tests
@@ -6,7 +6,7 @@
# directly. Instead, run scripts/run-end-to-end-tests
#
# For more information, see:
-# https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic/
+# https://torsion.org/borgmatic/how-to/develop-on-borgmatic/
set -e
+
borgmatic is simple, configuration-driven backup software for servers and
workstations. Protect your files with client-side encryption. Backup your
@@ -60,45 +60,45 @@ borgmatic is powered by [Borg Backup](https://www.borgbackup.org/).
### Data
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
### Monitoring
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
### Credentials
-
-
-
-
+
+
+
+
## Getting started
Your first step is to [install and configure
-borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/).
+borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/).
For additional documentation, check out the links above (left panel on wide screens)
for borgmatic how-to and
@@ -132,7 +132,7 @@ first. If you prefer to use an existing GitHub account, you can skip account
creation and [login directly](https://projects.torsion.org/user/login).
Also see the [security
-policy](https://torsion.org/borgmatic/docs/security-policy/) for any security
+policy](https://torsion.org/borgmatic/security-policy/) for any security
issues.
@@ -177,7 +177,7 @@ discuss your idea. Note that you'll need to
first. In general, contributions are very welcome. We don't bite!
Also, please check out the [borgmatic development
-how-to](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic/) for
+how-to](https://torsion.org/borgmatic/how-to/develop-on-borgmatic/) for
info on cloning source code, running tests, etc.
### Recent contributors
diff --git a/borgmatic/config/schema.yaml b/borgmatic/config/schema.yaml
index 933fa48c..4c2ee89c 100644
--- a/borgmatic/config/schema.yaml
+++ b/borgmatic/config/schema.yaml
@@ -275,7 +275,7 @@ properties:
encryption_passphrase takes precedence. This can also be used to
access encrypted systemd service credentials. Defaults to not set.
For more details, see:
- https://torsion.org/borgmatic/docs/how-to/provide-your-passwords/
+ https://torsion.org/borgmatic/how-to/provide-your-passwords/
example: "secret-tool lookup borg-repository repo-name"
encryption_passphrase:
type: string
diff --git a/docs/Dockerfile b/docs/Dockerfile
index cfa1b828..4379abb4 100644
--- a/docs/Dockerfile
+++ b/docs/Dockerfile
@@ -27,10 +27,9 @@ COPY --from=borgmatic /etc/borgmatic/config.yaml /source/docs/_includes/borgmati
COPY --from=borgmatic /command-line.txt /source/docs/_includes/borgmatic/command-line.txt
COPY --from=borgmatic /contributors.html /source/docs/_includes/borgmatic/contributors.html
COPY . /source
-RUN NODE_ENV=${ENVIRONMENT} npx eleventy --input=/source/docs --output=/output/docs \
- && mv /output/docs/index.html /output/index.html
+RUN NODE_ENV=${ENVIRONMENT} npx eleventy --input=/source/docs --output=/output
FROM docker.io/nginx:1.26.1-alpine
COPY --from=html /output /usr/share/nginx/html
-COPY --from=borgmatic /etc/borgmatic/config.yaml /usr/share/nginx/html/docs/reference/config.yaml
+COPY --from=borgmatic /etc/borgmatic/config.yaml /usr/share/nginx/html/reference/config.yaml
diff --git a/docs/_includes/asciinema.css b/docs/_includes/asciinema.css
deleted file mode 100644
index b2ddcffd..00000000
--- a/docs/_includes/asciinema.css
+++ /dev/null
@@ -1,3 +0,0 @@
-.asciicast > iframe {
- width: 100% !important;
-}
diff --git a/docs/_includes/components/toc.css b/docs/_includes/components/toc.css
index 4967c301..345b8f8a 100644
--- a/docs/_includes/components/toc.css
+++ b/docs/_includes/components/toc.css
@@ -48,9 +48,10 @@
padding-bottom: 0;
padding-left: 0.625rem; /* 10px /16 */
}
-/* Hide inactive menus 3 or more deep */
-.elv-toc-list ul ul > li:not(.elv-toc-active) > ul > li:not(.elv-toc-active) {
- display: none;
+
+/* Display inline menus 4 or more deep */
+.elv-toc-list ul ul ul > li {
+ display: inline-block;
}
/* List items */
@@ -98,3 +99,29 @@
.elv-cat-list-active {
font-weight: 600;
}
+
+.breadcrumb-item.active {
+ color: var(--primary-color);
+}
+
+#breadcrumb ol,
+#breadcrumb ul {
+ margin: 0;
+ padding: 0 0 1em;
+}
+
+#breadcrumb li {
+ display: inline;
+}
+
+#breadcrumb li:not(:last-child)::after {
+ content: " ˃"
+}
+
+#breadcrumb a:not(:hover) {
+ text-decoration: none;
+}
+
+#breadcrumb a {
+ text-decoration-color: #00bcd4;
+}
diff --git a/docs/_includes/header.njk b/docs/_includes/header.njk
index f511e012..c227f241 100644
--- a/docs/_includes/header.njk
+++ b/docs/_includes/header.njk
@@ -1,4 +1,8 @@