Support for Borg --chunker-params create option via "chunker_params" in borgmatic's storage section (#105 ).

Silence curl when posting release description to Gitea.
More concessions for Python 3.5 compatibility.
2026-07-22 10:13:00 +02:00 · 2018-10-27 15:57:28 -07:00 · 2018-10-15 22:32:13 -07:00 · 2018-10-15 09:37:26 -07:00 · 2018-10-15 09:30:04 -07:00 · 2018-10-15 09:25:57 -07:00
72 changed files with 3114 additions and 1436 deletions
@@ -0,0 +1,18 @@
+pipeline:
+  build:
+    image: python:${PYTHON_VERSION}-alpine${ALPINE_VERSION}
+    pull: true
+    commands:
+      - pip install tox
+      - tox
+      - apk add --no-cache borgbackup
+      - tox -e end-to-end
+
+matrix:
+  ALPINE_VERSION:
+    - 3.7
+    - 3.8
+  PYTHON_VERSION:
+    - 3.5
+    - 3.6
+    - 3.7
@@ -3,6 +3,7 @@
 *.swp
 .cache
 .coverage
+.pytest_cache
 .tox
 build
 dist
@@ -1,9 +1,12 @@
 Dan Helfman <witten@torsion.org>: Main developer

 Alexander Görtz: Python 3 compatibility
+Florian Lindner: Logging rewrite
 Henning Schroeder: Copy editing
 Johannes Feichtner: Support for user hooks
 Michele Lazzeri: Custom archive names
+Nick Whyte: Support prefix filtering for archive consistency checks
+newtonne: Read encryption password from external file
 Robin `ypid` Schneider: Support additional options of Borg
 Scott Squires: Custom archive names
-Thomas LÉVEIL: Support for a keep_minutely prune option
+Thomas LÉVEIL: Support for a keep_minutely prune option. Support for the --json option
@@ -1,24 +1,122 @@
-1.1.12.dev0
- * #45: Declare dependency on pykwalify 1.6 or above, as older versions yield "Unknown key: version"
+1.2.10
+ * #105: Support for Borg --chunker-params create option via "chunker_params" in borgmatic's storage
+   section.
+
+1.2.9
+ * #102: Fix for syntax error that occurred in Python 3.5 and below.
+ * Make automated tests support running in Python 3.5.
+
+1.2.8
+ * #73: Enable consistency checks for only certain repositories via "check_repositories" option in
+   borgmatic's consistency configuration. Handy for large repositories that take forever to check.
+ * Include link to issue tracker within various command output.
+ * Run continuous integration tests on a matrix of Python and Borg versions.
+
+1.2.7
+ * #98: Support for Borg --keep-secondly prune option.
+ * Use Black code formatter and Flake8 code checker as part of running automated tests.
+ * Add an end-to-end automated test that actually integrates with Borg.
+ * Set up continuous integration for borgmatic automated tests on projects.evoworx.org.
+
+1.2.6
+ * Fix generated configuration to also include a "keep_daily" value so pruning works out of the
+   box.
+
+1.2.5
+ * #57: When generating sample configuration with generate-borgmatic-config, comment out all
+   optional configuration so as to streamline the initial configuration process.
+
+1.2.4
+ * Fix for archive checking traceback due to parameter mismatch.
+
+1.2.3
+ * #64, #90, #92: Rewrite of logging system. Now verbosity flags passed to Borg are derived from
+   borgmatic's log level. Note that the output of borgmatic might slightly change.
+ * Part of #80: Support for Borg create --read-special via "read_special" option in borgmatic's
+   location configuration.
+ * #87: Support for Borg create --checkpoint-interval via "checkpoint_interval" option in
+   borgmatic's storage configuration.
+ * #88: Fix declared pykwalify compatibility version range in setup.py to prevent use of ancient
+   versions of pykwalify with large version numbers.
+ * #89: Pass --show-rc option to Borg when at highest verbosity level.
+ * #94: Support for Borg --json option via borgmatic command-line to --create archives.
+
+1.2.2
+ * #85: Fix compatibility issue between pykwalify and ruamel.yaml 0.15.52, which manifested in
+   borgmatic as a pykwalify RuleError.
+
+1.2.1
+ * Skip before/after backup hooks when only doing --prune, --check, --list, and/or --info.
+ * #71: Support for XDG_CONFIG_HOME environment variable for specifying alternate user ~/.config/
+   path.
+ * #74, #83: Support for Borg --json option via borgmatic command-line to --list archives or show
+   archive --info in JSON format, ideal for programmatic consumption.
+ * #38, #76: Upgrade ruamel.yaml compatibility version range and fix support for Python 3.7.
+ * #77: Skip non-"*.yaml" config filenames in /etc/borgmatic.d/ so as not to parse backup files,
+   editor swap files, etc.
+ * #81: Document user-defined hooks run before/after backup, or on error.
+ * Add code style guidelines to the documention.
+
+1.2.0
+ * #61: Support for Borg --list option via borgmatic command-line to list all archives.
+ * #61: Support for Borg --info option via borgmatic command-line to display summary information.
+ * #62: Update README to mention other ways of installing borgmatic.
+ * Support for Borg --prefix option for consistency checks via "prefix" option in borgmatic's
+   consistency configuration.
+ * Add introductory screencast link to documentation.
+ * #59: Ignore "check_last" and consistency "prefix" when "archives" not in consistency checks.
+ * #60: Add "Persistent" flag to systemd timer example.
+ * #63: Support for Borg --nobsdflags option to skip recording bsdflags (e.g. NODUMP, IMMUTABLE) in
+   archive.
+ * #69: Support for Borg prune --umask option using value of existing "umask" option in borgmatic's
+   storage configuration.
+ * Update tox.ini to only assume Python 3.x instead of Python 3.4 specifically.
+ * Add ~/.config/borgmatic/config.yaml to default configuration path probing.
+ * Document how to develop on and contribute to borgmatic.
+
+1.1.15
+ * Support for Borg BORG_PASSCOMMAND environment variable to read a password from an external file.
+ * Fix for Borg create error when using borgmatic's --dry-run and --verbosity options together.
+   Work-around for behavior introduced in Borg 1.1.3: https://github.com/borgbackup/borg/issues/3298
+ * #55: Fix for missing tags/releases on Gitea and GitHub project hosting.
+ * #56: Support for Borg --lock-wait option for the maximum wait for a repository/cache lock.
+ * #58: Support for using tilde in exclude_patterns to reference home directory.
+
+1.1.14
+ * #49: Fix for typo in --patterns-from option.
+ * #47: Support for Borg --dry-run option via borgmatic command-line.
+
+1.1.13
+ * #54: Fix for incorrect consistency check flags passed to Borg when all three checks ("repository",
+   "archives", and "extract") are specified in borgmatic configuration.
+ * #48: Add "local_path" to configuration for specifying an alternative Borg executable path.
+ * #49: Support for Borg experimental --patterns-from and --patterns options for specifying mixed
+   includes/excludes.
+ * Moved issue tracker from Taiga to integrated Gitea tracker at
+   https://projects.torsion.org/witten/borgmatic/issues
+
+1.1.12
+ * #46: Declare dependency on pykwalify 1.6 or above, as older versions yield "Unknown key: version"
   rule errors.
+ * Support for Borg --keep-minutely prune option.

 1.1.11
- * #25: Add "ssh_command" to configuration for specifying a custom SSH command or options.
+ * #26: Add "ssh_command" to configuration for specifying a custom SSH command or options.
 * Fix for incorrect /etc/borgmatic.d/ configuration path probing on macOS. This problem manifested
   as an error on startup: "[Errno 2] No such file or directory: '/etc/borgmatic.d'".

 1.1.10
 * Pass several Unix signals through to child processes like Borg. This means that Borg now properly
   shuts down if borgmatic is terminated (e.g. due to a system suspend).
- * #29: Support for using tilde in repository paths to reference home directory.
- * #42: Support for Borg --files-cache option for setting the files cache operation mode.
- * #44: Support for Borg --remote-ratelimit for limiting upload rate.
+ * #30: Support for using tilde in repository paths to reference home directory.
+ * #43: Support for Borg --files-cache option for setting the files cache operation mode.
+ * #45: Support for Borg --remote-ratelimit option for limiting upload rate.
 * Log invoked Borg commands when at highest verbosity level.

 1.1.9
- * #16, #38: Support for user-defined hooks before/after backup, or on error.
- * #33: Improve clarity of logging spew at high verbosity levels.
- * #29: Support for using tilde in source directory path to reference home directory.
+ * #17, #39: Support for user-defined hooks before/after backup, or on error.
+ * #34: Improve clarity of logging spew at high verbosity levels.
+ * #30: Support for using tilde in source directory path to reference home directory.
 * Require "prefix" in retention section when "archive_name_format" is set. This is to avoid
   accidental pruning of archives with a different archive name format. For similar reasons, default
   "prefix" to "{hostname}-" if not specified.
@@ -26,43 +124,43 @@
 * Update dead links to Borg documentation.

 1.1.8
- * #39: Fix to make /etc/borgmatic/config.yaml optional rather than required when using the default
+ * #40: Fix to make /etc/borgmatic/config.yaml optional rather than required when using the default
   config paths.

 1.1.7

- * #28: Add "archive_name_format" to configuration for customizing archive names.
+ * #29: Add "archive_name_format" to configuration for customizing archive names.
 * Fix for traceback when "exclude_from" value is empty in configuration file.
 * When pruning, make highest verbosity level list archives kept and pruned.
 * Clarification of Python 3 pip usage in documentation.

 1.1.6

- * #12, #35: Support for Borg --exclude-from, --exclude-caches, and --exclude-if-present options.
+ * #13, #36: Support for Borg --exclude-from, --exclude-caches, and --exclude-if-present options.

 1.1.5

- * #34: New "extract" consistency check that performs a dry-run extraction of the most recent
+ * #35: New "extract" consistency check that performs a dry-run extraction of the most recent
   archive.

 1.1.4

- * #17: Added command-line flags for performing a borgmatic run with only pruning, creating, or
+ * #18: Added command-line flags for performing a borgmatic run with only pruning, creating, or
   checking enabled. This supports use cases like running consistency checks from a different cron
   job with a different frequency, or running pruning with a different verbosity level.

 1.1.3

- * #14: Support for running multiple config files in /etc/borgmatic.d/ from a single borgmatic run.
+ * #15: Support for running multiple config files in /etc/borgmatic.d/ from a single borgmatic run.
 * Fix for generate-borgmatic-config writing config with invalid one_file_system value.

 1.1.2

- * #32: Fix for passing check_last as integer to subprocess when calling Borg.
+ * #33: Fix for passing check_last as integer to subprocess when calling Borg.

 1.1.1

- * Part of #32: Fix for upgrade-borgmatic-config converting check_last option as a string instead of
+ * Part of #33: Fix for upgrade-borgmatic-config converting check_last option as a string instead of
   an integer.
 * Fix for upgrade-borgmatic-config erroring when consistency checks option is not present.

@@ -71,8 +169,8 @@
 * Switched config file format to YAML. Run upgrade-borgmatic-config to upgrade.
 * Added generate-borgmatic-config command for initial config creation.
 * Dropped Python 2 support. Now Python 3 only.
- * #18: Fix for README mention of sample files not included in package.
- * #22: Sample files for triggering borgmatic from a systemd timer.
+ * #19: Fix for README mention of sample files not included in package.
+ * #23: Sample files for triggering borgmatic from a systemd timer.
 * Support for backing up to multiple repositories.
 * To free up space, now pruning backups prior to creating a new backup.
 * Enabled test coverage output during tox runs.
@@ -80,15 +178,15 @@

 1.0.3

- * #21: Fix for verbosity flag not actually causing verbose output.
+ * #22: Fix for verbosity flag not actually causing verbose output.

 1.0.2

- * #20: Fix for traceback when remote_path option is missing.
+ * #21: Fix for traceback when remote_path option is missing.

 1.0.1

- * #19: Support for Borg's --remote-path option to use an alternate Borg
+ * #20: Support for Borg's --remote-path option to use an alternate Borg
   executable. See sample/config.

 1.0.0
@@ -110,13 +208,13 @@

 0.1.7

- * #11: Fixed parsing of punctuation in configuration file.
+ * #12: Fixed parsing of punctuation in configuration file.
 * Better error message when configuration file is missing.

 0.1.6

- * #9: New configuration option for the encryption passphrase.
- * #10: Support for Borg's new archive compression feature.
+ * #10: New configuration option for the encryption passphrase.
+ * #11: Support for Borg's new archive compression feature.

 0.1.5

@@ -128,7 +226,7 @@

 0.1.3

- * #1: Add support for "borg check --last N" to Borg backend.
+ * #2: Add support for "borg check --last N" to Borg backend.

 0.1.2

@@ -1,13 +1,17 @@
-<img src="https://projects.torsion.org/witten/borgmatic/raw/branch/master/static/borgmatic.png" width="150px" style="float: right; padding-left: 1em;">
-
+---
+title: borgmatic
+permalink: borgmatic/index.html
+---
 ## Overview

-borgmatic (formerly atticmatic) is a simple Python wrapper script for the
-[Borg](https://borgbackup.readthedocs.org/en/stable/) backup software that
-initiates a backup, prunes any old backups according to a retention policy,
-and validates backups for consistency. The script supports specifying your
-settings in a declarative configuration file rather than having to put them
-all on the command-line, and handles common errors.
+<img src="https://projects.torsion.org/witten/borgmatic/raw/branch/master/static/borgmatic.png" width="150px" style="float: right; padding-left: 1em;">
+
+borgmatic is a simple Python wrapper script for the
+[Borg](https://www.borgbackup.org/) backup software that initiates a backup,
+prunes any old backups according to a retention policy, and validates backups
+for consistency. The script supports specifying your settings in a declarative
+configuration file rather than having to put them all on the command-line, and
+handles common errors.

 Here's an example config file:

@@ -44,19 +48,34 @@ borgmatic is hosted at <https://torsion.org/borgmatic> with [source code
 available](https://projects.torsion.org/witten/borgmatic). It's also mirrored
 on [GitHub](https://github.com/witten/borgmatic) for convenience.

+Want to see borgmatic in action? Check out the <a
+href="https://asciinema.org/a/203761" target="_blank">screencast</a>.
+
+<script src="https://asciinema.org/a/203761.js" id="asciicast-203761" async></script>
+

 ## Installation

-To get up and running, follow the [Borg Quick
+To get up and running, first [install
+Borg](https://borgbackup.readthedocs.io/en/latest/installation.html), at
+least version 1.1. Then, follow the [Borg Quick
 Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) to create
-a repository on a local or remote host. Note that if you plan to run borgmatic
-on a schedule with cron, and you encrypt your Borg repository with a
-passphrase instead of a key file, you'll either need to set the borgmatic
-`encryption_passphrase` configuration variable or set the `BORG_PASSPHRASE`
-environment variable. See the [repository encryption
+a repository on a local or remote host.
+
+Note that if you plan to run borgmatic on a schedule with cron, and you
+encrypt your Borg repository with a passphrase instead of a key file, you'll
+either need to set the borgmatic `encryption_passphrase` configuration
+variable or set the `BORG_PASSPHRASE` environment variable. See the
+[repository encryption
 section](https://borgbackup.readthedocs.io/en/latest/quickstart.html#repository-encryption)
 of the Quick Start for more info.

+Alternatively, the passphrase can be specified programatically by setting
+either the borgmatic `encryption_passcommand` configuration variable or the
+`BORG_PASSCOMMAND` environment variable. See the [Borg Security
+FAQ](http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically)
+for more info.
+
 If the repository is on a remote host, make sure that your local root user has
 key-based ssh access to the desired user account on the remote host.

@@ -69,11 +88,18 @@ sudo pip3 install --upgrade borgmatic
 Note that your pip binary may have a different name than "pip3". Make sure
 you're using Python 3, as borgmatic does not support Python 2.

-### Docker
+### Other ways to install

-If you would like to run borgmatic within Docker, please take a look at
-[b3vis/borgmatic](https://hub.docker.com/r/b3vis/borgmatic/) for more
-information.
+ * [A borgmatic Docker image](https://hub.docker.com/r/b3vis/borgmatic/) based
+   on Alpine Linux.
+ * [Another borgmatic Docker image](https://hub.docker.com/r/coaxial/borgmatic/) based
+   on Alpine Linux, updated automatically whenever the Alpine image updates.
+ * [A borgmatic package for
+   Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic).
+ * [A borgmatic package for Arch
+   Linux](https://aur.archlinux.org/packages/borgmatic/).
+ * [A borgmatic package for OpenBSD](http://ports.su/sysutils/borgmatic).
+<br><br>

 ## Configuration

@@ -83,10 +109,13 @@ After you install borgmatic, generate a sample configuration file:
 sudo generate-borgmatic-config
 ```

+If that command is not found, then it may be installed in a location that's
+not in your system `PATH`. Try looking in `/usr/local/bin/`.
+
 This generates a sample configuration file at /etc/borgmatic/config.yaml (by
 default). You should edit the file to suit your needs, as the values are just
 representative. All fields are optional except where indicated, so feel free
-to remove anything you don't need.
+to ignore anything you don't need.

 You can also have a look at the [full configuration
 schema](https://projects.torsion.org/witten/borgmatic/src/master/borgmatic/config/schema.yaml)
@@ -120,6 +149,31 @@ configuration paths on the command-line with borgmatic's `--config` option.
 See `borgmatic --help` for more information.


+### Hooks
+
+If you find yourself performing prepraration tasks before your backup runs, or
+cleanup work afterwards, borgmatic hooks may be of interest. They're simply
+shell commands that borgmatic executes for you at various points, and they're
+configured in the `hooks` section of your configuration file.
+
+For instance, you can specify `before_backup` hooks to dump a database to file
+before backing it up, and specify `after_backup` hooks to delete the temporary
+file afterwards.
+
+borgmatic hooks run once per configuration file. `before_backup` hooks run
+prior to backups of all repositories. `after_backup` hooks run afterwards, but
+not if an error occurs in a previous hook or in the backups themselves. And
+borgmatic runs `on_error` hooks if an error occurs.
+
+An important security note about hooks: borgmatic executes all hook commands
+with the user permissions of borgmatic itself. So to prevent potential shell
+injection or privilege escalation, do not forget to set secure permissions
+(`chmod 0700`) on borgmatic configuration files and scripts invoked by hooks.
+
+See the sample generated configuration file mentioned above for specifics
+about hook configuration syntax.
+
+
 ## Upgrading

 In general, all you should need to do to upgrade borgmatic is run the
@@ -248,6 +302,20 @@ any number of them. This supports use cases like running consistency checks
 from a different cron job with a different frequency, or running pruning with
 a different verbosity level.

+Additionally, borgmatic provides convenient flags for Borg's
+[list](https://borgbackup.readthedocs.io/en/stable/usage/list.html) and
+[info](https://borgbackup.readthedocs.io/en/stable/usage/info.html)
+functionality:
+
+
+```bash
+borgmatic --list
+borgmatic --info
+```
+
+You can include an optional `--json` flag with `--create`, `--list`, or
+`--info` to get the output formatted as JSON.
+

 ## Autopilot

@@ -257,7 +325,7 @@ configure a job runner to invoke it periodically.
 ### cron

 If you're using cron, download the [sample cron
-file](https://projects.torsion.org/witten/borgmatic/raw/master/sample/cron/borgmatic).
+file](https://projects.torsion.org/witten/borgmatic/src/master/sample/cron/borgmatic).
 Then, from the directory where you downloaded it:

 ```bash
@@ -286,20 +354,116 @@ Feel free to modify the timer file based on how frequently you'd like
 borgmatic to run.


-## Running tests
+## Support and contributing

-First install tox, which is used for setting up testing environments:
+### Issues
+
+You've got issues? Or an idea for a feature enhancement? We've got an [issue
+tracker](https://projects.torsion.org/witten/borgmatic/issues). In order to
+create a new issue or comment on an issue, you'll need to [login
+first](https://projects.torsion.org/user/login). Note that you can login with
+an existing GitHub account if you prefer.
+
+Other questions or comments? Contact <mailto:witten@torsion.org>.
+
+
+### Contributing
+
+If you'd like to contribute to borgmatic development, please feel free to
+submit a [Pull Request](https://projects.torsion.org/witten/borgmatic/pulls)
+or open an [issue](https://projects.torsion.org/witten/borgmatic/issues) first
+to discuss your idea. We also accept Pull Requests on GitHub, if that's more
+your thing. In general, contributions are very welcome. We don't bite! 
+
+
+### Code style
+
+Start with [PEP 8](https://www.python.org/dev/peps/pep-0008/). But then, apply
+the following deviations from it:
+
+ * For strings, prefer single quotes over double quotes.
+ * Limit all lines to a maximum of 100 characters.
+ * Use trailing commas within multiline values or argument lists.
+ * For multiline constructs, put opening and closing delimeters on lines
+   separate from their contents.
+ * Within multiline constructs, use standard four-space indentation. Don't align
+   indentation with an opening delimeter.
+
+borgmatic code uses the [Black](https://black.readthedocs.io/en/stable/) code
+formatter and [Flake8](http://flake8.pycqa.org/en/latest/) code checker, so
+certain code style requirements will be enforced when running automated tests.
+See the Black and Flake8 documentation for more information.
+
+
+### Development
+
+To get set up to hack on borgmatic, first clone master via HTTPS or SSH:

 ```bash
-pip3 install tox
+git clone https://projects.torsion.org/witten/borgmatic.git
 ```

-Then, to actually run tests, run:
+Or:

 ```bash
+git clone ssh://git@projects.torsion.org:3022/witten/borgmatic.git
+```
+
+Then, install borgmatic
+"[editable](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs)"
+so that you can easily run borgmatic commands while you're hacking on them to
+make sure your changes work.
+
+```bash
+cd borgmatic/
+pip3 install --editable --user .
+```
+
+Note that this will typically install the borgmatic commands into
+`~/.local/bin`, which may or may not be on your PATH. There are other ways to
+install borgmatic editable as well, for instance into the system Python
+install (so without `--user`, as root), or even into a
+[virtualenv](https://virtualenv.pypa.io/en/stable/). How or where you install
+borgmatic is up to you, but generally an editable install makes development
+and testing easier.
+
+
+### Running tests
+
+Assuming you've cloned the borgmatic source code as described above, and
+you're in the `borgmatic/` working copy, install tox, which is used for
+setting up testing environments:
+
+```bash
+sudo pip3 install tox
+```
+
+Finally, to actually run tests, run:
+
+```bash
+cd borgmatic
 tox
 ```

+If when running tests, you get an error from the
+[Black](https://black.readthedocs.io/en/stable/) code formatter about files
+that would be reformatted, you can ask Black to format them for you via the
+following:
+
+```bash
+tox -e black
+```
+
+### End-to-end tests
+
+borgmatic additionally includes some end-to-end tests that integration test
+with Borg for a few representative scenarios. These tests don't run by default
+because they're relatively slow and depend on Borg. If you would like to run
+them:
+
+```bash
+tox -e end-to-end
+```

 ## Troubleshooting

@@ -315,7 +479,7 @@ consistency:
 ```

 This error can be caused by an ssh timeout, which you can rectify by adding
-the following to the ~/.ssh/config file on the client:
+the following to the `~/.ssh/config` file on the client:

 ```text
    Host *
@@ -335,13 +499,3 @@ YAML library. If so, not to worry. borgmatic should install and function
 correctly even without the C YAML library. And borgmatic won't be any faster
 with the C library present, so you don't need to go out of your way to install
 it.
-
-
-## Issues and feedback
-
-Got an issue or an idea for a feature enhancement? Check out the [borgmatic
-issue tracker](https://tree.taiga.io/project/witten-borgmatic/issues?page=1&status=399951,399952,399955). In
-order to create a new issue or comment on an issue, you'll need to [login
-first](https://tree.taiga.io/login).
-
-Other questions or comments? Contact <mailto:witten@torsion.org>.
@@ -3,10 +3,10 @@ import os
 import subprocess

 from borgmatic.borg import extract
-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS


 DEFAULT_CHECKS = ('repository', 'archives')
+DEFAULT_PREFIX = '{hostname}-'


 logger = logging.getLogger(__name__)
@@ -31,10 +31,12 @@ def _parse_checks(consistency_config):
    if checks == ['disabled']:
        return ()

-    return tuple(check for check in checks if check.lower() not in ('disabled', '')) or DEFAULT_CHECKS
+    return (
+        tuple(check for check in checks if check.lower() not in ('disabled', '')) or DEFAULT_CHECKS
+    )


-def _make_check_flags(checks, check_last=None):
+def _make_check_flags(checks, check_last=None, prefix=None):
    '''
    Given a parsed sequence of checks, transform it into tuple of command-line flags.

@@ -46,39 +48,71 @@ def _make_check_flags(checks, check_last=None):

        ('--repository-only',)

-    Additionally, if a check_last value is given, a "--last" flag will be added.
+    However, if both "repository" and "archives" are in checks, then omit them from the returned
+    flags because Borg does both checks by default.
+
+    Additionally, if a check_last value is given and "archives" is in checks, then include a
+    "--last" flag. And if a prefix value is given and "archives" is in checks, then include a
+    "--prefix" flag.
    '''
-    last_flag = ('--last', str(check_last)) if check_last else ()
-    if checks == DEFAULT_CHECKS:
-        return last_flag
+    if 'archives' in checks:
+        last_flags = ('--last', str(check_last)) if check_last else ()
+        prefix_flags = ('--prefix', prefix) if prefix else ('--prefix', DEFAULT_PREFIX)
+    else:
+        last_flags = ()
+        prefix_flags = ()
+        if check_last:
+            logger.warning(
+                'Ignoring check_last option, as "archives" is not in consistency checks.'
+            )
+        if prefix:
+            logger.warning(
+                'Ignoring consistency prefix option, as "archives" is not in consistency checks.'
+            )

-    return tuple(
-        '--{}-only'.format(check) for check in checks
-        if check in DEFAULT_CHECKS
-    ) + last_flag
+    if set(DEFAULT_CHECKS).issubset(set(checks)):
+        return last_flags + prefix_flags
+
+    return (
+        tuple('--{}-only'.format(check) for check in checks if check in DEFAULT_CHECKS)
+        + last_flags
+        + prefix_flags
+    )


-def check_archives(verbosity, repository, consistency_config, remote_path=None):
+def check_archives(
+    repository, storage_config, consistency_config, local_path='borg', remote_path=None
+):
    '''
-    Given a verbosity flag, a local or remote repository path, a consistency config dict, and a
-    command to run, check the contained Borg archives for consistency.
+    Given a local or remote repository path, a storage config dict, a consistency config dict,
+    and a local/remote commands to run, check the contained Borg archives for consistency.

    If there are no consistency checks to run, skip running them.
    '''
    checks = _parse_checks(consistency_config)
    check_last = consistency_config.get('check_last', None)
+    lock_wait = None

    if set(checks).intersection(set(DEFAULT_CHECKS)):
        remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
-        verbosity_flags = {
-            VERBOSITY_SOME: ('--info',),
-            VERBOSITY_LOTS: ('--debug',),
-        }.get(verbosity, ())
+        lock_wait = storage_config.get('lock_wait', None)
+        lock_wait_flags = ('--lock-wait', str(lock_wait)) if lock_wait else ()
+
+        verbosity_flags = ()
+        if logger.isEnabledFor(logging.INFO):
+            verbosity_flags = ('--info',)
+        if logger.isEnabledFor(logging.DEBUG):
+            verbosity_flags = ('--debug', '--show-rc')
+
+        prefix = consistency_config.get('prefix')

        full_command = (
-            'borg', 'check',
-            repository,
-        ) + _make_check_flags(checks, check_last) + remote_path_flags + verbosity_flags
+            (local_path, 'check', repository)
+            + _make_check_flags(checks, check_last, prefix)
+            + remote_path_flags
+            + lock_wait_flags
+            + verbosity_flags
+        )

        # The check command spews to stdout/stderr even without the verbose flag. Suppress it.
        stdout = None if verbosity_flags else open(os.devnull, 'w')
@@ -87,4 +121,4 @@ def check_archives(verbosity, repository, consistency_config, remote_path=None):
        subprocess.check_call(full_command, stdout=stdout, stderr=subprocess.STDOUT)

    if 'extract' in checks:
-        extract.extract_last_archive_dry_run(verbosity, repository, remote_path)
+        extract.extract_last_archive_dry_run(repository, lock_wait, local_path, remote_path)
@@ -5,13 +5,15 @@ import os
 import subprocess
 import tempfile

-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
-

 logger = logging.getLogger(__name__)


 def initialize_environment(storage_config):
+    passcommand = storage_config.get('encryption_passcommand')
+    if passcommand:
+        os.environ['BORG_PASSCOMMAND'] = passcommand
+
    passphrase = storage_config.get('encryption_passphrase')
    if passphrase:
        os.environ['BORG_PASSPHRASE'] = passphrase
@@ -31,33 +33,61 @@ def _expand_directory(directory):
    return glob.glob(expanded_directory) or [expanded_directory]


-def _write_exclude_file(exclude_patterns=None):
+def _expand_directories(directories):
    '''
-    Given a sequence of exclude patterns, write them to a named temporary file and return it. Return
-    None if no patterns are provided.
+    Given a sequence of directory paths, expand tildes and globs in each one. Return all the
+    resulting directories as a single flattened tuple.
    '''
-    if not exclude_patterns:
+    if directories is None:
+        return ()
+
+    return tuple(
+        itertools.chain.from_iterable(_expand_directory(directory) for directory in directories)
+    )
+
+
+def _write_pattern_file(patterns=None):
+    '''
+    Given a sequence of patterns, write them to a named temporary file and return it. Return None
+    if no patterns are provided.
+    '''
+    if not patterns:
        return None

-    exclude_file = tempfile.NamedTemporaryFile('w')
-    exclude_file.write('\n'.join(exclude_patterns))
-    exclude_file.flush()
+    pattern_file = tempfile.NamedTemporaryFile('w')
+    pattern_file.write('\n'.join(patterns))
+    pattern_file.flush()

-    return exclude_file
+    return pattern_file


-def _make_exclude_flags(location_config, exclude_patterns_filename=None):
+def _make_pattern_flags(location_config, pattern_filename=None):
+    '''
+    Given a location config dict with a potential pattern_from option, and a filename containing any
+    additional patterns, return the corresponding Borg flags for those files as a tuple.
+    '''
+    pattern_filenames = tuple(location_config.get('patterns_from') or ()) + (
+        (pattern_filename,) if pattern_filename else ()
+    )
+
+    return tuple(
+        itertools.chain.from_iterable(
+            ('--patterns-from', pattern_filename) for pattern_filename in pattern_filenames
+        )
+    )
+
+
+def _make_exclude_flags(location_config, exclude_filename=None):
    '''
    Given a location config dict with various exclude options, and a filename containing any exclude
    patterns, return the corresponding Borg flags as a tuple.
    '''
    exclude_filenames = tuple(location_config.get('exclude_from') or ()) + (
-        (exclude_patterns_filename,) if exclude_patterns_filename else ()
+        (exclude_filename,) if exclude_filename else ()
    )
    exclude_from_flags = tuple(
        itertools.chain.from_iterable(
-            ('--exclude-from', exclude_filename)
-            for exclude_filename in exclude_filenames
+            ('--exclude-from', exclude_filename) for exclude_filename in exclude_filenames
        )
    )
    caches_flag = ('--exclude-caches',) if location_config.get('exclude_caches') else ()
@@ -68,51 +98,61 @@ def _make_exclude_flags(location_config, exclude_patterns_filename=None):


 def create_archive(
-    verbosity, repository, location_config, storage_config,
+    dry_run,
+    repository,
+    location_config,
+    storage_config,
+    local_path='borg',
+    remote_path=None,
+    json=False,
 ):
    '''
-    Given a vebosity flag, a local or remote repository path, a location config dict, and a storage
-    config dict, create a Borg archive.
+    Given vebosity/dry-run flags, a local or remote repository path, a location config dict, and a
+    storage config dict, create a Borg archive.
    '''
-    sources = tuple(
-        itertools.chain.from_iterable(
-            _expand_directory(directory)
-            for directory in location_config['source_directories']
-        )
-    )
+    sources = _expand_directories(location_config['source_directories'])

-    exclude_patterns_file = _write_exclude_file(location_config.get('exclude_patterns'))
-    exclude_flags = _make_exclude_flags(
-        location_config,
-        exclude_patterns_file.name if exclude_patterns_file else None,
-    )
+    pattern_file = _write_pattern_file(location_config.get('patterns'))
+    exclude_file = _write_pattern_file(_expand_directories(location_config.get('exclude_patterns')))
+    checkpoint_interval = storage_config.get('checkpoint_interval', None)
+    chunker_params = storage_config.get('chunker_params', None)
    compression = storage_config.get('compression', None)
-    compression_flags = ('--compression', compression) if compression else ()
    remote_rate_limit = storage_config.get('remote_rate_limit', None)
-    remote_rate_limit_flags = ('--remote-ratelimit', str(remote_rate_limit)) if remote_rate_limit else ()
    umask = storage_config.get('umask', None)
-    umask_flags = ('--umask', str(umask)) if umask else ()
-    one_file_system_flags = ('--one-file-system',) if location_config.get('one_file_system') else ()
+    lock_wait = storage_config.get('lock_wait', None)
    files_cache = location_config.get('files_cache')
-    files_cache_flags = ('--files-cache', files_cache) if files_cache else ()
-    remote_path = location_config.get('remote_path')
-    remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
-    verbosity_flags = {
-        VERBOSITY_SOME: ('--info', '--stats',),
-        VERBOSITY_LOTS: ('--debug', '--list', '--stats'),
-    }.get(verbosity, ())
    default_archive_name_format = '{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}'
    archive_name_format = storage_config.get('archive_name_format', default_archive_name_format)

    full_command = (
-        'borg', 'create',
-        '{repository}::{archive_name_format}'.format(
-            repository=repository,
-            archive_name_format=archive_name_format,
-        ),
-    ) + sources + exclude_flags + compression_flags + remote_rate_limit_flags + \
-        one_file_system_flags + files_cache_flags + remote_path_flags + umask_flags + \
-        verbosity_flags
+        (
+            local_path,
+            'create',
+            '{repository}::{archive_name_format}'.format(
+                repository=repository, archive_name_format=archive_name_format
+            ),
+        )
+        + sources
+        + _make_pattern_flags(location_config, pattern_file.name if pattern_file else None)
+        + _make_exclude_flags(location_config, exclude_file.name if exclude_file else None)
+        + (('--checkpoint-interval', str(checkpoint_interval)) if checkpoint_interval else ())
+        + (('--chunker-params', chunker_params) if chunker_params else ())
+        + (('--compression', compression) if compression else ())
+        + (('--remote-ratelimit', str(remote_rate_limit)) if remote_rate_limit else ())
+        + (('--one-file-system',) if location_config.get('one_file_system') else ())
+        + (('--read-special',) if location_config.get('read_special') else ())
+        + (('--nobsdflags',) if location_config.get('bsd_flags') is False else ())
+        + (('--files-cache', files_cache) if files_cache else ())
+        + (('--remote-path', remote_path) if remote_path else ())
+        + (('--umask', str(umask)) if umask else ())
+        + (('--lock-wait', str(lock_wait)) if lock_wait else ())
+        + (('--list', '--filter', 'AME') if logger.isEnabledFor(logging.INFO) else ())
+        + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+        + (('--stats',) if not dry_run and logger.isEnabledFor(logging.INFO) else ())
+        + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+        + (('--dry-run',) if dry_run else ())
+        + (('--json',) if json else ())
+    )

    logger.debug(' '.join(full_command))
    subprocess.check_call(full_command)
@@ -2,28 +2,29 @@ import logging
 import sys
 import subprocess

-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
-

 logger = logging.getLogger(__name__)


-def extract_last_archive_dry_run(verbosity, repository, remote_path=None):
+def extract_last_archive_dry_run(repository, lock_wait=None, local_path='borg', remote_path=None):
    '''
    Perform an extraction dry-run of just the most recent archive. If there are no archives, skip
    the dry-run.
    '''
    remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
-    verbosity_flags = {
-        VERBOSITY_SOME: ('--info',),
-        VERBOSITY_LOTS: ('--debug',),
-    }.get(verbosity, ())
+    lock_wait_flags = ('--lock-wait', str(lock_wait)) if lock_wait else ()
+    verbosity_flags = ()
+    if logger.isEnabledFor(logging.DEBUG):
+        verbosity_flags = ('--debug', '--show-rc')
+    elif logger.isEnabledFor(logging.INFO):
+        verbosity_flags = ('--info',)

    full_list_command = (
-        'borg', 'list',
-        '--short',
-        repository,
-    ) + remote_path_flags + verbosity_flags
+        (local_path, 'list', '--short', repository)
+        + remote_path_flags
+        + lock_wait_flags
+        + verbosity_flags
+    )

    list_output = subprocess.check_output(full_list_command).decode(sys.stdout.encoding)

@@ -31,15 +32,21 @@ def extract_last_archive_dry_run(verbosity, repository, remote_path=None):
    if not last_archive_name:
        return

-    list_flag = ('--list',) if verbosity == VERBOSITY_LOTS else ()
+    list_flag = ('--list',) if logger.isEnabledFor(logging.DEBUG) else ()
    full_extract_command = (
-        'borg', 'extract',
-        '--dry-run',
-        '{repository}::{last_archive_name}'.format(
-            repository=repository,
-            last_archive_name=last_archive_name,
-        ),
-    ) + remote_path_flags + verbosity_flags + list_flag
+        (
+            local_path,
+            'extract',
+            '--dry-run',
+            '{repository}::{last_archive_name}'.format(
+                repository=repository, last_archive_name=last_archive_name
+            ),
+        )
+        + remote_path_flags
+        + lock_wait_flags
+        + verbosity_flags
+        + list_flag
+    )

    logger.debug(' '.join(full_extract_command))
    subprocess.check_call(full_extract_command)
@@ -0,0 +1,29 @@
+import logging
+import subprocess
+
+
+logger = logging.getLogger(__name__)
+
+
+def display_archives_info(
+    repository, storage_config, local_path='borg', remote_path=None, json=False
+):
+    '''
+    Given a local or remote repository path, and a storage config dict,
+    display summary information for Borg archives in the repository.
+    '''
+    lock_wait = storage_config.get('lock_wait', None)
+
+    full_command = (
+        (local_path, 'info', repository)
+        + (('--remote-path', remote_path) if remote_path else ())
+        + (('--lock-wait', str(lock_wait)) if lock_wait else ())
+        + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+        + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+        + (('--json',) if json else ())
+    )
+
+    logger.debug(' '.join(full_command))
+
+    output = subprocess.check_output(full_command)
+    return output.decode() if output is not None else None
@@ -0,0 +1,26 @@
+import logging
+import subprocess
+
+
+logger = logging.getLogger(__name__)
+
+
+def list_archives(repository, storage_config, local_path='borg', remote_path=None, json=False):
+    '''
+    Given a local or remote repository path, and a storage config dict,
+    list Borg archives in the repository.
+    '''
+    lock_wait = storage_config.get('lock_wait', None)
+
+    full_command = (
+        (local_path, 'list', repository)
+        + (('--remote-path', remote_path) if remote_path else ())
+        + (('--lock-wait', str(lock_wait)) if lock_wait else ())
+        + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+        + (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+        + (('--json',) if json else ())
+    )
+    logger.debug(' '.join(full_command))
+
+    output = subprocess.check_output(full_command)
+    return output.decode() if output is not None else None
@@ -1,8 +1,6 @@
 import logging
 import subprocess

-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
-

 logger = logging.getLogger(__name__)

@@ -32,25 +30,28 @@ def _make_prune_flags(retention_config):
    )


-def prune_archives(verbosity, repository, retention_config, remote_path=None):
+def prune_archives(
+    dry_run, repository, storage_config, retention_config, local_path='borg', remote_path=None
+):
    '''
-    Given a verbosity flag, a local or remote repository path, a retention config dict, prune Borg
-    archives according the the retention policy specified in that configuration.
+    Given dry-run flag, a local or remote repository path, a storage config dict, and a
+    retention config dict, prune Borg archives according to the retention policy specified in that
+    configuration.
    '''
-    remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
-    verbosity_flags = {
-        VERBOSITY_SOME: ('--info', '--stats',),
-        VERBOSITY_LOTS: ('--debug', '--stats', '--list'),
-    }.get(verbosity, ())
+    umask = storage_config.get('umask', None)
+    lock_wait = storage_config.get('lock_wait', None)

    full_command = (
-        'borg', 'prune',
-        repository,
-    ) + tuple(
-        element
-        for pair in _make_prune_flags(retention_config)
-        for element in pair
-    ) + remote_path_flags + verbosity_flags
+        (local_path, 'prune', repository)
+        + tuple(element for pair in _make_prune_flags(retention_config) for element in pair)
+        + (('--remote-path', remote_path) if remote_path else ())
+        + (('--umask', str(umask)) if umask else ())
+        + (('--lock-wait', str(lock_wait)) if lock_wait else ())
+        + (('--stats',) if not dry_run and logger.isEnabledFor(logging.INFO) else ())
+        + (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+        + (('--debug', '--list', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+        + (('--dry-run',) if dry_run else ())
+    )

    logger.debug(' '.join(full_command))
    subprocess.check_call(full_command)
@@ -1,15 +1,21 @@
-
 from argparse import ArgumentParser
+import json
 import logging
 import os
 from subprocess import CalledProcessError
 import sys

-from borgmatic.borg import check, create, prune
+from borgmatic.borg import (
+    check as borg_check,
+    create as borg_create,
+    prune as borg_prune,
+    list as borg_list,
+    info as borg_info,
+)
 from borgmatic.commands import hook
-from borgmatic.config import collect, convert, validate
+from borgmatic.config import checks, collect, convert, validate
 from borgmatic.signals import configure_signals
-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS, verbosity_to_log_level
+from borgmatic.verbosity import verbosity_to_log_level


 logger = logging.getLogger(__name__)
@@ -23,20 +29,24 @@ def parse_arguments(*arguments):
    Given command-line arguments with which this script was invoked, parse the arguments and return
    them as an ArgumentParser instance.
    '''
+    config_paths = collect.get_default_config_paths()
+
    parser = ArgumentParser(
-        description=
-            '''
+        description='''
            A simple wrapper script for the Borg backup software that creates and prunes backups.
            If none of the --prune, --create, or --check options are given, then borgmatic defaults
            to all three: prune, create, and check archives.
            '''
    )
    parser.add_argument(
-        '-c', '--config',
+        '-c',
+        '--config',
        nargs='+',
        dest='config_paths',
-        default=collect.DEFAULT_CONFIG_PATHS,
-        help='Configuration filenames or directories, defaults to: {}'.format(' '.join(collect.DEFAULT_CONFIG_PATHS)),
+        default=config_paths,
+        help='Configuration filenames or directories, defaults to: {}'.format(
+            ' '.join(config_paths)
+        ),
    )
    parser.add_argument(
        '--excludes',
@@ -44,39 +54,73 @@ def parse_arguments(*arguments):
        help='Deprecated in favor of exclude_patterns within configuration',
    )
    parser.add_argument(
-        '-p', '--prune',
+        '-p',
+        '--prune',
        dest='prune',
        action='store_true',
        help='Prune archives according to the retention policy',
    )
    parser.add_argument(
-        '-C', '--create',
+        '-C',
+        '--create',
        dest='create',
        action='store_true',
        help='Create archives (actually perform backups)',
    )
    parser.add_argument(
-        '-k', '--check',
-        dest='check',
+        '-k', '--check', dest='check', action='store_true', help='Check archives for consistency'
+    )
+    parser.add_argument('-l', '--list', dest='list', action='store_true', help='List archives')
+    parser.add_argument(
+        '-i',
+        '--info',
+        dest='info',
        action='store_true',
-        help='Check archives for consistency',
+        help='Display summary information on archives',
    )
    parser.add_argument(
-        '-v', '--verbosity',
+        '--json',
+        dest='json',
+        default=False,
+        action='store_true',
+        help='Output results from the --create, --list, or --info options as json',
+    )
+    parser.add_argument(
+        '-n',
+        '--dry-run',
+        dest='dry_run',
+        action='store_true',
+        help='Go through the motions, but do not actually write to any repositories',
+    )
+    parser.add_argument(
+        '-v',
+        '--verbosity',
        type=int,
+        choices=range(0, 3),
+        default=0,
        help='Display verbose progress (1 for some, 2 for lots)',
    )

    args = parser.parse_args(arguments)

-    # If any of the three action flags in the given parse arguments have been explicitly requested,
-    # leave them as-is. Otherwise, assume defaults: Mutate the given arguments to enable all the
-    # actions.
-    if not args.prune and not args.create and not args.check:
-        args.prune = True
-        args.create = True
-        args.check = True
+    if args.json and not (args.create or args.list or args.info):
+        raise ValueError(
+            'The --json option can only be used with the --create, --list, or --info options'
+        )

+    if args.json and args.list and args.info:
+        raise ValueError(
+            'With the --json option, options --list and --info cannot be used together'
+        )
+
+    # If any of the action flags are explicitly requested, leave them as-is. Otherwise, assume
+    # defaults: Mutate the given arguments to enable the default actions.
+    if args.prune or args.create or args.check or args.list or args.info:
+        return args
+
+    args.prune = True
+    args.create = True
+    args.check = True
    return args


@@ -93,33 +137,107 @@ def run_configuration(config_filename, args):  # pragma: no cover
    )

    try:
+        local_path = location.get('local_path', 'borg')
        remote_path = location.get('remote_path')
-        create.initialize_environment(storage)
-        hook.execute_hook(hooks.get('before_backup'), config_filename, 'pre-backup')
+        borg_create.initialize_environment(storage)

-        for unexpanded_repository in location['repositories']:
-            repository = os.path.expanduser(unexpanded_repository)
-            if args.prune:
-                logger.info('{}: Pruning archives'.format(repository))
-                prune.prune_archives(args.verbosity, repository, retention, remote_path=remote_path)
-            if args.create:
-                logger.info('{}: Creating archive'.format(repository))
-                create.create_archive(
-                    args.verbosity,
-                    repository,
-                    location,
-                    storage,
-                )
-            if args.check:
-                logger.info('{}: Running consistency checks'.format(repository))
-                check.check_archives(args.verbosity, repository, consistency, remote_path=remote_path)
+        if args.create:
+            hook.execute_hook(hooks.get('before_backup'), config_filename, 'pre-backup')

-        hook.execute_hook(hooks.get('after_backup'), config_filename, 'post-backup')
+        _run_commands(
+            args=args,
+            consistency=consistency,
+            local_path=local_path,
+            location=location,
+            remote_path=remote_path,
+            retention=retention,
+            storage=storage,
+        )
+
+        if args.create:
+            hook.execute_hook(hooks.get('after_backup'), config_filename, 'post-backup')
    except (OSError, CalledProcessError):
        hook.execute_hook(hooks.get('on_error'), config_filename, 'on-error')
        raise


+def _run_commands(*, args, consistency, local_path, location, remote_path, retention, storage):
+    json_results = []
+    for unexpanded_repository in location['repositories']:
+        _run_commands_on_repository(
+            args=args,
+            consistency=consistency,
+            json_results=json_results,
+            local_path=local_path,
+            location=location,
+            remote_path=remote_path,
+            retention=retention,
+            storage=storage,
+            unexpanded_repository=unexpanded_repository,
+        )
+    if args.json:
+        sys.stdout.write(json.dumps(json_results))
+
+
+def _run_commands_on_repository(
+    *,
+    args,
+    consistency,
+    json_results,
+    local_path,
+    location,
+    remote_path,
+    retention,
+    storage,
+    unexpanded_repository
+):  # pragma: no cover
+    repository = os.path.expanduser(unexpanded_repository)
+    dry_run_label = ' (dry run; not making any changes)' if args.dry_run else ''
+    if args.prune:
+        logger.info('{}: Pruning archives{}'.format(repository, dry_run_label))
+        borg_prune.prune_archives(
+            args.dry_run,
+            repository,
+            storage,
+            retention,
+            local_path=local_path,
+            remote_path=remote_path,
+        )
+    if args.create:
+        logger.info('{}: Creating archive{}'.format(repository, dry_run_label))
+        borg_create.create_archive(
+            args.dry_run,
+            repository,
+            location,
+            storage,
+            local_path=local_path,
+            remote_path=remote_path,
+        )
+    if args.check and checks.repository_enabled_for_checks(repository, consistency):
+        logger.info('{}: Running consistency checks'.format(repository))
+        borg_check.check_archives(
+            repository, storage, consistency, local_path=local_path, remote_path=remote_path
+        )
+    if args.list:
+        logger.info('{}: Listing archives'.format(repository))
+        output = borg_list.list_archives(
+            repository, storage, local_path=local_path, remote_path=remote_path, json=args.json
+        )
+        if args.json:
+            json_results.append(json.loads(output))
+        else:
+            sys.stdout.write(output)
+    if args.info:
+        logger.info('{}: Displaying summary info for archives'.format(repository))
+        output = borg_info.display_archives_info(
+            repository, storage, local_path=local_path, remote_path=remote_path, json=args.json
+        )
+        if args.json:
+            json_results.append(json.loads(output))
+        else:
+            sys.stdout.write(output)
+
+
 def main():  # pragma: no cover
    try:
        configure_signals()
@@ -131,10 +249,14 @@ def main():  # pragma: no cover
        convert.guard_configuration_upgraded(LEGACY_CONFIG_PATH, config_filenames)

        if len(config_filenames) == 0:
-            raise ValueError('Error: No configuration files found in: {}'.format(' '.join(args.config_paths)))
+            raise ValueError(
+                'Error: No configuration files found in: {}'.format(' '.join(args.config_paths))
+            )

        for config_filename in config_filenames:
            run_configuration(config_filename, args)
    except (ValueError, OSError, CalledProcessError) as error:
        print(error, file=sys.stderr)
+        print(file=sys.stderr)
+        print('Need some help? https://torsion.org/borgmatic/#issues', file=sys.stderr)
        sys.exit(1)
@@ -1,6 +1,5 @@
 from argparse import ArgumentParser
 import os
-from subprocess import CalledProcessError
 import sys
 import textwrap

@@ -26,22 +25,31 @@ def parse_arguments(*arguments):
        '''
    )
    parser.add_argument(
-        '-s', '--source-config',
+        '-s',
+        '--source-config',
        dest='source_config_filename',
        default=DEFAULT_SOURCE_CONFIG_FILENAME,
-        help='Source INI-style configuration filename. Default: {}'.format(DEFAULT_SOURCE_CONFIG_FILENAME),
+        help='Source INI-style configuration filename. Default: {}'.format(
+            DEFAULT_SOURCE_CONFIG_FILENAME
+        ),
    )
    parser.add_argument(
-        '-e', '--source-excludes',
+        '-e',
+        '--source-excludes',
        dest='source_excludes_filename',
-        default=DEFAULT_SOURCE_EXCLUDES_FILENAME if os.path.exists(DEFAULT_SOURCE_EXCLUDES_FILENAME) else None,
+        default=DEFAULT_SOURCE_EXCLUDES_FILENAME
+        if os.path.exists(DEFAULT_SOURCE_EXCLUDES_FILENAME)
+        else None,
        help='Excludes filename',
    )
    parser.add_argument(
-        '-d', '--destination-config',
+        '-d',
+        '--destination-config',
        dest='destination_config_filename',
        default=DEFAULT_DESTINATION_CONFIG_FILENAME,
-        help='Destination YAML configuration filename. Default: {}'.format(DEFAULT_DESTINATION_CONFIG_FILENAME),
+        help='Destination YAML configuration filename. Default: {}'.format(
+            DEFAULT_DESTINATION_CONFIG_FILENAME
+        ),
    )

    return parser.parse_args(arguments)
@@ -57,11 +65,13 @@ def display_result(args):  # pragma: no cover
        ),
        TEXT_WRAP_CHARACTERS,
    )
-    
+
    delete_lines = textwrap.wrap(
        'Once you are satisfied, you can safely delete {}{}.'.format(
            args.source_config_filename,
-            ' and {}'.format(args.source_excludes_filename) if args.source_excludes_filename else '',
+            ' and {}'.format(args.source_excludes_filename)
+            if args.source_excludes_filename
+            else '',
        ),
        TEXT_WRAP_CHARACTERS,
    )
@@ -75,7 +85,9 @@ def main():  # pragma: no cover
    try:
        args = parse_arguments(*sys.argv[1:])
        schema = yaml.round_trip_load(open(validate.schema_filename()).read())
-        source_config = legacy.parse_configuration(args.source_config_filename, legacy.CONFIG_FORMAT)
+        source_config = legacy.parse_configuration(
+            args.source_config_filename, legacy.CONFIG_FORMAT
+        )
        source_config_file_mode = os.stat(args.source_config_filename).st_mode
        source_excludes = (
            open(args.source_excludes_filename).read().splitlines()
@@ -83,12 +95,12 @@ def main():  # pragma: no cover
            else []
        )

-        destination_config = convert.convert_legacy_parsed_config(source_config, source_excludes, schema)
+        destination_config = convert.convert_legacy_parsed_config(
+            source_config, source_excludes, schema
+        )

        generate.write_configuration(
-            args.destination_config_filename,
-            destination_config,
-            mode=source_config_file_mode,
+            args.destination_config_filename, destination_config, mode=source_config_file_mode
        )

        display_result(args)
@@ -1,9 +1,7 @@
 from argparse import ArgumentParser
-import os
-from subprocess import CalledProcessError
 import sys

-from borgmatic.config import convert, generate, validate
+from borgmatic.config import generate, validate


 DEFAULT_DESTINATION_CONFIG_FILENAME = '/etc/borgmatic/config.yaml'
@@ -16,10 +14,13 @@ def parse_arguments(*arguments):
    '''
    parser = ArgumentParser(description='Generate a sample borgmatic YAML configuration file.')
    parser.add_argument(
-        '-d', '--destination',
+        '-d',
+        '--destination',
        dest='destination_filename',
        default=DEFAULT_DESTINATION_CONFIG_FILENAME,
-        help='Destination YAML configuration filename. Default: {}'.format(DEFAULT_DESTINATION_CONFIG_FILENAME),
+        help='Destination YAML configuration filename. Default: {}'.format(
+            DEFAULT_DESTINATION_CONFIG_FILENAME
+        ),
    )

    return parser.parse_args(arguments)
@@ -29,12 +30,16 @@ def main():  # pragma: no cover
    try:
        args = parse_arguments(*sys.argv[1:])

-        generate.generate_sample_configuration(args.destination_filename, validate.schema_filename())
+        generate.generate_sample_configuration(
+            args.destination_filename, validate.schema_filename()
+        )

        print('Generated a sample configuration file at {}.'.format(args.destination_filename))
        print()
        print('Please edit the file to suit your needs. The values are just representative.')
        print('All fields are optional except where indicated.')
+        print()
+        print('If you ever need help: https://torsion.org/borgmatic/#issues')
    except (ValueError, OSError) as error:
        print(error, file=sys.stderr)
        sys.exit(1)
@@ -13,7 +13,11 @@ def execute_hook(commands, config_filename, description):
    if len(commands) == 1:
        logger.info('{}: Running command for {} hook'.format(config_filename, description))
    else:
-        logger.info('{}: Running {} commands for {} hook'.format(config_filename, len(commands), description))
+        logger.info(
+            '{}: Running {} commands for {} hook'.format(
+                config_filename, len(commands), description
+            )
+        )

    for command in commands:
        logger.debug('{}: Hook command: {}'.format(config_filename, command))
@@ -0,0 +1,9 @@
+def repository_enabled_for_checks(repository, consistency):
+    '''
+    Given a repository name and a consistency configuration dict, return whether the repository
+    is enabled to have consistency checks run.
+    '''
+    if not consistency.get('check_repositories'):
+        return True
+
+    return repository in consistency['check_repositories']
@@ -1,20 +1,35 @@
 import os


-DEFAULT_CONFIG_PATHS = ['/etc/borgmatic/config.yaml', '/etc/borgmatic.d']
+def get_default_config_paths():
+    '''
+    Based on the value of the XDG_CONFIG_HOME and HOME environment variables, return a list of
+    default configuration paths. This includes both system-wide configuration and configuration in
+    the current user's home directory.
+    '''
+    user_config_directory = os.getenv('XDG_CONFIG_HOME') or os.path.expandvars(
+        os.path.join('$HOME', '.config')
+    )
+
+    return [
+        '/etc/borgmatic/config.yaml',
+        '/etc/borgmatic.d',
+        '%s/borgmatic/config.yaml' % user_config_directory,
+    ]


 def collect_config_filenames(config_paths):
    '''
    Given a sequence of config paths, both filenames and directories, resolve that to just an
    iterable of files. Accomplish this by listing any given directories looking for contained config
-    files. This is non-recursive, so any directories within the given directories are ignored.
+    files (ending with the ".yaml" extension). This is non-recursive, so any directories within the
+    given directories are ignored.

    Return paths even if they don't exist on disk, so the user can find out about missing
    configuration paths. However, skip a default config path if it's missing, so the user doesn't
    have to create a default config path unless they need it.
    '''
-    real_default_config_paths = set(map(os.path.realpath, DEFAULT_CONFIG_PATHS))
+    real_default_config_paths = set(map(os.path.realpath, get_default_config_paths()))

    for path in config_paths:
        exists = os.path.exists(path)
@@ -28,5 +43,5 @@ def collect_config_filenames(config_paths):

        for filename in os.listdir(path):
            full_filename = os.path.join(path, filename)
-            if not os.path.isdir(full_filename):
+            if full_filename.endswith('.yaml') and not os.path.isdir(full_filename):
                yield full_filename
@@ -12,14 +12,17 @@ def _convert_section(source_section_config, section_schema):

    Where integer types exist in the given section schema, convert their values to integers.
    '''
-    destination_section_config = yaml.comments.CommentedMap([
-        (
-            option_name,
-            int(option_value)
-                if section_schema['map'].get(option_name, {}).get('type') == 'int' else option_value
-        )
-        for option_name, option_value in source_section_config.items()
-    ])
+    destination_section_config = yaml.comments.CommentedMap(
+        [
+            (
+                option_name,
+                int(option_value)
+                if section_schema['map'].get(option_name, {}).get('type') == 'int'
+                else option_value,
+            )
+            for option_name, option_value in source_section_config.items()
+        ]
+    )

    return destination_section_config

@@ -33,10 +36,12 @@ def convert_legacy_parsed_config(source_config, source_excludes, schema):
    Additionally, use the given schema as a source of helpful comments to include within the
    returned CommentedMap.
    '''
-    destination_config = yaml.comments.CommentedMap([
-        (section_name, _convert_section(section_config, schema['map'][section_name]))
-        for section_name, section_config in source_config._asdict().items()
-    ])
+    destination_config = yaml.comments.CommentedMap(
+        [
+            (section_name, _convert_section(section_config, schema['map'][section_name]))
+            for section_name, section_config in source_config._asdict().items()
+        ]
+    )

    # Split space-seperated values into actual lists, make "repository" into a list, and merge in
    # excludes.
@@ -53,9 +58,7 @@ def convert_legacy_parsed_config(source_config, source_excludes, schema):

    for section_name, section_config in destination_config.items():
        generate.add_comments_to_configuration(
-            section_config,
-            schema['map'][section_name],
-            indent=generate.INDENT,
+            section_config, schema['map'][section_name], indent=generate.INDENT
        )

    return destination_config
@@ -85,8 +88,7 @@ def guard_configuration_upgraded(source_config_filename, destination_config_file
    The idea is that we want to alert the user about upgrading their config if they haven't already.
    '''
    destination_config_exists = any(
-        os.path.exists(filename)
-        for filename in destination_config_filenames
+        os.path.exists(filename) for filename in destination_config_filenames
    )

    if os.path.exists(source_config_filename) and not destination_config_exists:
@@ -107,5 +109,5 @@ Please remove the "--excludes" argument and run borgmatic again.'''


 def guard_excludes_filename_omitted(excludes_filename):
-    if excludes_filename != None:
+    if excludes_filename is not None:
        raise LegacyExcludesFilenamePresent()
@@ -1,4 +1,3 @@
-from collections import OrderedDict
 import os

 from ruamel import yaml
@@ -9,12 +8,11 @@ INDENT = 4

 def _insert_newline_before_comment(config, field_name):
    '''
-    Using some ruamel.yaml black magic, insert a blank line in the config right befor the given
+    Using some ruamel.yaml black magic, insert a blank line in the config right before the given
    field and its comments.
    '''
    config.ca.items[field_name][1].insert(
-        0,
-        yaml.tokens.CommentToken('\n', yaml.error.CommentMark(0), None),
+        0, yaml.tokens.CommentToken('\n', yaml.error.CommentMark(0), None)
    )


@@ -27,23 +25,79 @@ def _schema_to_sample_configuration(schema, level=0):
    if example is not None:
        return example

-    config = yaml.comments.CommentedMap([
-        (
-            section_name,
-            _schema_to_sample_configuration(section_schema, level + 1),
-        )
-        for section_name, section_schema in schema['map'].items()
-    ])
+    config = yaml.comments.CommentedMap(
+        [
+            (section_name, _schema_to_sample_configuration(section_schema, level + 1))
+            for section_name, section_schema in schema['map'].items()
+        ]
+    )

    add_comments_to_configuration(config, schema, indent=(level * INDENT))

    return config


-def write_configuration(config_filename, config, mode=0o600):
+def _comment_out_line(line):
+    # If it's already is commented out (or empty), there's nothing further to do!
+    stripped_line = line.lstrip()
+    if not stripped_line or stripped_line.startswith('#'):
+        return line
+
+    # Comment out the names of optional sections.
+    one_indent = ' ' * INDENT
+    if not line.startswith(one_indent):
+        return '#' + line
+
+    # Otherwise, comment out the line, but insert the "#" after the first indent for aesthetics.
+    return '#'.join((one_indent, line[INDENT:]))
+
+
+REQUIRED_KEYS = {'source_directories', 'repositories', 'keep_daily'}
+REQUIRED_SECTION_NAMES = {'location', 'retention'}
+
+
+def _comment_out_optional_configuration(rendered_config):
    '''
-    Given a target config filename and a config data structure of nested OrderedDicts, write out the
-    config to file as YAML. Create any containing directories as needed.
+    Post-process a rendered configuration string to comment out optional key/values. The idea is
+    that this prevents the user from having to comment out a bunch of configuration they don't care
+    about to get to a minimal viable configuration file.
+
+    Ideally ruamel.yaml would support this during configuration generation, but it's not terribly
+    easy to accomplish that way.
+    '''
+    lines = []
+    required = False
+
+    for line in rendered_config.split('\n'):
+        key = line.strip().split(':')[0]
+
+        if key in REQUIRED_SECTION_NAMES:
+            lines.append(line)
+            continue
+
+        # Upon encountering a required configuration option, skip commenting out lines until the
+        # next blank line.
+        if key in REQUIRED_KEYS:
+            required = True
+        elif not key:
+            required = False
+
+        lines.append(_comment_out_line(line) if not required else line)
+
+    return '\n'.join(lines)
+
+
+def _render_configuration(config):
+    '''
+    Given a config data structure of nested OrderedDicts, render the config as YAML and return it.
+    '''
+    return yaml.round_trip_dump(config, indent=INDENT, block_seq_indent=INDENT)
+
+
+def write_configuration(config_filename, rendered_config, mode=0o600):
+    '''
+    Given a target config filename and rendered config YAML, write it out to file. Create any
+    containing directories as needed.
    '''
    if os.path.exists(config_filename):
        raise FileExistsError('{} already exists. Aborting.'.format(config_filename))
@@ -54,7 +108,7 @@ def write_configuration(config_filename, config, mode=0o600):
        pass

    with open(config_filename, 'w') as config_file:
-        config_file.write(yaml.round_trip_dump(config, indent=INDENT, block_seq_indent=INDENT))
+        config_file.write(rendered_config)

    os.chmod(config_filename, mode)

@@ -73,11 +127,7 @@ def add_comments_to_configuration(config, schema, indent=0):
        if not field_schema or not description:
            continue

-        config.yaml_set_comment_before_after_key(
-            key=field_name,
-            before=description,
-            indent=indent,
-        )
+        config.yaml_set_comment_before_after_key(key=field_name, before=description, indent=indent)
        if index > 0:
            _insert_newline_before_comment(config, field_name)

@@ -90,4 +140,6 @@ def generate_sample_configuration(config_filename, schema_filename):
    schema = yaml.round_trip_load(open(schema_filename))
    config = _schema_to_sample_configuration(schema)

-    write_configuration(config_filename, config)
+    write_configuration(
+        config_filename, _comment_out_optional_configuration(_render_configuration(config))
+    )
@@ -45,12 +45,8 @@ CONFIG_FORMAT = (
        ),
    ),
    Section_format(
-        'consistency',
-        (
-            option('checks', required=False),
-            option('check_last', required=False),
-        ),
-    )
+        'consistency', (option('checks', required=False), option('check_last', required=False))
+    ),
 )


@@ -66,7 +62,8 @@ def validate_configuration_format(parser, config_format):
    '''
    section_names = set(parser.sections())
    required_section_names = tuple(
-        section.name for section in config_format
+        section.name
+        for section in config_format
        if any(option.required for option in section.options)
    )

@@ -80,9 +77,7 @@ def validate_configuration_format(parser, config_format):

    missing_section_names = set(required_section_names) - section_names
    if missing_section_names:
-        raise ValueError(
-            'Missing config sections: {}'.format(', '.join(missing_section_names))
-        )
+        raise ValueError('Missing config sections: {}'.format(', '.join(missing_section_names)))

    for section_format in config_format:
        if section_format.name not in section_names:
@@ -91,26 +86,28 @@ def validate_configuration_format(parser, config_format):
        option_names = parser.options(section_format.name)
        expected_options = section_format.options

-        unexpected_option_names = set(option_names) - set(option.name for option in expected_options)
+        unexpected_option_names = set(option_names) - set(
+            option.name for option in expected_options
+        )

        if unexpected_option_names:
            raise ValueError(
                'Unexpected options found in config section {}: {}'.format(
-                    section_format.name,
-                    ', '.join(sorted(unexpected_option_names)),
+                    section_format.name, ', '.join(sorted(unexpected_option_names))
                )
            )

        missing_option_names = tuple(
-            option.name for option in expected_options if option.required
+            option.name
+            for option in expected_options
+            if option.required
            if option.name not in option_names
        )

        if missing_option_names:
            raise ValueError(
                'Required options missing from config section {}: {}'.format(
-                    section_format.name,
-                    ', '.join(missing_option_names)
+                    section_format.name, ', '.join(missing_option_names)
                )
            )

@@ -123,11 +120,7 @@ def parse_section_options(parser, section_format):

    Raise ValueError if any option values cannot be coerced to the expected Python data type.
    '''
-    type_getter = {
-        str: parser.get,
-        int: parser.getint,
-        bool: parser.getboolean,
-    }
+    type_getter = {str: parser.get, int: parser.getint, bool: parser.getboolean}

    return OrderedDict(
        (option.name, type_getter[option.value_type](section_format.name, option.name))
@@ -151,11 +144,10 @@ def parse_configuration(config_filename, config_format):

    # Describes a parsed configuration, where each attribute is the name of a configuration file
    # section and each value is a dict of that section's parsed options.
-    Parsed_config = namedtuple('Parsed_config', (section_format.name for section_format in config_format))
+    Parsed_config = namedtuple(
+        'Parsed_config', (section_format.name for section_format in config_format)
+    )

    return Parsed_config(
-        *(
-            parse_section_options(parser, section_format)
-            for section_format in config_format
-        )
+        *(parse_section_options(parser, section_format) for section_format in config_format)
    )
@@ -18,10 +18,31 @@ map:
                    - /home
                    - /etc
                    - /var/log/syslog*
+            repositories:
+                required: true
+                seq:
+                    - type: scalar
+                desc: |
+                    Paths to local or remote repositories (required). Tildes are expanded. Multiple
+                    repositories are backed up to in sequence. See ssh_command for SSH options like
+                    identity file or port.
+                example:
+                    - user@backupserver:sourcehostname.borg
            one_file_system:
                type: bool
                desc: Stay in same file system (do not cross mount points).
                example: true
+            read_special:
+                type: bool
+                desc: |
+                    Use Borg's --read-special flag to allow backup of block and other special
+                    devices. Use with caution, as it will lead to problems if used when
+                    backing up special devices such as /dev/zero.
+                example: false
+            bsd_flags:
+                type: bool
+                desc: Record bsdflags (e.g. NODUMP, IMMUTABLE) in archive. Defaults to true.
+                example: true
            files_cache:
                type: scalar
                desc: |
@@ -29,35 +50,52 @@ map:
                    https://borgbackup.readthedocs.io/en/stable/usage/create.html#description for
                    details.
                example: ctime,size,inode
+            local_path:
+                type: scalar
+                desc: Alternate Borg local executable. Defaults to "borg".
+                example: borg1
            remote_path:
                type: scalar
                desc: Alternate Borg remote executable. Defaults to "borg".
                example: borg1
-            repositories:
-                required: true
+            patterns:
                seq:
                    - type: scalar
                desc: |
-                    Paths to local or remote repositories (required). Tildes are expanded. Multiple
-                    repositories are backed up to in sequence.
+                    Any paths matching these patterns are included/excluded from backups. Globs are
+                    expanded. (Tildes are not.) Note that Borg considers this option experimental.
+                    See the output of "borg help patterns" for more details. Quote any value if it
+                    contains leading punctuation, so it parses correctly.
                example:
-                    - user@backupserver:sourcehostname.borg
+                    - 'R /'
+                    - '- /home/*/.cache'
+                    - '+ /home/susan'
+                    - '- /home/*'
+            patterns_from:
+                seq:
+                    - type: scalar
+                desc: |
+                    Read include/exclude patterns from one or more separate named files, one pattern
+                    per line. Note that Borg considers this option experimental. See the output of
+                    "borg help patterns" for more details.
+                example:
+                    - /etc/borgmatic/patterns
            exclude_patterns:
                seq:
                    - type: scalar
                desc: |
-                    Any paths matching these patterns are excluded from backups. Globs are expanded.
-                    See the output of "borg help patterns" for more details.
+                    Any paths matching these patterns are excluded from backups. Globs and tildes
+                    are expanded. See the output of "borg help patterns" for more details.
                example:
                    - '*.pyc'
-                    - /home/*/.cache
+                    - ~/*/.cache
                    - /etc/ssl
            exclude_from:
                seq:
                    - type: scalar
                desc: |
                    Read exclude patterns from one or more separate named files, one pattern per
-                    line.
+                    line. See the output of "borg help patterns" for more details.
                example:
                    - /etc/borgmatic/excludes
            exclude_caches:
@@ -77,6 +115,14 @@ map:
            https://borgbackup.readthedocs.io/en/stable/usage/general.html#environment-variables for
            details.
        map:
+            encryption_passcommand:
+                type: scalar
+                desc: |
+                    The standard output of this command is used to unlock the encryption key. Only
+                    use on repositories that were initialized with passcommand/repokey encryption.
+                    Note that if both encryption_passcommand and encryption_passphrase are set,
+                    then encryption_passphrase takes precedence.
+                example: "secret-tool lookup borg-repository repo-name"
            encryption_passphrase:
                type: scalar
                desc: |
@@ -85,6 +131,20 @@ map:
                    punctuation, so it parses correctly. And backslash any quote or backslash
                    literals as well.
                example: "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
+            checkpoint_interval:
+                type: int
+                desc: |
+                    Number of seconds between each checkpoint during a long-running backup. See
+                    https://borgbackup.readthedocs.io/en/stable/faq.html#if-a-backup-stops-mid-way-does-the-already-backed-up-data-stay-there
+                    for details. Defaults to checkpoints every 1800 seconds (30 minutes).
+                example: 1800
+            chunker_params:
+                type: scalar
+                desc: |
+                    Specify the parameters passed to then chunker (CHUNK_MIN_EXP, CHUNK_MAX_EXP,
+                    HASH_MASK_BITS, HASH_WINDOW_SIZE). See https://borgbackup.readthedocs.io/en/stable/internals.html
+                    for details.
+                example: 19,23,21,4095
            compression:
                type: scalar
                desc: |
@@ -104,6 +164,10 @@ map:
                type: scalar
                desc: Umask to be used for borg create.
                example: 0077
+            lock_wait:
+                type: int
+                desc: Maximum seconds to wait for acquiring a repository/cache lock.
+                example: 5
            archive_name_format:
                type: scalar
                desc: |
@@ -111,17 +175,23 @@ map:
                    "borg help placeholders" for details. Default is
                    "{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}". If you specify this option, you must
                    also specify a prefix in the retention section to avoid accidental pruning of
-                    archives with a different archive name format.
+                    archives with a different archive name format. And you should also specify a
+                    prefix in the consistency section as well.
                example: "{hostname}-documents-{now}"
    retention:
        desc: |
            Retention policy for how many backups to keep in each category. See
            https://borgbackup.readthedocs.org/en/stable/usage.html#borg-prune for details.
+            At least one of the "keep" options is required for pruning to work.
        map:
            keep_within:
                type: scalar
                desc: Keep all archives within this time interval.
                example: 3H
+            keep_secondly:
+                type: int
+                desc: Number of secondly archives to keep.
+                example: 60
            keep_minutely:
                type: int
                desc: Number of minutely archives to keep.
@@ -173,11 +243,28 @@ map:
                example:
                    - repository
                    - archives
+            check_repositories:
+                seq:
+                    - type: scalar
+                desc: |
+                    Paths to a subset of the repositories in the location section on which to run
+                    consistency checks. Handy in case some of your repositories are very large, and
+                    so running consistency checks on them would take too long. Defaults to running
+                    consistency checks on all repositories configured in the location section.
+                example:
+                    - user@backupserver:sourcehostname.borg
            check_last:
                type: int
                desc: Restrict the number of checked archives to the last n. Applies only to the
                      "archives" check.
                example: 3
+            prefix:
+                type: scalar
+                desc: |
+                    When performing the "archives" check, only consider archive names starting with
+                    this prefix. Borg placeholders can be used. See the output of
+                    "borg help placeholders" for details. Default is "{hostname}-".
+                example: sourcehostname
    hooks:
        desc: |
            Shell commands or scripts to execute before and after a backup or if an error has occurred.
@@ -1,6 +1,4 @@
 import logging
-import sys
-import warnings

 import pkg_resources
 import pykwalify.core
@@ -8,6 +6,9 @@ import pykwalify.errors
 from ruamel import yaml


+logger = logging.getLogger(__name__)
+
+
 def schema_filename():
    '''
    Path to the installed YAML configuration schema file, used to validate and parse the
@@ -21,6 +22,7 @@ class Validation_error(ValueError):
    A collection of error message strings generated when attempting to validate a particular
    configurartion file.
    '''
+
    def __init__(self, config_filename, error_messages):
        self.config_filename = config_filename
        self.error_messages = error_messages
@@ -45,9 +47,28 @@ def apply_logical_validation(config_filename, parsed_configuration):

    if archive_name_format and not prefix:
        raise Validation_error(
-            config_filename, (
-                'If you provide an archive_name_format, you must also specify a retention prefix.',
+            config_filename,
+            ('If you provide an archive_name_format, you must also specify a retention prefix.',),
+        )
+
+    location_repositories = parsed_configuration.get('location', {}).get('repositories')
+    check_repositories = parsed_configuration.get('consistency', {}).get('check_repositories', [])
+    for repository in check_repositories:
+        if repository not in location_repositories:
+            raise Validation_error(
+                config_filename,
+                (
+                    'Unknown repository in the consistency section\'s check_repositories: {}'.format(
+                        repository
+                    ),
+                ),
            )
+
+    consistency_prefix = parsed_configuration.get('consistency', {}).get('prefix')
+    if archive_name_format and not consistency_prefix:
+        logger.warning(
+            'Since version 1.1.16, if you provide `archive_name_format`, you should also'
+            ' specify `consistency.prefix`.'
        )


@@ -66,8 +87,8 @@ def parse_configuration(config_filename, schema_filename):
    logging.getLogger('pykwalify').setLevel(logging.ERROR)

    try:
-        config = yaml.round_trip_load(open(config_filename))
-        schema = yaml.round_trip_load(open(schema_filename))
+        config = yaml.safe_load(open(config_filename))
+        schema = yaml.safe_load(open(schema_filename))
    except yaml.error.YAMLError as error:
        raise Validation_error(config_filename, (str(error),))

@@ -1,66 +0,0 @@
-import os
-
-from flexmock import flexmock
-import pytest
-
-from borgmatic.commands import borgmatic as module
-
-
-def test_parse_arguments_with_no_arguments_uses_defaults():
-    parser = module.parse_arguments()
-
-    assert parser.config_paths == module.collect.DEFAULT_CONFIG_PATHS
-    assert parser.excludes_filename == None
-    assert parser.verbosity is None
-
-
-def test_parse_arguments_with_path_arguments_overrides_defaults():
-    parser = module.parse_arguments('--config', 'myconfig', '--excludes', 'myexcludes')
-
-    assert parser.config_paths == ['myconfig']
-    assert parser.excludes_filename == 'myexcludes'
-    assert parser.verbosity is None
-
-
-def test_parse_arguments_with_multiple_config_paths_parses_as_list():
-    parser = module.parse_arguments('--config', 'myconfig', 'otherconfig')
-
-    assert parser.config_paths == ['myconfig', 'otherconfig']
-    assert parser.verbosity is None
-
-
-def test_parse_arguments_with_verbosity_flag_overrides_default():
-    parser = module.parse_arguments('--verbosity', '1')
-
-    assert parser.config_paths == module.collect.DEFAULT_CONFIG_PATHS
-    assert parser.excludes_filename == None
-    assert parser.verbosity == 1
-
-
-def test_parse_arguments_with_no_actions_defaults_to_all_actions_enabled():
-    parser = module.parse_arguments()
-
-    assert parser.prune is True
-    assert parser.create is True
-    assert parser.check is True
-
-
-def test_parse_arguments_with_prune_action_leaves_other_actions_disabled():
-    parser = module.parse_arguments('--prune')
-
-    assert parser.prune is True
-    assert parser.create is False
-    assert parser.check is False
-
-
-def test_parse_arguments_with_multiple_actions_leaves_other_action_disabled():
-    parser = module.parse_arguments('--create', '--check')
-
-    assert parser.prune is False
-    assert parser.create is True
-    assert parser.check is True
-
-
-def test_parse_arguments_with_invalid_arguments_exits():
-    with pytest.raises(SystemExit):
-        module.parse_arguments('--posix-me-harder')
@@ -1,185 +0,0 @@
-from subprocess import STDOUT
-import sys
-
-from flexmock import flexmock
-import pytest
-
-from borgmatic.borg import check as module
-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
-
-
-def insert_subprocess_mock(check_call_command, **kwargs):
-    subprocess = flexmock(module.subprocess)
-    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
-
-
-def insert_subprocess_never():
-    subprocess = flexmock(module.subprocess)
-    subprocess.should_receive('check_call').never()
-
-
-def test_parse_checks_returns_them_as_tuple():
-    checks = module._parse_checks({'checks': ['foo', 'disabled', 'bar']})
-
-    assert checks == ('foo', 'bar')
-
-
-def test_parse_checks_with_missing_value_returns_defaults():
-    checks = module._parse_checks({})
-
-    assert checks == module.DEFAULT_CHECKS
-
-
-def test_parse_checks_with_blank_value_returns_defaults():
-    checks = module._parse_checks({'checks': []})
-
-    assert checks == module.DEFAULT_CHECKS
-
-
-def test_parse_checks_with_disabled_returns_no_checks():
-    checks = module._parse_checks({'checks': ['disabled']})
-
-    assert checks == ()
-
-
-def test_make_check_flags_with_checks_returns_flags():
-    flags = module._make_check_flags(('repository',))
-
-    assert flags == ('--repository-only',)
-
-
-def test_make_check_flags_with_extract_check_does_not_make_extract_flag():
-    flags = module._make_check_flags(('extract',))
-
-    assert flags == ()
-
-
-def test_make_check_flags_with_default_checks_returns_no_flags():
-    flags = module._make_check_flags(module.DEFAULT_CHECKS)
-
-    assert flags == ()
-
-
-def test_make_check_flags_with_checks_and_last_returns_flags_including_last():
-    flags = module._make_check_flags(('repository',), check_last=3)
-
-    assert flags == ('--repository-only', '--last', '3')
-
-
-def test_make_check_flags_with_default_checks_and_last_returns_last_flag():
-    flags = module._make_check_flags(module.DEFAULT_CHECKS, check_last=3)
-
-    assert flags == ('--last', '3')
-
-
-@pytest.mark.parametrize(
-    'checks',
-    (
-        ('repository',),
-        ('archives',),
-        ('repository', 'archives'),
-        ('repository', 'archives', 'other'),
-    ),
-)
-def test_check_archives_should_call_borg_with_parameters(checks):
-    check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
-    flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
-    stdout = flexmock()
-    insert_subprocess_mock(
-        ('borg', 'check', 'repo'),
-        stdout=stdout, stderr=STDOUT,
-    )
-    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
-    flexmock(module.os).should_receive('devnull')
-
-    module.check_archives(
-        verbosity=None,
-        repository='repo',
-        consistency_config=consistency_config,
-    )
-
-
-def test_check_archives_with_extract_check_should_call_extract_only():
-    checks = ('extract',)
-    check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
-    flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').never()
-    flexmock(module.extract).should_receive('extract_last_archive_dry_run').once()
-    insert_subprocess_never()
-
-    module.check_archives(
-        verbosity=None,
-        repository='repo',
-        consistency_config=consistency_config,
-    )
-
-
-def test_check_archives_with_verbosity_some_should_call_borg_with_info_parameter():
-    checks = ('repository',)
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
-    flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').and_return(())
-    insert_subprocess_mock(
-        ('borg', 'check', 'repo', '--info'),
-        stdout=None, stderr=STDOUT,
-    )
-
-    module.check_archives(
-        verbosity=VERBOSITY_SOME,
-        repository='repo',
-        consistency_config=consistency_config,
-    )
-
-
-def test_check_archives_with_verbosity_lots_should_call_borg_with_debug_parameter():
-    checks = ('repository',)
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
-    flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').and_return(())
-    insert_subprocess_mock(
-        ('borg', 'check', 'repo', '--debug'),
-        stdout=None, stderr=STDOUT,
-    )
-
-    module.check_archives(
-        verbosity=VERBOSITY_LOTS,
-        repository='repo',
-        consistency_config=consistency_config,
-    )
-
-
-def test_check_archives_without_any_checks_should_bail():
-    consistency_config = flexmock().should_receive('get').and_return(None).mock
-    flexmock(module).should_receive('_parse_checks').and_return(())
-    insert_subprocess_never()
-
-    module.check_archives(
-        verbosity=None,
-        repository='repo',
-        consistency_config=consistency_config,
-    )
-
-
-def test_check_archives_with_remote_path_should_call_borg_with_remote_path_parameters():
-    checks = ('repository',)
-    check_last = flexmock()
-    consistency_config = flexmock().should_receive('get').and_return(check_last).mock
-    flexmock(module).should_receive('_parse_checks').and_return(checks)
-    flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
-    stdout = flexmock()
-    insert_subprocess_mock(
-        ('borg', 'check', 'repo', '--remote-path', 'borg1'),
-        stdout=stdout, stderr=STDOUT,
-    )
-    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
-    flexmock(module.os).should_receive('devnull')
-
-    module.check_archives(
-        verbosity=None,
-        repository='repo',
-        consistency_config=consistency_config,
-        remote_path='borg1',
-    )
@@ -1,432 +0,0 @@
-import os
-
-from flexmock import flexmock
-
-from borgmatic.borg import create as module
-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
-
-
-def test_initialize_environment_with_passphrase_should_set_environment():
-    orig_environ = os.environ
-
-    try:
-        os.environ = {}
-        module.initialize_environment({'encryption_passphrase': 'pass'})
-        assert os.environ.get('BORG_PASSPHRASE') == 'pass'
-    finally:
-        os.environ = orig_environ
-
-
-def test_initialize_environment_with_ssh_command_should_set_environment():
-    orig_environ = os.environ
-
-    try:
-        os.environ = {}
-        module.initialize_environment({'ssh_command': 'ssh -C'})
-        assert os.environ.get('BORG_RSH') == 'ssh -C'
-    finally:
-        os.environ = orig_environ
-
-
-def test_initialize_environment_without_configuration_should_not_set_environment():
-    orig_environ = os.environ
-
-    try:
-        os.environ = {}
-        module.initialize_environment({})
-        assert os.environ.get('BORG_PASSPHRASE') == None
-        assert os.environ.get('BORG_RSH') == None
-    finally:
-        os.environ = orig_environ
-
-
-def test_expand_directory_with_basic_path_passes_it_through():
-    flexmock(module.os.path).should_receive('expanduser').and_return('foo')
-    flexmock(module.glob).should_receive('glob').and_return([])
-
-    paths = module._expand_directory('foo')
-
-    assert paths == ['foo']
-
-
-def test_expand_directory_with_glob_expands():
-    flexmock(module.os.path).should_receive('expanduser').and_return('foo*')
-    flexmock(module.glob).should_receive('glob').and_return(['foo', 'food'])
-
-    paths = module._expand_directory('foo*')
-
-    assert paths == ['foo', 'food']
-
-
-def test_write_exclude_file_does_not_raise():
-    temporary_file = flexmock(
-        name='filename',
-        write=lambda mode: None,
-        flush=lambda: None,
-    )
-    flexmock(module.tempfile).should_receive('NamedTemporaryFile').and_return(temporary_file)
-
-    module._write_exclude_file(['exclude'])
-
-
-def test_write_exclude_file_with_empty_exclude_patterns_does_not_raise():
-    module._write_exclude_file([])
-
-
-def insert_subprocess_mock(check_call_command, **kwargs):
-    subprocess = flexmock(module.subprocess)
-    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
-
-
-def test_make_exclude_flags_includes_exclude_patterns_filename_when_given():
-    exclude_flags = module._make_exclude_flags(
-        location_config={'exclude_patterns': ['*.pyc', '/var']},
-        exclude_patterns_filename='/tmp/excludes',
-    )
-
-    assert exclude_flags == ('--exclude-from', '/tmp/excludes')
-
-
-def test_make_exclude_flags_includes_exclude_from_filenames_when_in_config():
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-
-    exclude_flags = module._make_exclude_flags(
-        location_config={'exclude_from': ['excludes', 'other']},
-    )
-
-    assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', 'other')
-
-
-def test_make_exclude_flags_includes_both_filenames_when_patterns_given_and_exclude_from_in_config():
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-
-    exclude_flags = module._make_exclude_flags(
-        location_config={'exclude_from': ['excludes']},
-        exclude_patterns_filename='/tmp/excludes',
-    )
-
-    assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', '/tmp/excludes')
-
-
-def test_make_exclude_flags_considers_none_exclude_from_filenames_as_empty():
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-
-    exclude_flags = module._make_exclude_flags(
-        location_config={'exclude_from': None},
-    )
-
-    assert exclude_flags == ()
-
-
-def test_make_exclude_flags_includes_exclude_caches_when_true_in_config():
-    exclude_flags = module._make_exclude_flags(
-        location_config={'exclude_caches': True},
-    )
-
-    assert exclude_flags == ('--exclude-caches',)
-
-
-def test_make_exclude_flags_does_not_include_exclude_caches_when_false_in_config():
-    exclude_flags = module._make_exclude_flags(
-        location_config={'exclude_caches': False},
-    )
-
-    assert exclude_flags == ()
-
-
-def test_make_exclude_flags_includes_exclude_if_present_when_in_config():
-    exclude_flags = module._make_exclude_flags(
-        location_config={'exclude_if_present': 'exclude_me'},
-    )
-
-    assert exclude_flags == ('--exclude-if-present', 'exclude_me')
-
-
-def test_make_exclude_flags_is_empty_when_config_has_no_excludes():
-    exclude_flags = module._make_exclude_flags(location_config={})
-
-    assert exclude_flags == ()
-
-
-DEFAULT_ARCHIVE_NAME = '{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}'
-CREATE_COMMAND = ('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'bar')
-
-
-def test_create_archive_calls_borg_with_parameters():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND)
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_exclude_patterns_calls_borg_with_excludes():
-    exclude_flags = ('--exclude-from', 'excludes')
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(flexmock(name='/tmp/excludes'))
-    flexmock(module).should_receive('_make_exclude_flags').and_return(exclude_flags)
-    insert_subprocess_mock(CREATE_COMMAND + exclude_flags)
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': ['exclude'],
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_verbosity_some_calls_borg_with_info_parameter():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND + ('--info', '--stats',))
-
-    module.create_archive(
-        verbosity=VERBOSITY_SOME,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_verbosity_lots_calls_borg_with_debug_parameter():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND + ('--debug', '--list', '--stats'))
-
-    module.create_archive(
-        verbosity=VERBOSITY_LOTS,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_compression_calls_borg_with_compression_parameters():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND + ('--compression', 'rle'))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={'compression': 'rle'},
-    )
-
-
-def test_create_archive_with_remote_rate_limit_calls_borg_with_remote_ratelimit_parameters():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND + ('--remote-ratelimit', '100'))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={'remote_rate_limit': 100},
-    )
-
-
-def test_create_archive_with_one_file_system_calls_borg_with_one_file_system_parameters():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND + ('--one-file-system',))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'one_file_system': True,
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_files_cache_calls_borg_with_files_cache_parameters():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND + ('--files-cache', 'ctime,size'))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'files_cache': 'ctime,size',
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_remote_path_calls_borg_with_remote_path_parameters():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND + ('--remote-path', 'borg1'))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'remote_path': 'borg1',
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_umask_calls_borg_with_umask_parameters():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(CREATE_COMMAND + ('--umask', '740'))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={'umask': 740},
-    )
-
-
-def test_create_archive_with_source_directories_glob_expands():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo', 'food'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food'))
-    flexmock(module.glob).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo*'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_non_matching_source_directories_glob_passes_through():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo*'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo*'))
-    flexmock(module.glob).should_receive('glob').with_args('foo*').and_return([])
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo*'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_glob_calls_borg_with_expanded_directories():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo', 'food'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food'))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo*'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={},
-    )
-
-
-def test_create_archive_with_archive_name_format_calls_borg_with_archive_name():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(('borg', 'create', 'repo::ARCHIVE_NAME', 'foo', 'bar'))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={
-            'archive_name_format': 'ARCHIVE_NAME',
-        },
-    )
-
-
-def test_create_archive_with_archive_name_format_accepts_borg_placeholders():
-    flexmock(module).should_receive('_expand_directory').and_return(['foo']).and_return(['bar'])
-    flexmock(module).should_receive('_write_exclude_file').and_return(None)
-    flexmock(module).should_receive('_make_exclude_flags').and_return(())
-    insert_subprocess_mock(('borg', 'create', 'repo::Documents_{hostname}-{now}', 'foo', 'bar'))
-
-    module.create_archive(
-        verbosity=None,
-        repository='repo',
-        location_config={
-            'source_directories': ['foo', 'bar'],
-            'repositories': ['repo'],
-            'exclude_patterns': None,
-        },
-        storage_config={
-            'archive_name_format': 'Documents_{hostname}-{now}',
-        },
-    )
@@ -1,100 +0,0 @@
-import sys
-
-from flexmock import flexmock
-
-from borgmatic.borg import extract as module
-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
-
-
-def insert_subprocess_mock(check_call_command, **kwargs):
-    subprocess = flexmock(module.subprocess)
-    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
-
-
-def insert_subprocess_never():
-    subprocess = flexmock(module.subprocess)
-    subprocess.should_receive('check_call').never()
-
-
-def insert_subprocess_check_output_mock(check_output_command, result, **kwargs):
-    subprocess = flexmock(module.subprocess)
-    subprocess.should_receive('check_output').with_args(check_output_command, **kwargs).and_return(result).once()
-
-
-def test_extract_last_archive_dry_run_should_call_borg_with_last_archive():
-    flexmock(sys.stdout).encoding = 'utf-8'
-    insert_subprocess_check_output_mock(
-        ('borg', 'list', '--short', 'repo'),
-        result='archive1\narchive2\n'.encode('utf-8'),
-    )
-    insert_subprocess_mock(
-        ('borg', 'extract', '--dry-run', 'repo::archive2'),
-    )
-
-    module.extract_last_archive_dry_run(
-        verbosity=None,
-        repository='repo',
-    )
-
-
-def test_extract_last_archive_dry_run_without_any_archives_should_bail():
-    flexmock(sys.stdout).encoding = 'utf-8'
-    insert_subprocess_check_output_mock(
-        ('borg', 'list', '--short', 'repo'),
-        result='\n'.encode('utf-8'),
-    )
-    insert_subprocess_never()
-
-    module.extract_last_archive_dry_run(
-        verbosity=None,
-        repository='repo',
-    )
-
-
-def test_extract_last_archive_dry_run_with_verbosity_some_should_call_borg_with_info_parameter():
-    flexmock(sys.stdout).encoding = 'utf-8'
-    insert_subprocess_check_output_mock(
-        ('borg', 'list', '--short', 'repo', '--info'),
-        result='archive1\narchive2\n'.encode('utf-8'),
-    )
-    insert_subprocess_mock(
-        ('borg', 'extract', '--dry-run', 'repo::archive2', '--info'),
-    )
-
-    module.extract_last_archive_dry_run(
-        verbosity=VERBOSITY_SOME,
-        repository='repo',
-    )
-
-
-def test_extract_last_archive_dry_run_with_verbosity_lots_should_call_borg_with_debug_parameter():
-    flexmock(sys.stdout).encoding = 'utf-8'
-    insert_subprocess_check_output_mock(
-        ('borg', 'list', '--short', 'repo', '--debug'),
-        result='archive1\narchive2\n'.encode('utf-8'),
-    )
-    insert_subprocess_mock(
-        ('borg', 'extract', '--dry-run', 'repo::archive2', '--debug', '--list'),
-    )
-
-    module.extract_last_archive_dry_run(
-        verbosity=VERBOSITY_LOTS,
-        repository='repo',
-    )
-
-
-def test_extract_last_archive_dry_run_should_call_borg_with_remote_path_parameters():
-    flexmock(sys.stdout).encoding = 'utf-8'
-    insert_subprocess_check_output_mock(
-        ('borg', 'list', '--short', 'repo', '--remote-path', 'borg1'),
-        result='archive1\narchive2\n'.encode('utf-8'),
-    )
-    insert_subprocess_mock(
-        ('borg', 'extract', '--dry-run', 'repo::archive2', '--remote-path', 'borg1'),
-    )
-
-    module.extract_last_archive_dry_run(
-        verbosity=None,
-        repository='repo',
-        remote_path='borg1',
-    )
@@ -1,112 +0,0 @@
-from collections import OrderedDict
-
-from flexmock import flexmock
-
-from borgmatic.borg import prune as module
-from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
-
-
-def insert_subprocess_mock(check_call_command, **kwargs):
-    subprocess = flexmock(module.subprocess)
-    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
-
-
-BASE_PRUNE_FLAGS = (
-    ('--keep-daily', '1'),
-    ('--keep-weekly', '2'),
-    ('--keep-monthly', '3'),
-)
-
-
-def test_make_prune_flags_should_return_flags_from_config_plus_default_prefix():
-    retention_config = OrderedDict(
-        (
-            ('keep_daily', 1),
-            ('keep_weekly', 2),
-            ('keep_monthly', 3),
-        )
-    )
-
-    result = module._make_prune_flags(retention_config)
-
-    assert tuple(result) == BASE_PRUNE_FLAGS + (('--prefix', '{hostname}-'),)
-
-
-def test_make_prune_flags_accepts_prefix_with_placeholders():
-    retention_config = OrderedDict(
-        (
-            ('keep_daily', 1),
-            ('prefix', 'Documents_{hostname}-{now}'),
-        )
-    )
-
-    result = module._make_prune_flags(retention_config)
-
-    expected = (
-        ('--keep-daily', '1'),
-        ('--prefix', 'Documents_{hostname}-{now}'),
-    )
-
-    assert tuple(result) == expected
-
-
-PRUNE_COMMAND = (
-    'borg', 'prune', 'repo', '--keep-daily', '1', '--keep-weekly', '2', '--keep-monthly', '3',
-)
-
-
-def test_prune_archives_should_call_borg_with_parameters():
-    retention_config = flexmock()
-    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
-        BASE_PRUNE_FLAGS,
-    )
-    insert_subprocess_mock(PRUNE_COMMAND)
-
-    module.prune_archives(
-        verbosity=None,
-        repository='repo',
-        retention_config=retention_config,
-    )
-
-
-def test_prune_archives_with_verbosity_some_should_call_borg_with_info_parameter():
-    retention_config = flexmock()
-    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
-        BASE_PRUNE_FLAGS,
-    )
-    insert_subprocess_mock(PRUNE_COMMAND + ('--info', '--stats',))
-
-    module.prune_archives(
-        repository='repo',
-        verbosity=VERBOSITY_SOME,
-        retention_config=retention_config,
-    )
-
-
-def test_prune_archives_with_verbosity_lots_should_call_borg_with_debug_parameter():
-    retention_config = flexmock()
-    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
-        BASE_PRUNE_FLAGS,
-    )
-    insert_subprocess_mock(PRUNE_COMMAND + ('--debug', '--stats', '--list'))
-
-    module.prune_archives(
-        repository='repo',
-        verbosity=VERBOSITY_LOTS,
-        retention_config=retention_config,
-    )
-
-
-def test_prune_archives_with_remote_path_should_call_borg_with_remote_path_parameters():
-    retention_config = flexmock()
-    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
-        BASE_PRUNE_FLAGS,
-    )
-    insert_subprocess_mock(PRUNE_COMMAND + ('--remote-path', 'borg1'))
-
-    module.prune_archives(
-        verbosity=None,
-        repository='repo',
-        retention_config=retention_config,
-        remote_path='borg1',
-    )
@@ -1,49 +0,0 @@
-from collections import OrderedDict
-
-from flexmock import flexmock
-
-from borgmatic.config import generate as module
-
-
-def test_schema_to_sample_configuration_generates_config_with_examples():
-    flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
-    flexmock(module).should_receive('add_comments_to_configuration')
-    schema = {
-        'map': OrderedDict([
-            (
-                'section1', {
-                    'map': {
-                        'field1': OrderedDict([
-                            ('example', 'Example 1')
-                        ]),
-                    },
-                },
-            ),
-            (
-                'section2', {
-                    'map': OrderedDict([
-                        ('field2', {'example': 'Example 2'}),
-                        ('field3', {'example': 'Example 3'}),
-                    ]),
-                }
-            ),
-        ])
-    }
-
-    config = module._schema_to_sample_configuration(schema)
-
-    assert config == OrderedDict([
-        (
-            'section1',
-            OrderedDict([
-                ('field1', 'Example 1'),
-            ]),
-        ),
-        (
-            'section2',
-            OrderedDict([
-                ('field2', 'Example 2'),
-                ('field3', 'Example 3'),
-            ]),
-        )
-    ])
@@ -1,43 +0,0 @@
-import pytest
-
-from borgmatic.config import validate as module
-
-
-def test_validation_error_str_contains_error_messages_and_config_filename():
-    error = module.Validation_error('config.yaml', ('oops', 'uh oh'))
-
-    result = str(error)
-
-    assert 'config.yaml' in result
-    assert 'oops' in result
-    assert 'uh oh' in result
-
-
-def test_apply_logical_validation_raises_if_archive_name_format_present_without_prefix():
-    with pytest.raises(module.Validation_error):
-        module.apply_logical_validation(
-            'config.yaml',
-            {
-                'storage': {'archive_name_format': '{hostname}-{now}'},
-                'retention': {'keep_daily': 7},
-            },
-        )
-
-
-def test_apply_logical_validation_does_not_raise_if_archive_name_format_and_prefix_present():
-    module.apply_logical_validation(
-        'config.yaml',
-        {
-            'storage': {'archive_name_format': '{hostname}-{now}'},
-            'retention': {'prefix': '{hostname}-'},
-        },
-    )
-
-
-def test_apply_logical_validation_does_not_raise_otherwise():
-    module.apply_logical_validation(
-        'config.yaml',
-        {
-            'retention': {'keep_secondly': 1000},
-        },
-    )
@@ -1,11 +0,0 @@
-import logging
-
-from borgmatic import verbosity as module
-
-
-def test_verbosity_to_log_level_maps_known_verbosity_to_log_level():
-    assert module.verbosity_to_log_level(module.VERBOSITY_SOME) == logging.INFO
-
-
-def test_verbosity_to_log_level_maps_unknown_verbosity_to_error_level():
-    assert module.verbosity_to_log_level('my pants') == logging.ERROR
@@ -1,6 +1,7 @@
 import logging


+VERBOSITY_WARNING = 0
 VERBOSITY_SOME = 1
 VERBOSITY_LOTS = 2

@@ -10,6 +11,7 @@ def verbosity_to_log_level(verbosity):
    Given a borgmatic verbosity value, return the corresponding Python log level.
    '''
    return {
+        VERBOSITY_WARNING: logging.WARNING,
        VERBOSITY_SOME: logging.INFO,
        VERBOSITY_LOTS: logging.DEBUG,
-    }.get(verbosity, logging.ERROR)
+    }.get(verbosity, logging.WARNING)
@@ -0,0 +1,3 @@
+# You can drop this file into /etc/cron.d/ to run borgmatic nightly.
+
+0 3 * * * PATH=$PATH:/usr/bin /usr/bin/borgmatic
@@ -3,6 +3,7 @@ Description=Run borgmatic backup

 [Timer]
 OnCalendar=daily
+Persistent=true

 [Install]
 WantedBy=timers.target
@@ -0,0 +1,7 @@
+#!/usr/bin/env sh
+
+if which black; then
+    black --skip-string-normalization --line-length 100 --check .
+else
+    echo "Skipping black due to not being installed."
+fi
@@ -1,41 +1,55 @@
 #!/bin/bash

+set -o nounset
+
 # For each Borg sub-command that borgmatic uses, print out the Borg flags that borgmatic does not
 # appear to support yet. This script isn't terribly robust. It's intended as a basic tool to ferret
 # out unsupported Borg options so that they can be considered for addition to borgmatic.

-# Generate a sample borgmatic configuration with all options set.
+# Generate a sample borgmatic configuration with all options set, and uncomment all options.
 generate-borgmatic-config --destination temp.yaml
+cat temp.yaml | sed -e 's/# \S.*$//' | sed -e 's/#//' > temp.yaml.uncommented
+mv temp.yaml.uncommented temp.yaml

 # For each sub-command (prune, create, and check), collect the Borg command-line flags that result
 # from running borgmatic with the generated configuration. Then, collect the full set of available
 # Borg flags as reported by "borg --help" for that sub-command. Finally, compare the two lists of
 # flags to determine which Borg flags borgmatic doesn't yet support.
-for sub_command in prune create check; do
+for sub_command in prune create check list info; do
    echo "********** borg $sub_command **********"

-    for line in $(borgmatic --config temp.yaml --$sub_command -v 2 2>&1 | grep "borg $sub_command") ; do
+    for line in $(borgmatic --config temp.yaml --$sub_command -v 2 2>&1 | grep "borg\w* $sub_command") ; do
        echo "$line" | grep '^-' >> borgmatic_borg_flags
    done
    sort borgmatic_borg_flags > borgmatic_borg_flags.sorted
    mv borgmatic_borg_flags.sorted borgmatic_borg_flags

-    for line in $(borg $sub_command --help | awk -v RS= '/^usage:/') ; do
+    for word in $(borg $sub_command --help | grep '^  -') ; do
        # Exclude a bunch of flags that borgmatic actually supports, but don't get exercised by the
        # generated sample config, and also flags that don't make sense to support.
-        echo "$line" | grep -- -- | sed -r 's/(\[|\])//g' \
-            | grep -v '^-h$' \
+        echo "$word" | grep ^-- | sed -e 's/,$//' \
            | grep -v '^--archives-only$' \
-            | grep -v '^--repository-only$' \
-            | grep -v '^--stats$' \
-            | grep -v '^--list$' \
            | grep -v '^--critical$' \
-            | grep -v '^--error$' \
-            | grep -v '^--warning$' \
-            | grep -v '^--info$' \
            | grep -v '^--debug$' \
+            | grep -v '^--dry-run$' \
+            | grep -v '^--error$' \
+            | grep -v '^--help$' \
+            | grep -v '^--info$' \
+            | grep -v '^--json$' \
+            | grep -v '^--keep-last$' \
+            | grep -v '^--list$' \
+            | grep -v '^--nobsdflags$' \
+            | grep -v '^--pattern$' \
+            | grep -v '^--read-special$' \
+            | grep -v '^--repository-only$' \
+            | grep -v '^--show-rc$' \
+            | grep -v '^--stats$' \
+            | grep -v '^--verbose$' \
+            | grep -v '^--warning$' \
+            | grep -v '^-h$' \
            >> all_borg_flags
    done
+
    sort all_borg_flags > all_borg_flags.sorted
    mv all_borg_flags.sorted all_borg_flags

@@ -2,5 +2,5 @@

 set -e

-git push -u origin
-git push -u github
+git push -u origin master
+git push -u github master
@@ -2,8 +2,38 @@

 set -e

+projects_token=${1:-}
+github_token=${2:-}
+
+if [[ -z $github_token ]]; then
+    echo "Usage: $0 [projects-token] [github-token]"
+    exit 1
+fi
+if [[ ! -f NEWS ]]; then
+    echo "Missing NEWS file. Try running from root of repository."
+    exit 1
+fi
+
+version=$(head --lines=1 NEWS)
+git tag $version
+git push origin $version
+git push github $version
+
+# Build borgmatic and publish to pypi.
 rm -fr dist
 python3 setup.py bdist_wheel
 python3 setup.py sdist
 twine upload -r pypi dist/borgmatic-*.tar.gz
 twine upload -r pypi dist/borgmatic-*-py3-none-any.whl
+
+# Set release changelogs on projects.evoworx.org and GitHub.
+release_changelog="$(cat NEWS | sed '/^$/q' | grep -v '^\S')"
+escaped_release_changelog="$(echo "$release_changelog" | sed -z 's/\n/\\n/g' | sed -z 's/\"/\\"/g')"
+curl --silent --request POST \
+    "https://projects.torsion.org/api/v1/repos/witten/borgmatic/releases?access_token=$projects_token" \
+    --header "Accept: application/json" \
+    --header "Content-Type: application/json" \
+    --data "{\"body\": \"$escaped_release_changelog\", \"draft\": false, \"name\": \"borgmatic $version\", \"prerelease\": false, \"tag_name\": \"$version\"}"
+
+github-release create --token="$github_token" --owner=witten --repo=borgmatic --tag="$version" \
+    --name="borgmatic $version" --body="$release_changelog"
@@ -1,7 +1,7 @@
 from setuptools import setup, find_packages


-VERSION = '1.1.12.dev0'
+VERSION = '1.2.10'


 setup(
@@ -28,17 +28,7 @@ setup(
            'generate-borgmatic-config = borgmatic.commands.generate_config:main',
        ]
    },
-    obsoletes=[
-        'atticmatic',
-    ],
-    install_requires=(
-        'pykwalify>=1.6.0',
-        'ruamel.yaml<=0.15',
-        'setuptools',
-    ),
-    tests_require=(
-        'flexmock',
-        'pytest',
-    ),
+    obsoletes=['atticmatic'],
+    install_requires=('pykwalify>=1.6.0,<14.06', 'ruamel.yaml>0.15.0,<0.16.0', 'setuptools'),
    include_package_data=True,
 )
@@ -1,5 +1,23 @@
+appdirs==1.4.3
+atomicwrites==1.2.1
+attrs==18.2.0
+black==18.9b0; python_version >= '3.6'
+Click==7.0
+coverage==4.5.1
+docopt==0.6.2
+flake8==3.5.0
 flexmock==0.10.2
-pykwalify==1.6.0
-pytest==2.9.1
-pytest-cov==2.5.1
-ruamel.yaml==0.15.18
+mccabe==0.6.1
+more-itertools==4.3.0
+pluggy==0.7.1
+py==1.6.0
+pycodestyle==2.3.1
+pyflakes==2.0.0
+pykwalify==1.7.0
+pytest==3.8.2
+pytest-cov==2.6.0
+python-dateutil==2.7.3
+PyYAML==3.13
+ruamel.yaml>0.15.0,<0.16.0
+six==1.11.0
+toml==0.10.0
@@ -0,0 +1,63 @@
+import json
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+
+
+def generate_configuration(config_path, repository_path):
+    '''
+    Generate borgmatic configuration into a file at the config path, and update the defaults so as
+    to work for testing (including injecting the given repository path).
+    '''
+    subprocess.check_call(
+        'generate-borgmatic-config --destination {}'.format(config_path).split(' ')
+    )
+    config = (
+        open(config_path)
+        .read()
+        .replace('user@backupserver:sourcehostname.borg', repository_path)
+        .replace('- /home', '- {}'.format(config_path))
+        .replace('- /etc', '')
+        .replace('- /var/log/syslog*', '')
+    )
+    config_file = open(config_path, 'w')
+    config_file.write(config)
+    config_file.close()
+
+
+def test_borgmatic_command():
+    # Create a Borg repository.
+    temporary_directory = tempfile.mkdtemp()
+    repository_path = os.path.join(temporary_directory, 'test.borg')
+
+    try:
+        subprocess.check_call(
+            'borg init --encryption repokey {}'.format(repository_path).split(' '),
+            env={'BORG_PASSPHRASE': '', **os.environ},
+        )
+
+        config_path = os.path.join(temporary_directory, 'test.yaml')
+        generate_configuration(config_path, repository_path)
+
+        # Run borgmatic to generate a backup archive, and then list it to make sure it exists.
+        subprocess.check_call('borgmatic --config {}'.format(config_path).split(' '))
+        output = subprocess.check_output(
+            'borgmatic --config {} --list --json'.format(config_path).split(' ')
+        ).decode(sys.stdout.encoding)
+        parsed_output = json.loads(output)
+
+        assert len(parsed_output) == 1
+        assert len(parsed_output[0]['archives']) == 1
+
+        # Also exercise the info flag.
+        output = subprocess.check_output(
+            'borgmatic --config {} --info --json'.format(config_path).split(' ')
+        ).decode(sys.stdout.encoding)
+        parsed_output = json.loads(output)
+
+        assert len(parsed_output) == 1
+        assert 'repository' in parsed_output[0]
+    finally:
+        shutil.rmtree(temporary_directory)
@@ -0,0 +1,103 @@
+from flexmock import flexmock
+import pytest
+
+from borgmatic.commands import borgmatic as module
+
+
+def test_parse_arguments_with_no_arguments_uses_defaults():
+    config_paths = ['default']
+    flexmock(module.collect).should_receive('get_default_config_paths').and_return(config_paths)
+
+    parser = module.parse_arguments()
+
+    assert parser.config_paths == config_paths
+    assert parser.excludes_filename is None
+    assert parser.verbosity is 0
+    assert parser.json is False
+
+
+def test_parse_arguments_with_path_arguments_overrides_defaults():
+    flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
+
+    parser = module.parse_arguments('--config', 'myconfig', '--excludes', 'myexcludes')
+
+    assert parser.config_paths == ['myconfig']
+    assert parser.excludes_filename == 'myexcludes'
+    assert parser.verbosity is 0
+
+
+def test_parse_arguments_with_multiple_config_paths_parses_as_list():
+    flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
+
+    parser = module.parse_arguments('--config', 'myconfig', 'otherconfig')
+
+    assert parser.config_paths == ['myconfig', 'otherconfig']
+    assert parser.verbosity is 0
+
+
+def test_parse_arguments_with_verbosity_flag_overrides_default():
+    config_paths = ['default']
+    flexmock(module.collect).should_receive('get_default_config_paths').and_return(config_paths)
+
+    parser = module.parse_arguments('--verbosity', '1')
+
+    assert parser.config_paths == config_paths
+    assert parser.excludes_filename is None
+    assert parser.verbosity == 1
+
+
+def test_parse_arguments_with_json_flag_overrides_default():
+    parser = module.parse_arguments('--list', '--json')
+    assert parser.json is True
+
+
+def test_parse_arguments_with_no_actions_defaults_to_all_actions_enabled():
+    flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
+
+    parser = module.parse_arguments()
+
+    assert parser.prune is True
+    assert parser.create is True
+    assert parser.check is True
+
+
+def test_parse_arguments_with_prune_action_leaves_other_actions_disabled():
+    flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
+
+    parser = module.parse_arguments('--prune')
+
+    assert parser.prune is True
+    assert parser.create is False
+    assert parser.check is False
+
+
+def test_parse_arguments_with_multiple_actions_leaves_other_action_disabled():
+    flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
+
+    parser = module.parse_arguments('--create', '--check')
+
+    assert parser.prune is False
+    assert parser.create is True
+    assert parser.check is True
+
+
+def test_parse_arguments_with_invalid_arguments_exits():
+    flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
+
+    with pytest.raises(SystemExit):
+        module.parse_arguments('--posix-me-harder')
+
+
+def test_parse_arguments_with_json_flag_with_list_or_info_flag_does_not_raise_any_error():
+    module.parse_arguments('--list', '--json')
+    module.parse_arguments('--info', '--json')
+
+
+def test_parse_arguments_with_json_flag_but_no_list_or_info_flag_raises_value_error():
+    with pytest.raises(ValueError):
+        module.parse_arguments('--json')
+
+
+def test_parse_arguments_with_json_flag_and_both_list_and_info_flag_raises_value_error():
+    with pytest.raises(ValueError):
+        module.parse_arguments('--list', '--info', '--json')
@@ -20,9 +20,12 @@ def test_parse_arguments_with_filename_arguments_overrides_defaults():
    flexmock(os.path).should_receive('exists').and_return(True)

    parser = module.parse_arguments(
-        '--source-config', 'config',
-        '--source-excludes', 'excludes',
-        '--destination-config', 'config.yaml',
+        '--source-config',
+        'config',
+        '--source-excludes',
+        'excludes',
+        '--destination-config',
+        'config.yaml',
    )

    assert parser.source_config_filename == 'config'
@@ -6,6 +6,7 @@ def test_parse_arguments_with_no_arguments_uses_defaults():

    assert parser.destination_filename == module.DEFAULT_DESTINATION_CONFIG_FILENAME

+
 def test_parse_arguments_with_filename_argument_overrides_defaults():
    parser = module.parse_arguments('--destination', 'config.yaml')

@@ -11,11 +11,74 @@ from borgmatic.config import generate as module
 def test_insert_newline_before_comment_does_not_raise():
    field_name = 'foo'
    config = module.yaml.comments.CommentedMap([(field_name, 33)])
-    config.yaml_set_comment_before_after_key(key=field_name, before='Comment',)
+    config.yaml_set_comment_before_after_key(key=field_name, before='Comment')

    module._insert_newline_before_comment(config, field_name)


+def test_comment_out_line_skips_blank_line():
+    line = '    \n'
+
+    assert module._comment_out_line(line) == line
+
+
+def test_comment_out_line_skips_already_commented_out_line():
+    line = '    # foo'
+
+    assert module._comment_out_line(line) == line
+
+
+def test_comment_out_line_comments_section_name():
+    line = 'figgy-pudding:'
+
+    assert module._comment_out_line(line) == '#' + line
+
+
+def test_comment_out_line_comments_indented_option():
+    line = '    enabled: true'
+
+    assert module._comment_out_line(line) == '    #enabled: true'
+
+
+def test_comment_out_optional_configuration_comments_optional_config_only():
+    flexmock(module)._comment_out_line = lambda line: '#' + line
+    config = '''
+foo:
+    bar:
+        - baz
+        - quux
+
+location:
+    repositories:
+        - one
+        - two
+
+    other: thing
+    '''
+
+    expected_config = '''
+#foo:
+#    bar:
+#        - baz
+#        - quux
+#
+location:
+    repositories:
+        - one
+        - two
+#
+#    other: thing
+    '''
+
+    assert module._comment_out_optional_configuration(config.strip()) == expected_config.strip()
+
+
+def test_render_configuration_does_not_raise():
+    flexmock(module.yaml).should_receive('round_trip_dump')
+
+    module._render_configuration({})
+
+
 def test_write_configuration_does_not_raise():
    flexmock(os.path).should_receive('exists').and_return(False)
    flexmock(os).should_receive('makedirs')
@@ -23,14 +86,14 @@ def test_write_configuration_does_not_raise():
    builtins.should_receive('open').and_return(StringIO())
    flexmock(os).should_receive('chmod')

-    module.write_configuration('config.yaml', {})
+    module.write_configuration('config.yaml', 'config: yaml')


 def test_write_configuration_with_already_existing_file_raises():
    flexmock(os.path).should_receive('exists').and_return(True)

    with pytest.raises(FileExistsError):
-        module.write_configuration('config.yaml', {})
+        module.write_configuration('config.yaml', 'config: yaml')


 def test_write_configuration_with_already_existing_directory_does_not_raise():
@@ -40,18 +103,13 @@ def test_write_configuration_with_already_existing_directory_does_not_raise():
    builtins.should_receive('open').and_return(StringIO())
    flexmock(os).should_receive('chmod')

-    module.write_configuration('config.yaml', {})
+    module.write_configuration('config.yaml', 'config: yaml')


 def test_add_comments_to_configuration_does_not_raise():
    # Ensure that it can deal with fields both in the schema and missing from the schema.
    config = module.yaml.comments.CommentedMap([('foo', 33), ('bar', 44), ('baz', 55)])
-    schema = {
-        'map': {
-            'foo': {'desc': 'Foo'},
-            'bar': {'desc': 'Bar'},
-        }
-    }
+    schema = {'map': {'foo': {'desc': 'Foo'}, 'bar': {'desc': 'Bar'}}}

    module.add_comments_to_configuration(config, schema)

@@ -59,7 +117,9 @@ def test_add_comments_to_configuration_does_not_raise():
 def test_generate_sample_configuration_does_not_raise():
    builtins = flexmock(sys.modules['builtins'])
    builtins.should_receive('open').with_args('schema.yaml').and_return('')
-    flexmock(module).should_receive('write_configuration')
    flexmock(module).should_receive('_schema_to_sample_configuration')
+    flexmock(module).should_receive('_render_configuration')
+    flexmock(module).should_receive('_comment_out_optional_configuration')
+    flexmock(module).should_receive('write_configuration')

    module.generate_sample_configuration('config.yaml', 'schema.yaml')
@@ -8,17 +8,12 @@ from borgmatic.config import legacy as module

 def test_parse_section_options_with_punctuation_should_return_section_options():
    parser = module.RawConfigParser()
-    parser.readfp(StringIO('[section]\nfoo: {}\n'.format(string.punctuation)))
+    parser.read_file(StringIO('[section]\nfoo: {}\n'.format(string.punctuation)))

    section_format = module.Section_format(
-        'section',
-        (module.Config_option('foo', str, required=True),),
+        'section', (module.Config_option('foo', str, required=True),)
    )

    config = module.parse_section_options(parser, section_format)

-    assert config == OrderedDict(
-        (
-            ('foo', string.punctuation),
-        )
-    )
+    assert config == OrderedDict((('foo', string.punctuation),))
@@ -1,7 +1,6 @@
 import io
 import string
 import sys
-import os

 from flexmock import flexmock
 import pytest
@@ -10,7 +9,7 @@ from borgmatic.config import validate as module


 def test_schema_filename_returns_plausable_path():
-    schema_path = module.schema_filename()    
+    schema_path = module.schema_filename()

    assert schema_path.endswith('/schema.yaml')

@@ -75,7 +74,9 @@ def test_parse_configuration_passes_through_quoted_punctuation():

            repositories:
                - "{}.borg"
-        '''.format(escaped_punctuation)
+        '''.format(
+            escaped_punctuation
+        )
    )

    result = module.parse_configuration('config.yaml', 'schema.yaml')
@@ -84,7 +85,7 @@ def test_parse_configuration_passes_through_quoted_punctuation():
        'location': {
            'source_directories': ['/home'],
            'repositories': ['{}.borg'.format(string.punctuation)],
-        },
+        }
    }


@@ -111,7 +112,7 @@ def test_parse_configuration_with_schema_lacking_examples_does_not_raise():
                        required: true
                        seq:
                            - type: scalar
-        '''
+        ''',
    )

    module.parse_configuration('config.yaml', 'schema.yaml')
@@ -0,0 +1,265 @@
+from subprocess import STDOUT
+import logging
+import sys
+
+from flexmock import flexmock
+import pytest
+
+from borgmatic.borg import check as module
+from ..test_verbosity import insert_logging_mock
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
+
+
+def insert_subprocess_never():
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_call').never()
+
+
+def test_parse_checks_returns_them_as_tuple():
+    checks = module._parse_checks({'checks': ['foo', 'disabled', 'bar']})
+
+    assert checks == ('foo', 'bar')
+
+
+def test_parse_checks_with_missing_value_returns_defaults():
+    checks = module._parse_checks({})
+
+    assert checks == module.DEFAULT_CHECKS
+
+
+def test_parse_checks_with_blank_value_returns_defaults():
+    checks = module._parse_checks({'checks': []})
+
+    assert checks == module.DEFAULT_CHECKS
+
+
+def test_parse_checks_with_disabled_returns_no_checks():
+    checks = module._parse_checks({'checks': ['disabled']})
+
+    assert checks == ()
+
+
+def test_make_check_flags_with_repository_check_returns_flag():
+    flags = module._make_check_flags(('repository',))
+
+    assert flags == ('--repository-only',)
+
+
+def test_make_check_flags_with_extract_omits_extract_flag():
+    flags = module._make_check_flags(('extract',))
+
+    assert flags == ()
+
+
+def test_make_check_flags_with_default_checks_returns_default_flags():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS)
+
+    assert flags == ('--prefix', module.DEFAULT_PREFIX)
+
+
+def test_make_check_flags_with_all_checks_returns_default_flags():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS + ('extract',))
+
+    assert flags == ('--prefix', module.DEFAULT_PREFIX)
+
+
+def test_make_check_flags_with_archives_check_and_last_includes_last_flag():
+    flags = module._make_check_flags(('archives',), check_last=3)
+
+    assert flags == ('--archives-only', '--last', '3', '--prefix', module.DEFAULT_PREFIX)
+
+
+def test_make_check_flags_with_repository_check_and_last_omits_last_flag():
+    flags = module._make_check_flags(('repository',), check_last=3)
+
+    assert flags == ('--repository-only',)
+
+
+def test_make_check_flags_with_default_checks_and_last_includes_last_flag():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS, check_last=3)
+
+    assert flags == ('--last', '3', '--prefix', module.DEFAULT_PREFIX)
+
+
+def test_make_check_flags_with_archives_check_and_prefix_includes_prefix_flag():
+    flags = module._make_check_flags(('archives',), prefix='foo-')
+
+    assert flags == ('--archives-only', '--prefix', 'foo-')
+
+
+def test_make_check_flags_with_repository_check_and_prefix_omits_prefix_flag():
+    flags = module._make_check_flags(('repository',), prefix='foo-')
+
+    assert flags == ('--repository-only',)
+
+
+def test_make_check_flags_with_default_checks_and_prefix_includes_prefix_flag():
+    flags = module._make_check_flags(module.DEFAULT_CHECKS, prefix='foo-')
+
+    assert flags == ('--prefix', 'foo-')
+
+
+@pytest.mark.parametrize(
+    'checks',
+    (
+        ('repository',),
+        ('archives',),
+        ('repository', 'archives'),
+        ('repository', 'archives', 'other'),
+    ),
+)
+def test_check_archives_calls_borg_with_parameters(checks):
+    check_last = flexmock()
+    consistency_config = {'check_last': check_last}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(
+        checks, check_last, None
+    ).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(('borg', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        repository='repo', storage_config={}, consistency_config=consistency_config
+    )
+
+
+def test_check_archives_with_extract_check_calls_extract_only():
+    checks = ('extract',)
+    check_last = flexmock()
+    consistency_config = {'check_last': check_last}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').never()
+    flexmock(module.extract).should_receive('extract_last_archive_dry_run').once()
+    insert_subprocess_never()
+
+    module.check_archives(
+        repository='repo', storage_config={}, consistency_config=consistency_config
+    )
+
+
+def test_check_archives_with_log_info_calls_borg_with_info_parameter():
+    checks = ('repository',)
+    consistency_config = {'check_last': None}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').and_return(())
+    insert_logging_mock(logging.INFO)
+    insert_subprocess_mock(('borg', 'check', 'repo', '--info'), stdout=None, stderr=STDOUT)
+
+    module.check_archives(
+        repository='repo', storage_config={}, consistency_config=consistency_config
+    )
+
+
+def test_check_archives_with_log_debug_calls_borg_with_debug_parameter():
+    checks = ('repository',)
+    consistency_config = {'check_last': None}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').and_return(())
+    insert_logging_mock(logging.DEBUG)
+    insert_subprocess_mock(
+        ('borg', 'check', 'repo', '--debug', '--show-rc'), stdout=None, stderr=STDOUT
+    )
+
+    module.check_archives(
+        repository='repo', storage_config={}, consistency_config=consistency_config
+    )
+
+
+def test_check_archives_without_any_checks_bails():
+    consistency_config = {'check_last': None}
+    flexmock(module).should_receive('_parse_checks').and_return(())
+    insert_subprocess_never()
+
+    module.check_archives(
+        repository='repo', storage_config={}, consistency_config=consistency_config
+    )
+
+
+def test_check_archives_with_local_path_calls_borg_via_local_path():
+    checks = ('repository',)
+    check_last = flexmock()
+    consistency_config = {'check_last': check_last}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(
+        checks, check_last, None
+    ).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(('borg1', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        repository='repo',
+        storage_config={},
+        consistency_config=consistency_config,
+        local_path='borg1',
+    )
+
+
+def test_check_archives_with_remote_path_calls_borg_with_remote_path_parameters():
+    checks = ('repository',)
+    check_last = flexmock()
+    consistency_config = {'check_last': check_last}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(
+        checks, check_last, None
+    ).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(
+        ('borg', 'check', 'repo', '--remote-path', 'borg1'), stdout=stdout, stderr=STDOUT
+    )
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        repository='repo',
+        storage_config={},
+        consistency_config=consistency_config,
+        remote_path='borg1',
+    )
+
+
+def test_check_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
+    checks = ('repository',)
+    check_last = flexmock()
+    consistency_config = {'check_last': check_last}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(
+        checks, check_last, None
+    ).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(
+        ('borg', 'check', 'repo', '--lock-wait', '5'), stdout=stdout, stderr=STDOUT
+    )
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        repository='repo', storage_config={'lock_wait': 5}, consistency_config=consistency_config
+    )
+
+
+def test_check_archives_with_retention_prefix():
+    checks = ('repository',)
+    check_last = flexmock()
+    prefix = 'foo-'
+    consistency_config = {'check_last': check_last, 'prefix': prefix}
+    flexmock(module).should_receive('_parse_checks').and_return(checks)
+    flexmock(module).should_receive('_make_check_flags').with_args(
+        checks, check_last, prefix
+    ).and_return(())
+    stdout = flexmock()
+    insert_subprocess_mock(('borg', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
+
+    flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
+    flexmock(module.os).should_receive('devnull')
+
+    module.check_archives(
+        repository='repo', storage_config={}, consistency_config=consistency_config
+    )
@@ -0,0 +1,750 @@
+import logging
+import os
+
+from flexmock import flexmock
+
+from borgmatic.borg import create as module
+from ..test_verbosity import insert_logging_mock
+
+
+def test_initialize_environment_with_passcommand_should_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize_environment({'encryption_passcommand': 'command'})
+        assert os.environ.get('BORG_PASSCOMMAND') == 'command'
+    finally:
+        os.environ = orig_environ
+
+
+def test_initialize_environment_with_passphrase_should_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize_environment({'encryption_passphrase': 'pass'})
+        assert os.environ.get('BORG_PASSPHRASE') == 'pass'
+    finally:
+        os.environ = orig_environ
+
+
+def test_initialize_environment_with_ssh_command_should_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize_environment({'ssh_command': 'ssh -C'})
+        assert os.environ.get('BORG_RSH') == 'ssh -C'
+    finally:
+        os.environ = orig_environ
+
+
+def test_initialize_environment_without_configuration_should_not_set_environment():
+    orig_environ = os.environ
+
+    try:
+        os.environ = {}
+        module.initialize_environment({})
+        assert os.environ.get('BORG_PASSCOMMAND') is None
+        assert os.environ.get('BORG_PASSPHRASE') is None
+        assert os.environ.get('BORG_RSH') is None
+    finally:
+        os.environ = orig_environ
+
+
+def test_expand_directory_with_basic_path_passes_it_through():
+    flexmock(module.os.path).should_receive('expanduser').and_return('foo')
+    flexmock(module.glob).should_receive('glob').and_return([])
+
+    paths = module._expand_directory('foo')
+
+    assert paths == ['foo']
+
+
+def test_expand_directory_with_glob_expands():
+    flexmock(module.os.path).should_receive('expanduser').and_return('foo*')
+    flexmock(module.glob).should_receive('glob').and_return(['foo', 'food'])
+
+    paths = module._expand_directory('foo*')
+
+    assert paths == ['foo', 'food']
+
+
+def test_expand_directories_flattens_expanded_directories():
+    flexmock(module).should_receive('_expand_directory').with_args('~/foo').and_return(
+        ['/root/foo']
+    )
+    flexmock(module).should_receive('_expand_directory').with_args('bar*').and_return(
+        ['bar', 'barf']
+    )
+
+    paths = module._expand_directories(('~/foo', 'bar*'))
+
+    assert paths == ('/root/foo', 'bar', 'barf')
+
+
+def test_expand_directories_considers_none_as_no_directories():
+    paths = module._expand_directories(None)
+
+    assert paths == ()
+
+
+def test_write_pattern_file_does_not_raise():
+    temporary_file = flexmock(name='filename', write=lambda mode: None, flush=lambda: None)
+    flexmock(module.tempfile).should_receive('NamedTemporaryFile').and_return(temporary_file)
+
+    module._write_pattern_file(['exclude'])
+
+
+def test_write_pattern_file_with_empty_exclude_patterns_does_not_raise():
+    module._write_pattern_file([])
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
+
+
+def test_make_pattern_flags_includes_pattern_filename_when_given():
+    pattern_flags = module._make_pattern_flags(
+        location_config={'patterns': ['R /', '- /var']}, pattern_filename='/tmp/patterns'
+    )
+
+    assert pattern_flags == ('--patterns-from', '/tmp/patterns')
+
+
+def test_make_pattern_flags_includes_patterns_from_filenames_when_in_config():
+    pattern_flags = module._make_pattern_flags(
+        location_config={'patterns_from': ['patterns', 'other']}
+    )
+
+    assert pattern_flags == ('--patterns-from', 'patterns', '--patterns-from', 'other')
+
+
+def test_make_pattern_flags_includes_both_filenames_when_patterns_given_and_patterns_from_in_config():
+    pattern_flags = module._make_pattern_flags(
+        location_config={'patterns_from': ['patterns']}, pattern_filename='/tmp/patterns'
+    )
+
+    assert pattern_flags == ('--patterns-from', 'patterns', '--patterns-from', '/tmp/patterns')
+
+
+def test_make_pattern_flags_considers_none_patterns_from_filenames_as_empty():
+    pattern_flags = module._make_pattern_flags(location_config={'patterns_from': None})
+
+    assert pattern_flags == ()
+
+
+def test_make_exclude_flags_includes_exclude_patterns_filename_when_given():
+    exclude_flags = module._make_exclude_flags(
+        location_config={'exclude_patterns': ['*.pyc', '/var']}, exclude_filename='/tmp/excludes'
+    )
+
+    assert exclude_flags == ('--exclude-from', '/tmp/excludes')
+
+
+def test_make_exclude_flags_includes_exclude_from_filenames_when_in_config():
+
+    exclude_flags = module._make_exclude_flags(
+        location_config={'exclude_from': ['excludes', 'other']}
+    )
+
+    assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', 'other')
+
+
+def test_make_exclude_flags_includes_both_filenames_when_patterns_given_and_exclude_from_in_config():
+    exclude_flags = module._make_exclude_flags(
+        location_config={'exclude_from': ['excludes']}, exclude_filename='/tmp/excludes'
+    )
+
+    assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', '/tmp/excludes')
+
+
+def test_make_exclude_flags_considers_none_exclude_from_filenames_as_empty():
+    exclude_flags = module._make_exclude_flags(location_config={'exclude_from': None})
+
+    assert exclude_flags == ()
+
+
+def test_make_exclude_flags_includes_exclude_caches_when_true_in_config():
+    exclude_flags = module._make_exclude_flags(location_config={'exclude_caches': True})
+
+    assert exclude_flags == ('--exclude-caches',)
+
+
+def test_make_exclude_flags_does_not_include_exclude_caches_when_false_in_config():
+    exclude_flags = module._make_exclude_flags(location_config={'exclude_caches': False})
+
+    assert exclude_flags == ()
+
+
+def test_make_exclude_flags_includes_exclude_if_present_when_in_config():
+    exclude_flags = module._make_exclude_flags(location_config={'exclude_if_present': 'exclude_me'})
+
+    assert exclude_flags == ('--exclude-if-present', 'exclude_me')
+
+
+def test_make_exclude_flags_is_empty_when_config_has_no_excludes():
+    exclude_flags = module._make_exclude_flags(location_config={})
+
+    assert exclude_flags == ()
+
+
+DEFAULT_ARCHIVE_NAME = '{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}'
+CREATE_COMMAND = ('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'bar')
+
+
+def test_create_archive_calls_borg_with_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND)
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_patterns_calls_borg_with_patterns():
+    pattern_flags = ('--patterns-from', 'patterns')
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(
+        flexmock(name='/tmp/patterns')
+    ).and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(pattern_flags)
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + pattern_flags)
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'patterns': ['pattern'],
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_exclude_patterns_calls_borg_with_excludes():
+    exclude_flags = ('--exclude-from', 'excludes')
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(
+        ('exclude',)
+    )
+    flexmock(module).should_receive('_write_pattern_file').and_return(None).and_return(
+        flexmock(name='/tmp/excludes')
+    )
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(exclude_flags)
+    insert_subprocess_mock(CREATE_COMMAND + exclude_flags)
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': ['exclude'],
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_log_info_calls_borg_with_info_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--list', '--filter', 'AME', '--info', '--stats'))
+    insert_logging_mock(logging.INFO)
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_log_debug_calls_borg_with_debug_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(
+        CREATE_COMMAND + ('--list', '--filter', 'AME', '--stats', '--debug', '--show-rc')
+    )
+    insert_logging_mock(logging.DEBUG)
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_dry_run_calls_borg_with_dry_run_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--dry-run',))
+
+    module.create_archive(
+        dry_run=True,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_dry_run_and_log_info_calls_borg_without_stats_parameter():
+    """ --dry-run and --stats are mutually exclusive, see:
+    https://borgbackup.readthedocs.io/en/stable/usage/create.html#description """
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--list', '--filter', 'AME', '--info', '--dry-run'))
+    insert_logging_mock(logging.INFO)
+
+    module.create_archive(
+        dry_run=True,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_dry_run_and_log_debug_calls_borg_without_stats_parameter():
+    """ --dry-run and --stats are mutually exclusive, see:
+    https://borgbackup.readthedocs.io/en/stable/usage/create.html#description """
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(
+        CREATE_COMMAND + ('--list', '--filter', 'AME', '--debug', '--show-rc', '--dry-run')
+    )
+    insert_logging_mock(logging.DEBUG)
+
+    module.create_archive(
+        dry_run=True,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_checkpoint_interval_calls_borg_with_checkpoint_interval_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--checkpoint-interval', '600'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'checkpoint_interval': 600},
+    )
+
+
+def test_create_archive_with_chunker_params_calls_borg_with_chunker_params_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--chunker-params', '1,2,3,4'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'chunker_params': '1,2,3,4'},
+    )
+
+
+def test_create_archive_with_compression_calls_borg_with_compression_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--compression', 'rle'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'compression': 'rle'},
+    )
+
+
+def test_create_archive_with_remote_rate_limit_calls_borg_with_remote_ratelimit_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--remote-ratelimit', '100'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'remote_rate_limit': 100},
+    )
+
+
+def test_create_archive_with_one_file_system_calls_borg_with_one_file_system_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--one-file-system',))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'one_file_system': True,
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_read_special_calls_borg_with_read_special_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--read-special',))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'read_special': True,
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_bsd_flags_true_calls_borg_without_nobsdflags_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND)
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'bsd_flags': True,
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_bsd_flags_false_calls_borg_with_nobsdflags_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--nobsdflags',))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'bsd_flags': False,
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_files_cache_calls_borg_with_files_cache_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--files-cache', 'ctime,size'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'files_cache': 'ctime,size',
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_local_path_calls_borg_via_local_path():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(('borg1',) + CREATE_COMMAND[1:])
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        local_path='borg1',
+    )
+
+
+def test_create_archive_with_remote_path_calls_borg_with_remote_path_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--remote-path', 'borg1'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        remote_path='borg1',
+    )
+
+
+def test_create_archive_with_umask_calls_borg_with_umask_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--umask', '740'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'umask': 740},
+    )
+
+
+def test_create_archive_with_lock_wait_calls_borg_with_lock_wait_parameters():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--lock-wait', '5'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'lock_wait': 5},
+    )
+
+
+def test_create_archive_with_json_calls_borg_with_json_parameter():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(CREATE_COMMAND + ('--json',))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+        json=True,
+    )
+
+
+def test_create_archive_with_source_directories_glob_expands():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food')).and_return(
+        ()
+    )
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(
+        ('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food')
+    )
+    flexmock(module.glob).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo*'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_non_matching_source_directories_glob_passes_through():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo*',)).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo*'))
+    flexmock(module.glob).should_receive('glob').with_args('foo*').and_return([])
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo*'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_glob_calls_borg_with_expanded_directories():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food')).and_return(
+        ()
+    )
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(
+        ('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food')
+    )
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo*'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={},
+    )
+
+
+def test_create_archive_with_archive_name_format_calls_borg_with_archive_name():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(('borg', 'create', 'repo::ARCHIVE_NAME', 'foo', 'bar'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'archive_name_format': 'ARCHIVE_NAME'},
+    )
+
+
+def test_create_archive_with_archive_name_format_accepts_borg_placeholders():
+    flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return([])
+    flexmock(module).should_receive('_write_pattern_file').and_return(None)
+    flexmock(module).should_receive('_make_pattern_flags').and_return(())
+    flexmock(module).should_receive('_make_exclude_flags').and_return(())
+    insert_subprocess_mock(('borg', 'create', 'repo::Documents_{hostname}-{now}', 'foo', 'bar'))
+
+    module.create_archive(
+        dry_run=False,
+        repository='repo',
+        location_config={
+            'source_directories': ['foo', 'bar'],
+            'repositories': ['repo'],
+            'exclude_patterns': None,
+        },
+        storage_config={'archive_name_format': 'Documents_{hostname}-{now}'},
+    )
@@ -0,0 +1,103 @@
+import logging
+import sys
+
+from flexmock import flexmock
+
+from borgmatic.borg import extract as module
+from ..test_verbosity import insert_logging_mock
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
+
+
+def insert_subprocess_never():
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_call').never()
+
+
+def insert_subprocess_check_output_mock(check_output_command, result, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_output').with_args(check_output_command, **kwargs).and_return(
+        result
+    ).once()
+
+
+def test_extract_last_archive_dry_run_should_call_borg_with_last_archive():
+    flexmock(sys.stdout).encoding = 'utf-8'
+    insert_subprocess_check_output_mock(
+        ('borg', 'list', '--short', 'repo'), result='archive1\narchive2\n'.encode('utf-8')
+    )
+    insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2'))
+
+    module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
+
+
+def test_extract_last_archive_dry_run_without_any_archives_should_bail():
+    flexmock(sys.stdout).encoding = 'utf-8'
+    insert_subprocess_check_output_mock(
+        ('borg', 'list', '--short', 'repo'), result='\n'.encode('utf-8')
+    )
+    insert_subprocess_never()
+
+    module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
+
+
+def test_extract_last_archive_dry_run_with_log_info_should_call_borg_with_info_parameter():
+    flexmock(sys.stdout).encoding = 'utf-8'
+    insert_subprocess_check_output_mock(
+        ('borg', 'list', '--short', 'repo', '--info'), result='archive1\narchive2\n'.encode('utf-8')
+    )
+    insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2', '--info'))
+    insert_logging_mock(logging.INFO)
+
+    module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
+
+
+def test_extract_last_archive_dry_run_with_log_debug_should_call_borg_with_debug_parameter():
+    flexmock(sys.stdout).encoding = 'utf-8'
+    insert_subprocess_check_output_mock(
+        ('borg', 'list', '--short', 'repo', '--debug', '--show-rc'),
+        result='archive1\narchive2\n'.encode('utf-8'),
+    )
+    insert_subprocess_mock(
+        ('borg', 'extract', '--dry-run', 'repo::archive2', '--debug', '--show-rc', '--list')
+    )
+    insert_logging_mock(logging.DEBUG)
+
+    module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
+
+
+def test_extract_last_archive_dry_run_should_call_borg_via_local_path():
+    flexmock(sys.stdout).encoding = 'utf-8'
+    insert_subprocess_check_output_mock(
+        ('borg1', 'list', '--short', 'repo'), result='archive1\narchive2\n'.encode('utf-8')
+    )
+    insert_subprocess_mock(('borg1', 'extract', '--dry-run', 'repo::archive2'))
+
+    module.extract_last_archive_dry_run(repository='repo', lock_wait=None, local_path='borg1')
+
+
+def test_extract_last_archive_dry_run_should_call_borg_with_remote_path_parameters():
+    flexmock(sys.stdout).encoding = 'utf-8'
+    insert_subprocess_check_output_mock(
+        ('borg', 'list', '--short', 'repo', '--remote-path', 'borg1'),
+        result='archive1\narchive2\n'.encode('utf-8'),
+    )
+    insert_subprocess_mock(
+        ('borg', 'extract', '--dry-run', 'repo::archive2', '--remote-path', 'borg1')
+    )
+
+    module.extract_last_archive_dry_run(repository='repo', lock_wait=None, remote_path='borg1')
+
+
+def test_extract_last_archive_dry_run_should_call_borg_with_lock_wait_parameters():
+    flexmock(sys.stdout).encoding = 'utf-8'
+    insert_subprocess_check_output_mock(
+        ('borg', 'list', '--short', 'repo', '--lock-wait', '5'),
+        result='archive1\narchive2\n'.encode('utf-8'),
+    )
+    insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2', '--lock-wait', '5'))
+
+    module.extract_last_archive_dry_run(repository='repo', lock_wait=5)
@@ -0,0 +1,58 @@
+import logging
+
+from flexmock import flexmock
+
+from borgmatic.borg import info as module
+from ..test_verbosity import insert_logging_mock
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_output').with_args(check_call_command, **kwargs).once()
+
+
+INFO_COMMAND = ('borg', 'info', 'repo')
+
+
+def test_display_archives_info_calls_borg_with_parameters():
+    insert_subprocess_mock(INFO_COMMAND)
+
+    module.display_archives_info(repository='repo', storage_config={})
+
+
+def test_display_archives_info_with_log_info_calls_borg_with_info_parameter():
+    insert_subprocess_mock(INFO_COMMAND + ('--info',))
+    insert_logging_mock(logging.INFO)
+    module.display_archives_info(repository='repo', storage_config={})
+
+
+def test_display_archives_info_with_log_debug_calls_borg_with_debug_parameter():
+    insert_subprocess_mock(INFO_COMMAND + ('--debug', '--show-rc'))
+    insert_logging_mock(logging.DEBUG)
+
+    module.display_archives_info(repository='repo', storage_config={})
+
+
+def test_display_archives_info_with_json_calls_borg_with_json_parameter():
+    insert_subprocess_mock(INFO_COMMAND + ('--json',))
+
+    module.display_archives_info(repository='repo', storage_config={}, json=True)
+
+
+def test_display_archives_info_with_local_path_calls_borg_via_local_path():
+    insert_subprocess_mock(('borg1',) + INFO_COMMAND[1:])
+
+    module.display_archives_info(repository='repo', storage_config={}, local_path='borg1')
+
+
+def test_display_archives_info_with_remote_path_calls_borg_with_remote_path_parameters():
+    insert_subprocess_mock(INFO_COMMAND + ('--remote-path', 'borg1'))
+
+    module.display_archives_info(repository='repo', storage_config={}, remote_path='borg1')
+
+
+def test_display_archives_info_with_lock_wait_calls_borg_with_lock_wait_parameters():
+    storage_config = {'lock_wait': 5}
+    insert_subprocess_mock(INFO_COMMAND + ('--lock-wait', '5'))
+
+    module.display_archives_info(repository='repo', storage_config=storage_config)
@@ -0,0 +1,59 @@
+import logging
+
+from flexmock import flexmock
+
+from borgmatic.borg import list as module
+from ..test_verbosity import insert_logging_mock
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_output').with_args(check_call_command, **kwargs).once()
+
+
+LIST_COMMAND = ('borg', 'list', 'repo')
+
+
+def test_list_archives_calls_borg_with_parameters():
+    insert_subprocess_mock(LIST_COMMAND)
+
+    module.list_archives(repository='repo', storage_config={})
+
+
+def test_list_archives_with_log_info_calls_borg_with_info_parameter():
+    insert_subprocess_mock(LIST_COMMAND + ('--info',))
+    insert_logging_mock(logging.INFO)
+
+    module.list_archives(repository='repo', storage_config={})
+
+
+def test_list_archives_with_log_debug_calls_borg_with_debug_parameter():
+    insert_subprocess_mock(LIST_COMMAND + ('--debug', '--show-rc'))
+    insert_logging_mock(logging.DEBUG)
+
+    module.list_archives(repository='repo', storage_config={})
+
+
+def test_list_archives_with_json_calls_borg_with_json_parameter():
+    insert_subprocess_mock(LIST_COMMAND + ('--json',))
+
+    module.list_archives(repository='repo', storage_config={}, json=True)
+
+
+def test_list_archives_with_local_path_calls_borg_via_local_path():
+    insert_subprocess_mock(('borg1',) + LIST_COMMAND[1:])
+
+    module.list_archives(repository='repo', storage_config={}, local_path='borg1')
+
+
+def test_list_archives_with_remote_path_calls_borg_with_remote_path_parameters():
+    insert_subprocess_mock(LIST_COMMAND + ('--remote-path', 'borg1'))
+
+    module.list_archives(repository='repo', storage_config={}, remote_path='borg1')
+
+
+def test_list_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
+    storage_config = {'lock_wait': 5}
+    insert_subprocess_mock(LIST_COMMAND + ('--lock-wait', '5'))
+
+    module.list_archives(repository='repo', storage_config=storage_config)
@@ -0,0 +1,160 @@
+import logging
+from collections import OrderedDict
+
+from flexmock import flexmock
+
+from borgmatic.borg import prune as module
+from ..test_verbosity import insert_logging_mock
+
+
+def insert_subprocess_mock(check_call_command, **kwargs):
+    subprocess = flexmock(module.subprocess)
+    subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
+
+
+BASE_PRUNE_FLAGS = (('--keep-daily', '1'), ('--keep-weekly', '2'), ('--keep-monthly', '3'))
+
+
+def test_make_prune_flags_returns_flags_from_config_plus_default_prefix():
+    retention_config = OrderedDict((('keep_daily', 1), ('keep_weekly', 2), ('keep_monthly', 3)))
+
+    result = module._make_prune_flags(retention_config)
+
+    assert tuple(result) == BASE_PRUNE_FLAGS + (('--prefix', '{hostname}-'),)
+
+
+def test_make_prune_flags_accepts_prefix_with_placeholders():
+    retention_config = OrderedDict((('keep_daily', 1), ('prefix', 'Documents_{hostname}-{now}')))
+
+    result = module._make_prune_flags(retention_config)
+
+    expected = (('--keep-daily', '1'), ('--prefix', 'Documents_{hostname}-{now}'))
+
+    assert tuple(result) == expected
+
+
+PRUNE_COMMAND = (
+    'borg',
+    'prune',
+    'repo',
+    '--keep-daily',
+    '1',
+    '--keep-weekly',
+    '2',
+    '--keep-monthly',
+    '3',
+)
+
+
+def test_prune_archives_calls_borg_with_parameters():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS
+    )
+    insert_subprocess_mock(PRUNE_COMMAND)
+
+    module.prune_archives(
+        dry_run=False, repository='repo', storage_config={}, retention_config=retention_config
+    )
+
+
+def test_prune_archives_with_log_info_calls_borg_with_info_parameter():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--stats', '--info'))
+    insert_logging_mock(logging.INFO)
+
+    module.prune_archives(
+        repository='repo', storage_config={}, dry_run=False, retention_config=retention_config
+    )
+
+
+def test_prune_archives_with_log_debug_calls_borg_with_debug_parameter():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--stats', '--debug', '--list', '--show-rc'))
+    insert_logging_mock(logging.DEBUG)
+
+    module.prune_archives(
+        repository='repo', storage_config={}, dry_run=False, retention_config=retention_config
+    )
+
+
+def test_prune_archives_with_dry_run_calls_borg_with_dry_run_parameter():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--dry-run',))
+
+    module.prune_archives(
+        repository='repo', storage_config={}, dry_run=True, retention_config=retention_config
+    )
+
+
+def test_prune_archives_with_local_path_calls_borg_via_local_path():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS
+    )
+    insert_subprocess_mock(('borg1',) + PRUNE_COMMAND[1:])
+
+    module.prune_archives(
+        dry_run=False,
+        repository='repo',
+        storage_config={},
+        retention_config=retention_config,
+        local_path='borg1',
+    )
+
+
+def test_prune_archives_with_remote_path_calls_borg_with_remote_path_parameters():
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--remote-path', 'borg1'))
+
+    module.prune_archives(
+        dry_run=False,
+        repository='repo',
+        storage_config={},
+        retention_config=retention_config,
+        remote_path='borg1',
+    )
+
+
+def test_prune_archives_with_umask_calls_borg_with_umask_parameters():
+    storage_config = {'umask': '077'}
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--umask', '077'))
+
+    module.prune_archives(
+        dry_run=False,
+        repository='repo',
+        storage_config=storage_config,
+        retention_config=retention_config,
+    )
+
+
+def test_prune_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
+    storage_config = {'lock_wait': 5}
+    retention_config = flexmock()
+    flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
+        BASE_PRUNE_FLAGS
+    )
+    insert_subprocess_mock(PRUNE_COMMAND + ('--lock-wait', '5'))
+
+    module.prune_archives(
+        dry_run=False,
+        repository='repo',
+        storage_config=storage_config,
+        retention_config=retention_config,
+    )
@@ -0,0 +1,47 @@
+import json
+import sys
+
+from flexmock import flexmock
+
+from borgmatic.commands import borgmatic
+
+
+def test_run_commands_handles_multiple_json_outputs_in_array():
+    (
+        flexmock(borgmatic)
+        .should_receive('_run_commands_on_repository')
+        .times(3)
+        .replace_with(
+            lambda args, consistency, json_results, local_path, location, remote_path, retention, storage, unexpanded_repository: json_results.append(
+                {"whatever": unexpanded_repository}
+            )
+        )
+    )
+
+    (
+        flexmock(sys.stdout)
+        .should_call("write")
+        .with_args(
+            json.dumps(
+                json.loads(
+                    '''
+                        [
+                            {"whatever": "fake_repo1"},
+                            {"whatever": "fake_repo2"},
+                            {"whatever": "fake_repo3"}
+                        ]
+                    '''
+                )
+            )
+        )
+    )
+
+    borgmatic._run_commands(
+        args=flexmock(json=True),
+        consistency=None,
+        local_path=None,
+        location={'repositories': ['fake_repo1', 'fake_repo2', 'fake_repo3']},
+        remote_path=None,
+        retention=None,
+        storage=None,
+    )
@@ -0,0 +1,23 @@
+from borgmatic.config import checks as module
+
+
+def test_repository_enabled_for_checks_defaults_to_enabled_for_all_repositories():
+    enabled = module.repository_enabled_for_checks('repo.borg', consistency={})
+
+    assert enabled
+
+
+def test_repository_enabled_for_checks_is_enabled_for_specified_repositories():
+    enabled = module.repository_enabled_for_checks(
+        'repo.borg', consistency={'check_repositories': ['repo.borg', 'other.borg']}
+    )
+
+    assert enabled
+
+
+def test_repository_enabled_for_checks_is_disabled_for_other_repositories():
+    enabled = module.repository_enabled_for_checks(
+        'repo.borg', consistency={'check_repositories': ['other.borg']}
+    )
+
+    assert not enabled
@@ -3,6 +3,22 @@ from flexmock import flexmock
 from borgmatic.config import collect as module


+def test_get_default_config_paths_includes_absolute_user_config_path():
+    flexmock(module.os, environ={'XDG_CONFIG_HOME': None, 'HOME': '/home/user'})
+
+    config_paths = module.get_default_config_paths()
+
+    assert '/home/user/.config/borgmatic/config.yaml' in config_paths
+
+
+def test_get_default_config_paths_prefers_xdg_config_home_for_user_config_path():
+    flexmock(module.os, environ={'XDG_CONFIG_HOME': '/home/user/.etc', 'HOME': '/home/user'})
+
+    config_paths = module.get_default_config_paths()
+
+    assert '/home/user/.etc/borgmatic/config.yaml' in config_paths
+
+
 def test_collect_config_filenames_collects_given_files():
    config_paths = ('config.yaml', 'other.yaml')
    flexmock(module.os.path).should_receive('isdir').and_return(False)
@@ -32,6 +48,21 @@ def test_collect_config_filenames_collects_files_from_given_directories_and_igno
    )


+def test_collect_config_filenames_collects_files_from_given_directories_and_ignores_non_yaml_filenames():
+    config_paths = ('/etc/borgmatic.d',)
+    mock_path = flexmock(module.os.path)
+    mock_path.should_receive('exists').and_return(True)
+    mock_path.should_receive('isdir').with_args('/etc/borgmatic.d').and_return(True)
+    mock_path.should_receive('isdir').with_args('/etc/borgmatic.d/foo.yaml').and_return(False)
+    mock_path.should_receive('isdir').with_args('/etc/borgmatic.d/bar.yaml~').and_return(False)
+    mock_path.should_receive('isdir').with_args('/etc/borgmatic.d/baz.txt').and_return(False)
+    flexmock(module.os).should_receive('listdir').and_return(['foo.yaml', 'bar.yaml~', 'baz.txt'])
+
+    config_filenames = tuple(module.collect_config_filenames(config_paths))
+
+    assert config_filenames == ('/etc/borgmatic.d/foo.yaml',)
+
+
 def test_collect_config_filenames_skips_etc_borgmatic_config_dot_yaml_if_it_does_not_exist():
    config_paths = ('config.yaml', '/etc/borgmatic/config.yaml')
    mock_path = flexmock(module.os.path)
@@ -33,25 +33,31 @@ def test_convert_legacy_parsed_config_transforms_source_config_to_mapping():

    destination_config = module.convert_legacy_parsed_config(source_config, source_excludes, schema)

-    assert destination_config == OrderedDict([
-        (
-            'location',
-            OrderedDict([
-                ('source_directories', ['/home']),
-                ('repositories', ['hostname.borg']),
-                ('exclude_patterns', ['/var']),
-            ]),
-        ),
-        ('storage', OrderedDict([('encryption_passphrase', 'supersecret')])),
-        ('retention', OrderedDict([('keep_daily', 7)])),
-        ('consistency', OrderedDict([('checks', ['repository'])])),
-    ])
+    assert destination_config == OrderedDict(
+        [
+            (
+                'location',
+                OrderedDict(
+                    [
+                        ('source_directories', ['/home']),
+                        ('repositories', ['hostname.borg']),
+                        ('exclude_patterns', ['/var']),
+                    ]
+                ),
+            ),
+            ('storage', OrderedDict([('encryption_passphrase', 'supersecret')])),
+            ('retention', OrderedDict([('keep_daily', 7)])),
+            ('consistency', OrderedDict([('checks', ['repository'])])),
+        ]
+    )


 def test_convert_legacy_parsed_config_splits_space_separated_values():
    flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
    source_config = Parsed_config(
-        location=OrderedDict([('source_directories', '/home /etc'), ('repository', 'hostname.borg')]),
+        location=OrderedDict(
+            [('source_directories', '/home /etc'), ('repository', 'hostname.borg')]
+        ),
        storage=OrderedDict(),
        retention=OrderedDict(),
        consistency=OrderedDict([('checks', 'repository archives')]),
@@ -59,21 +65,25 @@ def test_convert_legacy_parsed_config_splits_space_separated_values():
    source_excludes = ['/var']
    schema = {'map': defaultdict(lambda: {'map': {}})}

-    destination_config = module.convert_legacy_parsed_config(source_config, source_excludes, schema) 
+    destination_config = module.convert_legacy_parsed_config(source_config, source_excludes, schema)

-    assert destination_config == OrderedDict([
-        (
-            'location',
-            OrderedDict([
-                ('source_directories', ['/home', '/etc']),
-                ('repositories', ['hostname.borg']),
-                ('exclude_patterns', ['/var']),
-            ]),
-        ),
-        ('storage', OrderedDict()),
-        ('retention', OrderedDict()),
-        ('consistency', OrderedDict([('checks', ['repository', 'archives'])])),
-    ])
+    assert destination_config == OrderedDict(
+        [
+            (
+                'location',
+                OrderedDict(
+                    [
+                        ('source_directories', ['/home', '/etc']),
+                        ('repositories', ['hostname.borg']),
+                        ('exclude_patterns', ['/var']),
+                    ]
+                ),
+            ),
+            ('storage', OrderedDict()),
+            ('retention', OrderedDict()),
+            ('consistency', OrderedDict([('checks', ['repository', 'archives'])])),
+        ]
+    )


 def test_guard_configuration_upgraded_raises_when_only_source_config_present():
@@ -0,0 +1,37 @@
+from collections import OrderedDict
+
+from flexmock import flexmock
+
+from borgmatic.config import generate as module
+
+
+def test_schema_to_sample_configuration_generates_config_with_examples():
+    flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
+    flexmock(module).should_receive('add_comments_to_configuration')
+    schema = {
+        'map': OrderedDict(
+            [
+                ('section1', {'map': {'field1': OrderedDict([('example', 'Example 1')])}}),
+                (
+                    'section2',
+                    {
+                        'map': OrderedDict(
+                            [
+                                ('field2', {'example': 'Example 2'}),
+                                ('field3', {'example': 'Example 3'}),
+                            ]
+                        )
+                    },
+                ),
+            ]
+        )
+    }
+
+    config = module._schema_to_sample_configuration(schema)
+
+    assert config == OrderedDict(
+        [
+            ('section1', OrderedDict([('field1', 'Example 1')])),
+            ('section2', OrderedDict([('field2', 'Example 2'), ('field3', 'Example 3')])),
+        ]
+    )
@@ -25,17 +25,9 @@ def test_validate_configuration_format_with_valid_config_should_not_raise():
    parser.should_receive('options').with_args('other').and_return(('such',))
    config_format = (
        module.Section_format(
-            'section',
-            options=(
-                module.Config_option('stuff', str, required=True),
-            ),
-        ),
-        module.Section_format(
-            'other',
-            options=(
-                module.Config_option('such', str, required=True),
-            ),
+            'section', options=(module.Config_option('stuff', str, required=True),)
        ),
+        module.Section_format('other', options=(module.Config_option('such', str, required=True),)),
    )

    module.validate_configuration_format(parser, config_format)
@@ -46,10 +38,7 @@ def test_validate_configuration_format_with_missing_required_section_should_rais
    parser.should_receive('sections').and_return(('section',))
    config_format = (
        module.Section_format(
-            'section',
-            options=(
-                module.Config_option('stuff', str, required=True),
-            ),
+            'section', options=(module.Config_option('stuff', str, required=True),)
        ),
        # At least one option in this section is required, so the section is required.
        module.Section_format(
@@ -71,10 +60,7 @@ def test_validate_configuration_format_with_missing_optional_section_should_not_
    parser.should_receive('options').with_args('section').and_return(('stuff',))
    config_format = (
        module.Section_format(
-            'section',
-            options=(
-                module.Config_option('stuff', str, required=True),
-            ),
+            'section', options=(module.Config_option('stuff', str, required=True),)
        ),
        # No options in the section are required, so the section is optional.
        module.Section_format(
@@ -92,9 +78,7 @@ def test_validate_configuration_format_with_missing_optional_section_should_not_
 def test_validate_configuration_format_with_unknown_section_should_raise():
    parser = flexmock()
    parser.should_receive('sections').and_return(('section', 'extra'))
-    config_format = (
-        module.Section_format('section', options=()),
-    )
+    config_format = (module.Section_format('section', options=()),)

    with pytest.raises(ValueError):
        module.validate_configuration_format(parser, config_format)
@@ -141,8 +125,7 @@ def test_validate_configuration_format_with_extra_option_should_raise():
    parser.should_receive('options').with_args('section').and_return(('option', 'extra'))
    config_format = (
        module.Section_format(
-            'section',
-            options=(module.Config_option('option', str, required=True),),
+            'section', options=(module.Config_option('option', str, required=True),)
        ),
    )

@@ -168,12 +151,7 @@ def test_parse_section_options_should_return_section_options():

    config = module.parse_section_options(parser, section_format)

-    assert config == OrderedDict(
-        (
-            ('foo', 'value'),
-            ('bar', 1),
-        )
-    )
+    assert config == OrderedDict((('foo', 'value'), ('bar', 1)))


 def test_parse_section_options_for_missing_section_should_return_empty_dict():
@@ -210,13 +188,13 @@ def test_parse_configuration_should_return_section_configs():
    config_format = (flexmock(name='items'), flexmock(name='things'))
    mock_module = flexmock(module)
    mock_module.should_receive('validate_configuration_format').with_args(
-        parser, config_format,
+        parser, config_format
    ).once()
    mock_section_configs = (flexmock(), flexmock())

    for section_format, section_config in zip(config_format, mock_section_configs):
        mock_module.should_receive('parse_section_options').with_args(
-            parser, section_format,
+            parser, section_format
        ).and_return(section_config).once()

    parsed_config = module.parse_configuration('filename', config_format)
@@ -0,0 +1,92 @@
+import pytest
+from flexmock import flexmock
+
+from borgmatic.config import validate as module
+
+
+def test_validation_error_str_contains_error_messages_and_config_filename():
+    error = module.Validation_error('config.yaml', ('oops', 'uh oh'))
+
+    result = str(error)
+
+    assert 'config.yaml' in result
+    assert 'oops' in result
+    assert 'uh oh' in result
+
+
+def test_apply_logical_validation_raises_if_archive_name_format_present_without_prefix():
+    with pytest.raises(module.Validation_error):
+        module.apply_logical_validation(
+            'config.yaml',
+            {
+                'storage': {'archive_name_format': '{hostname}-{now}'},
+                'retention': {'keep_daily': 7},
+            },
+        )
+
+
+def test_apply_logical_validation_raises_if_archive_name_format_present_without_retention_prefix():
+    with pytest.raises(module.Validation_error):
+        module.apply_logical_validation(
+            'config.yaml',
+            {
+                'storage': {'archive_name_format': '{hostname}-{now}'},
+                'retention': {'keep_daily': 7},
+                'consistency': {'prefix': '{hostname}-'},
+            },
+        )
+
+
+def test_apply_logical_validation_warns_if_archive_name_format_present_without_consistency_prefix():
+    logger = flexmock(module.logger)
+    logger.should_receive('warning').once()
+
+    module.apply_logical_validation(
+        'config.yaml',
+        {
+            'storage': {'archive_name_format': '{hostname}-{now}'},
+            'retention': {'prefix': '{hostname}-'},
+            'consistency': {},
+        },
+    )
+
+
+def test_apply_locical_validation_raises_if_unknown_repository_in_check_repositories():
+    with pytest.raises(module.Validation_error):
+        module.apply_logical_validation(
+            'config.yaml',
+            {
+                'location': {'repositories': ['repo.borg', 'other.borg']},
+                'retention': {'keep_secondly': 1000},
+                'consistency': {'check_repositories': ['repo.borg', 'unknown.borg']},
+            },
+        )
+
+
+def test_apply_locical_validation_does_not_raise_if_known_repository_in_check_repositories():
+    module.apply_logical_validation(
+        'config.yaml',
+        {
+            'location': {'repositories': ['repo.borg', 'other.borg']},
+            'retention': {'keep_secondly': 1000},
+            'consistency': {'check_repositories': ['repo.borg']},
+        },
+    )
+
+
+def test_apply_logical_validation_does_not_raise_or_warn_if_archive_name_format_and_prefix_present():
+    logger = flexmock(module.logger)
+    logger.should_receive('warning').never()
+
+    module.apply_logical_validation(
+        'config.yaml',
+        {
+            'storage': {'archive_name_format': '{hostname}-{now}'},
+            'retention': {'prefix': '{hostname}-'},
+            'consistency': {'prefix': '{hostname}-'},
+        },
+    )
+
+
+def test_apply_logical_validation_does_not_raise_otherwise():
+    module.apply_logical_validation('config.yaml', {'retention': {'keep_secondly': 1000}})
@@ -0,0 +1,21 @@
+import logging
+
+from flexmock import flexmock
+
+from borgmatic import verbosity as module
+
+
+def insert_logging_mock(log_level):
+    """ Mocks the isEnabledFor from python logging. """
+    logging = flexmock(module.logging.Logger)
+    logging.should_receive('isEnabledFor').replace_with(lambda lvl: lvl >= log_level)
+    logging.should_receive('getEffectiveLevel').replace_with(lambda: log_level)
+
+
+def test_verbosity_to_log_level_maps_known_verbosity_to_log_level():
+    assert module.verbosity_to_log_level(module.VERBOSITY_SOME) == logging.INFO
+    assert module.verbosity_to_log_level(module.VERBOSITY_LOTS) == logging.DEBUG
+
+
+def test_verbosity_to_log_level_maps_unknown_verbosity_to_warning_level():
+    assert module.verbosity_to_log_level('my pants') == logging.WARNING
@@ -1,8 +1,27 @@
 [tox]
-envlist=py34
-skipsdist=True
+envlist = py3
+skipsdist = True

 [testenv]
-usedevelop=True
-deps=-rtest_requirements.txt
-commands = py.test --cov-report term-missing:skip-covered --cov=borgmatic borgmatic []
+usedevelop = True
+deps = -rtest_requirements.txt
+whitelist_externals = sh
+commands =
+    py.test --cov-report term-missing:skip-covered --cov=borgmatic --ignore=tests/end-to-end \
+        tests []
+    sh scripts/black
+    flake8 .
+
+[testenv:black]
+basepython = python3.7
+commands =
+    black --skip-string-normalization --line-length 100 .
+
+[testenv:end-to-end]
+deps = -rtest_requirements.txt
+commands =
+    py.test tests/end-to-end []
+
+[flake8]
+ignore = E501,W503
+exclude = *.*/*
Author	SHA1	Message	Date
Dan Helfman	5c0b17ef39	Support for Borg --chunker-params create option via "chunker_params" in borgmatic's storage section (#105 ).	2018-10-27 15:57:28 -07:00
Dan Helfman	1697d8aaef	Silence curl when posting release description to Gitea.	2018-10-15 22:32:13 -07:00
Dan Helfman	fef441a8ff	More concessions for Python 3.5 compatibility.	2018-10-15 09:37:26 -07:00
Dan Helfman	c1ddc4268b	We can't have nice things.	2018-10-15 09:30:04 -07:00
Dan Helfman	e323290e61	Switch from bash to sh for black wrapper.	2018-10-15 09:25:57 -07:00
Dan Helfman	1ab44d4201	Wrap black with script that skips it if Python version < 3.6.	2018-10-15 09:20:35 -07:00
Dan Helfman	71b1c3dfb0	Make automated tests support running in Python 3.5.	2018-10-15 09:04:29 -07:00
Dan Helfman	695930a607	Fix for syntax error that occurred in Python 3.5 and below (#102 ).	2018-10-15 08:47:15 -07:00
Dan Helfman	eb2a4ff1f0	Add Python 3.5 to continuous integration.	2018-10-15 08:17:34 -07:00
Dan Helfman	531d5c80c0	Fix quoting and escaping in release script.	2018-10-14 12:14:29 -07:00
Dan Helfman	067ed27689	Rev to 1.2.8.	2018-10-14 11:45:34 -07:00
Dan Helfman	fa38de2de7	Enable consistency checks for only certain repositories via "check_repositories" (#73 ).	2018-10-13 20:34:51 -07:00
Dan Helfman	e4d1b49c39	Switch some functions with many arguments to kwargs only.	2018-10-13 15:19:16 -07:00
Dan Helfman	af7caec509	Mention minimum Borg version to install in README.	2018-10-13 13:35:42 -07:00
Dan Helfman	90c1f899fc	Use newer Alpine (with newer version of Borg) in matrix builds.	2018-10-13 13:35:18 -07:00
Dan Helfman	a0691ae4cd	Run continuous integration tests on a matrix of Python and Borg versions.	2018-10-13 13:09:12 -07:00
Dan Helfman	2f20e6f808	Include link to issue tracker within various command output.	2018-10-07 22:29:56 -07:00
Dan Helfman	7a4636ae0f	Remove curl --verbose in release script.	2018-10-06 22:35:00 -07:00
Dan Helfman	53435dcc3e	Post release changelogs to projects.evoworx.org.	2018-10-06 22:24:46 -07:00
Dan Helfman	4d01278037	Update release file to post changelogs to GitHub release descriptions.	2018-10-06 15:18:21 -07:00
Dan Helfman	2299e5d41e	Additional dependency version pins in test requirements.	2018-10-06 14:17:47 -07:00
Dan Helfman	d16f5d5df3	Add backticks to path literal in README.	2018-10-06 13:23:54 -07:00
Dan Helfman	da8e9638f4	Support for Borg --keep-secondly prune option (#98 ).	2018-10-04 21:54:23 -07:00
Dan Helfman	900ea80a42	Hack to uncomment all options in config file used for finding unsupported Borg options.	2018-10-04 21:45:31 -07:00
Dan Helfman	4b92d0f685	Remove unneeded Dockerfile for end-to-end tests.	2018-10-03 22:44:23 -07:00
Dan Helfman	3ce5533103	Make end-to-end test clean up after itself, and drop unnecessary use of Docker for it.	2018-10-03 22:36:25 -07:00
Dan Helfman	4a1ee8c911	Pull new base Docker images during CI.	2018-10-03 19:35:42 -07:00
Dan Helfman	3f22a99412	Rev pykwalify.	2018-10-03 08:59:08 -07:00
Dan Helfman	caf95cc913	Rebuild.	2018-09-30 22:58:23 -07:00
Dan Helfman	fd3130b4d9	Install tox before using it.	2018-09-30 22:47:07 -07:00
Dan Helfman	65bb5a49e2	CI?	2018-09-30 22:43:49 -07:00
Dan Helfman	4bcc517326	Attempted Drone CI configuration.	2018-09-30 22:09:53 -07:00
Dan Helfman	0b164973e0	Add an end-to-end automated test that actually integrates with Borg.	2018-09-30 17:30:04 -07:00
Dan Helfman	a125df991b	Move tests to the root of the repository, in keeping with more common convention.	2018-09-30 13:57:20 -07:00
Dan Helfman	f9a9b42c58	A little introductory text for the screencast.	2018-09-30 11:11:07 -07:00
Dan Helfman	56ad1d164a	Use Flake8 code checker as part of running automated tests.	2018-09-29 23:15:18 -07:00
Dan Helfman	3cce18919c	Switch Black link to documentation.	2018-09-29 22:46:34 -07:00
Dan Helfman	76d6a69f5a	Use Black code formatter as part of running automated tests.	2018-09-29 22:45:00 -07:00
Dan Helfman	3db17277b4	Replace broken screencast thumbnail with embedded player.	2018-09-29 21:38:38 -07:00
Dan Helfman	ece49eb500	Update screencast.	2018-09-29 18:56:39 -07:00
Dan Helfman	746428ed44	Fix generated configuration to also include a "keep_daily" value so pruning works out of the box.	2018-09-29 15:44:37 -07:00
Dan Helfman	984702b3b2	Fix various warnings.	2018-09-29 15:06:57 -07:00
Dan Helfman	1bc71e1c5d	Upgrade test requirements.	2018-09-29 15:04:42 -07:00
Dan Helfman	47efa88c9d	In generate-borgmatic-config, comment out all optional config (#57 ).	2018-09-29 15:03:11 -07:00
Dan Helfman	3821636b77	Bump version.	2018-09-27 08:13:23 -07:00
Dan Helfman	596f6f9dac	Update help/README about --create --json.	2018-09-27 08:12:54 -07:00
Dan HelfmanandGitHub	7ecdaea83a	Fix check_archives does not take json parameter.	2018-09-27 08:09:23 -07:00
Nils HesseandGitHub	98cb2644db	check_archives does not take json parameter	2018-09-27 12:21:14 +02:00
Dan Helfman	31db6faa19	Set to release version.	2018-09-26 21:32:28 -07:00
Dan Helfman	872d8b695a	Flesh out NEWS line item a bit.	2018-09-24 21:37:45 -07:00
Dan HelfmanandGitea	6db3e1dda5	Merge branch 'master' of floli/borgmatic into master	2018-09-25 04:36:09 +00:00
Florian Lindner	9aaf78b9dd	Add --json option for --create command line. Closes #94.	2018-09-24 21:53:09 +02:00
Dan Helfman	5d8ac158ce	Merge ssh://projects.torsion.org:3022/witten/borgmatic	2018-09-17 22:34:08 -07:00
Dan Helfman	d32a53d58f	Mention log level fix in NEWS.	2018-09-17 22:33:34 -07:00
floliandGitea	a836ec944f	Limit argument range for --verbose, make default log level more explicit. (#93 )	2018-09-18 05:31:27 +00:00
Dan Helfman	e7b128e735	--read-special is now supported.	2018-09-09 11:21:06 -07:00
Dan Helfman	ff3cb1d80f	Attach #64 to logging rewrite in NEWS.	2018-09-09 11:18:26 -07:00
Dan Helfman	c5ff08ee25	Remove now-gone verbosity parameter from test.	2018-09-09 11:14:33 -07:00
Dan Helfman	856db29180	Mention --read-special in NEWS.	2018-09-09 10:42:06 -07:00
Steve KerrisonandDan Helfman	20e09b4ea8	Support for Borg create --read-special via "read_special" option (#25 ).	2018-09-09 10:39:56 -07:00
Dan Helfman	1dd0682661	Merge branch 'master' of ssh://projects.torsion.org:3022/witten/borgmatic	2018-09-08 13:54:18 -07:00
floliandGitea	7252b8d614	Rework logging/verbosity system (#90 ) Looks great, merged! Thanks again for all your hard work here.	2018-09-08 20:53:37 +00:00
grerrgandDan Helfman	e5870a169b	Add example for cron in Alpine Linux (#24 )	2018-09-05 21:58:30 -07:00
Dan Helfman	94795a3560	Link to Borg home page instead of docs.	2018-09-02 22:06:57 -07:00
Dan	7705debab0	Switching back to table-like 11ty front matter. It looks less bad than JSON.	2018-09-01 22:45:13 -07:00
Dan	f87df0527f	Adding JSON front matter for 11ty.	2018-09-01 22:38:17 -07:00
Dan Helfman	e4512a40e0	Removing 11ty front matter out of README since it renders as a table on GitHub.	2018-09-01 22:11:38 -07:00
Dan Helfman	1d4a9510b8	Upgrade pytest.	2018-09-01 20:29:05 -07:00
Dan Helfman	2648f07e7a	Add missing syntax highlighting.	2018-08-29 23:01:11 -07:00
Dan Helfman	459bf1fcf6	Document --list and --info flags.	2018-08-29 22:57:32 -07:00
Dan	3930e63320	Merge branch 'master' of ssh://projects.torsion.org:3022/witten/borgmatic	2018-08-29 22:44:45 -07:00
Dan	acecb1e397	README metadata changes to support 11ty static site generator.	2018-08-29 22:44:12 -07:00
Dan Helfman	9b48eb5a61	Clarify that --json can be used with --info command-line flag.	2018-08-19 12:57:52 -07:00
Dan Helfman	7d40a448cb	Pass --show-rc option to Borg when at highest verbosity level (#89 ).	2018-08-19 12:44:40 -07:00
Dan Helfman	da7aed3814	Support for Borg create --checkpoint-interval (#87 ).	2018-08-19 11:41:49 -07:00
Dan Helfman	c7f4200417	Somewhat more robust mechanism to find unsupported Borg arguments.	2018-08-19 11:24:48 -07:00
Dan Helfman	5e2a5494af	Fix declared pykwalify compatibility version range in setup.py (#88 ).	2018-08-18 14:07:18 -07:00
Dan Helfman	7b77fd2510	Fix compatibility issue between pykwalify and ruamel.yaml 0.15.52 (#85 ).	2018-08-11 13:59:27 -07:00
Dan Helfman	ece5608677	Rev for release.	2018-07-28 22:27:39 -07:00
Dan Helfman	4644f613b2	Fix typo in README.	2018-07-28 22:24:24 -07:00
Dan Helfman	3afa5ac76d	Document hooks (#81 ).	2018-07-28 22:22:25 -07:00
Dan Helfman	27f8a1df04	Switch to non-raw link to sample cron job.	2018-07-28 20:29:55 -07:00
Dan Helfman	8e5b0bbf17	Remove errant ctrl-F character from docs.	2018-07-28 20:27:18 -07:00
Dan Helfman	282e9565c9	Mentioning new --info --json option in NEWS.	2018-07-28 20:24:19 -07:00
Thomas LÉVEILandGitea	b714ffd48b	add support for `--info --json` (#83 )	2018-07-29 03:17:45 +00:00
Dan Helfman	9968a15ef8	Clarifying code style for multiline constructs.	2018-07-28 15:21:19 -07:00
Dan Helfman	d93da55ce9	Add code style guidelines to the documention, and reformat some code accordingly.	2018-07-28 15:02:17 -07:00
Thomas LÉVEILandGitea	789bcd402a	add support for `--list --json` (#74 )	2018-07-28 21:21:38 +00:00
Dan HelfmanandGitea	cf6ab60d2e	Use XDG_CONFIG_HOME for user configuration directory, if set. (Thanks to floli.) (#71 ) Thanks! This will go out in the next release.	2018-07-25 01:34:05 +00:00
Dan Helfman	64364b20ff	Skip non-"*.yaml" config filenames in /etc/borgmatic.d/ so as not to parse backup files, editor swap files, etc. (#77 )	2018-07-22 12:08:49 -07:00
Dan Helfman	d29c7956bc	Upgrade ruamel.yaml compatibility version range and fix support for Python 3.7 (#38 , #76 ).	2018-07-22 11:25:06 -07:00
Dan Helfman	e5ef485d6b	Merge branch 'master' of ssh://projects.torsion.org:3022/witten/borgmatic	2018-07-01 14:54:15 -07:00
Dan Helfman	fc8046edc4	Adding NEWS item about skipping before/after backup hooks.	2018-07-01 14:51:57 -07:00
Dan HelfmanandGitea	4538017206	Merge branch 'fix-72-hooks-are-executed-when-list-or-info' of thomasleveil/borgmatic into master Thanks for fixing this! I agree about more specific hooks if and when `--check` or `--prune`-specific hook use cases are needed. I think what you've done here is fine until then.	2018-07-01 21:47:39 +00:00
Thomas LEVEIL	d664b6d253	only run hooks when creating an archive fix #72	2018-07-01 21:09:45 +02:00
Dan Helfman	f42aa0a6f2	Revving version for development.	2018-06-17 15:26:53 -07:00
Dan Helfman	9d4ba66f6e	Revving version for release.	2018-06-17 15:14:45 -07:00
Dan Helfman	cf846ab8ac	Support for Borg prune --umask option (#69 ).	2018-06-17 15:12:43 -07:00
Dan Helfman	219e287c6c	Document how to develop on and contribute to borgmatic.	2018-06-17 14:55:57 -07:00
Dan Helfman	dede8f9d4b	News for: ~/.config/borgmatic/config.yaml.	2018-06-17 14:30:47 -07:00
Dan HelfmanandGitea	7a1e3f5639	Merge branch 'add_user_config' of floli/borgmatic into master	2018-06-17 21:26:36 +00:00
Florian Lindner	9bd77292ff	Add default path for user configuration	2018-06-10 15:03:23 +02:00
Dan Helfman	f1a143de5b	Adding list and info Borg sub-commands to find-unsupported-borg-options script.	2018-05-26 20:53:03 -07:00
Dan Helfman	765e343c71	Support for Borg --info via borgmatic command-line (#61 ).	2018-05-26 16:19:05 -07:00
Dan Helfman	af4b91a048	Support for Borg --list option via borgmatic command-line to list all archives (#61 ).	2018-05-26 16:09:08 -07:00
Dan Helfman	cc9044487b	Support for Borg --nobsdflags option to skip recording bsdflags (e.g. NODUMP, IMMUTABLE) in archive (#63 ).	2018-05-26 15:09:23 -07:00
Dan Helfman	11c30001c3	Add "Persistent" flag to systemd timer example. (#60 )	2018-05-20 22:20:21 -07:00
Dan HelfmanandGitea	ac9161035a	Merge branch 'master' of floli/borgmatic into master Thanks for taking the time to add this! Makes sense.	2018-05-21 05:18:13 +00:00
Dan Helfman	007ec0644c	Ignore "check_last" and consistency "prefix" when "archives" not in consistency checks. (#59 )	2018-05-20 22:11:40 -07:00
Dan Helfman	1db808fb3d	Link to OpenBSD port of borgmatic.	2018-05-19 16:16:54 -07:00
Dan Helfman	76656275c3	Update README to mention other ways of installing borgmatic. (#62 )	2018-05-19 16:06:54 -07:00
Florian Lindner	64bdbc4bf0	Add Persistent, so that the timer is triggered if missed last time.	2018-05-17 21:47:58 +02:00
Dan Helfman	61033bb4e5	Update tox.ini to only assume Python 3.x instead of Python 3.4 specifically.	2018-04-09 20:34:59 -07:00
Dan Helfman	e608b7924a	Adding note about executable location.	2018-04-08 12:06:15 -07:00
Dan Helfman	f7f852a28b	Fix tests broken by addition of check --prefix default.	2018-03-03 22:36:51 -08:00
Dan Helfman	9b9c4c4abb	Clarifying note in schema about adding prefix to consistency section.	2018-03-03 22:33:34 -08:00
Dan Helfman	1b59f5b190	Changing version in warning to correspond with next release version.	2018-03-03 22:30:30 -08:00
Dan Helfman	65ab230961	Noting new Borg check --prefix feature in release notes.	2018-03-03 22:21:48 -08:00
Nick WhyteandDan Helfman	c64d0100d5	Only check archives with matching prefix.	2018-03-03 22:17:39 -08:00
Dan Helfman	0112407250	Add introductory screencast link to documentation.	2018-02-19 17:44:20 -08:00
Dan Helfman	2d3f5fa05d	Support for Borg --lock-wait option for the maximum wait for a repository/cache lock (#56 ).	2018-02-19 15:51:04 -08:00
Dan Helfman	a87036ee46	Support for using tilde in exclude_patterns to reference home directory (#58 ).	2018-02-18 15:34:19 -08:00
Dan Helfman	a72f5ff69a	Tests for --dry-run + --verbosity fix.	2018-02-18 14:26:51 -08:00
newtonneandDan Helfman	bb99009191	Fix issue when using both --dry-run and -v options.	2018-02-18 14:18:25 -08:00
Dan Helfman	4c45d60529	Mentioning ssh_command for additional SSH configuration..	2018-02-18 13:39:05 -08:00
Dan Helfman	2211f959ae	AUTHORS addition for recent encryption pass command changes.	2018-01-18 21:13:20 -08:00
Dan Helfman	cc1d6f53a0	55: Fix for missing tags/releases from Gitea and GitHub project hosting.	2018-01-17 20:27:09 -08:00
Dan Helfman	389778c716	Adding BORG_PASSCOMMAND update to NEWS.	2018-01-16 21:05:53 -08:00
newtonneandDan Helfman	e55e9e8139	Add `encryption_passcommand` configuration option	2018-01-16 21:03:25 -08:00
Dan Helfman	ef76e87477	Bumping version for release.	2018-01-15 20:55:49 -08:00
Dan Helfman	62526038d6	47: Support for Borg --dry-run option via borgmatic command-line.	2018-01-15 20:55:27 -08:00
Dan Helfman	bf2f39623e	49: Rename incorrect --pattern-from option to correct --patterns-from.	2018-01-15 20:22:53 -08:00
Dan Helfman	28c890a52d	Bumping version for release.	2018-01-14 16:37:02 -08:00
Dan Helfman	cd189c4fe4	48: Add "local_path" to configuration for specifying an alternative Borg executable path.	2018-01-14 16:35:24 -08:00
Dan Helfman	b8f6bab12d	49: Support for Borg experimental --patterns-from and --patterns options for specifying mixed includes/excludes.	2018-01-14 15:52:19 -08:00
Dan Helfman	50b3240c4f	54: Fix for incorrect consistency check flags passed to Borg when all three checks in borgmatic config.	2018-01-14 14:09:20 -08:00
Dan Helfman	18fbc75e16	Revising history to account for off-by-one error when importing issue numbers into Gitea.	2018-01-04 21:49:43 -08:00
Dan Helfman	0881da4a82	New issue tracker.	2018-01-04 21:37:43 -08:00
Dan	fa210766a2	Update for release.	2018-01-02 20:36:52 -08:00