mirror of
https://github.com/borgmatic-collective/borgmatic.git
synced 2026-07-22 10:13:00 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c3e7425f4c | ||
|
|
cc9dbb1def | ||
|
|
2045edc11b | ||
|
|
1dcac44d6c | ||
|
|
300ead65d3 | ||
|
|
6a0219a7a4 | ||
|
|
80c69aac05 | ||
|
|
7417a3cd00 | ||
|
|
9ca80a54d8 | ||
|
|
5c0b17ef39 | ||
|
|
1697d8aaef | ||
|
|
fef441a8ff | ||
|
|
c1ddc4268b | ||
|
|
e323290e61 | ||
|
|
1ab44d4201 | ||
|
|
71b1c3dfb0 | ||
|
|
695930a607 | ||
|
|
eb2a4ff1f0 | ||
|
|
531d5c80c0 | ||
|
|
067ed27689 | ||
|
|
fa38de2de7 | ||
|
|
e4d1b49c39 | ||
|
|
af7caec509 | ||
|
|
90c1f899fc | ||
|
|
a0691ae4cd | ||
|
|
2f20e6f808 | ||
|
|
7a4636ae0f | ||
|
|
53435dcc3e | ||
|
|
4d01278037 | ||
|
|
2299e5d41e | ||
|
|
d16f5d5df3 | ||
|
|
da8e9638f4 | ||
|
|
900ea80a42 | ||
|
|
4b92d0f685 | ||
|
|
3ce5533103 | ||
|
|
4a1ee8c911 | ||
|
|
3f22a99412 | ||
|
|
caf95cc913 | ||
|
|
fd3130b4d9 | ||
|
|
65bb5a49e2 | ||
|
|
4bcc517326 | ||
|
|
0b164973e0 | ||
|
|
a125df991b | ||
|
|
f9a9b42c58 | ||
|
|
56ad1d164a | ||
|
|
3cce18919c | ||
|
|
76d6a69f5a | ||
|
|
3db17277b4 | ||
|
|
ece49eb500 | ||
|
|
746428ed44 | ||
|
|
984702b3b2 | ||
|
|
1bc71e1c5d | ||
|
|
47efa88c9d |
+18
@@ -0,0 +1,18 @@
|
||||
pipeline:
|
||||
build:
|
||||
image: python:${PYTHON_VERSION}-alpine${ALPINE_VERSION}
|
||||
pull: true
|
||||
commands:
|
||||
- pip install tox
|
||||
- tox
|
||||
- apk add --no-cache borgbackup
|
||||
- tox -e end-to-end
|
||||
|
||||
matrix:
|
||||
ALPINE_VERSION:
|
||||
- 3.7
|
||||
- 3.8
|
||||
PYTHON_VERSION:
|
||||
- 3.5
|
||||
- 3.6
|
||||
- 3.7
|
||||
@@ -1,3 +1,42 @@
|
||||
1.2.12
|
||||
* #110: Support for Borg repository initialization via borgmatic --init command-line flag.
|
||||
* #111: Update Borg create --filter values so a dry run lists files to back up.
|
||||
* #113: Update README with link to a new/forked Docker image.
|
||||
* Prevent deprecated --excludes command-line option from being used.
|
||||
* Refactor README a bit to flow better for first-time users.
|
||||
* Update README with a few additional borgmatic packages (Debian and Ubuntu).
|
||||
|
||||
1.2.11
|
||||
* #108: Support for Borg create --progress via borgmatic command-line flag.
|
||||
|
||||
1.2.10
|
||||
* #105: Support for Borg --chunker-params create option via "chunker_params" in borgmatic's storage
|
||||
section.
|
||||
|
||||
1.2.9
|
||||
* #102: Fix for syntax error that occurred in Python 3.5 and below.
|
||||
* Make automated tests support running in Python 3.5.
|
||||
|
||||
1.2.8
|
||||
* #73: Enable consistency checks for only certain repositories via "check_repositories" option in
|
||||
borgmatic's consistency configuration. Handy for large repositories that take forever to check.
|
||||
* Include link to issue tracker within various command output.
|
||||
* Run continuous integration tests on a matrix of Python and Borg versions.
|
||||
|
||||
1.2.7
|
||||
* #98: Support for Borg --keep-secondly prune option.
|
||||
* Use Black code formatter and Flake8 code checker as part of running automated tests.
|
||||
* Add an end-to-end automated test that actually integrates with Borg.
|
||||
* Set up continuous integration for borgmatic automated tests on projects.evoworx.org.
|
||||
|
||||
1.2.6
|
||||
* Fix generated configuration to also include a "keep_daily" value so pruning works out of the
|
||||
box.
|
||||
|
||||
1.2.5
|
||||
* #57: When generating sample configuration with generate-borgmatic-config, comment out all
|
||||
optional configuration so as to streamline the initial configuration process.
|
||||
|
||||
1.2.4
|
||||
* Fix for archive checking traceback due to parameter mismatch.
|
||||
|
||||
|
||||
@@ -48,31 +48,19 @@ borgmatic is hosted at <https://torsion.org/borgmatic> with [source code
|
||||
available](https://projects.torsion.org/witten/borgmatic). It's also mirrored
|
||||
on [GitHub](https://github.com/witten/borgmatic) for convenience.
|
||||
|
||||
<a href="https://asciinema.org/a/164143" target="_blank"><img src="https://asciinema.org/a/164143.png" width="100%" /></a>
|
||||
Want to see borgmatic in action? Check out the <a
|
||||
href="https://asciinema.org/a/203761" target="_blank">screencast</a>.
|
||||
|
||||
<script src="https://asciinema.org/a/203761.js" id="asciicast-203761" async></script>
|
||||
|
||||
|
||||
## Installation
|
||||
|
||||
To get up and running, follow the [Borg Quick
|
||||
Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) to create
|
||||
a repository on a local or remote host. Note that if you plan to run borgmatic
|
||||
on a schedule with cron, and you encrypt your Borg repository with a
|
||||
passphrase instead of a key file, you'll either need to set the borgmatic
|
||||
`encryption_passphrase` configuration variable or set the `BORG_PASSPHRASE`
|
||||
environment variable. See the [repository encryption
|
||||
section](https://borgbackup.readthedocs.io/en/latest/quickstart.html#repository-encryption)
|
||||
of the Quick Start for more info.
|
||||
To get up and running, first [install
|
||||
Borg](https://borgbackup.readthedocs.io/en/latest/installation.html), at
|
||||
least version 1.1.
|
||||
|
||||
Alternatively, the passphrase can be specified programatically by setting
|
||||
either the borgmatic `encryption_passcommand` configuration variable or the
|
||||
`BORG_PASSCOMMAND` environment variable. See the [Borg Security
|
||||
FAQ](http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically)
|
||||
for more info.
|
||||
|
||||
If the repository is on a remote host, make sure that your local root user has
|
||||
key-based ssh access to the desired user account on the remote host.
|
||||
|
||||
To install borgmatic, run the following command to download and install it:
|
||||
Then, run the following command to download and install borgmatic:
|
||||
|
||||
```bash
|
||||
sudo pip3 install --upgrade borgmatic
|
||||
@@ -81,19 +69,19 @@ sudo pip3 install --upgrade borgmatic
|
||||
Note that your pip binary may have a different name than "pip3". Make sure
|
||||
you're using Python 3, as borgmatic does not support Python 2.
|
||||
|
||||
|
||||
### Other ways to install
|
||||
|
||||
* [A borgmatic Docker image](https://hub.docker.com/r/b3vis/borgmatic/) based
|
||||
on Alpine Linux.
|
||||
* [Another borgmatic Docker image](https://hub.docker.com/r/coaxial/borgmatic/) based
|
||||
on Alpine Linux, updated automatically whenever the Alpine image updates.
|
||||
* [A borgmatic package for
|
||||
Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic).
|
||||
* [A borgmatic package for Arch
|
||||
Linux](https://aur.archlinux.org/packages/borgmatic/).
|
||||
* [A borgmatic package for OpenBSD](http://ports.su/sysutils/borgmatic).
|
||||
* [Docker image](https://hub.docker.com/r/monachus/borgmatic/)
|
||||
* [Another Docker image](https://hub.docker.com/r/b3vis/borgmatic/)
|
||||
* [Debian](https://tracker.debian.org/pkg/borgmatic)
|
||||
* [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
|
||||
* [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
|
||||
* [Arch Linux](https://aur.archlinux.org/packages/borgmatic/)
|
||||
* [OpenBSD](http://ports.su/sysutils/borgmatic)
|
||||
<br><br>
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
After you install borgmatic, generate a sample configuration file:
|
||||
@@ -108,7 +96,7 @@ not in your system `PATH`. Try looking in `/usr/local/bin/`.
|
||||
This generates a sample configuration file at /etc/borgmatic/config.yaml (by
|
||||
default). You should edit the file to suit your needs, as the values are just
|
||||
representative. All fields are optional except where indicated, so feel free
|
||||
to remove anything you don't need.
|
||||
to ignore anything you don't need.
|
||||
|
||||
You can also have a look at the [full configuration
|
||||
schema](https://projects.torsion.org/witten/borgmatic/src/master/borgmatic/config/schema.yaml)
|
||||
@@ -117,6 +105,161 @@ borgmatic has added new options since you originally created your
|
||||
configuration file.
|
||||
|
||||
|
||||
### Encryption
|
||||
|
||||
Note that if you plan to run borgmatic on a schedule with cron, and you
|
||||
encrypt your Borg repository with a passphrase instead of a key file, you'll
|
||||
either need to set the borgmatic `encryption_passphrase` configuration
|
||||
variable or set the `BORG_PASSPHRASE` environment variable. See the
|
||||
[repository encryption
|
||||
section](https://borgbackup.readthedocs.io/en/latest/quickstart.html#repository-encryption)
|
||||
of the Quick Start for more info.
|
||||
|
||||
Alternatively, the passphrase can be specified programatically by setting
|
||||
either the borgmatic `encryption_passcommand` configuration variable or the
|
||||
`BORG_PASSCOMMAND` environment variable. See the [Borg Security
|
||||
FAQ](http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically)
|
||||
for more info.
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
### Initialization
|
||||
|
||||
Before you can create backups with borgmatic, you first need to initialize a
|
||||
Borg repository so you have a destination for your backup archives. (But skip
|
||||
this step if you already have a Borg repository.) To create a repository, run
|
||||
a command like the following:
|
||||
|
||||
```bash
|
||||
borgmatic --init --encryption repokey
|
||||
```
|
||||
|
||||
This uses the borgmatic configuration file you created above to determine
|
||||
which local or remote repository to create, and encrypts it with the
|
||||
encryption passphrase specified there if one is provided. Read about [Borg
|
||||
encryption
|
||||
modes](https://borgbackup.readthedocs.io/en/latest/usage/init.html#encryption-modes)
|
||||
for the menu of available encryption modes.
|
||||
|
||||
Also, optionally check out the [Borg Quick
|
||||
Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) for more
|
||||
background about repository initialization.
|
||||
|
||||
If the repository is on a remote host, make sure that your local user has
|
||||
key-based SSH access to the desired user account on the remote host.
|
||||
|
||||
|
||||
### Backups
|
||||
|
||||
You can run borgmatic and start a backup simply by invoking it without
|
||||
arguments:
|
||||
|
||||
```bash
|
||||
borgmatic
|
||||
```
|
||||
|
||||
This will also prune any old backups as per the configured retention policy,
|
||||
and check backups for consistency problems due to things like file damage.
|
||||
|
||||
If you'd like to see the available command-line arguments, view the help:
|
||||
|
||||
```bash
|
||||
borgmatic --help
|
||||
```
|
||||
|
||||
Note that borgmatic prunes archives *before* creating an archive, so as to
|
||||
free up space for archiving. This means that when a borgmatic run finishes,
|
||||
there may still be prune-able archives. Not to worry, as they will get cleaned
|
||||
up at the start of the next run.
|
||||
|
||||
|
||||
### Verbosity
|
||||
|
||||
By default, the backup will proceed silently except in the case of errors. But
|
||||
if you'd like to to get additional information about the progress of the
|
||||
backup as it proceeds, use the verbosity option:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 1
|
||||
```
|
||||
|
||||
Or, for even more progress spew:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 2
|
||||
```
|
||||
|
||||
### À la carte
|
||||
|
||||
If you want to run borgmatic with only pruning, creating, or checking enabled,
|
||||
the following optional flags are available:
|
||||
|
||||
```bash
|
||||
borgmatic --prune
|
||||
borgmatic --create
|
||||
borgmatic --check
|
||||
```
|
||||
|
||||
You can run with only one of these flags provided, or you can mix and match
|
||||
any number of them. This supports use cases like running consistency checks
|
||||
from a different cron job with a different frequency, or running pruning with
|
||||
a different verbosity level.
|
||||
|
||||
Additionally, borgmatic provides convenient flags for Borg's
|
||||
[list](https://borgbackup.readthedocs.io/en/stable/usage/list.html) and
|
||||
[info](https://borgbackup.readthedocs.io/en/stable/usage/info.html)
|
||||
functionality:
|
||||
|
||||
|
||||
```bash
|
||||
borgmatic --list
|
||||
borgmatic --info
|
||||
```
|
||||
|
||||
You can include an optional `--json` flag with `--create`, `--list`, or
|
||||
`--info` to get the output formatted as JSON.
|
||||
|
||||
|
||||
## Autopilot
|
||||
|
||||
If you want to run borgmatic automatically, say once a day, the you can
|
||||
configure a job runner to invoke it periodically.
|
||||
|
||||
### cron
|
||||
|
||||
If you're using cron, download the [sample cron
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/cron/borgmatic).
|
||||
Then, from the directory where you downloaded it:
|
||||
|
||||
```bash
|
||||
sudo mv borgmatic /etc/cron.d/borgmatic
|
||||
sudo chmod +x /etc/cron.d/borgmatic
|
||||
```
|
||||
|
||||
You can modify the cron file if you'd like to run borgmatic more or less frequently.
|
||||
|
||||
### systemd
|
||||
|
||||
If you're using systemd instead of cron to run jobs, download the [sample
|
||||
systemd service
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.service)
|
||||
and the [sample systemd timer
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.timer).
|
||||
Then, from the directory where you downloaded them:
|
||||
|
||||
```bash
|
||||
sudo mv borgmatic.service borgmatic.timer /etc/systemd/system/
|
||||
sudo systemctl enable borgmatic.timer
|
||||
sudo systemctl start borgmatic.timer
|
||||
```
|
||||
|
||||
Feel free to modify the timer file based on how frequently you'd like
|
||||
borgmatic to run.
|
||||
|
||||
|
||||
## Advanced configuration
|
||||
|
||||
### Multiple configuration files
|
||||
|
||||
A more advanced usage is to create multiple separate configuration files and
|
||||
@@ -240,113 +383,6 @@ That's it! borgmatic will continue using your /etc/borgmatic configuration
|
||||
files.
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
You can run borgmatic and start a backup simply by invoking it without
|
||||
arguments:
|
||||
|
||||
```bash
|
||||
borgmatic
|
||||
```
|
||||
|
||||
This will also prune any old backups as per the configured retention policy,
|
||||
and check backups for consistency problems due to things like file damage.
|
||||
|
||||
If you'd like to see the available command-line arguments, view the help:
|
||||
|
||||
```bash
|
||||
borgmatic --help
|
||||
```
|
||||
|
||||
Note that borgmatic prunes archives *before* creating an archive, so as to
|
||||
free up space for archiving. This means that when a borgmatic run finishes,
|
||||
there may still be prune-able archives. Not to worry, as they will get cleaned
|
||||
up at the start of the next run.
|
||||
|
||||
### Verbosity
|
||||
|
||||
By default, the backup will proceed silently except in the case of errors. But
|
||||
if you'd like to to get additional information about the progress of the
|
||||
backup as it proceeds, use the verbosity option:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 1
|
||||
```
|
||||
|
||||
Or, for even more progress spew:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 2
|
||||
```
|
||||
|
||||
### À la carte
|
||||
|
||||
If you want to run borgmatic with only pruning, creating, or checking enabled,
|
||||
the following optional flags are available:
|
||||
|
||||
```bash
|
||||
borgmatic --prune
|
||||
borgmatic --create
|
||||
borgmatic --check
|
||||
```
|
||||
|
||||
You can run with only one of these flags provided, or you can mix and match
|
||||
any number of them. This supports use cases like running consistency checks
|
||||
from a different cron job with a different frequency, or running pruning with
|
||||
a different verbosity level.
|
||||
|
||||
Additionally, borgmatic provides convenient flags for Borg's
|
||||
[list](https://borgbackup.readthedocs.io/en/stable/usage/list.html) and
|
||||
[info](https://borgbackup.readthedocs.io/en/stable/usage/info.html)
|
||||
functionality:
|
||||
|
||||
|
||||
```bash
|
||||
borgmatic --list
|
||||
borgmatic --info
|
||||
```
|
||||
|
||||
You can include an optional `--json` flag with `--create`, `--list`, or
|
||||
`--info` to get the output formatted as JSON.
|
||||
|
||||
|
||||
## Autopilot
|
||||
|
||||
If you want to run borgmatic automatically, say once a day, the you can
|
||||
configure a job runner to invoke it periodically.
|
||||
|
||||
### cron
|
||||
|
||||
If you're using cron, download the [sample cron
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/cron/borgmatic).
|
||||
Then, from the directory where you downloaded it:
|
||||
|
||||
```bash
|
||||
sudo mv borgmatic /etc/cron.d/borgmatic
|
||||
sudo chmod +x /etc/cron.d/borgmatic
|
||||
```
|
||||
|
||||
You can modify the cron file if you'd like to run borgmatic more or less frequently.
|
||||
|
||||
### systemd
|
||||
|
||||
If you're using systemd instead of cron to run jobs, download the [sample
|
||||
systemd service
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.service)
|
||||
and the [sample systemd timer
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.timer).
|
||||
Then, from the directory where you downloaded them:
|
||||
|
||||
```bash
|
||||
sudo mv borgmatic.service borgmatic.timer /etc/systemd/system/
|
||||
sudo systemctl enable borgmatic.timer
|
||||
sudo systemctl start borgmatic.timer
|
||||
```
|
||||
|
||||
Feel free to modify the timer file based on how frequently you'd like
|
||||
borgmatic to run.
|
||||
|
||||
|
||||
## Support and contributing
|
||||
|
||||
### Issues
|
||||
@@ -382,6 +418,11 @@ the following deviations from it:
|
||||
* Within multiline constructs, use standard four-space indentation. Don't align
|
||||
indentation with an opening delimeter.
|
||||
|
||||
borgmatic code uses the [Black](https://black.readthedocs.io/en/stable/) code
|
||||
formatter and [Flake8](http://flake8.pycqa.org/en/latest/) code checker, so
|
||||
certain code style requirements will be enforced when running automated tests.
|
||||
See the Black and Flake8 documentation for more information.
|
||||
|
||||
|
||||
### Development
|
||||
|
||||
@@ -433,6 +474,25 @@ cd borgmatic
|
||||
tox
|
||||
```
|
||||
|
||||
If when running tests, you get an error from the
|
||||
[Black](https://black.readthedocs.io/en/stable/) code formatter about files
|
||||
that would be reformatted, you can ask Black to format them for you via the
|
||||
following:
|
||||
|
||||
```bash
|
||||
tox -e black
|
||||
```
|
||||
|
||||
### End-to-end tests
|
||||
|
||||
borgmatic additionally includes some end-to-end tests that integration test
|
||||
with Borg for a few representative scenarios. These tests don't run by default
|
||||
because they're relatively slow and depend on Borg. If you would like to run
|
||||
them:
|
||||
|
||||
```bash
|
||||
tox -e end-to-end
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -448,7 +508,7 @@ consistency:
|
||||
```
|
||||
|
||||
This error can be caused by an ssh timeout, which you can rectify by adding
|
||||
the following to the ~/.ssh/config file on the client:
|
||||
the following to the `~/.ssh/config` file on the client:
|
||||
|
||||
```text
|
||||
Host *
|
||||
|
||||
+25
-13
@@ -31,7 +31,9 @@ def _parse_checks(consistency_config):
|
||||
if checks == ['disabled']:
|
||||
return ()
|
||||
|
||||
return tuple(check for check in checks if check.lower() not in ('disabled', '')) or DEFAULT_CHECKS
|
||||
return (
|
||||
tuple(check for check in checks if check.lower() not in ('disabled', '')) or DEFAULT_CHECKS
|
||||
)
|
||||
|
||||
|
||||
def _make_check_flags(checks, check_last=None, prefix=None):
|
||||
@@ -60,22 +62,29 @@ def _make_check_flags(checks, check_last=None, prefix=None):
|
||||
last_flags = ()
|
||||
prefix_flags = ()
|
||||
if check_last:
|
||||
logger.warn('Ignoring check_last option, as "archives" is not in consistency checks.')
|
||||
logger.warning(
|
||||
'Ignoring check_last option, as "archives" is not in consistency checks.'
|
||||
)
|
||||
if prefix:
|
||||
logger.warn('Ignoring consistency prefix option, as "archives" is not in consistency checks.')
|
||||
|
||||
logger.warning(
|
||||
'Ignoring consistency prefix option, as "archives" is not in consistency checks.'
|
||||
)
|
||||
|
||||
if set(DEFAULT_CHECKS).issubset(set(checks)):
|
||||
return last_flags + prefix_flags
|
||||
|
||||
return tuple(
|
||||
'--{}-only'.format(check) for check in checks
|
||||
if check in DEFAULT_CHECKS
|
||||
) + last_flags + prefix_flags
|
||||
return (
|
||||
tuple('--{}-only'.format(check) for check in checks if check in DEFAULT_CHECKS)
|
||||
+ last_flags
|
||||
+ prefix_flags
|
||||
)
|
||||
|
||||
|
||||
def check_archives(repository, storage_config, consistency_config, local_path='borg', remote_path=None):
|
||||
def check_archives(
|
||||
repository, storage_config, consistency_config, local_path='borg', remote_path=None
|
||||
):
|
||||
'''
|
||||
Given a local or remote repository path, a storage config dict, a consistency config dict,
|
||||
Given a local or remote repository path, a storage config dict, a consistency config dict,
|
||||
and a local/remote commands to run, check the contained Borg archives for consistency.
|
||||
|
||||
If there are no consistency checks to run, skip running them.
|
||||
@@ -98,9 +107,12 @@ def check_archives(repository, storage_config, consistency_config, local_path='b
|
||||
prefix = consistency_config.get('prefix')
|
||||
|
||||
full_command = (
|
||||
local_path, 'check',
|
||||
repository,
|
||||
) + _make_check_flags(checks, check_last, prefix) + remote_path_flags + lock_wait_flags + verbosity_flags
|
||||
(local_path, 'check', repository)
|
||||
+ _make_check_flags(checks, check_last, prefix)
|
||||
+ remote_path_flags
|
||||
+ lock_wait_flags
|
||||
+ verbosity_flags
|
||||
)
|
||||
|
||||
# The check command spews to stdout/stderr even without the verbose flag. Suppress it.
|
||||
stdout = None if verbosity_flags else open(os.devnull, 'w')
|
||||
|
||||
+22
-36
@@ -9,20 +9,6 @@ import tempfile
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def initialize_environment(storage_config):
|
||||
passcommand = storage_config.get('encryption_passcommand')
|
||||
if passcommand:
|
||||
os.environ['BORG_PASSCOMMAND'] = passcommand
|
||||
|
||||
passphrase = storage_config.get('encryption_passphrase')
|
||||
if passphrase:
|
||||
os.environ['BORG_PASSPHRASE'] = passphrase
|
||||
|
||||
ssh_command = storage_config.get('ssh_command')
|
||||
if ssh_command:
|
||||
os.environ['BORG_RSH'] = ssh_command
|
||||
|
||||
|
||||
def _expand_directory(directory):
|
||||
'''
|
||||
Given a directory path, expand any tilde (representing a user's home directory) and any globs
|
||||
@@ -42,10 +28,7 @@ def _expand_directories(directories):
|
||||
return ()
|
||||
|
||||
return tuple(
|
||||
itertools.chain.from_iterable(
|
||||
_expand_directory(directory)
|
||||
for directory in directories
|
||||
)
|
||||
itertools.chain.from_iterable(_expand_directory(directory) for directory in directories)
|
||||
)
|
||||
|
||||
|
||||
@@ -75,8 +58,7 @@ def _make_pattern_flags(location_config, pattern_filename=None):
|
||||
|
||||
return tuple(
|
||||
itertools.chain.from_iterable(
|
||||
('--patterns-from', pattern_filename)
|
||||
for pattern_filename in pattern_filenames
|
||||
('--patterns-from', pattern_filename) for pattern_filename in pattern_filenames
|
||||
)
|
||||
)
|
||||
|
||||
@@ -91,8 +73,7 @@ def _make_exclude_flags(location_config, exclude_filename=None):
|
||||
)
|
||||
exclude_from_flags = tuple(
|
||||
itertools.chain.from_iterable(
|
||||
('--exclude-from', exclude_filename)
|
||||
for exclude_filename in exclude_filenames
|
||||
('--exclude-from', exclude_filename) for exclude_filename in exclude_filenames
|
||||
)
|
||||
)
|
||||
caches_flag = ('--exclude-caches',) if location_config.get('exclude_caches') else ()
|
||||
@@ -103,7 +84,15 @@ def _make_exclude_flags(location_config, exclude_filename=None):
|
||||
|
||||
|
||||
def create_archive(
|
||||
dry_run, repository, location_config, storage_config, local_path='borg', remote_path=None, json=False):
|
||||
dry_run,
|
||||
repository,
|
||||
location_config,
|
||||
storage_config,
|
||||
local_path='borg',
|
||||
remote_path=None,
|
||||
progress=False,
|
||||
json=False,
|
||||
):
|
||||
'''
|
||||
Given vebosity/dry-run flags, a local or remote repository path, a location config dict, and a
|
||||
storage config dict, create a Borg archive.
|
||||
@@ -113,6 +102,7 @@ def create_archive(
|
||||
pattern_file = _write_pattern_file(location_config.get('patterns'))
|
||||
exclude_file = _write_pattern_file(_expand_directories(location_config.get('exclude_patterns')))
|
||||
checkpoint_interval = storage_config.get('checkpoint_interval', None)
|
||||
chunker_params = storage_config.get('chunker_params', None)
|
||||
compression = storage_config.get('compression', None)
|
||||
remote_rate_limit = storage_config.get('remote_rate_limit', None)
|
||||
umask = storage_config.get('umask', None)
|
||||
@@ -123,22 +113,17 @@ def create_archive(
|
||||
|
||||
full_command = (
|
||||
(
|
||||
local_path, 'create',
|
||||
local_path,
|
||||
'create',
|
||||
'{repository}::{archive_name_format}'.format(
|
||||
repository=repository,
|
||||
archive_name_format=archive_name_format,
|
||||
repository=repository, archive_name_format=archive_name_format
|
||||
),
|
||||
)
|
||||
+ sources
|
||||
+ _make_pattern_flags(
|
||||
location_config,
|
||||
pattern_file.name if pattern_file else None,
|
||||
)
|
||||
+ _make_exclude_flags(
|
||||
location_config,
|
||||
exclude_file.name if exclude_file else None,
|
||||
)
|
||||
+ _make_pattern_flags(location_config, pattern_file.name if pattern_file else None)
|
||||
+ _make_exclude_flags(location_config, exclude_file.name if exclude_file else None)
|
||||
+ (('--checkpoint-interval', str(checkpoint_interval)) if checkpoint_interval else ())
|
||||
+ (('--chunker-params', chunker_params) if chunker_params else ())
|
||||
+ (('--compression', compression) if compression else ())
|
||||
+ (('--remote-ratelimit', str(remote_rate_limit)) if remote_rate_limit else ())
|
||||
+ (('--one-file-system',) if location_config.get('one_file_system') else ())
|
||||
@@ -148,11 +133,12 @@ def create_archive(
|
||||
+ (('--remote-path', remote_path) if remote_path else ())
|
||||
+ (('--umask', str(umask)) if umask else ())
|
||||
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
|
||||
+ (('--list', '--filter', 'AME',) if logger.isEnabledFor(logging.INFO) else ())
|
||||
+ (( '--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--list', '--filter', 'AME-') if logger.isEnabledFor(logging.INFO) else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--stats',) if not dry_run and logger.isEnabledFor(logging.INFO) else ())
|
||||
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
|
||||
+ (('--dry-run',) if dry_run else ())
|
||||
+ (('--progress',) if progress else ())
|
||||
+ (('--json',) if json else ())
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
import os
|
||||
|
||||
|
||||
def initialize(storage_config):
|
||||
passcommand = storage_config.get('encryption_passcommand')
|
||||
if passcommand:
|
||||
os.environ['BORG_PASSCOMMAND'] = passcommand
|
||||
|
||||
passphrase = storage_config.get('encryption_passphrase')
|
||||
if passphrase:
|
||||
os.environ['BORG_PASSPHRASE'] = passphrase
|
||||
|
||||
ssh_command = storage_config.get('ssh_command')
|
||||
if ssh_command:
|
||||
os.environ['BORG_RSH'] = ssh_command
|
||||
+18
-12
@@ -18,13 +18,13 @@ def extract_last_archive_dry_run(repository, lock_wait=None, local_path='borg',
|
||||
verbosity_flags = ('--debug', '--show-rc')
|
||||
elif logger.isEnabledFor(logging.INFO):
|
||||
verbosity_flags = ('--info',)
|
||||
|
||||
|
||||
full_list_command = (
|
||||
local_path, 'list',
|
||||
'--short',
|
||||
repository,
|
||||
) + remote_path_flags + lock_wait_flags + verbosity_flags
|
||||
(local_path, 'list', '--short', repository)
|
||||
+ remote_path_flags
|
||||
+ lock_wait_flags
|
||||
+ verbosity_flags
|
||||
)
|
||||
|
||||
list_output = subprocess.check_output(full_list_command).decode(sys.stdout.encoding)
|
||||
|
||||
@@ -34,13 +34,19 @@ def extract_last_archive_dry_run(repository, lock_wait=None, local_path='borg',
|
||||
|
||||
list_flag = ('--list',) if logger.isEnabledFor(logging.DEBUG) else ()
|
||||
full_extract_command = (
|
||||
local_path, 'extract',
|
||||
'--dry-run',
|
||||
'{repository}::{last_archive_name}'.format(
|
||||
repository=repository,
|
||||
last_archive_name=last_archive_name,
|
||||
),
|
||||
) + remote_path_flags + lock_wait_flags + verbosity_flags + list_flag
|
||||
(
|
||||
local_path,
|
||||
'extract',
|
||||
'--dry-run',
|
||||
'{repository}::{last_archive_name}'.format(
|
||||
repository=repository, last_archive_name=last_archive_name
|
||||
),
|
||||
)
|
||||
+ remote_path_flags
|
||||
+ lock_wait_flags
|
||||
+ verbosity_flags
|
||||
+ list_flag
|
||||
)
|
||||
|
||||
logger.debug(' '.join(full_extract_command))
|
||||
subprocess.check_call(full_extract_command)
|
||||
|
||||
@@ -5,7 +5,9 @@ import subprocess
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def display_archives_info(repository, storage_config, local_path='borg', remote_path=None, json=False):
|
||||
def display_archives_info(
|
||||
repository, storage_config, local_path='borg', remote_path=None, json=False
|
||||
):
|
||||
'''
|
||||
Given a local or remote repository path, and a storage config dict,
|
||||
display summary information for Borg archives in the repository.
|
||||
@@ -16,7 +18,7 @@ def display_archives_info(repository, storage_config, local_path='borg', remote_
|
||||
(local_path, 'info', repository)
|
||||
+ (('--remote-path', remote_path) if remote_path else ())
|
||||
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
|
||||
+ (('--json',) if json else ())
|
||||
)
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
import logging
|
||||
import subprocess
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def initialize_repository(
|
||||
repository,
|
||||
encryption_mode,
|
||||
append_only=None,
|
||||
storage_quota=None,
|
||||
local_path='borg',
|
||||
remote_path=None,
|
||||
):
|
||||
'''
|
||||
Given a local or remote repository path, a Borg encryption mode, whether the repository should
|
||||
be append-only, and the storage quota to use, initialize the repository.
|
||||
'''
|
||||
full_command = (
|
||||
(local_path, 'init', repository)
|
||||
+ (('--encryption', encryption_mode) if encryption_mode else ())
|
||||
+ (('--append-only',) if append_only else ())
|
||||
+ (('--storage-quota', storage_quota) if storage_quota else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--debug',) if logger.isEnabledFor(logging.DEBUG) else ())
|
||||
+ (('--remote-path', remote_path) if remote_path else ())
|
||||
)
|
||||
|
||||
logger.debug(' '.join(full_command))
|
||||
subprocess.check_call(full_command)
|
||||
@@ -16,7 +16,7 @@ def list_archives(repository, storage_config, local_path='borg', remote_path=Non
|
||||
(local_path, 'list', repository)
|
||||
+ (('--remote-path', remote_path) if remote_path else ())
|
||||
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
|
||||
+ (('--json',) if json else ())
|
||||
)
|
||||
|
||||
+6
-11
@@ -2,7 +2,6 @@ import logging
|
||||
import subprocess
|
||||
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@@ -31,7 +30,9 @@ def _make_prune_flags(retention_config):
|
||||
)
|
||||
|
||||
|
||||
def prune_archives(dry_run, repository, storage_config, retention_config, local_path='borg', remote_path=None):
|
||||
def prune_archives(
|
||||
dry_run, repository, storage_config, retention_config, local_path='borg', remote_path=None
|
||||
):
|
||||
'''
|
||||
Given dry-run flag, a local or remote repository path, a storage config dict, and a
|
||||
retention config dict, prune Borg archives according to the retention policy specified in that
|
||||
@@ -41,19 +42,13 @@ def prune_archives(dry_run, repository, storage_config, retention_config, local_
|
||||
lock_wait = storage_config.get('lock_wait', None)
|
||||
|
||||
full_command = (
|
||||
(
|
||||
local_path, 'prune',
|
||||
repository,
|
||||
) + tuple(
|
||||
element
|
||||
for pair in _make_prune_flags(retention_config)
|
||||
for element in pair
|
||||
)
|
||||
(local_path, 'prune', repository)
|
||||
+ tuple(element for pair in _make_prune_flags(retention_config) for element in pair)
|
||||
+ (('--remote-path', remote_path) if remote_path else ())
|
||||
+ (('--umask', str(umask)) if umask else ())
|
||||
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
|
||||
+ (('--stats',) if not dry_run and logger.isEnabledFor(logging.INFO) else ())
|
||||
+ (( '--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--debug', '--list', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
|
||||
+ (('--dry-run',) if dry_run else ())
|
||||
)
|
||||
|
||||
+130
-49
@@ -5,10 +5,17 @@ import os
|
||||
from subprocess import CalledProcessError
|
||||
import sys
|
||||
|
||||
from borgmatic.borg import check as borg_check, create as borg_create, prune as borg_prune, \
|
||||
list as borg_list, info as borg_info
|
||||
from borgmatic.borg import (
|
||||
check as borg_check,
|
||||
create as borg_create,
|
||||
environment as borg_environment,
|
||||
prune as borg_prune,
|
||||
list as borg_list,
|
||||
info as borg_info,
|
||||
init as borg_init,
|
||||
)
|
||||
from borgmatic.commands import hook
|
||||
from borgmatic.config import collect, convert, validate
|
||||
from borgmatic.config import checks, collect, convert, validate
|
||||
from borgmatic.signals import configure_signals
|
||||
from borgmatic.verbosity import verbosity_to_log_level
|
||||
|
||||
@@ -27,19 +34,21 @@ def parse_arguments(*arguments):
|
||||
config_paths = collect.get_default_config_paths()
|
||||
|
||||
parser = ArgumentParser(
|
||||
description=
|
||||
'''
|
||||
description='''
|
||||
A simple wrapper script for the Borg backup software that creates and prunes backups.
|
||||
If none of the --prune, --create, or --check options are given, then borgmatic defaults
|
||||
to all three: prune, create, and check archives.
|
||||
'''
|
||||
)
|
||||
parser.add_argument(
|
||||
'-c', '--config',
|
||||
'-c',
|
||||
'--config',
|
||||
nargs='+',
|
||||
dest='config_paths',
|
||||
default=config_paths,
|
||||
help='Configuration filenames or directories, defaults to: {}'.format(' '.join(config_paths)),
|
||||
help='Configuration filenames or directories, defaults to: {}'.format(
|
||||
' '.join(config_paths)
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
'--excludes',
|
||||
@@ -47,35 +56,57 @@ def parse_arguments(*arguments):
|
||||
help='Deprecated in favor of exclude_patterns within configuration',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-p', '--prune',
|
||||
'-I', '--init', dest='init', action='store_true', help='Initialize an empty Borg repository'
|
||||
)
|
||||
parser.add_argument(
|
||||
'-e',
|
||||
'--encryption',
|
||||
dest='encryption_mode',
|
||||
help='Borg repository encryption mode (for use with --init)',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--append-only',
|
||||
dest='append_only',
|
||||
action='store_true',
|
||||
help='Create an append-only repository (for use with --init)',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--storage-quota',
|
||||
dest='storage_quota',
|
||||
help='Create a repository with a fixed storage quota (for use with --init)',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-p',
|
||||
'--prune',
|
||||
dest='prune',
|
||||
action='store_true',
|
||||
help='Prune archives according to the retention policy',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-C', '--create',
|
||||
'-C',
|
||||
'--create',
|
||||
dest='create',
|
||||
action='store_true',
|
||||
help='Create archives (actually perform backups)',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-k', '--check',
|
||||
dest='check',
|
||||
action='store_true',
|
||||
help='Check archives for consistency',
|
||||
'-k', '--check', dest='check', action='store_true', help='Check archives for consistency'
|
||||
)
|
||||
parser.add_argument('-l', '--list', dest='list', action='store_true', help='List archives')
|
||||
parser.add_argument(
|
||||
'-l', '--list',
|
||||
dest='list',
|
||||
action='store_true',
|
||||
help='List archives',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-i', '--info',
|
||||
'-i',
|
||||
'--info',
|
||||
dest='info',
|
||||
action='store_true',
|
||||
help='Display summary information on archives',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--progress',
|
||||
dest='progress',
|
||||
default=False,
|
||||
action='store_true',
|
||||
help='Display progress with --create option for each file as it is backed up',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--json',
|
||||
dest='json',
|
||||
@@ -84,23 +115,47 @@ def parse_arguments(*arguments):
|
||||
help='Output results from the --create, --list, or --info options as json',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-n', '--dry-run',
|
||||
'-n',
|
||||
'--dry-run',
|
||||
dest='dry_run',
|
||||
action='store_true',
|
||||
help='Go through the motions, but do not actually write to any repositories',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-v', '--verbosity',
|
||||
'-v',
|
||||
'--verbosity',
|
||||
type=int,
|
||||
choices=range(0,3),
|
||||
choices=range(0, 3),
|
||||
default=0,
|
||||
help='Display verbose progress (1 for some, 2 for lots)',
|
||||
)
|
||||
|
||||
args = parser.parse_args(arguments)
|
||||
|
||||
if args.excludes_filename:
|
||||
raise ValueError(
|
||||
'The --excludes option has been replaced with exclude_patterns in configuration'
|
||||
)
|
||||
|
||||
if (args.encryption_mode or args.append_only or args.storage_quota) and not args.init:
|
||||
raise ValueError(
|
||||
'The --encryption, --append-only, and --storage-quota options can only be used with the --init option'
|
||||
)
|
||||
|
||||
if args.init and (args.prune or args.create or args.dry_run):
|
||||
raise ValueError(
|
||||
'The --init option cannot be used with the --prune, --create, or --dry-run options'
|
||||
)
|
||||
if args.init and not args.encryption_mode:
|
||||
raise ValueError('The --encryption option is required with the --init option')
|
||||
|
||||
if args.progress and not args.create:
|
||||
raise ValueError('The --progress option can only be used with the --create option')
|
||||
|
||||
if args.json and not (args.create or args.list or args.info):
|
||||
raise ValueError('The --json option can only be used with the --create, --list, or --info options')
|
||||
raise ValueError(
|
||||
'The --json option can only be used with the --create, --list, or --info options'
|
||||
)
|
||||
|
||||
if args.json and args.list and args.info:
|
||||
raise ValueError(
|
||||
@@ -109,7 +164,7 @@ def parse_arguments(*arguments):
|
||||
|
||||
# If any of the action flags are explicitly requested, leave them as-is. Otherwise, assume
|
||||
# defaults: Mutate the given arguments to enable the default actions.
|
||||
if args.prune or args.create or args.check or args.list or args.info:
|
||||
if args.init or args.prune or args.create or args.check or args.list or args.info:
|
||||
return args
|
||||
|
||||
args.prune = True
|
||||
@@ -133,12 +188,20 @@ def run_configuration(config_filename, args): # pragma: no cover
|
||||
try:
|
||||
local_path = location.get('local_path', 'borg')
|
||||
remote_path = location.get('remote_path')
|
||||
borg_create.initialize_environment(storage)
|
||||
borg_environment.initialize(storage)
|
||||
|
||||
if args.create:
|
||||
hook.execute_hook(hooks.get('before_backup'), config_filename, 'pre-backup')
|
||||
|
||||
_run_commands(args, consistency, local_path, location, remote_path, retention, storage)
|
||||
_run_commands(
|
||||
args=args,
|
||||
consistency=consistency,
|
||||
local_path=local_path,
|
||||
location=location,
|
||||
remote_path=remote_path,
|
||||
retention=retention,
|
||||
storage=storage,
|
||||
)
|
||||
|
||||
if args.create:
|
||||
hook.execute_hook(hooks.get('after_backup'), config_filename, 'post-backup')
|
||||
@@ -147,23 +210,48 @@ def run_configuration(config_filename, args): # pragma: no cover
|
||||
raise
|
||||
|
||||
|
||||
def _run_commands(args, consistency, local_path, location, remote_path, retention, storage):
|
||||
def _run_commands(*, args, consistency, local_path, location, remote_path, retention, storage):
|
||||
json_results = []
|
||||
for unexpanded_repository in location['repositories']:
|
||||
_run_commands_on_repository(
|
||||
args, consistency, json_results, local_path, location, remote_path, retention, storage,
|
||||
unexpanded_repository,
|
||||
args=args,
|
||||
consistency=consistency,
|
||||
json_results=json_results,
|
||||
local_path=local_path,
|
||||
location=location,
|
||||
remote_path=remote_path,
|
||||
retention=retention,
|
||||
storage=storage,
|
||||
unexpanded_repository=unexpanded_repository,
|
||||
)
|
||||
if args.json:
|
||||
sys.stdout.write(json.dumps(json_results))
|
||||
|
||||
|
||||
def _run_commands_on_repository(
|
||||
args, consistency, json_results, local_path, location, remote_path,
|
||||
retention, storage, unexpanded_repository,
|
||||
*,
|
||||
args,
|
||||
consistency,
|
||||
json_results,
|
||||
local_path,
|
||||
location,
|
||||
remote_path,
|
||||
retention,
|
||||
storage,
|
||||
unexpanded_repository
|
||||
): # pragma: no cover
|
||||
repository = os.path.expanduser(unexpanded_repository)
|
||||
dry_run_label = ' (dry run; not making any changes)' if args.dry_run else ''
|
||||
if args.init:
|
||||
logger.info('{}: Initializing repository'.format(repository))
|
||||
borg_init.initialize_repository(
|
||||
repository,
|
||||
args.encryption_mode,
|
||||
args.append_only,
|
||||
args.storage_quota,
|
||||
local_path=local_path,
|
||||
remote_path=remote_path,
|
||||
)
|
||||
if args.prune:
|
||||
logger.info('{}: Pruning archives{}'.format(repository, dry_run_label))
|
||||
borg_prune.prune_archives(
|
||||
@@ -183,24 +271,17 @@ def _run_commands_on_repository(
|
||||
storage,
|
||||
local_path=local_path,
|
||||
remote_path=remote_path,
|
||||
progress=args.progress,
|
||||
)
|
||||
if args.check:
|
||||
if args.check and checks.repository_enabled_for_checks(repository, consistency):
|
||||
logger.info('{}: Running consistency checks'.format(repository))
|
||||
borg_check.check_archives(
|
||||
repository,
|
||||
storage,
|
||||
consistency,
|
||||
local_path=local_path,
|
||||
remote_path=remote_path
|
||||
repository, storage, consistency, local_path=local_path, remote_path=remote_path
|
||||
)
|
||||
if args.list:
|
||||
logger.info('{}: Listing archives'.format(repository))
|
||||
output = borg_list.list_archives(
|
||||
repository,
|
||||
storage,
|
||||
local_path=local_path,
|
||||
remote_path=remote_path,
|
||||
json=args.json,
|
||||
repository, storage, local_path=local_path, remote_path=remote_path, json=args.json
|
||||
)
|
||||
if args.json:
|
||||
json_results.append(json.loads(output))
|
||||
@@ -209,11 +290,7 @@ def _run_commands_on_repository(
|
||||
if args.info:
|
||||
logger.info('{}: Displaying summary info for archives'.format(repository))
|
||||
output = borg_info.display_archives_info(
|
||||
repository,
|
||||
storage,
|
||||
local_path=local_path,
|
||||
remote_path=remote_path,
|
||||
json=args.json,
|
||||
repository, storage, local_path=local_path, remote_path=remote_path, json=args.json
|
||||
)
|
||||
if args.json:
|
||||
json_results.append(json.loads(output))
|
||||
@@ -232,10 +309,14 @@ def main(): # pragma: no cover
|
||||
convert.guard_configuration_upgraded(LEGACY_CONFIG_PATH, config_filenames)
|
||||
|
||||
if len(config_filenames) == 0:
|
||||
raise ValueError('Error: No configuration files found in: {}'.format(' '.join(args.config_paths)))
|
||||
raise ValueError(
|
||||
'Error: No configuration files found in: {}'.format(' '.join(args.config_paths))
|
||||
)
|
||||
|
||||
for config_filename in config_filenames:
|
||||
run_configuration(config_filename, args)
|
||||
except (ValueError, OSError, CalledProcessError) as error:
|
||||
print(error, file=sys.stderr)
|
||||
print(file=sys.stderr)
|
||||
print('Need some help? https://torsion.org/borgmatic/#issues', file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
from argparse import ArgumentParser
|
||||
import os
|
||||
from subprocess import CalledProcessError
|
||||
import sys
|
||||
import textwrap
|
||||
|
||||
@@ -26,22 +25,31 @@ def parse_arguments(*arguments):
|
||||
'''
|
||||
)
|
||||
parser.add_argument(
|
||||
'-s', '--source-config',
|
||||
'-s',
|
||||
'--source-config',
|
||||
dest='source_config_filename',
|
||||
default=DEFAULT_SOURCE_CONFIG_FILENAME,
|
||||
help='Source INI-style configuration filename. Default: {}'.format(DEFAULT_SOURCE_CONFIG_FILENAME),
|
||||
help='Source INI-style configuration filename. Default: {}'.format(
|
||||
DEFAULT_SOURCE_CONFIG_FILENAME
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
'-e', '--source-excludes',
|
||||
'-e',
|
||||
'--source-excludes',
|
||||
dest='source_excludes_filename',
|
||||
default=DEFAULT_SOURCE_EXCLUDES_FILENAME if os.path.exists(DEFAULT_SOURCE_EXCLUDES_FILENAME) else None,
|
||||
default=DEFAULT_SOURCE_EXCLUDES_FILENAME
|
||||
if os.path.exists(DEFAULT_SOURCE_EXCLUDES_FILENAME)
|
||||
else None,
|
||||
help='Excludes filename',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-d', '--destination-config',
|
||||
'-d',
|
||||
'--destination-config',
|
||||
dest='destination_config_filename',
|
||||
default=DEFAULT_DESTINATION_CONFIG_FILENAME,
|
||||
help='Destination YAML configuration filename. Default: {}'.format(DEFAULT_DESTINATION_CONFIG_FILENAME),
|
||||
help='Destination YAML configuration filename. Default: {}'.format(
|
||||
DEFAULT_DESTINATION_CONFIG_FILENAME
|
||||
),
|
||||
)
|
||||
|
||||
return parser.parse_args(arguments)
|
||||
@@ -57,11 +65,13 @@ def display_result(args): # pragma: no cover
|
||||
),
|
||||
TEXT_WRAP_CHARACTERS,
|
||||
)
|
||||
|
||||
|
||||
delete_lines = textwrap.wrap(
|
||||
'Once you are satisfied, you can safely delete {}{}.'.format(
|
||||
args.source_config_filename,
|
||||
' and {}'.format(args.source_excludes_filename) if args.source_excludes_filename else '',
|
||||
' and {}'.format(args.source_excludes_filename)
|
||||
if args.source_excludes_filename
|
||||
else '',
|
||||
),
|
||||
TEXT_WRAP_CHARACTERS,
|
||||
)
|
||||
@@ -75,7 +85,9 @@ def main(): # pragma: no cover
|
||||
try:
|
||||
args = parse_arguments(*sys.argv[1:])
|
||||
schema = yaml.round_trip_load(open(validate.schema_filename()).read())
|
||||
source_config = legacy.parse_configuration(args.source_config_filename, legacy.CONFIG_FORMAT)
|
||||
source_config = legacy.parse_configuration(
|
||||
args.source_config_filename, legacy.CONFIG_FORMAT
|
||||
)
|
||||
source_config_file_mode = os.stat(args.source_config_filename).st_mode
|
||||
source_excludes = (
|
||||
open(args.source_excludes_filename).read().splitlines()
|
||||
@@ -83,12 +95,12 @@ def main(): # pragma: no cover
|
||||
else []
|
||||
)
|
||||
|
||||
destination_config = convert.convert_legacy_parsed_config(source_config, source_excludes, schema)
|
||||
destination_config = convert.convert_legacy_parsed_config(
|
||||
source_config, source_excludes, schema
|
||||
)
|
||||
|
||||
generate.write_configuration(
|
||||
args.destination_config_filename,
|
||||
destination_config,
|
||||
mode=source_config_file_mode,
|
||||
args.destination_config_filename, destination_config, mode=source_config_file_mode
|
||||
)
|
||||
|
||||
display_result(args)
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
from argparse import ArgumentParser
|
||||
import os
|
||||
from subprocess import CalledProcessError
|
||||
import sys
|
||||
|
||||
from borgmatic.config import convert, generate, validate
|
||||
from borgmatic.config import generate, validate
|
||||
|
||||
|
||||
DEFAULT_DESTINATION_CONFIG_FILENAME = '/etc/borgmatic/config.yaml'
|
||||
@@ -16,10 +14,13 @@ def parse_arguments(*arguments):
|
||||
'''
|
||||
parser = ArgumentParser(description='Generate a sample borgmatic YAML configuration file.')
|
||||
parser.add_argument(
|
||||
'-d', '--destination',
|
||||
'-d',
|
||||
'--destination',
|
||||
dest='destination_filename',
|
||||
default=DEFAULT_DESTINATION_CONFIG_FILENAME,
|
||||
help='Destination YAML configuration filename. Default: {}'.format(DEFAULT_DESTINATION_CONFIG_FILENAME),
|
||||
help='Destination YAML configuration filename. Default: {}'.format(
|
||||
DEFAULT_DESTINATION_CONFIG_FILENAME
|
||||
),
|
||||
)
|
||||
|
||||
return parser.parse_args(arguments)
|
||||
@@ -29,12 +30,16 @@ def main(): # pragma: no cover
|
||||
try:
|
||||
args = parse_arguments(*sys.argv[1:])
|
||||
|
||||
generate.generate_sample_configuration(args.destination_filename, validate.schema_filename())
|
||||
generate.generate_sample_configuration(
|
||||
args.destination_filename, validate.schema_filename()
|
||||
)
|
||||
|
||||
print('Generated a sample configuration file at {}.'.format(args.destination_filename))
|
||||
print()
|
||||
print('Please edit the file to suit your needs. The values are just representative.')
|
||||
print('All fields are optional except where indicated.')
|
||||
print()
|
||||
print('If you ever need help: https://torsion.org/borgmatic/#issues')
|
||||
except (ValueError, OSError) as error:
|
||||
print(error, file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
@@ -13,7 +13,11 @@ def execute_hook(commands, config_filename, description):
|
||||
if len(commands) == 1:
|
||||
logger.info('{}: Running command for {} hook'.format(config_filename, description))
|
||||
else:
|
||||
logger.info('{}: Running {} commands for {} hook'.format(config_filename, len(commands), description))
|
||||
logger.info(
|
||||
'{}: Running {} commands for {} hook'.format(
|
||||
config_filename, len(commands), description
|
||||
)
|
||||
)
|
||||
|
||||
for command in commands:
|
||||
logger.debug('{}: Hook command: {}'.format(config_filename, command))
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
def repository_enabled_for_checks(repository, consistency):
|
||||
'''
|
||||
Given a repository name and a consistency configuration dict, return whether the repository
|
||||
is enabled to have consistency checks run.
|
||||
'''
|
||||
if not consistency.get('check_repositories'):
|
||||
return True
|
||||
|
||||
return repository in consistency['check_repositories']
|
||||
@@ -7,8 +7,8 @@ def get_default_config_paths():
|
||||
default configuration paths. This includes both system-wide configuration and configuration in
|
||||
the current user's home directory.
|
||||
'''
|
||||
user_config_directory = (
|
||||
os.getenv('XDG_CONFIG_HOME') or os.path.expandvars(os.path.join('$HOME', '.config'))
|
||||
user_config_directory = os.getenv('XDG_CONFIG_HOME') or os.path.expandvars(
|
||||
os.path.join('$HOME', '.config')
|
||||
)
|
||||
|
||||
return [
|
||||
|
||||
+20
-18
@@ -12,14 +12,17 @@ def _convert_section(source_section_config, section_schema):
|
||||
|
||||
Where integer types exist in the given section schema, convert their values to integers.
|
||||
'''
|
||||
destination_section_config = yaml.comments.CommentedMap([
|
||||
(
|
||||
option_name,
|
||||
int(option_value)
|
||||
if section_schema['map'].get(option_name, {}).get('type') == 'int' else option_value
|
||||
)
|
||||
for option_name, option_value in source_section_config.items()
|
||||
])
|
||||
destination_section_config = yaml.comments.CommentedMap(
|
||||
[
|
||||
(
|
||||
option_name,
|
||||
int(option_value)
|
||||
if section_schema['map'].get(option_name, {}).get('type') == 'int'
|
||||
else option_value,
|
||||
)
|
||||
for option_name, option_value in source_section_config.items()
|
||||
]
|
||||
)
|
||||
|
||||
return destination_section_config
|
||||
|
||||
@@ -33,10 +36,12 @@ def convert_legacy_parsed_config(source_config, source_excludes, schema):
|
||||
Additionally, use the given schema as a source of helpful comments to include within the
|
||||
returned CommentedMap.
|
||||
'''
|
||||
destination_config = yaml.comments.CommentedMap([
|
||||
(section_name, _convert_section(section_config, schema['map'][section_name]))
|
||||
for section_name, section_config in source_config._asdict().items()
|
||||
])
|
||||
destination_config = yaml.comments.CommentedMap(
|
||||
[
|
||||
(section_name, _convert_section(section_config, schema['map'][section_name]))
|
||||
for section_name, section_config in source_config._asdict().items()
|
||||
]
|
||||
)
|
||||
|
||||
# Split space-seperated values into actual lists, make "repository" into a list, and merge in
|
||||
# excludes.
|
||||
@@ -53,9 +58,7 @@ def convert_legacy_parsed_config(source_config, source_excludes, schema):
|
||||
|
||||
for section_name, section_config in destination_config.items():
|
||||
generate.add_comments_to_configuration(
|
||||
section_config,
|
||||
schema['map'][section_name],
|
||||
indent=generate.INDENT,
|
||||
section_config, schema['map'][section_name], indent=generate.INDENT
|
||||
)
|
||||
|
||||
return destination_config
|
||||
@@ -85,8 +88,7 @@ def guard_configuration_upgraded(source_config_filename, destination_config_file
|
||||
The idea is that we want to alert the user about upgrading their config if they haven't already.
|
||||
'''
|
||||
destination_config_exists = any(
|
||||
os.path.exists(filename)
|
||||
for filename in destination_config_filenames
|
||||
os.path.exists(filename) for filename in destination_config_filenames
|
||||
)
|
||||
|
||||
if os.path.exists(source_config_filename) and not destination_config_exists:
|
||||
@@ -107,5 +109,5 @@ Please remove the "--excludes" argument and run borgmatic again.'''
|
||||
|
||||
|
||||
def guard_excludes_filename_omitted(excludes_filename):
|
||||
if excludes_filename != None:
|
||||
if excludes_filename is not None:
|
||||
raise LegacyExcludesFilenamePresent()
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
from collections import OrderedDict
|
||||
import os
|
||||
|
||||
from ruamel import yaml
|
||||
@@ -9,12 +8,11 @@ INDENT = 4
|
||||
|
||||
def _insert_newline_before_comment(config, field_name):
|
||||
'''
|
||||
Using some ruamel.yaml black magic, insert a blank line in the config right befor the given
|
||||
Using some ruamel.yaml black magic, insert a blank line in the config right before the given
|
||||
field and its comments.
|
||||
'''
|
||||
config.ca.items[field_name][1].insert(
|
||||
0,
|
||||
yaml.tokens.CommentToken('\n', yaml.error.CommentMark(0), None),
|
||||
0, yaml.tokens.CommentToken('\n', yaml.error.CommentMark(0), None)
|
||||
)
|
||||
|
||||
|
||||
@@ -27,23 +25,79 @@ def _schema_to_sample_configuration(schema, level=0):
|
||||
if example is not None:
|
||||
return example
|
||||
|
||||
config = yaml.comments.CommentedMap([
|
||||
(
|
||||
section_name,
|
||||
_schema_to_sample_configuration(section_schema, level + 1),
|
||||
)
|
||||
for section_name, section_schema in schema['map'].items()
|
||||
])
|
||||
config = yaml.comments.CommentedMap(
|
||||
[
|
||||
(section_name, _schema_to_sample_configuration(section_schema, level + 1))
|
||||
for section_name, section_schema in schema['map'].items()
|
||||
]
|
||||
)
|
||||
|
||||
add_comments_to_configuration(config, schema, indent=(level * INDENT))
|
||||
|
||||
return config
|
||||
|
||||
|
||||
def write_configuration(config_filename, config, mode=0o600):
|
||||
def _comment_out_line(line):
|
||||
# If it's already is commented out (or empty), there's nothing further to do!
|
||||
stripped_line = line.lstrip()
|
||||
if not stripped_line or stripped_line.startswith('#'):
|
||||
return line
|
||||
|
||||
# Comment out the names of optional sections.
|
||||
one_indent = ' ' * INDENT
|
||||
if not line.startswith(one_indent):
|
||||
return '#' + line
|
||||
|
||||
# Otherwise, comment out the line, but insert the "#" after the first indent for aesthetics.
|
||||
return '#'.join((one_indent, line[INDENT:]))
|
||||
|
||||
|
||||
REQUIRED_KEYS = {'source_directories', 'repositories', 'keep_daily'}
|
||||
REQUIRED_SECTION_NAMES = {'location', 'retention'}
|
||||
|
||||
|
||||
def _comment_out_optional_configuration(rendered_config):
|
||||
'''
|
||||
Given a target config filename and a config data structure of nested OrderedDicts, write out the
|
||||
config to file as YAML. Create any containing directories as needed.
|
||||
Post-process a rendered configuration string to comment out optional key/values. The idea is
|
||||
that this prevents the user from having to comment out a bunch of configuration they don't care
|
||||
about to get to a minimal viable configuration file.
|
||||
|
||||
Ideally ruamel.yaml would support this during configuration generation, but it's not terribly
|
||||
easy to accomplish that way.
|
||||
'''
|
||||
lines = []
|
||||
required = False
|
||||
|
||||
for line in rendered_config.split('\n'):
|
||||
key = line.strip().split(':')[0]
|
||||
|
||||
if key in REQUIRED_SECTION_NAMES:
|
||||
lines.append(line)
|
||||
continue
|
||||
|
||||
# Upon encountering a required configuration option, skip commenting out lines until the
|
||||
# next blank line.
|
||||
if key in REQUIRED_KEYS:
|
||||
required = True
|
||||
elif not key:
|
||||
required = False
|
||||
|
||||
lines.append(_comment_out_line(line) if not required else line)
|
||||
|
||||
return '\n'.join(lines)
|
||||
|
||||
|
||||
def _render_configuration(config):
|
||||
'''
|
||||
Given a config data structure of nested OrderedDicts, render the config as YAML and return it.
|
||||
'''
|
||||
return yaml.round_trip_dump(config, indent=INDENT, block_seq_indent=INDENT)
|
||||
|
||||
|
||||
def write_configuration(config_filename, rendered_config, mode=0o600):
|
||||
'''
|
||||
Given a target config filename and rendered config YAML, write it out to file. Create any
|
||||
containing directories as needed.
|
||||
'''
|
||||
if os.path.exists(config_filename):
|
||||
raise FileExistsError('{} already exists. Aborting.'.format(config_filename))
|
||||
@@ -54,7 +108,7 @@ def write_configuration(config_filename, config, mode=0o600):
|
||||
pass
|
||||
|
||||
with open(config_filename, 'w') as config_file:
|
||||
config_file.write(yaml.round_trip_dump(config, indent=INDENT, block_seq_indent=INDENT))
|
||||
config_file.write(rendered_config)
|
||||
|
||||
os.chmod(config_filename, mode)
|
||||
|
||||
@@ -73,11 +127,7 @@ def add_comments_to_configuration(config, schema, indent=0):
|
||||
if not field_schema or not description:
|
||||
continue
|
||||
|
||||
config.yaml_set_comment_before_after_key(
|
||||
key=field_name,
|
||||
before=description,
|
||||
indent=indent,
|
||||
)
|
||||
config.yaml_set_comment_before_after_key(key=field_name, before=description, indent=indent)
|
||||
if index > 0:
|
||||
_insert_newline_before_comment(config, field_name)
|
||||
|
||||
@@ -90,4 +140,6 @@ def generate_sample_configuration(config_filename, schema_filename):
|
||||
schema = yaml.round_trip_load(open(schema_filename))
|
||||
config = _schema_to_sample_configuration(schema)
|
||||
|
||||
write_configuration(config_filename, config)
|
||||
write_configuration(
|
||||
config_filename, _comment_out_optional_configuration(_render_configuration(config))
|
||||
)
|
||||
|
||||
+18
-26
@@ -45,12 +45,8 @@ CONFIG_FORMAT = (
|
||||
),
|
||||
),
|
||||
Section_format(
|
||||
'consistency',
|
||||
(
|
||||
option('checks', required=False),
|
||||
option('check_last', required=False),
|
||||
),
|
||||
)
|
||||
'consistency', (option('checks', required=False), option('check_last', required=False))
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -66,7 +62,8 @@ def validate_configuration_format(parser, config_format):
|
||||
'''
|
||||
section_names = set(parser.sections())
|
||||
required_section_names = tuple(
|
||||
section.name for section in config_format
|
||||
section.name
|
||||
for section in config_format
|
||||
if any(option.required for option in section.options)
|
||||
)
|
||||
|
||||
@@ -80,9 +77,7 @@ def validate_configuration_format(parser, config_format):
|
||||
|
||||
missing_section_names = set(required_section_names) - section_names
|
||||
if missing_section_names:
|
||||
raise ValueError(
|
||||
'Missing config sections: {}'.format(', '.join(missing_section_names))
|
||||
)
|
||||
raise ValueError('Missing config sections: {}'.format(', '.join(missing_section_names)))
|
||||
|
||||
for section_format in config_format:
|
||||
if section_format.name not in section_names:
|
||||
@@ -91,26 +86,28 @@ def validate_configuration_format(parser, config_format):
|
||||
option_names = parser.options(section_format.name)
|
||||
expected_options = section_format.options
|
||||
|
||||
unexpected_option_names = set(option_names) - set(option.name for option in expected_options)
|
||||
unexpected_option_names = set(option_names) - set(
|
||||
option.name for option in expected_options
|
||||
)
|
||||
|
||||
if unexpected_option_names:
|
||||
raise ValueError(
|
||||
'Unexpected options found in config section {}: {}'.format(
|
||||
section_format.name,
|
||||
', '.join(sorted(unexpected_option_names)),
|
||||
section_format.name, ', '.join(sorted(unexpected_option_names))
|
||||
)
|
||||
)
|
||||
|
||||
missing_option_names = tuple(
|
||||
option.name for option in expected_options if option.required
|
||||
option.name
|
||||
for option in expected_options
|
||||
if option.required
|
||||
if option.name not in option_names
|
||||
)
|
||||
|
||||
if missing_option_names:
|
||||
raise ValueError(
|
||||
'Required options missing from config section {}: {}'.format(
|
||||
section_format.name,
|
||||
', '.join(missing_option_names)
|
||||
section_format.name, ', '.join(missing_option_names)
|
||||
)
|
||||
)
|
||||
|
||||
@@ -123,11 +120,7 @@ def parse_section_options(parser, section_format):
|
||||
|
||||
Raise ValueError if any option values cannot be coerced to the expected Python data type.
|
||||
'''
|
||||
type_getter = {
|
||||
str: parser.get,
|
||||
int: parser.getint,
|
||||
bool: parser.getboolean,
|
||||
}
|
||||
type_getter = {str: parser.get, int: parser.getint, bool: parser.getboolean}
|
||||
|
||||
return OrderedDict(
|
||||
(option.name, type_getter[option.value_type](section_format.name, option.name))
|
||||
@@ -151,11 +144,10 @@ def parse_configuration(config_filename, config_format):
|
||||
|
||||
# Describes a parsed configuration, where each attribute is the name of a configuration file
|
||||
# section and each value is a dict of that section's parsed options.
|
||||
Parsed_config = namedtuple('Parsed_config', (section_format.name for section_format in config_format))
|
||||
Parsed_config = namedtuple(
|
||||
'Parsed_config', (section_format.name for section_format in config_format)
|
||||
)
|
||||
|
||||
return Parsed_config(
|
||||
*(
|
||||
parse_section_options(parser, section_format)
|
||||
for section_format in config_format
|
||||
)
|
||||
*(parse_section_options(parser, section_format) for section_format in config_format)
|
||||
)
|
||||
|
||||
@@ -18,6 +18,16 @@ map:
|
||||
- /home
|
||||
- /etc
|
||||
- /var/log/syslog*
|
||||
repositories:
|
||||
required: true
|
||||
seq:
|
||||
- type: scalar
|
||||
desc: |
|
||||
Paths to local or remote repositories (required). Tildes are expanded. Multiple
|
||||
repositories are backed up to in sequence. See ssh_command for SSH options like
|
||||
identity file or port.
|
||||
example:
|
||||
- user@backupserver:sourcehostname.borg
|
||||
one_file_system:
|
||||
type: bool
|
||||
desc: Stay in same file system (do not cross mount points).
|
||||
@@ -48,16 +58,6 @@ map:
|
||||
type: scalar
|
||||
desc: Alternate Borg remote executable. Defaults to "borg".
|
||||
example: borg1
|
||||
repositories:
|
||||
required: true
|
||||
seq:
|
||||
- type: scalar
|
||||
desc: |
|
||||
Paths to local or remote repositories (required). Tildes are expanded. Multiple
|
||||
repositories are backed up to in sequence. See ssh_command for SSH options like
|
||||
identity file or port.
|
||||
example:
|
||||
- user@backupserver:sourcehostname.borg
|
||||
patterns:
|
||||
seq:
|
||||
- type: scalar
|
||||
@@ -138,6 +138,13 @@ map:
|
||||
https://borgbackup.readthedocs.io/en/stable/faq.html#if-a-backup-stops-mid-way-does-the-already-backed-up-data-stay-there
|
||||
for details. Defaults to checkpoints every 1800 seconds (30 minutes).
|
||||
example: 1800
|
||||
chunker_params:
|
||||
type: scalar
|
||||
desc: |
|
||||
Specify the parameters passed to then chunker (CHUNK_MIN_EXP, CHUNK_MAX_EXP,
|
||||
HASH_MASK_BITS, HASH_WINDOW_SIZE). See https://borgbackup.readthedocs.io/en/stable/internals.html
|
||||
for details.
|
||||
example: 19,23,21,4095
|
||||
compression:
|
||||
type: scalar
|
||||
desc: |
|
||||
@@ -181,6 +188,10 @@ map:
|
||||
type: scalar
|
||||
desc: Keep all archives within this time interval.
|
||||
example: 3H
|
||||
keep_secondly:
|
||||
type: int
|
||||
desc: Number of secondly archives to keep.
|
||||
example: 60
|
||||
keep_minutely:
|
||||
type: int
|
||||
desc: Number of minutely archives to keep.
|
||||
@@ -232,6 +243,16 @@ map:
|
||||
example:
|
||||
- repository
|
||||
- archives
|
||||
check_repositories:
|
||||
seq:
|
||||
- type: scalar
|
||||
desc: |
|
||||
Paths to a subset of the repositories in the location section on which to run
|
||||
consistency checks. Handy in case some of your repositories are very large, and
|
||||
so running consistency checks on them would take too long. Defaults to running
|
||||
consistency checks on all repositories configured in the location section.
|
||||
example:
|
||||
- user@backupserver:sourcehostname.borg
|
||||
check_last:
|
||||
type: int
|
||||
desc: Restrict the number of checked archives to the last n. Applies only to the
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import logging
|
||||
import sys
|
||||
import warnings
|
||||
|
||||
import pkg_resources
|
||||
import pykwalify.core
|
||||
@@ -24,6 +22,7 @@ class Validation_error(ValueError):
|
||||
A collection of error message strings generated when attempting to validate a particular
|
||||
configurartion file.
|
||||
'''
|
||||
|
||||
def __init__(self, config_filename, error_messages):
|
||||
self.config_filename = config_filename
|
||||
self.error_messages = error_messages
|
||||
@@ -48,15 +47,29 @@ def apply_logical_validation(config_filename, parsed_configuration):
|
||||
|
||||
if archive_name_format and not prefix:
|
||||
raise Validation_error(
|
||||
config_filename, (
|
||||
'If you provide an archive_name_format, you must also specify a retention prefix.',
|
||||
)
|
||||
config_filename,
|
||||
('If you provide an archive_name_format, you must also specify a retention prefix.',),
|
||||
)
|
||||
|
||||
location_repositories = parsed_configuration.get('location', {}).get('repositories')
|
||||
check_repositories = parsed_configuration.get('consistency', {}).get('check_repositories', [])
|
||||
for repository in check_repositories:
|
||||
if repository not in location_repositories:
|
||||
raise Validation_error(
|
||||
config_filename,
|
||||
(
|
||||
'Unknown repository in the consistency section\'s check_repositories: {}'.format(
|
||||
repository
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
consistency_prefix = parsed_configuration.get('consistency', {}).get('prefix')
|
||||
if archive_name_format and not consistency_prefix:
|
||||
logger.warning('Since version 1.1.16, if you provide `archive_name_format`, you should also'
|
||||
' specify `consistency.prefix`.')
|
||||
logger.warning(
|
||||
'Since version 1.1.16, if you provide `archive_name_format`, you should also'
|
||||
' specify `consistency.prefix`.'
|
||||
)
|
||||
|
||||
|
||||
def parse_configuration(config_filename, schema_filename):
|
||||
|
||||
@@ -1,105 +0,0 @@
|
||||
import os
|
||||
|
||||
from flexmock import flexmock
|
||||
import pytest
|
||||
|
||||
from borgmatic.commands import borgmatic as module
|
||||
|
||||
|
||||
def test_parse_arguments_with_no_arguments_uses_defaults():
|
||||
config_paths = ['default']
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(config_paths)
|
||||
|
||||
parser = module.parse_arguments()
|
||||
|
||||
assert parser.config_paths == config_paths
|
||||
assert parser.excludes_filename == None
|
||||
assert parser.verbosity is 0
|
||||
assert parser.json is False
|
||||
|
||||
|
||||
def test_parse_arguments_with_path_arguments_overrides_defaults():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments('--config', 'myconfig', '--excludes', 'myexcludes')
|
||||
|
||||
assert parser.config_paths == ['myconfig']
|
||||
assert parser.excludes_filename == 'myexcludes'
|
||||
assert parser.verbosity is 0
|
||||
|
||||
|
||||
def test_parse_arguments_with_multiple_config_paths_parses_as_list():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments('--config', 'myconfig', 'otherconfig')
|
||||
|
||||
assert parser.config_paths == ['myconfig', 'otherconfig']
|
||||
assert parser.verbosity is 0
|
||||
|
||||
|
||||
def test_parse_arguments_with_verbosity_flag_overrides_default():
|
||||
config_paths = ['default']
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(config_paths)
|
||||
|
||||
parser = module.parse_arguments('--verbosity', '1')
|
||||
|
||||
assert parser.config_paths == config_paths
|
||||
assert parser.excludes_filename == None
|
||||
assert parser.verbosity == 1
|
||||
|
||||
|
||||
def test_parse_arguments_with_json_flag_overrides_default():
|
||||
parser = module.parse_arguments('--list', '--json')
|
||||
assert parser.json is True
|
||||
|
||||
|
||||
def test_parse_arguments_with_no_actions_defaults_to_all_actions_enabled():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments()
|
||||
|
||||
assert parser.prune is True
|
||||
assert parser.create is True
|
||||
assert parser.check is True
|
||||
|
||||
|
||||
def test_parse_arguments_with_prune_action_leaves_other_actions_disabled():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments('--prune')
|
||||
|
||||
assert parser.prune is True
|
||||
assert parser.create is False
|
||||
assert parser.check is False
|
||||
|
||||
|
||||
def test_parse_arguments_with_multiple_actions_leaves_other_action_disabled():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments('--create', '--check')
|
||||
|
||||
assert parser.prune is False
|
||||
assert parser.create is True
|
||||
assert parser.check is True
|
||||
|
||||
|
||||
def test_parse_arguments_with_invalid_arguments_exits():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(SystemExit):
|
||||
module.parse_arguments('--posix-me-harder')
|
||||
|
||||
|
||||
def test_parse_arguments_with_json_flag_with_list_or_info_flag_does_not_raise_any_error():
|
||||
module.parse_arguments('--list', '--json')
|
||||
module.parse_arguments('--info', '--json')
|
||||
|
||||
|
||||
def test_parse_arguments_with_json_flag_but_no_list_or_info_flag_raises_value_error():
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--json')
|
||||
|
||||
|
||||
def test_parse_arguments_with_json_flag_and_both_list_and_info_flag_raises_value_error():
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--list', '--info', '--json')
|
||||
@@ -1,49 +0,0 @@
|
||||
from collections import OrderedDict
|
||||
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.config import generate as module
|
||||
|
||||
|
||||
def test_schema_to_sample_configuration_generates_config_with_examples():
|
||||
flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
|
||||
flexmock(module).should_receive('add_comments_to_configuration')
|
||||
schema = {
|
||||
'map': OrderedDict([
|
||||
(
|
||||
'section1', {
|
||||
'map': {
|
||||
'field1': OrderedDict([
|
||||
('example', 'Example 1')
|
||||
]),
|
||||
},
|
||||
},
|
||||
),
|
||||
(
|
||||
'section2', {
|
||||
'map': OrderedDict([
|
||||
('field2', {'example': 'Example 2'}),
|
||||
('field3', {'example': 'Example 3'}),
|
||||
]),
|
||||
}
|
||||
),
|
||||
])
|
||||
}
|
||||
|
||||
config = module._schema_to_sample_configuration(schema)
|
||||
|
||||
assert config == OrderedDict([
|
||||
(
|
||||
'section1',
|
||||
OrderedDict([
|
||||
('field1', 'Example 1'),
|
||||
]),
|
||||
),
|
||||
(
|
||||
'section2',
|
||||
OrderedDict([
|
||||
('field2', 'Example 2'),
|
||||
('field3', 'Example 3'),
|
||||
]),
|
||||
)
|
||||
])
|
||||
Executable
+7
@@ -0,0 +1,7 @@
|
||||
#!/usr/bin/env sh
|
||||
|
||||
if which black; then
|
||||
black --skip-string-normalization --line-length 100 --check .
|
||||
else
|
||||
echo "Skipping black due to not being installed."
|
||||
fi
|
||||
@@ -6,8 +6,10 @@ set -o nounset
|
||||
# appear to support yet. This script isn't terribly robust. It's intended as a basic tool to ferret
|
||||
# out unsupported Borg options so that they can be considered for addition to borgmatic.
|
||||
|
||||
# Generate a sample borgmatic configuration with all options set.
|
||||
# Generate a sample borgmatic configuration with all options set, and uncomment all options.
|
||||
generate-borgmatic-config --destination temp.yaml
|
||||
cat temp.yaml | sed -e 's/# \S.*$//' | sed -e 's/#//' > temp.yaml.uncommented
|
||||
mv temp.yaml.uncommented temp.yaml
|
||||
|
||||
# For each sub-command (prune, create, and check), collect the Borg command-line flags that result
|
||||
# from running borgmatic with the generated configuration. Then, collect the full set of available
|
||||
@@ -33,9 +35,12 @@ for sub_command in prune create check list info; do
|
||||
| grep -v '^--error$' \
|
||||
| grep -v '^--help$' \
|
||||
| grep -v '^--info$' \
|
||||
| grep -v '^--json$' \
|
||||
| grep -v '^--keep-last$' \
|
||||
| grep -v '^--list$' \
|
||||
| grep -v '^--nobsdflags$' \
|
||||
| grep -v '^--pattern$' \
|
||||
| grep -v '^--progress$' \
|
||||
| grep -v '^--read-special$' \
|
||||
| grep -v '^--repository-only$' \
|
||||
| grep -v '^--show-rc$' \
|
||||
|
||||
@@ -2,13 +2,38 @@
|
||||
|
||||
set -e
|
||||
|
||||
projects_token=${1:-}
|
||||
github_token=${2:-}
|
||||
|
||||
if [[ -z $github_token ]]; then
|
||||
echo "Usage: $0 [projects-token] [github-token]"
|
||||
exit 1
|
||||
fi
|
||||
if [[ ! -f NEWS ]]; then
|
||||
echo "Missing NEWS file. Try running from root of repository."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
version=$(head --lines=1 NEWS)
|
||||
git tag $version
|
||||
git push origin $version
|
||||
git push github $version
|
||||
|
||||
# Build borgmatic and publish to pypi.
|
||||
rm -fr dist
|
||||
python3 setup.py bdist_wheel
|
||||
python3 setup.py sdist
|
||||
twine upload -r pypi dist/borgmatic-*.tar.gz
|
||||
twine upload -r pypi dist/borgmatic-*-py3-none-any.whl
|
||||
|
||||
# Set release changelogs on projects.evoworx.org and GitHub.
|
||||
release_changelog="$(cat NEWS | sed '/^$/q' | grep -v '^\S')"
|
||||
escaped_release_changelog="$(echo "$release_changelog" | sed -z 's/\n/\\n/g' | sed -z 's/\"/\\"/g')"
|
||||
curl --silent --request POST \
|
||||
"https://projects.torsion.org/api/v1/repos/witten/borgmatic/releases?access_token=$projects_token" \
|
||||
--header "Accept: application/json" \
|
||||
--header "Content-Type: application/json" \
|
||||
--data "{\"body\": \"$escaped_release_changelog\", \"draft\": false, \"name\": \"borgmatic $version\", \"prerelease\": false, \"tag_name\": \"$version\"}"
|
||||
|
||||
github-release create --token="$github_token" --owner=witten --repo=borgmatic --tag="$version" \
|
||||
--name="borgmatic $version" --body="$release_changelog"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
from setuptools import setup, find_packages
|
||||
|
||||
|
||||
VERSION = '1.2.4'
|
||||
VERSION = '1.2.12'
|
||||
|
||||
|
||||
setup(
|
||||
@@ -11,7 +11,7 @@ setup(
|
||||
author='Dan Helfman',
|
||||
author_email='witten@torsion.org',
|
||||
url='https://torsion.org/borgmatic',
|
||||
classifiers=(
|
||||
classifiers=[
|
||||
'Development Status :: 5 - Production/Stable',
|
||||
'Environment :: Console',
|
||||
'Intended Audience :: System Administrators',
|
||||
@@ -19,7 +19,7 @@ setup(
|
||||
'Programming Language :: Python',
|
||||
'Topic :: Security :: Cryptography',
|
||||
'Topic :: System :: Archiving :: Backup',
|
||||
),
|
||||
],
|
||||
packages=find_packages(),
|
||||
entry_points={
|
||||
'console_scripts': [
|
||||
@@ -28,17 +28,7 @@ setup(
|
||||
'generate-borgmatic-config = borgmatic.commands.generate_config:main',
|
||||
]
|
||||
},
|
||||
obsoletes=[
|
||||
'atticmatic',
|
||||
],
|
||||
install_requires=(
|
||||
'pykwalify>=1.6.0,<14.06',
|
||||
'ruamel.yaml>0.15.0,<0.16.0',
|
||||
'setuptools',
|
||||
),
|
||||
tests_require=(
|
||||
'flexmock',
|
||||
'pytest',
|
||||
),
|
||||
obsoletes=['atticmatic'],
|
||||
install_requires=('pykwalify>=1.6.0,<14.06', 'ruamel.yaml>0.15.0,<0.16.0', 'setuptools'),
|
||||
include_package_data=True,
|
||||
)
|
||||
|
||||
+21
-3
@@ -1,5 +1,23 @@
|
||||
appdirs==1.4.3
|
||||
atomicwrites==1.2.1
|
||||
attrs==18.2.0
|
||||
black==18.9b0; python_version >= '3.6'
|
||||
Click==7.0
|
||||
coverage==4.5.1
|
||||
docopt==0.6.2
|
||||
flake8==3.5.0
|
||||
flexmock==0.10.2
|
||||
pykwalify==1.6.1
|
||||
pytest==3.7.4
|
||||
pytest-cov==2.5.1
|
||||
mccabe==0.6.1
|
||||
more-itertools==4.3.0
|
||||
pluggy==0.7.1
|
||||
py==1.6.0
|
||||
pycodestyle==2.3.1
|
||||
pyflakes==2.0.0
|
||||
pykwalify==1.7.0
|
||||
pytest==3.8.2
|
||||
pytest-cov==2.6.0
|
||||
python-dateutil==2.7.3
|
||||
PyYAML==3.13
|
||||
ruamel.yaml>0.15.0,<0.16.0
|
||||
six==1.11.0
|
||||
toml==0.10.0
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
|
||||
def generate_configuration(config_path, repository_path):
|
||||
'''
|
||||
Generate borgmatic configuration into a file at the config path, and update the defaults so as
|
||||
to work for testing (including injecting the given repository path and tacking on an encryption
|
||||
passphrase).
|
||||
'''
|
||||
subprocess.check_call(
|
||||
'generate-borgmatic-config --destination {}'.format(config_path).split(' ')
|
||||
)
|
||||
config = (
|
||||
open(config_path)
|
||||
.read()
|
||||
.replace('user@backupserver:sourcehostname.borg', repository_path)
|
||||
.replace('- /home', '- {}'.format(config_path))
|
||||
.replace('- /etc', '')
|
||||
.replace('- /var/log/syslog*', '')
|
||||
+ 'storage:\n encryption_passphrase: "test"'
|
||||
)
|
||||
config_file = open(config_path, 'w')
|
||||
config_file.write(config)
|
||||
config_file.close()
|
||||
|
||||
|
||||
def test_borgmatic_command():
|
||||
# Create a Borg repository.
|
||||
temporary_directory = tempfile.mkdtemp()
|
||||
repository_path = os.path.join(temporary_directory, 'test.borg')
|
||||
|
||||
try:
|
||||
config_path = os.path.join(temporary_directory, 'test.yaml')
|
||||
generate_configuration(config_path, repository_path)
|
||||
|
||||
subprocess.check_call(
|
||||
'borgmatic -v 2 --config {} --init --encryption repokey'.format(config_path).split(' ')
|
||||
)
|
||||
|
||||
# Run borgmatic to generate a backup archive, and then list it to make sure it exists.
|
||||
subprocess.check_call('borgmatic --config {}'.format(config_path).split(' '))
|
||||
output = subprocess.check_output(
|
||||
'borgmatic --config {} --list --json'.format(config_path).split(' ')
|
||||
).decode(sys.stdout.encoding)
|
||||
parsed_output = json.loads(output)
|
||||
|
||||
assert len(parsed_output) == 1
|
||||
assert len(parsed_output[0]['archives']) == 1
|
||||
|
||||
# Also exercise the info flag.
|
||||
output = subprocess.check_output(
|
||||
'borgmatic --config {} --info --json'.format(config_path).split(' ')
|
||||
).decode(sys.stdout.encoding)
|
||||
parsed_output = json.loads(output)
|
||||
|
||||
assert len(parsed_output) == 1
|
||||
assert 'repository' in parsed_output[0]
|
||||
finally:
|
||||
shutil.rmtree(temporary_directory)
|
||||
@@ -0,0 +1,158 @@
|
||||
from flexmock import flexmock
|
||||
import pytest
|
||||
|
||||
from borgmatic.commands import borgmatic as module
|
||||
|
||||
|
||||
def test_parse_arguments_with_no_arguments_uses_defaults():
|
||||
config_paths = ['default']
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(config_paths)
|
||||
|
||||
parser = module.parse_arguments()
|
||||
|
||||
assert parser.config_paths == config_paths
|
||||
assert parser.excludes_filename is None
|
||||
assert parser.verbosity is 0
|
||||
assert parser.json is False
|
||||
|
||||
|
||||
def test_parse_arguments_with_multiple_config_paths_parses_as_list():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments('--config', 'myconfig', 'otherconfig')
|
||||
|
||||
assert parser.config_paths == ['myconfig', 'otherconfig']
|
||||
assert parser.verbosity is 0
|
||||
|
||||
|
||||
def test_parse_arguments_with_verbosity_overrides_default():
|
||||
config_paths = ['default']
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(config_paths)
|
||||
|
||||
parser = module.parse_arguments('--verbosity', '1')
|
||||
|
||||
assert parser.config_paths == config_paths
|
||||
assert parser.excludes_filename is None
|
||||
assert parser.verbosity == 1
|
||||
|
||||
|
||||
def test_parse_arguments_with_json_overrides_default():
|
||||
parser = module.parse_arguments('--list', '--json')
|
||||
assert parser.json is True
|
||||
|
||||
|
||||
def test_parse_arguments_with_no_actions_defaults_to_all_actions_enabled():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments()
|
||||
|
||||
assert parser.prune is True
|
||||
assert parser.create is True
|
||||
assert parser.check is True
|
||||
|
||||
|
||||
def test_parse_arguments_with_prune_action_leaves_other_actions_disabled():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments('--prune')
|
||||
|
||||
assert parser.prune is True
|
||||
assert parser.create is False
|
||||
assert parser.check is False
|
||||
|
||||
|
||||
def test_parse_arguments_with_multiple_actions_leaves_other_action_disabled():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
parser = module.parse_arguments('--create', '--check')
|
||||
|
||||
assert parser.prune is False
|
||||
assert parser.create is True
|
||||
assert parser.check is True
|
||||
|
||||
|
||||
def test_parse_arguments_with_invalid_arguments_exits():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(SystemExit):
|
||||
module.parse_arguments('--posix-me-harder')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_deprecated_excludes_option():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--excludes', 'myexcludes')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_encryption_mode_without_init():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--encryption', 'repokey')
|
||||
|
||||
|
||||
def test_parse_arguments_requires_encryption_mode_with_init():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--init')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_append_only_without_init():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--append-only')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_storage_quota_without_init():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--storage-quota', '5G')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_init_and_prune():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--prune')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_init_and_create():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--create')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_init_and_dry_run():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--dry-run')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_progress_and_create():
|
||||
module.parse_arguments('--progress', '--create', '--list')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_progress_without_create():
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--progress', '--list')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_json_with_list_or_info():
|
||||
module.parse_arguments('--list', '--json')
|
||||
module.parse_arguments('--info', '--json')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_json_without_list_or_info():
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--json')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_json_with_both_list_and_info():
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--list', '--info', '--json')
|
||||
+6
-3
@@ -20,9 +20,12 @@ def test_parse_arguments_with_filename_arguments_overrides_defaults():
|
||||
flexmock(os.path).should_receive('exists').and_return(True)
|
||||
|
||||
parser = module.parse_arguments(
|
||||
'--source-config', 'config',
|
||||
'--source-excludes', 'excludes',
|
||||
'--destination-config', 'config.yaml',
|
||||
'--source-config',
|
||||
'config',
|
||||
'--source-excludes',
|
||||
'excludes',
|
||||
'--destination-config',
|
||||
'config.yaml',
|
||||
)
|
||||
|
||||
assert parser.source_config_filename == 'config'
|
||||
+1
@@ -6,6 +6,7 @@ def test_parse_arguments_with_no_arguments_uses_defaults():
|
||||
|
||||
assert parser.destination_filename == module.DEFAULT_DESTINATION_CONFIG_FILENAME
|
||||
|
||||
|
||||
def test_parse_arguments_with_filename_argument_overrides_defaults():
|
||||
parser = module.parse_arguments('--destination', 'config.yaml')
|
||||
|
||||
+71
-11
@@ -11,11 +11,74 @@ from borgmatic.config import generate as module
|
||||
def test_insert_newline_before_comment_does_not_raise():
|
||||
field_name = 'foo'
|
||||
config = module.yaml.comments.CommentedMap([(field_name, 33)])
|
||||
config.yaml_set_comment_before_after_key(key=field_name, before='Comment',)
|
||||
config.yaml_set_comment_before_after_key(key=field_name, before='Comment')
|
||||
|
||||
module._insert_newline_before_comment(config, field_name)
|
||||
|
||||
|
||||
def test_comment_out_line_skips_blank_line():
|
||||
line = ' \n'
|
||||
|
||||
assert module._comment_out_line(line) == line
|
||||
|
||||
|
||||
def test_comment_out_line_skips_already_commented_out_line():
|
||||
line = ' # foo'
|
||||
|
||||
assert module._comment_out_line(line) == line
|
||||
|
||||
|
||||
def test_comment_out_line_comments_section_name():
|
||||
line = 'figgy-pudding:'
|
||||
|
||||
assert module._comment_out_line(line) == '#' + line
|
||||
|
||||
|
||||
def test_comment_out_line_comments_indented_option():
|
||||
line = ' enabled: true'
|
||||
|
||||
assert module._comment_out_line(line) == ' #enabled: true'
|
||||
|
||||
|
||||
def test_comment_out_optional_configuration_comments_optional_config_only():
|
||||
flexmock(module)._comment_out_line = lambda line: '#' + line
|
||||
config = '''
|
||||
foo:
|
||||
bar:
|
||||
- baz
|
||||
- quux
|
||||
|
||||
location:
|
||||
repositories:
|
||||
- one
|
||||
- two
|
||||
|
||||
other: thing
|
||||
'''
|
||||
|
||||
expected_config = '''
|
||||
#foo:
|
||||
# bar:
|
||||
# - baz
|
||||
# - quux
|
||||
#
|
||||
location:
|
||||
repositories:
|
||||
- one
|
||||
- two
|
||||
#
|
||||
# other: thing
|
||||
'''
|
||||
|
||||
assert module._comment_out_optional_configuration(config.strip()) == expected_config.strip()
|
||||
|
||||
|
||||
def test_render_configuration_does_not_raise():
|
||||
flexmock(module.yaml).should_receive('round_trip_dump')
|
||||
|
||||
module._render_configuration({})
|
||||
|
||||
|
||||
def test_write_configuration_does_not_raise():
|
||||
flexmock(os.path).should_receive('exists').and_return(False)
|
||||
flexmock(os).should_receive('makedirs')
|
||||
@@ -23,14 +86,14 @@ def test_write_configuration_does_not_raise():
|
||||
builtins.should_receive('open').and_return(StringIO())
|
||||
flexmock(os).should_receive('chmod')
|
||||
|
||||
module.write_configuration('config.yaml', {})
|
||||
module.write_configuration('config.yaml', 'config: yaml')
|
||||
|
||||
|
||||
def test_write_configuration_with_already_existing_file_raises():
|
||||
flexmock(os.path).should_receive('exists').and_return(True)
|
||||
|
||||
with pytest.raises(FileExistsError):
|
||||
module.write_configuration('config.yaml', {})
|
||||
module.write_configuration('config.yaml', 'config: yaml')
|
||||
|
||||
|
||||
def test_write_configuration_with_already_existing_directory_does_not_raise():
|
||||
@@ -40,18 +103,13 @@ def test_write_configuration_with_already_existing_directory_does_not_raise():
|
||||
builtins.should_receive('open').and_return(StringIO())
|
||||
flexmock(os).should_receive('chmod')
|
||||
|
||||
module.write_configuration('config.yaml', {})
|
||||
module.write_configuration('config.yaml', 'config: yaml')
|
||||
|
||||
|
||||
def test_add_comments_to_configuration_does_not_raise():
|
||||
# Ensure that it can deal with fields both in the schema and missing from the schema.
|
||||
config = module.yaml.comments.CommentedMap([('foo', 33), ('bar', 44), ('baz', 55)])
|
||||
schema = {
|
||||
'map': {
|
||||
'foo': {'desc': 'Foo'},
|
||||
'bar': {'desc': 'Bar'},
|
||||
}
|
||||
}
|
||||
schema = {'map': {'foo': {'desc': 'Foo'}, 'bar': {'desc': 'Bar'}}}
|
||||
|
||||
module.add_comments_to_configuration(config, schema)
|
||||
|
||||
@@ -59,7 +117,9 @@ def test_add_comments_to_configuration_does_not_raise():
|
||||
def test_generate_sample_configuration_does_not_raise():
|
||||
builtins = flexmock(sys.modules['builtins'])
|
||||
builtins.should_receive('open').with_args('schema.yaml').and_return('')
|
||||
flexmock(module).should_receive('write_configuration')
|
||||
flexmock(module).should_receive('_schema_to_sample_configuration')
|
||||
flexmock(module).should_receive('_render_configuration')
|
||||
flexmock(module).should_receive('_comment_out_optional_configuration')
|
||||
flexmock(module).should_receive('write_configuration')
|
||||
|
||||
module.generate_sample_configuration('config.yaml', 'schema.yaml')
|
||||
+3
-8
@@ -8,17 +8,12 @@ from borgmatic.config import legacy as module
|
||||
|
||||
def test_parse_section_options_with_punctuation_should_return_section_options():
|
||||
parser = module.RawConfigParser()
|
||||
parser.readfp(StringIO('[section]\nfoo: {}\n'.format(string.punctuation)))
|
||||
parser.read_file(StringIO('[section]\nfoo: {}\n'.format(string.punctuation)))
|
||||
|
||||
section_format = module.Section_format(
|
||||
'section',
|
||||
(module.Config_option('foo', str, required=True),),
|
||||
'section', (module.Config_option('foo', str, required=True),)
|
||||
)
|
||||
|
||||
config = module.parse_section_options(parser, section_format)
|
||||
|
||||
assert config == OrderedDict(
|
||||
(
|
||||
('foo', string.punctuation),
|
||||
)
|
||||
)
|
||||
assert config == OrderedDict((('foo', string.punctuation),))
|
||||
+6
-5
@@ -1,7 +1,6 @@
|
||||
import io
|
||||
import string
|
||||
import sys
|
||||
import os
|
||||
|
||||
from flexmock import flexmock
|
||||
import pytest
|
||||
@@ -10,7 +9,7 @@ from borgmatic.config import validate as module
|
||||
|
||||
|
||||
def test_schema_filename_returns_plausable_path():
|
||||
schema_path = module.schema_filename()
|
||||
schema_path = module.schema_filename()
|
||||
|
||||
assert schema_path.endswith('/schema.yaml')
|
||||
|
||||
@@ -75,7 +74,9 @@ def test_parse_configuration_passes_through_quoted_punctuation():
|
||||
|
||||
repositories:
|
||||
- "{}.borg"
|
||||
'''.format(escaped_punctuation)
|
||||
'''.format(
|
||||
escaped_punctuation
|
||||
)
|
||||
)
|
||||
|
||||
result = module.parse_configuration('config.yaml', 'schema.yaml')
|
||||
@@ -84,7 +85,7 @@ def test_parse_configuration_passes_through_quoted_punctuation():
|
||||
'location': {
|
||||
'source_directories': ['/home'],
|
||||
'repositories': ['{}.borg'.format(string.punctuation)],
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -111,7 +112,7 @@ def test_parse_configuration_with_schema_lacking_examples_does_not_raise():
|
||||
required: true
|
||||
seq:
|
||||
- type: scalar
|
||||
'''
|
||||
''',
|
||||
)
|
||||
|
||||
module.parse_configuration('config.yaml', 'schema.yaml')
|
||||
@@ -1,11 +1,13 @@
|
||||
from subprocess import STDOUT
|
||||
import logging, sys
|
||||
import logging
|
||||
import sys
|
||||
|
||||
from flexmock import flexmock
|
||||
import pytest
|
||||
|
||||
from borgmatic.borg import check as module
|
||||
from borgmatic.tests.unit.test_verbosity import insert_logging_mock
|
||||
from ..test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
subprocess = flexmock(module.subprocess)
|
||||
@@ -114,19 +116,16 @@ def test_check_archives_calls_borg_with_parameters(checks):
|
||||
check_last = flexmock()
|
||||
consistency_config = {'check_last': check_last}
|
||||
flexmock(module).should_receive('_parse_checks').and_return(checks)
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, None).and_return(())
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(
|
||||
checks, check_last, None
|
||||
).and_return(())
|
||||
stdout = flexmock()
|
||||
insert_subprocess_mock(
|
||||
('borg', 'check', 'repo'),
|
||||
stdout=stdout, stderr=STDOUT,
|
||||
)
|
||||
insert_subprocess_mock(('borg', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
|
||||
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
|
||||
flexmock(module.os).should_receive('devnull')
|
||||
|
||||
module.check_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
consistency_config=consistency_config,
|
||||
repository='repo', storage_config={}, consistency_config=consistency_config
|
||||
)
|
||||
|
||||
|
||||
@@ -140,9 +139,7 @@ def test_check_archives_with_extract_check_calls_extract_only():
|
||||
insert_subprocess_never()
|
||||
|
||||
module.check_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
consistency_config=consistency_config,
|
||||
repository='repo', storage_config={}, consistency_config=consistency_config
|
||||
)
|
||||
|
||||
|
||||
@@ -152,15 +149,10 @@ def test_check_archives_with_log_info_calls_borg_with_info_parameter():
|
||||
flexmock(module).should_receive('_parse_checks').and_return(checks)
|
||||
flexmock(module).should_receive('_make_check_flags').and_return(())
|
||||
insert_logging_mock(logging.INFO)
|
||||
insert_subprocess_mock(
|
||||
('borg', 'check', 'repo', '--info'),
|
||||
stdout=None, stderr=STDOUT,
|
||||
)
|
||||
insert_subprocess_mock(('borg', 'check', 'repo', '--info'), stdout=None, stderr=STDOUT)
|
||||
|
||||
module.check_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
consistency_config=consistency_config,
|
||||
repository='repo', storage_config={}, consistency_config=consistency_config
|
||||
)
|
||||
|
||||
|
||||
@@ -171,14 +163,11 @@ def test_check_archives_with_log_debug_calls_borg_with_debug_parameter():
|
||||
flexmock(module).should_receive('_make_check_flags').and_return(())
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
insert_subprocess_mock(
|
||||
('borg', 'check', 'repo', '--debug', '--show-rc'),
|
||||
stdout=None, stderr=STDOUT,
|
||||
('borg', 'check', 'repo', '--debug', '--show-rc'), stdout=None, stderr=STDOUT
|
||||
)
|
||||
|
||||
module.check_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
consistency_config=consistency_config,
|
||||
repository='repo', storage_config={}, consistency_config=consistency_config
|
||||
)
|
||||
|
||||
|
||||
@@ -188,9 +177,7 @@ def test_check_archives_without_any_checks_bails():
|
||||
insert_subprocess_never()
|
||||
|
||||
module.check_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
consistency_config=consistency_config,
|
||||
repository='repo', storage_config={}, consistency_config=consistency_config
|
||||
)
|
||||
|
||||
|
||||
@@ -199,12 +186,11 @@ def test_check_archives_with_local_path_calls_borg_via_local_path():
|
||||
check_last = flexmock()
|
||||
consistency_config = {'check_last': check_last}
|
||||
flexmock(module).should_receive('_parse_checks').and_return(checks)
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, None).and_return(())
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(
|
||||
checks, check_last, None
|
||||
).and_return(())
|
||||
stdout = flexmock()
|
||||
insert_subprocess_mock(
|
||||
('borg1', 'check', 'repo'),
|
||||
stdout=stdout, stderr=STDOUT,
|
||||
)
|
||||
insert_subprocess_mock(('borg1', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
|
||||
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
|
||||
flexmock(module.os).should_receive('devnull')
|
||||
|
||||
@@ -221,11 +207,12 @@ def test_check_archives_with_remote_path_calls_borg_with_remote_path_parameters(
|
||||
check_last = flexmock()
|
||||
consistency_config = {'check_last': check_last}
|
||||
flexmock(module).should_receive('_parse_checks').and_return(checks)
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, None).and_return(())
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(
|
||||
checks, check_last, None
|
||||
).and_return(())
|
||||
stdout = flexmock()
|
||||
insert_subprocess_mock(
|
||||
('borg', 'check', 'repo', '--remote-path', 'borg1'),
|
||||
stdout=stdout, stderr=STDOUT,
|
||||
('borg', 'check', 'repo', '--remote-path', 'borg1'), stdout=stdout, stderr=STDOUT
|
||||
)
|
||||
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
|
||||
flexmock(module.os).should_receive('devnull')
|
||||
@@ -243,19 +230,18 @@ def test_check_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
|
||||
check_last = flexmock()
|
||||
consistency_config = {'check_last': check_last}
|
||||
flexmock(module).should_receive('_parse_checks').and_return(checks)
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, None).and_return(())
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(
|
||||
checks, check_last, None
|
||||
).and_return(())
|
||||
stdout = flexmock()
|
||||
insert_subprocess_mock(
|
||||
('borg', 'check', 'repo', '--lock-wait', '5'),
|
||||
stdout=stdout, stderr=STDOUT,
|
||||
('borg', 'check', 'repo', '--lock-wait', '5'), stdout=stdout, stderr=STDOUT
|
||||
)
|
||||
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
|
||||
flexmock(module.os).should_receive('devnull')
|
||||
|
||||
module.check_archives(
|
||||
repository='repo',
|
||||
storage_config={'lock_wait': 5},
|
||||
consistency_config=consistency_config,
|
||||
repository='repo', storage_config={'lock_wait': 5}, consistency_config=consistency_config
|
||||
)
|
||||
|
||||
|
||||
@@ -265,18 +251,15 @@ def test_check_archives_with_retention_prefix():
|
||||
prefix = 'foo-'
|
||||
consistency_config = {'check_last': check_last, 'prefix': prefix}
|
||||
flexmock(module).should_receive('_parse_checks').and_return(checks)
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last, prefix).and_return(())
|
||||
flexmock(module).should_receive('_make_check_flags').with_args(
|
||||
checks, check_last, prefix
|
||||
).and_return(())
|
||||
stdout = flexmock()
|
||||
insert_subprocess_mock(
|
||||
('borg', 'check', 'repo'),
|
||||
stdout=stdout, stderr=STDOUT,
|
||||
)
|
||||
insert_subprocess_mock(('borg', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
|
||||
|
||||
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
|
||||
flexmock(module.os).should_receive('devnull')
|
||||
|
||||
module.check_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
consistency_config=consistency_config,
|
||||
repository='repo', storage_config={}, consistency_config=consistency_config
|
||||
)
|
||||
@@ -1,55 +1,9 @@
|
||||
import logging, os
|
||||
import logging
|
||||
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.borg import create as module
|
||||
from borgmatic.tests.unit.test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def test_initialize_environment_with_passcommand_should_set_environment():
|
||||
orig_environ = os.environ
|
||||
|
||||
try:
|
||||
os.environ = {}
|
||||
module.initialize_environment({'encryption_passcommand': 'command'})
|
||||
assert os.environ.get('BORG_PASSCOMMAND') == 'command'
|
||||
finally:
|
||||
os.environ = orig_environ
|
||||
|
||||
|
||||
def test_initialize_environment_with_passphrase_should_set_environment():
|
||||
orig_environ = os.environ
|
||||
|
||||
try:
|
||||
os.environ = {}
|
||||
module.initialize_environment({'encryption_passphrase': 'pass'})
|
||||
assert os.environ.get('BORG_PASSPHRASE') == 'pass'
|
||||
finally:
|
||||
os.environ = orig_environ
|
||||
|
||||
|
||||
def test_initialize_environment_with_ssh_command_should_set_environment():
|
||||
orig_environ = os.environ
|
||||
|
||||
try:
|
||||
os.environ = {}
|
||||
module.initialize_environment({'ssh_command': 'ssh -C'})
|
||||
assert os.environ.get('BORG_RSH') == 'ssh -C'
|
||||
finally:
|
||||
os.environ = orig_environ
|
||||
|
||||
|
||||
def test_initialize_environment_without_configuration_should_not_set_environment():
|
||||
orig_environ = os.environ
|
||||
|
||||
try:
|
||||
os.environ = {}
|
||||
module.initialize_environment({})
|
||||
assert os.environ.get('BORG_PASSCOMMAND') == None
|
||||
assert os.environ.get('BORG_PASSPHRASE') == None
|
||||
assert os.environ.get('BORG_RSH') == None
|
||||
finally:
|
||||
os.environ = orig_environ
|
||||
from ..test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def test_expand_directory_with_basic_path_passes_it_through():
|
||||
@@ -71,8 +25,12 @@ def test_expand_directory_with_glob_expands():
|
||||
|
||||
|
||||
def test_expand_directories_flattens_expanded_directories():
|
||||
flexmock(module).should_receive('_expand_directory').with_args('~/foo').and_return(['/root/foo'])
|
||||
flexmock(module).should_receive('_expand_directory').with_args('bar*').and_return(['bar', 'barf'])
|
||||
flexmock(module).should_receive('_expand_directory').with_args('~/foo').and_return(
|
||||
['/root/foo']
|
||||
)
|
||||
flexmock(module).should_receive('_expand_directory').with_args('bar*').and_return(
|
||||
['bar', 'barf']
|
||||
)
|
||||
|
||||
paths = module._expand_directories(('~/foo', 'bar*'))
|
||||
|
||||
@@ -86,11 +44,7 @@ def test_expand_directories_considers_none_as_no_directories():
|
||||
|
||||
|
||||
def test_write_pattern_file_does_not_raise():
|
||||
temporary_file = flexmock(
|
||||
name='filename',
|
||||
write=lambda mode: None,
|
||||
flush=lambda: None,
|
||||
)
|
||||
temporary_file = flexmock(name='filename', write=lambda mode: None, flush=lambda: None)
|
||||
flexmock(module.tempfile).should_receive('NamedTemporaryFile').and_return(temporary_file)
|
||||
|
||||
module._write_pattern_file(['exclude'])
|
||||
@@ -107,8 +61,7 @@ def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
|
||||
def test_make_pattern_flags_includes_pattern_filename_when_given():
|
||||
pattern_flags = module._make_pattern_flags(
|
||||
location_config={'patterns': ['R /', '- /var']},
|
||||
pattern_filename='/tmp/patterns',
|
||||
location_config={'patterns': ['R /', '- /var']}, pattern_filename='/tmp/patterns'
|
||||
)
|
||||
|
||||
assert pattern_flags == ('--patterns-from', '/tmp/patterns')
|
||||
@@ -116,7 +69,7 @@ def test_make_pattern_flags_includes_pattern_filename_when_given():
|
||||
|
||||
def test_make_pattern_flags_includes_patterns_from_filenames_when_in_config():
|
||||
pattern_flags = module._make_pattern_flags(
|
||||
location_config={'patterns_from': ['patterns', 'other']},
|
||||
location_config={'patterns_from': ['patterns', 'other']}
|
||||
)
|
||||
|
||||
assert pattern_flags == ('--patterns-from', 'patterns', '--patterns-from', 'other')
|
||||
@@ -124,25 +77,21 @@ def test_make_pattern_flags_includes_patterns_from_filenames_when_in_config():
|
||||
|
||||
def test_make_pattern_flags_includes_both_filenames_when_patterns_given_and_patterns_from_in_config():
|
||||
pattern_flags = module._make_pattern_flags(
|
||||
location_config={'patterns_from': ['patterns']},
|
||||
pattern_filename='/tmp/patterns',
|
||||
location_config={'patterns_from': ['patterns']}, pattern_filename='/tmp/patterns'
|
||||
)
|
||||
|
||||
assert pattern_flags == ('--patterns-from', 'patterns', '--patterns-from', '/tmp/patterns')
|
||||
|
||||
|
||||
def test_make_pattern_flags_considers_none_patterns_from_filenames_as_empty():
|
||||
pattern_flags = module._make_pattern_flags(
|
||||
location_config={'patterns_from': None},
|
||||
)
|
||||
pattern_flags = module._make_pattern_flags(location_config={'patterns_from': None})
|
||||
|
||||
assert pattern_flags == ()
|
||||
|
||||
|
||||
def test_make_exclude_flags_includes_exclude_patterns_filename_when_given():
|
||||
exclude_flags = module._make_exclude_flags(
|
||||
location_config={'exclude_patterns': ['*.pyc', '/var']},
|
||||
exclude_filename='/tmp/excludes',
|
||||
location_config={'exclude_patterns': ['*.pyc', '/var']}, exclude_filename='/tmp/excludes'
|
||||
)
|
||||
|
||||
assert exclude_flags == ('--exclude-from', '/tmp/excludes')
|
||||
@@ -151,7 +100,7 @@ def test_make_exclude_flags_includes_exclude_patterns_filename_when_given():
|
||||
def test_make_exclude_flags_includes_exclude_from_filenames_when_in_config():
|
||||
|
||||
exclude_flags = module._make_exclude_flags(
|
||||
location_config={'exclude_from': ['excludes', 'other']},
|
||||
location_config={'exclude_from': ['excludes', 'other']}
|
||||
)
|
||||
|
||||
assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', 'other')
|
||||
@@ -159,41 +108,32 @@ def test_make_exclude_flags_includes_exclude_from_filenames_when_in_config():
|
||||
|
||||
def test_make_exclude_flags_includes_both_filenames_when_patterns_given_and_exclude_from_in_config():
|
||||
exclude_flags = module._make_exclude_flags(
|
||||
location_config={'exclude_from': ['excludes']},
|
||||
exclude_filename='/tmp/excludes',
|
||||
location_config={'exclude_from': ['excludes']}, exclude_filename='/tmp/excludes'
|
||||
)
|
||||
|
||||
assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', '/tmp/excludes')
|
||||
|
||||
|
||||
def test_make_exclude_flags_considers_none_exclude_from_filenames_as_empty():
|
||||
exclude_flags = module._make_exclude_flags(
|
||||
location_config={'exclude_from': None},
|
||||
)
|
||||
exclude_flags = module._make_exclude_flags(location_config={'exclude_from': None})
|
||||
|
||||
assert exclude_flags == ()
|
||||
|
||||
|
||||
def test_make_exclude_flags_includes_exclude_caches_when_true_in_config():
|
||||
exclude_flags = module._make_exclude_flags(
|
||||
location_config={'exclude_caches': True},
|
||||
)
|
||||
exclude_flags = module._make_exclude_flags(location_config={'exclude_caches': True})
|
||||
|
||||
assert exclude_flags == ('--exclude-caches',)
|
||||
|
||||
|
||||
def test_make_exclude_flags_does_not_include_exclude_caches_when_false_in_config():
|
||||
exclude_flags = module._make_exclude_flags(
|
||||
location_config={'exclude_caches': False},
|
||||
)
|
||||
exclude_flags = module._make_exclude_flags(location_config={'exclude_caches': False})
|
||||
|
||||
assert exclude_flags == ()
|
||||
|
||||
|
||||
def test_make_exclude_flags_includes_exclude_if_present_when_in_config():
|
||||
exclude_flags = module._make_exclude_flags(
|
||||
location_config={'exclude_if_present': 'exclude_me'},
|
||||
)
|
||||
exclude_flags = module._make_exclude_flags(location_config={'exclude_if_present': 'exclude_me'})
|
||||
|
||||
assert exclude_flags == ('--exclude-if-present', 'exclude_me')
|
||||
|
||||
@@ -230,7 +170,9 @@ def test_create_archive_calls_borg_with_parameters():
|
||||
def test_create_archive_with_patterns_calls_borg_with_patterns():
|
||||
pattern_flags = ('--patterns-from', 'patterns')
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(flexmock(name='/tmp/patterns')).and_return(None)
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(
|
||||
flexmock(name='/tmp/patterns')
|
||||
).and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(pattern_flags)
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
insert_subprocess_mock(CREATE_COMMAND + pattern_flags)
|
||||
@@ -249,8 +191,12 @@ def test_create_archive_with_patterns_calls_borg_with_patterns():
|
||||
|
||||
def test_create_archive_with_exclude_patterns_calls_borg_with_excludes():
|
||||
exclude_flags = ('--exclude-from', 'excludes')
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(('exclude',))
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None).and_return(flexmock(name='/tmp/excludes'))
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(
|
||||
('exclude',)
|
||||
)
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None).and_return(
|
||||
flexmock(name='/tmp/excludes')
|
||||
)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(exclude_flags)
|
||||
insert_subprocess_mock(CREATE_COMMAND + exclude_flags)
|
||||
@@ -273,9 +219,9 @@ def test_create_archive_with_log_info_calls_borg_with_info_parameter():
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
insert_subprocess_mock(CREATE_COMMAND + ('--list', '--filter', 'AME', '--info', '--stats',))
|
||||
insert_subprocess_mock(CREATE_COMMAND + ('--list', '--filter', 'AME-', '--info', '--stats'))
|
||||
insert_logging_mock(logging.INFO)
|
||||
|
||||
|
||||
module.create_archive(
|
||||
dry_run=False,
|
||||
repository='repo',
|
||||
@@ -293,7 +239,9 @@ def test_create_archive_with_log_debug_calls_borg_with_debug_parameter():
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
insert_subprocess_mock(CREATE_COMMAND + ('--list', '--filter', 'AME','--stats', '--debug', '--show-rc'))
|
||||
insert_subprocess_mock(
|
||||
CREATE_COMMAND + ('--list', '--filter', 'AME-', '--stats', '--debug', '--show-rc')
|
||||
)
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.create_archive(
|
||||
@@ -336,7 +284,7 @@ def test_create_archive_with_dry_run_and_log_info_calls_borg_without_stats_param
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
insert_subprocess_mock(CREATE_COMMAND + ('--list', '--filter', 'AME', '--info', '--dry-run'))
|
||||
insert_subprocess_mock(CREATE_COMMAND + ('--list', '--filter', 'AME-', '--info', '--dry-run'))
|
||||
insert_logging_mock(logging.INFO)
|
||||
|
||||
module.create_archive(
|
||||
@@ -359,7 +307,9 @@ def test_create_archive_with_dry_run_and_log_debug_calls_borg_without_stats_para
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
insert_subprocess_mock(CREATE_COMMAND + ('--list', '--filter', 'AME', '--debug', '--show-rc', '--dry-run'))
|
||||
insert_subprocess_mock(
|
||||
CREATE_COMMAND + ('--list', '--filter', 'AME-', '--debug', '--show-rc', '--dry-run')
|
||||
)
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.create_archive(
|
||||
@@ -393,6 +343,25 @@ def test_create_archive_with_checkpoint_interval_calls_borg_with_checkpoint_inte
|
||||
)
|
||||
|
||||
|
||||
def test_create_archive_with_chunker_params_calls_borg_with_chunker_params_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
insert_subprocess_mock(CREATE_COMMAND + ('--chunker-params', '1,2,3,4'))
|
||||
|
||||
module.create_archive(
|
||||
dry_run=False,
|
||||
repository='repo',
|
||||
location_config={
|
||||
'source_directories': ['foo', 'bar'],
|
||||
'repositories': ['repo'],
|
||||
'exclude_patterns': None,
|
||||
},
|
||||
storage_config={'chunker_params': '1,2,3,4'},
|
||||
)
|
||||
|
||||
|
||||
def test_create_archive_with_compression_calls_borg_with_compression_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
@@ -625,16 +594,20 @@ def test_create_archive_with_json_calls_borg_with_json_parameter():
|
||||
'exclude_patterns': None,
|
||||
},
|
||||
storage_config={},
|
||||
json=True
|
||||
json=True,
|
||||
)
|
||||
|
||||
|
||||
def test_create_archive_with_source_directories_glob_expands():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food')).and_return(
|
||||
()
|
||||
)
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food'))
|
||||
insert_subprocess_mock(
|
||||
('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food')
|
||||
)
|
||||
flexmock(module.glob).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
|
||||
|
||||
module.create_archive(
|
||||
@@ -670,11 +643,15 @@ def test_create_archive_with_non_matching_source_directories_glob_passes_through
|
||||
|
||||
|
||||
def test_create_archive_with_glob_calls_borg_with_expanded_directories():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food')).and_return(
|
||||
()
|
||||
)
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
insert_subprocess_mock(('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food'))
|
||||
insert_subprocess_mock(
|
||||
('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food')
|
||||
)
|
||||
|
||||
module.create_archive(
|
||||
dry_run=False,
|
||||
@@ -703,9 +680,7 @@ def test_create_archive_with_archive_name_format_calls_borg_with_archive_name():
|
||||
'repositories': ['repo'],
|
||||
'exclude_patterns': None,
|
||||
},
|
||||
storage_config={
|
||||
'archive_name_format': 'ARCHIVE_NAME',
|
||||
},
|
||||
storage_config={'archive_name_format': 'ARCHIVE_NAME'},
|
||||
)
|
||||
|
||||
|
||||
@@ -724,7 +699,5 @@ def test_create_archive_with_archive_name_format_accepts_borg_placeholders():
|
||||
'repositories': ['repo'],
|
||||
'exclude_patterns': None,
|
||||
},
|
||||
storage_config={
|
||||
'archive_name_format': 'Documents_{hostname}-{now}',
|
||||
},
|
||||
storage_config={'archive_name_format': 'Documents_{hostname}-{now}'},
|
||||
)
|
||||
@@ -0,0 +1,49 @@
|
||||
import os
|
||||
|
||||
from borgmatic.borg import environment as module
|
||||
|
||||
|
||||
def test_initialize_with_passcommand_should_set_environment():
|
||||
orig_environ = os.environ
|
||||
|
||||
try:
|
||||
os.environ = {}
|
||||
module.initialize({'encryption_passcommand': 'command'})
|
||||
assert os.environ.get('BORG_PASSCOMMAND') == 'command'
|
||||
finally:
|
||||
os.environ = orig_environ
|
||||
|
||||
|
||||
def test_initialize_with_passphrase_should_set_environment():
|
||||
orig_environ = os.environ
|
||||
|
||||
try:
|
||||
os.environ = {}
|
||||
module.initialize({'encryption_passphrase': 'pass'})
|
||||
assert os.environ.get('BORG_PASSPHRASE') == 'pass'
|
||||
finally:
|
||||
os.environ = orig_environ
|
||||
|
||||
|
||||
def test_initialize_with_ssh_command_should_set_environment():
|
||||
orig_environ = os.environ
|
||||
|
||||
try:
|
||||
os.environ = {}
|
||||
module.initialize({'ssh_command': 'ssh -C'})
|
||||
assert os.environ.get('BORG_RSH') == 'ssh -C'
|
||||
finally:
|
||||
os.environ = orig_environ
|
||||
|
||||
|
||||
def test_initialize_without_configuration_should_not_set_environment():
|
||||
orig_environ = os.environ
|
||||
|
||||
try:
|
||||
os.environ = {}
|
||||
module.initialize({})
|
||||
assert os.environ.get('BORG_PASSCOMMAND') is None
|
||||
assert os.environ.get('BORG_PASSPHRASE') is None
|
||||
assert os.environ.get('BORG_RSH') is None
|
||||
finally:
|
||||
os.environ = orig_environ
|
||||
@@ -1,10 +1,10 @@
|
||||
import logging, sys
|
||||
import logging
|
||||
import sys
|
||||
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.borg import extract as module
|
||||
from borgmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
|
||||
from borgmatic.tests.unit.test_verbosity import insert_logging_mock
|
||||
from ..test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
@@ -19,54 +19,40 @@ def insert_subprocess_never():
|
||||
|
||||
def insert_subprocess_check_output_mock(check_output_command, result, **kwargs):
|
||||
subprocess = flexmock(module.subprocess)
|
||||
subprocess.should_receive('check_output').with_args(check_output_command, **kwargs).and_return(result).once()
|
||||
subprocess.should_receive('check_output').with_args(check_output_command, **kwargs).and_return(
|
||||
result
|
||||
).once()
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_should_call_borg_with_last_archive():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg', 'list', '--short', 'repo'),
|
||||
result='archive1\narchive2\n'.encode('utf-8'),
|
||||
)
|
||||
insert_subprocess_mock(
|
||||
('borg', 'extract', '--dry-run', 'repo::archive2'),
|
||||
('borg', 'list', '--short', 'repo'), result='archive1\narchive2\n'.encode('utf-8')
|
||||
)
|
||||
insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2'))
|
||||
|
||||
module.extract_last_archive_dry_run(
|
||||
repository='repo',
|
||||
lock_wait=None,
|
||||
)
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_without_any_archives_should_bail():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg', 'list', '--short', 'repo'),
|
||||
result='\n'.encode('utf-8'),
|
||||
('borg', 'list', '--short', 'repo'), result='\n'.encode('utf-8')
|
||||
)
|
||||
insert_subprocess_never()
|
||||
|
||||
module.extract_last_archive_dry_run(
|
||||
repository='repo',
|
||||
lock_wait=None,
|
||||
)
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_with_log_info_should_call_borg_with_info_parameter():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg', 'list', '--short', 'repo', '--info'),
|
||||
result='archive1\narchive2\n'.encode('utf-8'),
|
||||
)
|
||||
insert_subprocess_mock(
|
||||
('borg', 'extract', '--dry-run', 'repo::archive2', '--info'),
|
||||
('borg', 'list', '--short', 'repo', '--info'), result='archive1\narchive2\n'.encode('utf-8')
|
||||
)
|
||||
insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2', '--info'))
|
||||
insert_logging_mock(logging.INFO)
|
||||
|
||||
module.extract_last_archive_dry_run(
|
||||
repository='repo',
|
||||
lock_wait=None,
|
||||
)
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_with_log_debug_should_call_borg_with_debug_parameter():
|
||||
@@ -76,31 +62,21 @@ def test_extract_last_archive_dry_run_with_log_debug_should_call_borg_with_debug
|
||||
result='archive1\narchive2\n'.encode('utf-8'),
|
||||
)
|
||||
insert_subprocess_mock(
|
||||
('borg', 'extract', '--dry-run', 'repo::archive2', '--debug', '--show-rc', '--list'),
|
||||
('borg', 'extract', '--dry-run', 'repo::archive2', '--debug', '--show-rc', '--list')
|
||||
)
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.extract_last_archive_dry_run(
|
||||
repository='repo',
|
||||
lock_wait=None,
|
||||
)
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_should_call_borg_via_local_path():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg1', 'list', '--short', 'repo'),
|
||||
result='archive1\narchive2\n'.encode('utf-8'),
|
||||
)
|
||||
insert_subprocess_mock(
|
||||
('borg1', 'extract', '--dry-run', 'repo::archive2'),
|
||||
('borg1', 'list', '--short', 'repo'), result='archive1\narchive2\n'.encode('utf-8')
|
||||
)
|
||||
insert_subprocess_mock(('borg1', 'extract', '--dry-run', 'repo::archive2'))
|
||||
|
||||
module.extract_last_archive_dry_run(
|
||||
repository='repo',
|
||||
lock_wait=None,
|
||||
local_path='borg1',
|
||||
)
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None, local_path='borg1')
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_should_call_borg_with_remote_path_parameters():
|
||||
@@ -110,14 +86,10 @@ def test_extract_last_archive_dry_run_should_call_borg_with_remote_path_paramete
|
||||
result='archive1\narchive2\n'.encode('utf-8'),
|
||||
)
|
||||
insert_subprocess_mock(
|
||||
('borg', 'extract', '--dry-run', 'repo::archive2', '--remote-path', 'borg1'),
|
||||
('borg', 'extract', '--dry-run', 'repo::archive2', '--remote-path', 'borg1')
|
||||
)
|
||||
|
||||
module.extract_last_archive_dry_run(
|
||||
repository='repo',
|
||||
lock_wait=None,
|
||||
remote_path='borg1',
|
||||
)
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None, remote_path='borg1')
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_should_call_borg_with_lock_wait_parameters():
|
||||
@@ -126,11 +98,6 @@ def test_extract_last_archive_dry_run_should_call_borg_with_lock_wait_parameters
|
||||
('borg', 'list', '--short', 'repo', '--lock-wait', '5'),
|
||||
result='archive1\narchive2\n'.encode('utf-8'),
|
||||
)
|
||||
insert_subprocess_mock(
|
||||
('borg', 'extract', '--dry-run', 'repo::archive2', '--lock-wait', '5'),
|
||||
)
|
||||
insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2', '--lock-wait', '5'))
|
||||
|
||||
module.extract_last_archive_dry_run(
|
||||
repository='repo',
|
||||
lock_wait=5,
|
||||
)
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=5)
|
||||
@@ -1,10 +1,9 @@
|
||||
import logging
|
||||
from collections import OrderedDict
|
||||
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.borg import info as module
|
||||
from borgmatic.tests.unit.test_verbosity import insert_logging_mock
|
||||
from ..test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
@@ -18,66 +17,42 @@ INFO_COMMAND = ('borg', 'info', 'repo')
|
||||
def test_display_archives_info_calls_borg_with_parameters():
|
||||
insert_subprocess_mock(INFO_COMMAND)
|
||||
|
||||
module.display_archives_info(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
)
|
||||
module.display_archives_info(repository='repo', storage_config={})
|
||||
|
||||
|
||||
def test_display_archives_info_with_log_info_calls_borg_with_info_parameter():
|
||||
insert_subprocess_mock(INFO_COMMAND + ('--info',))
|
||||
insert_logging_mock(logging.INFO)
|
||||
module.display_archives_info(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
)
|
||||
module.display_archives_info(repository='repo', storage_config={})
|
||||
|
||||
|
||||
def test_display_archives_info_with_log_debug_calls_borg_with_debug_parameter():
|
||||
insert_subprocess_mock(INFO_COMMAND + ('--debug', '--show-rc'))
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.display_archives_info(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
)
|
||||
module.display_archives_info(repository='repo', storage_config={})
|
||||
|
||||
|
||||
def test_display_archives_info_with_json_calls_borg_with_json_parameter():
|
||||
insert_subprocess_mock(INFO_COMMAND + ('--json',))
|
||||
|
||||
module.display_archives_info(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
json=True,
|
||||
)
|
||||
module.display_archives_info(repository='repo', storage_config={}, json=True)
|
||||
|
||||
|
||||
def test_display_archives_info_with_local_path_calls_borg_via_local_path():
|
||||
insert_subprocess_mock(('borg1',) + INFO_COMMAND[1:])
|
||||
|
||||
module.display_archives_info(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
local_path='borg1',
|
||||
)
|
||||
module.display_archives_info(repository='repo', storage_config={}, local_path='borg1')
|
||||
|
||||
|
||||
def test_display_archives_info_with_remote_path_calls_borg_with_remote_path_parameters():
|
||||
insert_subprocess_mock(INFO_COMMAND + ('--remote-path', 'borg1'))
|
||||
|
||||
module.display_archives_info(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
remote_path='borg1',
|
||||
)
|
||||
module.display_archives_info(repository='repo', storage_config={}, remote_path='borg1')
|
||||
|
||||
|
||||
def test_display_archives_info_with_lock_wait_calls_borg_with_lock_wait_parameters():
|
||||
storage_config = {'lock_wait': 5}
|
||||
insert_subprocess_mock(INFO_COMMAND + ('--lock-wait', '5'))
|
||||
|
||||
module.display_archives_info(
|
||||
repository='repo',
|
||||
storage_config=storage_config,
|
||||
)
|
||||
module.display_archives_info(repository='repo', storage_config=storage_config)
|
||||
@@ -0,0 +1,58 @@
|
||||
import logging
|
||||
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.borg import init as module
|
||||
from ..test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
subprocess = flexmock(module.subprocess)
|
||||
subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
|
||||
|
||||
|
||||
INIT_COMMAND = ('borg', 'init', 'repo', '--encryption', 'repokey')
|
||||
|
||||
|
||||
def test_initialize_repository_calls_borg_with_parameters():
|
||||
insert_subprocess_mock(INIT_COMMAND)
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey')
|
||||
|
||||
|
||||
def test_initialize_repository_with_append_only_calls_borg_with_append_only_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--append-only',))
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey', append_only=True)
|
||||
|
||||
|
||||
def test_initialize_repository_with_storage_quota_calls_borg_with_storage_quota_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--storage-quota', '5G'))
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey', storage_quota='5G')
|
||||
|
||||
|
||||
def test_initialize_repository_with_log_info_calls_borg_with_info_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--info',))
|
||||
insert_logging_mock(logging.INFO)
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey')
|
||||
|
||||
|
||||
def test_initialize_repository_with_log_debug_calls_borg_with_debug_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--debug',))
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey')
|
||||
|
||||
|
||||
def test_initialize_repository_with_local_path_calls_borg_via_local_path():
|
||||
insert_subprocess_mock(('borg1',) + INIT_COMMAND[1:])
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey', local_path='borg1')
|
||||
|
||||
|
||||
def test_initialize_repository_with_remote_path_calls_borg_with_remote_path_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--remote-path', 'borg1'))
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey', remote_path='borg1')
|
||||
@@ -1,10 +1,9 @@
|
||||
import logging
|
||||
from collections import OrderedDict
|
||||
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.borg import list as module
|
||||
from borgmatic.tests.unit.test_verbosity import insert_logging_mock
|
||||
from ..test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
@@ -18,67 +17,43 @@ LIST_COMMAND = ('borg', 'list', 'repo')
|
||||
def test_list_archives_calls_borg_with_parameters():
|
||||
insert_subprocess_mock(LIST_COMMAND)
|
||||
|
||||
module.list_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
)
|
||||
module.list_archives(repository='repo', storage_config={})
|
||||
|
||||
|
||||
def test_list_archives_with_log_info_calls_borg_with_info_parameter():
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--info',))
|
||||
insert_logging_mock(logging.INFO)
|
||||
|
||||
module.list_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
)
|
||||
module.list_archives(repository='repo', storage_config={})
|
||||
|
||||
|
||||
def test_list_archives_with_log_debug_calls_borg_with_debug_parameter():
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--debug', '--show-rc'))
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.list_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
)
|
||||
module.list_archives(repository='repo', storage_config={})
|
||||
|
||||
|
||||
def test_list_archives_with_json_calls_borg_with_json_parameter():
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--json',))
|
||||
|
||||
module.list_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
json=True,
|
||||
)
|
||||
module.list_archives(repository='repo', storage_config={}, json=True)
|
||||
|
||||
|
||||
def test_list_archives_with_local_path_calls_borg_via_local_path():
|
||||
insert_subprocess_mock(('borg1',) + LIST_COMMAND[1:])
|
||||
|
||||
module.list_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
local_path='borg1',
|
||||
)
|
||||
module.list_archives(repository='repo', storage_config={}, local_path='borg1')
|
||||
|
||||
|
||||
def test_list_archives_with_remote_path_calls_borg_with_remote_path_parameters():
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--remote-path', 'borg1'))
|
||||
|
||||
module.list_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
remote_path='borg1',
|
||||
)
|
||||
module.list_archives(repository='repo', storage_config={}, remote_path='borg1')
|
||||
|
||||
|
||||
def test_list_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
|
||||
storage_config = {'lock_wait': 5}
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--lock-wait', '5'))
|
||||
|
||||
module.list_archives(
|
||||
repository='repo',
|
||||
storage_config=storage_config,
|
||||
)
|
||||
module.list_archives(repository='repo', storage_config=storage_config)
|
||||
@@ -4,7 +4,7 @@ from collections import OrderedDict
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.borg import prune as module
|
||||
from borgmatic.tests.unit.test_verbosity import insert_logging_mock
|
||||
from ..test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
@@ -12,22 +12,11 @@ def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
|
||||
|
||||
|
||||
|
||||
BASE_PRUNE_FLAGS = (
|
||||
('--keep-daily', '1'),
|
||||
('--keep-weekly', '2'),
|
||||
('--keep-monthly', '3'),
|
||||
)
|
||||
BASE_PRUNE_FLAGS = (('--keep-daily', '1'), ('--keep-weekly', '2'), ('--keep-monthly', '3'))
|
||||
|
||||
|
||||
def test_make_prune_flags_returns_flags_from_config_plus_default_prefix():
|
||||
retention_config = OrderedDict(
|
||||
(
|
||||
('keep_daily', 1),
|
||||
('keep_weekly', 2),
|
||||
('keep_monthly', 3),
|
||||
)
|
||||
)
|
||||
retention_config = OrderedDict((('keep_daily', 1), ('keep_weekly', 2), ('keep_monthly', 3)))
|
||||
|
||||
result = module._make_prune_flags(retention_config)
|
||||
|
||||
@@ -35,94 +24,82 @@ def test_make_prune_flags_returns_flags_from_config_plus_default_prefix():
|
||||
|
||||
|
||||
def test_make_prune_flags_accepts_prefix_with_placeholders():
|
||||
retention_config = OrderedDict(
|
||||
(
|
||||
('keep_daily', 1),
|
||||
('prefix', 'Documents_{hostname}-{now}'),
|
||||
)
|
||||
)
|
||||
retention_config = OrderedDict((('keep_daily', 1), ('prefix', 'Documents_{hostname}-{now}')))
|
||||
|
||||
result = module._make_prune_flags(retention_config)
|
||||
|
||||
expected = (
|
||||
('--keep-daily', '1'),
|
||||
('--prefix', 'Documents_{hostname}-{now}'),
|
||||
)
|
||||
expected = (('--keep-daily', '1'), ('--prefix', 'Documents_{hostname}-{now}'))
|
||||
|
||||
assert tuple(result) == expected
|
||||
|
||||
|
||||
PRUNE_COMMAND = (
|
||||
'borg', 'prune', 'repo', '--keep-daily', '1', '--keep-weekly', '2', '--keep-monthly', '3',
|
||||
'borg',
|
||||
'prune',
|
||||
'repo',
|
||||
'--keep-daily',
|
||||
'1',
|
||||
'--keep-weekly',
|
||||
'2',
|
||||
'--keep-monthly',
|
||||
'3',
|
||||
)
|
||||
|
||||
|
||||
def test_prune_archives_calls_borg_with_parameters():
|
||||
retention_config = flexmock()
|
||||
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
|
||||
BASE_PRUNE_FLAGS,
|
||||
BASE_PRUNE_FLAGS
|
||||
)
|
||||
insert_subprocess_mock(PRUNE_COMMAND)
|
||||
|
||||
module.prune_archives(
|
||||
dry_run=False,
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
retention_config=retention_config,
|
||||
dry_run=False, repository='repo', storage_config={}, retention_config=retention_config
|
||||
)
|
||||
|
||||
|
||||
def test_prune_archives_with_log_info_calls_borg_with_info_parameter():
|
||||
retention_config = flexmock()
|
||||
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
|
||||
BASE_PRUNE_FLAGS,
|
||||
BASE_PRUNE_FLAGS
|
||||
)
|
||||
insert_subprocess_mock(PRUNE_COMMAND + ('--stats', '--info',))
|
||||
insert_subprocess_mock(PRUNE_COMMAND + ('--stats', '--info'))
|
||||
insert_logging_mock(logging.INFO)
|
||||
|
||||
module.prune_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
dry_run=False,
|
||||
retention_config=retention_config,
|
||||
repository='repo', storage_config={}, dry_run=False, retention_config=retention_config
|
||||
)
|
||||
|
||||
|
||||
def test_prune_archives_with_log_debug_calls_borg_with_debug_parameter():
|
||||
retention_config = flexmock()
|
||||
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
|
||||
BASE_PRUNE_FLAGS,
|
||||
BASE_PRUNE_FLAGS
|
||||
)
|
||||
insert_subprocess_mock(PRUNE_COMMAND + ('--stats', '--debug', '--list', '--show-rc'))
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.prune_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
dry_run=False,
|
||||
retention_config=retention_config,
|
||||
repository='repo', storage_config={}, dry_run=False, retention_config=retention_config
|
||||
)
|
||||
|
||||
|
||||
def test_prune_archives_with_dry_run_calls_borg_with_dry_run_parameter():
|
||||
retention_config = flexmock()
|
||||
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
|
||||
BASE_PRUNE_FLAGS,
|
||||
BASE_PRUNE_FLAGS
|
||||
)
|
||||
insert_subprocess_mock(PRUNE_COMMAND + ('--dry-run',))
|
||||
|
||||
module.prune_archives(
|
||||
repository='repo',
|
||||
storage_config={},
|
||||
dry_run=True,
|
||||
retention_config=retention_config,
|
||||
repository='repo', storage_config={}, dry_run=True, retention_config=retention_config
|
||||
)
|
||||
|
||||
|
||||
def test_prune_archives_with_local_path_calls_borg_via_local_path():
|
||||
retention_config = flexmock()
|
||||
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
|
||||
BASE_PRUNE_FLAGS,
|
||||
BASE_PRUNE_FLAGS
|
||||
)
|
||||
insert_subprocess_mock(('borg1',) + PRUNE_COMMAND[1:])
|
||||
|
||||
@@ -138,7 +115,7 @@ def test_prune_archives_with_local_path_calls_borg_via_local_path():
|
||||
def test_prune_archives_with_remote_path_calls_borg_with_remote_path_parameters():
|
||||
retention_config = flexmock()
|
||||
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
|
||||
BASE_PRUNE_FLAGS,
|
||||
BASE_PRUNE_FLAGS
|
||||
)
|
||||
insert_subprocess_mock(PRUNE_COMMAND + ('--remote-path', 'borg1'))
|
||||
|
||||
@@ -155,7 +132,7 @@ def test_prune_archives_with_umask_calls_borg_with_umask_parameters():
|
||||
storage_config = {'umask': '077'}
|
||||
retention_config = flexmock()
|
||||
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
|
||||
BASE_PRUNE_FLAGS,
|
||||
BASE_PRUNE_FLAGS
|
||||
)
|
||||
insert_subprocess_mock(PRUNE_COMMAND + ('--umask', '077'))
|
||||
|
||||
@@ -171,7 +148,7 @@ def test_prune_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
|
||||
storage_config = {'lock_wait': 5}
|
||||
retention_config = flexmock()
|
||||
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
|
||||
BASE_PRUNE_FLAGS,
|
||||
BASE_PRUNE_FLAGS
|
||||
)
|
||||
insert_subprocess_mock(PRUNE_COMMAND + ('--lock-wait', '5'))
|
||||
|
||||
+6
-11
@@ -2,20 +2,19 @@ import json
|
||||
import sys
|
||||
|
||||
from flexmock import flexmock
|
||||
import pytest
|
||||
|
||||
from borgmatic.commands import borgmatic
|
||||
|
||||
|
||||
def test__run_commands_handles_multiple_json_outputs_in_array():
|
||||
def test_run_commands_handles_multiple_json_outputs_in_array():
|
||||
(
|
||||
flexmock(borgmatic)
|
||||
.should_receive('_run_commands_on_repository')
|
||||
.times(3)
|
||||
.replace_with(
|
||||
lambda args, consistency, json_results, local_path, location, remote_path, retention,
|
||||
storage,
|
||||
unexpanded_repository: json_results.append({"whatever": unexpanded_repository})
|
||||
lambda args, consistency, json_results, local_path, location, remote_path, retention, storage, unexpanded_repository: json_results.append(
|
||||
{"whatever": unexpanded_repository}
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
@@ -31,7 +30,7 @@ def test__run_commands_handles_multiple_json_outputs_in_array():
|
||||
{"whatever": "fake_repo2"},
|
||||
{"whatever": "fake_repo3"}
|
||||
]
|
||||
''',
|
||||
'''
|
||||
)
|
||||
)
|
||||
)
|
||||
@@ -41,11 +40,7 @@ def test__run_commands_handles_multiple_json_outputs_in_array():
|
||||
args=flexmock(json=True),
|
||||
consistency=None,
|
||||
local_path=None,
|
||||
location={'repositories': [
|
||||
'fake_repo1',
|
||||
'fake_repo2',
|
||||
'fake_repo3'
|
||||
]},
|
||||
location={'repositories': ['fake_repo1', 'fake_repo2', 'fake_repo3']},
|
||||
remote_path=None,
|
||||
retention=None,
|
||||
storage=None,
|
||||
@@ -0,0 +1,23 @@
|
||||
from borgmatic.config import checks as module
|
||||
|
||||
|
||||
def test_repository_enabled_for_checks_defaults_to_enabled_for_all_repositories():
|
||||
enabled = module.repository_enabled_for_checks('repo.borg', consistency={})
|
||||
|
||||
assert enabled
|
||||
|
||||
|
||||
def test_repository_enabled_for_checks_is_enabled_for_specified_repositories():
|
||||
enabled = module.repository_enabled_for_checks(
|
||||
'repo.borg', consistency={'check_repositories': ['repo.borg', 'other.borg']}
|
||||
)
|
||||
|
||||
assert enabled
|
||||
|
||||
|
||||
def test_repository_enabled_for_checks_is_disabled_for_other_repositories():
|
||||
enabled = module.repository_enabled_for_checks(
|
||||
'repo.borg', consistency={'check_repositories': ['other.borg']}
|
||||
)
|
||||
|
||||
assert not enabled
|
||||
@@ -33,25 +33,31 @@ def test_convert_legacy_parsed_config_transforms_source_config_to_mapping():
|
||||
|
||||
destination_config = module.convert_legacy_parsed_config(source_config, source_excludes, schema)
|
||||
|
||||
assert destination_config == OrderedDict([
|
||||
(
|
||||
'location',
|
||||
OrderedDict([
|
||||
('source_directories', ['/home']),
|
||||
('repositories', ['hostname.borg']),
|
||||
('exclude_patterns', ['/var']),
|
||||
]),
|
||||
),
|
||||
('storage', OrderedDict([('encryption_passphrase', 'supersecret')])),
|
||||
('retention', OrderedDict([('keep_daily', 7)])),
|
||||
('consistency', OrderedDict([('checks', ['repository'])])),
|
||||
])
|
||||
assert destination_config == OrderedDict(
|
||||
[
|
||||
(
|
||||
'location',
|
||||
OrderedDict(
|
||||
[
|
||||
('source_directories', ['/home']),
|
||||
('repositories', ['hostname.borg']),
|
||||
('exclude_patterns', ['/var']),
|
||||
]
|
||||
),
|
||||
),
|
||||
('storage', OrderedDict([('encryption_passphrase', 'supersecret')])),
|
||||
('retention', OrderedDict([('keep_daily', 7)])),
|
||||
('consistency', OrderedDict([('checks', ['repository'])])),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def test_convert_legacy_parsed_config_splits_space_separated_values():
|
||||
flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
|
||||
source_config = Parsed_config(
|
||||
location=OrderedDict([('source_directories', '/home /etc'), ('repository', 'hostname.borg')]),
|
||||
location=OrderedDict(
|
||||
[('source_directories', '/home /etc'), ('repository', 'hostname.borg')]
|
||||
),
|
||||
storage=OrderedDict(),
|
||||
retention=OrderedDict(),
|
||||
consistency=OrderedDict([('checks', 'repository archives')]),
|
||||
@@ -59,21 +65,25 @@ def test_convert_legacy_parsed_config_splits_space_separated_values():
|
||||
source_excludes = ['/var']
|
||||
schema = {'map': defaultdict(lambda: {'map': {}})}
|
||||
|
||||
destination_config = module.convert_legacy_parsed_config(source_config, source_excludes, schema)
|
||||
destination_config = module.convert_legacy_parsed_config(source_config, source_excludes, schema)
|
||||
|
||||
assert destination_config == OrderedDict([
|
||||
(
|
||||
'location',
|
||||
OrderedDict([
|
||||
('source_directories', ['/home', '/etc']),
|
||||
('repositories', ['hostname.borg']),
|
||||
('exclude_patterns', ['/var']),
|
||||
]),
|
||||
),
|
||||
('storage', OrderedDict()),
|
||||
('retention', OrderedDict()),
|
||||
('consistency', OrderedDict([('checks', ['repository', 'archives'])])),
|
||||
])
|
||||
assert destination_config == OrderedDict(
|
||||
[
|
||||
(
|
||||
'location',
|
||||
OrderedDict(
|
||||
[
|
||||
('source_directories', ['/home', '/etc']),
|
||||
('repositories', ['hostname.borg']),
|
||||
('exclude_patterns', ['/var']),
|
||||
]
|
||||
),
|
||||
),
|
||||
('storage', OrderedDict()),
|
||||
('retention', OrderedDict()),
|
||||
('consistency', OrderedDict([('checks', ['repository', 'archives'])])),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def test_guard_configuration_upgraded_raises_when_only_source_config_present():
|
||||
@@ -0,0 +1,37 @@
|
||||
from collections import OrderedDict
|
||||
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.config import generate as module
|
||||
|
||||
|
||||
def test_schema_to_sample_configuration_generates_config_with_examples():
|
||||
flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
|
||||
flexmock(module).should_receive('add_comments_to_configuration')
|
||||
schema = {
|
||||
'map': OrderedDict(
|
||||
[
|
||||
('section1', {'map': {'field1': OrderedDict([('example', 'Example 1')])}}),
|
||||
(
|
||||
'section2',
|
||||
{
|
||||
'map': OrderedDict(
|
||||
[
|
||||
('field2', {'example': 'Example 2'}),
|
||||
('field3', {'example': 'Example 3'}),
|
||||
]
|
||||
)
|
||||
},
|
||||
),
|
||||
]
|
||||
)
|
||||
}
|
||||
|
||||
config = module._schema_to_sample_configuration(schema)
|
||||
|
||||
assert config == OrderedDict(
|
||||
[
|
||||
('section1', OrderedDict([('field1', 'Example 1')])),
|
||||
('section2', OrderedDict([('field2', 'Example 2'), ('field3', 'Example 3')])),
|
||||
]
|
||||
)
|
||||
@@ -25,17 +25,9 @@ def test_validate_configuration_format_with_valid_config_should_not_raise():
|
||||
parser.should_receive('options').with_args('other').and_return(('such',))
|
||||
config_format = (
|
||||
module.Section_format(
|
||||
'section',
|
||||
options=(
|
||||
module.Config_option('stuff', str, required=True),
|
||||
),
|
||||
),
|
||||
module.Section_format(
|
||||
'other',
|
||||
options=(
|
||||
module.Config_option('such', str, required=True),
|
||||
),
|
||||
'section', options=(module.Config_option('stuff', str, required=True),)
|
||||
),
|
||||
module.Section_format('other', options=(module.Config_option('such', str, required=True),)),
|
||||
)
|
||||
|
||||
module.validate_configuration_format(parser, config_format)
|
||||
@@ -46,10 +38,7 @@ def test_validate_configuration_format_with_missing_required_section_should_rais
|
||||
parser.should_receive('sections').and_return(('section',))
|
||||
config_format = (
|
||||
module.Section_format(
|
||||
'section',
|
||||
options=(
|
||||
module.Config_option('stuff', str, required=True),
|
||||
),
|
||||
'section', options=(module.Config_option('stuff', str, required=True),)
|
||||
),
|
||||
# At least one option in this section is required, so the section is required.
|
||||
module.Section_format(
|
||||
@@ -71,10 +60,7 @@ def test_validate_configuration_format_with_missing_optional_section_should_not_
|
||||
parser.should_receive('options').with_args('section').and_return(('stuff',))
|
||||
config_format = (
|
||||
module.Section_format(
|
||||
'section',
|
||||
options=(
|
||||
module.Config_option('stuff', str, required=True),
|
||||
),
|
||||
'section', options=(module.Config_option('stuff', str, required=True),)
|
||||
),
|
||||
# No options in the section are required, so the section is optional.
|
||||
module.Section_format(
|
||||
@@ -92,9 +78,7 @@ def test_validate_configuration_format_with_missing_optional_section_should_not_
|
||||
def test_validate_configuration_format_with_unknown_section_should_raise():
|
||||
parser = flexmock()
|
||||
parser.should_receive('sections').and_return(('section', 'extra'))
|
||||
config_format = (
|
||||
module.Section_format('section', options=()),
|
||||
)
|
||||
config_format = (module.Section_format('section', options=()),)
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.validate_configuration_format(parser, config_format)
|
||||
@@ -141,8 +125,7 @@ def test_validate_configuration_format_with_extra_option_should_raise():
|
||||
parser.should_receive('options').with_args('section').and_return(('option', 'extra'))
|
||||
config_format = (
|
||||
module.Section_format(
|
||||
'section',
|
||||
options=(module.Config_option('option', str, required=True),),
|
||||
'section', options=(module.Config_option('option', str, required=True),)
|
||||
),
|
||||
)
|
||||
|
||||
@@ -168,12 +151,7 @@ def test_parse_section_options_should_return_section_options():
|
||||
|
||||
config = module.parse_section_options(parser, section_format)
|
||||
|
||||
assert config == OrderedDict(
|
||||
(
|
||||
('foo', 'value'),
|
||||
('bar', 1),
|
||||
)
|
||||
)
|
||||
assert config == OrderedDict((('foo', 'value'), ('bar', 1)))
|
||||
|
||||
|
||||
def test_parse_section_options_for_missing_section_should_return_empty_dict():
|
||||
@@ -210,13 +188,13 @@ def test_parse_configuration_should_return_section_configs():
|
||||
config_format = (flexmock(name='items'), flexmock(name='things'))
|
||||
mock_module = flexmock(module)
|
||||
mock_module.should_receive('validate_configuration_format').with_args(
|
||||
parser, config_format,
|
||||
parser, config_format
|
||||
).once()
|
||||
mock_section_configs = (flexmock(), flexmock())
|
||||
|
||||
for section_format, section_config in zip(config_format, mock_section_configs):
|
||||
mock_module.should_receive('parse_section_options').with_args(
|
||||
parser, section_format,
|
||||
parser, section_format
|
||||
).and_return(section_config).once()
|
||||
|
||||
parsed_config = module.parse_configuration('filename', config_format)
|
||||
@@ -24,6 +24,7 @@ def test_apply_logical_validation_raises_if_archive_name_format_present_without_
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_apply_logical_validation_raises_if_archive_name_format_present_without_retention_prefix():
|
||||
with pytest.raises(module.Validation_error):
|
||||
module.apply_logical_validation(
|
||||
@@ -31,7 +32,7 @@ def test_apply_logical_validation_raises_if_archive_name_format_present_without_
|
||||
{
|
||||
'storage': {'archive_name_format': '{hostname}-{now}'},
|
||||
'retention': {'keep_daily': 7},
|
||||
'consistency': {'prefix': '{hostname}-'}
|
||||
'consistency': {'prefix': '{hostname}-'},
|
||||
},
|
||||
)
|
||||
|
||||
@@ -50,6 +51,29 @@ def test_apply_logical_validation_warns_if_archive_name_format_present_without_c
|
||||
)
|
||||
|
||||
|
||||
def test_apply_locical_validation_raises_if_unknown_repository_in_check_repositories():
|
||||
with pytest.raises(module.Validation_error):
|
||||
module.apply_logical_validation(
|
||||
'config.yaml',
|
||||
{
|
||||
'location': {'repositories': ['repo.borg', 'other.borg']},
|
||||
'retention': {'keep_secondly': 1000},
|
||||
'consistency': {'check_repositories': ['repo.borg', 'unknown.borg']},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_apply_locical_validation_does_not_raise_if_known_repository_in_check_repositories():
|
||||
module.apply_logical_validation(
|
||||
'config.yaml',
|
||||
{
|
||||
'location': {'repositories': ['repo.borg', 'other.borg']},
|
||||
'retention': {'keep_secondly': 1000},
|
||||
'consistency': {'check_repositories': ['repo.borg']},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_apply_logical_validation_does_not_raise_or_warn_if_archive_name_format_and_prefix_present():
|
||||
logger = flexmock(module.logger)
|
||||
logger.should_receive('warning').never()
|
||||
@@ -59,15 +83,10 @@ def test_apply_logical_validation_does_not_raise_or_warn_if_archive_name_format_
|
||||
{
|
||||
'storage': {'archive_name_format': '{hostname}-{now}'},
|
||||
'retention': {'prefix': '{hostname}-'},
|
||||
'consistency': {'prefix': '{hostname}-'}
|
||||
'consistency': {'prefix': '{hostname}-'},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def test_apply_logical_validation_does_not_raise_otherwise():
|
||||
module.apply_logical_validation(
|
||||
'config.yaml',
|
||||
{
|
||||
'retention': {'keep_secondly': 1000},
|
||||
},
|
||||
)
|
||||
module.apply_logical_validation('config.yaml', {'retention': {'keep_secondly': 1000}})
|
||||
@@ -4,10 +4,13 @@ from flexmock import flexmock
|
||||
|
||||
from borgmatic import verbosity as module
|
||||
|
||||
|
||||
def insert_logging_mock(log_level):
|
||||
""" Mocks the isEnabledFor from python logging. """
|
||||
'''
|
||||
Mock the isEnabledFor from Python logging.
|
||||
'''
|
||||
logging = flexmock(module.logging.Logger)
|
||||
logging.should_receive('isEnabledFor').replace_with(lambda lvl : lvl >= log_level)
|
||||
logging.should_receive('isEnabledFor').replace_with(lambda level: level >= log_level)
|
||||
logging.should_receive('getEffectiveLevel').replace_with(lambda: log_level)
|
||||
|
||||
|
||||
@@ -1,8 +1,27 @@
|
||||
[tox]
|
||||
envlist=py3
|
||||
skipsdist=True
|
||||
envlist = py3
|
||||
skipsdist = True
|
||||
|
||||
[testenv]
|
||||
usedevelop=True
|
||||
deps=-rtest_requirements.txt
|
||||
commands = py.test --cov-report term-missing:skip-covered --cov=borgmatic borgmatic []
|
||||
usedevelop = True
|
||||
deps = -rtest_requirements.txt
|
||||
whitelist_externals = sh
|
||||
commands =
|
||||
py.test --cov-report term-missing:skip-covered --cov=borgmatic --ignore=tests/end-to-end \
|
||||
tests []
|
||||
sh scripts/black
|
||||
flake8 .
|
||||
|
||||
[testenv:black]
|
||||
basepython = python3.7
|
||||
commands =
|
||||
black --skip-string-normalization --line-length 100 .
|
||||
|
||||
[testenv:end-to-end]
|
||||
deps = -rtest_requirements.txt
|
||||
commands =
|
||||
py.test tests/end-to-end []
|
||||
|
||||
[flake8]
|
||||
ignore = E501,W503
|
||||
exclude = *.*/*
|
||||
|
||||
Reference in New Issue
Block a user