mirror of
https://github.com/borgmatic-collective/borgmatic.git
synced 2026-07-22 18:13:02 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4272c6b077 | ||
|
|
26071de2e7 | ||
|
|
fe92d9e838 | ||
|
|
5ea2d644a2 | ||
|
|
c35f90154f | ||
|
|
36305c50b1 | ||
|
|
2b3b8eab71 | ||
|
|
aa7c7651e5 | ||
|
|
c41ffb5ceb | ||
|
|
766a03375a | ||
|
|
2a4d4247e3 | ||
|
|
9de5083a7e | ||
|
|
d0557b2bcd | ||
|
|
1a980d6321 | ||
|
|
fb21d4e645 | ||
|
|
5933a4d778 | ||
|
|
8cf16c7831 | ||
|
|
fcf4e03c2f | ||
|
|
d1b29e82da | ||
|
|
290e969a22 | ||
|
|
18ae91ea6e | ||
|
|
0bce77a2ac | ||
|
|
19155607af | ||
|
|
f357c37e2c | ||
|
|
2980c14728 | ||
|
|
7e0e00d45d | ||
|
|
8b4ac0017b | ||
|
|
8ec1ec527e | ||
|
|
6096a7181c | ||
|
|
fa9dfb8ff7 | ||
|
|
2dc006aab4 | ||
|
|
031b9d6faf | ||
|
|
d9018a47f6 | ||
|
|
e893a20dfd | ||
|
|
09d521661f | ||
|
|
fd46efb193 | ||
|
|
426f54c9cc | ||
|
|
45a537b6b1 | ||
|
|
d6feca169c | ||
|
|
05e2900ab0 | ||
|
|
30b52e5523 | ||
|
|
14aeddc11f | ||
|
|
066399ecdb | ||
|
|
d4bbac4467 | ||
|
|
7516443a89 | ||
|
|
73d67e29b4 |
@@ -1,3 +1,32 @@
|
||||
1.2.17
|
||||
* #140: List the files within an archive via --list --archive option.
|
||||
|
||||
1.2.16
|
||||
* #119: Include a sample borgmatic configuration file in the documentation.
|
||||
* #123: Support for Borg archive restoration via borgmatic --extract command-line flag.
|
||||
* Refactor documentation into multiple separate pages for clarity and findability.
|
||||
* Organize options within command-line help into logical groups.
|
||||
* Exclude tests from distribution packages.
|
||||
|
||||
1.2.15
|
||||
* #127: Remove date echo from schema example, as it's not a substitute for real logging.
|
||||
* #132: Leave exclude_patterns glob expansion to Borg, since doing it in borgmatic leads to
|
||||
confusing behavior.
|
||||
* #136: Handle and format validation errors raised during argument parsing.
|
||||
* #138: Allow use of --stats flag when --create or --prune flags are implied.
|
||||
|
||||
1.2.14
|
||||
* #103: When generating sample configuration with generate-borgmatic-config, document the defaults
|
||||
for each option.
|
||||
* #116: When running multiple configuration files, attempt all configuration files even if one of
|
||||
them errors. Log a summary of results at the end.
|
||||
* Add borgmatic --version command-line flag to get the current installed version number.
|
||||
|
||||
1.2.13
|
||||
* #100: Support for --stats command-line flag independent of --verbosity.
|
||||
* #117: With borgmatic --init command-line flag, proceed without erroring if a repository already
|
||||
exists.
|
||||
|
||||
1.2.12
|
||||
* #110: Support for Borg repository initialization via borgmatic --init command-line flag.
|
||||
* #111: Update Borg create --filter values so a dry run lists files to back up.
|
||||
|
||||
@@ -54,333 +54,22 @@ href="https://asciinema.org/a/203761" target="_blank">screencast</a>.
|
||||
<script src="https://asciinema.org/a/203761.js" id="asciicast-203761" async></script>
|
||||
|
||||
|
||||
## Installation
|
||||
## How-to guides
|
||||
|
||||
To get up and running, first [install
|
||||
Borg](https://borgbackup.readthedocs.io/en/latest/installation.html), at
|
||||
least version 1.1.
|
||||
* [Set up backups with borgmatic](docs/how-to/set-up-backups.md) ⬅ *Start here!*
|
||||
* [Make per-application backups](docs/how-to/make-per-application-backups.md)
|
||||
* [Deal with very large backups](docs/how-to/deal-with-very-large-backups.md)
|
||||
* [Inspect your backups](docs/how-to/inspect-your-backups.md)
|
||||
* [Restore a backup](docs/how-to/restore-a-backup.md)
|
||||
* [Run preparation steps before backups](docs/how-to/run-preparation-steps-before-backups.md)
|
||||
* [Upgrade borgmatic](docs/how-to/upgrade.md)
|
||||
* [Develop on borgmatic](docs/how-to/develop-on-borgmatic.md)
|
||||
|
||||
Then, run the following command to download and install borgmatic:
|
||||
|
||||
```bash
|
||||
sudo pip3 install --upgrade borgmatic
|
||||
```
|
||||
## Reference guides
|
||||
|
||||
Note that your pip binary may have a different name than "pip3". Make sure
|
||||
you're using Python 3, as borgmatic does not support Python 2.
|
||||
|
||||
|
||||
### Other ways to install
|
||||
|
||||
* [Docker image](https://hub.docker.com/r/monachus/borgmatic/)
|
||||
* [Another Docker image](https://hub.docker.com/r/b3vis/borgmatic/)
|
||||
* [Debian](https://tracker.debian.org/pkg/borgmatic)
|
||||
* [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
|
||||
* [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
|
||||
* [Arch Linux](https://aur.archlinux.org/packages/borgmatic/)
|
||||
* [OpenBSD](http://ports.su/sysutils/borgmatic)
|
||||
<br><br>
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
After you install borgmatic, generate a sample configuration file:
|
||||
|
||||
```bash
|
||||
sudo generate-borgmatic-config
|
||||
```
|
||||
|
||||
If that command is not found, then it may be installed in a location that's
|
||||
not in your system `PATH`. Try looking in `/usr/local/bin/`.
|
||||
|
||||
This generates a sample configuration file at /etc/borgmatic/config.yaml (by
|
||||
default). You should edit the file to suit your needs, as the values are just
|
||||
representative. All fields are optional except where indicated, so feel free
|
||||
to ignore anything you don't need.
|
||||
|
||||
You can also have a look at the [full configuration
|
||||
schema](https://projects.torsion.org/witten/borgmatic/src/master/borgmatic/config/schema.yaml)
|
||||
for the authoritative set of all configuration options. This is handy if
|
||||
borgmatic has added new options since you originally created your
|
||||
configuration file.
|
||||
|
||||
|
||||
### Encryption
|
||||
|
||||
Note that if you plan to run borgmatic on a schedule with cron, and you
|
||||
encrypt your Borg repository with a passphrase instead of a key file, you'll
|
||||
either need to set the borgmatic `encryption_passphrase` configuration
|
||||
variable or set the `BORG_PASSPHRASE` environment variable. See the
|
||||
[repository encryption
|
||||
section](https://borgbackup.readthedocs.io/en/latest/quickstart.html#repository-encryption)
|
||||
of the Quick Start for more info.
|
||||
|
||||
Alternatively, the passphrase can be specified programatically by setting
|
||||
either the borgmatic `encryption_passcommand` configuration variable or the
|
||||
`BORG_PASSCOMMAND` environment variable. See the [Borg Security
|
||||
FAQ](http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically)
|
||||
for more info.
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
### Initialization
|
||||
|
||||
Before you can create backups with borgmatic, you first need to initialize a
|
||||
Borg repository so you have a destination for your backup archives. (But skip
|
||||
this step if you already have a Borg repository.) To create a repository, run
|
||||
a command like the following:
|
||||
|
||||
```bash
|
||||
borgmatic --init --encryption repokey
|
||||
```
|
||||
|
||||
This uses the borgmatic configuration file you created above to determine
|
||||
which local or remote repository to create, and encrypts it with the
|
||||
encryption passphrase specified there if one is provided. Read about [Borg
|
||||
encryption
|
||||
modes](https://borgbackup.readthedocs.io/en/latest/usage/init.html#encryption-modes)
|
||||
for the menu of available encryption modes.
|
||||
|
||||
Also, optionally check out the [Borg Quick
|
||||
Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) for more
|
||||
background about repository initialization.
|
||||
|
||||
If the repository is on a remote host, make sure that your local user has
|
||||
key-based SSH access to the desired user account on the remote host.
|
||||
|
||||
|
||||
### Backups
|
||||
|
||||
You can run borgmatic and start a backup simply by invoking it without
|
||||
arguments:
|
||||
|
||||
```bash
|
||||
borgmatic
|
||||
```
|
||||
|
||||
This will also prune any old backups as per the configured retention policy,
|
||||
and check backups for consistency problems due to things like file damage.
|
||||
|
||||
If you'd like to see the available command-line arguments, view the help:
|
||||
|
||||
```bash
|
||||
borgmatic --help
|
||||
```
|
||||
|
||||
Note that borgmatic prunes archives *before* creating an archive, so as to
|
||||
free up space for archiving. This means that when a borgmatic run finishes,
|
||||
there may still be prune-able archives. Not to worry, as they will get cleaned
|
||||
up at the start of the next run.
|
||||
|
||||
|
||||
### Verbosity
|
||||
|
||||
By default, the backup will proceed silently except in the case of errors. But
|
||||
if you'd like to to get additional information about the progress of the
|
||||
backup as it proceeds, use the verbosity option:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 1
|
||||
```
|
||||
|
||||
Or, for even more progress spew:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 2
|
||||
```
|
||||
|
||||
### À la carte
|
||||
|
||||
If you want to run borgmatic with only pruning, creating, or checking enabled,
|
||||
the following optional flags are available:
|
||||
|
||||
```bash
|
||||
borgmatic --prune
|
||||
borgmatic --create
|
||||
borgmatic --check
|
||||
```
|
||||
|
||||
You can run with only one of these flags provided, or you can mix and match
|
||||
any number of them. This supports use cases like running consistency checks
|
||||
from a different cron job with a different frequency, or running pruning with
|
||||
a different verbosity level.
|
||||
|
||||
Additionally, borgmatic provides convenient flags for Borg's
|
||||
[list](https://borgbackup.readthedocs.io/en/stable/usage/list.html) and
|
||||
[info](https://borgbackup.readthedocs.io/en/stable/usage/info.html)
|
||||
functionality:
|
||||
|
||||
|
||||
```bash
|
||||
borgmatic --list
|
||||
borgmatic --info
|
||||
```
|
||||
|
||||
You can include an optional `--json` flag with `--create`, `--list`, or
|
||||
`--info` to get the output formatted as JSON.
|
||||
|
||||
|
||||
## Autopilot
|
||||
|
||||
If you want to run borgmatic automatically, say once a day, the you can
|
||||
configure a job runner to invoke it periodically.
|
||||
|
||||
### cron
|
||||
|
||||
If you're using cron, download the [sample cron
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/cron/borgmatic).
|
||||
Then, from the directory where you downloaded it:
|
||||
|
||||
```bash
|
||||
sudo mv borgmatic /etc/cron.d/borgmatic
|
||||
sudo chmod +x /etc/cron.d/borgmatic
|
||||
```
|
||||
|
||||
You can modify the cron file if you'd like to run borgmatic more or less frequently.
|
||||
|
||||
### systemd
|
||||
|
||||
If you're using systemd instead of cron to run jobs, download the [sample
|
||||
systemd service
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.service)
|
||||
and the [sample systemd timer
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.timer).
|
||||
Then, from the directory where you downloaded them:
|
||||
|
||||
```bash
|
||||
sudo mv borgmatic.service borgmatic.timer /etc/systemd/system/
|
||||
sudo systemctl enable borgmatic.timer
|
||||
sudo systemctl start borgmatic.timer
|
||||
```
|
||||
|
||||
Feel free to modify the timer file based on how frequently you'd like
|
||||
borgmatic to run.
|
||||
|
||||
|
||||
## Advanced configuration
|
||||
|
||||
### Multiple configuration files
|
||||
|
||||
A more advanced usage is to create multiple separate configuration files and
|
||||
place each one in an /etc/borgmatic.d directory. For instance:
|
||||
|
||||
```bash
|
||||
sudo mkdir /etc/borgmatic.d
|
||||
sudo generate-borgmatic-config --destination /etc/borgmatic.d/app1.yaml
|
||||
sudo generate-borgmatic-config --destination /etc/borgmatic.d/app2.yaml
|
||||
```
|
||||
|
||||
With this approach, you can have entirely different backup policies for
|
||||
different applications on your system. For instance, you may want one backup
|
||||
configuration for your database data directory, and a different configuration
|
||||
for your user home directories.
|
||||
|
||||
When you set up multiple configuration files like this, borgmatic will run
|
||||
each one in turn from a single borgmatic invocation. This includes, by
|
||||
default, the traditional /etc/borgmatic/config.yaml as well.
|
||||
|
||||
And if you need even more customizability, you can specify alternate
|
||||
configuration paths on the command-line with borgmatic's `--config` option.
|
||||
See `borgmatic --help` for more information.
|
||||
|
||||
|
||||
### Hooks
|
||||
|
||||
If you find yourself performing prepraration tasks before your backup runs, or
|
||||
cleanup work afterwards, borgmatic hooks may be of interest. They're simply
|
||||
shell commands that borgmatic executes for you at various points, and they're
|
||||
configured in the `hooks` section of your configuration file.
|
||||
|
||||
For instance, you can specify `before_backup` hooks to dump a database to file
|
||||
before backing it up, and specify `after_backup` hooks to delete the temporary
|
||||
file afterwards.
|
||||
|
||||
borgmatic hooks run once per configuration file. `before_backup` hooks run
|
||||
prior to backups of all repositories. `after_backup` hooks run afterwards, but
|
||||
not if an error occurs in a previous hook or in the backups themselves. And
|
||||
borgmatic runs `on_error` hooks if an error occurs.
|
||||
|
||||
An important security note about hooks: borgmatic executes all hook commands
|
||||
with the user permissions of borgmatic itself. So to prevent potential shell
|
||||
injection or privilege escalation, do not forget to set secure permissions
|
||||
(`chmod 0700`) on borgmatic configuration files and scripts invoked by hooks.
|
||||
|
||||
See the sample generated configuration file mentioned above for specifics
|
||||
about hook configuration syntax.
|
||||
|
||||
|
||||
## Upgrading
|
||||
|
||||
In general, all you should need to do to upgrade borgmatic is run the
|
||||
following:
|
||||
|
||||
```bash
|
||||
sudo pip3 install --upgrade borgmatic
|
||||
```
|
||||
|
||||
However, see below about special cases.
|
||||
|
||||
|
||||
### Upgrading from borgmatic 1.0.x
|
||||
|
||||
borgmatic changed its configuration file format in version 1.1.0 from
|
||||
INI-style to YAML. This better supports validation, and has a more natural way
|
||||
to express lists of values. To upgrade your existing configuration, first
|
||||
upgrade to the new version of borgmatic.
|
||||
|
||||
As of version 1.1.0, borgmatic no longer supports Python 2. If you were
|
||||
already running borgmatic with Python 3, then you can simply upgrade borgmatic
|
||||
in-place:
|
||||
|
||||
```bash
|
||||
sudo pip3 install --upgrade borgmatic
|
||||
```
|
||||
|
||||
But if you were running borgmatic with Python 2, uninstall and reinstall instead:
|
||||
|
||||
```bash
|
||||
sudo pip uninstall borgmatic
|
||||
sudo pip3 install borgmatic
|
||||
```
|
||||
|
||||
The pip binary names for different versions of Python can differ, so the above
|
||||
commands may need some tweaking to work on your machine.
|
||||
|
||||
|
||||
Once borgmatic is upgraded, run:
|
||||
|
||||
```bash
|
||||
sudo upgrade-borgmatic-config
|
||||
```
|
||||
|
||||
That will generate a new YAML configuration file at /etc/borgmatic/config.yaml
|
||||
(by default) using the values from both your existing configuration and
|
||||
excludes files. The new version of borgmatic will consume the YAML
|
||||
configuration file instead of the old one.
|
||||
|
||||
|
||||
### Upgrading from atticmatic
|
||||
|
||||
You can ignore this section if you're not an atticmatic user (the former name
|
||||
of borgmatic).
|
||||
|
||||
borgmatic only supports Borg now and no longer supports Attic. So if you're
|
||||
an Attic user, consider switching to Borg. See the [Borg upgrade
|
||||
command](https://borgbackup.readthedocs.io/en/stable/usage.html#borg-upgrade)
|
||||
for more information. Then, follow the instructions above about setting up
|
||||
your borgmatic configuration files.
|
||||
|
||||
If you were already using Borg with atticmatic, then you can easily upgrade
|
||||
from atticmatic to borgmatic. Simply run the following commands:
|
||||
|
||||
```bash
|
||||
sudo pip3 uninstall atticmatic
|
||||
sudo pip3 install borgmatic
|
||||
```
|
||||
|
||||
That's it! borgmatic will continue using your /etc/borgmatic configuration
|
||||
files.
|
||||
* [borgmatic configuration reference](docs/reference/configuration.md)
|
||||
* [borgmatic command-line reference](docs/reference/command-line.md)
|
||||
|
||||
|
||||
## Support and contributing
|
||||
@@ -404,127 +93,6 @@ or open an [issue](https://projects.torsion.org/witten/borgmatic/issues) first
|
||||
to discuss your idea. We also accept Pull Requests on GitHub, if that's more
|
||||
your thing. In general, contributions are very welcome. We don't bite!
|
||||
|
||||
|
||||
### Code style
|
||||
|
||||
Start with [PEP 8](https://www.python.org/dev/peps/pep-0008/). But then, apply
|
||||
the following deviations from it:
|
||||
|
||||
* For strings, prefer single quotes over double quotes.
|
||||
* Limit all lines to a maximum of 100 characters.
|
||||
* Use trailing commas within multiline values or argument lists.
|
||||
* For multiline constructs, put opening and closing delimeters on lines
|
||||
separate from their contents.
|
||||
* Within multiline constructs, use standard four-space indentation. Don't align
|
||||
indentation with an opening delimeter.
|
||||
|
||||
borgmatic code uses the [Black](https://black.readthedocs.io/en/stable/) code
|
||||
formatter and [Flake8](http://flake8.pycqa.org/en/latest/) code checker, so
|
||||
certain code style requirements will be enforced when running automated tests.
|
||||
See the Black and Flake8 documentation for more information.
|
||||
|
||||
|
||||
### Development
|
||||
|
||||
To get set up to hack on borgmatic, first clone master via HTTPS or SSH:
|
||||
|
||||
```bash
|
||||
git clone https://projects.torsion.org/witten/borgmatic.git
|
||||
```
|
||||
|
||||
Or:
|
||||
|
||||
```bash
|
||||
git clone ssh://git@projects.torsion.org:3022/witten/borgmatic.git
|
||||
```
|
||||
|
||||
Then, install borgmatic
|
||||
"[editable](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs)"
|
||||
so that you can easily run borgmatic commands while you're hacking on them to
|
||||
make sure your changes work.
|
||||
|
||||
```bash
|
||||
cd borgmatic/
|
||||
pip3 install --editable --user .
|
||||
```
|
||||
|
||||
Note that this will typically install the borgmatic commands into
|
||||
`~/.local/bin`, which may or may not be on your PATH. There are other ways to
|
||||
install borgmatic editable as well, for instance into the system Python
|
||||
install (so without `--user`, as root), or even into a
|
||||
[virtualenv](https://virtualenv.pypa.io/en/stable/). How or where you install
|
||||
borgmatic is up to you, but generally an editable install makes development
|
||||
and testing easier.
|
||||
|
||||
|
||||
### Running tests
|
||||
|
||||
Assuming you've cloned the borgmatic source code as described above, and
|
||||
you're in the `borgmatic/` working copy, install tox, which is used for
|
||||
setting up testing environments:
|
||||
|
||||
```bash
|
||||
sudo pip3 install tox
|
||||
```
|
||||
|
||||
Finally, to actually run tests, run:
|
||||
|
||||
```bash
|
||||
cd borgmatic
|
||||
tox
|
||||
```
|
||||
|
||||
If when running tests, you get an error from the
|
||||
[Black](https://black.readthedocs.io/en/stable/) code formatter about files
|
||||
that would be reformatted, you can ask Black to format them for you via the
|
||||
following:
|
||||
|
||||
```bash
|
||||
tox -e black
|
||||
```
|
||||
|
||||
### End-to-end tests
|
||||
|
||||
borgmatic additionally includes some end-to-end tests that integration test
|
||||
with Borg for a few representative scenarios. These tests don't run by default
|
||||
because they're relatively slow and depend on Borg. If you would like to run
|
||||
them:
|
||||
|
||||
```bash
|
||||
tox -e end-to-end
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Broken pipe with remote repository
|
||||
|
||||
When running borgmatic on a large remote repository, you may receive errors
|
||||
like the following, particularly while "borg check" is validating backups for
|
||||
consistency:
|
||||
|
||||
```text
|
||||
Write failed: Broken pipe
|
||||
borg: Error: Connection closed by remote host
|
||||
```
|
||||
|
||||
This error can be caused by an ssh timeout, which you can rectify by adding
|
||||
the following to the `~/.ssh/config` file on the client:
|
||||
|
||||
```text
|
||||
Host *
|
||||
ServerAliveInterval 120
|
||||
```
|
||||
|
||||
This should make the client keep the connection alive while validating
|
||||
backups.
|
||||
|
||||
|
||||
### libyaml compilation errors
|
||||
|
||||
borgmatic depends on a Python YAML library (ruamel.yaml) that will optionally
|
||||
use a C YAML library (libyaml) if present. But if it's not installed, then
|
||||
when installing or upgrading borgmatic, you may see errors about compiling the
|
||||
YAML library. If so, not to worry. borgmatic should install and function
|
||||
correctly even without the C YAML library. And borgmatic won't be any faster
|
||||
with the C library present, so you don't need to go out of your way to install
|
||||
it.
|
||||
Also, please check out the [borgmatic development
|
||||
how-to](docs/how-to/develop-on-borgmatic.md) for info on cloning source code,
|
||||
running tests, etc.
|
||||
|
||||
@@ -32,6 +32,17 @@ def _expand_directories(directories):
|
||||
)
|
||||
|
||||
|
||||
def _expand_home_directories(directories):
|
||||
'''
|
||||
Given a sequence of directory paths, expand tildes in each one. Do not perform any globbing.
|
||||
Return the results as a tuple.
|
||||
'''
|
||||
if directories is None:
|
||||
return ()
|
||||
|
||||
return tuple(os.path.expanduser(directory) for directory in directories)
|
||||
|
||||
|
||||
def _write_pattern_file(patterns=None):
|
||||
'''
|
||||
Given a sequence of patterns, write them to a named temporary file and return it. Return None
|
||||
@@ -91,6 +102,7 @@ def create_archive(
|
||||
local_path='borg',
|
||||
remote_path=None,
|
||||
progress=False,
|
||||
stats=False,
|
||||
json=False,
|
||||
):
|
||||
'''
|
||||
@@ -100,7 +112,9 @@ def create_archive(
|
||||
sources = _expand_directories(location_config['source_directories'])
|
||||
|
||||
pattern_file = _write_pattern_file(location_config.get('patterns'))
|
||||
exclude_file = _write_pattern_file(_expand_directories(location_config.get('exclude_patterns')))
|
||||
exclude_file = _write_pattern_file(
|
||||
_expand_home_directories(location_config.get('exclude_patterns'))
|
||||
)
|
||||
checkpoint_interval = storage_config.get('checkpoint_interval', None)
|
||||
chunker_params = storage_config.get('chunker_params', None)
|
||||
compression = storage_config.get('compression', None)
|
||||
@@ -135,7 +149,7 @@ def create_archive(
|
||||
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
|
||||
+ (('--list', '--filter', 'AME-') if logger.isEnabledFor(logging.INFO) else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--stats',) if not dry_run and logger.isEnabledFor(logging.INFO) else ())
|
||||
+ (('--stats',) if not dry_run and (logger.isEnabledFor(logging.INFO) or stats) else ())
|
||||
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
|
||||
+ (('--dry-run',) if dry_run else ())
|
||||
+ (('--progress',) if progress else ())
|
||||
|
||||
@@ -8,8 +8,8 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
def extract_last_archive_dry_run(repository, lock_wait=None, local_path='borg', remote_path=None):
|
||||
'''
|
||||
Perform an extraction dry-run of just the most recent archive. If there are no archives, skip
|
||||
the dry-run.
|
||||
Perform an extraction dry-run of the most recent archive. If there are no archives, skip the
|
||||
dry-run.
|
||||
'''
|
||||
remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
|
||||
lock_wait_flags = ('--lock-wait', str(lock_wait)) if lock_wait else ()
|
||||
@@ -50,3 +50,37 @@ def extract_last_archive_dry_run(repository, lock_wait=None, local_path='borg',
|
||||
|
||||
logger.debug(' '.join(full_extract_command))
|
||||
subprocess.check_call(full_extract_command)
|
||||
|
||||
|
||||
def extract_archive(
|
||||
dry_run,
|
||||
repository,
|
||||
archive,
|
||||
restore_paths,
|
||||
storage_config,
|
||||
local_path='borg',
|
||||
remote_path=None,
|
||||
progress=False,
|
||||
):
|
||||
'''
|
||||
Given a dry-run flag, a local or remote repository path, an archive name, zero or more paths to
|
||||
restore from the archive, and a storage configuration dict, extract the archive into the current
|
||||
directory.
|
||||
'''
|
||||
umask = storage_config.get('umask', None)
|
||||
lock_wait = storage_config.get('lock_wait', None)
|
||||
|
||||
full_command = (
|
||||
(local_path, 'extract', '::'.join((repository, archive)))
|
||||
+ (tuple(restore_paths) if restore_paths else ())
|
||||
+ (('--remote-path', remote_path) if remote_path else ())
|
||||
+ (('--umask', str(umask)) if umask else ())
|
||||
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--debug', '--list', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
|
||||
+ (('--dry-run',) if dry_run else ())
|
||||
+ (('--progress',) if progress else ())
|
||||
)
|
||||
|
||||
logger.debug(' '.join(full_command))
|
||||
subprocess.check_call(full_command)
|
||||
|
||||
+12
-4
@@ -15,9 +15,17 @@ def initialize_repository(
|
||||
):
|
||||
'''
|
||||
Given a local or remote repository path, a Borg encryption mode, whether the repository should
|
||||
be append-only, and the storage quota to use, initialize the repository.
|
||||
be append-only, and the storage quota to use, initialize the repository. If the repository
|
||||
already exists, then log and skip initialization.
|
||||
'''
|
||||
full_command = (
|
||||
info_command = (local_path, 'info', repository)
|
||||
logger.debug(' '.join(info_command))
|
||||
|
||||
if subprocess.call(info_command, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) == 0:
|
||||
logger.info('Repository already exists. Skipping initialization.')
|
||||
return
|
||||
|
||||
init_command = (
|
||||
(local_path, 'init', repository)
|
||||
+ (('--encryption', encryption_mode) if encryption_mode else ())
|
||||
+ (('--append-only',) if append_only else ())
|
||||
@@ -27,5 +35,5 @@ def initialize_repository(
|
||||
+ (('--remote-path', remote_path) if remote_path else ())
|
||||
)
|
||||
|
||||
logger.debug(' '.join(full_command))
|
||||
subprocess.check_call(full_command)
|
||||
logger.debug(' '.join(init_command))
|
||||
subprocess.check_call(init_command)
|
||||
|
||||
@@ -5,15 +5,17 @@ import subprocess
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def list_archives(repository, storage_config, local_path='borg', remote_path=None, json=False):
|
||||
def list_archives(
|
||||
repository, storage_config, archive=None, local_path='borg', remote_path=None, json=False
|
||||
):
|
||||
'''
|
||||
Given a local or remote repository path, and a storage config dict,
|
||||
list Borg archives in the repository.
|
||||
Given a local or remote repository path and a storage config dict, list Borg archives in the
|
||||
repository. Or, if an archive name is given, list the files in that archive.
|
||||
'''
|
||||
lock_wait = storage_config.get('lock_wait', None)
|
||||
|
||||
full_command = (
|
||||
(local_path, 'list', repository)
|
||||
(local_path, 'list', '::'.join((repository, archive)) if archive else repository)
|
||||
+ (('--remote-path', remote_path) if remote_path else ())
|
||||
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
|
||||
@@ -31,7 +31,13 @@ def _make_prune_flags(retention_config):
|
||||
|
||||
|
||||
def prune_archives(
|
||||
dry_run, repository, storage_config, retention_config, local_path='borg', remote_path=None
|
||||
dry_run,
|
||||
repository,
|
||||
storage_config,
|
||||
retention_config,
|
||||
local_path='borg',
|
||||
remote_path=None,
|
||||
stats=False,
|
||||
):
|
||||
'''
|
||||
Given dry-run flag, a local or remote repository path, a storage config dict, and a
|
||||
@@ -51,6 +57,7 @@ def prune_archives(
|
||||
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
|
||||
+ (('--debug', '--list', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
|
||||
+ (('--dry-run',) if dry_run else ())
|
||||
+ (('--stats',) if stats else ())
|
||||
)
|
||||
|
||||
logger.debug(' '.join(full_command))
|
||||
|
||||
+284
-110
@@ -1,15 +1,19 @@
|
||||
from argparse import ArgumentParser
|
||||
import collections
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from subprocess import CalledProcessError
|
||||
import sys
|
||||
|
||||
import pkg_resources
|
||||
|
||||
from borgmatic.borg import (
|
||||
check as borg_check,
|
||||
create as borg_create,
|
||||
environment as borg_environment,
|
||||
prune as borg_prune,
|
||||
extract as borg_extract,
|
||||
list as borg_list,
|
||||
info as borg_info,
|
||||
init as borg_init,
|
||||
@@ -29,18 +33,121 @@ LEGACY_CONFIG_PATH = '/etc/borgmatic/config'
|
||||
def parse_arguments(*arguments):
|
||||
'''
|
||||
Given command-line arguments with which this script was invoked, parse the arguments and return
|
||||
them as an ArgumentParser instance.
|
||||
them as an argparse.ArgumentParser instance.
|
||||
'''
|
||||
config_paths = collect.get_default_config_paths()
|
||||
|
||||
parser = ArgumentParser(
|
||||
description='''
|
||||
A simple wrapper script for the Borg backup software that creates and prunes backups.
|
||||
If none of the --prune, --create, or --check options are given, then borgmatic defaults
|
||||
to all three: prune, create, and check archives.
|
||||
'''
|
||||
If none of the action options are given, then borgmatic defaults to: prune, create, and
|
||||
check archives.
|
||||
''',
|
||||
add_help=False,
|
||||
)
|
||||
parser.add_argument(
|
||||
|
||||
actions_group = parser.add_argument_group('actions')
|
||||
actions_group.add_argument(
|
||||
'-I', '--init', dest='init', action='store_true', help='Initialize an empty Borg repository'
|
||||
)
|
||||
actions_group.add_argument(
|
||||
'-p',
|
||||
'--prune',
|
||||
dest='prune',
|
||||
action='store_true',
|
||||
help='Prune archives according to the retention policy',
|
||||
)
|
||||
actions_group.add_argument(
|
||||
'-C',
|
||||
'--create',
|
||||
dest='create',
|
||||
action='store_true',
|
||||
help='Create archives (actually perform backups)',
|
||||
)
|
||||
actions_group.add_argument(
|
||||
'-k', '--check', dest='check', action='store_true', help='Check archives for consistency'
|
||||
)
|
||||
|
||||
actions_group.add_argument(
|
||||
'-x',
|
||||
'--extract',
|
||||
dest='extract',
|
||||
action='store_true',
|
||||
help='Extract a named archive to the current directory',
|
||||
)
|
||||
actions_group.add_argument(
|
||||
'-l', '--list', dest='list', action='store_true', help='List archives'
|
||||
)
|
||||
actions_group.add_argument(
|
||||
'-i',
|
||||
'--info',
|
||||
dest='info',
|
||||
action='store_true',
|
||||
help='Display summary information on archives',
|
||||
)
|
||||
|
||||
init_group = parser.add_argument_group('options for --init')
|
||||
init_group.add_argument(
|
||||
'-e', '--encryption', dest='encryption_mode', help='Borg repository encryption mode'
|
||||
)
|
||||
init_group.add_argument(
|
||||
'--append-only',
|
||||
dest='append_only',
|
||||
action='store_true',
|
||||
help='Create an append-only repository',
|
||||
)
|
||||
init_group.add_argument(
|
||||
'--storage-quota',
|
||||
dest='storage_quota',
|
||||
help='Create a repository with a fixed storage quota',
|
||||
)
|
||||
|
||||
prune_group = parser.add_argument_group('options for --prune')
|
||||
stats_argument = prune_group.add_argument(
|
||||
'--stats',
|
||||
dest='stats',
|
||||
default=False,
|
||||
action='store_true',
|
||||
help='Display statistics of archive',
|
||||
)
|
||||
|
||||
create_group = parser.add_argument_group('options for --create')
|
||||
progress_argument = create_group.add_argument(
|
||||
'--progress',
|
||||
dest='progress',
|
||||
default=False,
|
||||
action='store_true',
|
||||
help='Display progress for each file as it is processed',
|
||||
)
|
||||
create_group._group_actions.append(stats_argument)
|
||||
json_argument = create_group.add_argument(
|
||||
'--json', dest='json', default=False, action='store_true', help='Output results as JSON'
|
||||
)
|
||||
|
||||
extract_group = parser.add_argument_group('options for --extract')
|
||||
repository_argument = extract_group.add_argument(
|
||||
'--repository',
|
||||
help='Path of repository to use, defaults to the configured repository if there is only one',
|
||||
)
|
||||
archive_argument = extract_group.add_argument('--archive', help='Name of archive to operate on')
|
||||
extract_group.add_argument(
|
||||
'--restore-path',
|
||||
nargs='+',
|
||||
dest='restore_paths',
|
||||
help='Paths to restore from archive, defaults to the entire archive',
|
||||
)
|
||||
extract_group._group_actions.append(progress_argument)
|
||||
|
||||
list_group = parser.add_argument_group('options for --list')
|
||||
list_group._group_actions.append(repository_argument)
|
||||
list_group._group_actions.append(archive_argument)
|
||||
list_group._group_actions.append(json_argument)
|
||||
|
||||
info_group = parser.add_argument_group('options for --info')
|
||||
info_group._group_actions.append(json_argument)
|
||||
|
||||
common_group = parser.add_argument_group('common options')
|
||||
common_group.add_argument(
|
||||
'-c',
|
||||
'--config',
|
||||
nargs='+',
|
||||
@@ -50,78 +157,19 @@ def parse_arguments(*arguments):
|
||||
' '.join(config_paths)
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
common_group.add_argument(
|
||||
'--excludes',
|
||||
dest='excludes_filename',
|
||||
help='Deprecated in favor of exclude_patterns within configuration',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-I', '--init', dest='init', action='store_true', help='Initialize an empty Borg repository'
|
||||
)
|
||||
parser.add_argument(
|
||||
'-e',
|
||||
'--encryption',
|
||||
dest='encryption_mode',
|
||||
help='Borg repository encryption mode (for use with --init)',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--append-only',
|
||||
dest='append_only',
|
||||
action='store_true',
|
||||
help='Create an append-only repository (for use with --init)',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--storage-quota',
|
||||
dest='storage_quota',
|
||||
help='Create a repository with a fixed storage quota (for use with --init)',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-p',
|
||||
'--prune',
|
||||
dest='prune',
|
||||
action='store_true',
|
||||
help='Prune archives according to the retention policy',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-C',
|
||||
'--create',
|
||||
dest='create',
|
||||
action='store_true',
|
||||
help='Create archives (actually perform backups)',
|
||||
)
|
||||
parser.add_argument(
|
||||
'-k', '--check', dest='check', action='store_true', help='Check archives for consistency'
|
||||
)
|
||||
parser.add_argument('-l', '--list', dest='list', action='store_true', help='List archives')
|
||||
parser.add_argument(
|
||||
'-i',
|
||||
'--info',
|
||||
dest='info',
|
||||
action='store_true',
|
||||
help='Display summary information on archives',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--progress',
|
||||
dest='progress',
|
||||
default=False,
|
||||
action='store_true',
|
||||
help='Display progress with --create option for each file as it is backed up',
|
||||
)
|
||||
parser.add_argument(
|
||||
'--json',
|
||||
dest='json',
|
||||
default=False,
|
||||
action='store_true',
|
||||
help='Output results from the --create, --list, or --info options as json',
|
||||
)
|
||||
parser.add_argument(
|
||||
common_group.add_argument(
|
||||
'-n',
|
||||
'--dry-run',
|
||||
dest='dry_run',
|
||||
action='store_true',
|
||||
help='Go through the motions, but do not actually write to any repositories',
|
||||
)
|
||||
parser.add_argument(
|
||||
common_group.add_argument(
|
||||
'-v',
|
||||
'--verbosity',
|
||||
type=int,
|
||||
@@ -129,6 +177,14 @@ def parse_arguments(*arguments):
|
||||
default=0,
|
||||
help='Display verbose progress (1 for some, 2 for lots)',
|
||||
)
|
||||
common_group.add_argument(
|
||||
'--version',
|
||||
dest='version',
|
||||
default=False,
|
||||
action='store_true',
|
||||
help='Display installed version number of borgmatic and exit',
|
||||
)
|
||||
common_group.add_argument('--help', action='help', help='Show this help information and exit')
|
||||
|
||||
args = parser.parse_args(arguments)
|
||||
|
||||
@@ -142,15 +198,30 @@ def parse_arguments(*arguments):
|
||||
'The --encryption, --append-only, and --storage-quota options can only be used with the --init option'
|
||||
)
|
||||
|
||||
if args.init and (args.prune or args.create or args.dry_run):
|
||||
raise ValueError(
|
||||
'The --init option cannot be used with the --prune, --create, or --dry-run options'
|
||||
)
|
||||
if args.init and args.dry_run:
|
||||
raise ValueError('The --init option cannot be used with the --dry-run option')
|
||||
if args.init and not args.encryption_mode:
|
||||
raise ValueError('The --encryption option is required with the --init option')
|
||||
|
||||
if args.progress and not args.create:
|
||||
raise ValueError('The --progress option can only be used with the --create option')
|
||||
if not args.extract:
|
||||
if not args.list:
|
||||
if args.repository:
|
||||
raise ValueError(
|
||||
'The --repository option can only be used with the --extract and --list options'
|
||||
)
|
||||
if args.archive:
|
||||
raise ValueError(
|
||||
'The --archive option can only be used with the --extract and --list options'
|
||||
)
|
||||
if args.restore_paths:
|
||||
raise ValueError('The --restore-path option can only be used with the --extract option')
|
||||
if args.extract and not args.archive:
|
||||
raise ValueError('The --archive option is required with the --extract option')
|
||||
|
||||
if args.progress and not (args.create or args.extract):
|
||||
raise ValueError(
|
||||
'The --progress option can only be used with the --create and --extract options'
|
||||
)
|
||||
|
||||
if args.json and not (args.create or args.list or args.info):
|
||||
raise ValueError(
|
||||
@@ -164,22 +235,30 @@ def parse_arguments(*arguments):
|
||||
|
||||
# If any of the action flags are explicitly requested, leave them as-is. Otherwise, assume
|
||||
# defaults: Mutate the given arguments to enable the default actions.
|
||||
if args.init or args.prune or args.create or args.check or args.list or args.info:
|
||||
return args
|
||||
if (
|
||||
not args.init
|
||||
and not args.prune
|
||||
and not args.create
|
||||
and not args.check
|
||||
and not args.extract
|
||||
and not args.list
|
||||
and not args.info
|
||||
):
|
||||
args.prune = True
|
||||
args.create = True
|
||||
args.check = True
|
||||
|
||||
if args.stats and not (args.create or args.prune):
|
||||
raise ValueError('The --stats option can only be used when creating or pruning archives')
|
||||
|
||||
args.prune = True
|
||||
args.create = True
|
||||
args.check = True
|
||||
return args
|
||||
|
||||
|
||||
def run_configuration(config_filename, args): # pragma: no cover
|
||||
def run_configuration(config_filename, config, args): # pragma: no cover
|
||||
'''
|
||||
Parse a single configuration file, and execute its defined pruning, backups, and/or consistency
|
||||
checks.
|
||||
Given a config filename and the corresponding parsed config dict, execute its defined pruning,
|
||||
backups, consistency checks, and/or other actions.
|
||||
'''
|
||||
logger.info('{}: Parsing configuration file'.format(config_filename))
|
||||
config = validate.parse_configuration(config_filename, validate.schema_filename())
|
||||
(location, storage, retention, consistency, hooks) = (
|
||||
config.get(section_name, {})
|
||||
for section_name in ('location', 'storage', 'retention', 'consistency', 'hooks')
|
||||
@@ -261,6 +340,7 @@ def _run_commands_on_repository(
|
||||
retention,
|
||||
local_path=local_path,
|
||||
remote_path=remote_path,
|
||||
stats=args.stats,
|
||||
)
|
||||
if args.create:
|
||||
logger.info('{}: Creating archive{}'.format(repository, dry_run_label))
|
||||
@@ -272,21 +352,41 @@ def _run_commands_on_repository(
|
||||
local_path=local_path,
|
||||
remote_path=remote_path,
|
||||
progress=args.progress,
|
||||
stats=args.stats,
|
||||
)
|
||||
if args.check and checks.repository_enabled_for_checks(repository, consistency):
|
||||
logger.info('{}: Running consistency checks'.format(repository))
|
||||
borg_check.check_archives(
|
||||
repository, storage, consistency, local_path=local_path, remote_path=remote_path
|
||||
)
|
||||
if args.extract:
|
||||
if args.repository is None or repository == args.repository:
|
||||
logger.info('{}: Extracting archive {}'.format(repository, args.archive))
|
||||
borg_extract.extract_archive(
|
||||
args.dry_run,
|
||||
repository,
|
||||
args.archive,
|
||||
args.restore_paths,
|
||||
storage,
|
||||
local_path=local_path,
|
||||
remote_path=remote_path,
|
||||
progress=args.progress,
|
||||
)
|
||||
if args.list:
|
||||
logger.info('{}: Listing archives'.format(repository))
|
||||
output = borg_list.list_archives(
|
||||
repository, storage, local_path=local_path, remote_path=remote_path, json=args.json
|
||||
)
|
||||
if args.json:
|
||||
json_results.append(json.loads(output))
|
||||
else:
|
||||
sys.stdout.write(output)
|
||||
if args.repository is None or repository == args.repository:
|
||||
logger.info('{}: Listing archives'.format(repository))
|
||||
output = borg_list.list_archives(
|
||||
repository,
|
||||
storage,
|
||||
args.archive,
|
||||
local_path=local_path,
|
||||
remote_path=remote_path,
|
||||
json=args.json,
|
||||
)
|
||||
if args.json:
|
||||
json_results.append(json.loads(output))
|
||||
else:
|
||||
sys.stdout.write(output)
|
||||
if args.info:
|
||||
logger.info('{}: Displaying summary info for archives'.format(repository))
|
||||
output = borg_info.display_archives_info(
|
||||
@@ -298,25 +398,99 @@ def _run_commands_on_repository(
|
||||
sys.stdout.write(output)
|
||||
|
||||
|
||||
def main(): # pragma: no cover
|
||||
try:
|
||||
configure_signals()
|
||||
args = parse_arguments(*sys.argv[1:])
|
||||
logging.basicConfig(level=verbosity_to_log_level(args.verbosity), format='%(message)s')
|
||||
def collect_configuration_run_summary_logs(config_filenames, args):
|
||||
'''
|
||||
Given a sequence of configuration filenames and parsed command-line arguments as an
|
||||
argparse.ArgumentParser instance, run each configuration file and yield a series of
|
||||
logging.LogRecord instances containing summary information about each run.
|
||||
'''
|
||||
# Dict mapping from config filename to corresponding parsed config dict.
|
||||
configs = collections.OrderedDict()
|
||||
|
||||
config_filenames = tuple(collect.collect_config_filenames(args.config_paths))
|
||||
logger.debug('Ensuring legacy configuration is upgraded')
|
||||
convert.guard_configuration_upgraded(LEGACY_CONFIG_PATH, config_filenames)
|
||||
|
||||
if len(config_filenames) == 0:
|
||||
raise ValueError(
|
||||
'Error: No configuration files found in: {}'.format(' '.join(args.config_paths))
|
||||
# Parse and load each configuration file.
|
||||
for config_filename in config_filenames:
|
||||
try:
|
||||
logger.info('{}: Parsing configuration file'.format(config_filename))
|
||||
configs[config_filename] = validate.parse_configuration(
|
||||
config_filename, validate.schema_filename()
|
||||
)
|
||||
except (ValueError, OSError, validate.Validation_error) as error:
|
||||
yield logging.makeLogRecord(
|
||||
dict(
|
||||
levelno=logging.CRITICAL,
|
||||
msg='{}: Error parsing configuration file'.format(config_filename),
|
||||
)
|
||||
)
|
||||
yield logging.makeLogRecord(dict(levelno=logging.CRITICAL, msg=error))
|
||||
|
||||
for config_filename in config_filenames:
|
||||
run_configuration(config_filename, args)
|
||||
except (ValueError, OSError, CalledProcessError) as error:
|
||||
print(error, file=sys.stderr)
|
||||
print(file=sys.stderr)
|
||||
print('Need some help? https://torsion.org/borgmatic/#issues', file=sys.stderr)
|
||||
sys.exit(1)
|
||||
# Run cross-file validation checks.
|
||||
if args.extract or (args.list and args.archive):
|
||||
try:
|
||||
validate.guard_configuration_contains_repository(args.repository, configs)
|
||||
except ValueError as error:
|
||||
yield logging.makeLogRecord(dict(levelno=logging.CRITICAL, msg=error))
|
||||
return
|
||||
|
||||
# Execute the actions corresponding to each configuration file.
|
||||
for config_filename, config in configs.items():
|
||||
try:
|
||||
run_configuration(config_filename, config, args)
|
||||
yield logging.makeLogRecord(
|
||||
dict(
|
||||
levelno=logging.INFO,
|
||||
msg='{}: Successfully ran configuration file'.format(config_filename),
|
||||
)
|
||||
)
|
||||
except (ValueError, OSError, CalledProcessError) as error:
|
||||
yield logging.makeLogRecord(
|
||||
dict(
|
||||
levelno=logging.CRITICAL,
|
||||
msg='{}: Error running configuration file'.format(config_filename),
|
||||
)
|
||||
)
|
||||
yield logging.makeLogRecord(dict(levelno=logging.CRITICAL, msg=error))
|
||||
|
||||
if not config_filenames:
|
||||
yield logging.makeLogRecord(
|
||||
dict(
|
||||
levelno=logging.CRITICAL,
|
||||
msg='{}: No configuration files found'.format(' '.join(args.config_paths)),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def exit_with_help_link(): # pragma: no cover
|
||||
'''
|
||||
Display a link to get help and exit with an error code.
|
||||
'''
|
||||
logger.critical('\nNeed some help? https://torsion.org/borgmatic/#issues')
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main(): # pragma: no cover
|
||||
configure_signals()
|
||||
|
||||
try:
|
||||
args = parse_arguments(*sys.argv[1:])
|
||||
except ValueError as error:
|
||||
logging.basicConfig(level=logging.CRITICAL, format='%(message)s')
|
||||
logger.critical(error)
|
||||
exit_with_help_link()
|
||||
|
||||
logging.basicConfig(level=verbosity_to_log_level(args.verbosity), format='%(message)s')
|
||||
|
||||
if args.version:
|
||||
print(pkg_resources.require('borgmatic')[0].version)
|
||||
sys.exit(0)
|
||||
|
||||
config_filenames = tuple(collect.collect_config_filenames(args.config_paths))
|
||||
logger.debug('Ensuring legacy configuration is upgraded')
|
||||
convert.guard_configuration_upgraded(LEGACY_CONFIG_PATH, config_filenames)
|
||||
|
||||
summary_logs = tuple(collect_configuration_run_summary_logs(config_filenames, args))
|
||||
|
||||
logger.info('\nsummary:')
|
||||
[logger.handle(log) for log in summary_logs if log.levelno >= logger.getEffectiveLevel()]
|
||||
|
||||
if any(log.levelno == logging.CRITICAL for log in summary_logs):
|
||||
exit_with_help_link()
|
||||
|
||||
@@ -36,7 +36,7 @@ def main(): # pragma: no cover
|
||||
|
||||
print('Generated a sample configuration file at {}.'.format(args.destination_filename))
|
||||
print()
|
||||
print('Please edit the file to suit your needs. The values are just representative.')
|
||||
print('Please edit the file to suit your needs. The values are representative.')
|
||||
print('All fields are optional except where indicated.')
|
||||
print()
|
||||
print('If you ever need help: https://torsion.org/borgmatic/#issues')
|
||||
|
||||
@@ -20,10 +20,10 @@ def get_default_config_paths():
|
||||
|
||||
def collect_config_filenames(config_paths):
|
||||
'''
|
||||
Given a sequence of config paths, both filenames and directories, resolve that to just an
|
||||
iterable of files. Accomplish this by listing any given directories looking for contained config
|
||||
files (ending with the ".yaml" extension). This is non-recursive, so any directories within the
|
||||
given directories are ignored.
|
||||
Given a sequence of config paths, both filenames and directories, resolve that to an iterable
|
||||
of files. Accomplish this by listing any given directories looking for contained config files
|
||||
(ending with the ".yaml" extension). This is non-recursive, so any directories within the given
|
||||
directories are ignored.
|
||||
|
||||
Return paths even if they don't exist on disk, so the user can find out about missing
|
||||
configuration paths. However, skip a default config path if it's missing, so the user doesn't
|
||||
|
||||
@@ -93,21 +93,3 @@ def guard_configuration_upgraded(source_config_filename, destination_config_file
|
||||
|
||||
if os.path.exists(source_config_filename) and not destination_config_exists:
|
||||
raise LegacyConfigurationNotUpgraded()
|
||||
|
||||
|
||||
class LegacyExcludesFilenamePresent(FileNotFoundError):
|
||||
def __init__(self):
|
||||
super(LegacyExcludesFilenamePresent, self).__init__(
|
||||
'''borgmatic changed its configuration file format in version 1.1.0 from INI-style
|
||||
to YAML. This better supports validation, and has a more natural way to express
|
||||
lists of values. The new configuration file incorporates excludes, so you no
|
||||
longer need to provide an excludes filename on the command-line with an
|
||||
"--excludes" argument.
|
||||
|
||||
Please remove the "--excludes" argument and run borgmatic again.'''
|
||||
)
|
||||
|
||||
|
||||
def guard_excludes_filename_omitted(excludes_filename):
|
||||
if excludes_filename is not None:
|
||||
raise LegacyExcludesFilenamePresent()
|
||||
|
||||
@@ -30,14 +30,14 @@ map:
|
||||
- user@backupserver:sourcehostname.borg
|
||||
one_file_system:
|
||||
type: bool
|
||||
desc: Stay in same file system (do not cross mount points).
|
||||
desc: Stay in same file system (do not cross mount points). Defaults to false.
|
||||
example: true
|
||||
read_special:
|
||||
type: bool
|
||||
desc: |
|
||||
Use Borg's --read-special flag to allow backup of block and other special
|
||||
devices. Use with caution, as it will lead to problems if used when
|
||||
backing up special devices such as /dev/zero.
|
||||
backing up special devices such as /dev/zero. Defaults to false.
|
||||
example: false
|
||||
bsd_flags:
|
||||
type: bool
|
||||
@@ -48,7 +48,7 @@ map:
|
||||
desc: |
|
||||
Mode in which to operate the files cache. See
|
||||
https://borgbackup.readthedocs.io/en/stable/usage/create.html#description for
|
||||
details.
|
||||
details. Defaults to "ctime,size,inode".
|
||||
example: ctime,size,inode
|
||||
local_path:
|
||||
type: scalar
|
||||
@@ -102,11 +102,13 @@ map:
|
||||
type: bool
|
||||
desc: |
|
||||
Exclude directories that contain a CACHEDIR.TAG file. See
|
||||
http://www.brynosaurus.com/cachedir/spec.html for details.
|
||||
http://www.brynosaurus.com/cachedir/spec.html for details. Defaults to false.
|
||||
example: true
|
||||
exclude_if_present:
|
||||
type: scalar
|
||||
desc: Exclude directories that contain a file with the given filename.
|
||||
desc: |
|
||||
Exclude directories that contain a file with the given filename. Defaults to not
|
||||
set.
|
||||
example: .nobackup
|
||||
storage:
|
||||
desc: |
|
||||
@@ -121,7 +123,7 @@ map:
|
||||
The standard output of this command is used to unlock the encryption key. Only
|
||||
use on repositories that were initialized with passcommand/repokey encryption.
|
||||
Note that if both encryption_passcommand and encryption_passphrase are set,
|
||||
then encryption_passphrase takes precedence.
|
||||
then encryption_passphrase takes precedence. Defaults to not set.
|
||||
example: "secret-tool lookup borg-repository repo-name"
|
||||
encryption_passphrase:
|
||||
type: scalar
|
||||
@@ -129,7 +131,7 @@ map:
|
||||
Passphrase to unlock the encryption key with. Only use on repositories that were
|
||||
initialized with passphrase/repokey encryption. Quote the value if it contains
|
||||
punctuation, so it parses correctly. And backslash any quote or backslash
|
||||
literals as well.
|
||||
literals as well. Defaults to not set.
|
||||
example: "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
|
||||
checkpoint_interval:
|
||||
type: int
|
||||
@@ -143,7 +145,7 @@ map:
|
||||
desc: |
|
||||
Specify the parameters passed to then chunker (CHUNK_MIN_EXP, CHUNK_MAX_EXP,
|
||||
HASH_MASK_BITS, HASH_WINDOW_SIZE). See https://borgbackup.readthedocs.io/en/stable/internals.html
|
||||
for details.
|
||||
for details. Defaults to "19,23,21,4095".
|
||||
example: 19,23,21,4095
|
||||
compression:
|
||||
type: scalar
|
||||
@@ -154,25 +156,27 @@ map:
|
||||
example: lz4
|
||||
remote_rate_limit:
|
||||
type: int
|
||||
desc: Remote network upload rate limit in kiBytes/second.
|
||||
desc: Remote network upload rate limit in kiBytes/second. Defaults to unlimited.
|
||||
example: 100
|
||||
ssh_command:
|
||||
type: scalar
|
||||
desc: Command to use instead of just "ssh". This can be used to specify ssh options.
|
||||
desc: |
|
||||
Command to use instead of "ssh". This can be used to specify ssh options.
|
||||
Defaults to not set.
|
||||
example: ssh -i /path/to/private/key
|
||||
umask:
|
||||
type: scalar
|
||||
desc: Umask to be used for borg create.
|
||||
desc: Umask to be used for borg create. Defaults to 0077.
|
||||
example: 0077
|
||||
lock_wait:
|
||||
type: int
|
||||
desc: Maximum seconds to wait for acquiring a repository/cache lock.
|
||||
desc: Maximum seconds to wait for acquiring a repository/cache lock. Defaults to 1.
|
||||
example: 5
|
||||
archive_name_format:
|
||||
type: scalar
|
||||
desc: |
|
||||
Name of the archive. Borg placeholders can be used. See the output of
|
||||
"borg help placeholders" for details. Default is
|
||||
"borg help placeholders" for details. Defaults to
|
||||
"{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}". If you specify this option, you must
|
||||
also specify a prefix in the retention section to avoid accidental pruning of
|
||||
archives with a different archive name format. And you should also specify a
|
||||
@@ -221,7 +225,7 @@ map:
|
||||
desc: |
|
||||
When pruning, only consider archive names starting with this prefix.
|
||||
Borg placeholders can be used. See the output of "borg help placeholders" for
|
||||
details. Default is "{hostname}-".
|
||||
details. Defaults to "{hostname}-".
|
||||
example: sourcehostname
|
||||
consistency:
|
||||
desc: |
|
||||
@@ -239,7 +243,7 @@ map:
|
||||
"extract". Defaults to "repository" and "archives". Set to "disabled" to disable
|
||||
all consistency checks. "repository" checks the consistency of the repository,
|
||||
"archive" checks all of the archives, and "extract" does an extraction dry-run
|
||||
of just the most recent archive.
|
||||
of the most recent archive.
|
||||
example:
|
||||
- repository
|
||||
- archives
|
||||
@@ -256,14 +260,14 @@ map:
|
||||
check_last:
|
||||
type: int
|
||||
desc: Restrict the number of checked archives to the last n. Applies only to the
|
||||
"archives" check.
|
||||
"archives" check. Defaults to checking all archives.
|
||||
example: 3
|
||||
prefix:
|
||||
type: scalar
|
||||
desc: |
|
||||
When performing the "archives" check, only consider archive names starting with
|
||||
this prefix. Borg placeholders can be used. See the output of
|
||||
"borg help placeholders" for details. Default is "{hostname}-".
|
||||
"borg help placeholders" for details. Defaults to "{hostname}-".
|
||||
example: sourcehostname
|
||||
hooks:
|
||||
desc: |
|
||||
@@ -277,16 +281,16 @@ map:
|
||||
- type: scalar
|
||||
desc: List of one or more shell commands or scripts to execute before creating a backup.
|
||||
example:
|
||||
- echo "`date` - Starting a backup job."
|
||||
- echo "Starting a backup job."
|
||||
after_backup:
|
||||
seq:
|
||||
- type: scalar
|
||||
desc: List of one or more shell commands or scripts to execute after creating a backup.
|
||||
example:
|
||||
- echo "`date` - Backup created."
|
||||
- echo "Backup created."
|
||||
on_error:
|
||||
seq:
|
||||
- type: scalar
|
||||
desc: List of one or more shell commands or scripts to execute in case an exception has occurred.
|
||||
example:
|
||||
- echo "`date` - Error while creating a backup."
|
||||
- echo "Error while creating a backup."
|
||||
|
||||
@@ -93,7 +93,7 @@ def parse_configuration(config_filename, schema_filename):
|
||||
raise Validation_error(config_filename, (str(error),))
|
||||
|
||||
# pykwalify gets angry if the example field is not a string. So rather than bend to its will,
|
||||
# simply remove all examples before passing the schema to pykwalify.
|
||||
# remove all examples before passing the schema to pykwalify.
|
||||
for section_name, section_schema in schema['map'].items():
|
||||
for field_name, field_schema in section_schema['map'].items():
|
||||
field_schema.pop('example', None)
|
||||
@@ -107,3 +107,46 @@ def parse_configuration(config_filename, schema_filename):
|
||||
apply_logical_validation(config_filename, parsed_result)
|
||||
|
||||
return parsed_result
|
||||
|
||||
|
||||
def guard_configuration_contains_repository(repository, configurations):
|
||||
'''
|
||||
Given a repository path and a dict mapping from config filename to corresponding parsed config
|
||||
dict, ensure that the repository is declared exactly once in all of the configurations.
|
||||
|
||||
If no repository is given, then error if there are multiple configured repositories.
|
||||
|
||||
Raise ValueError if the repository is not found in a configuration, or is declared multiple
|
||||
times.
|
||||
'''
|
||||
if not repository:
|
||||
count = len(
|
||||
tuple(
|
||||
config_repository
|
||||
for config in configurations.values()
|
||||
for config_repository in config['location']['repositories']
|
||||
)
|
||||
)
|
||||
|
||||
if count > 1:
|
||||
raise ValueError(
|
||||
'Can\'t determine which repository to use. Use --repository option to disambiguate'.format(
|
||||
repository
|
||||
)
|
||||
)
|
||||
|
||||
return
|
||||
|
||||
count = len(
|
||||
tuple(
|
||||
config_repository
|
||||
for config in configurations.values()
|
||||
for config_repository in config['location']['repositories']
|
||||
if repository == config_repository
|
||||
)
|
||||
)
|
||||
|
||||
if count == 0:
|
||||
raise ValueError('Repository {} not found in configuration files'.format(repository))
|
||||
if count > 1:
|
||||
raise ValueError('Repository {} found in multiple configuration files'.format(repository))
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
---
|
||||
title: How to deal with very large backups
|
||||
---
|
||||
## Biggish data
|
||||
|
||||
Borg itself is great for efficiently de-duplicating data across successive
|
||||
backup archives, even when dealing with very large repositories. But you may
|
||||
find that while borgmatic's default mode of "prune, create, and check" works
|
||||
well on small repositories, it's not so great on larger ones. That's because
|
||||
running the default consistency checks takes a long time on large
|
||||
repositories.
|
||||
|
||||
### A la carte actions
|
||||
|
||||
If you find yourself in this situation, you have some options. First, you can
|
||||
run borgmatic's pruning, creating, or checking actions separately. For
|
||||
instance, the the following optional flags are available:
|
||||
|
||||
```bash
|
||||
borgmatic --prune
|
||||
borgmatic --create
|
||||
borgmatic --check
|
||||
```
|
||||
|
||||
You can run with only one of these flags provided, or you can mix and match
|
||||
any number of them in a single borgmatic run. This supports approaches like
|
||||
making backups with `--create` on a frequent schedule, while only running
|
||||
expensive consistency checks with `--check` on a much less frequent basis from
|
||||
a separate cron job.
|
||||
|
||||
### Consistency check configuration
|
||||
|
||||
Another option is to customize your consistency checks. The default
|
||||
consistency checks run both full-repository checks and per-archive checks
|
||||
within each repository.
|
||||
|
||||
But if you find that archive checks are too slow, for example, you can
|
||||
configure borgmatic to run repository checks only. Configure this in the
|
||||
`consistency` section of borgmatic configuration:
|
||||
|
||||
```yaml
|
||||
consistency:
|
||||
checks:
|
||||
- repository
|
||||
```
|
||||
|
||||
If that's still too slow, you can disable consistency checks entirely,
|
||||
either for a single repository or for all repositories.
|
||||
|
||||
Disabling all consistency checks looks like this:
|
||||
|
||||
```yaml
|
||||
consistency:
|
||||
checks:
|
||||
- disabled
|
||||
```
|
||||
|
||||
Or, if you have multiple repositories in your borgmatic configuration file,
|
||||
you can keep running consistency checks, but only against a subset of the
|
||||
repositories:
|
||||
|
||||
```yaml
|
||||
consistency:
|
||||
check_repositories:
|
||||
- path/of/repository_to_check.borg
|
||||
```
|
||||
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Broken pipe with remote repository
|
||||
|
||||
When running borgmatic on a large remote repository, you may receive errors
|
||||
like the following, particularly while "borg check" is validating backups for
|
||||
consistency:
|
||||
|
||||
```text
|
||||
Write failed: Broken pipe
|
||||
borg: Error: Connection closed by remote host
|
||||
```
|
||||
|
||||
This error can be caused by an ssh timeout, which you can rectify by adding
|
||||
the following to the `~/.ssh/config` file on the client:
|
||||
|
||||
```text
|
||||
Host *
|
||||
ServerAliveInterval 120
|
||||
```
|
||||
|
||||
This should make the client keep the connection alive while validating
|
||||
backups.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: How to develop on borgmatic
|
||||
---
|
||||
## Source code
|
||||
|
||||
To get set up to hack on borgmatic, first clone master via HTTPS or SSH:
|
||||
|
||||
```bash
|
||||
git clone https://projects.torsion.org/witten/borgmatic.git
|
||||
```
|
||||
|
||||
Or:
|
||||
|
||||
```bash
|
||||
git clone ssh://git@projects.torsion.org:3022/witten/borgmatic.git
|
||||
```
|
||||
|
||||
Then, install borgmatic
|
||||
"[editable](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs)"
|
||||
so that you can run borgmatic commands while you're hacking on them to
|
||||
make sure your changes work.
|
||||
|
||||
```bash
|
||||
cd borgmatic/
|
||||
pip3 install --editable --user .
|
||||
```
|
||||
|
||||
Note that this will typically install the borgmatic commands into
|
||||
`~/.local/bin`, which may or may not be on your PATH. There are other ways to
|
||||
install borgmatic editable as well, for instance into the system Python
|
||||
install (so without `--user`, as root), or even into a
|
||||
[virtualenv](https://virtualenv.pypa.io/en/stable/). How or where you install
|
||||
borgmatic is up to you, but generally an editable install makes development
|
||||
and testing easier.
|
||||
|
||||
|
||||
## Automated tests
|
||||
|
||||
Assuming you've cloned the borgmatic source code as described above, and
|
||||
you're in the `borgmatic/` working copy, install tox, which is used for
|
||||
setting up testing environments:
|
||||
|
||||
```bash
|
||||
sudo pip3 install tox
|
||||
```
|
||||
|
||||
Finally, to actually run tests, run:
|
||||
|
||||
```bash
|
||||
cd borgmatic
|
||||
tox
|
||||
```
|
||||
|
||||
### Code formatting
|
||||
|
||||
If when running tests, you get an error from the
|
||||
[Black](https://black.readthedocs.io/en/stable/) code formatter about files
|
||||
that would be reformatted, you can ask Black to format them for you via the
|
||||
following:
|
||||
|
||||
```bash
|
||||
tox -e black
|
||||
```
|
||||
|
||||
### End-to-end tests
|
||||
|
||||
borgmatic additionally includes some end-to-end tests that integration test
|
||||
with Borg for a few representative scenarios. These tests don't run by default
|
||||
because they're relatively slow and depend on Borg. If you would like to run
|
||||
them:
|
||||
|
||||
```bash
|
||||
tox -e end-to-end
|
||||
```
|
||||
|
||||
## Code style
|
||||
|
||||
Start with [PEP 8](https://www.python.org/dev/peps/pep-0008/). But then, apply
|
||||
the following deviations from it:
|
||||
|
||||
* For strings, prefer single quotes over double quotes.
|
||||
* Limit all lines to a maximum of 100 characters.
|
||||
* Use trailing commas within multiline values or argument lists.
|
||||
* For multiline constructs, put opening and closing delimeters on lines
|
||||
separate from their contents.
|
||||
* Within multiline constructs, use standard four-space indentation. Don't align
|
||||
indentation with an opening delimeter.
|
||||
|
||||
borgmatic code uses the [Black](https://black.readthedocs.io/en/stable/) code
|
||||
formatter and [Flake8](http://flake8.pycqa.org/en/latest/) code checker, so
|
||||
certain code style requirements will be enforced when running automated tests.
|
||||
See the Black and Flake8 documentation for more information.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Inspect your backups](../../docs/how-to/inspect-your-backups.md)
|
||||
@@ -0,0 +1,46 @@
|
||||
---
|
||||
title: How to inspect your backups
|
||||
---
|
||||
## Backup progress
|
||||
|
||||
By default, borgmatic runs proceed silently except in the case of errors. But
|
||||
if you'd like to to get additional information about the progress of the
|
||||
backup as it proceeds, use the verbosity option:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 1
|
||||
```
|
||||
|
||||
This lists the files that borgmatic is archiving, which are those that are new
|
||||
or changed since the last backup.
|
||||
|
||||
Or, for even more progress and debug spew:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 2
|
||||
```
|
||||
|
||||
## Existing backups
|
||||
|
||||
Borgmatic provides convenient flags for Borg's
|
||||
[list](https://borgbackup.readthedocs.io/en/stable/usage/list.html) and
|
||||
[info](https://borgbackup.readthedocs.io/en/stable/usage/info.html)
|
||||
functionality:
|
||||
|
||||
|
||||
```bash
|
||||
borgmatic --list
|
||||
borgmatic --info
|
||||
```
|
||||
|
||||
## Scripting borgmatic
|
||||
|
||||
To consume the output of borgmatic in other software, you can include an
|
||||
optional `--json` flag with `--create`, `--list`, or `--info` to get the
|
||||
output formatted as JSON.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
|
||||
* [Develop on borgmatic](../../docs/how-to/develop-on-borgmatic.md)
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
title: How to make per-application backups
|
||||
---
|
||||
## Multiple backup configurations
|
||||
|
||||
You may find yourself wanting to create different backup policies for
|
||||
different applications on your system. For instance, you may want one backup
|
||||
configuration for your database data directory, and a different configuration
|
||||
for your user home directories.
|
||||
|
||||
The way to accomplish that is pretty simple: Create multiple separate
|
||||
configuration files and place each one in a `/etc/borgmatic.d/` directory. For
|
||||
instance:
|
||||
|
||||
```bash
|
||||
sudo mkdir /etc/borgmatic.d
|
||||
sudo generate-borgmatic-config --destination /etc/borgmatic.d/app1.yaml
|
||||
sudo generate-borgmatic-config --destination /etc/borgmatic.d/app2.yaml
|
||||
```
|
||||
|
||||
When you set up multiple configuration files like this, borgmatic will run
|
||||
each one in turn from a single borgmatic invocation. This includes, by
|
||||
default, the traditional `/etc/borgmatic/config.yaml` as well.
|
||||
|
||||
And if you need even more customizability, you can specify alternate
|
||||
configuration paths on the command-line with borgmatic's `--config` option.
|
||||
See `borgmatic --help` for more information.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
title: How to restore a backup
|
||||
---
|
||||
## Extract
|
||||
|
||||
When the worst happens—or you want to test your backups—the first step is
|
||||
to figure out which archive to restore. A good way to do that is to use the
|
||||
`--list` action:
|
||||
|
||||
```bash
|
||||
borgmatic --list
|
||||
```
|
||||
|
||||
That should yield output looking something like:
|
||||
|
||||
```text
|
||||
host-2019-01-01T04:05:06.070809 Tue, 2019-01-01 04:05:06 [...]
|
||||
host-2019-01-02T04:06:07.080910 Wed, 2019-01-02 04:06:07 [...]
|
||||
```
|
||||
|
||||
Assuming that you want to restore the archive with the most up-to-date files
|
||||
and therefore the latest timestamp, run a command like:
|
||||
|
||||
```bash
|
||||
borgmatic --extract --archive host-2019-01-02T04:06:07.080910
|
||||
```
|
||||
|
||||
The `--archive` value is the name of the archive to restore. This extracts the
|
||||
entire contents of the archive to the current directory, so make sure you're
|
||||
in the right place before running the command.
|
||||
|
||||
|
||||
## Repository selection
|
||||
|
||||
If you have a single repository in your borgmatic configuration file(s), no
|
||||
problem: the `--extract` action figures out which repository to use.
|
||||
|
||||
But if you have multiple repositories configured, then you'll need to specify
|
||||
the repository path containing the archive to extract. Here's an example:
|
||||
|
||||
```bash
|
||||
borgmatic --extract --repository repo.borg --archive host-2019-...
|
||||
```
|
||||
|
||||
## Restore particular files
|
||||
|
||||
Sometimes, you want to restore a single deleted file, rather than restoring
|
||||
everything from an archive. To do that, tack on one or more `--restore-path`
|
||||
values. For instance:
|
||||
|
||||
```bash
|
||||
borgmatic --extract --archive host-2019-... --restore-path /path/1 /path/2
|
||||
```
|
||||
|
||||
Like a whole-archive restore, this also restores into the current directory.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
|
||||
* [Inspect your backups](../../docs/how-to/inspect-your-backups.md)
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
title: How to run preparation steps before backups
|
||||
---
|
||||
## Preparation and cleanup hooks
|
||||
|
||||
If you find yourself performing prepraration tasks before your backup runs, or
|
||||
cleanup work afterwards, borgmatic hooks may be of interest. Hooks are
|
||||
shell commands that borgmatic executes for you at various points, and they're
|
||||
configured in the `hooks` section of your configuration file.
|
||||
|
||||
For instance, you can specify `before_backup` hooks to dump a database to file
|
||||
before backing it up, and specify `after_backup` hooks to delete the temporary
|
||||
file afterwards. Here's an example:
|
||||
|
||||
```yaml
|
||||
hooks:
|
||||
before_backup:
|
||||
- dump-a-database /to/file.sql
|
||||
after_backup:
|
||||
- rm /to/file.sql
|
||||
```
|
||||
|
||||
borgmatic hooks run once per configuration file. `before_backup` hooks run
|
||||
prior to backups of all repositories. `after_backup` hooks run afterwards, but
|
||||
not if an error occurs in a previous hook or in the backups themselves.
|
||||
|
||||
|
||||
## Error hooks
|
||||
|
||||
borgmatic also runs `on_error` hooks if an error occurs. Here's an example
|
||||
configuration:
|
||||
|
||||
```yaml
|
||||
hooks:
|
||||
on_error:
|
||||
- echo "Error while creating a backup."
|
||||
```
|
||||
|
||||
|
||||
## Security
|
||||
|
||||
An important security note about hooks: borgmatic executes all hook commands
|
||||
with the user permissions of borgmatic itself. So to prevent potential shell
|
||||
injection or privilege escalation, do not forget to set secure permissions
|
||||
(`chmod 0700`) on borgmatic configuration files and scripts invoked by hooks.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
|
||||
* [Make per-application backups](../../docs/how-to/make-per-application-backups.md)
|
||||
@@ -0,0 +1,180 @@
|
||||
---
|
||||
title: How to set up backups with borgmatic
|
||||
---
|
||||
## Installation
|
||||
|
||||
To get up and running, first [install
|
||||
Borg](https://borgbackup.readthedocs.io/en/latest/installation.html), at
|
||||
least version 1.1.
|
||||
|
||||
Then, run the following command to download and install borgmatic:
|
||||
|
||||
```bash
|
||||
sudo pip3 install --upgrade borgmatic
|
||||
```
|
||||
|
||||
Note that your pip binary may have a different name than "pip3". Make sure
|
||||
you're using Python 3, as borgmatic does not support Python 2.
|
||||
|
||||
|
||||
### Other ways to install
|
||||
|
||||
Along with the above process, you have several other options for installing
|
||||
borgmatic:
|
||||
|
||||
* [Docker image](https://hub.docker.com/r/monachus/borgmatic/)
|
||||
* [Another Docker image](https://hub.docker.com/r/b3vis/borgmatic/)
|
||||
* [Debian](https://tracker.debian.org/pkg/borgmatic)
|
||||
* [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
|
||||
* [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
|
||||
* [Arch Linux](https://aur.archlinux.org/packages/borgmatic/)
|
||||
* [OpenBSD](http://ports.su/sysutils/borgmatic)
|
||||
* [openSUSE](https://software.opensuse.org/package/borgmatic)
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
After you install borgmatic, generate a sample configuration file:
|
||||
|
||||
```bash
|
||||
sudo generate-borgmatic-config
|
||||
```
|
||||
|
||||
If that command is not found, then it may be installed in a location that's
|
||||
not in your system `PATH`. Try looking in `/usr/local/bin/`.
|
||||
|
||||
This generates a sample configuration file at /etc/borgmatic/config.yaml (by
|
||||
default). You should edit the file to suit your needs, as the values are
|
||||
representative. All fields are optional except where indicated, so feel free
|
||||
to ignore anything you don't need.
|
||||
|
||||
You can also get the same sample configuration file from the [configuration
|
||||
reference](../../docs/reference/configuration.md), the authoritative set of
|
||||
all configuration options. This is handy if borgmatic has added new options
|
||||
since you originally created your configuration file.
|
||||
|
||||
|
||||
### Encryption
|
||||
|
||||
Note that if you plan to run borgmatic on a schedule with cron, and you
|
||||
encrypt your Borg repository with a passphrase instead of a key file, you'll
|
||||
either need to set the borgmatic `encryption_passphrase` configuration
|
||||
variable or set the `BORG_PASSPHRASE` environment variable. See the
|
||||
[repository encryption
|
||||
section](https://borgbackup.readthedocs.io/en/latest/quickstart.html#repository-encryption)
|
||||
of the Borg Quick Start for more info.
|
||||
|
||||
Alternatively, you can specify the passphrase programatically by setting
|
||||
either the borgmatic `encryption_passcommand` configuration variable or the
|
||||
`BORG_PASSCOMMAND` environment variable. See the [Borg Security
|
||||
FAQ](http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically)
|
||||
for more info.
|
||||
|
||||
|
||||
## Initialization
|
||||
|
||||
Before you can create backups with borgmatic, you first need to initialize a
|
||||
Borg repository so you have a destination for your backup archives. (But skip
|
||||
this step if you already have a Borg repository.) To create a repository, run
|
||||
a command like the following:
|
||||
|
||||
```bash
|
||||
borgmatic --init --encryption repokey
|
||||
```
|
||||
|
||||
This uses the borgmatic configuration file you created above to determine
|
||||
which local or remote repository to create, and encrypts it with the
|
||||
encryption passphrase specified there if one is provided. Read about [Borg
|
||||
encryption
|
||||
modes](https://borgbackup.readthedocs.io/en/latest/usage/init.html#encryption-modes)
|
||||
for the menu of available encryption modes.
|
||||
|
||||
Also, optionally check out the [Borg Quick
|
||||
Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) for more
|
||||
background about repository initialization.
|
||||
|
||||
Note that borgmatic skips repository initialization if the repository already
|
||||
exists. This supports use cases like ensuring a repository exists prior to
|
||||
performing a backup.
|
||||
|
||||
If the repository is on a remote host, make sure that your local user has
|
||||
key-based SSH access to the desired user account on the remote host.
|
||||
|
||||
|
||||
## Backups
|
||||
|
||||
Now that you've configured borgmatic and initialized a repository, it's a
|
||||
good idea to test that borgmatic is working. So to run borgmatic and start a
|
||||
backup, you can invoke it like this:
|
||||
|
||||
```bash
|
||||
borgmatic --verbosity 1
|
||||
```
|
||||
|
||||
By default, this will also prune any old backups as per the configured
|
||||
retention policy, and check backups for consistency problems due to things
|
||||
like file damage.
|
||||
|
||||
The verbosity flag makes borgmatic list the files that it's archiving, which
|
||||
are those that are new or changed since the last backup. Eyeball the list and
|
||||
see if it matches your expectations based on the configuration.
|
||||
|
||||
|
||||
## Autopilot
|
||||
|
||||
Running backups manually is good for validating your configuration, but I'm
|
||||
guessing that you want to run borgmatic automatically, say once a day. To do
|
||||
that, you can configure a separate job runner to invoke it periodically.
|
||||
|
||||
### cron
|
||||
|
||||
If you're using cron, download the [sample cron
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/cron/borgmatic).
|
||||
Then, from the directory where you downloaded it:
|
||||
|
||||
```bash
|
||||
sudo mv borgmatic /etc/cron.d/borgmatic
|
||||
sudo chmod +x /etc/cron.d/borgmatic
|
||||
```
|
||||
|
||||
You can modify the cron file if you'd like to run borgmatic more or less frequently.
|
||||
|
||||
### systemd
|
||||
|
||||
If you're using systemd instead of cron to run jobs, download the [sample
|
||||
systemd service
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.service)
|
||||
and the [sample systemd timer
|
||||
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.timer).
|
||||
Then, from the directory where you downloaded them:
|
||||
|
||||
```bash
|
||||
sudo mv borgmatic.service borgmatic.timer /etc/systemd/system/
|
||||
sudo systemctl enable borgmatic.timer
|
||||
sudo systemctl start borgmatic.timer
|
||||
```
|
||||
|
||||
Feel free to modify the timer file based on how frequently you'd like
|
||||
borgmatic to run.
|
||||
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### libyaml compilation errors
|
||||
|
||||
borgmatic depends on a Python YAML library (ruamel.yaml) that will optionally
|
||||
use a C YAML library (libyaml) if present. But if it's not installed, then
|
||||
when installing or upgrading borgmatic, you may see errors about compiling the
|
||||
YAML library. If so, not to worry. borgmatic should install and function
|
||||
correctly even without the C YAML library. And borgmatic won't be any faster
|
||||
with the C library present, so you don't need to go out of your way to install
|
||||
it.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Make per-application backups](../../docs/how-to/make-per-application-backups.md)
|
||||
* [Deal with very large backups](../../docs/how-to/deal-with-very-large-backups.md)
|
||||
* [Inspect your backups](../../docs/how-to/inspect-your-backups.md)
|
||||
* [borgmatic configuration reference](../../docs/reference/configuration.md)
|
||||
* [borgmatic command-line reference](../../docs/reference/command-line.md)
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: How to upgrade borgmatic
|
||||
---
|
||||
## Upgrading
|
||||
|
||||
In general, all you should need to do to upgrade borgmatic is run the
|
||||
following:
|
||||
|
||||
```bash
|
||||
sudo pip3 install --upgrade borgmatic
|
||||
```
|
||||
|
||||
See below about special cases.
|
||||
|
||||
|
||||
### Upgrading from borgmatic 1.0.x
|
||||
|
||||
borgmatic changed its configuration file format in version 1.1.0 from
|
||||
INI-style to YAML. This better supports validation, and has a more natural way
|
||||
to express lists of values. To upgrade your existing configuration, first
|
||||
upgrade to the new version of borgmatic.
|
||||
|
||||
As of version 1.1.0, borgmatic no longer supports Python 2. If you were
|
||||
already running borgmatic with Python 3, then you can upgrade borgmatic
|
||||
in-place:
|
||||
|
||||
```bash
|
||||
sudo pip3 install --upgrade borgmatic
|
||||
```
|
||||
|
||||
But if you were running borgmatic with Python 2, uninstall and reinstall instead:
|
||||
|
||||
```bash
|
||||
sudo pip uninstall borgmatic
|
||||
sudo pip3 install borgmatic
|
||||
```
|
||||
|
||||
The pip binary names for different versions of Python can differ, so the above
|
||||
commands may need some tweaking to work on your machine.
|
||||
|
||||
|
||||
Once borgmatic is upgraded, run:
|
||||
|
||||
```bash
|
||||
sudo upgrade-borgmatic-config
|
||||
```
|
||||
|
||||
That will generate a new YAML configuration file at /etc/borgmatic/config.yaml
|
||||
(by default) using the values from both your existing configuration and
|
||||
excludes files. The new version of borgmatic will consume the YAML
|
||||
configuration file instead of the old one.
|
||||
|
||||
|
||||
### Upgrading from atticmatic
|
||||
|
||||
You can ignore this section if you're not an atticmatic user (the former name
|
||||
of borgmatic).
|
||||
|
||||
borgmatic only supports Borg now and no longer supports Attic. So if you're
|
||||
an Attic user, consider switching to Borg. See the [Borg upgrade
|
||||
command](https://borgbackup.readthedocs.io/en/stable/usage.html#borg-upgrade)
|
||||
for more information. Then, follow the instructions above about setting up
|
||||
your borgmatic configuration files.
|
||||
|
||||
If you were already using Borg with atticmatic, then you can upgrade
|
||||
from atticmatic to borgmatic by running the following commands:
|
||||
|
||||
```bash
|
||||
sudo pip3 uninstall atticmatic
|
||||
sudo pip3 install borgmatic
|
||||
```
|
||||
|
||||
That's it! borgmatic will continue using your /etc/borgmatic configuration
|
||||
files.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Develop on borgmatic](../../docs/how-to/develop-on-borgmatic.md)
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
title: borgmatic command-line reference
|
||||
---
|
||||
## borgmatic options
|
||||
|
||||
Here are all of the available borgmatic command-line options:
|
||||
|
||||
```
|
||||
{% include borgmatic/command-line.txt %}
|
||||
```
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
|
||||
* [borgmatic configuration reference](../../docs/reference/configuration.md)
|
||||
@@ -0,0 +1,19 @@
|
||||
---
|
||||
title: borgmatic configuration reference
|
||||
---
|
||||
## Configuration file
|
||||
|
||||
Here is a full sample borgmatic configuration file including all available options:
|
||||
|
||||
```yaml
|
||||
{% include borgmatic/config.yaml %}
|
||||
```
|
||||
|
||||
Note that you can also [download this configuration
|
||||
file](../../docs/reference/config.yaml) for use locally.
|
||||
|
||||
|
||||
## Related documentation
|
||||
|
||||
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
|
||||
* [borgmatic command-line reference](../../docs/reference/command-line.md)
|
||||
@@ -41,6 +41,7 @@ for sub_command in prune create check list info; do
|
||||
| grep -v '^--nobsdflags$' \
|
||||
| grep -v '^--pattern$' \
|
||||
| grep -v '^--progress$' \
|
||||
| grep -v '^--stats$' \
|
||||
| grep -v '^--read-special$' \
|
||||
| grep -v '^--repository-only$' \
|
||||
| grep -v '^--show-rc$' \
|
||||
|
||||
+4
-2
@@ -2,5 +2,7 @@
|
||||
|
||||
set -e
|
||||
|
||||
git push -u origin master
|
||||
git push -u github master
|
||||
branch_name=$(git rev-parse --abbrev-ref HEAD)
|
||||
|
||||
git push -u github "$branch_name"
|
||||
git push -u origin "$branch_name"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
from setuptools import setup, find_packages
|
||||
|
||||
|
||||
VERSION = '1.2.12'
|
||||
VERSION = '1.2.17'
|
||||
|
||||
|
||||
setup(
|
||||
@@ -20,7 +20,7 @@ setup(
|
||||
'Topic :: Security :: Cryptography',
|
||||
'Topic :: System :: Archiving :: Backup',
|
||||
],
|
||||
packages=find_packages(),
|
||||
packages=find_packages(exclude=['tests*']),
|
||||
entry_points={
|
||||
'console_scripts': [
|
||||
'borgmatic = borgmatic.commands.borgmatic:main',
|
||||
|
||||
@@ -33,6 +33,11 @@ def test_borgmatic_command():
|
||||
# Create a Borg repository.
|
||||
temporary_directory = tempfile.mkdtemp()
|
||||
repository_path = os.path.join(temporary_directory, 'test.borg')
|
||||
extract_path = os.path.join(temporary_directory, 'extract')
|
||||
|
||||
original_working_directory = os.getcwd()
|
||||
os.mkdir(extract_path)
|
||||
os.chdir(extract_path)
|
||||
|
||||
try:
|
||||
config_path = os.path.join(temporary_directory, 'test.yaml')
|
||||
@@ -51,8 +56,19 @@ def test_borgmatic_command():
|
||||
|
||||
assert len(parsed_output) == 1
|
||||
assert len(parsed_output[0]['archives']) == 1
|
||||
archive_name = parsed_output[0]['archives'][0]['archive']
|
||||
|
||||
# Also exercise the info flag.
|
||||
# Extract the created archive into the current (temporary) directory, and confirm that the
|
||||
# extracted file looks right.
|
||||
output = subprocess.check_output(
|
||||
'borgmatic --config {} --extract --archive {}'.format(config_path, archive_name).split(
|
||||
' '
|
||||
)
|
||||
).decode(sys.stdout.encoding)
|
||||
extracted_config_path = os.path.join(extract_path, config_path)
|
||||
assert open(extracted_config_path).read() == open(config_path).read()
|
||||
|
||||
# Exercise the info flag.
|
||||
output = subprocess.check_output(
|
||||
'borgmatic --config {} --info --json'.format(config_path).split(' ')
|
||||
).decode(sys.stdout.encoding)
|
||||
@@ -61,4 +77,5 @@ def test_borgmatic_command():
|
||||
assert len(parsed_output) == 1
|
||||
assert 'repository' in parsed_output[0]
|
||||
finally:
|
||||
os.chdir(original_working_directory)
|
||||
shutil.rmtree(temporary_directory)
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import subprocess
|
||||
|
||||
from flexmock import flexmock
|
||||
import pytest
|
||||
|
||||
@@ -92,6 +94,12 @@ def test_parse_arguments_disallows_encryption_mode_without_init():
|
||||
module.parse_arguments('--config', 'myconfig', '--encryption', 'repokey')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_encryption_mode_with_init():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--encryption', 'repokey')
|
||||
|
||||
|
||||
def test_parse_arguments_requires_encryption_mode_with_init():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
@@ -113,46 +121,150 @@ def test_parse_arguments_disallows_storage_quota_without_init():
|
||||
module.parse_arguments('--config', 'myconfig', '--storage-quota', '5G')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_init_and_prune():
|
||||
def test_parse_arguments_allows_init_and_prune():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--prune')
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--encryption', 'repokey', '--prune')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_init_and_create():
|
||||
def test_parse_arguments_allows_init_and_create():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--create')
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--encryption', 'repokey', '--create')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_init_and_dry_run():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--init', '--dry-run')
|
||||
module.parse_arguments(
|
||||
'--config', 'myconfig', '--init', '--encryption', 'repokey', '--dry-run'
|
||||
)
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_repository_without_extract_or_list():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--repository', 'test.borg')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_repository_with_extract():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments(
|
||||
'--config', 'myconfig', '--extract', '--repository', 'test.borg', '--archive', 'test'
|
||||
)
|
||||
|
||||
|
||||
def test_parse_arguments_allows_repository_with_list():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--config', 'myconfig', '--list', '--repository', 'test.borg')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_archive_without_extract_or_list():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--archive', 'test')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_restore_paths_without_extract():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--restore-path', 'test')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_archive_with_extract():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--config', 'myconfig', '--extract', '--archive', 'test')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_archive_with_list():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--config', 'myconfig', '--list', '--archive', 'test')
|
||||
|
||||
|
||||
def test_parse_arguments_requires_archive_with_extract():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--config', 'myconfig', '--extract')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_progress_and_create():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--progress', '--create', '--list')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_progress_and_extract():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--progress', '--extract', '--archive', 'test', '--list')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_progress_without_create():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--progress', '--list')
|
||||
|
||||
|
||||
def test_parse_arguments_with_stats_and_create_flags_does_not_raise():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--stats', '--create', '--list')
|
||||
|
||||
|
||||
def test_parse_arguments_with_stats_and_prune_flags_does_not_raise():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--stats', '--prune', '--list')
|
||||
|
||||
|
||||
def test_parse_arguments_with_stats_flag_but_no_create_or_prune_flag_raises_value_error():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--stats', '--list')
|
||||
|
||||
|
||||
def test_parse_arguments_with_just_stats_flag_does_not_raise():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--stats')
|
||||
|
||||
|
||||
def test_parse_arguments_allows_json_with_list_or_info():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
module.parse_arguments('--list', '--json')
|
||||
module.parse_arguments('--info', '--json')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_json_without_list_or_info():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--json')
|
||||
|
||||
|
||||
def test_parse_arguments_disallows_json_with_both_list_and_info():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
module.parse_arguments('--list', '--info', '--json')
|
||||
|
||||
|
||||
def test_borgmatic_version_matches_news_version():
|
||||
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
|
||||
|
||||
borgmatic_version = subprocess.check_output(('borgmatic', '--version')).decode('ascii')
|
||||
news_version = open('NEWS').readline()
|
||||
|
||||
assert borgmatic_version == news_version
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
import subprocess
|
||||
|
||||
|
||||
def test_setup_version_matches_news_version():
|
||||
setup_version = subprocess.check_output(('python', 'setup.py', '--version')).decode('ascii')
|
||||
news_version = open('NEWS').readline()
|
||||
|
||||
assert setup_version == news_version
|
||||
@@ -43,6 +43,21 @@ def test_expand_directories_considers_none_as_no_directories():
|
||||
assert paths == ()
|
||||
|
||||
|
||||
def test_expand_home_directories_expands_tildes():
|
||||
flexmock(module.os.path).should_receive('expanduser').with_args('~/bar').and_return('/foo/bar')
|
||||
flexmock(module.os.path).should_receive('expanduser').with_args('baz').and_return('baz')
|
||||
|
||||
paths = module._expand_home_directories(('~/bar', 'baz'))
|
||||
|
||||
assert paths == ('/foo/bar', 'baz')
|
||||
|
||||
|
||||
def test_expand_home_directories_considers_none_as_no_directories():
|
||||
paths = module._expand_home_directories(None)
|
||||
|
||||
assert paths == ()
|
||||
|
||||
|
||||
def test_write_pattern_file_does_not_raise():
|
||||
temporary_file = flexmock(name='filename', write=lambda mode: None, flush=lambda: None)
|
||||
flexmock(module.tempfile).should_receive('NamedTemporaryFile').and_return(temporary_file)
|
||||
@@ -149,7 +164,8 @@ CREATE_COMMAND = ('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'fo
|
||||
|
||||
|
||||
def test_create_archive_calls_borg_with_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -169,7 +185,8 @@ def test_create_archive_calls_borg_with_parameters():
|
||||
|
||||
def test_create_archive_with_patterns_calls_borg_with_patterns():
|
||||
pattern_flags = ('--patterns-from', 'patterns')
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(
|
||||
flexmock(name='/tmp/patterns')
|
||||
).and_return(None)
|
||||
@@ -191,9 +208,8 @@ def test_create_archive_with_patterns_calls_borg_with_patterns():
|
||||
|
||||
def test_create_archive_with_exclude_patterns_calls_borg_with_excludes():
|
||||
exclude_flags = ('--exclude-from', 'excludes')
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(
|
||||
('exclude',)
|
||||
)
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(('exclude',))
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None).and_return(
|
||||
flexmock(name='/tmp/excludes')
|
||||
)
|
||||
@@ -214,7 +230,8 @@ def test_create_archive_with_exclude_patterns_calls_borg_with_excludes():
|
||||
|
||||
|
||||
def test_create_archive_with_log_info_calls_borg_with_info_parameter():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
@@ -235,7 +252,8 @@ def test_create_archive_with_log_info_calls_borg_with_info_parameter():
|
||||
|
||||
|
||||
def test_create_archive_with_log_debug_calls_borg_with_debug_parameter():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -257,7 +275,8 @@ def test_create_archive_with_log_debug_calls_borg_with_debug_parameter():
|
||||
|
||||
|
||||
def test_create_archive_with_dry_run_calls_borg_with_dry_run_parameter():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
@@ -277,9 +296,10 @@ def test_create_archive_with_dry_run_calls_borg_with_dry_run_parameter():
|
||||
|
||||
|
||||
def test_create_archive_with_dry_run_and_log_info_calls_borg_without_stats_parameter():
|
||||
""" --dry-run and --stats are mutually exclusive, see:
|
||||
https://borgbackup.readthedocs.io/en/stable/usage/create.html#description """
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
# --dry-run and --stats are mutually exclusive, see:
|
||||
# https://borgbackup.readthedocs.io/en/stable/usage/create.html#description
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
@@ -300,9 +320,10 @@ def test_create_archive_with_dry_run_and_log_info_calls_borg_without_stats_param
|
||||
|
||||
|
||||
def test_create_archive_with_dry_run_and_log_debug_calls_borg_without_stats_parameter():
|
||||
""" --dry-run and --stats are mutually exclusive, see:
|
||||
https://borgbackup.readthedocs.io/en/stable/usage/create.html#description """
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
# --dry-run and --stats are mutually exclusive, see:
|
||||
# https://borgbackup.readthedocs.io/en/stable/usage/create.html#description
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
@@ -325,7 +346,8 @@ def test_create_archive_with_dry_run_and_log_debug_calls_borg_without_stats_para
|
||||
|
||||
|
||||
def test_create_archive_with_checkpoint_interval_calls_borg_with_checkpoint_interval_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -344,7 +366,8 @@ def test_create_archive_with_checkpoint_interval_calls_borg_with_checkpoint_inte
|
||||
|
||||
|
||||
def test_create_archive_with_chunker_params_calls_borg_with_chunker_params_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -363,7 +386,8 @@ def test_create_archive_with_chunker_params_calls_borg_with_chunker_params_param
|
||||
|
||||
|
||||
def test_create_archive_with_compression_calls_borg_with_compression_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -382,7 +406,8 @@ def test_create_archive_with_compression_calls_borg_with_compression_parameters(
|
||||
|
||||
|
||||
def test_create_archive_with_remote_rate_limit_calls_borg_with_remote_ratelimit_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -401,7 +426,8 @@ def test_create_archive_with_remote_rate_limit_calls_borg_with_remote_ratelimit_
|
||||
|
||||
|
||||
def test_create_archive_with_one_file_system_calls_borg_with_one_file_system_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -421,7 +447,8 @@ def test_create_archive_with_one_file_system_calls_borg_with_one_file_system_par
|
||||
|
||||
|
||||
def test_create_archive_with_read_special_calls_borg_with_read_special_parameter():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -441,7 +468,8 @@ def test_create_archive_with_read_special_calls_borg_with_read_special_parameter
|
||||
|
||||
|
||||
def test_create_archive_with_bsd_flags_true_calls_borg_without_nobsdflags_parameter():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -461,7 +489,8 @@ def test_create_archive_with_bsd_flags_true_calls_borg_without_nobsdflags_parame
|
||||
|
||||
|
||||
def test_create_archive_with_bsd_flags_false_calls_borg_with_nobsdflags_parameter():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -481,7 +510,8 @@ def test_create_archive_with_bsd_flags_false_calls_borg_with_nobsdflags_paramete
|
||||
|
||||
|
||||
def test_create_archive_with_files_cache_calls_borg_with_files_cache_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -501,7 +531,8 @@ def test_create_archive_with_files_cache_calls_borg_with_files_cache_parameters(
|
||||
|
||||
|
||||
def test_create_archive_with_local_path_calls_borg_via_local_path():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -521,7 +552,8 @@ def test_create_archive_with_local_path_calls_borg_via_local_path():
|
||||
|
||||
|
||||
def test_create_archive_with_remote_path_calls_borg_with_remote_path_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -541,7 +573,8 @@ def test_create_archive_with_remote_path_calls_borg_with_remote_path_parameters(
|
||||
|
||||
|
||||
def test_create_archive_with_umask_calls_borg_with_umask_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -560,7 +593,8 @@ def test_create_archive_with_umask_calls_borg_with_umask_parameters():
|
||||
|
||||
|
||||
def test_create_archive_with_lock_wait_calls_borg_with_lock_wait_parameters():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -579,7 +613,8 @@ def test_create_archive_with_lock_wait_calls_borg_with_lock_wait_parameters():
|
||||
|
||||
|
||||
def test_create_archive_with_json_calls_borg_with_json_parameter():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -599,9 +634,8 @@ def test_create_archive_with_json_calls_borg_with_json_parameter():
|
||||
|
||||
|
||||
def test_create_archive_with_source_directories_glob_expands():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food')).and_return(
|
||||
()
|
||||
)
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -623,7 +657,8 @@ def test_create_archive_with_source_directories_glob_expands():
|
||||
|
||||
|
||||
def test_create_archive_with_non_matching_source_directories_glob_passes_through():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo*',)).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo*',))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -643,9 +678,8 @@ def test_create_archive_with_non_matching_source_directories_glob_passes_through
|
||||
|
||||
|
||||
def test_create_archive_with_glob_calls_borg_with_expanded_directories():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food')).and_return(
|
||||
()
|
||||
)
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -666,7 +700,8 @@ def test_create_archive_with_glob_calls_borg_with_expanded_directories():
|
||||
|
||||
|
||||
def test_create_archive_with_archive_name_format_calls_borg_with_archive_name():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return(())
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
@@ -685,7 +720,8 @@ def test_create_archive_with_archive_name_format_calls_borg_with_archive_name():
|
||||
|
||||
|
||||
def test_create_archive_with_archive_name_format_accepts_borg_placeholders():
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar')).and_return([])
|
||||
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
|
||||
flexmock(module).should_receive('_expand_home_directories').and_return(())
|
||||
flexmock(module).should_receive('_write_pattern_file').and_return(None)
|
||||
flexmock(module).should_receive('_make_pattern_flags').and_return(())
|
||||
flexmock(module).should_receive('_make_exclude_flags').and_return(())
|
||||
|
||||
@@ -24,7 +24,7 @@ def insert_subprocess_check_output_mock(check_output_command, result, **kwargs):
|
||||
).once()
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_should_call_borg_with_last_archive():
|
||||
def test_extract_last_archive_dry_run_calls_borg_with_last_archive():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg', 'list', '--short', 'repo'), result='archive1\narchive2\n'.encode('utf-8')
|
||||
@@ -44,7 +44,7 @@ def test_extract_last_archive_dry_run_without_any_archives_should_bail():
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_with_log_info_should_call_borg_with_info_parameter():
|
||||
def test_extract_last_archive_dry_run_with_log_info_calls_borg_with_info_parameter():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg', 'list', '--short', 'repo', '--info'), result='archive1\narchive2\n'.encode('utf-8')
|
||||
@@ -55,7 +55,7 @@ def test_extract_last_archive_dry_run_with_log_info_should_call_borg_with_info_p
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_with_log_debug_should_call_borg_with_debug_parameter():
|
||||
def test_extract_last_archive_dry_run_with_log_debug_calls_borg_with_debug_parameter():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg', 'list', '--short', 'repo', '--debug', '--show-rc'),
|
||||
@@ -69,7 +69,7 @@ def test_extract_last_archive_dry_run_with_log_debug_should_call_borg_with_debug
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_should_call_borg_via_local_path():
|
||||
def test_extract_last_archive_dry_run_calls_borg_via_local_path():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg1', 'list', '--short', 'repo'), result='archive1\narchive2\n'.encode('utf-8')
|
||||
@@ -79,7 +79,7 @@ def test_extract_last_archive_dry_run_should_call_borg_via_local_path():
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None, local_path='borg1')
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_should_call_borg_with_remote_path_parameters():
|
||||
def test_extract_last_archive_dry_run_calls_borg_with_remote_path_parameters():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg', 'list', '--short', 'repo', '--remote-path', 'borg1'),
|
||||
@@ -92,7 +92,7 @@ def test_extract_last_archive_dry_run_should_call_borg_with_remote_path_paramete
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=None, remote_path='borg1')
|
||||
|
||||
|
||||
def test_extract_last_archive_dry_run_should_call_borg_with_lock_wait_parameters():
|
||||
def test_extract_last_archive_dry_run_calls_borg_with_lock_wait_parameters():
|
||||
flexmock(sys.stdout).encoding = 'utf-8'
|
||||
insert_subprocess_check_output_mock(
|
||||
('borg', 'list', '--short', 'repo', '--lock-wait', '5'),
|
||||
@@ -101,3 +101,91 @@ def test_extract_last_archive_dry_run_should_call_borg_with_lock_wait_parameters
|
||||
insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2', '--lock-wait', '5'))
|
||||
|
||||
module.extract_last_archive_dry_run(repository='repo', lock_wait=5)
|
||||
|
||||
|
||||
def test_extract_archive_calls_borg_with_restore_path_parameters():
|
||||
insert_subprocess_mock(('borg', 'extract', 'repo::archive', 'path1', 'path2'))
|
||||
|
||||
module.extract_archive(
|
||||
dry_run=False,
|
||||
repository='repo',
|
||||
archive='archive',
|
||||
restore_paths=['path1', 'path2'],
|
||||
storage_config={},
|
||||
)
|
||||
|
||||
|
||||
def test_extract_archive_calls_borg_with_remote_path_parameters():
|
||||
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--remote-path', 'borg1'))
|
||||
|
||||
module.extract_archive(
|
||||
dry_run=False,
|
||||
repository='repo',
|
||||
archive='archive',
|
||||
restore_paths=None,
|
||||
storage_config={},
|
||||
remote_path='borg1',
|
||||
)
|
||||
|
||||
|
||||
def test_extract_archive_calls_borg_with_umask_parameters():
|
||||
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--umask', '0770'))
|
||||
|
||||
module.extract_archive(
|
||||
dry_run=False,
|
||||
repository='repo',
|
||||
archive='archive',
|
||||
restore_paths=None,
|
||||
storage_config={'umask': '0770'},
|
||||
)
|
||||
|
||||
|
||||
def test_extract_archive_calls_borg_with_lock_wait_parameters():
|
||||
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--lock-wait', '5'))
|
||||
|
||||
module.extract_archive(
|
||||
dry_run=False,
|
||||
repository='repo',
|
||||
archive='archive',
|
||||
restore_paths=None,
|
||||
storage_config={'lock_wait': '5'},
|
||||
)
|
||||
|
||||
|
||||
def test_extract_archive_with_log_info_calls_borg_with_info_parameter():
|
||||
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--info'))
|
||||
insert_logging_mock(logging.INFO)
|
||||
|
||||
module.extract_archive(
|
||||
dry_run=False, repository='repo', archive='archive', restore_paths=None, storage_config={}
|
||||
)
|
||||
|
||||
|
||||
def test_extract_archive_with_log_debug_calls_borg_with_debug_parameters():
|
||||
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--debug', '--list', '--show-rc'))
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.extract_archive(
|
||||
dry_run=False, repository='repo', archive='archive', restore_paths=None, storage_config={}
|
||||
)
|
||||
|
||||
|
||||
def test_extract_archive_calls_borg_with_dry_run_parameter():
|
||||
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--dry-run'))
|
||||
|
||||
module.extract_archive(
|
||||
dry_run=True, repository='repo', archive='archive', restore_paths=None, storage_config={}
|
||||
)
|
||||
|
||||
|
||||
def test_extract_archive_calls_borg_with_progress_parameter():
|
||||
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--progress'))
|
||||
|
||||
module.extract_archive(
|
||||
dry_run=False,
|
||||
repository='repo',
|
||||
archive='archive',
|
||||
restore_paths=None,
|
||||
storage_config={},
|
||||
progress=True,
|
||||
)
|
||||
|
||||
@@ -6,53 +6,74 @@ from borgmatic.borg import init as module
|
||||
from ..test_verbosity import insert_logging_mock
|
||||
|
||||
|
||||
def insert_subprocess_mock(check_call_command, **kwargs):
|
||||
subprocess = flexmock(module.subprocess)
|
||||
subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
|
||||
|
||||
|
||||
INFO_REPOSITORY_EXISTS_RESPONSE_CODE = 0
|
||||
INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE = 2
|
||||
INIT_COMMAND = ('borg', 'init', 'repo', '--encryption', 'repokey')
|
||||
|
||||
|
||||
def insert_info_command_mock(info_response):
|
||||
subprocess = flexmock(module.subprocess)
|
||||
subprocess.should_receive('call').and_return(info_response)
|
||||
|
||||
|
||||
def insert_init_command_mock(init_command, **kwargs):
|
||||
subprocess = flexmock(module.subprocess)
|
||||
subprocess.should_receive('check_call').with_args(init_command, **kwargs).once()
|
||||
|
||||
|
||||
def test_initialize_repository_calls_borg_with_parameters():
|
||||
insert_subprocess_mock(INIT_COMMAND)
|
||||
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
|
||||
insert_init_command_mock(INIT_COMMAND)
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey')
|
||||
|
||||
|
||||
def test_initialize_repository_skips_initialization_when_repository_already_exists():
|
||||
insert_info_command_mock(INFO_REPOSITORY_EXISTS_RESPONSE_CODE)
|
||||
flexmock(module.subprocess).should_receive('check_call').never()
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey')
|
||||
|
||||
|
||||
def test_initialize_repository_with_append_only_calls_borg_with_append_only_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--append-only',))
|
||||
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
|
||||
insert_init_command_mock(INIT_COMMAND + ('--append-only',))
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey', append_only=True)
|
||||
|
||||
|
||||
def test_initialize_repository_with_storage_quota_calls_borg_with_storage_quota_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--storage-quota', '5G'))
|
||||
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
|
||||
insert_init_command_mock(INIT_COMMAND + ('--storage-quota', '5G'))
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey', storage_quota='5G')
|
||||
|
||||
|
||||
def test_initialize_repository_with_log_info_calls_borg_with_info_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--info',))
|
||||
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
|
||||
insert_init_command_mock(INIT_COMMAND + ('--info',))
|
||||
insert_logging_mock(logging.INFO)
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey')
|
||||
|
||||
|
||||
def test_initialize_repository_with_log_debug_calls_borg_with_debug_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--debug',))
|
||||
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
|
||||
insert_init_command_mock(INIT_COMMAND + ('--debug',))
|
||||
insert_logging_mock(logging.DEBUG)
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey')
|
||||
|
||||
|
||||
def test_initialize_repository_with_local_path_calls_borg_via_local_path():
|
||||
insert_subprocess_mock(('borg1',) + INIT_COMMAND[1:])
|
||||
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
|
||||
insert_init_command_mock(('borg1',) + INIT_COMMAND[1:])
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey', local_path='borg1')
|
||||
|
||||
|
||||
def test_initialize_repository_with_remote_path_calls_borg_with_remote_path_parameter():
|
||||
insert_subprocess_mock(INIT_COMMAND + ('--remote-path', 'borg1'))
|
||||
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
|
||||
insert_init_command_mock(INIT_COMMAND + ('--remote-path', 'borg1'))
|
||||
|
||||
module.initialize_repository(repository='repo', encryption_mode='repokey', remote_path='borg1')
|
||||
|
||||
@@ -34,10 +34,18 @@ def test_list_archives_with_log_debug_calls_borg_with_debug_parameter():
|
||||
module.list_archives(repository='repo', storage_config={})
|
||||
|
||||
|
||||
def test_list_archives_with_json_calls_borg_with_json_parameter():
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--json',))
|
||||
def test_list_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
|
||||
storage_config = {'lock_wait': 5}
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--lock-wait', '5'))
|
||||
|
||||
module.list_archives(repository='repo', storage_config={}, json=True)
|
||||
module.list_archives(repository='repo', storage_config=storage_config)
|
||||
|
||||
|
||||
def test_list_archives_with_archive_calls_borg_with_archive_parameter():
|
||||
storage_config = {}
|
||||
insert_subprocess_mock(('borg', 'list', 'repo::archive'))
|
||||
|
||||
module.list_archives(repository='repo', storage_config=storage_config, archive='archive')
|
||||
|
||||
|
||||
def test_list_archives_with_local_path_calls_borg_via_local_path():
|
||||
@@ -52,8 +60,7 @@ def test_list_archives_with_remote_path_calls_borg_with_remote_path_parameters()
|
||||
module.list_archives(repository='repo', storage_config={}, remote_path='borg1')
|
||||
|
||||
|
||||
def test_list_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
|
||||
storage_config = {'lock_wait': 5}
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--lock-wait', '5'))
|
||||
def test_list_archives_with_json_calls_borg_with_json_parameter():
|
||||
insert_subprocess_mock(LIST_COMMAND + ('--json',))
|
||||
|
||||
module.list_archives(repository='repo', storage_config=storage_config)
|
||||
module.list_archives(repository='repo', storage_config={}, json=True)
|
||||
|
||||
@@ -3,12 +3,12 @@ import sys
|
||||
|
||||
from flexmock import flexmock
|
||||
|
||||
from borgmatic.commands import borgmatic
|
||||
from borgmatic.commands import borgmatic as module
|
||||
|
||||
|
||||
def test_run_commands_handles_multiple_json_outputs_in_array():
|
||||
(
|
||||
flexmock(borgmatic)
|
||||
flexmock(module)
|
||||
.should_receive('_run_commands_on_repository')
|
||||
.times(3)
|
||||
.replace_with(
|
||||
@@ -36,7 +36,7 @@ def test_run_commands_handles_multiple_json_outputs_in_array():
|
||||
)
|
||||
)
|
||||
|
||||
borgmatic._run_commands(
|
||||
module._run_commands(
|
||||
args=flexmock(json=True),
|
||||
consistency=None,
|
||||
local_path=None,
|
||||
@@ -45,3 +45,88 @@ def test_run_commands_handles_multiple_json_outputs_in_array():
|
||||
retention=None,
|
||||
storage=None,
|
||||
)
|
||||
|
||||
|
||||
def test_collect_configuration_run_summary_logs_info_for_success():
|
||||
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
|
||||
flexmock(module).should_receive('run_configuration')
|
||||
args = flexmock(extract=False, list=False)
|
||||
|
||||
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
|
||||
|
||||
assert all(log for log in logs if log.levelno == module.logging.INFO)
|
||||
|
||||
|
||||
def test_collect_configuration_run_summary_logs_info_for_success_with_extract():
|
||||
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
|
||||
flexmock(module.validate).should_receive('guard_configuration_contains_repository')
|
||||
flexmock(module).should_receive('run_configuration')
|
||||
args = flexmock(extract=True, list=False, repository='repo')
|
||||
|
||||
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
|
||||
|
||||
assert all(log for log in logs if log.levelno == module.logging.INFO)
|
||||
|
||||
|
||||
def test_collect_configuration_run_summary_logs_critical_for_extract_with_repository_error():
|
||||
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
|
||||
flexmock(module.validate).should_receive('guard_configuration_contains_repository').and_raise(
|
||||
ValueError
|
||||
)
|
||||
args = flexmock(extract=True, list=False, repository='repo')
|
||||
|
||||
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
|
||||
|
||||
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
|
||||
|
||||
|
||||
def test_collect_configuration_run_summary_logs_critical_for_list_with_archive_and_repository_error():
|
||||
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
|
||||
flexmock(module.validate).should_receive('guard_configuration_contains_repository').and_raise(
|
||||
ValueError
|
||||
)
|
||||
args = flexmock(extract=False, list=True, repository='repo', archive='test')
|
||||
|
||||
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
|
||||
|
||||
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
|
||||
|
||||
|
||||
def test_collect_configuration_run_summary_logs_info_for_success_with_list():
|
||||
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
|
||||
flexmock(module).should_receive('run_configuration')
|
||||
args = flexmock(extract=False, list=True, repository='repo', archive=None)
|
||||
|
||||
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
|
||||
|
||||
assert all(log for log in logs if log.levelno == module.logging.INFO)
|
||||
|
||||
|
||||
def test_collect_configuration_run_summary_logs_critical_for_parse_error():
|
||||
flexmock(module.validate).should_receive('parse_configuration').and_raise(ValueError)
|
||||
args = flexmock(extract=False, list=False)
|
||||
|
||||
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
|
||||
|
||||
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
|
||||
|
||||
|
||||
def test_collect_configuration_run_summary_logs_critical_for_run_error():
|
||||
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
|
||||
flexmock(module.validate).should_receive('guard_configuration_contains_repository')
|
||||
flexmock(module).should_receive('run_configuration').and_raise(ValueError)
|
||||
args = flexmock(extract=False, list=False)
|
||||
|
||||
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
|
||||
|
||||
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
|
||||
|
||||
|
||||
def test_collect_configuration_run_summary_logs_critical_for_missing_configs():
|
||||
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
|
||||
flexmock(module).should_receive('run_configuration')
|
||||
args = flexmock(config_paths=(), extract=False, list=False)
|
||||
|
||||
logs = tuple(module.collect_configuration_run_summary_logs(config_filenames=(), args=args))
|
||||
|
||||
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
|
||||
|
||||
@@ -117,12 +117,3 @@ def test_guard_configuration_upgraded_does_not_raise_when_neither_config_present
|
||||
flexmock(os.path).should_receive('exists').with_args('other.yaml').and_return(False)
|
||||
|
||||
module.guard_configuration_upgraded('config', ('config.yaml', 'other.yaml'))
|
||||
|
||||
|
||||
def test_guard_excludes_filename_omitted_raises_when_filename_provided():
|
||||
with pytest.raises(module.LegacyExcludesFilenamePresent):
|
||||
module.guard_excludes_filename_omitted(excludes_filename='/etc/borgmatic/excludes')
|
||||
|
||||
|
||||
def test_guard_excludes_filename_omitted_does_not_raise_when_filename_not_provided():
|
||||
module.guard_excludes_filename_omitted(excludes_filename=None)
|
||||
|
||||
@@ -90,3 +90,42 @@ def test_apply_logical_validation_does_not_raise_or_warn_if_archive_name_format_
|
||||
|
||||
def test_apply_logical_validation_does_not_raise_otherwise():
|
||||
module.apply_logical_validation('config.yaml', {'retention': {'keep_secondly': 1000}})
|
||||
|
||||
|
||||
def test_guard_configuration_contains_repository_does_not_raise_when_repository_in_config():
|
||||
module.guard_configuration_contains_repository(
|
||||
repository='repo', configurations={'config.yaml': {'location': {'repositories': ['repo']}}}
|
||||
)
|
||||
|
||||
|
||||
def test_guard_configuration_contains_repository_does_not_raise_when_repository_not_given():
|
||||
module.guard_configuration_contains_repository(
|
||||
repository=None, configurations={'config.yaml': {'location': {'repositories': ['repo']}}}
|
||||
)
|
||||
|
||||
|
||||
def test_guard_configuration_contains_repository_errors_when_repository_assumed_to_match_config_twice():
|
||||
with pytest.raises(ValueError):
|
||||
module.guard_configuration_contains_repository(
|
||||
repository=None,
|
||||
configurations={'config.yaml': {'location': {'repositories': ['repo', 'repo2']}}},
|
||||
)
|
||||
|
||||
|
||||
def test_guard_configuration_contains_repository_errors_when_repository_missing_from_config():
|
||||
with pytest.raises(ValueError):
|
||||
module.guard_configuration_contains_repository(
|
||||
repository='nope',
|
||||
configurations={'config.yaml': {'location': {'repositories': ['repo', 'repo2']}}},
|
||||
)
|
||||
|
||||
|
||||
def test_guard_configuration_contains_repository_errors_when_repository_matches_config_twice():
|
||||
with pytest.raises(ValueError):
|
||||
module.guard_configuration_contains_repository(
|
||||
repository='repo',
|
||||
configurations={
|
||||
'config.yaml': {'location': {'repositories': ['repo', 'repo2']}},
|
||||
'other.yaml': {'location': {'repositories': ['repo']}},
|
||||
},
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user