Compare commits

...
530 Commits
Author SHA1 Message Date
Dan Helfman 1c88dda76a Fix for invalid JSON output when using multiple borgmatic configuration files (#155). 2019-04-02 22:30:14 -07:00
Dan Helfman 0b59c22c23 Fix for seemingly random filename ordering when running through a directory of configuration files (#157). 2019-03-30 16:41:21 -07:00
Dan Helfman 576377e2b2 Clarify differences between Docker images. 2019-03-16 15:04:48 -07:00
Dan Helfman 6ff1867312 Configuration files includes and merging (#148). 2019-03-06 12:06:27 -08:00
Dan Helfman 3cb52423d2 Support for Borg create/extract --numeric-owner flag (#147). 2019-03-05 09:11:35 -08:00
Dan Helfman 5a5b6491ac Add note about uncommenting section names. 2019-03-04 15:15:49 -08:00
Dan Helfman 4272c6b077 List the files within an archive via --list --archive option (#140). 2019-02-23 23:02:17 -08:00
Dan Helfman 26071de2e7 Update extraction docs. 2019-02-18 22:43:32 -08:00
Dan Helfman fe92d9e838 Fix restore paths list to tuple conversion. 2019-02-18 21:59:09 -08:00
Dan Helfman 5ea2d644a2 Fix error handling when --extract repository guard fails. 2019-02-18 21:52:56 -08:00
Dan Helfman c35f90154f Only guard repository when --extract is used. 2019-02-18 21:43:30 -08:00
Dan Helfman 36305c50b1 Update push script to support branches. 2019-02-18 13:51:33 -08:00
Dan Helfman 2b3b8eab71 Add archive extract to end-to-end test. 2019-02-18 13:47:18 -08:00
Dan Helfman aa7c7651e5 Fix config repositories consumption. 2019-02-18 13:27:35 -08:00
Dan Helfman c41ffb5ceb If no extract repository is given, then error if there are multiple configured repositories. 2019-02-18 13:22:14 -08:00
Dan Helfman 766a03375a Guard that the given repository occurs in config exactly once. 2019-02-18 12:58:39 -08:00
Dan Helfman 2a4d4247e3 Tests for extract_archive(). 2019-02-18 10:31:52 -08:00
Dan Helfman 9de5083a7e Additional test coverage for extract options in borgmatic command. 2019-02-18 09:52:56 -08:00
Dan Helfman d0557b2bcd Initial work on #123: Support for Borg extract. 2019-02-18 09:30:34 -08:00
Dan Helfman 1a980d6321 Organize options within command-line help into logical groups. 2019-02-12 22:27:04 -08:00
Dan Helfman fb21d4e645 Remove dead code. 2019-02-09 21:17:55 -08:00
Dan Helfman 5933a4d778 Note tests exclusion in changelog. 2019-02-08 20:59:50 -08:00
Dan HelfmanandGitea 8cf16c7831 Exclude tests from distribution packages. 2019-02-09 05:02:19 +00:00
Julien Nicoulaud fcf4e03c2f exclude tests from distribution packages 2019-02-08 19:22:56 +01:00
Dan Helfman d1b29e82da borgmatic command-line reference. 2019-02-04 22:27:40 -08:00
Dan Helfman 290e969a22 Include a sample borgmatic configuration file in the documentation (#119). 2019-02-04 22:12:33 -08:00
Dan Helfman 18ae91ea6e Strike some unnecessary words from docs. 2019-02-04 20:58:27 -08:00
Dan Helfman 0bce77a2ac Distribute troubleshooting among relevant how-to guides. 2019-02-04 20:53:47 -08:00
Dan Helfman 19155607af Include link to development how-to. 2019-02-03 22:42:33 -08:00
Dan Helfman f357c37e2c Fix/remove some documentation links. 2019-02-03 22:35:38 -08:00
Dan Helfman 2980c14728 Fix README links on GitHub. 2019-02-03 22:26:39 -08:00
Dan Helfman 7e0e00d45d Refactor documentation into multiple separate pages for clarity and findability. 2019-02-03 22:20:59 -08:00
Dan Helfman 8b4ac0017b Fix ticket number in changelog. 2019-01-27 14:00:24 -08:00
Dan Helfman 8ec1ec527e Bump version for release. 2019-01-27 13:54:26 -08:00
Dan Helfman 6096a7181c Leave exclude_patterns glob expansion to Borg, since doing it in borgmatic leads to confusing behavior (#132). 2019-01-27 13:47:26 -08:00
Dan Helfman fa9dfb8ff7 Remove date echo from schema example, as it's not a substitute for real logging (#127). 2019-01-27 12:22:22 -08:00
Dan Helfman 2dc006aab4 Allow use of --stats flag when --create or --prune flags are implied (#139). 2019-01-27 12:15:47 -08:00
Dan Helfman 031b9d6faf Handle and format validation errors raised during argument parsing (#136). 2019-01-27 11:58:04 -08:00
Dan HelfmanandGitHub d9018a47f6 Add link to openSUSE packages from README. 2019-01-21 09:57:05 -08:00
Antonio LarrosaandGitHub e893a20dfd Add link to openSUSE packages
Add a link to the software.opensuse.org page were both official and community packages of borgmatic are available to be downloaded or installed using 1-click-install.
2019-01-21 13:13:40 +01:00
Dan Helfman 09d521661f Remove weasel words like "easily" and "simply". 2018-12-25 22:03:34 -08:00
Dan Helfman fd46efb193 Add borgmatic --version command-line flag to get the current installed version number. 2018-12-25 21:01:08 -08:00
Dan Helfman 426f54c9cc When generating sample configuration, document the defaults for each option (#103). 2018-12-25 17:05:22 -08:00
Dan Helfman 45a537b6b1 When running multiple configuration files, attempt all of them even if one errors (#116). 2018-12-25 15:23:54 -08:00
Dan Helfman d6feca169c Fix duplicate issue number. 2018-12-24 22:35:16 -08:00
Dan Helfman 05e2900ab0 Rev version. 2018-12-24 22:29:41 -08:00
Dan Helfman 30b52e5523 With --init command-line flag, if a repository already exists, proceed without erroring (#117). 2018-12-24 22:28:02 -08:00
Dan Helfman 14aeddc11f Black re-formatting. 2018-12-24 14:38:57 -08:00
Dan Helfman 066399ecdb Mention --stats command-line flag in NEWS file. 2018-12-23 16:06:08 -08:00
Dan HelfmanandGitea d4bbac4467 Support for --stats command-line flag independent of --verbosity (#100). 2018-12-24 00:04:23 +00:00
Felix Buehler 7516443a89 fix changes requested about stats 2018-12-22 23:46:03 +01:00
Felix Buehler 73d67e29b4 Support for Borg create & prune --stats via borgmatic command-line flag (#100) 2018-12-22 23:27:24 +01:00
Dan Helfman c3e7425f4c Some late-breaking README additions (new borgmatic packages). 2018-12-10 22:30:10 -08:00
Dan Helfman cc9dbb1def Support for Borg repository initialization via borgmatic --init command-line flag (#110). 2018-12-10 22:20:57 -08:00
Dan Helfman 2045edc11b Fix warning about classifiers as tuple. 2018-12-09 15:49:58 -08:00
Dan Helfman 1dcac44d6c Fix broken test of deprecated --excludes option. 2018-12-09 15:49:05 -08:00
Dan Helfman 300ead65d3 Error when deprecated --excludes command-line option is used. 2018-12-09 14:57:14 -08:00
Dan Helfman 6a0219a7a4 Update README with link to a new/forked Docker image (#113). 2018-12-02 15:16:52 -08:00
Dan Helfman 80c69aac05 Fix incomplete test coverage around --progress argument validation. 2018-12-02 15:08:42 -08:00
Dan Helfman 7417a3cd00 Update Borg create --filter values so a dry run lists files to back up. (#111). 2018-12-02 15:03:07 -08:00
Dan Helfman 9ca80a54d8 Support for Borg create --progress via borgmatic command-line flag (#108). 2018-11-21 22:03:39 -08:00
Dan Helfman 5c0b17ef39 Support for Borg --chunker-params create option via "chunker_params" in borgmatic's storage section (#105). 2018-10-27 15:57:28 -07:00
Dan Helfman 1697d8aaef Silence curl when posting release description to Gitea. 2018-10-15 22:32:13 -07:00
Dan Helfman fef441a8ff More concessions for Python 3.5 compatibility. 2018-10-15 09:37:26 -07:00
Dan Helfman c1ddc4268b We can't have nice things. 2018-10-15 09:30:04 -07:00
Dan Helfman e323290e61 Switch from bash to sh for black wrapper. 2018-10-15 09:25:57 -07:00
Dan Helfman 1ab44d4201 Wrap black with script that skips it if Python version < 3.6. 2018-10-15 09:20:35 -07:00
Dan Helfman 71b1c3dfb0 Make automated tests support running in Python 3.5. 2018-10-15 09:04:29 -07:00
Dan Helfman 695930a607 Fix for syntax error that occurred in Python 3.5 and below (#102). 2018-10-15 08:47:15 -07:00
Dan Helfman eb2a4ff1f0 Add Python 3.5 to continuous integration. 2018-10-15 08:17:34 -07:00
Dan Helfman 531d5c80c0 Fix quoting and escaping in release script. 2018-10-14 12:14:29 -07:00
Dan Helfman 067ed27689 Rev to 1.2.8. 2018-10-14 11:45:34 -07:00
Dan Helfman fa38de2de7 Enable consistency checks for only certain repositories via "check_repositories" (#73). 2018-10-13 20:34:51 -07:00
Dan Helfman e4d1b49c39 Switch some functions with many arguments to kwargs only. 2018-10-13 15:19:16 -07:00
Dan Helfman af7caec509 Mention minimum Borg version to install in README. 2018-10-13 13:35:42 -07:00
Dan Helfman 90c1f899fc Use newer Alpine (with newer version of Borg) in matrix builds. 2018-10-13 13:35:18 -07:00
Dan Helfman a0691ae4cd Run continuous integration tests on a matrix of Python and Borg versions. 2018-10-13 13:09:12 -07:00
Dan Helfman 2f20e6f808 Include link to issue tracker within various command output. 2018-10-07 22:29:56 -07:00
Dan Helfman 7a4636ae0f Remove curl --verbose in release script. 2018-10-06 22:35:00 -07:00
Dan Helfman 53435dcc3e Post release changelogs to projects.evoworx.org. 2018-10-06 22:24:46 -07:00
Dan Helfman 4d01278037 Update release file to post changelogs to GitHub release descriptions. 2018-10-06 15:18:21 -07:00
Dan Helfman 2299e5d41e Additional dependency version pins in test requirements. 2018-10-06 14:17:47 -07:00
Dan Helfman d16f5d5df3 Add backticks to path literal in README. 2018-10-06 13:23:54 -07:00
Dan Helfman da8e9638f4 Support for Borg --keep-secondly prune option (#98). 2018-10-04 21:54:23 -07:00
Dan Helfman 900ea80a42 Hack to uncomment all options in config file used for finding unsupported Borg options. 2018-10-04 21:45:31 -07:00
Dan Helfman 4b92d0f685 Remove unneeded Dockerfile for end-to-end tests. 2018-10-03 22:44:23 -07:00
Dan Helfman 3ce5533103 Make end-to-end test clean up after itself, and drop unnecessary use of Docker for it. 2018-10-03 22:36:25 -07:00
Dan Helfman 4a1ee8c911 Pull new base Docker images during CI. 2018-10-03 19:35:42 -07:00
Dan Helfman 3f22a99412 Rev pykwalify. 2018-10-03 08:59:08 -07:00
Dan Helfman caf95cc913 Rebuild. 2018-09-30 22:58:23 -07:00
Dan Helfman fd3130b4d9 Install tox before using it. 2018-09-30 22:47:07 -07:00
Dan Helfman 65bb5a49e2 CI? 2018-09-30 22:43:49 -07:00
Dan Helfman 4bcc517326 Attempted Drone CI configuration. 2018-09-30 22:09:53 -07:00
Dan Helfman 0b164973e0 Add an end-to-end automated test that actually integrates with Borg. 2018-09-30 17:30:04 -07:00
Dan Helfman a125df991b Move tests to the root of the repository, in keeping with more common convention. 2018-09-30 13:57:20 -07:00
Dan Helfman f9a9b42c58 A little introductory text for the screencast. 2018-09-30 11:11:07 -07:00
Dan Helfman 56ad1d164a Use Flake8 code checker as part of running automated tests. 2018-09-29 23:15:18 -07:00
Dan Helfman 3cce18919c Switch Black link to documentation. 2018-09-29 22:46:34 -07:00
Dan Helfman 76d6a69f5a Use Black code formatter as part of running automated tests. 2018-09-29 22:45:00 -07:00
Dan Helfman 3db17277b4 Replace broken screencast thumbnail with embedded player. 2018-09-29 21:38:38 -07:00
Dan Helfman ece49eb500 Update screencast. 2018-09-29 18:56:39 -07:00
Dan Helfman 746428ed44 Fix generated configuration to also include a "keep_daily" value so pruning works out of the box. 2018-09-29 15:44:37 -07:00
Dan Helfman 984702b3b2 Fix various warnings. 2018-09-29 15:06:57 -07:00
Dan Helfman 1bc71e1c5d Upgrade test requirements. 2018-09-29 15:04:42 -07:00
Dan Helfman 47efa88c9d In generate-borgmatic-config, comment out all optional config (#57). 2018-09-29 15:03:11 -07:00
Dan Helfman 3821636b77 Bump version. 2018-09-27 08:13:23 -07:00
Dan Helfman 596f6f9dac Update help/README about --create --json. 2018-09-27 08:12:54 -07:00
Dan HelfmanandGitHub 7ecdaea83a Fix check_archives does not take json parameter. 2018-09-27 08:09:23 -07:00
Nils HesseandGitHub 98cb2644db check_archives does not take json parameter 2018-09-27 12:21:14 +02:00
Dan Helfman 31db6faa19 Set to release version. 2018-09-26 21:32:28 -07:00
Dan Helfman 872d8b695a Flesh out NEWS line item a bit. 2018-09-24 21:37:45 -07:00
Dan HelfmanandGitea 6db3e1dda5 Merge branch 'master' of floli/borgmatic into master 2018-09-25 04:36:09 +00:00
Florian Lindner 9aaf78b9dd Add --json option for --create command line.
Closes #94.
2018-09-24 21:53:09 +02:00
Dan Helfman 5d8ac158ce Merge ssh://projects.torsion.org:3022/witten/borgmatic 2018-09-17 22:34:08 -07:00
Dan Helfman d32a53d58f Mention log level fix in NEWS. 2018-09-17 22:33:34 -07:00
floliandGitea a836ec944f Limit argument range for --verbose, make default log level more explicit. (#93) 2018-09-18 05:31:27 +00:00
Dan Helfman e7b128e735 --read-special is now supported. 2018-09-09 11:21:06 -07:00
Dan Helfman ff3cb1d80f Attach #64 to logging rewrite in NEWS. 2018-09-09 11:18:26 -07:00
Dan Helfman c5ff08ee25 Remove now-gone verbosity parameter from test. 2018-09-09 11:14:33 -07:00
Dan Helfman 856db29180 Mention --read-special in NEWS. 2018-09-09 10:42:06 -07:00
Steve KerrisonandDan Helfman 20e09b4ea8 Support for Borg create --read-special via "read_special" option (#25). 2018-09-09 10:39:56 -07:00
Dan Helfman 1dd0682661 Merge branch 'master' of ssh://projects.torsion.org:3022/witten/borgmatic 2018-09-08 13:54:18 -07:00
floliandGitea 7252b8d614 Rework logging/verbosity system (#90)
Looks great, merged! Thanks again for all your hard work here.
2018-09-08 20:53:37 +00:00
grerrgandDan Helfman e5870a169b Add example for cron in Alpine Linux (#24) 2018-09-05 21:58:30 -07:00
Dan Helfman 94795a3560 Link to Borg home page instead of docs. 2018-09-02 22:06:57 -07:00
Dan 7705debab0 Switching back to table-like 11ty front matter. It looks less bad than JSON. 2018-09-01 22:45:13 -07:00
Dan f87df0527f Adding JSON front matter for 11ty. 2018-09-01 22:38:17 -07:00
Dan Helfman e4512a40e0 Removing 11ty front matter out of README since it renders as a table on GitHub. 2018-09-01 22:11:38 -07:00
Dan Helfman 1d4a9510b8 Upgrade pytest. 2018-09-01 20:29:05 -07:00
Dan Helfman 2648f07e7a Add missing syntax highlighting. 2018-08-29 23:01:11 -07:00
Dan Helfman 459bf1fcf6 Document --list and --info flags. 2018-08-29 22:57:32 -07:00
Dan 3930e63320 Merge branch 'master' of ssh://projects.torsion.org:3022/witten/borgmatic 2018-08-29 22:44:45 -07:00
Dan acecb1e397 README metadata changes to support 11ty static site generator. 2018-08-29 22:44:12 -07:00
Dan Helfman 9b48eb5a61 Clarify that --json can be used with --info command-line flag. 2018-08-19 12:57:52 -07:00
Dan Helfman 7d40a448cb Pass --show-rc option to Borg when at highest verbosity level (#89). 2018-08-19 12:44:40 -07:00
Dan Helfman da7aed3814 Support for Borg create --checkpoint-interval (#87). 2018-08-19 11:41:49 -07:00
Dan Helfman c7f4200417 Somewhat more robust mechanism to find unsupported Borg arguments. 2018-08-19 11:24:48 -07:00
Dan Helfman 5e2a5494af Fix declared pykwalify compatibility version range in setup.py (#88). 2018-08-18 14:07:18 -07:00
Dan Helfman 7b77fd2510 Fix compatibility issue between pykwalify and ruamel.yaml 0.15.52 (#85). 2018-08-11 13:59:27 -07:00
Dan Helfman ece5608677 Rev for release. 2018-07-28 22:27:39 -07:00
Dan Helfman 4644f613b2 Fix typo in README. 2018-07-28 22:24:24 -07:00
Dan Helfman 3afa5ac76d Document hooks (#81). 2018-07-28 22:22:25 -07:00
Dan Helfman 27f8a1df04 Switch to non-raw link to sample cron job. 2018-07-28 20:29:55 -07:00
Dan Helfman 8e5b0bbf17 Remove errant ctrl-F character from docs. 2018-07-28 20:27:18 -07:00
Dan Helfman 282e9565c9 Mentioning new --info --json option in NEWS. 2018-07-28 20:24:19 -07:00
Thomas LÉVEILandGitea b714ffd48b add support for --info --json (#83) 2018-07-29 03:17:45 +00:00
Dan Helfman 9968a15ef8 Clarifying code style for multiline constructs. 2018-07-28 15:21:19 -07:00
Dan Helfman d93da55ce9 Add code style guidelines to the documention, and reformat some code accordingly. 2018-07-28 15:02:17 -07:00
Thomas LÉVEILandGitea 789bcd402a add support for --list --json (#74) 2018-07-28 21:21:38 +00:00
Dan HelfmanandGitea cf6ab60d2e Use XDG_CONFIG_HOME for user configuration directory, if set. (Thanks to floli.) (#71)
Thanks! This will go out in the next release.
2018-07-25 01:34:05 +00:00
Dan Helfman 64364b20ff Skip non-"*.yaml" config filenames in /etc/borgmatic.d/ so as not to parse backup files, editor swap files, etc. (#77) 2018-07-22 12:08:49 -07:00
Dan Helfman d29c7956bc Upgrade ruamel.yaml compatibility version range and fix support for Python 3.7 (#38, #76). 2018-07-22 11:25:06 -07:00
Dan Helfman e5ef485d6b Merge branch 'master' of ssh://projects.torsion.org:3022/witten/borgmatic 2018-07-01 14:54:15 -07:00
Dan Helfman fc8046edc4 Adding NEWS item about skipping before/after backup hooks. 2018-07-01 14:51:57 -07:00
Dan HelfmanandGitea 4538017206 Merge branch 'fix-72-hooks-are-executed-when-list-or-info' of thomasleveil/borgmatic into master
Thanks for fixing this!

I agree about more specific hooks if and when `--check` or `--prune`-specific hook use cases are needed. I think what you've done here is fine until then.
2018-07-01 21:47:39 +00:00
Thomas LEVEIL d664b6d253 only run hooks when creating an archive
fix #72
2018-07-01 21:09:45 +02:00
Dan Helfman f42aa0a6f2 Revving version for development. 2018-06-17 15:26:53 -07:00
Dan Helfman 9d4ba66f6e Revving version for release. 2018-06-17 15:14:45 -07:00
Dan Helfman cf846ab8ac Support for Borg prune --umask option (#69). 2018-06-17 15:12:43 -07:00
Dan Helfman 219e287c6c Document how to develop on and contribute to borgmatic. 2018-06-17 14:55:57 -07:00
Dan Helfman dede8f9d4b News for: ~/.config/borgmatic/config.yaml. 2018-06-17 14:30:47 -07:00
Dan HelfmanandGitea 7a1e3f5639 Merge branch 'add_user_config' of floli/borgmatic into master 2018-06-17 21:26:36 +00:00
Florian Lindner 9bd77292ff Add default path for user configuration 2018-06-10 15:03:23 +02:00
Dan Helfman f1a143de5b Adding list and info Borg sub-commands to find-unsupported-borg-options script. 2018-05-26 20:53:03 -07:00
Dan Helfman 765e343c71 Support for Borg --info via borgmatic command-line (#61). 2018-05-26 16:19:05 -07:00
Dan Helfman af4b91a048 Support for Borg --list option via borgmatic command-line to list all archives (#61). 2018-05-26 16:09:08 -07:00
Dan Helfman cc9044487b Support for Borg --nobsdflags option to skip recording bsdflags (e.g. NODUMP, IMMUTABLE) in archive (#63). 2018-05-26 15:09:23 -07:00
Dan Helfman 11c30001c3 Add "Persistent" flag to systemd timer example. (#60) 2018-05-20 22:20:21 -07:00
Dan HelfmanandGitea ac9161035a Merge branch 'master' of floli/borgmatic into master
Thanks for taking the time to add this! Makes sense.
2018-05-21 05:18:13 +00:00
Dan Helfman 007ec0644c Ignore "check_last" and consistency "prefix" when "archives" not in consistency checks. (#59) 2018-05-20 22:11:40 -07:00
Dan Helfman 1db808fb3d Link to OpenBSD port of borgmatic. 2018-05-19 16:16:54 -07:00
Dan Helfman 76656275c3 Update README to mention other ways of installing borgmatic. (#62) 2018-05-19 16:06:54 -07:00
Florian Lindner 64bdbc4bf0 Add Persistent, so that the timer is triggered if missed last time. 2018-05-17 21:47:58 +02:00
Dan Helfman 61033bb4e5 Update tox.ini to only assume Python 3.x instead of Python 3.4 specifically. 2018-04-09 20:34:59 -07:00
Dan Helfman e608b7924a Adding note about executable location. 2018-04-08 12:06:15 -07:00
Dan Helfman f7f852a28b Fix tests broken by addition of check --prefix default. 2018-03-03 22:36:51 -08:00
Dan Helfman 9b9c4c4abb Clarifying note in schema about adding prefix to consistency section. 2018-03-03 22:33:34 -08:00
Dan Helfman 1b59f5b190 Changing version in warning to correspond with next release version. 2018-03-03 22:30:30 -08:00
Dan Helfman 65ab230961 Noting new Borg check --prefix feature in release notes. 2018-03-03 22:21:48 -08:00
Nick WhyteandDan Helfman c64d0100d5 Only check archives with matching prefix. 2018-03-03 22:17:39 -08:00
Dan Helfman 0112407250 Add introductory screencast link to documentation. 2018-02-19 17:44:20 -08:00
Dan Helfman 2d3f5fa05d Support for Borg --lock-wait option for the maximum wait for a repository/cache lock (#56). 2018-02-19 15:51:04 -08:00
Dan Helfman a87036ee46 Support for using tilde in exclude_patterns to reference home directory (#58). 2018-02-18 15:34:19 -08:00
Dan Helfman a72f5ff69a Tests for --dry-run + --verbosity fix. 2018-02-18 14:26:51 -08:00
newtonneandDan Helfman bb99009191 Fix issue when using both --dry-run and -v options. 2018-02-18 14:18:25 -08:00
Dan Helfman 4c45d60529 Mentioning ssh_command for additional SSH configuration.. 2018-02-18 13:39:05 -08:00
Dan Helfman 2211f959ae AUTHORS addition for recent encryption pass command changes. 2018-01-18 21:13:20 -08:00
Dan Helfman cc1d6f53a0 55: Fix for missing tags/releases from Gitea and GitHub project hosting. 2018-01-17 20:27:09 -08:00
Dan Helfman 389778c716 Adding BORG_PASSCOMMAND update to NEWS. 2018-01-16 21:05:53 -08:00
newtonneandDan Helfman e55e9e8139 Add encryption_passcommand configuration option 2018-01-16 21:03:25 -08:00
Dan Helfman ef76e87477 Bumping version for release. 2018-01-15 20:55:49 -08:00
Dan Helfman 62526038d6 47: Support for Borg --dry-run option via borgmatic command-line. 2018-01-15 20:55:27 -08:00
Dan Helfman bf2f39623e 49: Rename incorrect --pattern-from option to correct --patterns-from. 2018-01-15 20:22:53 -08:00
Dan Helfman 28c890a52d Bumping version for release. 2018-01-14 16:37:02 -08:00
Dan Helfman cd189c4fe4 48: Add "local_path" to configuration for specifying an alternative Borg executable path. 2018-01-14 16:35:24 -08:00
Dan Helfman b8f6bab12d 49: Support for Borg experimental --patterns-from and --patterns options for specifying mixed includes/excludes. 2018-01-14 15:52:19 -08:00
Dan Helfman 50b3240c4f 54: Fix for incorrect consistency check flags passed to Borg when all three checks in borgmatic config. 2018-01-14 14:09:20 -08:00
Dan Helfman 18fbc75e16 Revising history to account for off-by-one error when importing issue numbers into Gitea. 2018-01-04 21:49:43 -08:00
Dan Helfman 0881da4a82 New issue tracker. 2018-01-04 21:37:43 -08:00
Dan fa210766a2 Update for release. 2018-01-02 20:36:52 -08:00
Dan d4f52e3137 Update AUTHORS with recent changes + sort. 2018-01-02 20:33:27 -08:00
Dan 8b2ebdc5f7 Simplifying example. 2018-01-02 20:31:06 -08:00
Dan HelfmanandGitea a00407256d Merge branch 'keep_minutely' of thomasleveil/borgmatic into master 2018-01-03 04:30:13 +00:00
Thomas LEVEIL 24b5eccefc add support for Borg --keep-minutely prune option 2018-01-03 00:13:44 +01:00
Dan 815fb39a05 Declare dependency on pykwalify 1.6 or above, as older versions yield "Unknown key: version" rule errors. 2017-11-26 10:30:31 -08:00
Dan 24c196d2a4 Script to find unsupported Borg options in borgmatic, to assist with #13. 2017-11-11 16:07:54 -08:00
Dan 3e26e70d0c Fix for incorrect /etc/borgmatic.d/ configuration path probing on macOS. 2017-11-10 21:33:29 -08:00
Dan 5ce25e2790 Re-fixing logo image. 2017-11-04 11:55:48 -07:00
Dan 8243552c8c Fixing PNG path. 2017-11-04 11:54:29 -07:00
Dan 425e27dee5 Add "ssh_command" to configuration for specifying a custom SSH command or options. 2017-11-03 22:01:04 -07:00
Dan 9ec9269a18 Link to repository encryption section of Borg Quick Start. 2017-11-03 20:28:31 -07:00
Dan bf5cbd1deb Mentioning use of BORG_PASSPHRASE environment variable. 2017-11-03 20:27:21 -07:00
Dan 4c09cbf1a4 Releasing. 2017-11-02 22:38:16 -07:00
Dan fc077af4ce Mentioning borgmatic --config option in docs. 2017-11-02 22:28:53 -07:00
Dan ca4312bb85 Support for Borg --remote-ratelimit for limiting upload rate. And log Borg commands. 2017-11-02 22:22:40 -07:00
Dan fc3b1fccba Support for Borg --files-cache option for setting the files cache operation mode. 2017-11-02 22:03:11 -07:00
Dan f83346b9b3 Support for using tilde in repository paths to reference home directory. 2017-11-02 21:34:04 -07:00
Dan 63c7241aec Typo in comment. 2017-10-31 22:21:49 -07:00
Dan fd77dc579e Pass through several more Unix signals that Borg happens to consume. 2017-10-31 22:10:00 -07:00
Dan f017ed648f Disabling code coverage on this one-line functions. 2017-10-31 22:01:18 -07:00
Dan 27a6745743 Passing the Unix SIGTERM signal through to child processes like Borg. 2017-10-31 21:58:35 -07:00
Dan 95be0c8e46 Removing broken download URL. 2017-10-29 21:42:01 -07:00
Dan 17a774ba7e Removing .hg* files. 2017-10-29 20:32:21 -07:00
Dan a1d2bd173b Bumping version for release. 2017-10-29 20:29:53 -07:00
Dan f495550ad7 Default "prefix" to "{hostname}-" if not specified. 2017-10-29 20:14:18 -07:00
Dan 43d0e597a2 Require "prefix" in retention section when "archive_name_format" is set. 2017-10-29 19:36:26 -07:00
Dan f1c07b5cf5 Updated dead links to Borg documentation. 2017-10-29 17:05:49 -07:00
Dan f2782426d5 Comment typo. 2017-10-29 16:46:30 -07:00
Dan f13ed92b0e Breaking borgmatic command main() apart, since it was getting a little unwieldy. 2017-10-29 16:44:15 -07:00
Dan 6e9e7c45d7 Being explicit about markdown syntax highlighting in README. 2017-10-28 10:45:27 -07:00
Dan c1ca4b9421 Using absolute path for logo. 2017-10-28 10:33:36 -07:00
Dan 469feadbc0 Adding missing PNG logo. 2017-10-28 10:31:30 -07:00
Dan a5403a4373 Switched logo from SVG to PNG for compatibility reasons. 2017-10-28 10:29:18 -07:00
Dan 56c902258d Setting up download URL for new hosting location. 2017-10-27 22:40:34 -07:00
Dan 9c1660f467 Fix typo in README. 2017-10-27 22:27:28 -07:00
Dan dd926b5762 Updating links. 2017-10-27 22:26:33 -07:00
Dan 9d03351b5d Converted main source repository from Mercurial to Git. 2017-10-27 21:55:08 -07:00
Dan 719d9a9835 Merge branch 'master' of github.com:witten/borgmatic 2017-10-27 21:51:34 -07:00
Dan 731c8c9ad9 Adding push and release scripts. 2017-10-27 21:51:10 -07:00
Dan Helfman 2ae8ac2947 Add tests for verbosity mapping. 2017-10-25 22:36:23 -07:00
Dan Helfman cc94e5f52f Add tests for verbosity mapping. 2017-10-25 22:36:23 -07:00
Dan Helfman a09c9f248e Adding logging to hook execution! 2017-10-25 22:32:06 -07:00
Dan Helfman 16f0a3976c Adding logging to hook execution! 2017-10-25 22:32:06 -07:00
Dan Helfman cc78223164 Fixing inconsistent indentation. 2017-10-25 21:58:02 -07:00
Dan Helfman 30f56235c1 Fixing inconsistent indentation. 2017-10-25 21:58:02 -07:00
Dan Helfman 7458769cb3 Merge. 2017-10-25 21:54:50 -07:00
Dan Helfman a5aa9355f5 Merge. 2017-10-25 21:54:50 -07:00
Dan Helfman 5c229639f0 Improve clarity of logging spew at high verbosity levels. 2017-10-25 21:47:33 -07:00
Dan Helfman 059322b7f8 Improve clarity of logging spew at high verbosity levels. 2017-10-25 21:47:33 -07:00
Johannes FeichtnerandDan Helfman f1a98d82c6 #16, #38: Support for user-defined hooks before/after backup, or on error. 2017-10-25 21:38:27 -07:00
Johannes Feichtner 80e2c023dd #16, #38: Support for user-defined hooks before/after backup, or on error. 2017-10-26 06:38:27 +02:00
b3visandDan Helfman 86511deac4 Added section about docker (#18) 2017-10-25 21:24:24 -07:00
b3vis bb3475b3f8 Added section about docker (#18) 2017-10-26 05:24:24 +01:00
Dan Helfman bd196c1fb9 Removing "from __future__ import print_function". This isn't Python 2 anymore, Toto. 2017-09-09 17:38:14 -07:00
Dan Helfman 873fc22cfb Removing "from __future__ import print_function". This isn't Python 2 anymore, Toto. 2017-09-09 17:38:14 -07:00
Dan Helfman f3d6d7c0a3 #29: Support for using tilde in source directory path to reference home directory. 2017-09-09 17:23:31 -07:00
Dan Helfman 86cc6ca869 #29: Support for using tilde in source directory path to reference home directory. 2017-09-09 17:23:31 -07:00
Dan Helfman d30caa422e #39: Fix to make /etc/borgmatic/config.yaml optional rather than required when using the default config paths. 2017-09-08 21:25:42 -07:00
Dan Helfman 84c148fb3b #39: Fix to make /etc/borgmatic/config.yaml optional rather than required when using the default config paths. 2017-09-08 21:25:42 -07:00
Dan Helfman 6c4f641c1e Added tag 1.1.7 for changeset ec7949a14a20 2017-09-03 11:33:10 -07:00
Dan Helfman b44bc57548 Added tag 1.1.7 for changeset ec7949a14a20 2017-09-03 11:33:10 -07:00
Dan Helfman bb18a9a3f2 Update NEWS and AUTHORS for release. 2017-09-03 11:33:07 -07:00
Dan Helfman f7dcbe40d4 Update NEWS and AUTHORS for release. 2017-09-03 11:33:07 -07:00
Michele LazzeriandDan Helfman 95533d2b31 Added storage.archive_name_format to config (#16)
* Added storage.archive_name_format to config
2017-09-03 11:13:14 -07:00
Michele Lazzeri 867d3fceb0 Added storage.archive_name_format to config (#16)
* Added storage.archive_name_format to config
2017-09-03 20:13:14 +02:00
Dan Helfman 3af92f8b92 Fix for traceback when "exclude_from" value is empty in configuration file. 2017-08-27 10:01:49 -07:00
Dan Helfman 7c048d1989 Fix for traceback when "exclude_from" value is empty in configuration file. 2017-08-27 10:01:49 -07:00
Dan Helfman d127e73590 Clarification of Python 3 pip usage in documentation. 2017-08-26 16:18:53 -07:00
Dan Helfman 13ba5c84de Clarification of Python 3 pip usage in documentation. 2017-08-26 16:18:53 -07:00
Dan Helfman 50c4f6f2a1 Adding documentation note about pruning happening before archiving. 2017-08-26 16:13:41 -07:00
Dan Helfman 9588e111c4 Adding documentation note about pruning happening before archiving. 2017-08-26 16:13:41 -07:00
Dan Helfman 37ae34a432 When pruning, make highest verbosity level list archives kept and pruned. 2017-08-26 16:07:30 -07:00
Dan Helfman e3a559e13b When pruning, make highest verbosity level list archives kept and pruned. 2017-08-26 16:07:30 -07:00
Dan Helfman 3664ac7418 Added tag 1.1.6 for changeset 4daa944c122c 2017-08-05 23:33:08 -07:00
Dan Helfman 3f83788858 Added tag 1.1.6 for changeset 4daa944c122c 2017-08-05 23:33:08 -07:00
Dan Helfman 10cac46f4c #12, #35: Support for Borg --exclude-from, --exclude-caches, and --exclude-if-present options. 2017-08-05 23:32:39 -07:00
Dan Helfman b1f429f4b5 #12, #35: Support for Borg --exclude-from, --exclude-caches, and --exclude-if-present options. 2017-08-05 23:32:39 -07:00
Dan Helfman 51095cd419 Remove unused imports. 2017-08-05 22:26:38 -07:00
Dan Helfman ddd56bf2a7 Remove unused imports. 2017-08-05 22:26:38 -07:00
Dan Helfman 674a6153f3 Fix imports of borg/*.py modules now that they've been split out. 2017-08-05 22:26:28 -07:00
Dan Helfman 793ffbd048 Fix imports of borg/*.py modules now that they've been split out. 2017-08-05 22:26:28 -07:00
Dan Helfman aa04473521 Split out Borg integration code into multiple files, as it was getting kind of hairy all in one. 2017-08-05 16:21:39 -07:00
Dan Helfman 247d36a309 Split out Borg integration code into multiple files, as it was getting kind of hairy all in one. 2017-08-05 16:21:39 -07:00
Dan Helfman 77d3c66fb9 Added tag 1.1.5 for changeset 64ca13bfe050 2017-07-30 11:16:41 -07:00
Dan Helfman 9f5b808042 Added tag 1.1.5 for changeset 64ca13bfe050 2017-07-30 11:16:41 -07:00
Dan Helfman 9bea7ae5ed #34: New "extract" consistency check that performs a dry-run extraction of the most recent archive. 2017-07-30 11:16:26 -07:00
Dan Helfman e85d487c3a #34: New "extract" consistency check that performs a dry-run extraction of the most recent archive. 2017-07-30 11:16:26 -07:00
Dan Helfman 23679a6edd Removing Pelican-specific title metadata out of README markdown. 2017-07-29 16:05:11 -07:00
Dan Helfman 525ffa6a28 Removing Pelican-specific title metadata out of README markdown. 2017-07-29 16:05:11 -07:00
Dan Helfman 0f44fbedf4 Getting logo to show up on GitHub. 2017-07-28 22:36:16 -07:00
Dan Helfman ac47301a64 Getting logo to show up on GitHub. 2017-07-28 22:36:16 -07:00
Dan Helfman ae15e0f404 Added tag 1.1.4 for changeset 3d605962d891 2017-07-28 22:02:43 -07:00
Dan Helfman 9347c02268 Added tag 1.1.4 for changeset 3d605962d891 2017-07-28 22:02:43 -07:00
Dan Helfman a2e8abc537 #17: Added command-line flags for performing a borgmatic run with only pruning, creating, or checking enabled. 2017-07-28 22:02:18 -07:00
Dan Helfman ceeaf25443 #17: Added command-line flags for performing a borgmatic run with only pruning, creating, or checking enabled. 2017-07-28 22:02:18 -07:00
Dan Helfman 10404143c6 Added tag 1.1.3 for changeset 3f838f661546 2017-07-25 21:21:50 -07:00
Dan Helfman 62d2b267da Added tag 1.1.3 for changeset 3f838f661546 2017-07-25 21:21:50 -07:00
Dan Helfman 94aaf4554f Releasing. 2017-07-25 21:21:47 -07:00
Dan Helfman 03d50d74ca Releasing. 2017-07-25 21:21:47 -07:00
Dan Helfman 0c8816e6cc #14: Support for running multiple config files in /etc/borgmatic.d/ from a single borgmatic run. 2017-07-25 21:18:51 -07:00
Dan Helfman 7ed5b33db5 #14: Support for running multiple config files in /etc/borgmatic.d/ from a single borgmatic run. 2017-07-25 21:18:51 -07:00
Dan Helfman e3e4aeff94 Fix for generate-borgmatic-config writing config with invalid one_file_system value. 2017-07-25 20:32:32 -07:00
Dan Helfman 57b3066987 Fix for generate-borgmatic-config writing config with invalid one_file_system value. 2017-07-25 20:32:32 -07:00
Dan Helfman 89cd879529 Added tag 1.1.2 for changeset f052a77a8ad5 2017-07-24 19:29:28 -07:00
Dan Helfman 1527ff7898 Added tag 1.1.2 for changeset f052a77a8ad5 2017-07-24 19:29:28 -07:00
Dan Helfman 2c61c0bc08 #32: Fix for passing check_last as integer to subprocess when calling Borg. 2017-07-24 19:29:26 -07:00
Dan Helfman 3967e1b5f0 #32: Fix for passing check_last as integer to subprocess when calling Borg. 2017-07-24 19:29:26 -07:00
Dan Helfman bcd8b9982d Added tag 1.1.1 for changeset 7d3d11eff6c0 2017-07-24 08:41:05 -07:00
Dan Helfman 8cbd51512b Added tag 1.1.1 for changeset 7d3d11eff6c0 2017-07-24 08:41:05 -07:00
Dan Helfman b36b923c5d #32: Fix for upgrade-borgmatic-config converting check_last option as a string instead of an integer. 2017-07-24 08:41:02 -07:00
Dan Helfman c38f7a3693 #32: Fix for upgrade-borgmatic-config converting check_last option as a string instead of an integer. 2017-07-24 08:41:02 -07:00
Dan Helfman f44a7884e6 No longer producing univeral (Python 2 + 3) wheel. 2017-07-23 17:34:17 -07:00
Dan Helfman 7c77a5a8a5 No longer producing univeral (Python 2 + 3) wheel. 2017-07-23 17:34:17 -07:00
Dan Helfman b61b09f55c Added tag 1.1.0 for changeset 5a003056a8ff 2017-07-22 23:27:26 -07:00
Dan Helfman 9caaee18b5 Added tag 1.1.0 for changeset 5a003056a8ff 2017-07-22 23:27:26 -07:00
Dan Helfman 588955a467 Setting release version. 2017-07-22 23:27:21 -07:00
Dan Helfman 7c0407ed22 Setting release version. 2017-07-22 23:27:21 -07:00
Dan Helfman ee3edeaac2 Support for backing up to multiple repositories. 2017-07-22 22:56:46 -07:00
Dan Helfman 499f8aa0a4 Support for backing up to multiple repositories. 2017-07-22 22:56:46 -07:00
Dan Helfman 90a0d3b1e0 Renaming group to section for consistency. 2017-07-22 22:17:37 -07:00
Dan Helfman 548212274f Renaming group to section for consistency. 2017-07-22 22:17:37 -07:00
Dan Helfman cd8ceccfaf To free up space, now pruning backups prior to creating a new backup. 2017-07-22 21:50:29 -07:00
Dan Helfman 1292dd2162 To free up space, now pruning backups prior to creating a new backup. 2017-07-22 21:50:29 -07:00
Dan Helfman e5c12fc81c Mentioning test coverage addition in NEWS. 2017-07-22 21:23:01 -07:00
Dan Helfman b02ac44cfc Mentioning test coverage addition in NEWS. 2017-07-22 21:23:01 -07:00
Dan Helfman f5abe05ce9 Instructions to make cron file executable. 2017-07-22 21:20:48 -07:00
Dan Helfman 52963adfc9 Instructions to make cron file executable. 2017-07-22 21:20:48 -07:00
Dan Helfman 6af53d1163 Fixing gets on config group names. 2017-07-22 21:19:26 -07:00
Dan Helfman 2274cfe480 Fixing gets on config group names. 2017-07-22 21:19:26 -07:00
Dan Helfman 3cccac8cb1 Mentioning libyaml compile errors in troubleshooting. 2017-07-22 21:07:09 -07:00
Dan Helfman 8cf52651fe Mentioning libyaml compile errors in troubleshooting. 2017-07-22 21:07:09 -07:00
Dan Helfman 919d7573c3 Upgrading instructions to super clarify Python 3 upgrade. 2017-07-22 20:52:29 -07:00
Dan Helfman 166ef8faae Upgrading instructions to super clarify Python 3 upgrade. 2017-07-22 20:52:29 -07:00
Dan Helfman 8bfffd8cf7 Removing TODO that basically entails testing ruamel.yaml round-tripping, which in theory already has its own tests. 2017-07-22 20:31:26 -07:00
Dan Helfman ac2a63763f Removing TODO that basically entails testing ruamel.yaml round-tripping, which in theory already has its own tests. 2017-07-22 20:31:26 -07:00
Dan Helfman edb54b300b Fixing up borg module to deal with new parsed config file structures. 2017-07-22 20:11:49 -07:00
Dan Helfman 8b2b41eefc Fixing up borg module to deal with new parsed config file structures. 2017-07-22 20:11:49 -07:00
Dan Helfman 41d202c2e7 TODO about using the new exclude_patterns. 2017-07-10 16:26:32 -07:00
Dan Helfman fb172f018a TODO about using the new exclude_patterns. 2017-07-10 16:26:32 -07:00
Dan Helfman 8ef6c6fcbe Bail if "--excludes" argument is provided, as it's now deprecated in favor of configuration file. 2017-07-10 16:25:13 -07:00
Dan Helfman b1355e75c4 Bail if "--excludes" argument is provided, as it's now deprecated in favor of configuration file. 2017-07-10 16:25:13 -07:00
Dan Helfman 0691cda46f Mention generate-borgmatic-config in changelog. 2017-07-10 16:07:07 -07:00
Dan Helfman d2c143d39c Mention generate-borgmatic-config in changelog. 2017-07-10 16:07:07 -07:00
Dan Helfman 8bf07e4766 Provide helpful message when borgmatic is run with only legacy config present. 2017-07-10 16:06:02 -07:00
Dan Helfman ef32b292a8 Provide helpful message when borgmatic is run with only legacy config present. 2017-07-10 16:06:02 -07:00
Dan Helfman b3d0fb0cee When writing config, make containing directory if necessary. Also default to tighter permissions. 2017-07-10 15:20:50 -07:00
Dan Helfman 61f88228b0 When writing config, make containing directory if necessary. Also default to tighter permissions. 2017-07-10 15:20:50 -07:00
Dan Helfman ff28be7724 Documentation updates based on the new YAML configuration. 2017-07-10 11:06:28 -07:00
Dan Helfman f98558546c Documentation updates based on the new YAML configuration. 2017-07-10 11:06:28 -07:00
Dan Helfman 5ff016238e Don't overwrite config files. And retain file permissions when upgrading config. 2017-07-10 10:37:11 -07:00
Dan Helfman 9cc7c77ba9 Don't overwrite config files. And retain file permissions when upgrading config. 2017-07-10 10:37:11 -07:00
Dan Helfman 618e56b2a5 Display result of config upgrade. 2017-07-10 10:13:57 -07:00
Dan Helfman 3b1b058ffe Display result of config upgrade. 2017-07-10 10:13:57 -07:00
Dan Helfman 338b80903c Fixing tests broken by excludes merging. 2017-07-10 10:09:06 -07:00
Dan Helfman 9a3b52e1fd Fixing tests broken by excludes merging. 2017-07-10 10:09:06 -07:00
Dan Helfman fea97b5149 Merge excludes into config file format. 2017-07-10 09:43:25 -07:00
Dan Helfman 0dfc935af6 Merge excludes into config file format. 2017-07-10 09:43:25 -07:00
Dan Helfman 17c87f8758 Completed test coverage of commands (except for main()s). 2017-07-09 17:03:45 -07:00
Dan Helfman 2f7527a333 Completed test coverage of commands (except for main()s). 2017-07-09 17:03:45 -07:00
Dan Helfman d49be19544 Add a version to the schema, because inevitably I'll want to revise the schema. 2017-07-09 16:18:10 -07:00
Dan Helfman 263891f414 Add a version to the schema, because inevitably I'll want to revise the schema. 2017-07-09 16:18:10 -07:00
Dan Helfman d4ae7814a0 Adding TODO about a helpful notice about legacy config. 2017-07-09 11:49:51 -07:00
Dan Helfman 644c2e6612 Adding TODO about a helpful notice about legacy config. 2017-07-09 11:49:51 -07:00
Dan Helfman dc9b075d5a Rename convert-borgmatic-config to upgrade-borgmatic-config. 2017-07-09 11:48:24 -07:00
Dan Helfman 999feb81ca Rename convert-borgmatic-config to upgrade-borgmatic-config. 2017-07-09 11:48:24 -07:00
Dan Helfman 1bcb2a8be4 More test coverage, and simplification of config generation. 2017-07-09 11:41:55 -07:00
Dan Helfman f581f4b8d9 More test coverage, and simplification of config generation. 2017-07-09 11:41:55 -07:00
Dan Helfman a16d90ff46 Adding a "does not raise" test for displaying errors. 2017-07-09 10:27:34 -07:00
Dan Helfman c7803a2814 Adding a "does not raise" test for displaying errors. 2017-07-09 10:27:34 -07:00
Dan Helfman e50fd04750 Adding test coverage report. Making tests a little less brittle. 2017-07-08 23:01:41 -07:00
Dan Helfman f4e5dc8382 Adding test coverage report. Making tests a little less brittle. 2017-07-08 23:01:41 -07:00
Dan Helfman 745de200df Basic YAML generating / validating / converting to. 2017-07-08 22:33:51 -07:00
Dan Helfman f19a40ef9c Basic YAML generating / validating / converting to. 2017-07-08 22:33:51 -07:00
Dan Helfman bff6980eee Tests for YAML config code. 2017-07-04 18:32:37 -07:00
Dan Helfman 483bd50bdf Tests for YAML config code. 2017-07-04 18:32:37 -07:00
Dan Helfman 1dc60d2856 Integrating YAML config into borgmatic and updating README. 2017-07-04 18:23:59 -07:00
Dan Helfman 5110e64e63 Integrating YAML config into borgmatic and updating README. 2017-07-04 18:23:59 -07:00
Dan Helfman 6e85940d63 Basic YAML configuration file parsing. 2017-07-04 16:52:24 -07:00
Dan Helfman 4d7556f68b Basic YAML configuration file parsing. 2017-07-04 16:52:24 -07:00
Dan Helfman e00f74ddf7 Dropped Python 2 support. Now Python 3 only. 2017-07-02 17:18:33 -07:00
Dan Helfman 9212f87735 Dropped Python 2 support. Now Python 3 only. 2017-07-02 17:18:33 -07:00
Dan Helfman 1aaf27dfb2 Changed example umask config to be more realistic. 2017-06-25 10:36:36 -07:00
Dan Helfman ebd34f1695 Changed example umask config to be more realistic. 2017-06-25 10:36:36 -07:00
Dan Helfman 87c65fb723 Removing unnecessary curlies from bash command. 2016-07-04 09:35:51 -07:00
Dan Helfman a34dccbd27 Removing unnecessary curlies from bash command. 2016-07-04 09:35:51 -07:00
Dan Helfman abb6bed459 Sample files for triggering borgmatic from a systemd timer. 2016-07-04 09:19:34 -07:00
Dan Helfman 49c4f483fd Sample files for triggering borgmatic from a systemd timer. 2016-07-04 09:19:34 -07:00
Dan Helfman 5bd1cc5580 #18: Fix for README mention of sample files not included in package. Also, added logo. 2016-07-03 22:07:53 -07:00
Dan Helfman 4447956da7 #18: Fix for README mention of sample files not included in package. Also, added logo. 2016-07-03 22:07:53 -07:00
Dan Helfman f6d2e983d9 Added tag 1.0.3 for changeset 32c6341dda9f 2016-06-23 07:13:29 -07:00
Dan Helfman 9a96a277e6 Added tag 1.0.3 for changeset 32c6341dda9f 2016-06-23 07:13:29 -07:00
Dan Helfman 6bfe524bac #21: Fix for verbosity flag not actually causing verbose output. 2016-06-23 07:13:25 -07:00
Dan Helfman a45d7bec81 #21: Fix for verbosity flag not actually causing verbose output. 2016-06-23 07:13:25 -07:00
Dan Helfman ead991dcd1 Added tag 1.0.2 for changeset 9603d13910b3 2016-06-13 12:02:37 -07:00
Dan Helfman 63c4bf3bf9 Added tag 1.0.2 for changeset 9603d13910b3 2016-06-13 12:02:37 -07:00
Dan Helfman b22b552bf3 #20: Fix for traceback when remote_path option is missing. 2016-06-13 08:53:41 -07:00
Dan Helfman ed0127df91 #20: Fix for traceback when remote_path option is missing. 2016-06-13 08:53:41 -07:00
Dan Helfman 938392b25b Restricting issues list to open issues. 2016-06-12 22:40:04 -07:00
Dan Helfman 17e9f21fb9 Restricting issues list to open issues. 2016-06-12 22:40:04 -07:00
Dan Helfman 481dbc14c3 Rename issues URL. 2016-06-12 22:37:42 -07:00
Dan Helfman 16cc77fd9d Rename issues URL. 2016-06-12 22:37:42 -07:00
Dan Helfman 600c438951 Reverting to pre-rename issues link, because that link isn't yet renamed to borgmatic. 2016-06-10 17:11:28 -07:00
Dan Helfman 5d9bb13410 Reverting to pre-rename issues link, because that link isn't yet renamed to borgmatic. 2016-06-10 17:11:28 -07:00
Dan Helfman 2e3e68d2cb Added tag 1.0.1 for changeset de2d7721cdec 2016-06-10 13:34:23 -07:00
Dan Helfman da513c1089 Added tag 1.0.1 for changeset de2d7721cdec 2016-06-10 13:34:23 -07:00
Dan Helfman 331adca23e #19: Support for Borg's --remote-path option to use an alternate Borg executable. 2016-06-10 13:31:37 -07:00
Dan Helfman 3579dbe813 #19: Support for Borg's --remote-path option to use an alternate Borg executable. 2016-06-10 13:31:37 -07:00
Dan Helfman e1e5db22f8 Making a univeral wheel that supports both Python 2 and 3. 2016-06-10 12:34:49 -07:00
Dan Helfman 8e3a2c7a85 Making a univeral wheel that supports both Python 2 and 3. 2016-06-10 12:34:49 -07:00
Dan Helfman 377e3948ff Added tag 1.0.0 for changeset 0e1fbee9358d 2016-06-10 12:15:48 -07:00
Dan Helfman 706a31f189 Added tag 1.0.0 for changeset 0e1fbee9358d 2016-06-10 12:15:48 -07:00
Dan Helfman 4533fec167 Documenting how to upgrade from atticmatic to borgmatic. 2016-06-10 11:53:45 -07:00
Dan Helfman e5c772d8a5 Documenting how to upgrade from atticmatic to borgmatic. 2016-06-10 11:53:45 -07:00
Dan Helfman 633700c0af Dropping support for Attic. 2016-06-10 11:21:53 -07:00
Dan Helfman 40a215802f Dropping support for Attic. 2016-06-10 11:21:53 -07:00
Dan Helfman 007b3e6d4e Merge pull request #12 from dawez/master
fixed typos in readme
2016-04-17 21:21:44 -07:00
Dan Helfman 5d46acbe41 Merge pull request #12 from dawez/master
fixed typos in readme
2016-04-17 21:21:44 -07:00
dawez 175761c757 fixed typos in readme 2016-04-17 22:26:07 +02:00
dawez 9dc4960277 fixed typos in readme 2016-04-17 22:26:07 +02:00
Dan Helfman c7e23fe9ed Added tag 0.1.8 for changeset dbc96d3f83bd 2016-04-10 16:01:18 -07:00
Dan Helfman 7644a38574 Added tag 0.1.8 for changeset dbc96d3f83bd 2016-04-10 16:01:18 -07:00
Dan Helfman 2359135327 Cutting a release. 2016-04-10 16:01:05 -07:00
Dan Helfman df9cc6a7d2 Switching from the no-longer-maintained nose test runner to pytest. 2016-04-10 15:59:36 -07:00
Dan Helfman 448c2593ed Refreshing flexmock version in test requirements. 2016-04-10 15:48:10 -07:00
Dan Helfman a364369f25 Fixing "check" backend tests to support new use of stderr=STDOUT. 2016-04-10 15:46:43 -07:00
Dan Helfman e430b8c281 At verbosity zero, suppressing Borg check spew to stderr about "Checking segments". 2016-04-10 15:29:42 -07:00
Dan Helfman 219408647c Normalizing recent changes. No new content. 2016-04-10 15:27:21 -07:00
Dan Helfman 5bffa35741 Fixed links to Borg documentation. 2016-04-10 10:23:32 -07:00
Dan Helfman a938a2ad61 Merge pull request #11 from jangondol/patch-1
Fix broken link to Borg quickstart
2016-04-10 10:21:23 -07:00
Jan Gondol 37b6f043dd Fix broken link to Borg quickstart 2016-04-06 14:54:06 +02:00
Dan Helfman 0c97dba5f1 Mocking out glob() in test so it doesn't hit the filesystem, and simplifying comprehension. 2016-02-13 16:41:17 -08:00
Dan Helfman f149fa4b4b Merge pull request #5 from ypid/support-file-globs
Added support for file globs in source_directories.
2016-02-13 16:32:05 -08:00
Robin Schneider f669e31305 Made globing for source_directories the default.
Don’t remove non existing files/directories from the list and let
attic/borg handle this.
2016-02-13 21:05:34 +01:00
Dan Helfman dd2be365b1 Support borg create --umask. (Merge PR from ypid.) 2016-02-13 10:59:43 -08:00
Dan Helfman d6585811d6 Added support for --one-file-system for Borg. 2016-02-13 10:43:31 -08:00
Dan Helfman 14a277a64f Merge pull request #8 from ypid/fixed-source-split-bug
Use /\s+/ to split source_directories to handle 1+ spaces.
2016-02-13 10:29:31 -08:00
Dan Helfman 9723dbdd4c Merge pull request #6 from ypid/added-gitignore
Added .gitignore file.
2016-02-13 10:27:55 -08:00
Robin Schneider 9f5c5c8e13 Use /\s+/ to split source_directories to handle 1+ spaces.
This bug is can be quite annoying because when you accidentally used
something like:

```ini
[location]
source_directories: backup_one  backup_two
;                              A (Additional space here)
```

It would call Attic/Borg with ('backup_one', '', 'backup_two') which in
turn backups your whole $PWD.
2016-01-31 11:42:07 +01:00
Robin Schneider 4453f41967 Added .gitignore file. 2016-01-30 20:24:29 +01:00
Robin Schneider e4cf193cd7 Added support for file globs in source_directories.
source_directories_glob can be used to enable glob support (defaults to
disabled).
2016-01-25 23:52:16 +01:00
Dan Helfman e1605ae6ab Merge pull request #3 from ypid/README-ini-hightlighting
Also allow the INI example to be highlighted on GitHub.
2016-01-24 20:52:01 -08:00
Robin Schneider d6ea9b6658 Also allow the INI example to be highlighted on GitHub. 2016-01-20 13:11:15 +01:00
Dan Helfman 147996b234 Added tag github/master for changeset 28434dd0440c 2015-11-08 17:06:48 -08:00
Dan Helfman ac374f89d2 Merge. 2015-11-08 17:04:49 -08:00
Dan Helfman ddde37ff59 Removed tag github/yaml_config_files 2015-11-08 17:04:14 -08:00
Dan Helfman 278aaabeb9 Removed tag github/yaml_config_files 2015-11-08 17:03:40 -08:00
Dan Helfman 06cdc2c46c Merge pull request #2 from ThomasWaldmann/patch-1
fixed typo in README
2015-10-20 18:33:16 -07:00
TW e5586f7d87 fixed typo in README 2015-10-20 23:08:43 +02:00
Dan Helfman d044504693 Added tag 0.1.7 for changeset 5a458ebef804 2015-09-06 16:40:46 -07:00
Dan Helfman ec416cb152 Revving version. 2015-09-06 16:40:39 -07:00
Dan Helfman 0da1c6ec7b Better error message when configuration file is missing. 2015-09-06 15:55:14 -07:00
Dan Helfman f54acc9bbf #11: Fixed parsing of punctuation in configuration file. 2015-09-06 15:33:56 -07:00
Dan Helfman d86faa30d3 Added tag 0.1.6 for changeset 4c63f3d90ec2 2015-09-02 22:48:14 -07:00
Dan Helfman a44212ff00 #9: New configuration option for the encryption passphrase. #10: Support for Borg's new archive compression feature. 2015-09-02 22:48:07 -07:00
Dan Helfman 178e56e77b Adding documentation note about logging into the issue tracker in order to create issues. 2015-09-02 18:45:15 -07:00
Dan Helfman 1cb44da6b3 Adding build to hgignore. 2015-08-09 11:04:57 -07:00
Dan Helfman 7e8123379f Added tag 0.1.5 for changeset 0afff209b902 2015-08-09 10:59:40 -07:00
Dan Helfman c554d1d36d Changes to support release on PyPI. Now pip installable by name! 2015-08-09 10:59:27 -07:00
Dan Helfman a56529871c Added tag 0.1.4 for changeset e58246fc92bb 2015-07-30 08:13:32 -07:00
Dan Helfman 820492f9a9 Adding version test. 2015-07-30 08:13:27 -07:00
Dan Helfman f803198f14 Added tag 0.1.4 for changeset 6dda59c12de8 2015-07-30 08:12:36 -07:00
Dan Helfman f59bc98d79 Adding test that setup.py version matches release version. 2015-07-30 08:12:31 -07:00
Dan Helfman 158e889deb Added tag 0.1.3 for changeset acc7fb61566f 2015-07-27 21:48:21 -07:00
Dan Helfman 2444c4b372 #1: Add support for "borg check --last N" to Borg backend. 2015-07-27 21:47:52 -07:00
Dan Helfman 9ecc207139 #6: Fixing example config file to use valid keep_within value. 2015-07-27 19:06:39 -07:00
Dan Helfman 803fd3a851 Linking to both Attic and Borg check docs from sample config. 2015-07-26 22:02:43 -07:00
Dan Helfman e18cfd6c80 Removing some annoying Pelican metadata from docs. 2015-07-26 21:29:14 -07:00
Dan Helfman 7a287ba289 Added tag 0.1.2 for changeset 83067f995dd3 2015-07-26 21:06:06 -07:00
Dan Helfman 3f99dc6db2 As a convenience to new users, allow a missing default excludes file. 2015-07-26 21:06:03 -07:00
Dan Helfman 837d25cfd8 Linking to both Attic and Borg prune docs from sample config. 2015-07-26 20:57:31 -07:00
Dan Helfman fedc75cafc New issue tracker, linked from documentation. 2015-07-21 21:29:40 -07:00
Dan Helfman 29106b9645 Added tag 0.1.1 for changeset 7b6c87dca7ea 2015-07-18 23:49:06 -07:00
Dan Helfman 4f6337c46e Bumping version. 2015-07-18 23:48:58 -07:00
Dan Helfman 55183bf890 Added tag 0.1.1 for changeset ac5dfa01e9d1 2015-07-18 18:44:14 -07:00
Dan Helfman cc09d7fc10 Adding borgmatic cron example. 2015-07-18 18:44:11 -07:00
Dan Helfman 96ce9309e0 Added tag 0.1.0 for changeset 38d72677343f 2015-07-18 18:39:33 -07:00
Dan Helfman f2f8503e77 New "borgmatic" command to support Borg backup software, a fork of Attic. 2015-07-18 18:35:29 -07:00
Dan Helfman bdce17ae71 Added tag 0.0.7 for changeset cf4c7065f071 2015-07-17 21:58:58 -07:00
Dan Helfman b501a568aa Flag for multiple levels of verbosity: some, and lots. 2015-07-17 21:58:50 -07:00
Dan Helfman 1578b44536 Improved mocking of Python builtins in unit tests. 2015-06-14 11:00:46 -07:00
Dan Helfman 932145f20b Adding some explanitory text about consistency checks to README example. 2015-05-10 22:06:48 -07:00
Dan Helfman 6c2e668262 Added tag 0.0.6 for changeset 7ea93ca83f42 2015-05-10 22:00:34 -07:00
Dan Helfman df2d059af2 New configuration section for customizing which Attic consistency checks run, if any. 2015-05-10 22:00:31 -07:00
Dan Helfman 301eb4926e Passing through command-line options from tox to nosetests. 2015-03-15 11:15:40 -07:00
Dan Helfman 897046cf51 Added tag 0.0.5 for changeset a03495a8e8b4 2015-03-15 10:47:58 -07:00
Dan Helfman 645c66c98a Re-fixing version. 2015-03-15 10:47:49 -07:00
Dan Helfman 811678396a Added tag 0.0.5 for changeset 569aef47a9b2 2015-03-15 10:46:55 -07:00
Dan Helfman fd94debba6 Added tag 0.0.5 for changeset aa8a807f4ba2 2015-03-15 10:44:25 -07:00
Dan Helfman 851dd69785 Backout out "helpful" error message that broke --verbose. 2015-03-15 10:44:18 -07:00
Dan Helfman 6430e3799a Added tag 0.0.5 for changeset b31d51b63370 2015-03-15 10:39:08 -07:00
Dan Helfman c15a89b9f8 Added tag 0.0.4 for changeset 4bb2e81fc770 2015-03-15 10:19:12 -07:00
Dan Helfman 0bf033de06 Releasing 0.0.4. 2015-03-15 10:15:03 -07:00
Dan Helfman 5bc7c04cdf Now using tox to run tests against multiple versions of Python in one go. 2015-03-15 10:14:30 -07:00
Dan Helfman a5eda0d305 Fixing Python 3 test incompatibility with builtins. 2015-03-15 10:14:16 -07:00
Dan Helfman 04216922c4 Bumping setup.py version. 2015-03-15 09:52:40 -07:00
Dan Helfman aa482d088c Added nosetests config file (setup.cfg) with defaults. 2015-03-15 09:41:58 -07:00
Dan Helfman b755d09b0a Added a troubleshooting section with steps to deal with broken pipes. 2015-02-28 11:03:22 -08:00
Dan Helfman 89703111d0 Updating install instructions so you can upgrade from one release of atticmatic to the next. 2015-02-14 09:31:42 -08:00
Dan Helfman 0f21919626 Added tag 0.0.3 for changeset 7730ae34665c 2015-02-14 09:24:15 -08:00
Dan Helfman e3fa18b892 After pruning, run attic's consistency checks on all archives. 2015-02-14 09:23:40 -08:00
Dan Helfman 6d4e40a819 Correcting doc string based on updated command-line arguments source. 2014-12-20 11:42:27 -08:00
Dan Helfman de40570db4 Integration tests for argument parsing. 2014-12-20 11:37:25 -08:00
Dan Helfman 2a45f43167 Adding a note about repository encryption. 2014-12-20 10:56:03 -08:00
Dan Helfman f8c6da6ff4 Added tag 0.0.2 for changeset 467d3a3ce918 2014-12-06 18:35:28 -08:00
Dan Helfman 63018fad4e Configuration support for additional attic prune flags: keep_within, keep_hourly, keep_yearly, and prefix. 2014-12-06 18:35:20 -08:00
Dan Helfman 2038354b61 For convenience, adding some short-form arguments in addition to the long-form arguments. 2014-12-01 22:47:51 -08:00
Dan Helfman 9d7db93655 Adding sudo to installation of test dependencies, for consistency with installation of main dependencies. 2014-12-01 22:39:11 -08:00
Dan Helfman f4d8469085 Preventing ConfigParser from swallowing file read IOErrors, so that the user gets a more useful message. 2014-12-01 22:35:25 -08:00
Dan Helfman 273ca83f6b Fixing configparser import for Python 3. 2014-12-01 22:14:35 -08:00
Dan Helfman 41008b3ef4 Expanding description. 2014-12-01 20:36:43 -08:00
Dan Helfman 16ba9ef4c6 Adding authors/contributors file. 2014-12-01 20:30:07 -08:00
Dan Helfman 25b78d6de4 Python 3 ConfigParser compatibility. 2014-12-01 20:26:19 -08:00
Dan Helfman d311e90792 Python 3 compatible exceptions. 2014-12-01 20:23:29 -08:00
Dan Helfman 0fc9acce7c Mentioning source code location explicitly. 2014-12-01 20:22:49 -08:00
Dan Helfman 5a3c66a667 Saving README when rendered such that it can be served easily. 2014-12-01 20:15:21 -08:00
Dan Helfman f8387c7bd1 Markdown metadata and link formatting updates. 2014-12-01 19:49:25 -08:00
Dan Helfman 181fed83df Renaming README to indicate markdown. 2014-11-27 09:34:13 -08:00
Dan Helfman 243e2b698d Playing nicely with markdown. 2014-11-27 09:29:31 -08:00
Dan Helfman f2f4ce38c9 Adding note about making etc configuration directory before copying a file to it. 2014-11-26 20:15:21 -08:00
Dan Helfman 965ec51171 Merge pull request #1 from hajs/master
fixed README: copy cronjob to /etc/cron.d instead of /etc/init.d
2014-11-26 08:03:47 -08:00
Henning Schroder cd11e89fde fixed README: copy cronjob to /etc/cron.d instead of /etc/init.d (like comment in sample/atticmatic.cron correctly explains) 2014-11-26 13:04:14 +01:00
Dan Helfman 866c707f58 Updating README with clarifications and examples. 2014-11-25 16:01:59 -08:00
Dan Helfman c459c42ffc Note about hosting arrangement. 2014-11-18 18:32:16 -08:00
Dan Helfman 0bca430c02 Merge with Github head! 2014-11-18 18:28:20 -08:00
Dan Helfman 7b3efddcbd Adding GPL v3 license. 2014-11-18 18:22:51 -08:00
Dan Helfman cdb4035bab Unit tests for attic invocation code. 2014-11-17 22:19:34 -08:00
Dan Helfman c7d68246cd Adding unit tests for config module. 2014-11-17 21:57:44 -08:00
Dan Helfman 8784e19e45 Adding contact info. 2014-11-17 18:35:47 -08:00
Dan Helfman 48523810ec Adding PATH necessary to find the attic binary. 2014-11-01 17:46:04 -07:00
Dan Helfman 6dff335c8b Initial import. 2014-10-30 22:34:03 -07:00
107 changed files with 6608 additions and 1213 deletions
+18
View File
@@ -0,0 +1,18 @@
pipeline:
build:
image: python:${PYTHON_VERSION}-alpine${ALPINE_VERSION}
pull: true
commands:
- pip install tox
- tox
- apk add --no-cache borgbackup
- tox -e end-to-end
matrix:
ALPINE_VERSION:
- 3.7
- 3.8
PYTHON_VERSION:
- 3.5
- 3.6
- 3.7
+3
View File
@@ -1,6 +1,9 @@
*.egg-info
*.pyc
*.swp
.cache
.coverage
.pytest_cache
.tox
build
dist
-7
View File
@@ -1,7 +0,0 @@
syntax: glob
*.egg-info
*.pyc
*.swp
.tox
build
dist
-27
View File
@@ -1,27 +0,0 @@
467d3a3ce9185e84ee51ca9156499162efd94f9a 0.0.2
7730ae34665c0dedf46deab90b32780abf6dbaff 0.0.3
4bb2e81fc77038be4499b7ea6797ab7d109460e0 0.0.4
b31d51b633701554e84f996cc0c73bad2990780b 0.0.5
b31d51b633701554e84f996cc0c73bad2990780b 0.0.5
aa8a807f4ba28f0652764ed14713ffea2fd6922d 0.0.5
aa8a807f4ba28f0652764ed14713ffea2fd6922d 0.0.5
569aef47a9b25c55b13753f94706f5d330219995 0.0.5
569aef47a9b25c55b13753f94706f5d330219995 0.0.5
a03495a8e8b471da63b5e2ae79d3ff9065839c2a 0.0.5
7ea93ca83f426ec0a608a68580c72c0775b81f86 0.0.6
cf4c7065f0711deda1cba878398bc05390e2c3f9 0.0.7
38d72677343f0a5d6845f4ac50d6778397083d45 0.1.0
ac5dfa01e9d14d09845f5e94c2c679e21c5eb2f9 0.1.1
ac5dfa01e9d14d09845f5e94c2c679e21c5eb2f9 0.1.1
7b6c87dca7ea312b2257ac1b46857b3f8c56b39c 0.1.1
83067f995dd391e38544a7722dc3b254b59c5521 0.1.2
acc7fb61566fe8028c179f43ecc735c851220b06 0.1.3
6dda59c12de88f060eb7244e6d330173985a9639 0.1.4
6dda59c12de88f060eb7244e6d330173985a9639 0.1.4
e58246fc92bb22c2b2fd8b86a1227de69d2d0315 0.1.4
0afff209b902698c2266986129d6dc9f5f913101 0.1.5
4c63f3d90ec2bf6af1714a3acec84654a7c9edf3 0.1.6
5a458ebef804be14e30d7375e3e9fbc26aedb80d 0.1.7
977f19c2f6a515be6c5ef69cf17b0e0989532209 github/yaml_config_files
0000000000000000000000000000000000000000 github/yaml_config_files
28434dd0440cc8da44c2f3e9bd7e9402a59c3b40 github/master
+7
View File
@@ -1,5 +1,12 @@
Dan Helfman <witten@torsion.org>: Main developer
Alexander Görtz: Python 3 compatibility
Florian Lindner: Logging rewrite
Henning Schroeder: Copy editing
Johannes Feichtner: Support for user hooks
Michele Lazzeri: Custom archive names
Nick Whyte: Support prefix filtering for archive consistency checks
newtonne: Read encryption password from external file
Robin `ypid` Schneider: Support additional options of Borg
Scott Squires: Custom archive names
Thomas LÉVEIL: Support for a keep_minutely prune option. Support for the --json option
+1
View File
@@ -0,0 +1 @@
include borgmatic/config/schema.yaml
+259 -4
View File
@@ -1,3 +1,258 @@
1.3.1
* #155: Fix for invalid JSON output when using multiple borgmatic configuration files.
* #157: Fix for seemingly random filename ordering when running through a directory of
configuration files.
* Fix for empty JSON output when using --create --json.
* Now capturing Borg output only when --json flag is used. Previously, borgmatic delayed Borg
output even without the --json flag.
1.3.0
* #148: Configuration file includes and merging via "!include" tag to support reuse of common
options across configuration files.
1.2.18
* #147: Support for Borg create/extract --numeric-owner flag via "numeric_owner" option in
borgmatic's location section.
1.2.17
* #140: List the files within an archive via --list --archive option.
1.2.16
* #119: Include a sample borgmatic configuration file in the documentation.
* #123: Support for Borg archive restoration via borgmatic --extract command-line flag.
* Refactor documentation into multiple separate pages for clarity and findability.
* Organize options within command-line help into logical groups.
* Exclude tests from distribution packages.
1.2.15
* #127: Remove date echo from schema example, as it's not a substitute for real logging.
* #132: Leave exclude_patterns glob expansion to Borg, since doing it in borgmatic leads to
confusing behavior.
* #136: Handle and format validation errors raised during argument parsing.
* #138: Allow use of --stats flag when --create or --prune flags are implied.
1.2.14
* #103: When generating sample configuration with generate-borgmatic-config, document the defaults
for each option.
* #116: When running multiple configuration files, attempt all configuration files even if one of
them errors. Log a summary of results at the end.
* Add borgmatic --version command-line flag to get the current installed version number.
1.2.13
* #100: Support for --stats command-line flag independent of --verbosity.
* #117: With borgmatic --init command-line flag, proceed without erroring if a repository already
exists.
1.2.12
* #110: Support for Borg repository initialization via borgmatic --init command-line flag.
* #111: Update Borg create --filter values so a dry run lists files to back up.
* #113: Update README with link to a new/forked Docker image.
* Prevent deprecated --excludes command-line option from being used.
* Refactor README a bit to flow better for first-time users.
* Update README with a few additional borgmatic packages (Debian and Ubuntu).
1.2.11
* #108: Support for Borg create --progress via borgmatic command-line flag.
1.2.10
* #105: Support for Borg --chunker-params create option via "chunker_params" option in borgmatic's
storage section.
1.2.9
* #102: Fix for syntax error that occurred in Python 3.5 and below.
* Make automated tests support running in Python 3.5.
1.2.8
* #73: Enable consistency checks for only certain repositories via "check_repositories" option in
borgmatic's consistency configuration. Handy for large repositories that take forever to check.
* Include link to issue tracker within various command output.
* Run continuous integration tests on a matrix of Python and Borg versions.
1.2.7
* #98: Support for Borg --keep-secondly prune option.
* Use Black code formatter and Flake8 code checker as part of running automated tests.
* Add an end-to-end automated test that actually integrates with Borg.
* Set up continuous integration for borgmatic automated tests on projects.evoworx.org.
1.2.6
* Fix generated configuration to also include a "keep_daily" value so pruning works out of the
box.
1.2.5
* #57: When generating sample configuration with generate-borgmatic-config, comment out all
optional configuration so as to streamline the initial configuration process.
1.2.4
* Fix for archive checking traceback due to parameter mismatch.
1.2.3
* #64, #90, #92: Rewrite of logging system. Now verbosity flags passed to Borg are derived from
borgmatic's log level. Note that the output of borgmatic might slightly change.
* Part of #80: Support for Borg create --read-special via "read_special" option in borgmatic's
location configuration.
* #87: Support for Borg create --checkpoint-interval via "checkpoint_interval" option in
borgmatic's storage configuration.
* #88: Fix declared pykwalify compatibility version range in setup.py to prevent use of ancient
versions of pykwalify with large version numbers.
* #89: Pass --show-rc option to Borg when at highest verbosity level.
* #94: Support for Borg --json option via borgmatic command-line to --create archives.
1.2.2
* #85: Fix compatibility issue between pykwalify and ruamel.yaml 0.15.52, which manifested in
borgmatic as a pykwalify RuleError.
1.2.1
* Skip before/after backup hooks when only doing --prune, --check, --list, and/or --info.
* #71: Support for XDG_CONFIG_HOME environment variable for specifying alternate user ~/.config/
path.
* #74, #83: Support for Borg --json option via borgmatic command-line to --list archives or show
archive --info in JSON format, ideal for programmatic consumption.
* #38, #76: Upgrade ruamel.yaml compatibility version range and fix support for Python 3.7.
* #77: Skip non-"*.yaml" config filenames in /etc/borgmatic.d/ so as not to parse backup files,
editor swap files, etc.
* #81: Document user-defined hooks run before/after backup, or on error.
* Add code style guidelines to the documention.
1.2.0
* #61: Support for Borg --list option via borgmatic command-line to list all archives.
* #61: Support for Borg --info option via borgmatic command-line to display summary information.
* #62: Update README to mention other ways of installing borgmatic.
* Support for Borg --prefix option for consistency checks via "prefix" option in borgmatic's
consistency configuration.
* Add introductory screencast link to documentation.
* #59: Ignore "check_last" and consistency "prefix" when "archives" not in consistency checks.
* #60: Add "Persistent" flag to systemd timer example.
* #63: Support for Borg --nobsdflags option to skip recording bsdflags (e.g. NODUMP, IMMUTABLE) in
archive.
* #69: Support for Borg prune --umask option using value of existing "umask" option in borgmatic's
storage configuration.
* Update tox.ini to only assume Python 3.x instead of Python 3.4 specifically.
* Add ~/.config/borgmatic/config.yaml to default configuration path probing.
* Document how to develop on and contribute to borgmatic.
1.1.15
* Support for Borg BORG_PASSCOMMAND environment variable to read a password from an external file.
* Fix for Borg create error when using borgmatic's --dry-run and --verbosity options together.
Work-around for behavior introduced in Borg 1.1.3: https://github.com/borgbackup/borg/issues/3298
* #55: Fix for missing tags/releases on Gitea and GitHub project hosting.
* #56: Support for Borg --lock-wait option for the maximum wait for a repository/cache lock.
* #58: Support for using tilde in exclude_patterns to reference home directory.
1.1.14
* #49: Fix for typo in --patterns-from option.
* #47: Support for Borg --dry-run option via borgmatic command-line.
1.1.13
* #54: Fix for incorrect consistency check flags passed to Borg when all three checks ("repository",
"archives", and "extract") are specified in borgmatic configuration.
* #48: Add "local_path" to configuration for specifying an alternative Borg executable path.
* #49: Support for Borg experimental --patterns-from and --patterns options for specifying mixed
includes/excludes.
* Moved issue tracker from Taiga to integrated Gitea tracker at
https://projects.torsion.org/witten/borgmatic/issues
1.1.12
* #46: Declare dependency on pykwalify 1.6 or above, as older versions yield "Unknown key: version"
rule errors.
* Support for Borg --keep-minutely prune option.
1.1.11
* #26: Add "ssh_command" to configuration for specifying a custom SSH command or options.
* Fix for incorrect /etc/borgmatic.d/ configuration path probing on macOS. This problem manifested
as an error on startup: "[Errno 2] No such file or directory: '/etc/borgmatic.d'".
1.1.10
* Pass several Unix signals through to child processes like Borg. This means that Borg now properly
shuts down if borgmatic is terminated (e.g. due to a system suspend).
* #30: Support for using tilde in repository paths to reference home directory.
* #43: Support for Borg --files-cache option for setting the files cache operation mode.
* #45: Support for Borg --remote-ratelimit option for limiting upload rate.
* Log invoked Borg commands when at highest verbosity level.
1.1.9
* #17, #39: Support for user-defined hooks before/after backup, or on error.
* #34: Improve clarity of logging spew at high verbosity levels.
* #30: Support for using tilde in source directory path to reference home directory.
* Require "prefix" in retention section when "archive_name_format" is set. This is to avoid
accidental pruning of archives with a different archive name format. For similar reasons, default
"prefix" to "{hostname}-" if not specified.
* Convert main source repository from Mercurial to Git.
* Update dead links to Borg documentation.
1.1.8
* #40: Fix to make /etc/borgmatic/config.yaml optional rather than required when using the default
config paths.
1.1.7
* #29: Add "archive_name_format" to configuration for customizing archive names.
* Fix for traceback when "exclude_from" value is empty in configuration file.
* When pruning, make highest verbosity level list archives kept and pruned.
* Clarification of Python 3 pip usage in documentation.
1.1.6
* #13, #36: Support for Borg --exclude-from, --exclude-caches, and --exclude-if-present options.
1.1.5
* #35: New "extract" consistency check that performs a dry-run extraction of the most recent
archive.
1.1.4
* #18: Added command-line flags for performing a borgmatic run with only pruning, creating, or
checking enabled. This supports use cases like running consistency checks from a different cron
job with a different frequency, or running pruning with a different verbosity level.
1.1.3
* #15: Support for running multiple config files in /etc/borgmatic.d/ from a single borgmatic run.
* Fix for generate-borgmatic-config writing config with invalid one_file_system value.
1.1.2
* #33: Fix for passing check_last as integer to subprocess when calling Borg.
1.1.1
* Part of #33: Fix for upgrade-borgmatic-config converting check_last option as a string instead of
an integer.
* Fix for upgrade-borgmatic-config erroring when consistency checks option is not present.
1.1.0
* Switched config file format to YAML. Run upgrade-borgmatic-config to upgrade.
* Added generate-borgmatic-config command for initial config creation.
* Dropped Python 2 support. Now Python 3 only.
* #19: Fix for README mention of sample files not included in package.
* #23: Sample files for triggering borgmatic from a systemd timer.
* Support for backing up to multiple repositories.
* To free up space, now pruning backups prior to creating a new backup.
* Enabled test coverage output during tox runs.
* Added logo.
1.0.3
* #22: Fix for verbosity flag not actually causing verbose output.
1.0.2
* #21: Fix for traceback when remote_path option is missing.
1.0.1
* #20: Support for Borg's --remote-path option to use an alternate Borg
executable. See sample/config.
1.0.0
* Attic is no longer supported, as there hasn't been any recent development on
it. Dropping Attic support will allow faster iteration on Borg-specific
features. If you're still using Attic, this is a good time to switch to Borg!
* Project renamed from atticmatic to borgmatic. See the borgmatic README for
information on upgrading.
0.1.8
* Fix for handling of spaces in source_directories which resulted in backup up everything.
@@ -9,13 +264,13 @@
0.1.7
* #11: Fixed parsing of punctuation in configuration file.
* #12: Fixed parsing of punctuation in configuration file.
* Better error message when configuration file is missing.
0.1.6
* #9: New configuration option for the encryption passphrase.
* #10: Support for Borg's new archive compression feature.
* #10: New configuration option for the encryption passphrase.
* #11: Support for Borg's new archive compression feature.
0.1.5
@@ -27,7 +282,7 @@
0.1.3
* #1: Add support for "borg check --last N" to Borg backend.
* #2: Add support for "borg check --last N" to Borg backend.
0.1.2
+75 -123
View File
@@ -1,146 +1,98 @@
title: Atticmatic
---
title: borgmatic
permalink: borgmatic/index.html
---
## Overview
atticmatic is a simple Python wrapper script for the
[Attic](https://attic-backup.org/) and
[Borg](https://borgbackup.readthedocs.org/en/stable/) backup software that
initiates a backup, prunes any old backups according to a retention policy,
and validates backups for consistency. The script supports specifying your
settings in a declarative configuration file rather than having to put them
all on the command-line, and handles common errors.
<img src="https://projects.torsion.org/witten/borgmatic/raw/branch/master/static/borgmatic.png" width="150px" style="float: right; padding-left: 1em;">
borgmatic is a simple Python wrapper script for the
[Borg](https://www.borgbackup.org/) backup software that initiates a backup,
prunes any old backups according to a retention policy, and validates backups
for consistency. The script supports specifying your settings in a declarative
configuration file rather than having to put them all on the command-line, and
handles common errors.
Here's an example config file:
```INI
[location]
# Space-separated list of source directories to backup.
# Globs are expanded.
source_directories: /home /etc /var/log/syslog*
```yaml
location:
# List of source directories to backup. Globs are expanded.
source_directories:
- /home
- /etc
- /var/log/syslog*
# Path to local or remote backup repository.
repository: user@backupserver:sourcehostname.attic
# Paths to local or remote repositories.
repositories:
- user@backupserver:sourcehostname.borg
[retention]
# Retention policy for how many backups to keep in each category.
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
# Any paths matching these patterns are excluded from backups.
exclude_patterns:
- /home/*/.cache
[consistency]
# Consistency checks to run, or "disabled" to prevent checks.
checks: repository archives
retention:
# Retention policy for how many backups to keep in each category.
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
consistency:
# List of consistency checks to run: "repository", "archives", or both.
checks:
- repository
- archives
```
Additionally, exclude patterns can be specified in a separate excludes config
file, one pattern per line.
borgmatic is hosted at <https://torsion.org/borgmatic> with [source code
available](https://projects.torsion.org/witten/borgmatic). It's also mirrored
on [GitHub](https://github.com/witten/borgmatic) for convenience.
atticmatic is hosted at <https://torsion.org/atticmatic> with [source code
available](https://torsion.org/hg/atticmatic). It's also mirrored on
[GitHub](https://github.com/witten/atticmatic) and
[BitBucket](https://bitbucket.org/dhelfman/atticmatic) for convenience.
Want to see borgmatic in action? Check out the <a
href="https://asciinema.org/a/203761" target="_blank">screencast</a>.
<script src="https://asciinema.org/a/203761.js" id="asciicast-203761" async></script>
## Setup
## How-to guides
To get up and running, follow the [Attic Quick
Start](https://attic-backup.org/quickstart.html) or the [Borg Quick
Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) to create a
repository on a local or remote host. Note that if you plan to run atticmatic
on a schedule with cron, and you encrypt your attic repository with a
passphrase instead of a key file, you'll need to set the atticmatic
`encryption_passphrase` configuration variable. See the repository encryption
section of the Quick Start for more info.
If the repository is on a remote host, make sure that your local root user has
key-based ssh access to the desired user account on the remote host.
To install atticmatic, run the following command to download and install it:
sudo pip install --upgrade atticmatic
If you are using Attic, copy the following configuration files:
sudo cp sample/atticmatic.cron /etc/cron.d/atticmatic
sudo mkdir /etc/atticmatic/
sudo cp sample/config sample/excludes /etc/atticmatic/
If you are using Borg, copy the files like this instead:
sudo cp sample/borgmatic.cron /etc/cron.d/borgmatic
sudo mkdir /etc/borgmatic/
sudo cp sample/config sample/excludes /etc/borgmatic/
Lastly, modify the /etc files with your desired configuration.
* [Set up backups with borgmatic](docs/how-to/set-up-backups.md) ⬅ *Start here!*
* [Make per-application backups](docs/how-to/make-per-application-backups.md)
* [Deal with very large backups](docs/how-to/deal-with-very-large-backups.md)
* [Inspect your backups](docs/how-to/inspect-your-backups.md)
* [Restore a backup](docs/how-to/restore-a-backup.md)
* [Run preparation steps before backups](docs/how-to/run-preparation-steps-before-backups.md)
* [Upgrade borgmatic](docs/how-to/upgrade.md)
* [Develop on borgmatic](docs/how-to/develop-on-borgmatic.md)
## Usage
## Reference guides
You can run atticmatic and start a backup simply by invoking it without
arguments:
atticmatic
Or, if you're using Borg, use this command instead to make use of the Borg
backend:
borgmatic
This will also prune any old backups as per the configured retention policy,
and check backups for consistency problems due to things like file damage.
By default, the backup will proceed silently except in the case of errors. But
if you'd like to to get additional information about the progress of the
backup as it proceeds, use the verbosity option:
atticmattic --verbosity 1
Or, for even more progress spew:
atticmattic --verbosity 2
If you'd like to see the available command-line arguments, view the help:
atticmattic --help
* [borgmatic configuration reference](docs/reference/configuration.md)
* [borgmatic command-line reference](docs/reference/command-line.md)
## Running tests
## Support and contributing
First install tox, which is used for setting up testing environments:
### Issues
pip install tox
Then, to actually run tests, run:
tox
## Troubleshooting
### Broken pipe with remote repository
When running atticmatic on a large remote repository, you may receive errors
like the following, particularly while "attic check" is validating backups for
consistency:
Write failed: Broken pipe
attic: Error: Connection closed by remote host
This error can be caused by an ssh timeout, which you can rectify by adding
the following to the ~/.ssh/config file on the client:
Host *
ServerAliveInterval 120
This should make the client keep the connection alive while validating
backups.
## Issues and feedback
Got an issue or an idea for a feature enhancement? Check out the [atticmatic
issue tracker](https://tree.taiga.io/project/witten-atticmatic/issues). In
order to create a new issue or comment on an issue, you'll need to [login
first](https://tree.taiga.io/login).
You've got issues? Or an idea for a feature enhancement? We've got an [issue
tracker](https://projects.torsion.org/witten/borgmatic/issues). In order to
create a new issue or comment on an issue, you'll need to [login
first](https://projects.torsion.org/user/login). Note that you can login with
an existing GitHub account if you prefer.
Other questions or comments? Contact <mailto:witten@torsion.org>.
### Contributing
If you'd like to contribute to borgmatic development, please feel free to
submit a [Pull Request](https://projects.torsion.org/witten/borgmatic/pulls)
or open an [issue](https://projects.torsion.org/witten/borgmatic/issues) first
to discuss your idea. We also accept Pull Requests on GitHub, if that's more
your thing. In general, contributions are very welcome. We don't bite!
Also, please check out the [borgmatic development
how-to](docs/how-to/develop-on-borgmatic.md) for info on cloning source code,
running tests, etc.
-14
View File
@@ -1,14 +0,0 @@
from functools import partial
from atticmatic.backends import shared
# An atticmatic backend that supports Attic for actually handling backups.
COMMAND = 'attic'
CONFIG_FORMAT = shared.CONFIG_FORMAT
initialize = partial(shared.initialize, command=COMMAND)
create_archive = partial(shared.create_archive, command=COMMAND)
prune_archives = partial(shared.prune_archives, command=COMMAND)
check_archives = partial(shared.check_archives, command=COMMAND)
-41
View File
@@ -1,41 +0,0 @@
from functools import partial
from atticmatic.config import Section_format, option
from atticmatic.backends import shared
# An atticmatic backend that supports Borg for actually handling backups.
COMMAND = 'borg'
CONFIG_FORMAT = (
Section_format(
'location',
(
option('source_directories'),
option('one_file_system', value_type=bool, required=False),
option('repository'),
),
),
Section_format(
'storage',
(
option('encryption_passphrase', required=False),
option('compression', required=False),
option('umask', required=False),
),
),
shared.CONFIG_FORMAT[2], # retention
Section_format(
'consistency',
(
option('checks', required=False),
option('check_last', required=False),
),
)
)
initialize = partial(shared.initialize, command=COMMAND)
create_archive = partial(shared.create_archive, command=COMMAND)
prune_archives = partial(shared.prune_archives, command=COMMAND)
check_archives = partial(shared.check_archives, command=COMMAND)
-219
View File
@@ -1,219 +0,0 @@
from datetime import datetime
import os
import re
import platform
import subprocess
from glob import glob
from itertools import chain
from atticmatic.config import Section_format, option
from atticmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
# Common backend functionality shared by Attic and Borg. As the two backup
# commands diverge, these shared functions will likely need to be replaced
# with non-shared functions within atticmatic.backends.attic and
# atticmatic.backends.borg.
CONFIG_FORMAT = (
Section_format(
'location',
(
option('source_directories'),
option('repository'),
),
),
Section_format(
'storage',
(
option('encryption_passphrase', required=False),
),
),
Section_format(
'retention',
(
option('keep_within', required=False),
option('keep_hourly', int, required=False),
option('keep_daily', int, required=False),
option('keep_weekly', int, required=False),
option('keep_monthly', int, required=False),
option('keep_yearly', int, required=False),
option('prefix', required=False),
),
),
Section_format(
'consistency',
(
option('checks', required=False),
),
)
)
def initialize(storage_config, command):
passphrase = storage_config.get('encryption_passphrase')
if passphrase:
os.environ['{}_PASSPHRASE'.format(command.upper())] = passphrase
def create_archive(
excludes_filename, verbosity, storage_config, source_directories, repository, command,
one_file_system=None
):
'''
Given an excludes filename (or None), a vebosity flag, a storage config dict, a space-separated
list of source directories, a local or remote repository path, and a command to run, create an
attic archive.
'''
sources = re.split('\s+', source_directories)
sources = tuple(chain.from_iterable(glob(x) or [x] for x in sources))
exclude_flags = ('--exclude-from', excludes_filename) if excludes_filename else ()
compression = storage_config.get('compression', None)
compression_flags = ('--compression', compression) if compression else ()
umask = storage_config.get('umask', None)
umask_flags = ('--umask', str(umask)) if umask else ()
one_file_system_flags = ('--one-file-system',) if one_file_system else ()
verbosity_flags = {
VERBOSITY_SOME: ('--stats',),
VERBOSITY_LOTS: ('--verbose', '--stats'),
}.get(verbosity, ())
full_command = (
command, 'create',
'{repo}::{hostname}-{timestamp}'.format(
repo=repository,
hostname=platform.node(),
timestamp=datetime.now().isoformat(),
),
) + sources + exclude_flags + compression_flags + one_file_system_flags + \
umask_flags + verbosity_flags
subprocess.check_call(full_command)
def _make_prune_flags(retention_config):
'''
Given a retention config dict mapping from option name to value, tranform it into an iterable of
command-line name-value flag pairs.
For example, given a retention config of:
{'keep_weekly': 4, 'keep_monthly': 6}
This will be returned as an iterable of:
(
('--keep-weekly', '4'),
('--keep-monthly', '6'),
)
'''
return (
('--' + option_name.replace('_', '-'), str(retention_config[option_name]))
for option_name, value in retention_config.items()
)
def prune_archives(verbosity, repository, retention_config, command):
'''
Given a verbosity flag, a local or remote repository path, a retention config dict, and a
command to run, prune attic archives according the the retention policy specified in that
configuration.
'''
verbosity_flags = {
VERBOSITY_SOME: ('--stats',),
VERBOSITY_LOTS: ('--verbose', '--stats'),
}.get(verbosity, ())
full_command = (
command, 'prune',
repository,
) + tuple(
element
for pair in _make_prune_flags(retention_config)
for element in pair
) + verbosity_flags
subprocess.check_call(full_command)
DEFAULT_CHECKS = ('repository', 'archives')
def _parse_checks(consistency_config):
'''
Given a consistency config with a space-separated "checks" option, transform it to a tuple of
named checks to run.
For example, given a retention config of:
{'checks': 'repository archives'}
This will be returned as:
('repository', 'archives')
If no "checks" option is present, return the DEFAULT_CHECKS. If the checks value is the string
"disabled", return an empty tuple, meaning that no checks should be run.
'''
checks = consistency_config.get('checks', '').strip()
if not checks:
return DEFAULT_CHECKS
return tuple(
check for check in consistency_config['checks'].split(' ')
if check.lower() not in ('disabled', '')
)
def _make_check_flags(checks, check_last=None):
'''
Given a parsed sequence of checks, transform it into tuple of command-line flags.
For example, given parsed checks of:
('repository',)
This will be returned as:
('--repository-only',)
Additionally, if a check_last value is given, a "--last" flag will be added. Note that only
Borg supports this flag.
'''
last_flag = ('--last', check_last) if check_last else ()
if checks == DEFAULT_CHECKS:
return last_flag
return tuple(
'--{}-only'.format(check) for check in checks
) + last_flag
def check_archives(verbosity, repository, consistency_config, command):
'''
Given a verbosity flag, a local or remote repository path, a consistency config dict, and a
command to run, check the contained attic archives for consistency.
If there are no consistency checks to run, skip running them.
'''
checks = _parse_checks(consistency_config)
check_last = consistency_config.get('check_last', None)
if not checks:
return
verbosity_flags = {
VERBOSITY_SOME: ('--verbose',),
VERBOSITY_LOTS: ('--verbose',),
}.get(verbosity, ())
full_command = (
command, 'check',
repository,
) + _make_check_flags(checks, check_last) + verbosity_flags
# The check command spews to stdout/stderr even without the verbose flag. Suppress it.
stdout = None if verbosity_flags else open(os.devnull, 'w')
subprocess.check_call(full_command, stdout=stdout, stderr=subprocess.STDOUT)
-75
View File
@@ -1,75 +0,0 @@
from __future__ import print_function
from argparse import ArgumentParser
from importlib import import_module
import os
from subprocess import CalledProcessError
import sys
from atticmatic.config import parse_configuration
DEFAULT_CONFIG_FILENAME_PATTERN = '/etc/{}/config'
DEFAULT_EXCLUDES_FILENAME_PATTERN = '/etc/{}/excludes'
def parse_arguments(command_name, *arguments):
'''
Given the name of the command with which this script was invoked and command-line arguments,
parse the arguments and return them as an ArgumentParser instance. Use the command name to
determine the default configuration and excludes paths.
'''
config_filename_default = DEFAULT_CONFIG_FILENAME_PATTERN.format(command_name)
excludes_filename_default = DEFAULT_EXCLUDES_FILENAME_PATTERN.format(command_name)
parser = ArgumentParser()
parser.add_argument(
'-c', '--config',
dest='config_filename',
default=config_filename_default,
help='Configuration filename',
)
parser.add_argument(
'--excludes',
dest='excludes_filename',
default=excludes_filename_default if os.path.exists(excludes_filename_default) else None,
help='Excludes filename',
)
parser.add_argument(
'-v', '--verbosity',
type=int,
help='Display verbose progress (1 for some, 2 for lots)',
)
return parser.parse_args(arguments)
def load_backend(command_name):
'''
Given the name of the command with which this script was invoked, return the corresponding
backend module responsible for actually dealing with backups.
'''
backend_name = {
'atticmatic': 'attic',
'borgmatic': 'borg',
}.get(command_name, 'attic')
return import_module('atticmatic.backends.{}'.format(backend_name))
def main():
try:
command_name = os.path.basename(sys.argv[0])
args = parse_arguments(command_name, *sys.argv[1:])
backend = load_backend(command_name)
config = parse_configuration(args.config_filename, backend.CONFIG_FORMAT)
repository = config.location['repository']
backend.initialize(config.storage)
backend.create_archive(
args.excludes_filename, args.verbosity, config.storage, **config.location
)
backend.prune_archives(args.verbosity, repository, config.retention)
backend.check_archives(args.verbosity, repository, config.consistency)
except (ValueError, IOError, CalledProcessError) as error:
print(error, file=sys.stderr)
sys.exit(1)
-11
View File
@@ -1,11 +0,0 @@
from flexmock import flexmock
import sys
def builtins_mock():
try:
# Python 2
return flexmock(sys.modules['__builtin__'])
except KeyError:
# Python 3
return flexmock(sys.modules['builtins'])
@@ -1,72 +0,0 @@
import os
import sys
from flexmock import flexmock
import pytest
from atticmatic import command as module
COMMAND_NAME = 'foomatic'
def test_parse_arguments_with_no_arguments_uses_defaults():
flexmock(os.path).should_receive('exists').and_return(True)
parser = module.parse_arguments(COMMAND_NAME)
assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME_PATTERN.format(COMMAND_NAME)
assert parser.excludes_filename == module.DEFAULT_EXCLUDES_FILENAME_PATTERN.format(COMMAND_NAME)
assert parser.verbosity == None
def test_parse_arguments_with_filename_arguments_overrides_defaults():
flexmock(os.path).should_receive('exists').and_return(True)
parser = module.parse_arguments(COMMAND_NAME, '--config', 'myconfig', '--excludes', 'myexcludes')
assert parser.config_filename == 'myconfig'
assert parser.excludes_filename == 'myexcludes'
assert parser.verbosity == None
def test_parse_arguments_with_missing_default_excludes_file_sets_filename_to_none():
flexmock(os.path).should_receive('exists').and_return(False)
parser = module.parse_arguments(COMMAND_NAME)
assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME_PATTERN.format(COMMAND_NAME)
assert parser.excludes_filename == None
assert parser.verbosity == None
def test_parse_arguments_with_missing_overridden_excludes_file_retains_filename():
flexmock(os.path).should_receive('exists').and_return(False)
parser = module.parse_arguments(COMMAND_NAME, '--excludes', 'myexcludes')
assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME_PATTERN.format(COMMAND_NAME)
assert parser.excludes_filename == 'myexcludes'
assert parser.verbosity == None
def test_parse_arguments_with_verbosity_flag_overrides_default():
flexmock(os.path).should_receive('exists').and_return(True)
parser = module.parse_arguments(COMMAND_NAME, '--verbosity', '1')
assert parser.config_filename == module.DEFAULT_CONFIG_FILENAME_PATTERN.format(COMMAND_NAME)
assert parser.excludes_filename == module.DEFAULT_EXCLUDES_FILENAME_PATTERN.format(COMMAND_NAME)
assert parser.verbosity == 1
def test_parse_arguments_with_invalid_arguments_exits():
flexmock(os.path).should_receive('exists').and_return(True)
original_stderr = sys.stderr
sys.stderr = sys.stdout
try:
with pytest.raises(SystemExit):
module.parse_arguments(COMMAND_NAME, '--posix-me-harder')
finally:
sys.stderr = original_stderr
@@ -1,29 +0,0 @@
try:
# Python 2
from cStringIO import StringIO
except ImportError:
# Python 3
from io import StringIO
from collections import OrderedDict
import string
from atticmatic import config as module
def test_parse_section_options_with_punctuation_should_return_section_options():
parser = module.RawConfigParser()
parser.readfp(StringIO('[section]\nfoo: {}\n'.format(string.punctuation)))
section_format = module.Section_format(
'section',
(module.Config_option('foo', str, required=True),),
)
config = module.parse_section_options(parser, section_format)
assert config == OrderedDict(
(
('foo', string.punctuation),
)
)
@@ -1,8 +0,0 @@
import subprocess
def test_setup_version_matches_news_version():
setup_version = subprocess.check_output(('python', 'setup.py', '--version')).decode('ascii')
news_version = open('NEWS').readline()
assert setup_version == news_version
@@ -1,420 +0,0 @@
from collections import OrderedDict
from subprocess import STDOUT
import os
from flexmock import flexmock
from atticmatic.backends import shared as module
from atticmatic.tests.builtins import builtins_mock
from atticmatic.verbosity import VERBOSITY_SOME, VERBOSITY_LOTS
def test_initialize_with_passphrase_should_set_environment():
orig_environ = os.environ
try:
os.environ = {}
module.initialize({'encryption_passphrase': 'pass'}, command='attic')
assert os.environ.get('ATTIC_PASSPHRASE') == 'pass'
finally:
os.environ = orig_environ
def test_initialize_without_passphrase_should_not_set_environment():
orig_environ = os.environ
try:
os.environ = {}
module.initialize({}, command='attic')
assert os.environ.get('ATTIC_PASSPHRASE') == None
finally:
os.environ = orig_environ
def insert_subprocess_mock(check_call_command, **kwargs):
subprocess = flexmock(STDOUT=STDOUT)
subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
flexmock(module).subprocess = subprocess
def insert_subprocess_never():
subprocess = flexmock()
subprocess.should_receive('check_call').never()
flexmock(module).subprocess = subprocess
def insert_platform_mock():
flexmock(module.platform).should_receive('node').and_return('host')
def insert_datetime_mock():
flexmock(module).datetime = flexmock().should_receive('now').and_return(
flexmock().should_receive('isoformat').and_return('now').mock
).mock
CREATE_COMMAND_WITHOUT_EXCLUDES = ('attic', 'create', 'repo::host-now', 'foo', 'bar')
CREATE_COMMAND = CREATE_COMMAND_WITHOUT_EXCLUDES + ('--exclude-from', 'excludes')
def test_create_archive_should_call_attic_with_parameters():
insert_subprocess_mock(CREATE_COMMAND)
insert_platform_mock()
insert_datetime_mock()
module.create_archive(
excludes_filename='excludes',
verbosity=None,
storage_config={},
source_directories='foo bar',
repository='repo',
command='attic',
)
def test_create_archive_with_two_spaces_in_source_directories():
insert_subprocess_mock(CREATE_COMMAND)
insert_platform_mock()
insert_datetime_mock()
module.create_archive(
excludes_filename='excludes',
verbosity=None,
storage_config={},
source_directories='foo bar',
repository='repo',
command='attic',
)
def test_create_archive_with_none_excludes_filename_should_call_attic_without_excludes():
insert_subprocess_mock(CREATE_COMMAND_WITHOUT_EXCLUDES)
insert_platform_mock()
insert_datetime_mock()
module.create_archive(
excludes_filename=None,
verbosity=None,
storage_config={},
source_directories='foo bar',
repository='repo',
command='attic',
)
def test_create_archive_with_verbosity_some_should_call_attic_with_stats_parameter():
insert_subprocess_mock(CREATE_COMMAND + ('--stats',))
insert_platform_mock()
insert_datetime_mock()
module.create_archive(
excludes_filename='excludes',
verbosity=VERBOSITY_SOME,
storage_config={},
source_directories='foo bar',
repository='repo',
command='attic',
)
def test_create_archive_with_verbosity_lots_should_call_attic_with_verbose_parameter():
insert_subprocess_mock(CREATE_COMMAND + ('--verbose', '--stats'))
insert_platform_mock()
insert_datetime_mock()
module.create_archive(
excludes_filename='excludes',
verbosity=VERBOSITY_LOTS,
storage_config={},
source_directories='foo bar',
repository='repo',
command='attic',
)
def test_create_archive_with_compression_should_call_attic_with_compression_parameters():
insert_subprocess_mock(CREATE_COMMAND + ('--compression', 'rle'))
insert_platform_mock()
insert_datetime_mock()
module.create_archive(
excludes_filename='excludes',
verbosity=None,
storage_config={'compression': 'rle'},
source_directories='foo bar',
repository='repo',
command='attic',
)
def test_create_archive_with_one_file_system_should_call_attic_with_one_file_system_parameters():
insert_subprocess_mock(CREATE_COMMAND + ('--one-file-system',))
insert_platform_mock()
insert_datetime_mock()
module.create_archive(
excludes_filename='excludes',
verbosity=None,
storage_config={},
source_directories='foo bar',
repository='repo',
command='attic',
one_file_system=True,
)
def test_create_archive_with_umask_should_call_attic_with_umask_parameters():
insert_subprocess_mock(CREATE_COMMAND + ('--umask', '740'))
insert_platform_mock()
insert_datetime_mock()
module.create_archive(
excludes_filename='excludes',
verbosity=None,
storage_config={'umask': 740},
source_directories='foo bar',
repository='repo',
command='attic',
)
def test_create_archive_with_source_directories_glob_expands():
insert_subprocess_mock(('attic', 'create', 'repo::host-now', 'foo', 'food'))
insert_platform_mock()
insert_datetime_mock()
flexmock(module).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
module.create_archive(
excludes_filename=None,
verbosity=None,
storage_config={},
source_directories='foo*',
repository='repo',
command='attic',
)
def test_create_archive_with_non_matching_source_directories_glob_passes_through():
insert_subprocess_mock(('attic', 'create', 'repo::host-now', 'foo*'))
insert_platform_mock()
insert_datetime_mock()
flexmock(module).should_receive('glob').with_args('foo*').and_return([])
module.create_archive(
excludes_filename=None,
verbosity=None,
storage_config={},
source_directories='foo*',
repository='repo',
command='attic',
)
def test_create_archive_with_glob_should_call_attic_with_expanded_directories():
insert_subprocess_mock(('attic', 'create', 'repo::host-now', 'foo', 'food'))
insert_platform_mock()
insert_datetime_mock()
flexmock(module).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
module.create_archive(
excludes_filename=None,
verbosity=None,
storage_config={},
source_directories='foo*',
repository='repo',
command='attic',
)
BASE_PRUNE_FLAGS = (
('--keep-daily', '1'),
('--keep-weekly', '2'),
('--keep-monthly', '3'),
)
def test_make_prune_flags_should_return_flags_from_config():
retention_config = OrderedDict(
(
('keep_daily', 1),
('keep_weekly', 2),
('keep_monthly', 3),
)
)
result = module._make_prune_flags(retention_config)
assert tuple(result) == BASE_PRUNE_FLAGS
PRUNE_COMMAND = (
'attic', 'prune', 'repo', '--keep-daily', '1', '--keep-weekly', '2', '--keep-monthly', '3',
)
def test_prune_archives_should_call_attic_with_parameters():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS,
)
insert_subprocess_mock(PRUNE_COMMAND)
module.prune_archives(
verbosity=None,
repository='repo',
retention_config=retention_config,
command='attic',
)
def test_prune_archives_with_verbosity_some_should_call_attic_with_stats_parameter():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS,
)
insert_subprocess_mock(PRUNE_COMMAND + ('--stats',))
module.prune_archives(
repository='repo',
verbosity=VERBOSITY_SOME,
retention_config=retention_config,
command='attic',
)
def test_prune_archives_with_verbosity_lots_should_call_attic_with_verbose_parameter():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS,
)
insert_subprocess_mock(PRUNE_COMMAND + ('--verbose', '--stats',))
module.prune_archives(
repository='repo',
verbosity=VERBOSITY_LOTS,
retention_config=retention_config,
command='attic',
)
def test_parse_checks_returns_them_as_tuple():
checks = module._parse_checks({'checks': 'foo disabled bar'})
assert checks == ('foo', 'bar')
def test_parse_checks_with_missing_value_returns_defaults():
checks = module._parse_checks({})
assert checks == module.DEFAULT_CHECKS
def test_parse_checks_with_blank_value_returns_defaults():
checks = module._parse_checks({'checks': ''})
assert checks == module.DEFAULT_CHECKS
def test_parse_checks_with_disabled_returns_no_checks():
checks = module._parse_checks({'checks': 'disabled'})
assert checks == ()
def test_make_check_flags_with_checks_returns_flags():
flags = module._make_check_flags(('foo', 'bar'))
assert flags == ('--foo-only', '--bar-only')
def test_make_check_flags_with_default_checks_returns_no_flags():
flags = module._make_check_flags(module.DEFAULT_CHECKS)
assert flags == ()
def test_make_check_flags_with_checks_and_last_returns_flags_including_last():
flags = module._make_check_flags(('foo', 'bar'), check_last=3)
assert flags == ('--foo-only', '--bar-only', '--last', 3)
def test_make_check_flags_with_last_returns_last_flag():
flags = module._make_check_flags(module.DEFAULT_CHECKS, check_last=3)
assert flags == ('--last', 3)
def test_check_archives_should_call_attic_with_parameters():
checks = flexmock()
check_last = flexmock()
consistency_config = flexmock().should_receive('get').and_return(check_last).mock
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').with_args(checks, check_last).and_return(())
stdout = flexmock()
insert_subprocess_mock(
('attic', 'check', 'repo'),
stdout=stdout, stderr=STDOUT,
)
insert_platform_mock()
insert_datetime_mock()
builtins_mock().should_receive('open').and_return(stdout)
flexmock(module.os).should_receive('devnull')
module.check_archives(
verbosity=None,
repository='repo',
consistency_config=consistency_config,
command='attic',
)
def test_check_archives_with_verbosity_some_should_call_attic_with_verbose_parameter():
consistency_config = flexmock().should_receive('get').and_return(None).mock
flexmock(module).should_receive('_parse_checks').and_return(flexmock())
flexmock(module).should_receive('_make_check_flags').and_return(())
insert_subprocess_mock(
('attic', 'check', 'repo', '--verbose'),
stdout=None, stderr=STDOUT,
)
insert_platform_mock()
insert_datetime_mock()
module.check_archives(
verbosity=VERBOSITY_SOME,
repository='repo',
consistency_config=consistency_config,
command='attic',
)
def test_check_archives_with_verbosity_lots_should_call_attic_with_verbose_parameter():
consistency_config = flexmock().should_receive('get').and_return(None).mock
flexmock(module).should_receive('_parse_checks').and_return(flexmock())
flexmock(module).should_receive('_make_check_flags').and_return(())
insert_subprocess_mock(
('attic', 'check', 'repo', '--verbose'),
stdout=None, stderr=STDOUT,
)
insert_platform_mock()
insert_datetime_mock()
module.check_archives(
verbosity=VERBOSITY_LOTS,
repository='repo',
consistency_config=consistency_config,
command='attic',
)
def test_check_archives_without_any_checks_should_bail():
consistency_config = flexmock().should_receive('get').and_return(None).mock
flexmock(module).should_receive('_parse_checks').and_return(())
insert_subprocess_never()
module.check_archives(
verbosity=None,
repository='repo',
consistency_config=consistency_config,
command='attic',
)
-33
View File
@@ -1,33 +0,0 @@
from flexmock import flexmock
from atticmatic import command as module
def test_load_backend_with_atticmatic_command_should_return_attic_backend():
backend = flexmock()
(
flexmock(module).should_receive('import_module').with_args('atticmatic.backends.attic')
.and_return(backend).once()
)
assert module.load_backend('atticmatic') == backend
def test_load_backend_with_unknown_command_should_return_attic_backend():
backend = flexmock()
(
flexmock(module).should_receive('import_module').with_args('atticmatic.backends.attic')
.and_return(backend).once()
)
assert module.load_backend('unknownmatic') == backend
def test_load_backend_with_borgmatic_command_should_return_borg_backend():
backend = flexmock()
(
flexmock(module).should_receive('import_module').with_args('atticmatic.backends.borg')
.and_return(backend).once()
)
assert module.load_backend('borgmatic') == backend
-2
View File
@@ -1,2 +0,0 @@
VERBOSITY_SOME = 1
VERBOSITY_LOTS = 2
+124
View File
@@ -0,0 +1,124 @@
import logging
import os
import subprocess
from borgmatic.borg import extract
DEFAULT_CHECKS = ('repository', 'archives')
DEFAULT_PREFIX = '{hostname}-'
logger = logging.getLogger(__name__)
def _parse_checks(consistency_config):
'''
Given a consistency config with a "checks" list, transform it to a tuple of named checks to run.
For example, given a retention config of:
{'checks': ['repository', 'archives']}
This will be returned as:
('repository', 'archives')
If no "checks" option is present, return the DEFAULT_CHECKS. If the checks value is the string
"disabled", return an empty tuple, meaning that no checks should be run.
'''
checks = consistency_config.get('checks', [])
if checks == ['disabled']:
return ()
return (
tuple(check for check in checks if check.lower() not in ('disabled', '')) or DEFAULT_CHECKS
)
def _make_check_flags(checks, check_last=None, prefix=None):
'''
Given a parsed sequence of checks, transform it into tuple of command-line flags.
For example, given parsed checks of:
('repository',)
This will be returned as:
('--repository-only',)
However, if both "repository" and "archives" are in checks, then omit them from the returned
flags because Borg does both checks by default.
Additionally, if a check_last value is given and "archives" is in checks, then include a
"--last" flag. And if a prefix value is given and "archives" is in checks, then include a
"--prefix" flag.
'''
if 'archives' in checks:
last_flags = ('--last', str(check_last)) if check_last else ()
prefix_flags = ('--prefix', prefix) if prefix else ('--prefix', DEFAULT_PREFIX)
else:
last_flags = ()
prefix_flags = ()
if check_last:
logger.warning(
'Ignoring check_last option, as "archives" is not in consistency checks.'
)
if prefix:
logger.warning(
'Ignoring consistency prefix option, as "archives" is not in consistency checks.'
)
if set(DEFAULT_CHECKS).issubset(set(checks)):
return last_flags + prefix_flags
return (
tuple('--{}-only'.format(check) for check in checks if check in DEFAULT_CHECKS)
+ last_flags
+ prefix_flags
)
def check_archives(
repository, storage_config, consistency_config, local_path='borg', remote_path=None
):
'''
Given a local or remote repository path, a storage config dict, a consistency config dict,
and a local/remote commands to run, check the contained Borg archives for consistency.
If there are no consistency checks to run, skip running them.
'''
checks = _parse_checks(consistency_config)
check_last = consistency_config.get('check_last', None)
lock_wait = None
if set(checks).intersection(set(DEFAULT_CHECKS)):
remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
lock_wait = storage_config.get('lock_wait', None)
lock_wait_flags = ('--lock-wait', str(lock_wait)) if lock_wait else ()
verbosity_flags = ()
if logger.isEnabledFor(logging.INFO):
verbosity_flags = ('--info',)
if logger.isEnabledFor(logging.DEBUG):
verbosity_flags = ('--debug', '--show-rc')
prefix = consistency_config.get('prefix')
full_command = (
(local_path, 'check', repository)
+ _make_check_flags(checks, check_last, prefix)
+ remote_path_flags
+ lock_wait_flags
+ verbosity_flags
)
# The check command spews to stdout/stderr even without the verbose flag. Suppress it.
stdout = None if verbosity_flags else open(os.devnull, 'w')
logger.debug(' '.join(full_command))
subprocess.check_call(full_command, stdout=stdout, stderr=subprocess.STDOUT)
if 'extract' in checks:
extract.extract_last_archive_dry_run(repository, lock_wait, local_path, remote_path)
+161
View File
@@ -0,0 +1,161 @@
import glob
import itertools
import logging
import os
import tempfile
from borgmatic.borg.execute import execute_command
logger = logging.getLogger(__name__)
def _expand_directory(directory):
'''
Given a directory path, expand any tilde (representing a user's home directory) and any globs
therein. Return a list of one or more resulting paths.
'''
expanded_directory = os.path.expanduser(directory)
return glob.glob(expanded_directory) or [expanded_directory]
def _expand_directories(directories):
'''
Given a sequence of directory paths, expand tildes and globs in each one. Return all the
resulting directories as a single flattened tuple.
'''
if directories is None:
return ()
return tuple(
itertools.chain.from_iterable(_expand_directory(directory) for directory in directories)
)
def _expand_home_directories(directories):
'''
Given a sequence of directory paths, expand tildes in each one. Do not perform any globbing.
Return the results as a tuple.
'''
if directories is None:
return ()
return tuple(os.path.expanduser(directory) for directory in directories)
def _write_pattern_file(patterns=None):
'''
Given a sequence of patterns, write them to a named temporary file and return it. Return None
if no patterns are provided.
'''
if not patterns:
return None
pattern_file = tempfile.NamedTemporaryFile('w')
pattern_file.write('\n'.join(patterns))
pattern_file.flush()
return pattern_file
def _make_pattern_flags(location_config, pattern_filename=None):
'''
Given a location config dict with a potential pattern_from option, and a filename containing any
additional patterns, return the corresponding Borg flags for those files as a tuple.
'''
pattern_filenames = tuple(location_config.get('patterns_from') or ()) + (
(pattern_filename,) if pattern_filename else ()
)
return tuple(
itertools.chain.from_iterable(
('--patterns-from', pattern_filename) for pattern_filename in pattern_filenames
)
)
def _make_exclude_flags(location_config, exclude_filename=None):
'''
Given a location config dict with various exclude options, and a filename containing any exclude
patterns, return the corresponding Borg flags as a tuple.
'''
exclude_filenames = tuple(location_config.get('exclude_from') or ()) + (
(exclude_filename,) if exclude_filename else ()
)
exclude_from_flags = tuple(
itertools.chain.from_iterable(
('--exclude-from', exclude_filename) for exclude_filename in exclude_filenames
)
)
caches_flag = ('--exclude-caches',) if location_config.get('exclude_caches') else ()
if_present = location_config.get('exclude_if_present')
if_present_flags = ('--exclude-if-present', if_present) if if_present else ()
return exclude_from_flags + caches_flag + if_present_flags
def create_archive(
dry_run,
repository,
location_config,
storage_config,
local_path='borg',
remote_path=None,
progress=False,
stats=False,
json=False,
):
'''
Given vebosity/dry-run flags, a local or remote repository path, a location config dict, and a
storage config dict, create a Borg archive and return Borg's JSON output (if any).
'''
sources = _expand_directories(location_config['source_directories'])
pattern_file = _write_pattern_file(location_config.get('patterns'))
exclude_file = _write_pattern_file(
_expand_home_directories(location_config.get('exclude_patterns'))
)
checkpoint_interval = storage_config.get('checkpoint_interval', None)
chunker_params = storage_config.get('chunker_params', None)
compression = storage_config.get('compression', None)
remote_rate_limit = storage_config.get('remote_rate_limit', None)
umask = storage_config.get('umask', None)
lock_wait = storage_config.get('lock_wait', None)
files_cache = location_config.get('files_cache')
default_archive_name_format = '{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}'
archive_name_format = storage_config.get('archive_name_format', default_archive_name_format)
full_command = (
(
local_path,
'create',
'{repository}::{archive_name_format}'.format(
repository=repository, archive_name_format=archive_name_format
),
)
+ sources
+ _make_pattern_flags(location_config, pattern_file.name if pattern_file else None)
+ _make_exclude_flags(location_config, exclude_file.name if exclude_file else None)
+ (('--checkpoint-interval', str(checkpoint_interval)) if checkpoint_interval else ())
+ (('--chunker-params', chunker_params) if chunker_params else ())
+ (('--compression', compression) if compression else ())
+ (('--remote-ratelimit', str(remote_rate_limit)) if remote_rate_limit else ())
+ (('--one-file-system',) if location_config.get('one_file_system') else ())
+ (('--numeric-owner',) if location_config.get('numeric_owner') else ())
+ (('--read-special',) if location_config.get('read_special') else ())
+ (('--nobsdflags',) if location_config.get('bsd_flags') is False else ())
+ (('--files-cache', files_cache) if files_cache else ())
+ (('--remote-path', remote_path) if remote_path else ())
+ (('--umask', str(umask)) if umask else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--list', '--filter', 'AME-') if logger.isEnabledFor(logging.INFO) else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--stats',) if not dry_run and (logger.isEnabledFor(logging.INFO) or stats) else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ (('--dry-run',) if dry_run else ())
+ (('--progress',) if progress else ())
+ (('--json',) if json else ())
)
return execute_command(full_command, capture_output=json)
+15
View File
@@ -0,0 +1,15 @@
import os
def initialize(storage_config):
passcommand = storage_config.get('encryption_passcommand')
if passcommand:
os.environ['BORG_PASSCOMMAND'] = passcommand
passphrase = storage_config.get('encryption_passphrase')
if passphrase:
os.environ['BORG_PASSPHRASE'] = passphrase
ssh_command = storage_config.get('ssh_command')
if ssh_command:
os.environ['BORG_RSH'] = ssh_command
+19
View File
@@ -0,0 +1,19 @@
import logging
import subprocess
logger = logging.getLogger(__name__)
def execute_command(full_command, capture_output=False):
'''
Execute the given command (a sequence of command/argument strings). If capture output is True,
then return the command's output as a string.
'''
logger.debug(' '.join(full_command))
if capture_output:
output = subprocess.check_output(full_command)
return output.decode() if output is not None else None
else:
subprocess.check_call(full_command)
+88
View File
@@ -0,0 +1,88 @@
import logging
import sys
import subprocess
logger = logging.getLogger(__name__)
def extract_last_archive_dry_run(repository, lock_wait=None, local_path='borg', remote_path=None):
'''
Perform an extraction dry-run of the most recent archive. If there are no archives, skip the
dry-run.
'''
remote_path_flags = ('--remote-path', remote_path) if remote_path else ()
lock_wait_flags = ('--lock-wait', str(lock_wait)) if lock_wait else ()
verbosity_flags = ()
if logger.isEnabledFor(logging.DEBUG):
verbosity_flags = ('--debug', '--show-rc')
elif logger.isEnabledFor(logging.INFO):
verbosity_flags = ('--info',)
full_list_command = (
(local_path, 'list', '--short', repository)
+ remote_path_flags
+ lock_wait_flags
+ verbosity_flags
)
list_output = subprocess.check_output(full_list_command).decode(sys.stdout.encoding)
last_archive_name = list_output.strip().split('\n')[-1]
if not last_archive_name:
return
list_flag = ('--list',) if logger.isEnabledFor(logging.DEBUG) else ()
full_extract_command = (
(
local_path,
'extract',
'--dry-run',
'{repository}::{last_archive_name}'.format(
repository=repository, last_archive_name=last_archive_name
),
)
+ remote_path_flags
+ lock_wait_flags
+ verbosity_flags
+ list_flag
)
logger.debug(' '.join(full_extract_command))
subprocess.check_call(full_extract_command)
def extract_archive(
dry_run,
repository,
archive,
restore_paths,
location_config,
storage_config,
local_path='borg',
remote_path=None,
progress=False,
):
'''
Given a dry-run flag, a local or remote repository path, an archive name, zero or more paths to
restore from the archive, and location/storage configuration dicts, extract the archive into the
current directory.
'''
umask = storage_config.get('umask', None)
lock_wait = storage_config.get('lock_wait', None)
full_command = (
(local_path, 'extract', '::'.join((repository, archive)))
+ (tuple(restore_paths) if restore_paths else ())
+ (('--remote-path', remote_path) if remote_path else ())
+ (('--numeric-owner',) if location_config.get('numeric_owner') else ())
+ (('--umask', str(umask)) if umask else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug', '--list', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ (('--dry-run',) if dry_run else ())
+ (('--progress',) if progress else ())
)
logger.debug(' '.join(full_command))
subprocess.check_call(full_command)
+27
View File
@@ -0,0 +1,27 @@
import logging
from borgmatic.borg.execute import execute_command
logger = logging.getLogger(__name__)
def display_archives_info(
repository, storage_config, local_path='borg', remote_path=None, json=False
):
'''
Given a local or remote repository path, and a storage config dict, display summary information
for Borg archives in the repository or return JSON summary information.
'''
lock_wait = storage_config.get('lock_wait', None)
full_command = (
(local_path, 'info', repository)
+ (('--remote-path', remote_path) if remote_path else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ (('--json',) if json else ())
)
return execute_command(full_command, capture_output=json)
+39
View File
@@ -0,0 +1,39 @@
import logging
import subprocess
logger = logging.getLogger(__name__)
def initialize_repository(
repository,
encryption_mode,
append_only=None,
storage_quota=None,
local_path='borg',
remote_path=None,
):
'''
Given a local or remote repository path, a Borg encryption mode, whether the repository should
be append-only, and the storage quota to use, initialize the repository. If the repository
already exists, then log and skip initialization.
'''
info_command = (local_path, 'info', repository)
logger.debug(' '.join(info_command))
if subprocess.call(info_command, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) == 0:
logger.info('Repository already exists. Skipping initialization.')
return
init_command = (
(local_path, 'init', repository)
+ (('--encryption', encryption_mode) if encryption_mode else ())
+ (('--append-only',) if append_only else ())
+ (('--storage-quota', storage_quota) if storage_quota else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug',) if logger.isEnabledFor(logging.DEBUG) else ())
+ (('--remote-path', remote_path) if remote_path else ())
)
logger.debug(' '.join(init_command))
subprocess.check_call(init_command)
+28
View File
@@ -0,0 +1,28 @@
import logging
from borgmatic.borg.execute import execute_command
logger = logging.getLogger(__name__)
def list_archives(
repository, storage_config, archive=None, local_path='borg', remote_path=None, json=False
):
'''
Given a local or remote repository path and a storage config dict, display the output of listing
Borg archives in the repository or return JSON output. Or, if an archive name is given, listing
the files in that archive.
'''
lock_wait = storage_config.get('lock_wait', None)
full_command = (
(local_path, 'list', '::'.join((repository, archive)) if archive else repository)
+ (('--remote-path', remote_path) if remote_path else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ (('--json',) if json else ())
)
return execute_command(full_command, capture_output=json)
+64
View File
@@ -0,0 +1,64 @@
import logging
import subprocess
logger = logging.getLogger(__name__)
def _make_prune_flags(retention_config):
'''
Given a retention config dict mapping from option name to value, tranform it into an iterable of
command-line name-value flag pairs.
For example, given a retention config of:
{'keep_weekly': 4, 'keep_monthly': 6}
This will be returned as an iterable of:
(
('--keep-weekly', '4'),
('--keep-monthly', '6'),
)
'''
if not retention_config.get('prefix'):
retention_config['prefix'] = '{hostname}-'
return (
('--' + option_name.replace('_', '-'), str(retention_config[option_name]))
for option_name, value in retention_config.items()
)
def prune_archives(
dry_run,
repository,
storage_config,
retention_config,
local_path='borg',
remote_path=None,
stats=False,
):
'''
Given dry-run flag, a local or remote repository path, a storage config dict, and a
retention config dict, prune Borg archives according to the retention policy specified in that
configuration.
'''
umask = storage_config.get('umask', None)
lock_wait = storage_config.get('lock_wait', None)
full_command = (
(local_path, 'prune', repository)
+ tuple(element for pair in _make_prune_flags(retention_config) for element in pair)
+ (('--remote-path', remote_path) if remote_path else ())
+ (('--umask', str(umask)) if umask else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--stats',) if not dry_run and logger.isEnabledFor(logging.INFO) else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug', '--list', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ (('--dry-run',) if dry_run else ())
+ (('--stats',) if stats else ())
)
logger.debug(' '.join(full_command))
subprocess.check_call(full_command)
+487
View File
@@ -0,0 +1,487 @@
from argparse import ArgumentParser
import collections
import json
import logging
import os
from subprocess import CalledProcessError
import sys
import pkg_resources
from borgmatic.borg import (
check as borg_check,
create as borg_create,
environment as borg_environment,
prune as borg_prune,
extract as borg_extract,
list as borg_list,
info as borg_info,
init as borg_init,
)
from borgmatic.commands import hook
from borgmatic.config import checks, collect, convert, validate
from borgmatic.signals import configure_signals
from borgmatic.verbosity import verbosity_to_log_level
logger = logging.getLogger(__name__)
LEGACY_CONFIG_PATH = '/etc/borgmatic/config'
def parse_arguments(*arguments):
'''
Given command-line arguments with which this script was invoked, parse the arguments and return
them as an argparse.ArgumentParser instance.
'''
config_paths = collect.get_default_config_paths()
parser = ArgumentParser(
description='''
A simple wrapper script for the Borg backup software that creates and prunes backups.
If none of the action options are given, then borgmatic defaults to: prune, create, and
check archives.
''',
add_help=False,
)
actions_group = parser.add_argument_group('actions')
actions_group.add_argument(
'-I', '--init', dest='init', action='store_true', help='Initialize an empty Borg repository'
)
actions_group.add_argument(
'-p',
'--prune',
dest='prune',
action='store_true',
help='Prune archives according to the retention policy',
)
actions_group.add_argument(
'-C',
'--create',
dest='create',
action='store_true',
help='Create archives (actually perform backups)',
)
actions_group.add_argument(
'-k', '--check', dest='check', action='store_true', help='Check archives for consistency'
)
actions_group.add_argument(
'-x',
'--extract',
dest='extract',
action='store_true',
help='Extract a named archive to the current directory',
)
actions_group.add_argument(
'-l', '--list', dest='list', action='store_true', help='List archives'
)
actions_group.add_argument(
'-i',
'--info',
dest='info',
action='store_true',
help='Display summary information on archives',
)
init_group = parser.add_argument_group('options for --init')
init_group.add_argument(
'-e', '--encryption', dest='encryption_mode', help='Borg repository encryption mode'
)
init_group.add_argument(
'--append-only',
dest='append_only',
action='store_true',
help='Create an append-only repository',
)
init_group.add_argument(
'--storage-quota',
dest='storage_quota',
help='Create a repository with a fixed storage quota',
)
prune_group = parser.add_argument_group('options for --prune')
stats_argument = prune_group.add_argument(
'--stats',
dest='stats',
default=False,
action='store_true',
help='Display statistics of archive',
)
create_group = parser.add_argument_group('options for --create')
progress_argument = create_group.add_argument(
'--progress',
dest='progress',
default=False,
action='store_true',
help='Display progress for each file as it is processed',
)
create_group._group_actions.append(stats_argument)
json_argument = create_group.add_argument(
'--json', dest='json', default=False, action='store_true', help='Output results as JSON'
)
extract_group = parser.add_argument_group('options for --extract')
repository_argument = extract_group.add_argument(
'--repository',
help='Path of repository to use, defaults to the configured repository if there is only one',
)
archive_argument = extract_group.add_argument('--archive', help='Name of archive to operate on')
extract_group.add_argument(
'--restore-path',
nargs='+',
dest='restore_paths',
help='Paths to restore from archive, defaults to the entire archive',
)
extract_group._group_actions.append(progress_argument)
list_group = parser.add_argument_group('options for --list')
list_group._group_actions.append(repository_argument)
list_group._group_actions.append(archive_argument)
list_group._group_actions.append(json_argument)
info_group = parser.add_argument_group('options for --info')
info_group._group_actions.append(json_argument)
common_group = parser.add_argument_group('common options')
common_group.add_argument(
'-c',
'--config',
nargs='+',
dest='config_paths',
default=config_paths,
help='Configuration filenames or directories, defaults to: {}'.format(
' '.join(config_paths)
),
)
common_group.add_argument(
'--excludes',
dest='excludes_filename',
help='Deprecated in favor of exclude_patterns within configuration',
)
common_group.add_argument(
'-n',
'--dry-run',
dest='dry_run',
action='store_true',
help='Go through the motions, but do not actually write to any repositories',
)
common_group.add_argument(
'-v',
'--verbosity',
type=int,
choices=range(0, 3),
default=0,
help='Display verbose progress (1 for some, 2 for lots)',
)
common_group.add_argument(
'--version',
dest='version',
default=False,
action='store_true',
help='Display installed version number of borgmatic and exit',
)
common_group.add_argument('--help', action='help', help='Show this help information and exit')
args = parser.parse_args(arguments)
if args.excludes_filename:
raise ValueError(
'The --excludes option has been replaced with exclude_patterns in configuration'
)
if (args.encryption_mode or args.append_only or args.storage_quota) and not args.init:
raise ValueError(
'The --encryption, --append-only, and --storage-quota options can only be used with the --init option'
)
if args.init and args.dry_run:
raise ValueError('The --init option cannot be used with the --dry-run option')
if args.init and not args.encryption_mode:
raise ValueError('The --encryption option is required with the --init option')
if not args.extract:
if not args.list:
if args.repository:
raise ValueError(
'The --repository option can only be used with the --extract and --list options'
)
if args.archive:
raise ValueError(
'The --archive option can only be used with the --extract and --list options'
)
if args.restore_paths:
raise ValueError('The --restore-path option can only be used with the --extract option')
if args.extract and not args.archive:
raise ValueError('The --archive option is required with the --extract option')
if args.progress and not (args.create or args.extract):
raise ValueError(
'The --progress option can only be used with the --create and --extract options'
)
if args.json and not (args.create or args.list or args.info):
raise ValueError(
'The --json option can only be used with the --create, --list, or --info options'
)
if args.json and args.list and args.info:
raise ValueError(
'With the --json option, options --list and --info cannot be used together'
)
# If any of the action flags are explicitly requested, leave them as-is. Otherwise, assume
# defaults: Mutate the given arguments to enable the default actions.
if (
not args.init
and not args.prune
and not args.create
and not args.check
and not args.extract
and not args.list
and not args.info
):
args.prune = True
args.create = True
args.check = True
if args.stats and not (args.create or args.prune):
raise ValueError('The --stats option can only be used when creating or pruning archives')
return args
def run_configuration(config_filename, config, args): # pragma: no cover
'''
Given a config filename and the corresponding parsed config dict, execute its defined pruning,
backups, consistency checks, and/or other actions.
Yield JSON output strings from executing any actions that produce JSON.
'''
(location, storage, retention, consistency, hooks) = (
config.get(section_name, {})
for section_name in ('location', 'storage', 'retention', 'consistency', 'hooks')
)
try:
local_path = location.get('local_path', 'borg')
remote_path = location.get('remote_path')
borg_environment.initialize(storage)
if args.create:
hook.execute_hook(hooks.get('before_backup'), config_filename, 'pre-backup')
for repository_path in location['repositories']:
yield from run_actions(
args=args,
location=location,
storage=storage,
retention=retention,
consistency=consistency,
local_path=local_path,
remote_path=remote_path,
repository_path=repository_path,
)
if args.create:
hook.execute_hook(hooks.get('after_backup'), config_filename, 'post-backup')
except (OSError, CalledProcessError):
hook.execute_hook(hooks.get('on_error'), config_filename, 'on-error')
raise
def run_actions(
*, args, location, storage, retention, consistency, local_path, remote_path, repository_path
): # pragma: no cover
'''
Given parsed command-line arguments as an argparse.ArgumentParser instance, several different
configuration dicts, local and remote paths to Borg, and a repository name, run all actions
from the command-line arguments on the given repository.
Yield JSON output strings from executing any actions that produce JSON.
'''
repository = os.path.expanduser(repository_path)
dry_run_label = ' (dry run; not making any changes)' if args.dry_run else ''
if args.init:
logger.info('{}: Initializing repository'.format(repository))
borg_init.initialize_repository(
repository,
args.encryption_mode,
args.append_only,
args.storage_quota,
local_path=local_path,
remote_path=remote_path,
)
if args.prune:
logger.info('{}: Pruning archives{}'.format(repository, dry_run_label))
borg_prune.prune_archives(
args.dry_run,
repository,
storage,
retention,
local_path=local_path,
remote_path=remote_path,
stats=args.stats,
)
if args.create:
logger.info('{}: Creating archive{}'.format(repository, dry_run_label))
json_output = borg_create.create_archive(
args.dry_run,
repository,
location,
storage,
local_path=local_path,
remote_path=remote_path,
progress=args.progress,
stats=args.stats,
json=args.json,
)
if json_output:
yield json.loads(json_output)
if args.check and checks.repository_enabled_for_checks(repository, consistency):
logger.info('{}: Running consistency checks'.format(repository))
borg_check.check_archives(
repository, storage, consistency, local_path=local_path, remote_path=remote_path
)
if args.extract:
if args.repository is None or repository == args.repository:
logger.info('{}: Extracting archive {}'.format(repository, args.archive))
borg_extract.extract_archive(
args.dry_run,
repository,
args.archive,
args.restore_paths,
location,
storage,
local_path=local_path,
remote_path=remote_path,
progress=args.progress,
)
if args.list:
if args.repository is None or repository == args.repository:
logger.info('{}: Listing archives'.format(repository))
json_output = borg_list.list_archives(
repository,
storage,
args.archive,
local_path=local_path,
remote_path=remote_path,
json=args.json,
)
if json_output:
yield json.loads(json_output)
if args.info:
logger.info('{}: Displaying summary info for archives'.format(repository))
json_output = borg_info.display_archives_info(
repository, storage, local_path=local_path, remote_path=remote_path, json=args.json
)
if json_output:
yield json.loads(json_output)
def collect_configuration_run_summary_logs(config_filenames, args):
'''
Given a sequence of configuration filenames and parsed command-line arguments as an
argparse.ArgumentParser instance, run each configuration file and yield a series of
logging.LogRecord instances containing summary information about each run.
As a side effect of running through these configuration files, output their JSON results, if
any, to stdout.
'''
# Dict mapping from config filename to corresponding parsed config dict.
configs = collections.OrderedDict()
# Parse and load each configuration file.
for config_filename in config_filenames:
try:
logger.info('{}: Parsing configuration file'.format(config_filename))
configs[config_filename] = validate.parse_configuration(
config_filename, validate.schema_filename()
)
except (ValueError, OSError, validate.Validation_error) as error:
yield logging.makeLogRecord(
dict(
levelno=logging.CRITICAL,
msg='{}: Error parsing configuration file'.format(config_filename),
)
)
yield logging.makeLogRecord(dict(levelno=logging.CRITICAL, msg=error))
# Run cross-file validation checks.
if args.extract or (args.list and args.archive):
try:
validate.guard_configuration_contains_repository(args.repository, configs)
except ValueError as error:
yield logging.makeLogRecord(dict(levelno=logging.CRITICAL, msg=error))
return
# Execute the actions corresponding to each configuration file.
json_results = []
for config_filename, config in configs.items():
try:
json_results.extend(list(run_configuration(config_filename, config, args)))
yield logging.makeLogRecord(
dict(
levelno=logging.INFO,
msg='{}: Successfully ran configuration file'.format(config_filename),
)
)
except (ValueError, OSError, CalledProcessError) as error:
yield logging.makeLogRecord(
dict(
levelno=logging.CRITICAL,
msg='{}: Error running configuration file'.format(config_filename),
)
)
yield logging.makeLogRecord(dict(levelno=logging.CRITICAL, msg=error))
if json_results:
sys.stdout.write(json.dumps(json_results))
if not config_filenames:
yield logging.makeLogRecord(
dict(
levelno=logging.CRITICAL,
msg='{}: No configuration files found'.format(' '.join(args.config_paths)),
)
)
def exit_with_help_link(): # pragma: no cover
'''
Display a link to get help and exit with an error code.
'''
logger.critical('\nNeed some help? https://torsion.org/borgmatic/#issues')
sys.exit(1)
def main(): # pragma: no cover
configure_signals()
try:
args = parse_arguments(*sys.argv[1:])
except ValueError as error:
logging.basicConfig(level=logging.CRITICAL, format='%(message)s')
logger.critical(error)
exit_with_help_link()
logging.basicConfig(level=verbosity_to_log_level(args.verbosity), format='%(message)s')
if args.version:
print(pkg_resources.require('borgmatic')[0].version)
sys.exit(0)
config_filenames = tuple(collect.collect_config_filenames(args.config_paths))
logger.debug('Ensuring legacy configuration is upgraded')
convert.guard_configuration_upgraded(LEGACY_CONFIG_PATH, config_filenames)
summary_logs = tuple(collect_configuration_run_summary_logs(config_filenames, args))
logger.info('\nsummary:')
[logger.handle(log) for log in summary_logs if log.levelno >= logger.getEffectiveLevel()]
if any(log.levelno == logging.CRITICAL for log in summary_logs):
exit_with_help_link()
+109
View File
@@ -0,0 +1,109 @@
from argparse import ArgumentParser
import os
import sys
import textwrap
from ruamel import yaml
from borgmatic.config import convert, generate, legacy, validate
DEFAULT_SOURCE_CONFIG_FILENAME = '/etc/borgmatic/config'
DEFAULT_SOURCE_EXCLUDES_FILENAME = '/etc/borgmatic/excludes'
DEFAULT_DESTINATION_CONFIG_FILENAME = '/etc/borgmatic/config.yaml'
def parse_arguments(*arguments):
'''
Given command-line arguments with which this script was invoked, parse the arguments and return
them as an ArgumentParser instance.
'''
parser = ArgumentParser(
description='''
Convert legacy INI-style borgmatic configuration and excludes files to a single YAML
configuration file. Note that this replaces any comments from the source files.
'''
)
parser.add_argument(
'-s',
'--source-config',
dest='source_config_filename',
default=DEFAULT_SOURCE_CONFIG_FILENAME,
help='Source INI-style configuration filename. Default: {}'.format(
DEFAULT_SOURCE_CONFIG_FILENAME
),
)
parser.add_argument(
'-e',
'--source-excludes',
dest='source_excludes_filename',
default=DEFAULT_SOURCE_EXCLUDES_FILENAME
if os.path.exists(DEFAULT_SOURCE_EXCLUDES_FILENAME)
else None,
help='Excludes filename',
)
parser.add_argument(
'-d',
'--destination-config',
dest='destination_config_filename',
default=DEFAULT_DESTINATION_CONFIG_FILENAME,
help='Destination YAML configuration filename. Default: {}'.format(
DEFAULT_DESTINATION_CONFIG_FILENAME
),
)
return parser.parse_args(arguments)
TEXT_WRAP_CHARACTERS = 80
def display_result(args): # pragma: no cover
result_lines = textwrap.wrap(
'Your borgmatic configuration has been upgraded. Please review the result in {}.'.format(
args.destination_config_filename
),
TEXT_WRAP_CHARACTERS,
)
delete_lines = textwrap.wrap(
'Once you are satisfied, you can safely delete {}{}.'.format(
args.source_config_filename,
' and {}'.format(args.source_excludes_filename)
if args.source_excludes_filename
else '',
),
TEXT_WRAP_CHARACTERS,
)
print('\n'.join(result_lines))
print()
print('\n'.join(delete_lines))
def main(): # pragma: no cover
try:
args = parse_arguments(*sys.argv[1:])
schema = yaml.round_trip_load(open(validate.schema_filename()).read())
source_config = legacy.parse_configuration(
args.source_config_filename, legacy.CONFIG_FORMAT
)
source_config_file_mode = os.stat(args.source_config_filename).st_mode
source_excludes = (
open(args.source_excludes_filename).read().splitlines()
if args.source_excludes_filename
else []
)
destination_config = convert.convert_legacy_parsed_config(
source_config, source_excludes, schema
)
generate.write_configuration(
args.destination_config_filename, destination_config, mode=source_config_file_mode
)
display_result(args)
except (ValueError, OSError) as error:
print(error, file=sys.stderr)
sys.exit(1)
+45
View File
@@ -0,0 +1,45 @@
from argparse import ArgumentParser
import sys
from borgmatic.config import generate, validate
DEFAULT_DESTINATION_CONFIG_FILENAME = '/etc/borgmatic/config.yaml'
def parse_arguments(*arguments):
'''
Given command-line arguments with which this script was invoked, parse the arguments and return
them as an ArgumentParser instance.
'''
parser = ArgumentParser(description='Generate a sample borgmatic YAML configuration file.')
parser.add_argument(
'-d',
'--destination',
dest='destination_filename',
default=DEFAULT_DESTINATION_CONFIG_FILENAME,
help='Destination YAML configuration filename. Default: {}'.format(
DEFAULT_DESTINATION_CONFIG_FILENAME
),
)
return parser.parse_args(arguments)
def main(): # pragma: no cover
try:
args = parse_arguments(*sys.argv[1:])
generate.generate_sample_configuration(
args.destination_filename, validate.schema_filename()
)
print('Generated a sample configuration file at {}.'.format(args.destination_filename))
print()
print('Please edit the file to suit your needs. The values are representative.')
print('All fields are optional except where indicated.')
print()
print('If you ever need help: https://torsion.org/borgmatic/#issues')
except (ValueError, OSError) as error:
print(error, file=sys.stderr)
sys.exit(1)
+24
View File
@@ -0,0 +1,24 @@
import logging
import subprocess
logger = logging.getLogger(__name__)
def execute_hook(commands, config_filename, description):
if not commands:
logger.debug('{}: No commands to run for {} hook'.format(config_filename, description))
return
if len(commands) == 1:
logger.info('{}: Running command for {} hook'.format(config_filename, description))
else:
logger.info(
'{}: Running {} commands for {} hook'.format(
config_filename, len(commands), description
)
)
for command in commands:
logger.debug('{}: Hook command: {}'.format(config_filename, command))
subprocess.check_call(command, shell=True)
+9
View File
@@ -0,0 +1,9 @@
def repository_enabled_for_checks(repository, consistency):
'''
Given a repository name and a consistency configuration dict, return whether the repository
is enabled to have consistency checks run.
'''
if not consistency.get('check_repositories'):
return True
return repository in consistency['check_repositories']
+47
View File
@@ -0,0 +1,47 @@
import os
def get_default_config_paths():
'''
Based on the value of the XDG_CONFIG_HOME and HOME environment variables, return a list of
default configuration paths. This includes both system-wide configuration and configuration in
the current user's home directory.
'''
user_config_directory = os.getenv('XDG_CONFIG_HOME') or os.path.expandvars(
os.path.join('$HOME', '.config')
)
return [
'/etc/borgmatic/config.yaml',
'/etc/borgmatic.d',
'%s/borgmatic/config.yaml' % user_config_directory,
]
def collect_config_filenames(config_paths):
'''
Given a sequence of config paths, both filenames and directories, resolve that to an iterable
of files. Accomplish this by listing any given directories looking for contained config files
(ending with the ".yaml" extension). This is non-recursive, so any directories within the given
directories are ignored.
Return paths even if they don't exist on disk, so the user can find out about missing
configuration paths. However, skip a default config path if it's missing, so the user doesn't
have to create a default config path unless they need it.
'''
real_default_config_paths = set(map(os.path.realpath, get_default_config_paths()))
for path in config_paths:
exists = os.path.exists(path)
if os.path.realpath(path) in real_default_config_paths and not exists:
continue
if not os.path.isdir(path) or not exists:
yield path
continue
for filename in sorted(os.listdir(path)):
full_filename = os.path.join(path, filename)
if full_filename.endswith('.yaml') and not os.path.isdir(full_filename):
yield full_filename
+95
View File
@@ -0,0 +1,95 @@
import os
from ruamel import yaml
from borgmatic.config import generate
def _convert_section(source_section_config, section_schema):
'''
Given a legacy Parsed_config instance for a single section, convert it to its corresponding
yaml.comments.CommentedMap representation in preparation for actual serialization to YAML.
Where integer types exist in the given section schema, convert their values to integers.
'''
destination_section_config = yaml.comments.CommentedMap(
[
(
option_name,
int(option_value)
if section_schema['map'].get(option_name, {}).get('type') == 'int'
else option_value,
)
for option_name, option_value in source_section_config.items()
]
)
return destination_section_config
def convert_legacy_parsed_config(source_config, source_excludes, schema):
'''
Given a legacy Parsed_config instance loaded from an INI-style config file and a list of exclude
patterns, convert them to a corresponding yaml.comments.CommentedMap representation in
preparation for serialization to a single YAML config file.
Additionally, use the given schema as a source of helpful comments to include within the
returned CommentedMap.
'''
destination_config = yaml.comments.CommentedMap(
[
(section_name, _convert_section(section_config, schema['map'][section_name]))
for section_name, section_config in source_config._asdict().items()
]
)
# Split space-seperated values into actual lists, make "repository" into a list, and merge in
# excludes.
location = destination_config['location']
location['source_directories'] = source_config.location['source_directories'].split(' ')
location['repositories'] = [location.pop('repository')]
location['exclude_patterns'] = source_excludes
if source_config.consistency.get('checks'):
destination_config['consistency']['checks'] = source_config.consistency['checks'].split(' ')
# Add comments to each section, and then add comments to the fields in each section.
generate.add_comments_to_configuration(destination_config, schema)
for section_name, section_config in destination_config.items():
generate.add_comments_to_configuration(
section_config, schema['map'][section_name], indent=generate.INDENT
)
return destination_config
class LegacyConfigurationNotUpgraded(FileNotFoundError):
def __init__(self):
super(LegacyConfigurationNotUpgraded, self).__init__(
'''borgmatic changed its configuration file format in version 1.1.0 from INI-style
to YAML. This better supports validation, and has a more natural way to express
lists of values. To upgrade your existing configuration, run:
sudo upgrade-borgmatic-config
That will generate a new YAML configuration file at /etc/borgmatic/config.yaml
(by default) using the values from both your existing configuration and excludes
files. The new version of borgmatic will consume the YAML configuration file
instead of the old one.'''
)
def guard_configuration_upgraded(source_config_filename, destination_config_filenames):
'''
If legacy source configuration exists but no destination upgraded configs do, raise
LegacyConfigurationNotUpgraded.
The idea is that we want to alert the user about upgrading their config if they haven't already.
'''
destination_config_exists = any(
os.path.exists(filename) for filename in destination_config_filenames
)
if os.path.exists(source_config_filename) and not destination_config_exists:
raise LegacyConfigurationNotUpgraded()
+145
View File
@@ -0,0 +1,145 @@
import os
from ruamel import yaml
INDENT = 4
def _insert_newline_before_comment(config, field_name):
'''
Using some ruamel.yaml black magic, insert a blank line in the config right before the given
field and its comments.
'''
config.ca.items[field_name][1].insert(
0, yaml.tokens.CommentToken('\n', yaml.error.CommentMark(0), None)
)
def _schema_to_sample_configuration(schema, level=0):
'''
Given a loaded configuration schema, generate and return sample config for it. Include comments
for each section based on the schema "desc" description.
'''
example = schema.get('example')
if example is not None:
return example
config = yaml.comments.CommentedMap(
[
(section_name, _schema_to_sample_configuration(section_schema, level + 1))
for section_name, section_schema in schema['map'].items()
]
)
add_comments_to_configuration(config, schema, indent=(level * INDENT))
return config
def _comment_out_line(line):
# If it's already is commented out (or empty), there's nothing further to do!
stripped_line = line.lstrip()
if not stripped_line or stripped_line.startswith('#'):
return line
# Comment out the names of optional sections.
one_indent = ' ' * INDENT
if not line.startswith(one_indent):
return '#' + line
# Otherwise, comment out the line, but insert the "#" after the first indent for aesthetics.
return '#'.join((one_indent, line[INDENT:]))
REQUIRED_KEYS = {'source_directories', 'repositories', 'keep_daily'}
REQUIRED_SECTION_NAMES = {'location', 'retention'}
def _comment_out_optional_configuration(rendered_config):
'''
Post-process a rendered configuration string to comment out optional key/values. The idea is
that this prevents the user from having to comment out a bunch of configuration they don't care
about to get to a minimal viable configuration file.
Ideally ruamel.yaml would support this during configuration generation, but it's not terribly
easy to accomplish that way.
'''
lines = []
required = False
for line in rendered_config.split('\n'):
key = line.strip().split(':')[0]
if key in REQUIRED_SECTION_NAMES:
lines.append(line)
continue
# Upon encountering a required configuration option, skip commenting out lines until the
# next blank line.
if key in REQUIRED_KEYS:
required = True
elif not key:
required = False
lines.append(_comment_out_line(line) if not required else line)
return '\n'.join(lines)
def _render_configuration(config):
'''
Given a config data structure of nested OrderedDicts, render the config as YAML and return it.
'''
return yaml.round_trip_dump(config, indent=INDENT, block_seq_indent=INDENT)
def write_configuration(config_filename, rendered_config, mode=0o600):
'''
Given a target config filename and rendered config YAML, write it out to file. Create any
containing directories as needed.
'''
if os.path.exists(config_filename):
raise FileExistsError('{} already exists. Aborting.'.format(config_filename))
try:
os.makedirs(os.path.dirname(config_filename), mode=0o700)
except (FileExistsError, FileNotFoundError):
pass
with open(config_filename, 'w') as config_file:
config_file.write(rendered_config)
os.chmod(config_filename, mode)
def add_comments_to_configuration(config, schema, indent=0):
'''
Using descriptions from a schema as a source, add those descriptions as comments to the given
config before each field. This function only adds comments for the top-most config map level.
Indent the comment the given number of characters.
'''
for index, field_name in enumerate(config.keys()):
field_schema = schema['map'].get(field_name, {})
description = field_schema.get('desc')
# No description to use? Skip it.
if not field_schema or not description:
continue
config.yaml_set_comment_before_after_key(key=field_name, before=description, indent=indent)
if index > 0:
_insert_newline_before_comment(config, field_name)
def generate_sample_configuration(config_filename, schema_filename):
'''
Given a target config filename and the path to a schema filename in pykwalify YAML schema
format, write out a sample configuration file based on that schema.
'''
schema = yaml.round_trip_load(open(schema_filename))
config = _schema_to_sample_configuration(schema)
write_configuration(
config_filename, _comment_out_optional_configuration(_render_configuration(config))
)
@@ -1,11 +1,5 @@
from collections import OrderedDict, namedtuple
try:
# Python 2
from ConfigParser import RawConfigParser
except ImportError:
# Python 3
from configparser import RawConfigParser
from configparser import RawConfigParser
Section_format = namedtuple('Section_format', ('name', 'options'))
@@ -20,6 +14,42 @@ def option(name, value_type=str, required=True):
return Config_option(name, value_type, required)
CONFIG_FORMAT = (
Section_format(
'location',
(
option('source_directories'),
option('one_file_system', value_type=bool, required=False),
option('remote_path', required=False),
option('repository'),
),
),
Section_format(
'storage',
(
option('encryption_passphrase', required=False),
option('compression', required=False),
option('umask', required=False),
),
),
Section_format(
'retention',
(
option('keep_within', required=False),
option('keep_hourly', int, required=False),
option('keep_daily', int, required=False),
option('keep_weekly', int, required=False),
option('keep_monthly', int, required=False),
option('keep_yearly', int, required=False),
option('prefix', required=False),
),
),
Section_format(
'consistency', (option('checks', required=False), option('check_last', required=False))
),
)
def validate_configuration_format(parser, config_format):
'''
Given an open RawConfigParser and an expected config file format, validate that the parsed
@@ -32,7 +62,8 @@ def validate_configuration_format(parser, config_format):
'''
section_names = set(parser.sections())
required_section_names = tuple(
section.name for section in config_format
section.name
for section in config_format
if any(option.required for option in section.options)
)
@@ -46,9 +77,7 @@ def validate_configuration_format(parser, config_format):
missing_section_names = set(required_section_names) - section_names
if missing_section_names:
raise ValueError(
'Missing config sections: {}'.format(', '.join(missing_section_names))
)
raise ValueError('Missing config sections: {}'.format(', '.join(missing_section_names)))
for section_format in config_format:
if section_format.name not in section_names:
@@ -57,26 +86,28 @@ def validate_configuration_format(parser, config_format):
option_names = parser.options(section_format.name)
expected_options = section_format.options
unexpected_option_names = set(option_names) - set(option.name for option in expected_options)
unexpected_option_names = set(option_names) - set(
option.name for option in expected_options
)
if unexpected_option_names:
raise ValueError(
'Unexpected options found in config section {}: {}'.format(
section_format.name,
', '.join(sorted(unexpected_option_names)),
section_format.name, ', '.join(sorted(unexpected_option_names))
)
)
missing_option_names = tuple(
option.name for option in expected_options if option.required
option.name
for option in expected_options
if option.required
if option.name not in option_names
)
if missing_option_names:
raise ValueError(
'Required options missing from config section {}: {}'.format(
section_format.name,
', '.join(missing_option_names)
section_format.name, ', '.join(missing_option_names)
)
)
@@ -89,11 +120,7 @@ def parse_section_options(parser, section_format):
Raise ValueError if any option values cannot be coerced to the expected Python data type.
'''
type_getter = {
str: parser.get,
int: parser.getint,
bool: parser.getboolean,
}
type_getter = {str: parser.get, int: parser.getint, bool: parser.getboolean}
return OrderedDict(
(option.name, type_getter[option.value_type](section_format.name, option.name))
@@ -117,11 +144,10 @@ def parse_configuration(config_filename, config_format):
# Describes a parsed configuration, where each attribute is the name of a configuration file
# section and each value is a dict of that section's parsed options.
Parsed_config = namedtuple('Parsed_config', (section_format.name for section_format in config_format))
Parsed_config = namedtuple(
'Parsed_config', (section_format.name for section_format in config_format)
)
return Parsed_config(
*(
parse_section_options(parser, section_format)
for section_format in config_format
)
*(parse_section_options(parser, section_format) for section_format in config_format)
)
+62
View File
@@ -0,0 +1,62 @@
import os
import logging
import ruamel.yaml
logger = logging.getLogger(__name__)
def load_configuration(filename):
'''
Load the given configuration file and return its contents as a data structure of nested dicts
and lists.
Raise ruamel.yaml.error.YAMLError if something goes wrong parsing the YAML, or RecursionError
if there are too many recursive includes.
'''
yaml = ruamel.yaml.YAML(typ='safe')
yaml.Constructor = Include_constructor
return yaml.load(open(filename))
def include_configuration(loader, filename_node):
'''
Load the given YAML filename (ignoring the given loader so we can use our own), and return its
contents as a data structure of nested dicts and lists.
'''
return load_configuration(os.path.expanduser(filename_node.value))
class Include_constructor(ruamel.yaml.SafeConstructor):
'''
A YAML "constructor" (a ruamel.yaml concept) that supports a custom "!include" tag for including
separate YAML configuration files. Example syntax: `retention: !include common.yaml`
'''
def __init__(self, preserve_quotes=None, loader=None):
super(Include_constructor, self).__init__(preserve_quotes, loader)
self.add_constructor('!include', include_configuration)
def flatten_mapping(self, node):
'''
Support the special case of shallow merging included configuration into an existing mapping
using the YAML '<<' merge key. Example syntax:
```
retention:
keep_daily: 1
<<: !include common.yaml
```
'''
representer = ruamel.yaml.representer.SafeRepresenter()
for index, (key_node, value_node) in enumerate(node.value):
if key_node.tag == u'tag:yaml.org,2002:merge' and value_node.tag == '!include':
included_value = representer.represent_mapping(
tag='tag:yaml.org,2002:map', mapping=self.construct_object(value_node)
)
node.value[index] = (key_node, included_value)
super(Include_constructor, self).flatten_mapping(node)
+300
View File
@@ -0,0 +1,300 @@
name: Borgmatic configuration file schema
version: 1
map:
location:
desc: |
Where to look for files to backup, and where to store those backups. See
https://borgbackup.readthedocs.io/en/stable/quickstart.html and
https://borgbackup.readthedocs.io/en/stable/usage.html#borg-create for details.
required: true
map:
source_directories:
required: true
seq:
- type: scalar
desc: |
List of source directories to backup (required). Globs and tildes are expanded.
example:
- /home
- /etc
- /var/log/syslog*
repositories:
required: true
seq:
- type: scalar
desc: |
Paths to local or remote repositories (required). Tildes are expanded. Multiple
repositories are backed up to in sequence. See ssh_command for SSH options like
identity file or port.
example:
- user@backupserver:sourcehostname.borg
one_file_system:
type: bool
desc: Stay in same file system (do not cross mount points). Defaults to false.
example: true
numeric_owner:
type: bool
desc: Only store/extract numeric user and group identifiers. Defaults to false.
example: true
read_special:
type: bool
desc: |
Use Borg's --read-special flag to allow backup of block and other special
devices. Use with caution, as it will lead to problems if used when
backing up special devices such as /dev/zero. Defaults to false.
example: false
bsd_flags:
type: bool
desc: Record bsdflags (e.g. NODUMP, IMMUTABLE) in archive. Defaults to true.
example: true
files_cache:
type: scalar
desc: |
Mode in which to operate the files cache. See
https://borgbackup.readthedocs.io/en/stable/usage/create.html#description for
details. Defaults to "ctime,size,inode".
example: ctime,size,inode
local_path:
type: scalar
desc: Alternate Borg local executable. Defaults to "borg".
example: borg1
remote_path:
type: scalar
desc: Alternate Borg remote executable. Defaults to "borg".
example: borg1
patterns:
seq:
- type: scalar
desc: |
Any paths matching these patterns are included/excluded from backups. Globs are
expanded. (Tildes are not.) Note that Borg considers this option experimental.
See the output of "borg help patterns" for more details. Quote any value if it
contains leading punctuation, so it parses correctly.
example:
- 'R /'
- '- /home/*/.cache'
- '+ /home/susan'
- '- /home/*'
patterns_from:
seq:
- type: scalar
desc: |
Read include/exclude patterns from one or more separate named files, one pattern
per line. Note that Borg considers this option experimental. See the output of
"borg help patterns" for more details.
example:
- /etc/borgmatic/patterns
exclude_patterns:
seq:
- type: scalar
desc: |
Any paths matching these patterns are excluded from backups. Globs and tildes
are expanded. See the output of "borg help patterns" for more details.
example:
- '*.pyc'
- ~/*/.cache
- /etc/ssl
exclude_from:
seq:
- type: scalar
desc: |
Read exclude patterns from one or more separate named files, one pattern per
line. See the output of "borg help patterns" for more details.
example:
- /etc/borgmatic/excludes
exclude_caches:
type: bool
desc: |
Exclude directories that contain a CACHEDIR.TAG file. See
http://www.brynosaurus.com/cachedir/spec.html for details. Defaults to false.
example: true
exclude_if_present:
type: scalar
desc: |
Exclude directories that contain a file with the given filename. Defaults to not
set.
example: .nobackup
storage:
desc: |
Repository storage options. See
https://borgbackup.readthedocs.io/en/stable/usage.html#borg-create and
https://borgbackup.readthedocs.io/en/stable/usage/general.html#environment-variables for
details.
map:
encryption_passcommand:
type: scalar
desc: |
The standard output of this command is used to unlock the encryption key. Only
use on repositories that were initialized with passcommand/repokey encryption.
Note that if both encryption_passcommand and encryption_passphrase are set,
then encryption_passphrase takes precedence. Defaults to not set.
example: "secret-tool lookup borg-repository repo-name"
encryption_passphrase:
type: scalar
desc: |
Passphrase to unlock the encryption key with. Only use on repositories that were
initialized with passphrase/repokey encryption. Quote the value if it contains
punctuation, so it parses correctly. And backslash any quote or backslash
literals as well. Defaults to not set.
example: "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
checkpoint_interval:
type: int
desc: |
Number of seconds between each checkpoint during a long-running backup. See
https://borgbackup.readthedocs.io/en/stable/faq.html#if-a-backup-stops-mid-way-does-the-already-backed-up-data-stay-there
for details. Defaults to checkpoints every 1800 seconds (30 minutes).
example: 1800
chunker_params:
type: scalar
desc: |
Specify the parameters passed to then chunker (CHUNK_MIN_EXP, CHUNK_MAX_EXP,
HASH_MASK_BITS, HASH_WINDOW_SIZE). See https://borgbackup.readthedocs.io/en/stable/internals.html
for details. Defaults to "19,23,21,4095".
example: 19,23,21,4095
compression:
type: scalar
desc: |
Type of compression to use when creating archives. See
https://borgbackup.readthedocs.org/en/stable/usage.html#borg-create for details.
Defaults to no compression.
example: lz4
remote_rate_limit:
type: int
desc: Remote network upload rate limit in kiBytes/second. Defaults to unlimited.
example: 100
ssh_command:
type: scalar
desc: |
Command to use instead of "ssh". This can be used to specify ssh options.
Defaults to not set.
example: ssh -i /path/to/private/key
umask:
type: scalar
desc: Umask to be used for borg create. Defaults to 0077.
example: 0077
lock_wait:
type: int
desc: Maximum seconds to wait for acquiring a repository/cache lock. Defaults to 1.
example: 5
archive_name_format:
type: scalar
desc: |
Name of the archive. Borg placeholders can be used. See the output of
"borg help placeholders" for details. Defaults to
"{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}". If you specify this option, you must
also specify a prefix in the retention section to avoid accidental pruning of
archives with a different archive name format. And you should also specify a
prefix in the consistency section as well.
example: "{hostname}-documents-{now}"
retention:
desc: |
Retention policy for how many backups to keep in each category. See
https://borgbackup.readthedocs.org/en/stable/usage.html#borg-prune for details.
At least one of the "keep" options is required for pruning to work.
map:
keep_within:
type: scalar
desc: Keep all archives within this time interval.
example: 3H
keep_secondly:
type: int
desc: Number of secondly archives to keep.
example: 60
keep_minutely:
type: int
desc: Number of minutely archives to keep.
example: 60
keep_hourly:
type: int
desc: Number of hourly archives to keep.
example: 24
keep_daily:
type: int
desc: Number of daily archives to keep.
example: 7
keep_weekly:
type: int
desc: Number of weekly archives to keep.
example: 4
keep_monthly:
type: int
desc: Number of monthly archives to keep.
example: 6
keep_yearly:
type: int
desc: Number of yearly archives to keep.
example: 1
prefix:
type: scalar
desc: |
When pruning, only consider archive names starting with this prefix.
Borg placeholders can be used. See the output of "borg help placeholders" for
details. Defaults to "{hostname}-".
example: sourcehostname
consistency:
desc: |
Consistency checks to run after backups. See
https://borgbackup.readthedocs.org/en/stable/usage.html#borg-check and
https://borgbackup.readthedocs.org/en/stable/usage.html#borg-extract for details.
map:
checks:
seq:
- type: str
enum: ['repository', 'archives', 'extract', 'disabled']
unique: true
desc: |
List of one or more consistency checks to run: "repository", "archives", and/or
"extract". Defaults to "repository" and "archives". Set to "disabled" to disable
all consistency checks. "repository" checks the consistency of the repository,
"archive" checks all of the archives, and "extract" does an extraction dry-run
of the most recent archive.
example:
- repository
- archives
check_repositories:
seq:
- type: scalar
desc: |
Paths to a subset of the repositories in the location section on which to run
consistency checks. Handy in case some of your repositories are very large, and
so running consistency checks on them would take too long. Defaults to running
consistency checks on all repositories configured in the location section.
example:
- user@backupserver:sourcehostname.borg
check_last:
type: int
desc: Restrict the number of checked archives to the last n. Applies only to the
"archives" check. Defaults to checking all archives.
example: 3
prefix:
type: scalar
desc: |
When performing the "archives" check, only consider archive names starting with
this prefix. Borg placeholders can be used. See the output of
"borg help placeholders" for details. Defaults to "{hostname}-".
example: sourcehostname
hooks:
desc: |
Shell commands or scripts to execute before and after a backup or if an error has occurred.
IMPORTANT: All provided commands and scripts are executed with user permissions of borgmatic.
Do not forget to set secure permissions on this file as well as on any script listed (chmod 0700) to
prevent potential shell injection or privilege escalation.
map:
before_backup:
seq:
- type: scalar
desc: List of one or more shell commands or scripts to execute before creating a backup.
example:
- echo "Starting a backup job."
after_backup:
seq:
- type: scalar
desc: List of one or more shell commands or scripts to execute after creating a backup.
example:
- echo "Backup created."
on_error:
seq:
- type: scalar
desc: List of one or more shell commands or scripts to execute in case an exception has occurred.
example:
- echo "Error while creating a backup."
+154
View File
@@ -0,0 +1,154 @@
import logging
import pkg_resources
import pykwalify.core
import pykwalify.errors
import ruamel.yaml
from borgmatic.config import load
logger = logging.getLogger(__name__)
def schema_filename():
'''
Path to the installed YAML configuration schema file, used to validate and parse the
configuration.
'''
return pkg_resources.resource_filename('borgmatic', 'config/schema.yaml')
class Validation_error(ValueError):
'''
A collection of error message strings generated when attempting to validate a particular
configurartion file.
'''
def __init__(self, config_filename, error_messages):
self.config_filename = config_filename
self.error_messages = error_messages
def __str__(self):
'''
Render a validation error as a user-facing string.
'''
return 'An error occurred while parsing a configuration file at {}:\n'.format(
self.config_filename
) + '\n'.join(self.error_messages)
def apply_logical_validation(config_filename, parsed_configuration):
'''
Given a parsed and schematically valid configuration as a data structure of nested dicts (see
below), run through any additional logical validation checks. If there are any such validation
problems, raise a Validation_error.
'''
archive_name_format = parsed_configuration.get('storage', {}).get('archive_name_format')
prefix = parsed_configuration.get('retention', {}).get('prefix')
if archive_name_format and not prefix:
raise Validation_error(
config_filename,
('If you provide an archive_name_format, you must also specify a retention prefix.',),
)
location_repositories = parsed_configuration.get('location', {}).get('repositories')
check_repositories = parsed_configuration.get('consistency', {}).get('check_repositories', [])
for repository in check_repositories:
if repository not in location_repositories:
raise Validation_error(
config_filename,
(
'Unknown repository in the consistency section\'s check_repositories: {}'.format(
repository
),
),
)
consistency_prefix = parsed_configuration.get('consistency', {}).get('prefix')
if archive_name_format and not consistency_prefix:
logger.warning(
'Since version 1.1.16, if you provide `archive_name_format`, you should also'
' specify `consistency.prefix`.'
)
def parse_configuration(config_filename, schema_filename):
'''
Given the path to a config filename in YAML format and the path to a schema filename in
pykwalify YAML schema format, return the parsed configuration as a data structure of nested
dicts and lists corresponding to the schema. Example return value:
{'location': {'source_directories': ['/home', '/etc'], 'repository': 'hostname.borg'},
'retention': {'keep_daily': 7}, 'consistency': {'checks': ['repository', 'archives']}}
Raise FileNotFoundError if the file does not exist, PermissionError if the user does not
have permissions to read the file, or Validation_error if the config does not match the schema.
'''
logging.getLogger('pykwalify').setLevel(logging.ERROR)
try:
config = load.load_configuration(config_filename)
schema = load.load_configuration(schema_filename)
except (ruamel.yaml.error.YAMLError, RecursionError) as error:
raise Validation_error(config_filename, (str(error),))
# pykwalify gets angry if the example field is not a string. So rather than bend to its will,
# remove all examples before passing the schema to pykwalify.
for section_name, section_schema in schema['map'].items():
for field_name, field_schema in section_schema['map'].items():
field_schema.pop('example', None)
validator = pykwalify.core.Core(source_data=config, schema_data=schema)
parsed_result = validator.validate(raise_exception=False)
if validator.validation_errors:
raise Validation_error(config_filename, validator.validation_errors)
apply_logical_validation(config_filename, parsed_result)
return parsed_result
def guard_configuration_contains_repository(repository, configurations):
'''
Given a repository path and a dict mapping from config filename to corresponding parsed config
dict, ensure that the repository is declared exactly once in all of the configurations.
If no repository is given, then error if there are multiple configured repositories.
Raise ValueError if the repository is not found in a configuration, or is declared multiple
times.
'''
if not repository:
count = len(
tuple(
config_repository
for config in configurations.values()
for config_repository in config['location']['repositories']
)
)
if count > 1:
raise ValueError(
'Can\'t determine which repository to use. Use --repository option to disambiguate'.format(
repository
)
)
return
count = len(
tuple(
config_repository
for config in configurations.values()
for config_repository in config['location']['repositories']
if repository == config_repository
)
)
if count == 0:
raise ValueError('Repository {} not found in configuration files'.format(repository))
if count > 1:
raise ValueError('Repository {} found in multiple configuration files'.format(repository))
+18
View File
@@ -0,0 +1,18 @@
import os
import signal
def _handle_signal(signal_number, frame): # pragma: no cover
'''
Send the signal to all processes in borgmatic's process group, which includes child process.
'''
os.killpg(os.getpgrp(), signal_number)
def configure_signals(): # pragma: no cover
'''
Configure borgmatic's signal handlers to pass relevant signals through to any child processes
like Borg. Note that SIGINT gets passed through even without these changes.
'''
for signal_number in (signal.SIGHUP, signal.SIGTERM, signal.SIGUSR1, signal.SIGUSR2):
signal.signal(signal_number, _handle_signal)
+17
View File
@@ -0,0 +1,17 @@
import logging
VERBOSITY_WARNING = 0
VERBOSITY_SOME = 1
VERBOSITY_LOTS = 2
def verbosity_to_log_level(verbosity):
'''
Given a borgmatic verbosity value, return the corresponding Python log level.
'''
return {
VERBOSITY_WARNING: logging.WARNING,
VERBOSITY_SOME: logging.INFO,
VERBOSITY_LOTS: logging.DEBUG,
}.get(verbosity, logging.WARNING)
@@ -0,0 +1,96 @@
---
title: How to deal with very large backups
---
## Biggish data
Borg itself is great for efficiently de-duplicating data across successive
backup archives, even when dealing with very large repositories. But you may
find that while borgmatic's default mode of "prune, create, and check" works
well on small repositories, it's not so great on larger ones. That's because
running the default consistency checks takes a long time on large
repositories.
### A la carte actions
If you find yourself in this situation, you have some options. First, you can
run borgmatic's pruning, creating, or checking actions separately. For
instance, the the following optional flags are available:
```bash
borgmatic --prune
borgmatic --create
borgmatic --check
```
You can run with only one of these flags provided, or you can mix and match
any number of them in a single borgmatic run. This supports approaches like
making backups with `--create` on a frequent schedule, while only running
expensive consistency checks with `--check` on a much less frequent basis from
a separate cron job.
### Consistency check configuration
Another option is to customize your consistency checks. The default
consistency checks run both full-repository checks and per-archive checks
within each repository.
But if you find that archive checks are too slow, for example, you can
configure borgmatic to run repository checks only. Configure this in the
`consistency` section of borgmatic configuration:
```yaml
consistency:
checks:
- repository
```
If that's still too slow, you can disable consistency checks entirely,
either for a single repository or for all repositories.
Disabling all consistency checks looks like this:
```yaml
consistency:
checks:
- disabled
```
Or, if you have multiple repositories in your borgmatic configuration file,
you can keep running consistency checks, but only against a subset of the
repositories:
```yaml
consistency:
check_repositories:
- path/of/repository_to_check.borg
```
## Troubleshooting
### Broken pipe with remote repository
When running borgmatic on a large remote repository, you may receive errors
like the following, particularly while "borg check" is validating backups for
consistency:
```text
Write failed: Broken pipe
borg: Error: Connection closed by remote host
```
This error can be caused by an ssh timeout, which you can rectify by adding
the following to the `~/.ssh/config` file on the client:
```text
Host *
ServerAliveInterval 120
```
This should make the client keep the connection alive while validating
backups.
## Related documentation
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
+97
View File
@@ -0,0 +1,97 @@
---
title: How to develop on borgmatic
---
## Source code
To get set up to hack on borgmatic, first clone master via HTTPS or SSH:
```bash
git clone https://projects.torsion.org/witten/borgmatic.git
```
Or:
```bash
git clone ssh://git@projects.torsion.org:3022/witten/borgmatic.git
```
Then, install borgmatic
"[editable](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs)"
so that you can run borgmatic commands while you're hacking on them to
make sure your changes work.
```bash
cd borgmatic/
pip3 install --editable --user .
```
Note that this will typically install the borgmatic commands into
`~/.local/bin`, which may or may not be on your PATH. There are other ways to
install borgmatic editable as well, for instance into the system Python
install (so without `--user`, as root), or even into a
[virtualenv](https://virtualenv.pypa.io/en/stable/). How or where you install
borgmatic is up to you, but generally an editable install makes development
and testing easier.
## Automated tests
Assuming you've cloned the borgmatic source code as described above, and
you're in the `borgmatic/` working copy, install tox, which is used for
setting up testing environments:
```bash
sudo pip3 install tox
```
Finally, to actually run tests, run:
```bash
cd borgmatic
tox
```
### Code formatting
If when running tests, you get an error from the
[Black](https://black.readthedocs.io/en/stable/) code formatter about files
that would be reformatted, you can ask Black to format them for you via the
following:
```bash
tox -e black
```
### End-to-end tests
borgmatic additionally includes some end-to-end tests that integration test
with Borg for a few representative scenarios. These tests don't run by default
because they're relatively slow and depend on Borg. If you would like to run
them:
```bash
tox -e end-to-end
```
## Code style
Start with [PEP 8](https://www.python.org/dev/peps/pep-0008/). But then, apply
the following deviations from it:
* For strings, prefer single quotes over double quotes.
* Limit all lines to a maximum of 100 characters.
* Use trailing commas within multiline values or argument lists.
* For multiline constructs, put opening and closing delimeters on lines
separate from their contents.
* Within multiline constructs, use standard four-space indentation. Don't align
indentation with an opening delimeter.
borgmatic code uses the [Black](https://black.readthedocs.io/en/stable/) code
formatter and [Flake8](http://flake8.pycqa.org/en/latest/) code checker, so
certain code style requirements will be enforced when running automated tests.
See the Black and Flake8 documentation for more information.
## Related documentation
* [Inspect your backups](../../docs/how-to/inspect-your-backups.md)
+46
View File
@@ -0,0 +1,46 @@
---
title: How to inspect your backups
---
## Backup progress
By default, borgmatic runs proceed silently except in the case of errors. But
if you'd like to to get additional information about the progress of the
backup as it proceeds, use the verbosity option:
```bash
borgmatic --verbosity 1
```
This lists the files that borgmatic is archiving, which are those that are new
or changed since the last backup.
Or, for even more progress and debug spew:
```bash
borgmatic --verbosity 2
```
## Existing backups
Borgmatic provides convenient flags for Borg's
[list](https://borgbackup.readthedocs.io/en/stable/usage/list.html) and
[info](https://borgbackup.readthedocs.io/en/stable/usage/info.html)
functionality:
```bash
borgmatic --list
borgmatic --info
```
## Scripting borgmatic
To consume the output of borgmatic in other software, you can include an
optional `--json` flag with `--create`, `--list`, or `--info` to get the
output formatted as JSON.
## Related documentation
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
* [Develop on borgmatic](../../docs/how-to/develop-on-borgmatic.md)
+111
View File
@@ -0,0 +1,111 @@
---
title: How to make per-application backups
---
## Multiple backup configurations
You may find yourself wanting to create different backup policies for
different applications on your system. For instance, you may want one backup
configuration for your database data directory, and a different configuration
for your user home directories.
The way to accomplish that is pretty simple: Create multiple separate
configuration files and place each one in a `/etc/borgmatic.d/` directory. For
instance:
```bash
sudo mkdir /etc/borgmatic.d
sudo generate-borgmatic-config --destination /etc/borgmatic.d/app1.yaml
sudo generate-borgmatic-config --destination /etc/borgmatic.d/app2.yaml
```
When you set up multiple configuration files like this, borgmatic will run
each one in turn from a single borgmatic invocation. This includes, by
default, the traditional `/etc/borgmatic/config.yaml` as well.
And if you need even more customizability, you can specify alternate
configuration paths on the command-line with borgmatic's `--config` option.
See `borgmatic --help` for more information.
## Configuration includes
Once you have multiple different configuration files, you might want to share
common configuration options across these files with having to copy and paste
them. To achieve this, you can put fragments of common configuration options
into a file, and then include or inline that file into one or more borgmatic
configuration files.
Let's say that you want to include common retention configuration across all
of your configuration files. You could do that in each configuration file with
the following:
```yaml
location:
...
retention:
!include /etc/borgmatic/common_retention.yaml
```
And then the contents of `common_retention.yaml` could be:
```yaml
keep_hourly: 24
keep_daily: 7
```
To prevent borgmatic from trying to load these configuration fragments by
themselves and complaining that they are not valid configuration files, you
should put them in a directory other than `/etc/borgmatic.d/`. (A subdirectory
is fine.)
Note that this form of include must be a YAML value rather than a key. For
example, this will not work:
```yaml
location:
...
# Don't do this. It won't work!
!include /etc/borgmatic/common_retention.yaml
```
But if you do want to merge in a YAML key and its values, keep reading!
## Include merging
If you need to get even fancier and pull in common configuration options while
potentially overriding individual options, you can perform a YAML merge of
included configuration using the YAML `<<` key. For instance, here's an
example of a main configuration file that pulls in two retention options via
an include, and then overrides one of them locally:
```yaml
location:
...
retention:
keep_daily: 5
<<: !include /etc/borgmatic/common_retention.yaml
```
This is what `common_retention.yaml` might look like:
```yaml
keep_hourly: 24
keep_daily: 7
```
Once this include gets merged in, the resulting configuration would have a
`keep_hourly` value of `24` and an overridden `keep_daily` value of `5`.
When there is a collision of an option between the local file and the merged
include, the local file's option takes precedent. And note that this is a
shallow merge rather than a deep merge, so the merging does not descend into
nested values.
## Related documentation
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
+61
View File
@@ -0,0 +1,61 @@
---
title: How to restore a backup
---
## Extract
When the worst happens—or you want to test your backups—the first step is
to figure out which archive to restore. A good way to do that is to use the
`--list` action:
```bash
borgmatic --list
```
That should yield output looking something like:
```text
host-2019-01-01T04:05:06.070809 Tue, 2019-01-01 04:05:06 [...]
host-2019-01-02T04:06:07.080910 Wed, 2019-01-02 04:06:07 [...]
```
Assuming that you want to restore the archive with the most up-to-date files
and therefore the latest timestamp, run a command like:
```bash
borgmatic --extract --archive host-2019-01-02T04:06:07.080910
```
The `--archive` value is the name of the archive to restore. This extracts the
entire contents of the archive to the current directory, so make sure you're
in the right place before running the command.
## Repository selection
If you have a single repository in your borgmatic configuration file(s), no
problem: the `--extract` action figures out which repository to use.
But if you have multiple repositories configured, then you'll need to specify
the repository path containing the archive to extract. Here's an example:
```bash
borgmatic --extract --repository repo.borg --archive host-2019-...
```
## Restore particular files
Sometimes, you want to restore a single deleted file, rather than restoring
everything from an archive. To do that, tack on one or more `--restore-path`
values. For instance:
```bash
borgmatic --extract --archive host-2019-... --restore-path /path/1 /path/2
```
Like a whole-archive restore, this also restores into the current directory.
## Related documentation
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
* [Inspect your backups](../../docs/how-to/inspect-your-backups.md)
@@ -0,0 +1,51 @@
---
title: How to run preparation steps before backups
---
## Preparation and cleanup hooks
If you find yourself performing prepraration tasks before your backup runs, or
cleanup work afterwards, borgmatic hooks may be of interest. Hooks are
shell commands that borgmatic executes for you at various points, and they're
configured in the `hooks` section of your configuration file.
For instance, you can specify `before_backup` hooks to dump a database to file
before backing it up, and specify `after_backup` hooks to delete the temporary
file afterwards. Here's an example:
```yaml
hooks:
before_backup:
- dump-a-database /to/file.sql
after_backup:
- rm /to/file.sql
```
borgmatic hooks run once per configuration file. `before_backup` hooks run
prior to backups of all repositories. `after_backup` hooks run afterwards, but
not if an error occurs in a previous hook or in the backups themselves.
## Error hooks
borgmatic also runs `on_error` hooks if an error occurs. Here's an example
configuration:
```yaml
hooks:
on_error:
- echo "Error while creating a backup."
```
## Security
An important security note about hooks: borgmatic executes all hook commands
with the user permissions of borgmatic itself. So to prevent potential shell
injection or privilege escalation, do not forget to set secure permissions
(`chmod 0700`) on borgmatic configuration files and scripts invoked by hooks.
## Related documentation
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
* [Make per-application backups](../../docs/how-to/make-per-application-backups.md)
+188
View File
@@ -0,0 +1,188 @@
---
title: How to set up backups with borgmatic
---
## Installation
To get up and running, first [install
Borg](https://borgbackup.readthedocs.io/en/latest/installation.html), at
least version 1.1.
Then, run the following command to download and install borgmatic:
```bash
sudo pip3 install --upgrade borgmatic
```
Note that your pip binary may have a different name than "pip3". Make sure
you're using Python 3, as borgmatic does not support Python 2.
### Other ways to install
Along with the above process, you have several other options for installing
borgmatic:
* [Docker base image](https://hub.docker.com/r/monachus/borgmatic/)
* [Docker image with support for scheduled backups](https://hub.docker.com/r/b3vis/borgmatic/)
* [Debian](https://tracker.debian.org/pkg/borgmatic)
* [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
* [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
* [Arch Linux](https://aur.archlinux.org/packages/borgmatic/)
* [OpenBSD](http://ports.su/sysutils/borgmatic)
* [openSUSE](https://software.opensuse.org/package/borgmatic)
## Configuration
After you install borgmatic, generate a sample configuration file:
```bash
sudo generate-borgmatic-config
```
If that command is not found, then it may be installed in a location that's
not in your system `PATH`. Try looking in `/usr/local/bin/`.
This generates a sample configuration file at /etc/borgmatic/config.yaml (by
default). You should edit the file to suit your needs, as the values are
representative. All options are optional except where indicated, so feel free
to ignore anything you don't need.
Note that the configuration file is organized into distinct sections, each
with a section name like `location:` or `storage:`. So take care that if you
uncomment a particular option, also uncomment its containing section name, or
else borgmatic won't recognize the option.
You can also get the same sample configuration file from the [configuration
reference](../../docs/reference/configuration.md), the authoritative set of
all configuration options. This is handy if borgmatic has added new options
since you originally created your configuration file.
### Encryption
Note that if you plan to run borgmatic on a schedule with cron, and you
encrypt your Borg repository with a passphrase instead of a key file, you'll
either need to set the borgmatic `encryption_passphrase` configuration
variable or set the `BORG_PASSPHRASE` environment variable. See the
[repository encryption
section](https://borgbackup.readthedocs.io/en/latest/quickstart.html#repository-encryption)
of the Borg Quick Start for more info.
Alternatively, you can specify the passphrase programatically by setting
either the borgmatic `encryption_passcommand` configuration variable or the
`BORG_PASSCOMMAND` environment variable. See the [Borg Security
FAQ](http://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically)
for more info.
###
## Initialization
Before you can create backups with borgmatic, you first need to initialize a
Borg repository so you have a destination for your backup archives. (But skip
this step if you already have a Borg repository.) To create a repository, run
a command like the following:
```bash
borgmatic --init --encryption repokey
```
This uses the borgmatic configuration file you created above to determine
which local or remote repository to create, and encrypts it with the
encryption passphrase specified there if one is provided. Read about [Borg
encryption
modes](https://borgbackup.readthedocs.io/en/latest/usage/init.html#encryption-modes)
for the menu of available encryption modes.
Also, optionally check out the [Borg Quick
Start](https://borgbackup.readthedocs.org/en/latest/quickstart.html) for more
background about repository initialization.
Note that borgmatic skips repository initialization if the repository already
exists. This supports use cases like ensuring a repository exists prior to
performing a backup.
If the repository is on a remote host, make sure that your local user has
key-based SSH access to the desired user account on the remote host.
## Backups
Now that you've configured borgmatic and initialized a repository, it's a
good idea to test that borgmatic is working. So to run borgmatic and start a
backup, you can invoke it like this:
```bash
borgmatic --verbosity 1
```
By default, this will also prune any old backups as per the configured
retention policy, and check backups for consistency problems due to things
like file damage.
The verbosity flag makes borgmatic list the files that it's archiving, which
are those that are new or changed since the last backup. Eyeball the list and
see if it matches your expectations based on the configuration.
## Autopilot
Running backups manually is good for validating your configuration, but I'm
guessing that you want to run borgmatic automatically, say once a day. To do
that, you can configure a separate job runner to invoke it periodically.
### cron
If you're using cron, download the [sample cron
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/cron/borgmatic).
Then, from the directory where you downloaded it:
```bash
sudo mv borgmatic /etc/cron.d/borgmatic
sudo chmod +x /etc/cron.d/borgmatic
```
You can modify the cron file if you'd like to run borgmatic more or less frequently.
### systemd
If you're using systemd instead of cron to run jobs, download the [sample
systemd service
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.service)
and the [sample systemd timer
file](https://projects.torsion.org/witten/borgmatic/src/master/sample/systemd/borgmatic.timer).
Then, from the directory where you downloaded them:
```bash
sudo mv borgmatic.service borgmatic.timer /etc/systemd/system/
sudo systemctl enable borgmatic.timer
sudo systemctl start borgmatic.timer
```
Feel free to modify the timer file based on how frequently you'd like
borgmatic to run.
## Troubleshooting
### libyaml compilation errors
borgmatic depends on a Python YAML library (ruamel.yaml) that will optionally
use a C YAML library (libyaml) if present. But if it's not installed, then
when installing or upgrading borgmatic, you may see errors about compiling the
YAML library. If so, not to worry. borgmatic should install and function
correctly even without the C YAML library. And borgmatic won't be any faster
with the C library present, so you don't need to go out of your way to install
it.
## Related documentation
* [Make per-application backups](../../docs/how-to/make-per-application-backups.md)
* [Deal with very large backups](../../docs/how-to/deal-with-very-large-backups.md)
* [Inspect your backups](../../docs/how-to/inspect-your-backups.md)
* [borgmatic configuration reference](../../docs/reference/configuration.md)
* [borgmatic command-line reference](../../docs/reference/command-line.md)
+79
View File
@@ -0,0 +1,79 @@
---
title: How to upgrade borgmatic
---
## Upgrading
In general, all you should need to do to upgrade borgmatic is run the
following:
```bash
sudo pip3 install --upgrade borgmatic
```
See below about special cases.
### Upgrading from borgmatic 1.0.x
borgmatic changed its configuration file format in version 1.1.0 from
INI-style to YAML. This better supports validation, and has a more natural way
to express lists of values. To upgrade your existing configuration, first
upgrade to the new version of borgmatic.
As of version 1.1.0, borgmatic no longer supports Python 2. If you were
already running borgmatic with Python 3, then you can upgrade borgmatic
in-place:
```bash
sudo pip3 install --upgrade borgmatic
```
But if you were running borgmatic with Python 2, uninstall and reinstall instead:
```bash
sudo pip uninstall borgmatic
sudo pip3 install borgmatic
```
The pip binary names for different versions of Python can differ, so the above
commands may need some tweaking to work on your machine.
Once borgmatic is upgraded, run:
```bash
sudo upgrade-borgmatic-config
```
That will generate a new YAML configuration file at /etc/borgmatic/config.yaml
(by default) using the values from both your existing configuration and
excludes files. The new version of borgmatic will consume the YAML
configuration file instead of the old one.
### Upgrading from atticmatic
You can ignore this section if you're not an atticmatic user (the former name
of borgmatic).
borgmatic only supports Borg now and no longer supports Attic. So if you're
an Attic user, consider switching to Borg. See the [Borg upgrade
command](https://borgbackup.readthedocs.io/en/stable/usage.html#borg-upgrade)
for more information. Then, follow the instructions above about setting up
your borgmatic configuration files.
If you were already using Borg with atticmatic, then you can upgrade
from atticmatic to borgmatic by running the following commands:
```bash
sudo pip3 uninstall atticmatic
sudo pip3 install borgmatic
```
That's it! borgmatic will continue using your /etc/borgmatic configuration
files.
## Related documentation
* [Develop on borgmatic](../../docs/how-to/develop-on-borgmatic.md)
+16
View File
@@ -0,0 +1,16 @@
---
title: borgmatic command-line reference
---
## borgmatic options
Here are all of the available borgmatic command-line options:
```
{% include borgmatic/command-line.txt %}
```
## Related documentation
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
* [borgmatic configuration reference](../../docs/reference/configuration.md)
+19
View File
@@ -0,0 +1,19 @@
---
title: borgmatic configuration reference
---
## Configuration file
Here is a full sample borgmatic configuration file including all available options:
```yaml
{% include borgmatic/config.yaml %}
```
Note that you can also [download this configuration
file](../../docs/reference/config.yaml) for use locally.
## Related documentation
* [Set up backups with borgmatic](../../docs/how-to/set-up-backups.md)
* [borgmatic command-line reference](../../docs/reference/command-line.md)
-3
View File
@@ -1,3 +0,0 @@
# You can drop this file into /etc/cron.d/ to run atticmatic nightly.
0 3 * * * root PATH=$PATH:/usr/local/bin /usr/local/bin/atticmatic
-43
View File
@@ -1,43 +0,0 @@
[location]
# Space-separated list of source directories to backup.
# Globs are expanded.
source_directories: /home /etc /var/log/syslog*
# For Borg only, you can specify to stay in same file system (do not cross
# mount points).
#one_file_system: True
# Path to local or remote repository.
repository: user@backupserver:sourcehostname.attic
[storage]
# Passphrase to unlock the encryption key with. Only use on repositories that
# were initialized with passphrase/repokey encryption.
#encryption_passphrase: foo
# For Borg only, you can specify the type of compression to use when creating
# archives. See https://borgbackup.readthedocs.org/en/stable/usage.html#borg-create
# for details. Defaults to no compression.
#compression: lz4
# For Borg only, you can specify the umask to be used for borg create.
#umask: 0740
[retention]
# Retention policy for how many backups to keep in each category. See
# https://attic-backup.org/usage.html#attic-prune or
# https://borgbackup.readthedocs.org/en/stable/usage.html#borg-prune for details.
#keep_within: 3H
#keep_hourly: 24
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
keep_yearly: 1
#prefix: sourcehostname
[consistency]
# Space-separated list of consistency checks to run: "repository", "archives",
# or both. Defaults to both. Set to "disabled" to disable all consistency
# checks. See https://attic-backup.org/usage.html#attic-check or
# https://borgbackup.readthedocs.org/en/stable/usage.html#borg-check for details.
checks: repository archives
# For Borg only, you can restrict the number of checked archives to the last n.
#check_last: 3
+3
View File
@@ -0,0 +1,3 @@
# You can drop this file into /etc/cron.d/ to run borgmatic nightly.
0 3 * * * PATH=$PATH:/usr/bin /usr/bin/borgmatic
-3
View File
@@ -1,3 +0,0 @@
*.pyc
/home/*/.cache
/etc/ssl
+6
View File
@@ -0,0 +1,6 @@
[Unit]
Description=borgmatic backup
[Service]
Type=oneshot
ExecStart=/usr/local/bin/borgmatic
+9
View File
@@ -0,0 +1,9 @@
[Unit]
Description=Run borgmatic backup
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
Executable
+7
View File
@@ -0,0 +1,7 @@
#!/usr/bin/env sh
if which black; then
black --skip-string-normalization --line-length 100 --check .
else
echo "Skipping black due to not being installed."
fi
+63
View File
@@ -0,0 +1,63 @@
#!/bin/bash
set -o nounset
# For each Borg sub-command that borgmatic uses, print out the Borg flags that borgmatic does not
# appear to support yet. This script isn't terribly robust. It's intended as a basic tool to ferret
# out unsupported Borg options so that they can be considered for addition to borgmatic.
# Generate a sample borgmatic configuration with all options set, and uncomment all options.
generate-borgmatic-config --destination temp.yaml
cat temp.yaml | sed -e 's/# \S.*$//' | sed -e 's/#//' > temp.yaml.uncommented
mv temp.yaml.uncommented temp.yaml
# For each sub-command (prune, create, and check), collect the Borg command-line flags that result
# from running borgmatic with the generated configuration. Then, collect the full set of available
# Borg flags as reported by "borg --help" for that sub-command. Finally, compare the two lists of
# flags to determine which Borg flags borgmatic doesn't yet support.
for sub_command in prune create check list info; do
echo "********** borg $sub_command **********"
for line in $(borgmatic --config temp.yaml --$sub_command -v 2 2>&1 | grep "borg\w* $sub_command") ; do
echo "$line" | grep '^-' >> borgmatic_borg_flags
done
sort borgmatic_borg_flags > borgmatic_borg_flags.sorted
mv borgmatic_borg_flags.sorted borgmatic_borg_flags
for word in $(borg $sub_command --help | grep '^ -') ; do
# Exclude a bunch of flags that borgmatic actually supports, but don't get exercised by the
# generated sample config, and also flags that don't make sense to support.
echo "$word" | grep ^-- | sed -e 's/,$//' \
| grep -v '^--archives-only$' \
| grep -v '^--critical$' \
| grep -v '^--debug$' \
| grep -v '^--dry-run$' \
| grep -v '^--error$' \
| grep -v '^--help$' \
| grep -v '^--info$' \
| grep -v '^--json$' \
| grep -v '^--keep-last$' \
| grep -v '^--list$' \
| grep -v '^--nobsdflags$' \
| grep -v '^--pattern$' \
| grep -v '^--progress$' \
| grep -v '^--stats$' \
| grep -v '^--read-special$' \
| grep -v '^--repository-only$' \
| grep -v '^--show-rc$' \
| grep -v '^--stats$' \
| grep -v '^--verbose$' \
| grep -v '^--warning$' \
| grep -v '^-h$' \
>> all_borg_flags
done
sort all_borg_flags > all_borg_flags.sorted
mv all_borg_flags.sorted all_borg_flags
comm -13 borgmatic_borg_flags all_borg_flags
rm ./*_borg_flags
done
rm temp.yaml
Executable
+8
View File
@@ -0,0 +1,8 @@
#!/bin/bash
set -e
branch_name=$(git rev-parse --abbrev-ref HEAD)
git push -u github "$branch_name"
git push -u origin "$branch_name"
+39
View File
@@ -0,0 +1,39 @@
#!/bin/bash
set -e
projects_token=${1:-}
github_token=${2:-}
if [[ -z $github_token ]]; then
echo "Usage: $0 [projects-token] [github-token]"
exit 1
fi
if [[ ! -f NEWS ]]; then
echo "Missing NEWS file. Try running from root of repository."
exit 1
fi
version=$(head --lines=1 NEWS)
git tag $version
git push origin $version
git push github $version
# Build borgmatic and publish to pypi.
rm -fr dist
python3 setup.py bdist_wheel
python3 setup.py sdist
twine upload -r pypi dist/borgmatic-*.tar.gz
twine upload -r pypi dist/borgmatic-*-py3-none-any.whl
# Set release changelogs on projects.evoworx.org and GitHub.
release_changelog="$(cat NEWS | sed '/^$/q' | grep -v '^\S')"
escaped_release_changelog="$(echo "$release_changelog" | sed -z 's/\n/\\n/g' | sed -z 's/\"/\\"/g')"
curl --silent --request POST \
"https://projects.torsion.org/api/v1/repos/witten/borgmatic/releases?access_token=$projects_token" \
--header "Accept: application/json" \
--header "Content-Type: application/json" \
--data "{\"body\": \"$escaped_release_changelog\", \"draft\": false, \"name\": \"borgmatic $version\", \"prerelease\": false, \"tag_name\": \"$version\"}"
github-release create --token="$github_token" --owner=witten --repo=borgmatic --tag="$version" \
--name="borgmatic $version" --body="$release_changelog"
+13 -14
View File
@@ -1,18 +1,17 @@
from setuptools import setup, find_packages
VERSION = '0.1.8'
VERSION = '1.3.1'
setup(
name='atticmatic',
name='borgmatic',
version=VERSION,
description='A wrapper script for Attic/Borg backup software that creates and prunes backups',
description='A wrapper script for Borg backup software that creates and prunes backups',
author='Dan Helfman',
author_email='witten@torsion.org',
url='https://torsion.org/atticmatic',
download_url='https://torsion.org/hg/atticmatic/archive/%s.tar.gz' % VERSION,
classifiers=(
url='https://torsion.org/borgmatic',
classifiers=[
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'Intended Audience :: System Administrators',
@@ -20,16 +19,16 @@ setup(
'Programming Language :: Python',
'Topic :: Security :: Cryptography',
'Topic :: System :: Archiving :: Backup',
),
packages=find_packages(),
],
packages=find_packages(exclude=['tests*']),
entry_points={
'console_scripts': [
'atticmatic = atticmatic.command:main',
'borgmatic = atticmatic.command:main',
'borgmatic = borgmatic.commands.borgmatic:main',
'upgrade-borgmatic-config = borgmatic.commands.convert_config:main',
'generate-borgmatic-config = borgmatic.commands.generate_config:main',
]
},
tests_require=(
'flexmock',
'pytest',
)
obsoletes=['atticmatic'],
install_requires=('pykwalify>=1.6.0,<14.06', 'ruamel.yaml>0.15.0,<0.16.0', 'setuptools'),
include_package_data=True,
)
Binary file not shown.

After

Width:  |  Height:  |  Size: 3.5 KiB

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" x="0px" y="0px" viewBox="0 0 100 100" enable-background="new 0 0 100 100" xml:space="preserve"><g><circle cx="63.461" cy="59.615" r="5.626"></circle><path d="M81.432,48.015c0.779-10.849-1.758-19.894-7.407-26.293C68.447,15.403,59.915,11.923,50,11.923 c-6.729,0-12.834,1.645-17.81,4.724c-0.405-0.06-0.823-0.073-1.248,0.005C8.128,20.817,0,34.623,0,46.154 c0,6.155,1.856,11.005,5.519,14.417c2.082,1.941,4.461,3.223,6.659,3.921C13.041,74.256,18.345,81,23.077,81h1.438 C30.168,94,44.847,99.614,50,99.614c6.708,0,24.493-7.875,27.38-25.392c4.758-1.888,10.697-7.373,10.697-13.711 C88.077,54.639,85.395,49.855,81.432,48.015z M7.691,46.154c0-3.397,1.238-13.399,14.747-19.083 c-2.043,3.821-3.31,8.234-3.771,13.136c-3.06,2.521-5.577,7.792-6.366,15.873c-0.534-0.316-1.056-0.686-1.538-1.135 C8.725,53.045,7.691,50.088,7.691,46.154z M74.23,67.947c-1.806,0-3.309,1.389-3.451,3.188C69.608,85.928,53.914,92.575,50,92.575 c-3.275,0-14.607-4.474-19.03-14.325c1.001-0.709,2.168-1.25,3.646-1.25c3.809,0,3.847,0,3.847,0c2.126,0,7.692-4.023,7.692-19.494 C46.154,46.241,40.588,42,38.462,42c0,0-0.195,0-3.847,0c-3.65,0-5.566-4-7.692-4h-1.046c0.813-4,2.646-8.756,5.473-11.94 c4.321-4.87,10.771-7.329,18.65-7.329c8.016,0,14.528,2.636,18.835,7.515c4.821,5.462,6.697,13.762,5.424,23.95 c-0.123,0.984,0.183,1.99,0.84,2.734s1.602,1.179,2.595,1.179c1.671,0,3.46,2.562,3.46,6.371 C81.153,64.016,75.754,67.812,74.23,67.947z M30.432,59.615c0-3.377,2.734-6.107,6.106-6.107c3.373,0,6.107,2.73,6.107,6.107 s-2.734,6.107-6.107,6.107C33.166,65.723,30.432,62.992,30.432,59.615z"></path></g></svg>

After

Width:  |  Height:  |  Size: 1.6 KiB

+22 -1
View File
@@ -1,2 +1,23 @@
appdirs==1.4.3
atomicwrites==1.2.1
attrs==18.2.0
black==18.9b0; python_version >= '3.6'
Click==7.0
coverage==4.5.1
docopt==0.6.2
flake8==3.5.0
flexmock==0.10.2
pytest==2.9.1
mccabe==0.6.1
more-itertools==4.3.0
pluggy==0.7.1
py==1.6.0
pycodestyle==2.3.1
pyflakes==2.0.0
pykwalify==1.7.0
pytest==3.8.2
pytest-cov==2.6.0
python-dateutil==2.7.3
PyYAML==3.13
ruamel.yaml>0.15.0,<0.16.0
six==1.11.0
toml==0.10.0
+81
View File
@@ -0,0 +1,81 @@
import json
import os
import shutil
import subprocess
import sys
import tempfile
def generate_configuration(config_path, repository_path):
'''
Generate borgmatic configuration into a file at the config path, and update the defaults so as
to work for testing (including injecting the given repository path and tacking on an encryption
passphrase).
'''
subprocess.check_call(
'generate-borgmatic-config --destination {}'.format(config_path).split(' ')
)
config = (
open(config_path)
.read()
.replace('user@backupserver:sourcehostname.borg', repository_path)
.replace('- /home', '- {}'.format(config_path))
.replace('- /etc', '')
.replace('- /var/log/syslog*', '')
+ 'storage:\n encryption_passphrase: "test"'
)
config_file = open(config_path, 'w')
config_file.write(config)
config_file.close()
def test_borgmatic_command():
# Create a Borg repository.
temporary_directory = tempfile.mkdtemp()
repository_path = os.path.join(temporary_directory, 'test.borg')
extract_path = os.path.join(temporary_directory, 'extract')
original_working_directory = os.getcwd()
os.mkdir(extract_path)
os.chdir(extract_path)
try:
config_path = os.path.join(temporary_directory, 'test.yaml')
generate_configuration(config_path, repository_path)
subprocess.check_call(
'borgmatic -v 2 --config {} --init --encryption repokey'.format(config_path).split(' ')
)
# Run borgmatic to generate a backup archive, and then list it to make sure it exists.
subprocess.check_call('borgmatic --config {}'.format(config_path).split(' '))
output = subprocess.check_output(
'borgmatic --config {} --list --json'.format(config_path).split(' ')
).decode(sys.stdout.encoding)
parsed_output = json.loads(output)
assert len(parsed_output) == 1
assert len(parsed_output[0]['archives']) == 1
archive_name = parsed_output[0]['archives'][0]['archive']
# Extract the created archive into the current (temporary) directory, and confirm that the
# extracted file looks right.
output = subprocess.check_output(
'borgmatic --config {} --extract --archive {}'.format(config_path, archive_name).split(
' '
)
).decode(sys.stdout.encoding)
extracted_config_path = os.path.join(extract_path, config_path)
assert open(extracted_config_path).read() == open(config_path).read()
# Exercise the info flag.
output = subprocess.check_output(
'borgmatic --config {} --info --json'.format(config_path).split(' ')
).decode(sys.stdout.encoding)
parsed_output = json.loads(output)
assert len(parsed_output) == 1
assert 'repository' in parsed_output[0]
finally:
os.chdir(original_working_directory)
shutil.rmtree(temporary_directory)
View File
@@ -0,0 +1,270 @@
import subprocess
from flexmock import flexmock
import pytest
from borgmatic.commands import borgmatic as module
def test_parse_arguments_with_no_arguments_uses_defaults():
config_paths = ['default']
flexmock(module.collect).should_receive('get_default_config_paths').and_return(config_paths)
parser = module.parse_arguments()
assert parser.config_paths == config_paths
assert parser.excludes_filename is None
assert parser.verbosity is 0
assert parser.json is False
def test_parse_arguments_with_multiple_config_paths_parses_as_list():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
parser = module.parse_arguments('--config', 'myconfig', 'otherconfig')
assert parser.config_paths == ['myconfig', 'otherconfig']
assert parser.verbosity is 0
def test_parse_arguments_with_verbosity_overrides_default():
config_paths = ['default']
flexmock(module.collect).should_receive('get_default_config_paths').and_return(config_paths)
parser = module.parse_arguments('--verbosity', '1')
assert parser.config_paths == config_paths
assert parser.excludes_filename is None
assert parser.verbosity == 1
def test_parse_arguments_with_json_overrides_default():
parser = module.parse_arguments('--list', '--json')
assert parser.json is True
def test_parse_arguments_with_no_actions_defaults_to_all_actions_enabled():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
parser = module.parse_arguments()
assert parser.prune is True
assert parser.create is True
assert parser.check is True
def test_parse_arguments_with_prune_action_leaves_other_actions_disabled():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
parser = module.parse_arguments('--prune')
assert parser.prune is True
assert parser.create is False
assert parser.check is False
def test_parse_arguments_with_multiple_actions_leaves_other_action_disabled():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
parser = module.parse_arguments('--create', '--check')
assert parser.prune is False
assert parser.create is True
assert parser.check is True
def test_parse_arguments_with_invalid_arguments_exits():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(SystemExit):
module.parse_arguments('--posix-me-harder')
def test_parse_arguments_disallows_deprecated_excludes_option():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--excludes', 'myexcludes')
def test_parse_arguments_disallows_encryption_mode_without_init():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--encryption', 'repokey')
def test_parse_arguments_allows_encryption_mode_with_init():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--config', 'myconfig', '--init', '--encryption', 'repokey')
def test_parse_arguments_requires_encryption_mode_with_init():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--init')
def test_parse_arguments_disallows_append_only_without_init():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--append-only')
def test_parse_arguments_disallows_storage_quota_without_init():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--storage-quota', '5G')
def test_parse_arguments_allows_init_and_prune():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--config', 'myconfig', '--init', '--encryption', 'repokey', '--prune')
def test_parse_arguments_allows_init_and_create():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--config', 'myconfig', '--init', '--encryption', 'repokey', '--create')
def test_parse_arguments_disallows_init_and_dry_run():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments(
'--config', 'myconfig', '--init', '--encryption', 'repokey', '--dry-run'
)
def test_parse_arguments_disallows_repository_without_extract_or_list():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--repository', 'test.borg')
def test_parse_arguments_allows_repository_with_extract():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments(
'--config', 'myconfig', '--extract', '--repository', 'test.borg', '--archive', 'test'
)
def test_parse_arguments_allows_repository_with_list():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--config', 'myconfig', '--list', '--repository', 'test.borg')
def test_parse_arguments_disallows_archive_without_extract_or_list():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--archive', 'test')
def test_parse_arguments_disallows_restore_paths_without_extract():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--restore-path', 'test')
def test_parse_arguments_allows_archive_with_extract():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--config', 'myconfig', '--extract', '--archive', 'test')
def test_parse_arguments_allows_archive_with_list():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--config', 'myconfig', '--list', '--archive', 'test')
def test_parse_arguments_requires_archive_with_extract():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--config', 'myconfig', '--extract')
def test_parse_arguments_allows_progress_and_create():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--progress', '--create', '--list')
def test_parse_arguments_allows_progress_and_extract():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--progress', '--extract', '--archive', 'test', '--list')
def test_parse_arguments_disallows_progress_without_create():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--progress', '--list')
def test_parse_arguments_with_stats_and_create_flags_does_not_raise():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--stats', '--create', '--list')
def test_parse_arguments_with_stats_and_prune_flags_does_not_raise():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--stats', '--prune', '--list')
def test_parse_arguments_with_stats_flag_but_no_create_or_prune_flag_raises_value_error():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--stats', '--list')
def test_parse_arguments_with_just_stats_flag_does_not_raise():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--stats')
def test_parse_arguments_allows_json_with_list_or_info():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('--list', '--json')
module.parse_arguments('--info', '--json')
def test_parse_arguments_disallows_json_without_list_or_info():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--json')
def test_parse_arguments_disallows_json_with_both_list_and_info():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--list', '--info', '--json')
def test_borgmatic_version_matches_news_version():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
borgmatic_version = subprocess.check_output(('borgmatic', '--version')).decode('ascii')
news_version = open('NEWS').readline()
assert borgmatic_version == news_version
@@ -0,0 +1,50 @@
import os
from flexmock import flexmock
import pytest
from borgmatic.commands import convert_config as module
def test_parse_arguments_with_no_arguments_uses_defaults():
flexmock(os.path).should_receive('exists').and_return(True)
parser = module.parse_arguments()
assert parser.source_config_filename == module.DEFAULT_SOURCE_CONFIG_FILENAME
assert parser.source_excludes_filename == module.DEFAULT_SOURCE_EXCLUDES_FILENAME
assert parser.destination_config_filename == module.DEFAULT_DESTINATION_CONFIG_FILENAME
def test_parse_arguments_with_filename_arguments_overrides_defaults():
flexmock(os.path).should_receive('exists').and_return(True)
parser = module.parse_arguments(
'--source-config',
'config',
'--source-excludes',
'excludes',
'--destination-config',
'config.yaml',
)
assert parser.source_config_filename == 'config'
assert parser.source_excludes_filename == 'excludes'
assert parser.destination_config_filename == 'config.yaml'
def test_parse_arguments_with_missing_default_excludes_file_sets_filename_to_none():
flexmock(os.path).should_receive('exists').and_return(False)
parser = module.parse_arguments()
assert parser.source_config_filename == module.DEFAULT_SOURCE_CONFIG_FILENAME
assert parser.source_excludes_filename is None
assert parser.destination_config_filename == module.DEFAULT_DESTINATION_CONFIG_FILENAME
def test_parse_arguments_with_invalid_arguments_exits():
flexmock(os.path).should_receive('exists').and_return(True)
with pytest.raises(SystemExit):
module.parse_arguments('--posix-me-harder')
@@ -0,0 +1,13 @@
from borgmatic.commands import generate_config as module
def test_parse_arguments_with_no_arguments_uses_defaults():
parser = module.parse_arguments()
assert parser.destination_filename == module.DEFAULT_DESTINATION_CONFIG_FILENAME
def test_parse_arguments_with_filename_argument_overrides_defaults():
parser = module.parse_arguments('--destination', 'config.yaml')
assert parser.destination_filename == 'config.yaml'
+125
View File
@@ -0,0 +1,125 @@
from io import StringIO
import os
import sys
from flexmock import flexmock
import pytest
from borgmatic.config import generate as module
def test_insert_newline_before_comment_does_not_raise():
field_name = 'foo'
config = module.yaml.comments.CommentedMap([(field_name, 33)])
config.yaml_set_comment_before_after_key(key=field_name, before='Comment')
module._insert_newline_before_comment(config, field_name)
def test_comment_out_line_skips_blank_line():
line = ' \n'
assert module._comment_out_line(line) == line
def test_comment_out_line_skips_already_commented_out_line():
line = ' # foo'
assert module._comment_out_line(line) == line
def test_comment_out_line_comments_section_name():
line = 'figgy-pudding:'
assert module._comment_out_line(line) == '#' + line
def test_comment_out_line_comments_indented_option():
line = ' enabled: true'
assert module._comment_out_line(line) == ' #enabled: true'
def test_comment_out_optional_configuration_comments_optional_config_only():
flexmock(module)._comment_out_line = lambda line: '#' + line
config = '''
foo:
bar:
- baz
- quux
location:
repositories:
- one
- two
other: thing
'''
expected_config = '''
#foo:
# bar:
# - baz
# - quux
#
location:
repositories:
- one
- two
#
# other: thing
'''
assert module._comment_out_optional_configuration(config.strip()) == expected_config.strip()
def test_render_configuration_does_not_raise():
flexmock(module.yaml).should_receive('round_trip_dump')
module._render_configuration({})
def test_write_configuration_does_not_raise():
flexmock(os.path).should_receive('exists').and_return(False)
flexmock(os).should_receive('makedirs')
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').and_return(StringIO())
flexmock(os).should_receive('chmod')
module.write_configuration('config.yaml', 'config: yaml')
def test_write_configuration_with_already_existing_file_raises():
flexmock(os.path).should_receive('exists').and_return(True)
with pytest.raises(FileExistsError):
module.write_configuration('config.yaml', 'config: yaml')
def test_write_configuration_with_already_existing_directory_does_not_raise():
flexmock(os.path).should_receive('exists').and_return(False)
flexmock(os).should_receive('makedirs').and_raise(FileExistsError)
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').and_return(StringIO())
flexmock(os).should_receive('chmod')
module.write_configuration('config.yaml', 'config: yaml')
def test_add_comments_to_configuration_does_not_raise():
# Ensure that it can deal with fields both in the schema and missing from the schema.
config = module.yaml.comments.CommentedMap([('foo', 33), ('bar', 44), ('baz', 55)])
schema = {'map': {'foo': {'desc': 'Foo'}, 'bar': {'desc': 'Bar'}}}
module.add_comments_to_configuration(config, schema)
def test_generate_sample_configuration_does_not_raise():
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('schema.yaml').and_return('')
flexmock(module).should_receive('_schema_to_sample_configuration')
flexmock(module).should_receive('_render_configuration')
flexmock(module).should_receive('_comment_out_optional_configuration')
flexmock(module).should_receive('write_configuration')
module.generate_sample_configuration('config.yaml', 'schema.yaml')
+19
View File
@@ -0,0 +1,19 @@
from io import StringIO
from collections import OrderedDict
import string
from borgmatic.config import legacy as module
def test_parse_section_options_with_punctuation_should_return_section_options():
parser = module.RawConfigParser()
parser.read_file(StringIO('[section]\nfoo: {}\n'.format(string.punctuation)))
section_format = module.Section_format(
'section', (module.Config_option('foo', str, required=True),)
)
config = module.parse_section_options(parser, section_format)
assert config == OrderedDict((('foo', string.punctuation),))
+40
View File
@@ -0,0 +1,40 @@
import sys
from flexmock import flexmock
from borgmatic.config import load as module
def test_load_configuration_parses_contents():
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('config.yaml').and_return('key: value')
assert module.load_configuration('config.yaml') == {'key': 'value'}
def test_load_configuration_inlines_include():
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('include.yaml').and_return('value')
builtins.should_receive('open').with_args('config.yaml').and_return(
'key: !include include.yaml'
)
assert module.load_configuration('config.yaml') == {'key': 'value'}
def test_load_configuration_merges_include():
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('include.yaml').and_return(
'''
foo: bar
baz: quux
'''
)
builtins.should_receive('open').with_args('config.yaml').and_return(
'''
foo: override
<<: !include include.yaml
'''
)
assert module.load_configuration('config.yaml') == {'foo': 'override', 'baz': 'quux'}
+214
View File
@@ -0,0 +1,214 @@
import io
import string
import sys
from flexmock import flexmock
import pytest
from borgmatic.config import validate as module
def test_schema_filename_returns_plausable_path():
schema_path = module.schema_filename()
assert schema_path.endswith('/schema.yaml')
def mock_config_and_schema(config_yaml, schema_yaml=None):
'''
Set up mocks for the given config config YAML string and the schema YAML string, or the default
schema if no schema is provided. The idea is that that the code under test consumes these mocks
when parsing the configuration.
'''
config_stream = io.StringIO(config_yaml)
if schema_yaml is None:
schema_stream = open(module.schema_filename())
else:
schema_stream = io.StringIO(schema_yaml)
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('config.yaml').and_return(config_stream)
builtins.should_receive('open').with_args('schema.yaml').and_return(schema_stream)
def test_parse_configuration_transforms_file_into_mapping():
mock_config_and_schema(
'''
location:
source_directories:
- /home
- /etc
repositories:
- hostname.borg
retention:
keep_minutely: 60
keep_hourly: 24
keep_daily: 7
consistency:
checks:
- repository
- archives
'''
)
result = module.parse_configuration('config.yaml', 'schema.yaml')
assert result == {
'location': {'source_directories': ['/home', '/etc'], 'repositories': ['hostname.borg']},
'retention': {'keep_daily': 7, 'keep_hourly': 24, 'keep_minutely': 60},
'consistency': {'checks': ['repository', 'archives']},
}
def test_parse_configuration_passes_through_quoted_punctuation():
escaped_punctuation = string.punctuation.replace('\\', r'\\').replace('"', r'\"')
mock_config_and_schema(
'''
location:
source_directories:
- /home
repositories:
- "{}.borg"
'''.format(
escaped_punctuation
)
)
result = module.parse_configuration('config.yaml', 'schema.yaml')
assert result == {
'location': {
'source_directories': ['/home'],
'repositories': ['{}.borg'.format(string.punctuation)],
}
}
def test_parse_configuration_with_schema_lacking_examples_does_not_raise():
mock_config_and_schema(
'''
location:
source_directories:
- /home
repositories:
- hostname.borg
''',
'''
map:
location:
required: true
map:
source_directories:
required: true
seq:
- type: scalar
repositories:
required: true
seq:
- type: scalar
''',
)
module.parse_configuration('config.yaml', 'schema.yaml')
def test_parse_configuration_inlines_include():
mock_config_and_schema(
'''
location:
source_directories:
- /home
repositories:
- hostname.borg
retention:
!include include.yaml
'''
)
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('include.yaml').and_return(
'''
keep_daily: 7
keep_hourly: 24
'''
)
result = module.parse_configuration('config.yaml', 'schema.yaml')
assert result == {
'location': {'source_directories': ['/home'], 'repositories': ['hostname.borg']},
'retention': {'keep_daily': 7, 'keep_hourly': 24},
}
def test_parse_configuration_merges_include():
mock_config_and_schema(
'''
location:
source_directories:
- /home
repositories:
- hostname.borg
retention:
keep_daily: 1
<<: !include include.yaml
'''
)
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('include.yaml').and_return(
'''
keep_daily: 7
keep_hourly: 24
'''
)
result = module.parse_configuration('config.yaml', 'schema.yaml')
assert result == {
'location': {'source_directories': ['/home'], 'repositories': ['hostname.borg']},
'retention': {'keep_daily': 1, 'keep_hourly': 24},
}
def test_parse_configuration_raises_for_missing_config_file():
with pytest.raises(FileNotFoundError):
module.parse_configuration('config.yaml', 'schema.yaml')
def test_parse_configuration_raises_for_missing_schema_file():
mock_config_and_schema('')
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('schema.yaml').and_raise(FileNotFoundError)
with pytest.raises(FileNotFoundError):
module.parse_configuration('config.yaml', 'schema.yaml')
def test_parse_configuration_raises_for_syntax_error():
mock_config_and_schema('foo:\nbar')
with pytest.raises(ValueError):
module.parse_configuration('config.yaml', 'schema.yaml')
def test_parse_configuration_raises_for_validation_error():
mock_config_and_schema(
'''
location:
source_directories: yes
repositories:
- hostname.borg
'''
)
with pytest.raises(module.Validation_error):
module.parse_configuration('config.yaml', 'schema.yaml')
View File
View File
+265
View File
@@ -0,0 +1,265 @@
from subprocess import STDOUT
import logging
import sys
from flexmock import flexmock
import pytest
from borgmatic.borg import check as module
from ..test_verbosity import insert_logging_mock
def insert_subprocess_mock(check_call_command, **kwargs):
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
def insert_subprocess_never():
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').never()
def test_parse_checks_returns_them_as_tuple():
checks = module._parse_checks({'checks': ['foo', 'disabled', 'bar']})
assert checks == ('foo', 'bar')
def test_parse_checks_with_missing_value_returns_defaults():
checks = module._parse_checks({})
assert checks == module.DEFAULT_CHECKS
def test_parse_checks_with_blank_value_returns_defaults():
checks = module._parse_checks({'checks': []})
assert checks == module.DEFAULT_CHECKS
def test_parse_checks_with_disabled_returns_no_checks():
checks = module._parse_checks({'checks': ['disabled']})
assert checks == ()
def test_make_check_flags_with_repository_check_returns_flag():
flags = module._make_check_flags(('repository',))
assert flags == ('--repository-only',)
def test_make_check_flags_with_extract_omits_extract_flag():
flags = module._make_check_flags(('extract',))
assert flags == ()
def test_make_check_flags_with_default_checks_returns_default_flags():
flags = module._make_check_flags(module.DEFAULT_CHECKS)
assert flags == ('--prefix', module.DEFAULT_PREFIX)
def test_make_check_flags_with_all_checks_returns_default_flags():
flags = module._make_check_flags(module.DEFAULT_CHECKS + ('extract',))
assert flags == ('--prefix', module.DEFAULT_PREFIX)
def test_make_check_flags_with_archives_check_and_last_includes_last_flag():
flags = module._make_check_flags(('archives',), check_last=3)
assert flags == ('--archives-only', '--last', '3', '--prefix', module.DEFAULT_PREFIX)
def test_make_check_flags_with_repository_check_and_last_omits_last_flag():
flags = module._make_check_flags(('repository',), check_last=3)
assert flags == ('--repository-only',)
def test_make_check_flags_with_default_checks_and_last_includes_last_flag():
flags = module._make_check_flags(module.DEFAULT_CHECKS, check_last=3)
assert flags == ('--last', '3', '--prefix', module.DEFAULT_PREFIX)
def test_make_check_flags_with_archives_check_and_prefix_includes_prefix_flag():
flags = module._make_check_flags(('archives',), prefix='foo-')
assert flags == ('--archives-only', '--prefix', 'foo-')
def test_make_check_flags_with_repository_check_and_prefix_omits_prefix_flag():
flags = module._make_check_flags(('repository',), prefix='foo-')
assert flags == ('--repository-only',)
def test_make_check_flags_with_default_checks_and_prefix_includes_prefix_flag():
flags = module._make_check_flags(module.DEFAULT_CHECKS, prefix='foo-')
assert flags == ('--prefix', 'foo-')
@pytest.mark.parametrize(
'checks',
(
('repository',),
('archives',),
('repository', 'archives'),
('repository', 'archives', 'other'),
),
)
def test_check_archives_calls_borg_with_parameters(checks):
check_last = flexmock()
consistency_config = {'check_last': check_last}
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').with_args(
checks, check_last, None
).and_return(())
stdout = flexmock()
insert_subprocess_mock(('borg', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
flexmock(module.os).should_receive('devnull')
module.check_archives(
repository='repo', storage_config={}, consistency_config=consistency_config
)
def test_check_archives_with_extract_check_calls_extract_only():
checks = ('extract',)
check_last = flexmock()
consistency_config = {'check_last': check_last}
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').never()
flexmock(module.extract).should_receive('extract_last_archive_dry_run').once()
insert_subprocess_never()
module.check_archives(
repository='repo', storage_config={}, consistency_config=consistency_config
)
def test_check_archives_with_log_info_calls_borg_with_info_parameter():
checks = ('repository',)
consistency_config = {'check_last': None}
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').and_return(())
insert_logging_mock(logging.INFO)
insert_subprocess_mock(('borg', 'check', 'repo', '--info'), stdout=None, stderr=STDOUT)
module.check_archives(
repository='repo', storage_config={}, consistency_config=consistency_config
)
def test_check_archives_with_log_debug_calls_borg_with_debug_parameter():
checks = ('repository',)
consistency_config = {'check_last': None}
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').and_return(())
insert_logging_mock(logging.DEBUG)
insert_subprocess_mock(
('borg', 'check', 'repo', '--debug', '--show-rc'), stdout=None, stderr=STDOUT
)
module.check_archives(
repository='repo', storage_config={}, consistency_config=consistency_config
)
def test_check_archives_without_any_checks_bails():
consistency_config = {'check_last': None}
flexmock(module).should_receive('_parse_checks').and_return(())
insert_subprocess_never()
module.check_archives(
repository='repo', storage_config={}, consistency_config=consistency_config
)
def test_check_archives_with_local_path_calls_borg_via_local_path():
checks = ('repository',)
check_last = flexmock()
consistency_config = {'check_last': check_last}
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').with_args(
checks, check_last, None
).and_return(())
stdout = flexmock()
insert_subprocess_mock(('borg1', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
flexmock(module.os).should_receive('devnull')
module.check_archives(
repository='repo',
storage_config={},
consistency_config=consistency_config,
local_path='borg1',
)
def test_check_archives_with_remote_path_calls_borg_with_remote_path_parameters():
checks = ('repository',)
check_last = flexmock()
consistency_config = {'check_last': check_last}
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').with_args(
checks, check_last, None
).and_return(())
stdout = flexmock()
insert_subprocess_mock(
('borg', 'check', 'repo', '--remote-path', 'borg1'), stdout=stdout, stderr=STDOUT
)
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
flexmock(module.os).should_receive('devnull')
module.check_archives(
repository='repo',
storage_config={},
consistency_config=consistency_config,
remote_path='borg1',
)
def test_check_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
checks = ('repository',)
check_last = flexmock()
consistency_config = {'check_last': check_last}
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').with_args(
checks, check_last, None
).and_return(())
stdout = flexmock()
insert_subprocess_mock(
('borg', 'check', 'repo', '--lock-wait', '5'), stdout=stdout, stderr=STDOUT
)
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
flexmock(module.os).should_receive('devnull')
module.check_archives(
repository='repo', storage_config={'lock_wait': 5}, consistency_config=consistency_config
)
def test_check_archives_with_retention_prefix():
checks = ('repository',)
check_last = flexmock()
prefix = 'foo-'
consistency_config = {'check_last': check_last, 'prefix': prefix}
flexmock(module).should_receive('_parse_checks').and_return(checks)
flexmock(module).should_receive('_make_check_flags').with_args(
checks, check_last, prefix
).and_return(())
stdout = flexmock()
insert_subprocess_mock(('borg', 'check', 'repo'), stdout=stdout, stderr=STDOUT)
flexmock(sys.modules['builtins']).should_receive('open').and_return(stdout)
flexmock(module.os).should_receive('devnull')
module.check_archives(
repository='repo', storage_config={}, consistency_config=consistency_config
)
+809
View File
@@ -0,0 +1,809 @@
import logging
from flexmock import flexmock
from borgmatic.borg import create as module
from ..test_verbosity import insert_logging_mock
def test_expand_directory_with_basic_path_passes_it_through():
flexmock(module.os.path).should_receive('expanduser').and_return('foo')
flexmock(module.glob).should_receive('glob').and_return([])
paths = module._expand_directory('foo')
assert paths == ['foo']
def test_expand_directory_with_glob_expands():
flexmock(module.os.path).should_receive('expanduser').and_return('foo*')
flexmock(module.glob).should_receive('glob').and_return(['foo', 'food'])
paths = module._expand_directory('foo*')
assert paths == ['foo', 'food']
def test_expand_directories_flattens_expanded_directories():
flexmock(module).should_receive('_expand_directory').with_args('~/foo').and_return(
['/root/foo']
)
flexmock(module).should_receive('_expand_directory').with_args('bar*').and_return(
['bar', 'barf']
)
paths = module._expand_directories(('~/foo', 'bar*'))
assert paths == ('/root/foo', 'bar', 'barf')
def test_expand_directories_considers_none_as_no_directories():
paths = module._expand_directories(None)
assert paths == ()
def test_expand_home_directories_expands_tildes():
flexmock(module.os.path).should_receive('expanduser').with_args('~/bar').and_return('/foo/bar')
flexmock(module.os.path).should_receive('expanduser').with_args('baz').and_return('baz')
paths = module._expand_home_directories(('~/bar', 'baz'))
assert paths == ('/foo/bar', 'baz')
def test_expand_home_directories_considers_none_as_no_directories():
paths = module._expand_home_directories(None)
assert paths == ()
def test_write_pattern_file_does_not_raise():
temporary_file = flexmock(name='filename', write=lambda mode: None, flush=lambda: None)
flexmock(module.tempfile).should_receive('NamedTemporaryFile').and_return(temporary_file)
module._write_pattern_file(['exclude'])
def test_write_pattern_file_with_empty_exclude_patterns_does_not_raise():
module._write_pattern_file([])
def test_make_pattern_flags_includes_pattern_filename_when_given():
pattern_flags = module._make_pattern_flags(
location_config={'patterns': ['R /', '- /var']}, pattern_filename='/tmp/patterns'
)
assert pattern_flags == ('--patterns-from', '/tmp/patterns')
def test_make_pattern_flags_includes_patterns_from_filenames_when_in_config():
pattern_flags = module._make_pattern_flags(
location_config={'patterns_from': ['patterns', 'other']}
)
assert pattern_flags == ('--patterns-from', 'patterns', '--patterns-from', 'other')
def test_make_pattern_flags_includes_both_filenames_when_patterns_given_and_patterns_from_in_config():
pattern_flags = module._make_pattern_flags(
location_config={'patterns_from': ['patterns']}, pattern_filename='/tmp/patterns'
)
assert pattern_flags == ('--patterns-from', 'patterns', '--patterns-from', '/tmp/patterns')
def test_make_pattern_flags_considers_none_patterns_from_filenames_as_empty():
pattern_flags = module._make_pattern_flags(location_config={'patterns_from': None})
assert pattern_flags == ()
def test_make_exclude_flags_includes_exclude_patterns_filename_when_given():
exclude_flags = module._make_exclude_flags(
location_config={'exclude_patterns': ['*.pyc', '/var']}, exclude_filename='/tmp/excludes'
)
assert exclude_flags == ('--exclude-from', '/tmp/excludes')
def test_make_exclude_flags_includes_exclude_from_filenames_when_in_config():
exclude_flags = module._make_exclude_flags(
location_config={'exclude_from': ['excludes', 'other']}
)
assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', 'other')
def test_make_exclude_flags_includes_both_filenames_when_patterns_given_and_exclude_from_in_config():
exclude_flags = module._make_exclude_flags(
location_config={'exclude_from': ['excludes']}, exclude_filename='/tmp/excludes'
)
assert exclude_flags == ('--exclude-from', 'excludes', '--exclude-from', '/tmp/excludes')
def test_make_exclude_flags_considers_none_exclude_from_filenames_as_empty():
exclude_flags = module._make_exclude_flags(location_config={'exclude_from': None})
assert exclude_flags == ()
def test_make_exclude_flags_includes_exclude_caches_when_true_in_config():
exclude_flags = module._make_exclude_flags(location_config={'exclude_caches': True})
assert exclude_flags == ('--exclude-caches',)
def test_make_exclude_flags_does_not_include_exclude_caches_when_false_in_config():
exclude_flags = module._make_exclude_flags(location_config={'exclude_caches': False})
assert exclude_flags == ()
def test_make_exclude_flags_includes_exclude_if_present_when_in_config():
exclude_flags = module._make_exclude_flags(location_config={'exclude_if_present': 'exclude_me'})
assert exclude_flags == ('--exclude-if-present', 'exclude_me')
def test_make_exclude_flags_is_empty_when_config_has_no_excludes():
exclude_flags = module._make_exclude_flags(location_config={})
assert exclude_flags == ()
DEFAULT_ARCHIVE_NAME = '{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}'
CREATE_COMMAND = ('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'bar')
def test_create_archive_calls_borg_with_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND, capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_patterns_calls_borg_with_patterns():
pattern_flags = ('--patterns-from', 'patterns')
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(
flexmock(name='/tmp/patterns')
).and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(pattern_flags)
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + pattern_flags, capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'patterns': ['pattern'],
},
storage_config={},
)
def test_create_archive_with_exclude_patterns_calls_borg_with_excludes():
exclude_flags = ('--exclude-from', 'excludes')
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(('exclude',))
flexmock(module).should_receive('_write_pattern_file').and_return(None).and_return(
flexmock(name='/tmp/excludes')
)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(exclude_flags)
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + exclude_flags, capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': ['exclude'],
},
storage_config={},
)
def test_create_archive_with_log_info_calls_borg_with_info_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--list', '--filter', 'AME-', '--info', '--stats'), capture_output=False
)
insert_logging_mock(logging.INFO)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_log_debug_calls_borg_with_debug_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--list', '--filter', 'AME-', '--stats', '--debug', '--show-rc'),
capture_output=False,
)
insert_logging_mock(logging.DEBUG)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_dry_run_calls_borg_with_dry_run_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--dry-run',), capture_output=False
)
module.create_archive(
dry_run=True,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_dry_run_and_log_info_calls_borg_without_stats_parameter():
# --dry-run and --stats are mutually exclusive, see:
# https://borgbackup.readthedocs.io/en/stable/usage/create.html#description
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--list', '--filter', 'AME-', '--info', '--dry-run'), capture_output=False
)
insert_logging_mock(logging.INFO)
module.create_archive(
dry_run=True,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_dry_run_and_log_debug_calls_borg_without_stats_parameter():
# --dry-run and --stats are mutually exclusive, see:
# https://borgbackup.readthedocs.io/en/stable/usage/create.html#description
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--list', '--filter', 'AME-', '--debug', '--show-rc', '--dry-run'),
capture_output=False,
)
insert_logging_mock(logging.DEBUG)
module.create_archive(
dry_run=True,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_checkpoint_interval_calls_borg_with_checkpoint_interval_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--checkpoint-interval', '600'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={'checkpoint_interval': 600},
)
def test_create_archive_with_chunker_params_calls_borg_with_chunker_params_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--chunker-params', '1,2,3,4'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={'chunker_params': '1,2,3,4'},
)
def test_create_archive_with_compression_calls_borg_with_compression_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--compression', 'rle'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={'compression': 'rle'},
)
def test_create_archive_with_remote_rate_limit_calls_borg_with_remote_ratelimit_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--remote-ratelimit', '100'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={'remote_rate_limit': 100},
)
def test_create_archive_with_one_file_system_calls_borg_with_one_file_system_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--one-file-system',), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'one_file_system': True,
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_numeric_owner_calls_borg_with_numeric_owner_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--numeric-owner',), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'numeric_owner': True,
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_read_special_calls_borg_with_read_special_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--read-special',), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'read_special': True,
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_bsd_flags_true_calls_borg_without_nobsdflags_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND, capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'bsd_flags': True,
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_bsd_flags_false_calls_borg_with_nobsdflags_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--nobsdflags',), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'bsd_flags': False,
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_files_cache_calls_borg_with_files_cache_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--files-cache', 'ctime,size'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'files_cache': 'ctime,size',
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_local_path_calls_borg_via_local_path():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
('borg1',) + CREATE_COMMAND[1:], capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
local_path='borg1',
)
def test_create_archive_with_remote_path_calls_borg_with_remote_path_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--remote-path', 'borg1'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
remote_path='borg1',
)
def test_create_archive_with_umask_calls_borg_with_umask_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--umask', '740'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={'umask': 740},
)
def test_create_archive_with_lock_wait_calls_borg_with_lock_wait_parameters():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--lock-wait', '5'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={'lock_wait': 5},
)
def test_create_archive_with_json_calls_borg_with_json_parameter():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
CREATE_COMMAND + ('--json',), capture_output=True
).and_return('[]')
json_output = module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
json=True,
)
assert json_output == '[]'
def test_create_archive_with_source_directories_glob_expands():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food'),
capture_output=False,
)
flexmock(module.glob).should_receive('glob').with_args('foo*').and_return(['foo', 'food'])
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo*'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_non_matching_source_directories_glob_passes_through():
flexmock(module).should_receive('_expand_directories').and_return(('foo*',))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo*'), capture_output=False
)
flexmock(module.glob).should_receive('glob').with_args('foo*').and_return([])
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo*'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_glob_calls_borg_with_expanded_directories():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'food'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
('borg', 'create', 'repo::{}'.format(DEFAULT_ARCHIVE_NAME), 'foo', 'food'),
capture_output=False,
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo*'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={},
)
def test_create_archive_with_archive_name_format_calls_borg_with_archive_name():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
('borg', 'create', 'repo::ARCHIVE_NAME', 'foo', 'bar'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={'archive_name_format': 'ARCHIVE_NAME'},
)
def test_create_archive_with_archive_name_format_accepts_borg_placeholders():
flexmock(module).should_receive('_expand_directories').and_return(('foo', 'bar'))
flexmock(module).should_receive('_expand_home_directories').and_return(())
flexmock(module).should_receive('_write_pattern_file').and_return(None)
flexmock(module).should_receive('_make_pattern_flags').and_return(())
flexmock(module).should_receive('_make_exclude_flags').and_return(())
flexmock(module).should_receive('execute_command').with_args(
('borg', 'create', 'repo::Documents_{hostname}-{now}', 'foo', 'bar'), capture_output=False
)
module.create_archive(
dry_run=False,
repository='repo',
location_config={
'source_directories': ['foo', 'bar'],
'repositories': ['repo'],
'exclude_patterns': None,
},
storage_config={'archive_name_format': 'Documents_{hostname}-{now}'},
)
+49
View File
@@ -0,0 +1,49 @@
import os
from borgmatic.borg import environment as module
def test_initialize_with_passcommand_should_set_environment():
orig_environ = os.environ
try:
os.environ = {}
module.initialize({'encryption_passcommand': 'command'})
assert os.environ.get('BORG_PASSCOMMAND') == 'command'
finally:
os.environ = orig_environ
def test_initialize_with_passphrase_should_set_environment():
orig_environ = os.environ
try:
os.environ = {}
module.initialize({'encryption_passphrase': 'pass'})
assert os.environ.get('BORG_PASSPHRASE') == 'pass'
finally:
os.environ = orig_environ
def test_initialize_with_ssh_command_should_set_environment():
orig_environ = os.environ
try:
os.environ = {}
module.initialize({'ssh_command': 'ssh -C'})
assert os.environ.get('BORG_RSH') == 'ssh -C'
finally:
os.environ = orig_environ
def test_initialize_without_configuration_should_not_set_environment():
orig_environ = os.environ
try:
os.environ = {}
module.initialize({})
assert os.environ.get('BORG_PASSCOMMAND') is None
assert os.environ.get('BORG_PASSPHRASE') is None
assert os.environ.get('BORG_RSH') is None
finally:
os.environ = orig_environ
+25
View File
@@ -0,0 +1,25 @@
from flexmock import flexmock
from borgmatic.borg import execute as module
def test_execute_command_calls_full_command():
full_command = ['foo', 'bar']
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').with_args(full_command).once()
output = module.execute_command(full_command)
assert output is None
def test_execute_command_captures_output():
full_command = ['foo', 'bar']
expected_output = '[]'
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_output').with_args(full_command).and_return(
flexmock(decode=lambda: expected_output)
).once()
output = module.execute_command(full_command, capture_output=True)
assert output == expected_output
+224
View File
@@ -0,0 +1,224 @@
import logging
import sys
from flexmock import flexmock
from borgmatic.borg import extract as module
from ..test_verbosity import insert_logging_mock
def insert_subprocess_mock(check_call_command, **kwargs):
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
def insert_subprocess_never():
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').never()
def insert_subprocess_check_output_mock(check_output_command, result, **kwargs):
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_output').with_args(check_output_command, **kwargs).and_return(
result
).once()
def test_extract_last_archive_dry_run_calls_borg_with_last_archive():
flexmock(sys.stdout).encoding = 'utf-8'
insert_subprocess_check_output_mock(
('borg', 'list', '--short', 'repo'), result='archive1\narchive2\n'.encode('utf-8')
)
insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2'))
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
def test_extract_last_archive_dry_run_without_any_archives_should_bail():
flexmock(sys.stdout).encoding = 'utf-8'
insert_subprocess_check_output_mock(
('borg', 'list', '--short', 'repo'), result='\n'.encode('utf-8')
)
insert_subprocess_never()
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
def test_extract_last_archive_dry_run_with_log_info_calls_borg_with_info_parameter():
flexmock(sys.stdout).encoding = 'utf-8'
insert_subprocess_check_output_mock(
('borg', 'list', '--short', 'repo', '--info'), result='archive1\narchive2\n'.encode('utf-8')
)
insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2', '--info'))
insert_logging_mock(logging.INFO)
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
def test_extract_last_archive_dry_run_with_log_debug_calls_borg_with_debug_parameter():
flexmock(sys.stdout).encoding = 'utf-8'
insert_subprocess_check_output_mock(
('borg', 'list', '--short', 'repo', '--debug', '--show-rc'),
result='archive1\narchive2\n'.encode('utf-8'),
)
insert_subprocess_mock(
('borg', 'extract', '--dry-run', 'repo::archive2', '--debug', '--show-rc', '--list')
)
insert_logging_mock(logging.DEBUG)
module.extract_last_archive_dry_run(repository='repo', lock_wait=None)
def test_extract_last_archive_dry_run_calls_borg_via_local_path():
flexmock(sys.stdout).encoding = 'utf-8'
insert_subprocess_check_output_mock(
('borg1', 'list', '--short', 'repo'), result='archive1\narchive2\n'.encode('utf-8')
)
insert_subprocess_mock(('borg1', 'extract', '--dry-run', 'repo::archive2'))
module.extract_last_archive_dry_run(repository='repo', lock_wait=None, local_path='borg1')
def test_extract_last_archive_dry_run_calls_borg_with_remote_path_parameters():
flexmock(sys.stdout).encoding = 'utf-8'
insert_subprocess_check_output_mock(
('borg', 'list', '--short', 'repo', '--remote-path', 'borg1'),
result='archive1\narchive2\n'.encode('utf-8'),
)
insert_subprocess_mock(
('borg', 'extract', '--dry-run', 'repo::archive2', '--remote-path', 'borg1')
)
module.extract_last_archive_dry_run(repository='repo', lock_wait=None, remote_path='borg1')
def test_extract_last_archive_dry_run_calls_borg_with_lock_wait_parameters():
flexmock(sys.stdout).encoding = 'utf-8'
insert_subprocess_check_output_mock(
('borg', 'list', '--short', 'repo', '--lock-wait', '5'),
result='archive1\narchive2\n'.encode('utf-8'),
)
insert_subprocess_mock(('borg', 'extract', '--dry-run', 'repo::archive2', '--lock-wait', '5'))
module.extract_last_archive_dry_run(repository='repo', lock_wait=5)
def test_extract_archive_calls_borg_with_restore_path_parameters():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', 'path1', 'path2'))
module.extract_archive(
dry_run=False,
repository='repo',
archive='archive',
restore_paths=['path1', 'path2'],
location_config={},
storage_config={},
)
def test_extract_archive_calls_borg_with_remote_path_parameters():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--remote-path', 'borg1'))
module.extract_archive(
dry_run=False,
repository='repo',
archive='archive',
restore_paths=None,
location_config={},
storage_config={},
remote_path='borg1',
)
def test_extract_archive_calls_borg_with_numeric_owner_parameter():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--numeric-owner'))
module.extract_archive(
dry_run=False,
repository='repo',
archive='archive',
restore_paths=None,
location_config={'numeric_owner': True},
storage_config={},
)
def test_extract_archive_calls_borg_with_umask_parameters():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--umask', '0770'))
module.extract_archive(
dry_run=False,
repository='repo',
archive='archive',
restore_paths=None,
location_config={},
storage_config={'umask': '0770'},
)
def test_extract_archive_calls_borg_with_lock_wait_parameters():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--lock-wait', '5'))
module.extract_archive(
dry_run=False,
repository='repo',
archive='archive',
restore_paths=None,
location_config={},
storage_config={'lock_wait': '5'},
)
def test_extract_archive_with_log_info_calls_borg_with_info_parameter():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--info'))
insert_logging_mock(logging.INFO)
module.extract_archive(
dry_run=False,
repository='repo',
archive='archive',
restore_paths=None,
location_config={},
storage_config={},
)
def test_extract_archive_with_log_debug_calls_borg_with_debug_parameters():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--debug', '--list', '--show-rc'))
insert_logging_mock(logging.DEBUG)
module.extract_archive(
dry_run=False,
repository='repo',
archive='archive',
restore_paths=None,
location_config={},
storage_config={},
)
def test_extract_archive_calls_borg_with_dry_run_parameter():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--dry-run'))
module.extract_archive(
dry_run=True,
repository='repo',
archive='archive',
restore_paths=None,
location_config={},
storage_config={},
)
def test_extract_archive_calls_borg_with_progress_parameter():
insert_subprocess_mock(('borg', 'extract', 'repo::archive', '--progress'))
module.extract_archive(
dry_run=False,
repository='repo',
archive='archive',
restore_paths=None,
location_config={},
storage_config={},
progress=True,
)
+22
View File
@@ -0,0 +1,22 @@
from flexmock import flexmock
from borgmatic.commands import hook as module
def test_execute_hook_invokes_each_command():
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').with_args(':', shell=True).once()
module.execute_hook([':'], 'config.yaml', 'pre-backup')
def test_execute_hook_with_multiple_commands_invokes_each_command():
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').with_args(':', shell=True).once()
subprocess.should_receive('check_call').with_args('true', shell=True).once()
module.execute_hook([':', 'true'], 'config.yaml', 'pre-backup')
def test_execute_hook_with_empty_commands_does_not_raise():
module.execute_hook([], 'config.yaml', 'post-backup')
+67
View File
@@ -0,0 +1,67 @@
import logging
from flexmock import flexmock
from borgmatic.borg import info as module
from ..test_verbosity import insert_logging_mock
INFO_COMMAND = ('borg', 'info', 'repo')
def test_display_archives_info_calls_borg_with_parameters():
flexmock(module).should_receive('execute_command').with_args(INFO_COMMAND, capture_output=False)
module.display_archives_info(repository='repo', storage_config={})
def test_display_archives_info_with_log_info_calls_borg_with_info_parameter():
flexmock(module).should_receive('execute_command').with_args(
INFO_COMMAND + ('--info',), capture_output=False
)
insert_logging_mock(logging.INFO)
module.display_archives_info(repository='repo', storage_config={})
def test_display_archives_info_with_log_debug_calls_borg_with_debug_parameter():
flexmock(module).should_receive('execute_command').with_args(
INFO_COMMAND + ('--debug', '--show-rc'), capture_output=False
)
insert_logging_mock(logging.DEBUG)
module.display_archives_info(repository='repo', storage_config={})
def test_display_archives_info_with_json_calls_borg_with_json_parameter():
flexmock(module).should_receive('execute_command').with_args(
INFO_COMMAND + ('--json',), capture_output=True
).and_return('[]')
json_output = module.display_archives_info(repository='repo', storage_config={}, json=True)
assert json_output == '[]'
def test_display_archives_info_with_local_path_calls_borg_via_local_path():
flexmock(module).should_receive('execute_command').with_args(
('borg1',) + INFO_COMMAND[1:], capture_output=False
)
module.display_archives_info(repository='repo', storage_config={}, local_path='borg1')
def test_display_archives_info_with_remote_path_calls_borg_with_remote_path_parameters():
flexmock(module).should_receive('execute_command').with_args(
INFO_COMMAND + ('--remote-path', 'borg1'), capture_output=False
)
module.display_archives_info(repository='repo', storage_config={}, remote_path='borg1')
def test_display_archives_info_with_lock_wait_calls_borg_with_lock_wait_parameters():
storage_config = {'lock_wait': 5}
flexmock(module).should_receive('execute_command').with_args(
INFO_COMMAND + ('--lock-wait', '5'), capture_output=False
)
module.display_archives_info(repository='repo', storage_config=storage_config)
+79
View File
@@ -0,0 +1,79 @@
import logging
from flexmock import flexmock
from borgmatic.borg import init as module
from ..test_verbosity import insert_logging_mock
INFO_REPOSITORY_EXISTS_RESPONSE_CODE = 0
INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE = 2
INIT_COMMAND = ('borg', 'init', 'repo', '--encryption', 'repokey')
def insert_info_command_mock(info_response):
subprocess = flexmock(module.subprocess)
subprocess.should_receive('call').and_return(info_response)
def insert_init_command_mock(init_command, **kwargs):
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').with_args(init_command, **kwargs).once()
def test_initialize_repository_calls_borg_with_parameters():
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
insert_init_command_mock(INIT_COMMAND)
module.initialize_repository(repository='repo', encryption_mode='repokey')
def test_initialize_repository_skips_initialization_when_repository_already_exists():
insert_info_command_mock(INFO_REPOSITORY_EXISTS_RESPONSE_CODE)
flexmock(module.subprocess).should_receive('check_call').never()
module.initialize_repository(repository='repo', encryption_mode='repokey')
def test_initialize_repository_with_append_only_calls_borg_with_append_only_parameter():
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
insert_init_command_mock(INIT_COMMAND + ('--append-only',))
module.initialize_repository(repository='repo', encryption_mode='repokey', append_only=True)
def test_initialize_repository_with_storage_quota_calls_borg_with_storage_quota_parameter():
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
insert_init_command_mock(INIT_COMMAND + ('--storage-quota', '5G'))
module.initialize_repository(repository='repo', encryption_mode='repokey', storage_quota='5G')
def test_initialize_repository_with_log_info_calls_borg_with_info_parameter():
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
insert_init_command_mock(INIT_COMMAND + ('--info',))
insert_logging_mock(logging.INFO)
module.initialize_repository(repository='repo', encryption_mode='repokey')
def test_initialize_repository_with_log_debug_calls_borg_with_debug_parameter():
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
insert_init_command_mock(INIT_COMMAND + ('--debug',))
insert_logging_mock(logging.DEBUG)
module.initialize_repository(repository='repo', encryption_mode='repokey')
def test_initialize_repository_with_local_path_calls_borg_via_local_path():
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
insert_init_command_mock(('borg1',) + INIT_COMMAND[1:])
module.initialize_repository(repository='repo', encryption_mode='repokey', local_path='borg1')
def test_initialize_repository_with_remote_path_calls_borg_with_remote_path_parameter():
insert_info_command_mock(INFO_REPOSITORY_NOT_FOUND_RESPONSE_CODE)
insert_init_command_mock(INIT_COMMAND + ('--remote-path', 'borg1'))
module.initialize_repository(repository='repo', encryption_mode='repokey', remote_path='borg1')
+77
View File
@@ -0,0 +1,77 @@
import logging
from flexmock import flexmock
from borgmatic.borg import list as module
from ..test_verbosity import insert_logging_mock
LIST_COMMAND = ('borg', 'list', 'repo')
def test_list_archives_calls_borg_with_parameters():
flexmock(module).should_receive('execute_command').with_args(LIST_COMMAND, capture_output=False)
module.list_archives(repository='repo', storage_config={})
def test_list_archives_with_log_info_calls_borg_with_info_parameter():
flexmock(module).should_receive('execute_command').with_args(
LIST_COMMAND + ('--info',), capture_output=False
)
insert_logging_mock(logging.INFO)
module.list_archives(repository='repo', storage_config={})
def test_list_archives_with_log_debug_calls_borg_with_debug_parameter():
flexmock(module).should_receive('execute_command').with_args(
LIST_COMMAND + ('--debug', '--show-rc'), capture_output=False
)
insert_logging_mock(logging.DEBUG)
module.list_archives(repository='repo', storage_config={})
def test_list_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
storage_config = {'lock_wait': 5}
flexmock(module).should_receive('execute_command').with_args(
LIST_COMMAND + ('--lock-wait', '5'), capture_output=False
)
module.list_archives(repository='repo', storage_config=storage_config)
def test_list_archives_with_archive_calls_borg_with_archive_parameter():
storage_config = {}
flexmock(module).should_receive('execute_command').with_args(
('borg', 'list', 'repo::archive'), capture_output=False
)
module.list_archives(repository='repo', storage_config=storage_config, archive='archive')
def test_list_archives_with_local_path_calls_borg_via_local_path():
flexmock(module).should_receive('execute_command').with_args(
('borg1',) + LIST_COMMAND[1:], capture_output=False
)
module.list_archives(repository='repo', storage_config={}, local_path='borg1')
def test_list_archives_with_remote_path_calls_borg_with_remote_path_parameters():
flexmock(module).should_receive('execute_command').with_args(
LIST_COMMAND + ('--remote-path', 'borg1'), capture_output=False
)
module.list_archives(repository='repo', storage_config={}, remote_path='borg1')
def test_list_archives_with_json_calls_borg_with_json_parameter():
flexmock(module).should_receive('execute_command').with_args(
LIST_COMMAND + ('--json',), capture_output=True
).and_return('[]')
json_output = module.list_archives(repository='repo', storage_config={}, json=True)
assert json_output == '[]'
+160
View File
@@ -0,0 +1,160 @@
import logging
from collections import OrderedDict
from flexmock import flexmock
from borgmatic.borg import prune as module
from ..test_verbosity import insert_logging_mock
def insert_subprocess_mock(check_call_command, **kwargs):
subprocess = flexmock(module.subprocess)
subprocess.should_receive('check_call').with_args(check_call_command, **kwargs).once()
BASE_PRUNE_FLAGS = (('--keep-daily', '1'), ('--keep-weekly', '2'), ('--keep-monthly', '3'))
def test_make_prune_flags_returns_flags_from_config_plus_default_prefix():
retention_config = OrderedDict((('keep_daily', 1), ('keep_weekly', 2), ('keep_monthly', 3)))
result = module._make_prune_flags(retention_config)
assert tuple(result) == BASE_PRUNE_FLAGS + (('--prefix', '{hostname}-'),)
def test_make_prune_flags_accepts_prefix_with_placeholders():
retention_config = OrderedDict((('keep_daily', 1), ('prefix', 'Documents_{hostname}-{now}')))
result = module._make_prune_flags(retention_config)
expected = (('--keep-daily', '1'), ('--prefix', 'Documents_{hostname}-{now}'))
assert tuple(result) == expected
PRUNE_COMMAND = (
'borg',
'prune',
'repo',
'--keep-daily',
'1',
'--keep-weekly',
'2',
'--keep-monthly',
'3',
)
def test_prune_archives_calls_borg_with_parameters():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS
)
insert_subprocess_mock(PRUNE_COMMAND)
module.prune_archives(
dry_run=False, repository='repo', storage_config={}, retention_config=retention_config
)
def test_prune_archives_with_log_info_calls_borg_with_info_parameter():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS
)
insert_subprocess_mock(PRUNE_COMMAND + ('--stats', '--info'))
insert_logging_mock(logging.INFO)
module.prune_archives(
repository='repo', storage_config={}, dry_run=False, retention_config=retention_config
)
def test_prune_archives_with_log_debug_calls_borg_with_debug_parameter():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS
)
insert_subprocess_mock(PRUNE_COMMAND + ('--stats', '--debug', '--list', '--show-rc'))
insert_logging_mock(logging.DEBUG)
module.prune_archives(
repository='repo', storage_config={}, dry_run=False, retention_config=retention_config
)
def test_prune_archives_with_dry_run_calls_borg_with_dry_run_parameter():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS
)
insert_subprocess_mock(PRUNE_COMMAND + ('--dry-run',))
module.prune_archives(
repository='repo', storage_config={}, dry_run=True, retention_config=retention_config
)
def test_prune_archives_with_local_path_calls_borg_via_local_path():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS
)
insert_subprocess_mock(('borg1',) + PRUNE_COMMAND[1:])
module.prune_archives(
dry_run=False,
repository='repo',
storage_config={},
retention_config=retention_config,
local_path='borg1',
)
def test_prune_archives_with_remote_path_calls_borg_with_remote_path_parameters():
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS
)
insert_subprocess_mock(PRUNE_COMMAND + ('--remote-path', 'borg1'))
module.prune_archives(
dry_run=False,
repository='repo',
storage_config={},
retention_config=retention_config,
remote_path='borg1',
)
def test_prune_archives_with_umask_calls_borg_with_umask_parameters():
storage_config = {'umask': '077'}
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS
)
insert_subprocess_mock(PRUNE_COMMAND + ('--umask', '077'))
module.prune_archives(
dry_run=False,
repository='repo',
storage_config=storage_config,
retention_config=retention_config,
)
def test_prune_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
storage_config = {'lock_wait': 5}
retention_config = flexmock()
flexmock(module).should_receive('_make_prune_flags').with_args(retention_config).and_return(
BASE_PRUNE_FLAGS
)
insert_subprocess_mock(PRUNE_COMMAND + ('--lock-wait', '5'))
module.prune_archives(
dry_run=False,
repository='repo',
storage_config=storage_config,
retention_config=retention_config,
)
View File
+101
View File
@@ -0,0 +1,101 @@
from flexmock import flexmock
from borgmatic.commands import borgmatic as module
def test_collect_configuration_run_summary_logs_info_for_success():
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
flexmock(module).should_receive('run_configuration').and_return([])
args = flexmock(extract=False, list=False)
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
assert all(log for log in logs if log.levelno == module.logging.INFO)
def test_collect_configuration_run_summary_logs_info_for_success_with_extract():
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
flexmock(module.validate).should_receive('guard_configuration_contains_repository')
flexmock(module).should_receive('run_configuration').and_return([])
args = flexmock(extract=True, list=False, repository='repo')
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
assert all(log for log in logs if log.levelno == module.logging.INFO)
def test_collect_configuration_run_summary_logs_critical_for_extract_with_repository_error():
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
flexmock(module.validate).should_receive('guard_configuration_contains_repository').and_raise(
ValueError
)
args = flexmock(extract=True, list=False, repository='repo')
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
def test_collect_configuration_run_summary_logs_critical_for_list_with_archive_and_repository_error():
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
flexmock(module.validate).should_receive('guard_configuration_contains_repository').and_raise(
ValueError
)
args = flexmock(extract=False, list=True, repository='repo', archive='test')
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
def test_collect_configuration_run_summary_logs_info_for_success_with_list():
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
flexmock(module).should_receive('run_configuration').and_return([])
args = flexmock(extract=False, list=True, repository='repo', archive=None)
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
assert all(log for log in logs if log.levelno == module.logging.INFO)
def test_collect_configuration_run_summary_logs_critical_for_parse_error():
flexmock(module.validate).should_receive('parse_configuration').and_raise(ValueError)
args = flexmock(extract=False, list=False)
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
def test_collect_configuration_run_summary_logs_critical_for_run_error():
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
flexmock(module.validate).should_receive('guard_configuration_contains_repository')
flexmock(module).should_receive('run_configuration').and_raise(ValueError)
args = flexmock(extract=False, list=False)
logs = tuple(module.collect_configuration_run_summary_logs(('test.yaml',), args=args))
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
def test_collect_configuration_run_summary_logs_critical_for_missing_configs():
flexmock(module.validate).should_receive('parse_configuration').and_return({'test.yaml': {}})
flexmock(module).should_receive('run_configuration').and_return([])
args = flexmock(config_paths=(), extract=False, list=False)
logs = tuple(module.collect_configuration_run_summary_logs(config_filenames=(), args=args))
assert any(log for log in logs if log.levelno == module.logging.CRITICAL)
def test_collect_configuration_run_summary_logs_outputs_merged_json_results():
flexmock(module.validate).should_receive('parse_configuration').and_return(
{'test.yaml': {}, 'test2.yaml': {}}
)
flexmock(module).should_receive('run_configuration').and_return(['foo', 'bar']).and_return(
['baz']
)
flexmock(module.sys.stdout).should_receive('write').with_args('["foo", "bar", "baz"]').once()
args = flexmock(extract=False, list=False)
tuple(module.collect_configuration_run_summary_logs(('test.yaml', 'test2.yaml'), args=args))
View File
+23
View File
@@ -0,0 +1,23 @@
from borgmatic.config import checks as module
def test_repository_enabled_for_checks_defaults_to_enabled_for_all_repositories():
enabled = module.repository_enabled_for_checks('repo.borg', consistency={})
assert enabled
def test_repository_enabled_for_checks_is_enabled_for_specified_repositories():
enabled = module.repository_enabled_for_checks(
'repo.borg', consistency={'check_repositories': ['repo.borg', 'other.borg']}
)
assert enabled
def test_repository_enabled_for_checks_is_disabled_for_other_repositories():
enabled = module.repository_enabled_for_checks(
'repo.borg', consistency={'check_repositories': ['other.borg']}
)
assert not enabled

Some files were not shown because too many files have changed in this diff Show More