Compare commits

...
57 Commits
Author SHA1 Message Date
Dan Helfman fdb353d358 Bump version for release. 2023-11-19 21:14:56 -08:00
Dan Helfman 3b99f7c75a Constants support includes and command-line overrides (#745, #782) 2023-11-19 21:13:35 -08:00
Dan Helfman 8b9abc6cf8 Documentation clarifications (#791). 2023-11-15 09:05:52 -08:00
Dan Helfman da034c316a Add another mention of "skip_actions" to the docs (#701). 2023-11-08 18:22:17 -08:00
Dan Helfman 08d01d8bcd Documentation formatting. 2023-11-08 17:57:31 -08:00
Dan Helfman eef69e23ee Document the possible units of times for a configured check frequency. 2023-11-08 17:53:59 -08:00
Dan Helfman 26bb54a9dd Remove broken link in documentation (#786). 2023-11-08 10:26:09 -08:00
Dan Helfman 715e2ac127 Add test support for Python 3.12. 2023-11-07 10:17:55 -08:00
Dan Helfman f39cea4abf Remove additional Python 3.7-isms (#784). 2023-11-07 10:17:16 -08:00
Dan Helfman 22101bdd49 Drop support for Python 3.7, which has been end-of-lifed (#784). 2023-11-07 10:11:29 -08:00
Dan Helfman 13cf863d89 Fix tests (#783). 2023-11-07 10:09:31 -08:00
Dan Helfman dcf25fa041 Upgrade ruamel.yaml dependency to support version 0.18.x (#783). 2023-11-07 10:00:13 -08:00
Dan Helfman 12b75f9075 Update documentation about logging changes from version 1.8.3 (#665). 2023-11-06 21:13:46 -08:00
Dan HelfmanandGitHub 9baf06a2f7 Fix typo.
Typo
2023-11-04 08:56:39 -07:00
tdltdc 56302e22cd Typo 2023-11-04 15:05:19 +01:00
Dan Helfman 6cc93c4eb9 Fix environment variable interpolation within configured repository paths (#782). 2023-11-03 21:16:04 -07:00
Dan Helfman 2da43239f6 Fix docs: minor typos (#781).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/781
2023-11-03 00:59:29 +00:00
debuglevel 4beef36d3c Update docs/how-to/inspect-your-backups.md 2023-11-02 23:14:31 +00:00
debuglevel eacfbd742b Typo 2023-11-02 23:13:45 +00:00
debuglevel 82a85986b6 Typo 2023-11-02 22:57:46 +00:00
Dan Helfman ef448e2dd1 Add a "skip_actions" option to skip running particular actions (#701). 2023-10-31 21:54:41 -07:00
Dan Helfman c3efe1b90e Only parse "--override" values as complex data types when they're for options of those types (#779). 2023-10-29 19:02:28 -07:00
Dan Helfman d85c1ee216 Correct changelog addition (#779). 2023-10-29 16:25:40 -07:00
Dan Helfman b47088067c Add a "--match-archives" flag to the "check" action (#779). 2023-10-29 16:22:39 -07:00
Dan Helfman c5732aa4fc Fix home page CSS layout to prevent overflow at certain window widths (#777). 2023-10-27 14:12:35 -07:00
Dan Helfman a0323d9d6c Bump version for release. 2023-10-26 22:20:26 -07:00
Dan Helfman 8ad7b473f1 When an archive filter causes no matching archives for the "rlist" or "info" actions, warn (#748). 2023-10-26 22:12:13 -07:00
Dan Helfman 895a0ccb3c Upgrade to tox 4. (Now a minimum requirement.) 2023-10-23 17:39:27 -07:00
Dan Helfman 257ab77bea Disallow the "--dry-run" flag with the "borg" action (#774). 2023-10-23 17:23:04 -07:00
Dan HelfmanandGitHub dccaa4014b Update systemd service example with better filesystem protection options.
Merge pull request #78 from Alphix/update-systemd-service
2023-10-15 08:59:39 -07:00
David Härdeman 2f3c0bec5b Update systemd .service example
First, ProtectSystem=strict will make the entire file system hierarchy (except
/dev, /proc/ and /sys) read-only, so separate ReadOnlyPaths= is not necessary.

Second, ProtectHome=tmpfs will not just mount an empty tmpfs on /root, but also
on /home and /run/user. As it's likely quite common to want to backup /home,
this seems like a footgun.

Finally, it's quite likely that borgbackup will want access to root's SSH keys
in order to connect to remote backup servers.

Note that all these options are commented out by default, so this is more of
a documentation change than any real change in functionality.
2023-10-15 11:30:11 +02:00
Dan Helfman 487d8ffd32 Fix normalization of deprecated sections to support empty sections without erroring (#771). 2023-10-14 13:04:18 -07:00
Dan Helfman 30523a7c89 Update home page example of Healthchecks configuration not to use deprecated config. 2023-10-11 12:56:21 -07:00
Dan Helfman 77b1907d03 Update Healthchecks deprecation warning message for clarity. 2023-10-11 12:17:57 -07:00
Dan Helfman 09594c85bf Be more explicit in documentation that you don't have to use an environment variable for passphrases. 2023-10-10 09:34:55 -07:00
Dan Helfman e07efdf68f Add documentation note about using includes for specifying passphrases (#769). 2023-10-10 09:16:58 -07:00
Dan Helfman 1fed44f905 Add documentation note about sudo and sudoers "secure_path" option (#757). 2023-10-09 14:15:54 -07:00
Dan Helfman c687dafdd2 Fix a traceback when an invalid command-line flag or action is used (#768). 2023-10-06 21:00:23 -07:00
Dan Helfman 3eff2c4248 Add Grafana Loki badge to integrations documentation. 2023-10-05 09:06:06 -07:00
Dan Helfman d94fdb6faf Add apprise logo to integrations in readme (#715).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/765
2023-10-05 15:51:04 +00:00
Pim Kunis a83282faf0 add apprise logo to integrations in readme 2023-10-05 15:38:32 +02:00
Dan Helfman e7169f6fb2 Upgrade certifi test dependency to fix security alert. 2023-10-04 22:59:15 -07:00
Dan Helfman 9587fc2366 Update Apprise documentation to use sudo for pipx install (#715). 2023-10-04 22:54:11 -07:00
Dan Helfman 5f06884d5a Fix Apprise/PyYAML end-to-end test breakage (#715). 2023-10-04 22:51:05 -07:00
Dan Helfman f011431463 Apprise hook documentation (#715). 2023-10-04 19:23:53 -07:00
Dan Helfman 9e14f209f1 Merge branch 'main' of ssh://projects.torsion.org:3022/borgmatic-collective/borgmatic 2023-10-04 14:58:48 -07:00
Dan Helfman 9d34d2eec5 Support for Apprise (#759).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/759
2023-10-04 21:58:20 +00:00
Pim Kunis 7a9625cd44 fix PR comments 2023-10-04 13:19:40 +02:00
Pim Kunisandpizzapim 4763c323d0 add unit tests for apprise hook 2023-10-01 16:59:59 +00:00
Pim Kunisandpizzapim eaa22be3db fix PR comments 2023-10-01 16:59:59 +00:00
Pim Kunisandpizzapim a587e207f9 pin Apprise dependencies for test requirements 2023-10-01 16:59:59 +00:00
Pim Kunisandpizzapim db8079b699 fix typo in setup.py
handle if apprise cannot be imported
2023-10-01 16:59:59 +00:00
Pim Kunisandpizzapim 5a989826a1 convert map to list for apprise function call
fix apprise config schema
remove apprise from required dependencies
2023-10-01 16:59:59 +00:00
Pim Kunisandpizzapim 21f4266273 incorporate PR review comments 2023-10-01 16:59:59 +00:00
Pim Kunisandpizzapim e7252c7545 remove comments about tags 2023-10-01 16:59:59 +00:00
Pim Kunisandpizzapim 86011c8418 default apprise notify type per borgmatic state 2023-10-01 16:59:59 +00:00
Pim Kunisandpizzapim f3295ccb4a add support for apprise 2023-10-01 16:59:59 +00:00
61 changed files with 1714 additions and 628 deletions
+31 -1
View File
@@ -1,10 +1,40 @@
1.8.5
* #701: Add a "skip_actions" option to skip running particular actions, handy for append-only or
checkless configurations. See the documentation for more information:
https://torsion.org/borgmatic/docs/how-to/set-up-backups/#skipping-actions
* #701: Deprecate the "disabled" value for the "checks" option in favor of the new "skip_actions"
option.
* #745: Constants now apply to included configuration, not just the file doing the includes. As a
side effect of this change, constants no longer apply to option names and only substitute into
configuration values.
* #779: Add a "--match-archives" flag to the "check" action for selecting the archives to check,
overriding the existing "archive_name_format" and "match_archives" options in configuration.
* #779: Only parse "--override" values as complex data types when they're for options of those
types.
* #782: Fix environment variable interpolation within configured repository paths.
* #782: Add configuration constant overriding via the existing "--override" flag.
* #783: Upgrade ruamel.yaml dependency to support version 0.18.x.
* #784: Drop support for Python 3.7, which has been end-of-lifed.
1.8.4
* #715: Add a monitoring hook for sending backup status to a variety of monitoring services via the
Apprise library. See the documentation for more information:
https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#apprise-hook
* #748: When an archive filter causes no matching archives for the "rlist" or "info"
actions, warn the user and suggest how to remove the filter.
* #768: Fix a traceback when an invalid command-line flag or action is used.
* #771: Fix normalization of deprecated sections ("location:", "storage:", "hooks:", etc.) to
support empty sections without erroring.
* #774: Disallow the "--dry-run" flag with the "borg" action, as borgmatic can't guarantee the Borg
command won't have side effects.
1.8.3
* #665: BREAKING: Simplify logging logic as follows: Syslog verbosity is now disabled by
default, but setting the "--syslog-verbosity" flag enables it regardless of whether you're at an
interactive console. Additionally, "--log-file-verbosity" and "--monitoring-verbosity" now
default to 1 (info about steps borgmatic is taking) instead of 0. And both syslog logging and
file logging can be enabled simultaneously.
* #743: Add a monitoring hook for sending backup status and logs to to Grafana Loki. See the
* #743: Add a monitoring hook for sending backup status and logs to Grafana Loki. See the
documentation for more information:
https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#loki-hook
* #753: When "archive_name_format" is not set, filter archives using the default archive name
+15 -12
View File
@@ -48,24 +48,27 @@ postgresql_databases:
- name: users
# Third-party services to notify you if backups aren't happening.
healthchecks: https://hc-ping.com/be067061-cf96-4412-8eae-62b0c50d6a8c
healthchecks:
ping_url: https://hc-ping.com/be067061-cf96-4412-8eae-62b0c50d6a8c
```
borgmatic is powered by [Borg Backup](https://www.borgbackup.org/).
## Integrations
<a href="https://www.postgresql.org/"><img src="docs/static/postgresql.png" alt="PostgreSQL" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://www.mysql.com/"><img src="docs/static/mysql.png" alt="MySQL" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://mariadb.com/"><img src="docs/static/mariadb.png" alt="MariaDB" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://www.mongodb.com/"><img src="docs/static/mongodb.png" alt="MongoDB" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://sqlite.org/"><img src="docs/static/sqlite.png" alt="SQLite" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://healthchecks.io/"><img src="docs/static/healthchecks.png" alt="Healthchecks" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://cronitor.io/"><img src="docs/static/cronitor.png" alt="Cronitor" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://cronhub.io/"><img src="docs/static/cronhub.png" alt="Cronhub" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://www.pagerduty.com/"><img src="docs/static/pagerduty.png" alt="PagerDuty" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://ntfy.sh/"><img src="docs/static/ntfy.png" alt="ntfy" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://www.borgbase.com/?utm_source=borgmatic"><img src="docs/static/borgbase.png" alt="BorgBase" height="60px" style="margin-bottom:20px;"></a>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="https://www.postgresql.org/"><img src="docs/static/postgresql.png" alt="PostgreSQL" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://www.mysql.com/"><img src="docs/static/mysql.png" alt="MySQL" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://mariadb.com/"><img src="docs/static/mariadb.png" alt="MariaDB" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://www.mongodb.com/"><img src="docs/static/mongodb.png" alt="MongoDB" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://sqlite.org/"><img src="docs/static/sqlite.png" alt="SQLite" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://healthchecks.io/"><img src="docs/static/healthchecks.png" alt="Healthchecks" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://cronitor.io/"><img src="docs/static/cronitor.png" alt="Cronitor" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://cronhub.io/"><img src="docs/static/cronhub.png" alt="Cronhub" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://www.pagerduty.com/"><img src="docs/static/pagerduty.png" alt="PagerDuty" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://ntfy.sh/"><img src="docs/static/ntfy.png" alt="ntfy" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://grafana.com/oss/loki/"><img src="docs/static/loki.png" alt="Loki" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://github.com/caronc/apprise/wiki"><img src="docs/static/apprise.png" alt="Apprise" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://www.borgbase.com/?utm_source=borgmatic"><img src="docs/static/borgbase.png" alt="BorgBase" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
## Getting started
+1 -4
View File
@@ -39,13 +39,10 @@ def run_check(
repository['path'],
config,
local_borg_version,
check_arguments,
global_arguments,
local_path=local_path,
remote_path=remote_path,
progress=check_arguments.progress,
repair=check_arguments.repair,
only_checks=check_arguments.only,
force=check_arguments.force,
)
borgmatic.hooks.command.execute_hook(
config.get('after_check'),
+2 -6
View File
@@ -1,12 +1,8 @@
import importlib.metadata
import json
import logging
import os
try:
import importlib_metadata
except ModuleNotFoundError: # pragma: nocover
import importlib.metadata as importlib_metadata
import borgmatic.borg.create
import borgmatic.borg.state
import borgmatic.config.validate
@@ -39,7 +35,7 @@ def create_borgmatic_manifest(config, config_paths, dry_run):
with open(borgmatic_manifest_path, 'w') as config_list_file:
json.dump(
{
'borgmatic_version': importlib_metadata.version('borgmatic'),
'borgmatic_version': importlib.metadata.version('borgmatic'),
'config_paths': config_paths,
},
config_list_file,
+24 -18
View File
@@ -39,7 +39,11 @@ def parse_checks(config, only_checks=None):
check_config['name'] for check_config in (config.get('checks', None) or DEFAULT_CHECKS)
)
checks = tuple(check.lower() for check in checks)
if 'disabled' in checks:
logger.warning(
'The "disabled" value for the "checks" option is deprecated and will be removed from a future release; use "skip_actions" instead'
)
if len(checks) > 1:
logger.warning(
'Multiple checks are configured, but one of them is "disabled"; not running any checks'
@@ -119,6 +123,9 @@ def filter_checks_on_frequency(
Raise ValueError if a frequency cannot be parsed.
'''
if not checks:
return checks
filtered_checks = list(checks)
if force:
@@ -149,11 +156,13 @@ def filter_checks_on_frequency(
return tuple(filtered_checks)
def make_archive_filter_flags(local_borg_version, config, checks, check_last=None, prefix=None):
def make_archive_filter_flags(
local_borg_version, config, checks, check_arguments, check_last=None, prefix=None
):
'''
Given the local Borg version, a configuration dict, a parsed sequence of checks, the check last
value, and a consistency check prefix, transform the checks into tuple of command-line flags for
filtering archives in a check command.
Given the local Borg version, a configuration dict, a parsed sequence of checks, check arguments
as an argparse.Namespace instance, the check last value, and a consistency check prefix,
transform the checks into tuple of command-line flags for filtering archives in a check command.
If a check_last value is given and "archives" is in checks, then include a "--last" flag. And if
a prefix value is given and "archives" is in checks, then include a "--match-archives" flag.
@@ -168,7 +177,7 @@ def make_archive_filter_flags(local_borg_version, config, checks, check_last=Non
if prefix
else (
flags.make_match_archives_flags(
config.get('match_archives'),
check_arguments.match_archives or config.get('match_archives'),
config.get('archive_name_format'),
local_borg_version,
)
@@ -353,18 +362,15 @@ def check_archives(
repository_path,
config,
local_borg_version,
check_arguments,
global_arguments,
local_path='borg',
remote_path=None,
progress=None,
repair=None,
only_checks=None,
force=None,
):
'''
Given a local or remote repository path, a configuration dict, local/remote commands to run,
whether to include progress information, whether to attempt a repair, and an optional list of
checks to use instead of configured checks, check the contained Borg archives for consistency.
Given a local or remote repository path, a configuration dict, the local Borg version, check
arguments as an argparse.Namespace instance, global arguments, and local/remote commands to run,
check the contained Borg archives for consistency.
If there are no consistency checks to run, skip running them.
@@ -389,11 +395,11 @@ def check_archives(
check_last = config.get('check_last', None)
prefix = config.get('prefix')
configured_checks = parse_checks(config, only_checks)
configured_checks = parse_checks(config, check_arguments.only_checks)
lock_wait = None
extra_borg_options = config.get('extra_borg_options', {}).get('check', '')
archive_filter_flags = make_archive_filter_flags(
local_borg_version, config, configured_checks, check_last, prefix
local_borg_version, config, configured_checks, check_arguments, check_last, prefix
)
archives_check_id = make_archives_check_id(archive_filter_flags)
@@ -401,7 +407,7 @@ def check_archives(
config,
borg_repository_id,
configured_checks,
force,
check_arguments.force,
archives_check_id,
)
@@ -416,13 +422,13 @@ def check_archives(
full_command = (
(local_path, 'check')
+ (('--repair',) if repair else ())
+ (('--repair',) if check_arguments.repair else ())
+ make_check_flags(checks, archive_filter_flags)
+ (('--remote-path', remote_path) if remote_path else ())
+ (('--log-json',) if global_arguments.log_json else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ verbosity_flags
+ (('--progress',) if progress else ())
+ (('--progress',) if check_arguments.progress else ())
+ (tuple(extra_borg_options.split(' ')) if extra_borg_options else ())
+ flags.make_repository_flags(repository_path, local_borg_version)
)
@@ -431,7 +437,7 @@ def check_archives(
# The Borg repair option triggers an interactive prompt, which won't work when output is
# captured. And progress messes with the terminal directly.
if repair or progress:
if check_arguments.repair or check_arguments.progress:
execute_command(
full_command, output_file=DO_NOT_CAPTURE, extra_environment=borg_environment
)
+27
View File
@@ -1,8 +1,12 @@
import itertools
import json
import logging
import re
from borgmatic.borg import feature
logger = logging.getLogger(__name__)
def make_flags(name, value):
'''
@@ -86,3 +90,26 @@ def make_match_archives_flags(match_archives, archive_name_format, local_borg_ve
return ('--match-archives', f'sh:{derived_match_archives}')
else:
return ('--glob-archives', f'{derived_match_archives}')
def warn_for_aggressive_archive_flags(json_command, json_output):
'''
Given a JSON archives command and the resulting JSON string output from running it, parse the
JSON and warn if the command used an archive flag but the output indicates zero archives were
found.
'''
archive_flags_used = {'--glob-archives', '--match-archives'}.intersection(set(json_command))
if not archive_flags_used:
return
try:
if len(json.loads(json_output)['archives']) == 0:
logger.warning('An archive filter was applied, but no matching archives were found.')
logger.warning(
'Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration.'
)
except json.JSONDecodeError as error:
logger.debug(f'Cannot parse JSON output from archive command: {error}')
except (TypeError, KeyError):
logger.debug('Cannot parse JSON output from archive command: No "archives" key found')
+61 -23
View File
@@ -1,3 +1,4 @@
import argparse
import logging
import borgmatic.logger
@@ -7,24 +8,21 @@ from borgmatic.execute import execute_command, execute_command_and_capture_outpu
logger = logging.getLogger(__name__)
def display_archives_info(
def make_info_command(
repository_path,
config,
local_borg_version,
info_arguments,
global_arguments,
local_path='borg',
remote_path=None,
local_path,
remote_path,
):
'''
Given a local or remote repository path, a configuration dict, the local Borg version, global
arguments as an argparse.Namespace, and the arguments to the info action, display summary
information for Borg archives in the repository or return JSON summary information.
Given a local or remote repository path, a configuration dict, the local Borg version, the
arguments to the info action as an argparse.Namespace, and global arguments, return a command
as a tuple to display summary information for archives in the repository.
'''
borgmatic.logger.add_custom_log_levels()
lock_wait = config.get('lock_wait', None)
full_command = (
return (
(local_path, 'info')
+ (
('--info',)
@@ -38,7 +36,7 @@ def display_archives_info(
)
+ flags.make_flags('remote-path', remote_path)
+ flags.make_flags('log-json', global_arguments.log_json)
+ flags.make_flags('lock-wait', lock_wait)
+ flags.make_flags('lock-wait', config.get('lock_wait'))
+ (
(
flags.make_flags('match-archives', f'sh:{info_arguments.prefix}*')
@@ -62,16 +60,56 @@ def display_archives_info(
+ flags.make_repository_flags(repository_path, local_borg_version)
)
def display_archives_info(
repository_path,
config,
local_borg_version,
info_arguments,
global_arguments,
local_path='borg',
remote_path=None,
):
'''
Given a local or remote repository path, a configuration dict, the local Borg version, the
arguments to the info action as an argparse.Namespace, and global arguments, display summary
information for Borg archives in the repository or return JSON summary information.
'''
borgmatic.logger.add_custom_log_levels()
main_command = make_info_command(
repository_path,
config,
local_borg_version,
info_arguments,
global_arguments,
local_path,
remote_path,
)
json_command = make_info_command(
repository_path,
config,
local_borg_version,
argparse.Namespace(**dict(info_arguments.__dict__, json=True)),
global_arguments,
local_path,
remote_path,
)
json_info = execute_command_and_capture_output(
json_command,
extra_environment=environment.make_environment(config),
borg_local_path=local_path,
)
if info_arguments.json:
return execute_command_and_capture_output(
full_command,
extra_environment=environment.make_environment(config),
borg_local_path=local_path,
)
else:
execute_command(
full_command,
output_log_level=logging.ANSWER,
borg_local_path=local_path,
extra_environment=environment.make_environment(config),
)
return json_info
flags.warn_for_aggressive_archive_flags(json_command, json_info)
execute_command(
main_command,
output_log_level=logging.ANSWER,
borg_local_path=local_path,
extra_environment=environment.make_environment(config),
)
+24 -10
View File
@@ -1,3 +1,4 @@
import argparse
import logging
import borgmatic.logger
@@ -137,15 +138,28 @@ def list_repository(
local_path,
remote_path,
)
json_command = make_rlist_command(
repository_path,
config,
local_borg_version,
argparse.Namespace(**dict(rlist_arguments.__dict__, json=True)),
global_arguments,
local_path,
remote_path,
)
json_listing = execute_command_and_capture_output(
json_command, extra_environment=borg_environment, borg_local_path=local_path
)
if rlist_arguments.json:
return execute_command_and_capture_output(
main_command, extra_environment=borg_environment, borg_local_path=local_path
)
else:
execute_command(
main_command,
output_log_level=logging.ANSWER,
borg_local_path=local_path,
extra_environment=borg_environment,
)
return json_listing
flags.warn_for_aggressive_archive_flags(json_command, json_listing)
execute_command(
main_command,
output_log_level=logging.ANSWER,
borg_local_path=local_path,
extra_environment=borg_environment,
)
+11 -1
View File
@@ -604,11 +604,18 @@ def make_parsers():
action='store_true',
help='Attempt to repair any inconsistencies found (for interactive use)',
)
check_group.add_argument(
'-a',
'--match-archives',
'--glob-archives',
metavar='PATTERN',
help='Only check archives with names matching this pattern',
)
check_group.add_argument(
'--only',
metavar='CHECK',
choices=('repository', 'archives', 'data', 'extract'),
dest='only',
dest='only_checks',
action='append',
help='Run a particular consistency check (repository, archives, data, or extract) instead of configured checks (subject to configured frequency, can specify flag multiple times)',
)
@@ -1328,4 +1335,7 @@ def parse_arguments(*unparsed_arguments):
'With the info action, only one of --archive, --prefix, or --match-archives flags can be used.'
)
if 'borg' in arguments and arguments['global'].dry_run:
raise ValueError('With the borg action, --dry-run is not supported.')
return arguments
+26 -23
View File
@@ -1,4 +1,5 @@
import collections
import importlib.metadata
import json
import logging
import os
@@ -9,11 +10,6 @@ from subprocess import CalledProcessError
import colorama
try:
import importlib_metadata
except ModuleNotFoundError: # pragma: nocover
import importlib.metadata as importlib_metadata
import borgmatic.actions.borg
import borgmatic.actions.break_lock
import borgmatic.actions.check
@@ -70,6 +66,12 @@ def run_configuration(config_filename, config, arguments):
using_primary_action = {'create', 'prune', 'compact', 'check'}.intersection(arguments)
monitoring_log_level = verbosity_to_log_level(global_arguments.monitoring_verbosity)
monitoring_hooks_are_activated = using_primary_action and monitoring_log_level != DISABLED
skip_actions = config.get('skip_actions')
if skip_actions:
logger.debug(
f"{config_filename}: Skipping {'/'.join(skip_actions)} action{'s' if len(skip_actions) > 1 else ''} due to configured skip_actions"
)
try:
local_borg_version = borg_version.local_borg_version(config, local_path)
@@ -274,6 +276,7 @@ def run_actions(
'repositories': ','.join([repo['path'] for repo in config['repositories']]),
'log_file': global_arguments.log_file if global_arguments.log_file else '',
}
skip_actions = set(config.get('skip_actions', {}))
command.execute_hook(
config.get('before_actions'),
@@ -285,7 +288,7 @@ def run_actions(
)
for action_name, action_arguments in arguments.items():
if action_name == 'rcreate':
if action_name == 'rcreate' and action_name not in skip_actions:
borgmatic.actions.rcreate.run_rcreate(
repository,
config,
@@ -295,7 +298,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'transfer':
elif action_name == 'transfer' and action_name not in skip_actions:
borgmatic.actions.transfer.run_transfer(
repository,
config,
@@ -305,7 +308,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'create':
elif action_name == 'create' and action_name not in skip_actions:
yield from borgmatic.actions.create.run_create(
config_filename,
repository,
@@ -318,7 +321,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'prune':
elif action_name == 'prune' and action_name not in skip_actions:
borgmatic.actions.prune.run_prune(
config_filename,
repository,
@@ -331,7 +334,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'compact':
elif action_name == 'compact' and action_name not in skip_actions:
borgmatic.actions.compact.run_compact(
config_filename,
repository,
@@ -344,7 +347,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'check':
elif action_name == 'check' and action_name not in skip_actions:
if checks.repository_enabled_for_checks(repository, config):
borgmatic.actions.check.run_check(
config_filename,
@@ -357,7 +360,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'extract':
elif action_name == 'extract' and action_name not in skip_actions:
borgmatic.actions.extract.run_extract(
config_filename,
repository,
@@ -369,7 +372,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'export-tar':
elif action_name == 'export-tar' and action_name not in skip_actions:
borgmatic.actions.export_tar.run_export_tar(
repository,
config,
@@ -379,7 +382,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'mount':
elif action_name == 'mount' and action_name not in skip_actions:
borgmatic.actions.mount.run_mount(
repository,
config,
@@ -389,7 +392,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'restore':
elif action_name == 'restore' and action_name not in skip_actions:
borgmatic.actions.restore.run_restore(
repository,
config,
@@ -399,7 +402,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'rlist':
elif action_name == 'rlist' and action_name not in skip_actions:
yield from borgmatic.actions.rlist.run_rlist(
repository,
config,
@@ -409,7 +412,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'list':
elif action_name == 'list' and action_name not in skip_actions:
yield from borgmatic.actions.list.run_list(
repository,
config,
@@ -419,7 +422,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'rinfo':
elif action_name == 'rinfo' and action_name not in skip_actions:
yield from borgmatic.actions.rinfo.run_rinfo(
repository,
config,
@@ -429,7 +432,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'info':
elif action_name == 'info' and action_name not in skip_actions:
yield from borgmatic.actions.info.run_info(
repository,
config,
@@ -439,7 +442,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'break-lock':
elif action_name == 'break-lock' and action_name not in skip_actions:
borgmatic.actions.break_lock.run_break_lock(
repository,
config,
@@ -449,7 +452,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'export':
elif action_name == 'export' and action_name not in skip_actions:
borgmatic.actions.export_key.run_export_key(
repository,
config,
@@ -459,7 +462,7 @@ def run_actions(
local_path,
remote_path,
)
elif action_name == 'borg':
elif action_name == 'borg' and action_name not in skip_actions:
borgmatic.actions.borg.run_borg(
repository,
config,
@@ -827,7 +830,7 @@ def main(extra_summary_logs=[]): # pragma: no cover
global_arguments = arguments['global']
if global_arguments.version:
print(importlib_metadata.version('borgmatic'))
print(importlib.metadata.version('borgmatic'))
sys.exit(0)
if global_arguments.bash_completion:
print(borgmatic.commands.completion.bash.bash_completion())
+5 -5
View File
@@ -1,9 +1,9 @@
def repository_enabled_for_checks(repository, consistency):
def repository_enabled_for_checks(repository, config):
'''
Given a repository name and a consistency configuration dict, return whether the repository
is enabled to have consistency checks run.
Given a repository name and a configuration dict, return whether the
repository is enabled to have consistency checks run.
'''
if not consistency.get('check_repositories'):
if not config.get('check_repositories'):
return True
return repository in consistency['check_repositories']
return repository in config['check_repositories']
+47
View File
@@ -0,0 +1,47 @@
def coerce_scalar(value):
'''
Given a configuration value, coerce it to an integer or a boolean as appropriate and return the
result.
'''
try:
return int(value)
except (TypeError, ValueError):
pass
if value == 'true' or value == 'True':
return True
if value == 'false' or value == 'False':
return False
return value
def apply_constants(value, constants):
'''
Given a configuration value (bool, dict, int, list, or string) and a dict of named constants,
replace any configuration string values of the form "{constant}" (or containing it) with the
value of the correspondingly named key from the constants. Recurse as necessary into nested
configuration to find values to replace.
For instance, if a configuration value contains "{foo}", replace it with the value of the "foo"
key found within the configuration's "constants".
Return the configuration value and modify the original.
'''
if not value or not constants:
return value
if isinstance(value, str):
for constant_name, constant_value in constants.items():
value = value.replace('{' + constant_name + '}', str(constant_value))
# Support constants within non-string scalars by coercing the value to its appropriate type.
value = coerce_scalar(value)
elif isinstance(value, list):
for index, list_value in enumerate(value):
value[index] = apply_constants(list_value, constants)
elif isinstance(value, dict):
for option_name, option_value in value.items():
value[option_name] = apply_constants(option_value, constants)
return value
+20 -14
View File
@@ -1,21 +1,22 @@
import os
import re
_VARIABLE_PATTERN = re.compile(
VARIABLE_PATTERN = re.compile(
r'(?P<escape>\\)?(?P<variable>\$\{(?P<name>[A-Za-z0-9_]+)((:?-)(?P<default>[^}]+))?\})'
)
def _resolve_string(matcher):
def resolve_string(matcher):
'''
Get the value from environment given a matcher containing a name and an optional default value.
If the variable is not defined in environment and no default value is provided, an Error is raised.
Given a matcher containing a name and an optional default value, get the value from environment.
Raise ValueError if the variable is not defined in environment and no default value is provided.
'''
if matcher.group('escape') is not None:
# in case of escaped envvar, unescape it
# In the case of an escaped environment variable, unescape it.
return matcher.group('variable')
# resolve the env var
# Resolve the environment variable.
name, default = matcher.group('name'), matcher.group('default')
out = os.getenv(name, default=default)
@@ -27,19 +28,24 @@ def _resolve_string(matcher):
def resolve_env_variables(item):
'''
Resolves variables like or ${FOO} from given configuration with values from process environment
Supported formats:
- ${FOO} will return FOO env variable
- ${FOO-bar} or ${FOO:-bar} will return FOO env variable if it exists, else "bar"
Resolves variables like or ${FOO} from given configuration with values from process environment.
If any variable is missing in environment and no default value is provided, an Error is raised.
Supported formats:
* ${FOO} will return FOO env variable
* ${FOO-bar} or ${FOO:-bar} will return FOO env variable if it exists, else "bar"
Raise if any variable is missing in environment and no default value is provided.
'''
if isinstance(item, str):
return _VARIABLE_PATTERN.sub(_resolve_string, item)
return VARIABLE_PATTERN.sub(resolve_string, item)
if isinstance(item, list):
for i, subitem in enumerate(item):
item[i] = resolve_env_variables(subitem)
for index, subitem in enumerate(item):
item[index] = resolve_env_variables(subitem)
if isinstance(item, dict):
for key, value in item.items():
item[key] = resolve_env_variables(value)
return item
+10 -8
View File
@@ -3,7 +3,7 @@ import io
import os
import re
from ruamel import yaml
import ruamel.yaml
from borgmatic.config import load, normalize
@@ -17,7 +17,7 @@ def insert_newline_before_comment(config, field_name):
field and its comments.
'''
config.ca.items[field_name][1].insert(
0, yaml.tokens.CommentToken('\n', yaml.error.CommentMark(0), None)
0, ruamel.yaml.tokens.CommentToken('\n', ruamel.yaml.error.CommentMark(0), None)
)
@@ -32,12 +32,12 @@ def schema_to_sample_configuration(schema, level=0, parent_is_sequence=False):
return example
if schema_type == 'array':
config = yaml.comments.CommentedSeq(
config = ruamel.yaml.comments.CommentedSeq(
[schema_to_sample_configuration(schema['items'], level, parent_is_sequence=True)]
)
add_comments_to_configuration_sequence(config, schema, indent=(level * INDENT))
elif schema_type == 'object':
config = yaml.comments.CommentedMap(
config = ruamel.yaml.comments.CommentedMap(
[
(field_name, schema_to_sample_configuration(sub_schema, level + 1))
for field_name, sub_schema in schema['properties'].items()
@@ -101,7 +101,7 @@ def render_configuration(config):
'''
Given a config data structure of nested OrderedDicts, render the config as YAML and return it.
'''
dumper = yaml.YAML()
dumper = ruamel.yaml.YAML(typ='rt')
dumper.indent(mapping=INDENT, sequence=INDENT + SEQUENCE_INDENT, offset=INDENT)
rendered = io.StringIO()
dumper.dump(config, rendered)
@@ -236,7 +236,9 @@ def merge_source_configuration_into_destination(destination_config, source_confi
for field_name, source_value in source_config.items():
# Since this key/value is from the source configuration, leave it uncommented and remove any
# sentinel that would cause it to get commented out.
remove_commented_out_sentinel(destination_config, field_name)
remove_commented_out_sentinel(
ruamel.yaml.comments.CommentedMap(destination_config), field_name
)
# This is a mapping. Recurse for this key/value.
if isinstance(source_value, collections.abc.Mapping):
@@ -248,7 +250,7 @@ def merge_source_configuration_into_destination(destination_config, source_confi
# This is a sequence. Recurse for each item in it.
if isinstance(source_value, collections.abc.Sequence) and not isinstance(source_value, str):
destination_value = destination_config[field_name]
destination_config[field_name] = yaml.comments.CommentedSeq(
destination_config[field_name] = ruamel.yaml.comments.CommentedSeq(
[
merge_source_configuration_into_destination(
destination_value[index] if index < len(destination_value) else None,
@@ -275,7 +277,7 @@ def generate_sample_configuration(
schema. If a source filename is provided, merge the parsed contents of that configuration into
the generated configuration.
'''
schema = yaml.round_trip_load(open(schema_filename))
schema = ruamel.yaml.YAML(typ='safe').load(open(schema_filename))
source_config = None
if source_filename:
+2 -20
View File
@@ -1,6 +1,5 @@
import functools
import itertools
import json
import logging
import operator
import os
@@ -159,8 +158,7 @@ class Include_constructor(ruamel.yaml.SafeConstructor):
def load_configuration(filename):
'''
Load the given configuration file and return its contents as a data structure of nested dicts
and lists. Also, replace any "{constant}" strings with the value of the "constant" key in the
"constants" option of the configuration file.
and lists.
Raise ruamel.yaml.error.YAMLError if something goes wrong parsing the YAML, or RecursionError
if there are too many recursive includes.
@@ -179,23 +177,7 @@ def load_configuration(filename):
yaml.Constructor = Include_constructor_with_include_directory
with open(filename) as file:
file_contents = file.read()
config = yaml.load(file_contents)
try:
has_constants = bool(config and 'constants' in config)
except TypeError:
has_constants = False
if has_constants:
for key, value in config['constants'].items():
value = json.dumps(value)
file_contents = file_contents.replace(f'{{{key}}}', value.strip('"'))
config = yaml.load(file_contents)
del config['constants']
return config
return yaml.load(file.read())
def filter_omitted_nodes(nodes, values):
+2 -2
View File
@@ -39,7 +39,7 @@ def normalize_sections(config_filename, config):
for section_name in ('location', 'storage', 'retention', 'consistency', 'output', 'hooks'):
section_config = config.get(section_name)
if section_config:
if section_config is not None:
any_section_upgraded = True
del config[section_name]
config.update(section_config)
@@ -90,7 +90,7 @@ def normalize(config_filename, config):
dict(
levelno=logging.WARNING,
levelname='WARNING',
msg=f'{config_filename}: The healthchecks hook now expects a mapping value. String values for this option are deprecated and support will be removed from a future release.',
msg=f'{config_filename}: The healthchecks hook now expects a key/value pair with "ping_url" as a key. String values for this option are deprecated and support will be removed from a future release.',
)
)
)
+44 -13
View File
@@ -22,13 +22,19 @@ def set_values(config, keys, value):
set_values(config[first_key], keys[1:], value)
def convert_value_type(value):
def convert_value_type(value, option_type):
'''
Given a string value, determine its logical type (string, boolean, integer, etc.), and return it
converted to that type.
Given a string value and its schema type as a string, determine its logical type (string,
boolean, integer, etc.), and return it converted to that type.
If the option type is a string, leave the value as a string so that special characters in it
don't get interpreted as YAML during conversion.
Raise ruamel.yaml.error.YAMLError if there's a parse issue with the YAML.
'''
if option_type == 'string':
return value
return ruamel.yaml.YAML(typ='safe').load(io.StringIO(value))
@@ -46,11 +52,32 @@ def strip_section_names(parsed_override_key):
return parsed_override_key
def parse_overrides(raw_overrides):
def type_for_option(schema, option_keys):
'''
Given a sequence of configuration file override strings in the form of "option.suboption=value",
parse and return a sequence of tuples (keys, values), where keys is a sequence of strings. For
instance, given the following raw overrides:
Given a configuration schema and a sequence of keys identifying an option, e.g.
('extra_borg_options', 'init'), return the schema type of that option as a string.
Return None if the option or its type cannot be found in the schema.
'''
option_schema = schema
for key in option_keys:
try:
option_schema = option_schema['properties'][key]
except KeyError:
return None
try:
return option_schema['type']
except KeyError:
return None
def parse_overrides(raw_overrides, schema):
'''
Given a sequence of configuration file override strings in the form of "option.suboption=value"
and a configuration schema dict, parse and return a sequence of tuples (keys, values), where
keys is a sequence of strings. For instance, given the following raw overrides:
['my_option.suboption=value1', 'other_option=value2']
@@ -71,10 +98,13 @@ def parse_overrides(raw_overrides):
for raw_override in raw_overrides:
try:
raw_keys, value = raw_override.split('=', 1)
keys = strip_section_names(tuple(raw_keys.split('.')))
option_type = type_for_option(schema, keys)
parsed_overrides.append(
(
strip_section_names(tuple(raw_keys.split('.'))),
convert_value_type(value),
keys,
convert_value_type(value, option_type),
)
)
except ValueError:
@@ -87,12 +117,13 @@ def parse_overrides(raw_overrides):
return tuple(parsed_overrides)
def apply_overrides(config, raw_overrides):
def apply_overrides(config, schema, raw_overrides):
'''
Given a configuration dict and a sequence of configuration file override strings in the form of
"option.suboption=value", parse each override and set it the configuration dict.
Given a configuration dict, a corresponding configuration schema dict, and a sequence of
configuration file override strings in the form of "option.suboption=value", parse each override
and set it into the configuration dict.
'''
overrides = parse_overrides(raw_overrides)
overrides = parse_overrides(raw_overrides, schema)
for keys, value in overrides:
set_values(config, keys, value)
+124 -14
View File
@@ -216,7 +216,7 @@ properties:
Store configuration files used to create a backup in the backup
itself. Defaults to true. Changing this to false prevents "borgmatic
bootstrap" from extracting configuration files from the backup.
example: true
example: false
source_directories_must_exist:
type: boolean
description: |
@@ -287,14 +287,17 @@ properties:
retry_wait:
type: integer
description: |
Wait time between retries (in seconds) to allow transient issues to
pass. Increases after each retry as a form of backoff. Defaults to 0
(no wait).
Wait time between retries (in seconds) to allow transient issues
to pass. Increases after each retry by that same wait time as a
form of backoff. Defaults to 0 (no wait).
example: 10
temporary_directory:
type: string
description: |
Directory where temporary files are stored. Defaults to $TMPDIR.
Directory where temporary Borg files are stored. Defaults to
$TMPDIR. See "Resource Usage" at
https://borgbackup.readthedocs.io/en/stable/usage/general.html for
details.
example: /path/to/tmpdir
ssh_command:
type: string
@@ -423,7 +426,9 @@ properties:
command-line invocation.
keep_within:
type: string
description: Keep all archives within this time interval.
description: |
Keep all archives within this time interval. See "skip_actions" for
disabling pruning altogether.
example: 3H
keep_secondly:
type: integer
@@ -479,13 +484,13 @@ properties:
- disabled
description: |
Name of consistency check to run: "repository",
"archives", "data", and/or "extract". Set to "disabled"
to disable all consistency checks. "repository" checks
the consistency of the repository, "archives" checks all
of the archives, "data" verifies the integrity of the
data within the archives, and "extract" does an
extraction dry-run of the most recent archive. Note that
"data" implies "archives".
"archives", "data", and/or "extract". "repository"
checks the consistency of the repository, "archives"
checks all of the archives, "data" verifies the
integrity of the data within the archives, and "extract"
does an extraction dry-run of the most recent archive.
Note that "data" implies "archives". See "skip_actions"
for disabling checks altogether.
example: repository
frequency:
type: string
@@ -525,6 +530,18 @@ properties:
Apply color to console output. Can be overridden with --no-color
command-line flag. Defaults to true.
example: false
skip_actions:
type: array
items:
type: string
description: |
List of one or more actions to skip running for this configuration
file, even if specified on the command-line (explicitly or
implicitly). This is handy for append-only configurations where you
never want to run "compact" or checkless configuration where you
want to skip "check". Defaults to not skipping any actions.
example:
- compact
before_actions:
type: array
items:
@@ -1306,6 +1323,99 @@ properties:
example:
- start
- finish
apprise:
type: object
required: ['services']
additionalProperties: false
properties:
services:
type: array
items:
type: object
required:
- url
- label
properties:
url:
type: string
example: "gotify://hostname/token"
label:
type: string
example: gotify
description: |
A list of Apprise services to publish to with URLs and
labels. The labels are used for logging. A full list of
services and their configuration can be found at
https://github.com/caronc/apprise/wiki.
example:
- url: "kodi://user@hostname"
label: kodi
- url: "line://Token@User"
label: line
start:
type: object
required: ['body']
properties:
title:
type: string
description: |
Specify the message title. If left unspecified, no
title is sent.
example: Ping!
body:
type: string
description: |
Specify the message body.
example: Starting backup process.
finish:
type: object
required: ['body']
properties:
title:
type: string
description: |
Specify the message title. If left unspecified, no
title is sent.
example: Ping!
body:
type: string
description: |
Specify the message body.
example: Backups successfully made.
fail:
type: object
required: ['body']
properties:
title:
type: string
description: |
Specify the message title. If left unspecified, no
title is sent.
example: Ping!
body:
type: string
description: |
Specify the message body.
example: Your backups have failed.
states:
type: array
items:
type: string
enum:
- start
- finish
- fail
uniqueItems: true
description: |
List of one or more monitoring states to ping for: "start",
"finish", and/or "fail". Defaults to pinging for failure
only. For each selected state, corresponding configuration
for the message title and body should be given. If any is
left unspecified, a generic message is emitted instead.
example:
- start
- finish
healthchecks:
type: object
required: ['ping_url']
@@ -1400,7 +1510,7 @@ properties:
ends, or errors.
example: https://cronhub.io/ping/1f5e3410-254c-5587
description: |
Configuration for a monitoring integration with Crunhub. Create an
Configuration for a monitoring integration with Cronhub. Create an
account at https://cronhub.io if you'd like to use this service. See
borgmatic monitoring documentation for details.
loki:
+6 -3
View File
@@ -4,7 +4,7 @@ import jsonschema
import ruamel.yaml
import borgmatic.config
from borgmatic.config import environment, load, normalize, override
from borgmatic.config import constants, environment, load, normalize, override
def schema_filename():
@@ -109,11 +109,14 @@ def parse_configuration(config_filename, schema_filename, overrides=None, resolv
except (ruamel.yaml.error.YAMLError, RecursionError) as error:
raise Validation_error(config_filename, (str(error),))
override.apply_overrides(config, overrides)
logs = normalize.normalize(config_filename, config)
override.apply_overrides(config, schema, overrides)
constants.apply_constants(config, config.get('constants') if config else {})
if resolve_env:
environment.resolve_env_variables(config)
logs = normalize.normalize(config_filename, config)
try:
validator = jsonschema.Draft7Validator(schema)
except AttributeError: # pragma: no cover
+79
View File
@@ -0,0 +1,79 @@
import logging
import operator
logger = logging.getLogger(__name__)
def initialize_monitor(
ping_url, config, config_filename, monitoring_log_level, dry_run
): # pragma: no cover
'''
No initialization is necessary for this monitor.
'''
pass
def ping_monitor(hook_config, config, config_filename, state, monitoring_log_level, dry_run):
'''
Ping the configured Apprise service URLs. Use the given configuration filename in any log
entries. If this is a dry run, then don't actually ping anything.
'''
try:
import apprise
from apprise import NotifyFormat, NotifyType
except ImportError: # pragma: no cover
logger.warning('Unable to import Apprise in monitoring hook')
return
state_to_notify_type = {
'start': NotifyType.INFO,
'finish': NotifyType.SUCCESS,
'fail': NotifyType.FAILURE,
'log': NotifyType.INFO,
}
run_states = hook_config.get('states', ['fail'])
if state.name.lower() not in run_states:
return
state_config = hook_config.get(
state.name.lower(),
{
'title': f'A borgmatic {state.name} event happened',
'body': f'A borgmatic {state.name} event happened',
},
)
if not hook_config.get('services'):
logger.info(f'{config_filename}: No Apprise services to ping')
return
dry_run_string = ' (dry run; not actually pinging)' if dry_run else ''
labels_string = ', '.join(map(operator.itemgetter('label'), hook_config.get('services')))
logger.info(f'{config_filename}: Pinging Apprise services: {labels_string}{dry_run_string}')
apprise_object = apprise.Apprise()
apprise_object.add(list(map(operator.itemgetter('url'), hook_config.get('services'))))
if dry_run:
return
result = apprise_object.notify(
title=state_config.get('title', ''),
body=state_config.get('body'),
body_format=NotifyFormat.TEXT,
notify_type=state_to_notify_type[state.name.lower()],
)
if result is False:
logger.warning(f'{config_filename}: Error sending some Apprise notifications')
def destroy_monitor(
ping_url_or_uuid, config, config_filename, monitoring_log_level, dry_run
): # pragma: no cover
'''
No destruction is necessary for this monitor.
'''
pass
+2
View File
@@ -1,6 +1,7 @@
import logging
from borgmatic.hooks import (
apprise,
cronhub,
cronitor,
healthchecks,
@@ -17,6 +18,7 @@ from borgmatic.hooks import (
logger = logging.getLogger(__name__)
HOOK_NAME_TO_MODULE = {
'apprise': apprise,
'cronhub': cronhub,
'cronitor': cronitor,
'healthchecks': healthchecks,
+1 -1
View File
@@ -1,6 +1,6 @@
from enum import Enum
MONITOR_HOOK_NAMES = ('healthchecks', 'cronitor', 'cronhub', 'pagerduty', 'ntfy', 'loki')
MONITOR_HOOK_NAMES = ('apprise', 'healthchecks', 'cronitor', 'cronhub', 'pagerduty', 'ntfy', 'loki')
class State(Enum):
+2 -2
View File
@@ -166,6 +166,8 @@ def configure_logging(
Raise FileNotFoundError or PermissionError if the log file could not be opened for writing.
'''
add_custom_log_levels()
if syslog_log_level is None:
syslog_log_level = logging.DISABLED
if log_file_log_level is None:
@@ -173,8 +175,6 @@ def configure_logging(
if monitoring_log_level is None:
monitoring_log_level = console_log_level
add_custom_log_levels()
# Log certain log levels to console stderr and others to stdout. This supports use cases like
# grepping (non-error) output.
console_disabled = logging.NullHandler()
+2 -2
View File
@@ -2,13 +2,13 @@
font-size: 1rem; /* Reset */
}
.elv-toc details {
--details-force-closed: (max-width: 63.9375em); /* 1023px */
--details-force-closed: (max-width: 79.9375em); /* 1023px */
}
.elv-toc details > summary {
font-size: 1.375rem; /* 22px /16 */
margin-bottom: .5em;
}
@media (min-width: 64em) { /* 1024px */
@media (min-width: 80em) {
.elv-toc {
position: absolute;
left: 3rem;
+3 -3
View File
@@ -121,7 +121,7 @@ main h1:first-child,
main .elv-toc + h1 {
border-bottom: 2px dotted #666;
}
@media (min-width: 64em) { /* 1024px */
@media (min-width: 80em) {
main .elv-toc + h1,
main .elv-toc + h2 {
margin-top: 0;
@@ -243,10 +243,10 @@ footer.elv-layout {
.elv-layout-full {
max-width: none;
}
@media (min-width: 64em) { /* 1024px */
@media (min-width: 80em) {
.elv-layout-toc {
padding-left: 15rem;
max-width: 60rem;
max-width: 76rem;
margin-right: 1rem;
position: relative;
}
@@ -126,7 +126,7 @@ for more information.
## Hook output
Any output produced by your hooks shows up both at the console and in syslog
(when run in a non-interactive console). For more information, read about <a
(when enabled). For more information, read about <a
href="https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/">inspecting
your backups</a>.
+29 -6
View File
@@ -51,6 +51,11 @@ cron job), while only running expensive consistency checks with `check` on a
much less frequent basis (e.g. with `borgmatic check` called from a separate
cron job).
<span class="minilink minilink-addedin">New in version 1.8.5</span> Instead of
(or in addition to) specifying actions on the command-line, you can configure
borgmatic to [skip particular
actions](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#skipping-actions).
### Consistency check configuration
@@ -116,8 +121,17 @@ this option in the `consistency:` section of your configuration.
This tells borgmatic to run the `repository` consistency check at most once
every two weeks for a given repository and the `archives` check at most once a
month. The `frequency` value is a number followed by a unit of time, e.g. "3
days", "1 week", "2 months", etc.
month. The `frequency` value is a number followed by a unit of time, e.g. `3
days`, `1 week`, `2 months`, etc. The set of possible time units is as
follows (singular or plural):
* `second`
* `minute`
* `hour`
* `day`
* `week` (7 days)
* `month` (30 days)
* `year` (365 days)
The `frequency` defaults to `always` for a check configured without a
`frequency`, which means run this check every time checks run. But if you omit
@@ -162,7 +176,16 @@ location:
If that's still too slow, you can disable consistency checks entirely,
either for a single repository or for all repositories.
Disabling all consistency checks looks like this:
<span class="minilink minilink-addedin">New in version 1.8.5</span> Disabling
all consistency checks looks like this:
```yaml
skip_actions:
- check
```
<span class="minilink minilink-addedin">Prior to version 1.8.5</span> Use this
configuration instead:
```yaml
checks:
@@ -170,10 +193,10 @@ checks:
```
<span class="minilink minilink-addedin">Prior to version 1.8.0</span> Put
this option in the `consistency:` section of your configuration.
`checks:` in the `consistency:` section of your configuration.
<span class="minilink minilink-addedin">Prior to version 1.6.2</span> `checks`
was a plain list of strings without the `name:` part. For instance:
<span class="minilink minilink-addedin">Prior to version 1.6.2</span>
`checks:` was a plain list of strings without the `name:` part. For instance:
```yaml
checks:
+4 -3
View File
@@ -149,9 +149,10 @@ borgmatic umount --mount-point /mnt
<span class="minilink minilink-addedin">New in version 1.7.15</span> borgmatic
automatically stores all the configuration files used to create an archive
inside the archive itself. This is useful in cases where you've lost a
configuration file or you want to see what configurations were used to create a
particular archive.
inside the archive itself. They are stored in the archive using their full
paths from the machine being backed up. This is useful in cases where you've
lost a configuration file or you want to see what configurations were used to
create a particular archive.
To extract the configuration files from an archive, use the `config bootstrap`
action. For example:
+16 -13
View File
@@ -116,27 +116,30 @@ archive, complete with file sizes.
## Logging
By default, borgmatic logs to a local syslog-compatible daemon if one is
present and borgmatic is running in a non-interactive console. Where those
logs show up depends on your particular system. If you're using systemd, try
running `journalctl -xe`. Otherwise, try viewing `/var/log/syslog` or
similar.
You can customize the log level used for syslog logging with the
`--syslog-verbosity` flag, and this is independent from the console logging
`--verbosity` flag described above. For instance, to get additional
information about the progress of the backup as it proceeds:
By default, borgmatic logs to the console. You can enable simultaneous syslog
logging and customize its log level with the `--syslog-verbosity` flag, which
is independent from the console logging `--verbosity` flag described above.
For instance, to enable syslog logging, run:
```bash
borgmatic --syslog-verbosity 1
```
Or to increase syslog logging to include debug spew:
To increase syslog logging further to include debugging information, run:
```bash
borgmatic --syslog-verbosity 2
```
See above for further details about the verbosity levels.
Where these logs show up depends on your particular system. If you're using
systemd, try running `journalctl -xe`. Otherwise, try viewing
`/var/log/syslog` or similar.
<span class="minilink minilink-addedin">Prior to version 1.8.3</span>borgmatic
logged to syslog by default whenever run at a non-interactive console.
### Rate limiting
If you are using rsyslog or systemd's journal, be aware that by default they
@@ -165,7 +168,7 @@ Note that if you use the `--log-file` flag, you are responsible for rotating
the log file so it doesn't grow too large, for example with
[logrotate](https://wiki.archlinux.org/index.php/Logrotate).
You can the `--log-file-verbosity` flag to customize the log file's log level:
You can use the `--log-file-verbosity` flag to customize the log file's log level:
```bash
borgmatic --log-file /path/to/file.log --log-file-verbosity 2
@@ -197,5 +200,5 @@ See the [Python logging
documentation](https://docs.python.org/3/library/logging.html#logrecord-attributes)
for additional placeholders.
Note that this `--log-file-format` flg only applies to the specified
Note that this `--log-file-format` flag only applies to the specified
`--log-file` and not to syslog or other logging.
+1 -1
View File
@@ -151,7 +151,7 @@ in newer versions of borgmatic.
## Configuration includes
Once you have multiple different configuration files, you might want to share
common configuration options across these files with having to copy and paste
common configuration options across these files without having to copy and paste
them. To achieve this, you can put fragments of common configuration options
into a file and then include or inline that file into one or more borgmatic
configuration files.
+85 -26
View File
@@ -36,28 +36,24 @@ below for how to configure this.
### Third-party monitoring services
borgmatic integrates with monitoring services like
[Healthchecks](https://healthchecks.io/), [Cronitor](https://cronitor.io),
[Cronhub](https://cronhub.io), [PagerDuty](https://www.pagerduty.com/),
[ntfy](https://ntfy.sh/), and [Grafana Loki](https://grafana.com/oss/loki/)
and pings these services whenever borgmatic runs. That way, you'll receive an
alert when something goes wrong or (for certain hooks) the service doesn't
hear from borgmatic for a configured interval. See [Healthchecks
hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#healthchecks-hook),
[Cronitor
hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#cronitor-hook),
[Cronhub
hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#cronhub-hook),
[PagerDuty
hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#pagerduty-hook),
[ntfy
hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#ntfy-hook),
and [Loki
hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#loki-hook),
below for how to configure this.
borgmatic integrates with these monitoring services and libraries, pinging
them as backups happen:
* [Healthchecks](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#healthchecks-hook)
* [Cronitor](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#cronitor-hook)
* [Cronhub](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#cronhub-hook)
* [PagerDuty](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#pagerduty-hook)
* [ntfy](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#ntfy-hook)
* [Grafana Loki](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#loki-hook)
* [Apprise](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#apprise-hook)
The idea is that you'll receive an alert when something goes wrong or when the
service doesn't hear from borgmatic for a configured interval (if supported).
See the documentation links above for configuration information.
While these services and libraries offer different features, you probably only
need to use one of them at most.
While these services offer different features, you probably only need to use
one of them at most.
### Third-party monitoring software
@@ -146,7 +142,7 @@ healthchecks:
<span class="minilink minilink-addedin">Prior to version 1.8.0</span> Put
this option in the `hooks:` section of your configuration.
With this hook in place, borgmatic pings your Healthchecks project when a
With this configuration, borgmatic pings your Healthchecks project when a
backup begins, ends, or errors, but only when any of the `create`, `prune`,
`compact`, or `check` actions are run.
@@ -190,7 +186,7 @@ cronitor:
<span class="minilink minilink-addedin">Prior to version 1.8.0</span> Put
this option in the `hooks:` section of your configuration.
With this hook in place, borgmatic pings your Cronitor monitor when a backup
With this configuration, borgmatic pings your Cronitor monitor when a backup
begins, ends, or errors, but only when any of the `prune`, `compact`,
`create`, or `check` actions are run. Then, if the actions complete
successfully or errors, borgmatic notifies Cronitor accordingly.
@@ -217,7 +213,7 @@ cronhub:
<span class="minilink minilink-addedin">Prior to version 1.8.0</span> Put
this option in the `hooks:` section of your configuration.
With this hook in place, borgmatic pings your Cronhub monitor when a backup
With this configuration, borgmatic pings your Cronhub monitor when a backup
begins, ends, or errors, but only when any of the `prune`, `compact`,
`create`, or `check` actions are run. Then, if the actions complete
successfully or errors, borgmatic notifies Cronhub accordingly.
@@ -258,7 +254,7 @@ pagerduty:
<span class="minilink minilink-addedin">Prior to version 1.8.0</span> Put
this option in the `hooks:` section of your configuration.
With this hook in place, borgmatic creates a PagerDuty event for your service
With this configuration, borgmatic creates a PagerDuty event for your service
whenever backups fail, but only when any of the `create`, `prune`, `compact`,
or `check` actions are run. Note that borgmatic does not contact PagerDuty
when a backup starts or when it ends without error.
@@ -340,7 +336,7 @@ loki:
url: http://localhost:3100/loki/api/v1/push
```
With this hook in place, borgmatic sends its logs to your Loki instance as any
With this configuration, borgmatic sends its logs to your Loki instance as any
of the `prune`, `compact`, `create`, or `check` actions are run. Then, after
the actions complete, borgmatic notifies Loki of success or failure.
@@ -375,6 +371,69 @@ loki:
```
## Apprise hook
<span class="minilink minilink-addedin">New in version 1.8.4</span>
[Apprise](https://github.com/caronc/apprise/wiki) is a local notification library
that "allows you to send a notification to almost all of the most popular
[notification services](https://github.com/caronc/apprise/wiki) available to
us today such as: Telegram, Discord, Slack, Amazon SNS, Gotify, etc."
Depending on how you installed borgmatic, it may not have come with Apprise.
For instance, if you originally [installed borgmatic with
pipx](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#installation),
run the following to install Apprise so borgmatic can use it:
```bash
sudo pipx install --editable --force borgmatic[Apprise]
```
Omit `sudo` if borgmatic is installed as a non-root user.
Once Apprise is installed, configure borgmatic to notify one or more [Apprise
services](https://github.com/caronc/apprise/wiki). For example:
```yaml
apprise:
services:
- url: gotify://hostname/token
label: gotify
- url: mastodons://access_key@hostname/@user
label: mastodon
```
With this configuration, borgmatic pings each of the configured Apprise
services when a backup begins, ends, or errors, but only when any of the
`prune`, `compact`, `create`, or `check` actions are run.
You can optionally customize the contents of the default messages sent to
these services:
```yaml
apprise:
services:
- url: gotify://hostname/token
label: gotify
start:
title: Ping!
body: Starting backup process.
finish:
title: Ping!
body: Backups successfully made.
fail:
title: Ping!
body: Your backups have failed.
states:
- start
- finish
- fail
```
See the [configuration
reference](https://torsion.org/borgmatic/docs/reference/configuration/) for
details.
## Scripting borgmatic
To consume the output of borgmatic in other software, you can include an
+24 -11
View File
@@ -9,9 +9,15 @@ eleventyNavigation:
If you want to use a Borg repository passphrase or database passwords with
borgmatic, you can set them directly in your borgmatic configuration file,
treating those secrets like any other option value. But if you'd rather store
them outside of borgmatic, whether for convenience or security reasons, read
on.
treating those secrets like any other option value. For instance, you can
specify your Borg passhprase with:
```yaml
encryption_passphrase: yourpassphrase
```
But if you'd rather store them outside of borgmatic, whether for convenience
or security reasons, read on.
<span class="minilink minilink-addedin">New in version 1.6.4</span> borgmatic
supports interpolating arbitrary environment variables directly into option
@@ -20,14 +26,14 @@ pull your repository passphrase, your database passwords, or any other option
values from environment variables. For instance:
```yaml
encryption_passphrase: ${MY_PASSPHRASE}
encryption_passphrase: ${YOUR_PASSPHRASE}
```
<span class="minilink minilink-addedin">Prior to version 1.8.0</span> Put
this option in the `storage:` section of your configuration.
This uses the `MY_PASSPHRASE` environment variable as your encryption
passphrase. Note that the `{` `}` brackets are required. `$MY_PASSPHRASE` by
This uses the `YOUR_PASSPHRASE` environment variable as your encryption
passphrase. Note that the `{` `}` brackets are required. `$YOUR_PASSPHRASE` by
itself will not work.
In the case of `encryption_passphrase` in particular, an alternate approach
@@ -42,25 +48,26 @@ the same approach applies. For example:
```yaml
postgresql_databases:
- name: users
password: ${MY_DATABASE_PASSWORD}
password: ${YOUR_DATABASE_PASSWORD}
```
<span class="minilink minilink-addedin">Prior to version 1.8.0</span> Put
this option in the `hooks:` section of your configuration.
This uses the `MY_DATABASE_PASSWORD` environment variable as your database
This uses the `YOUR_DATABASE_PASSWORD` environment variable as your database
password.
### Interpolation defaults
If you'd like to set a default for your environment variables, you can do so with the following syntax:
If you'd like to set a default for your environment variables, you can do so
with the following syntax:
```yaml
encryption_passphrase: ${MY_PASSPHRASE:-defaultpass}
encryption_passphrase: ${YOUR_PASSPHRASE:-defaultpass}
```
Here, "`defaultpass`" is the default passphrase if the `MY_PASSPHRASE`
Here, "`defaultpass`" is the default passphrase if the `YOUR_PASSPHRASE`
environment variable is not set. Without a default, if the environment
variable doesn't exist, borgmatic will error.
@@ -90,3 +97,9 @@ Additionally, borgmatic action hooks support their own [variable
interpolation](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/#variable-interpolation),
although in that case it's for particular borgmatic runtime values rather than
(only) environment variables.
Lastly, if you do want to specify your passhprase directly within borgmatic
configuration, but you'd like to keep it in a separate file from your main
configuration, you can [use a configuration include or a merge
include](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#configuration-includes)
to pull in an external password.
+17 -1
View File
@@ -69,7 +69,8 @@ borgmatic --version
If borgmatic is properly installed, that should output your borgmatic version.
You can also try `sudo borgmatic --version` if you intend to run borgmatic
with `sudo`.
with `sudo`. If that doesn't work, you may need to update your [sudoers
`secure_path` option](https://wiki.archlinux.org/title/Sudo).
### Other ways to install
@@ -281,6 +282,21 @@ due to things like file damage. For instance:
sudo borgmatic --verbosity 1 --list --stats
```
### Skipping actions
<span class="minilink minilink-addedin">New in version 1.8.5</span> You can
configure borgmatic to skip running certain actions (default or otherwise).
For instance, to always skip the `compact` action when using [Borg's
append-only
mode](https://borgbackup.readthedocs.io/en/stable/usage/notes.html#append-only-mode-forbid-compaction),
set the `skip_actions` option:
```
skip_actions:
- compact
```
## Autopilot
Running backups manually is good for validating your configuration, but I'm
-2
View File
@@ -21,5 +21,3 @@ version](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#configuration
```yaml
{% include borgmatic/config.yaml %}
```
Note that you can also [download this configuration
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 157 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

+6 -6
View File
@@ -32,16 +32,16 @@ RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
# To restrict write access further, change "ProtectSystem" to "strict" and uncomment
# "ReadWritePaths", "ReadOnlyPaths", "ProtectHome", and "BindPaths". Then add any local repository
# paths to the list of "ReadWritePaths" and local backup source paths to "ReadOnlyPaths". This
# leaves most of the filesystem read-only to borgmatic.
# To restrict write access further, change "ProtectSystem" to "strict" and
# uncomment "ReadWritePaths", "TemporaryFileSystem", "BindPaths" and
# "BindReadOnlyPaths". Then add any local repository paths to the list of
# "ReadWritePaths". This leaves most of the filesystem read-only to borgmatic.
ProtectSystem=full
# ReadWritePaths=-/mnt/my_backup_drive
# ReadOnlyPaths=-/var/lib/my_backup_source
# This will mount a tmpfs on top of /root and pass through needed paths
# ProtectHome=tmpfs
# TemporaryFileSystem=/root:ro
# BindPaths=-/root/.cache/borg -/root/.config/borg -/root/.borgmatic
# BindReadOnlyPaths=-/root/.ssh
# May interfere with running external programs within borgmatic hooks.
CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW
+3 -3
View File
@@ -18,11 +18,11 @@ if [ -z "$TEST_CONTAINER" ]; then
fi
apk add --no-cache python3 py3-pip borgbackup postgresql-client mariadb-client mongodb-tools \
py3-ruamel.yaml py3-ruamel.yaml.clib bash sqlite fish
py3-ruamel.yaml py3-ruamel.yaml.clib py3-yaml bash sqlite fish
# If certain dependencies of black are available in this version of Alpine, install them.
apk add --no-cache py3-typed-ast py3-regex || true
python3 -m pip install --no-cache --upgrade pip==22.2.2 setuptools==64.0.1 pymongo==4.4.1
pip3 install --ignore-installed tox==3.25.1
python3 -m pip install --no-cache --upgrade pip==22.2.2 setuptools==64.0.1
pip3 install --ignore-installed tox==4.11.3
export COVERAGE_FILE=/tmp/.coverage
if [ "$1" != "--end-to-end-only" ]; then
+4 -3
View File
@@ -1,6 +1,6 @@
from setuptools import find_packages, setup
VERSION = '1.8.3'
VERSION = '1.8.5'
setup(
@@ -33,9 +33,10 @@ setup(
'jsonschema',
'packaging',
'requests',
'ruamel.yaml>0.15.0,<0.18.0',
'ruamel.yaml>0.15.0',
'setuptools',
),
extras_require={"Apprise": ["apprise"]},
include_package_data=True,
python_requires='>=3.7',
python_requires='>=3.8',
)
+10 -9
View File
@@ -1,6 +1,8 @@
appdirs==1.4.4; python_version >= '3.8'
apprise==1.3.0
attrs==22.2.0; python_version >= '3.8'
black==23.3.0; python_version >= '3.8'
certifi==2023.7.22
chardet==5.1.0
click==8.1.3; python_version >= '3.8'
codespell==2.2.4
@@ -12,22 +14,21 @@ flake8-use-fstring==1.4
flake8-variables-names==0.0.5
flexmock==0.11.3
idna==3.4
importlib_metadata==6.3.0; python_version < '3.8'
isort==5.12.0
jsonschema==4.17.3
Markdown==3.4.1
mccabe==0.7.0
packaging==23.1
pathspec==0.11.1
pluggy==1.0.0
pathspec==0.11.1; python_version >= '3.8'
py==1.11.0
pycodestyle==2.10.0
pyflakes==3.0.1
jsonschema==4.17.3
pytest==7.3.0
pytest-cov==4.0.0
regex; python_version >= '3.8'
PyYAML>5.0.0
regex
requests==2.31.0
ruamel.yaml>0.15.0,<0.18.0
toml==0.10.2; python_version >= '3.8'
typed-ast; python_version >= '3.8'
typing-extensions==4.5.0; python_version < '3.8'
zipp==3.15.0; python_version < '3.8'
ruamel.yaml>0.15.0
toml==0.10.2
typed-ast
+11
View File
@@ -0,0 +1,11 @@
import subprocess
import sys
def test_borgmatic_command_with_invalid_flag_shows_error_but_not_traceback():
output = subprocess.run(
'borgmatic -v 2 --invalid'.split(' '), stdout=subprocess.PIPE, stderr=subprocess.STDOUT
).stdout.decode(sys.stdout.encoding)
assert 'Unrecognized argument' in output
assert 'Traceback' not in output
+3
View File
@@ -32,6 +32,9 @@ def assert_command_does_not_duplicate_flags(command, *args, **kwargs):
flag_name: 1 for flag_name in flag_counts
}, f"Duplicate flags found in: {' '.join(command)}"
if '--json' in command:
return '{}'
def fuzz_argument(arguments, argument_name):
'''
@@ -622,3 +622,16 @@ def test_parse_arguments_config_with_subaction_and_explicit_config_file_does_not
module.parse_arguments(
'config', 'bootstrap', '--repository', 'repo.borg', '--config', 'test.yaml'
)
def test_parse_arguments_with_borg_action_and_dry_run_raises():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
with pytest.raises(ValueError):
module.parse_arguments('--dry-run', 'borg', 'list')
def test_parse_arguments_with_borg_action_and_no_dry_run_does_not_raise():
flexmock(module.collect).should_receive('get_default_config_paths').and_return(['default'])
module.parse_arguments('borg', 'list')
+22 -12
View File
@@ -10,7 +10,7 @@ from borgmatic.config import generate as module
def test_insert_newline_before_comment_does_not_raise():
field_name = 'foo'
config = module.yaml.comments.CommentedMap([(field_name, 33)])
config = module.ruamel.yaml.comments.CommentedMap([(field_name, 33)])
config.yaml_set_comment_before_after_key(key=field_name, before='Comment')
module.insert_newline_before_comment(config, field_name)
@@ -125,14 +125,16 @@ def test_write_configuration_with_already_existing_directory_does_not_raise():
def test_add_comments_to_configuration_sequence_of_strings_does_not_raise():
config = module.yaml.comments.CommentedSeq(['foo', 'bar'])
config = module.ruamel.yaml.comments.CommentedSeq(['foo', 'bar'])
schema = {'type': 'array', 'items': {'type': 'string'}}
module.add_comments_to_configuration_sequence(config, schema)
def test_add_comments_to_configuration_sequence_of_maps_does_not_raise():
config = module.yaml.comments.CommentedSeq([module.yaml.comments.CommentedMap([('foo', 'yo')])])
config = module.ruamel.yaml.comments.CommentedSeq(
[module.ruamel.yaml.comments.CommentedMap([('foo', 'yo')])]
)
schema = {
'type': 'array',
'items': {'type': 'object', 'properties': {'foo': {'description': 'yo'}}},
@@ -142,7 +144,9 @@ def test_add_comments_to_configuration_sequence_of_maps_does_not_raise():
def test_add_comments_to_configuration_sequence_of_maps_without_description_does_not_raise():
config = module.yaml.comments.CommentedSeq([module.yaml.comments.CommentedMap([('foo', 'yo')])])
config = module.ruamel.yaml.comments.CommentedSeq(
[module.ruamel.yaml.comments.CommentedMap([('foo', 'yo')])]
)
schema = {'type': 'array', 'items': {'type': 'object', 'properties': {'foo': {}}}}
module.add_comments_to_configuration_sequence(config, schema)
@@ -150,7 +154,7 @@ def test_add_comments_to_configuration_sequence_of_maps_without_description_does
def test_add_comments_to_configuration_object_does_not_raise():
# Ensure that it can deal with fields both in the schema and missing from the schema.
config = module.yaml.comments.CommentedMap([('foo', 33), ('bar', 44), ('baz', 55)])
config = module.ruamel.yaml.comments.CommentedMap([('foo', 33), ('bar', 44), ('baz', 55)])
schema = {
'type': 'object',
'properties': {'foo': {'description': 'Foo'}, 'bar': {'description': 'Bar'}},
@@ -160,7 +164,7 @@ def test_add_comments_to_configuration_object_does_not_raise():
def test_add_comments_to_configuration_object_with_skip_first_does_not_raise():
config = module.yaml.comments.CommentedMap([('foo', 33)])
config = module.ruamel.yaml.comments.CommentedMap([('foo', 33)])
schema = {'type': 'object', 'properties': {'foo': {'description': 'Foo'}}}
module.add_comments_to_configuration_object(config, schema, skip_first=True)
@@ -168,7 +172,7 @@ def test_add_comments_to_configuration_object_with_skip_first_does_not_raise():
def test_remove_commented_out_sentinel_keeps_other_comments():
field_name = 'foo'
config = module.yaml.comments.CommentedMap([(field_name, 33)])
config = module.ruamel.yaml.comments.CommentedMap([(field_name, 33)])
config.yaml_set_comment_before_after_key(key=field_name, before='Actual comment.\nCOMMENT_OUT')
module.remove_commented_out_sentinel(config, field_name)
@@ -180,7 +184,7 @@ def test_remove_commented_out_sentinel_keeps_other_comments():
def test_remove_commented_out_sentinel_without_sentinel_keeps_other_comments():
field_name = 'foo'
config = module.yaml.comments.CommentedMap([(field_name, 33)])
config = module.ruamel.yaml.comments.CommentedMap([(field_name, 33)])
config.yaml_set_comment_before_after_key(key=field_name, before='Actual comment.')
module.remove_commented_out_sentinel(config, field_name)
@@ -192,7 +196,7 @@ def test_remove_commented_out_sentinel_without_sentinel_keeps_other_comments():
def test_remove_commented_out_sentinel_on_unknown_field_does_not_raise():
field_name = 'foo'
config = module.yaml.comments.CommentedMap([(field_name, 33)])
config = module.ruamel.yaml.comments.CommentedMap([(field_name, 33)])
config.yaml_set_comment_before_after_key(key=field_name, before='Actual comment.')
module.remove_commented_out_sentinel(config, 'unknown')
@@ -201,7 +205,9 @@ def test_remove_commented_out_sentinel_on_unknown_field_does_not_raise():
def test_generate_sample_configuration_does_not_raise():
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('schema.yaml').and_return('')
flexmock(module.yaml).should_receive('round_trip_load')
flexmock(module.ruamel.yaml).should_receive('YAML').and_return(
flexmock(load=lambda filename: {})
)
flexmock(module).should_receive('schema_to_sample_configuration')
flexmock(module).should_receive('merge_source_configuration_into_destination')
flexmock(module).should_receive('render_configuration')
@@ -214,7 +220,9 @@ def test_generate_sample_configuration_does_not_raise():
def test_generate_sample_configuration_with_source_filename_does_not_raise():
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('schema.yaml').and_return('')
flexmock(module.yaml).should_receive('round_trip_load')
flexmock(module.ruamel.yaml).should_receive('YAML').and_return(
flexmock(load=lambda filename: {})
)
flexmock(module.load).should_receive('load_configuration')
flexmock(module.normalize).should_receive('normalize')
flexmock(module).should_receive('schema_to_sample_configuration')
@@ -229,7 +237,9 @@ def test_generate_sample_configuration_with_source_filename_does_not_raise():
def test_generate_sample_configuration_with_dry_run_does_not_write_file():
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args('schema.yaml').and_return('')
flexmock(module.yaml).should_receive('round_trip_load')
flexmock(module.ruamel.yaml).should_receive('YAML').and_return(
flexmock(load=lambda filename: {})
)
flexmock(module).should_receive('schema_to_sample_configuration')
flexmock(module).should_receive('merge_source_configuration_into_destination')
flexmock(module).should_receive('render_configuration')
-29
View File
@@ -15,35 +15,6 @@ def test_load_configuration_parses_contents():
assert module.load_configuration('config.yaml') == {'key': 'value'}
def test_load_configuration_replaces_constants():
builtins = flexmock(sys.modules['builtins'])
config_file = io.StringIO(
'''
constants:
key: value
key: {key}
'''
)
config_file.name = 'config.yaml'
builtins.should_receive('open').with_args('config.yaml').and_return(config_file)
assert module.load_configuration('config.yaml') == {'key': 'value'}
def test_load_configuration_replaces_complex_constants():
builtins = flexmock(sys.modules['builtins'])
config_file = io.StringIO(
'''
constants:
key:
subkey: value
key: {key}
'''
)
config_file.name = 'config.yaml'
builtins.should_receive('open').with_args('config.yaml').and_return(config_file)
assert module.load_configuration('config.yaml') == {'key': {'subkey': 'value'}}
def test_load_configuration_with_only_integer_value_does_not_raise():
builtins = flexmock(sys.modules['builtins'])
config_file = io.StringIO('33')
+24 -12
View File
@@ -4,19 +4,24 @@ from borgmatic.config import override as module
@pytest.mark.parametrize(
'value,expected_result',
'value,expected_result,option_type',
(
('thing', 'thing'),
('33', 33),
('33b', '33b'),
('true', True),
('false', False),
('[foo]', ['foo']),
('[foo, bar]', ['foo', 'bar']),
('thing', 'thing', 'string'),
('33', 33, 'integer'),
('33', '33', 'string'),
('33b', '33b', 'integer'),
('33b', '33b', 'string'),
('true', True, 'boolean'),
('false', False, 'boolean'),
('true', 'true', 'string'),
('[foo]', ['foo'], 'array'),
('[foo]', '[foo]', 'string'),
('[foo, bar]', ['foo', 'bar'], 'array'),
('[foo, bar]', '[foo, bar]', 'string'),
),
)
def test_convert_value_type_coerces_values(value, expected_result):
assert module.convert_value_type(value) == expected_result
def test_convert_value_type_coerces_values(value, expected_result, option_type):
assert module.convert_value_type(value, option_type) == expected_result
def test_apply_overrides_updates_config():
@@ -25,16 +30,23 @@ def test_apply_overrides_updates_config():
'other_section.thing=value2',
'section.nested.key=value3',
'new.foo=bar',
'new.mylist=[baz]',
'new.nonlist=[quux]',
]
config = {
'section': {'key': 'value', 'other': 'other_value'},
'other_section': {'thing': 'thing_value'},
}
schema = {
'properties': {
'new': {'properties': {'mylist': {'type': 'array'}, 'nonlist': {'type': 'string'}}}
}
}
module.apply_overrides(config, raw_overrides)
module.apply_overrides(config, schema, raw_overrides)
assert config == {
'section': {'key': 'value1', 'other': 'other_value', 'nested': {'key': 'value3'}},
'other_section': {'thing': 'value2'},
'new': {'foo': 'bar'},
'new': {'foo': 'bar', 'mylist': ['baz'], 'nonlist': '[quux]'},
}
+5 -3
View File
@@ -1,4 +1,5 @@
import io
import os
import string
import sys
@@ -244,7 +245,7 @@ def test_parse_configuration_applies_overrides():
assert logs == []
def test_parse_configuration_applies_normalization():
def test_parse_configuration_applies_normalization_after_environment_variable_interpolation():
mock_config_and_schema(
'''
location:
@@ -252,17 +253,18 @@ def test_parse_configuration_applies_normalization():
- /home
repositories:
- path: hostname.borg
- ${NO_EXIST:-user@hostname:repo}
exclude_if_present: .nobackup
'''
)
flexmock(os).should_receive('getenv').replace_with(lambda variable_name, default: default)
config, logs = module.parse_configuration('/tmp/config.yaml', '/tmp/schema.yaml')
assert config == {
'source_directories': ['/home'],
'repositories': [{'path': 'hostname.borg'}],
'repositories': [{'path': 'ssh://user@hostname/./repo'}],
'exclude_if_present': ['.nobackup'],
}
assert logs
+2 -2
View File
@@ -151,7 +151,7 @@ def test_create_borgmatic_manifest_creates_manifest_file():
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.os).should_receive('makedirs').and_return(True)
flexmock(module.importlib_metadata).should_receive('version').and_return('1.0.0')
flexmock(module.importlib.metadata).should_receive('version').and_return('1.0.0')
flexmock(sys.modules['builtins']).should_receive('open').with_args(
'/home/user/.borgmatic/bootstrap/manifest.json', 'w'
).and_return(
@@ -172,7 +172,7 @@ def test_create_borgmatic_manifest_creates_manifest_file_with_custom_borgmatic_s
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.os).should_receive('makedirs').and_return(True)
flexmock(module.importlib_metadata).should_receive('version').and_return('1.0.0')
flexmock(module.importlib.metadata).should_receive('version').and_return('1.0.0')
flexmock(sys.modules['builtins']).should_receive('open').with_args(
'/borgmatic/bootstrap/manifest.json', 'w'
).and_return(
+160 -20
View File
@@ -193,6 +193,19 @@ def test_filter_checks_on_frequency_restains_check_with_unelapsed_frequency_and_
) == ('archives',)
def test_filter_checks_on_frequency_passes_through_empty_checks():
assert (
module.filter_checks_on_frequency(
config={'checks': [{'name': 'archives', 'frequency': '1 hour'}]},
borg_repository_id='repo',
checks=(),
force=False,
archives_check_id='1234',
)
== ()
)
def test_make_archive_filter_flags_with_default_checks_and_prefix_returns_default_flags():
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
@@ -201,6 +214,7 @@ def test_make_archive_filter_flags_with_default_checks_and_prefix_returns_defaul
'1.2.3',
{},
('repository', 'archives'),
check_arguments=flexmock(match_archives=None),
prefix='foo',
)
@@ -215,6 +229,7 @@ def test_make_archive_filter_flags_with_all_checks_and_prefix_returns_default_fl
'1.2.3',
{},
('repository', 'archives', 'extract'),
check_arguments=flexmock(match_archives=None),
prefix='foo',
)
@@ -229,6 +244,7 @@ def test_make_archive_filter_flags_with_all_checks_and_prefix_without_borg_featu
'1.2.3',
{},
('repository', 'archives', 'extract'),
check_arguments=flexmock(match_archives=None),
prefix='foo',
)
@@ -239,7 +255,9 @@ def test_make_archive_filter_flags_with_archives_check_and_last_includes_last_fl
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('archives',), check_last=3)
flags = module.make_archive_filter_flags(
'1.2.3', {}, ('archives',), check_arguments=flexmock(match_archives=None), check_last=3
)
assert flags == ('--last', '3')
@@ -248,7 +266,9 @@ def test_make_archive_filter_flags_with_data_check_and_last_includes_last_flag()
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('data',), check_last=3)
flags = module.make_archive_filter_flags(
'1.2.3', {}, ('data',), check_arguments=flexmock(match_archives=None), check_last=3
)
assert flags == ('--last', '3')
@@ -257,7 +277,9 @@ def test_make_archive_filter_flags_with_repository_check_and_last_omits_last_fla
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('repository',), check_last=3)
flags = module.make_archive_filter_flags(
'1.2.3', {}, ('repository',), check_arguments=flexmock(match_archives=None), check_last=3
)
assert flags == ()
@@ -266,7 +288,13 @@ def test_make_archive_filter_flags_with_default_checks_and_last_includes_last_fl
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('repository', 'archives'), check_last=3)
flags = module.make_archive_filter_flags(
'1.2.3',
{},
('repository', 'archives'),
check_arguments=flexmock(match_archives=None),
check_last=3,
)
assert flags == ('--last', '3')
@@ -275,7 +303,9 @@ def test_make_archive_filter_flags_with_archives_check_and_prefix_includes_match
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('archives',), prefix='foo-')
flags = module.make_archive_filter_flags(
'1.2.3', {}, ('archives',), check_arguments=flexmock(match_archives=None), prefix='foo-'
)
assert flags == ('--match-archives', 'sh:foo-*')
@@ -284,11 +314,30 @@ def test_make_archive_filter_flags_with_data_check_and_prefix_includes_match_arc
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('data',), prefix='foo-')
flags = module.make_archive_filter_flags(
'1.2.3', {}, ('data',), check_arguments=flexmock(match_archives=None), prefix='foo-'
)
assert flags == ('--match-archives', 'sh:foo-*')
def test_make_archive_filter_flags_prefers_check_arguments_match_archives_to_config_match_archives():
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
'baz-*', None, '1.2.3'
).and_return(('--match-archives', 'sh:baz-*'))
flags = module.make_archive_filter_flags(
'1.2.3',
{'match_archives': 'bar-{now}'}, # noqa: FS003
('archives',),
check_arguments=flexmock(match_archives='baz-*'),
prefix='',
)
assert flags == ('--match-archives', 'sh:baz-*')
def test_make_archive_filter_flags_with_archives_check_and_empty_prefix_uses_archive_name_format_instead():
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
@@ -296,7 +345,11 @@ def test_make_archive_filter_flags_with_archives_check_and_empty_prefix_uses_arc
).and_return(('--match-archives', 'sh:bar-*'))
flags = module.make_archive_filter_flags(
'1.2.3', {'archive_name_format': 'bar-{now}'}, ('archives',), prefix='' # noqa: FS003
'1.2.3',
{'archive_name_format': 'bar-{now}'}, # noqa: FS003
('archives',),
check_arguments=flexmock(match_archives=None),
prefix='',
)
assert flags == ('--match-archives', 'sh:bar-*')
@@ -306,7 +359,9 @@ def test_make_archive_filter_flags_with_archives_check_and_none_prefix_omits_mat
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('archives',), prefix=None)
flags = module.make_archive_filter_flags(
'1.2.3', {}, ('archives',), check_arguments=flexmock(match_archives=None), prefix=None
)
assert flags == ()
@@ -315,7 +370,9 @@ def test_make_archive_filter_flags_with_repository_check_and_prefix_omits_match_
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('repository',), prefix='foo-')
flags = module.make_archive_filter_flags(
'1.2.3', {}, ('repository',), check_arguments=flexmock(match_archives=None), prefix='foo-'
)
assert flags == ()
@@ -324,7 +381,13 @@ def test_make_archive_filter_flags_with_default_checks_and_prefix_includes_match
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flags = module.make_archive_filter_flags('1.2.3', {}, ('repository', 'archives'), prefix='foo-')
flags = module.make_archive_filter_flags(
'1.2.3',
{},
('repository', 'archives'),
check_arguments=flexmock(match_archives=None),
prefix='foo-',
)
assert flags == ('--match-archives', 'sh:foo-*')
@@ -607,7 +670,7 @@ def test_upgrade_check_times_renames_stale_temporary_check_path():
module.upgrade_check_times(flexmock(), flexmock())
def test_check_archives_with_progress_calls_borg_with_progress_parameter():
def test_check_archives_with_progress_passes_through_to_borg():
checks = ('repository',)
config = {'check_last': None}
flexmock(module.rinfo).should_receive('display_repository_info').and_return(
@@ -634,12 +697,14 @@ def test_check_archives_with_progress_calls_borg_with_progress_parameter():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=True, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
progress=True,
)
def test_check_archives_with_repair_calls_borg_with_repair_parameter():
def test_check_archives_with_repair_passes_through_to_borg():
checks = ('repository',)
config = {'check_last': None}
flexmock(module.rinfo).should_receive('display_repository_info').and_return(
@@ -666,8 +731,10 @@ def test_check_archives_with_repair_calls_borg_with_repair_parameter():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=True, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
repair=True,
)
@@ -701,6 +768,9 @@ def test_check_archives_calls_borg_with_parameters(checks):
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
@@ -723,6 +793,9 @@ def test_check_archives_with_json_error_raises():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
@@ -743,6 +816,9 @@ def test_check_archives_with_missing_json_keys_raises():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
@@ -769,11 +845,14 @@ def test_check_archives_with_extract_check_calls_extract_only():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
def test_check_archives_with_log_info_calls_borg_with_info_parameter():
def test_check_archives_with_log_info_passes_through_to_borg():
checks = ('repository',)
config = {'check_last': None}
flexmock(module.rinfo).should_receive('display_repository_info').and_return(
@@ -795,11 +874,14 @@ def test_check_archives_with_log_info_calls_borg_with_info_parameter():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
def test_check_archives_with_log_debug_calls_borg_with_debug_parameter():
def test_check_archives_with_log_debug_passes_through_to_borg():
checks = ('repository',)
config = {'check_last': None}
flexmock(module.rinfo).should_receive('display_repository_info').and_return(
@@ -821,6 +903,9 @@ def test_check_archives_with_log_debug_calls_borg_with_debug_parameter():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
@@ -841,6 +926,9 @@ def test_check_archives_without_any_checks_bails():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
@@ -867,12 +955,15 @@ def test_check_archives_with_local_path_calls_borg_via_local_path():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
local_path='borg1',
)
def test_check_archives_with_remote_path_calls_borg_with_remote_path_parameters():
def test_check_archives_with_remote_path_passes_through_to_borg():
checks = ('repository',)
check_last = flexmock()
config = {'check_last': check_last}
@@ -894,12 +985,15 @@ def test_check_archives_with_remote_path_calls_borg_with_remote_path_parameters(
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
remote_path='borg1',
)
def test_check_archives_with_log_json_calls_borg_with_log_json_parameters():
def test_check_archives_with_log_json_passes_through_to_borg():
checks = ('repository',)
check_last = flexmock()
config = {'check_last': check_last}
@@ -921,11 +1015,14 @@ def test_check_archives_with_log_json_calls_borg_with_log_json_parameters():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=True),
)
def test_check_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
def test_check_archives_with_lock_wait_passes_through_to_borg():
checks = ('repository',)
check_last = flexmock()
config = {'lock_wait': 5, 'check_last': check_last}
@@ -947,6 +1044,9 @@ def test_check_archives_with_lock_wait_calls_borg_with_lock_wait_parameters():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
@@ -974,11 +1074,14 @@ def test_check_archives_with_retention_prefix():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
def test_check_archives_with_extra_borg_options_calls_borg_with_extra_options():
def test_check_archives_with_extra_borg_options_passes_through_to_borg():
checks = ('repository',)
config = {'check_last': None, 'extra_borg_options': {'check': '--extra --options'}}
flexmock(module.rinfo).should_receive('display_repository_info').and_return(
@@ -999,5 +1102,42 @@ def test_check_archives_with_extra_borg_options_calls_borg_with_extra_options():
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
def test_check_archives_with_match_archives_passes_through_to_borg():
checks = ('archives',)
config = {'check_last': None}
flexmock(module.rinfo).should_receive('display_repository_info').and_return(
'{"repository": {"id": "repo"}}'
)
flexmock(module).should_receive('upgrade_check_times')
flexmock(module).should_receive('parse_checks')
flexmock(module).should_receive('make_archive_filter_flags').and_return(
('--match-archives', 'foo-*')
)
flexmock(module).should_receive('make_archives_check_id').and_return(None)
flexmock(module).should_receive('filter_checks_on_frequency').and_return(checks)
flexmock(module).should_receive('make_check_flags').and_return(('--match-archives', 'foo-*'))
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'check', '--match-archives', 'foo-*', 'repo'),
extra_environment=None,
).once()
flexmock(module).should_receive('make_check_time_path')
flexmock(module).should_receive('write_check_time')
module.check_archives(
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None, repair=None, only_checks=None, force=None, match_archives='foo-*'
),
global_arguments=flexmock(log_json=False),
)
+50
View File
@@ -164,3 +164,53 @@ def test_make_match_archives_flags_makes_flags_with_globs(
)
== expected_result
)
def test_warn_for_aggressive_archive_flags_without_archive_flags_bails():
flexmock(module.logger).should_receive('warning').never()
module.warn_for_aggressive_archive_flags(('borg', '--do-stuff'), '{}')
def test_warn_for_aggressive_archive_flags_with_glob_archives_and_zero_archives_warns():
flexmock(module.logger).should_receive('warning').twice()
module.warn_for_aggressive_archive_flags(
('borg', '--glob-archives', 'foo*'), '{"archives": []}'
)
def test_warn_for_aggressive_archive_flags_with_match_archives_and_zero_archives_warns():
flexmock(module.logger).should_receive('warning').twice()
module.warn_for_aggressive_archive_flags(
('borg', '--match-archives', 'foo*'), '{"archives": []}'
)
def test_warn_for_aggressive_archive_flags_with_glob_archives_and_one_archive_does_not_warn():
flexmock(module.logger).should_receive('warning').never()
module.warn_for_aggressive_archive_flags(
('borg', '--glob-archives', 'foo*'), '{"archives": [{"name": "foo"]}'
)
def test_warn_for_aggressive_archive_flags_with_match_archives_and_one_archive_does_not_warn():
flexmock(module.logger).should_receive('warning').never()
module.warn_for_aggressive_archive_flags(
('borg', '--match-archives', 'foo*'), '{"archives": [{"name": "foo"]}'
)
def test_warn_for_aggressive_archive_flags_with_glob_archives_and_invalid_json_does_not_warn():
flexmock(module.logger).should_receive('warning').never()
module.warn_for_aggressive_archive_flags(('borg', '--glob-archives', 'foo*'), '{"archives": [}')
def test_warn_for_aggressive_archive_flags_with_glob_archives_and_json_missing_archives_does_not_warn():
flexmock(module.logger).should_receive('warning').never()
module.warn_for_aggressive_archive_flags(('borg', '--glob-archives', 'foo*'), '{}')
+157 -210
View File
@@ -8,224 +8,178 @@ from borgmatic.borg import info as module
from ..test_verbosity import insert_logging_mock
def test_display_archives_info_calls_borg_with_parameters():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_constructs_borg_info_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--repo', 'repo')
def test_display_archives_info_with_log_info_calls_borg_with_info_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_log_info_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--info', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
insert_logging_mock(logging.INFO)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--info', '--repo', 'repo')
def test_display_archives_info_with_log_info_and_json_suppresses_most_borg_output():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_log_info_and_json_omits_borg_logging_flags():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(('--json',))
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'info', '--json', '--repo', 'repo'),
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
insert_logging_mock(logging.INFO)
json_output = module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=True, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert json_output == '[]'
assert command == ('borg', 'info', '--json', '--repo', 'repo')
def test_display_archives_info_with_log_debug_calls_borg_with_debug_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_log_debug_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--debug', '--show-rc', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
insert_logging_mock(logging.DEBUG)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--debug', '--show-rc', '--repo', 'repo')
def test_display_archives_info_with_log_debug_and_json_suppresses_most_borg_output():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_log_debug_and_json_omits_borg_logging_flags():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(('--json',))
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'info', '--json', '--repo', 'repo'),
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
insert_logging_mock(logging.DEBUG)
json_output = module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=True, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert json_output == '[]'
assert command == ('borg', 'info', '--json', '--repo', 'repo')
def test_display_archives_info_with_json_calls_borg_with_json_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_json_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(('--json',))
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'info', '--json', '--repo', 'repo'),
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
json_output = module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=True, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert json_output == '[]'
assert command == ('borg', 'info', '--json', '--repo', 'repo')
def test_display_archives_info_with_archive_calls_borg_with_match_archives_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_archive_uses_match_archives_flags():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
'archive', None, '2.3.4'
).and_return(('--match-archives', 'archive'))
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--match-archives', 'archive', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive='archive', json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--match-archives', 'archive', '--repo', 'repo')
def test_display_archives_info_with_local_path_calls_borg_via_local_path():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_local_path_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg1', 'info', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg1',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg1',
remote_path=None,
)
command == ('borg1', 'info', '--repo', 'repo')
def test_display_archives_info_with_remote_path_calls_borg_with_remote_path_parameters():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_remote_path_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_flags').with_args(
'remote-path', 'borg1'
@@ -235,27 +189,21 @@ def test_display_archives_info_with_remote_path_calls_borg_with_remote_path_para
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--remote-path', 'borg1', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path='borg1',
)
assert command == ('borg', 'info', '--remote-path', 'borg1', '--repo', 'repo')
def test_display_archives_info_with_log_json_calls_borg_with_log_json_parameters():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_log_json_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_flags').with_args('log-json', True).and_return(
('--log-json',)
@@ -265,26 +213,21 @@ def test_display_archives_info_with_log_json_calls_borg_with_log_json_parameters
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--log-json', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=True),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--log-json', '--repo', 'repo')
def test_display_archives_info_with_lock_wait_calls_borg_with_lock_wait_parameters():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_lock_wait_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_flags').with_args('lock-wait', 5).and_return(
('--lock-wait', '5')
@@ -295,26 +238,21 @@ def test_display_archives_info_with_lock_wait_calls_borg_with_lock_wait_paramete
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
config = {'lock_wait': 5}
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--lock-wait', '5', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config=config,
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--lock-wait', '5', '--repo', 'repo')
def test_display_archives_info_transforms_prefix_into_match_archives_parameters():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_transforms_prefix_into_match_archives_flags():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_flags').with_args(
'match-archives', 'sh:foo*'
@@ -324,26 +262,21 @@ def test_display_archives_info_transforms_prefix_into_match_archives_parameters(
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--match-archives', 'sh:foo*', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix='foo'),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--match-archives', 'sh:foo*', '--repo', 'repo')
def test_display_archives_info_prefers_prefix_over_archive_name_format():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_prefers_prefix_over_archive_name_format():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_flags').with_args(
'match-archives', 'sh:foo*'
@@ -353,52 +286,42 @@ def test_display_archives_info_prefers_prefix_over_archive_name_format():
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--match-archives', 'sh:foo*', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={'archive_name_format': 'bar-{now}'}, # noqa: FS003
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix='foo'),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--match-archives', 'sh:foo*', '--repo', 'repo')
def test_display_archives_info_transforms_archive_name_format_into_match_archives_parameters():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_transforms_archive_name_format_into_match_archives_flags():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, 'bar-{now}', '2.3.4' # noqa: FS003
).and_return(('--match-archives', 'sh:bar-*'))
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--match-archives', 'sh:bar-*', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={'archive_name_format': 'bar-{now}'}, # noqa: FS003
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--match-archives', 'sh:bar-*', '--repo', 'repo')
def test_display_archives_with_match_archives_option_calls_borg_with_match_archives_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_match_archives_option_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
'sh:foo-*', 'bar-{now}', '2.3.4' # noqa: FS003
@@ -406,14 +329,8 @@ def test_display_archives_with_match_archives_option_calls_borg_with_match_archi
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--match-archives', 'sh:foo-*', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={
'archive_name_format': 'bar-{now}', # noqa: FS003
@@ -422,12 +339,14 @@ def test_display_archives_with_match_archives_option_calls_borg_with_match_archi
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--match-archives', 'sh:foo-*', '--repo', 'repo')
def test_display_archives_with_match_archives_flag_calls_borg_with_match_archives_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_match_archives_flag_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
'sh:foo-*', 'bar-{now}', '2.3.4' # noqa: FS003
@@ -435,26 +354,22 @@ def test_display_archives_with_match_archives_flag_calls_borg_with_match_archive
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', '--match-archives', 'sh:foo-*', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={'archive_name_format': 'bar-{now}'}, # noqa: FS003
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives='sh:foo-*'),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--match-archives', 'sh:foo-*', '--repo', 'repo')
@pytest.mark.parametrize('argument_name', ('sort_by', 'first', 'last'))
def test_display_archives_info_passes_through_arguments_to_borg(argument_name):
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_passes_arguments_through_to_command(argument_name):
flag_name = f"--{argument_name.replace('_', ' ')}"
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
@@ -465,14 +380,8 @@ def test_display_archives_info_passes_through_arguments_to_borg(argument_name):
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', flag_name, 'value', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
@@ -480,12 +389,14 @@ def test_display_archives_info_passes_through_arguments_to_borg(argument_name):
info_arguments=flexmock(
archive=None, json=False, prefix=None, match_archives=None, **{argument_name: 'value'}
),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', flag_name, 'value', '--repo', 'repo')
def test_display_archives_info_with_date_based_matching_calls_borg_with_date_based_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
def test_make_info_command_with_date_based_matching_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
@@ -494,26 +405,6 @@ def test_display_archives_info_with_date_based_matching_calls_borg_with_date_bas
('--newer', '1d', '--newest', '1y', '--older', '1m', '--oldest', '1w')
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
(
'borg',
'info',
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'--repo',
'repo',
),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
info_arguments = flexmock(
archive=None,
json=False,
@@ -524,10 +415,66 @@ def test_display_archives_info_with_date_based_matching_calls_borg_with_date_bas
older='1m',
oldest='1w',
)
module.display_archives_info(
command = module.make_info_command(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=info_arguments,
local_path='borg',
remote_path=None,
)
assert command == (
'borg',
'info',
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'--repo',
'repo',
)
def test_display_archives_info_calls_two_commands():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module).should_receive('make_info_command')
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').once()
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').once()
module.display_archives_info(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
)
def test_display_archives_info_with_json_calls_json_command_only():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module).should_receive('make_info_command')
flexmock(module.environment).should_receive('make_environment')
json_output = flexmock()
flexmock(module).should_receive('execute_command_and_capture_output').and_return(json_output)
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags').never()
flexmock(module).should_receive('execute_command').never()
assert (
module.display_archives_info(
repository_path='repo',
config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=True, prefix=None, match_archives=None),
)
== json_output
)
+51
View File
@@ -18,6 +18,12 @@ def test_display_repository_info_calls_borg_with_flags():
)
)
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'rinfo', '--json', '--repo', 'repo'),
borg_local_path='borg',
extra_environment=None,
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'rinfo', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
@@ -40,6 +46,12 @@ def test_display_repository_info_without_borg_features_calls_borg_with_info_sub_
flexmock(module.feature).should_receive('available').and_return(False)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'rinfo', '--json', 'repo'),
borg_local_path='borg',
extra_environment=None,
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'info', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
@@ -67,6 +79,12 @@ def test_display_repository_info_with_log_info_calls_borg_with_info_flag():
)
)
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'rinfo', '--info', '--json', '--repo', 'repo'),
borg_local_path='borg',
extra_environment=None,
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'rinfo', '--info', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
@@ -99,6 +117,7 @@ def test_display_repository_info_with_log_info_and_json_suppresses_most_borg_out
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags').never()
insert_logging_mock(logging.INFO)
json_output = module.display_repository_info(
@@ -123,6 +142,12 @@ def test_display_repository_info_with_log_debug_calls_borg_with_debug_flag():
)
)
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'rinfo', '--debug', '--show-rc', '--json', '--repo', 'repo'),
borg_local_path='borg',
extra_environment=None,
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'rinfo', '--debug', '--show-rc', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
@@ -156,6 +181,7 @@ def test_display_repository_info_with_log_debug_and_json_suppresses_most_borg_ou
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags').never()
insert_logging_mock(logging.DEBUG)
json_output = module.display_repository_info(
@@ -185,6 +211,7 @@ def test_display_repository_info_with_json_calls_borg_with_json_flag():
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags').never()
json_output = module.display_repository_info(
repository_path='repo',
@@ -208,6 +235,12 @@ def test_display_repository_info_with_local_path_calls_borg_via_local_path():
)
)
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg1', 'rinfo', '--json', '--repo', 'repo'),
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg1', 'rinfo', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
@@ -236,6 +269,12 @@ def test_display_repository_info_with_remote_path_calls_borg_with_remote_path_fl
)
)
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'rinfo', '--remote-path', 'borg1', '--json', '--repo', 'repo'),
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'rinfo', '--remote-path', 'borg1', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
@@ -264,6 +303,12 @@ def test_display_repository_info_with_log_json_calls_borg_with_log_json_flags():
)
)
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'rinfo', '--log-json', '--json', '--repo', 'repo'),
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'rinfo', '--log-json', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
@@ -292,6 +337,12 @@ def test_display_repository_info_with_lock_wait_calls_borg_with_lock_wait_flags(
)
)
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'rinfo', '--lock-wait', '5', '--json', '--repo', 'repo'),
extra_environment=None,
borg_local_path='borg',
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'rinfo', '--lock-wait', '5', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
+14 -41
View File
@@ -559,66 +559,39 @@ def test_make_rlist_command_with_match_archives_calls_borg_with_match_archives_f
assert command == ('borg', 'list', '--match-archives', 'foo-*', 'repo')
def test_list_repository_calls_borg_with_flags():
def test_list_repository_calls_two_commands():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
rlist_arguments = argparse.Namespace(json=False)
global_arguments = flexmock()
flexmock(module.feature).should_receive('available').and_return(False)
flexmock(module).should_receive('make_rlist_command').with_args(
repository_path='repo',
config={},
local_borg_version='1.2.3',
rlist_arguments=rlist_arguments,
global_arguments=global_arguments,
local_path='borg',
remote_path=None,
).and_return(('borg', 'rlist', 'repo'))
flexmock(module).should_receive('make_rlist_command')
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'rlist', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
).once()
flexmock(module).should_receive('execute_command_and_capture_output').once()
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').once()
module.list_repository(
repository_path='repo',
config={},
local_borg_version='1.2.3',
rlist_arguments=rlist_arguments,
global_arguments=global_arguments,
rlist_arguments=argparse.Namespace(json=False),
global_arguments=flexmock(),
)
def test_list_repository_with_json_returns_borg_output():
def test_list_repository_with_json_calls_json_command_only():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
rlist_arguments = argparse.Namespace(json=True)
global_arguments = flexmock()
json_output = flexmock()
flexmock(module.feature).should_receive('available').and_return(False)
flexmock(module).should_receive('make_rlist_command').with_args(
repository_path='repo',
config={},
local_borg_version='1.2.3',
rlist_arguments=rlist_arguments,
global_arguments=global_arguments,
local_path='borg',
remote_path=None,
).and_return(('borg', 'rlist', 'repo'))
flexmock(module).should_receive('make_rlist_command')
flexmock(module.environment).should_receive('make_environment')
json_output = flexmock()
flexmock(module).should_receive('execute_command_and_capture_output').and_return(json_output)
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags').never()
flexmock(module).should_receive('execute_command').never()
assert (
module.list_repository(
repository_path='repo',
config={},
local_borg_version='1.2.3',
rlist_arguments=rlist_arguments,
global_arguments=global_arguments,
rlist_arguments=argparse.Namespace(json=True),
global_arguments=flexmock(),
)
== json_output
)
+83
View File
@@ -23,6 +23,16 @@ def test_run_configuration_runs_actions_for_each_repository():
assert results == expected_results
def test_run_configuration_with_skip_actions_does_not_raise():
flexmock(module).should_receive('verbosity_to_log_level').and_return(logging.INFO)
flexmock(module.borg_version).should_receive('local_borg_version').and_return(flexmock())
flexmock(module).should_receive('run_actions').and_return(flexmock()).and_return(flexmock())
config = {'repositories': [{'path': 'foo'}, {'path': 'bar'}], 'skip_actions': ['compact']}
arguments = {'global': flexmock(monitoring_verbosity=1)}
list(module.run_configuration('test.yaml', config, arguments))
def test_run_configuration_with_invalid_borg_version_errors():
flexmock(module).should_receive('verbosity_to_log_level').and_return(logging.INFO)
flexmock(module.borg_version).should_receive('local_borg_version').and_raise(ValueError)
@@ -504,6 +514,24 @@ def test_run_actions_runs_create():
assert result == (expected,)
def test_run_actions_with_skip_actions_skips_create():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.command).should_receive('execute_hook')
flexmock(borgmatic.actions.create).should_receive('run_create').never()
tuple(
module.run_actions(
arguments={'global': flexmock(dry_run=False, log_file='foo'), 'create': flexmock()},
config_filename=flexmock(),
config={'repositories': [], 'skip_actions': ['create']},
local_path=flexmock(),
remote_path=flexmock(),
local_borg_version=flexmock(),
repository={'path': 'repo'},
)
)
def test_run_actions_runs_prune():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.command).should_receive('execute_hook')
@@ -522,6 +550,24 @@ def test_run_actions_runs_prune():
)
def test_run_actions_with_skip_actions_skips_prune():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.command).should_receive('execute_hook')
flexmock(borgmatic.actions.prune).should_receive('run_prune').never()
tuple(
module.run_actions(
arguments={'global': flexmock(dry_run=False, log_file='foo'), 'prune': flexmock()},
config_filename=flexmock(),
config={'repositories': [], 'skip_actions': ['prune']},
local_path=flexmock(),
remote_path=flexmock(),
local_borg_version=flexmock(),
repository={'path': 'repo'},
)
)
def test_run_actions_runs_compact():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.command).should_receive('execute_hook')
@@ -540,6 +586,24 @@ def test_run_actions_runs_compact():
)
def test_run_actions_with_skip_actions_skips_compact():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.command).should_receive('execute_hook')
flexmock(borgmatic.actions.compact).should_receive('run_compact').never()
tuple(
module.run_actions(
arguments={'global': flexmock(dry_run=False, log_file='foo'), 'compact': flexmock()},
config_filename=flexmock(),
config={'repositories': [], 'skip_actions': ['compact']},
local_path=flexmock(),
remote_path=flexmock(),
local_borg_version=flexmock(),
repository={'path': 'repo'},
)
)
def test_run_actions_runs_check_when_repository_enabled_for_checks():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.command).should_receive('execute_hook')
@@ -578,6 +642,25 @@ def test_run_actions_skips_check_when_repository_not_enabled_for_checks():
)
def test_run_actions_with_skip_actions_skips_check():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.command).should_receive('execute_hook')
flexmock(module.checks).should_receive('repository_enabled_for_checks').and_return(True)
flexmock(borgmatic.actions.check).should_receive('run_check').never()
tuple(
module.run_actions(
arguments={'global': flexmock(dry_run=False, log_file='foo'), 'check': flexmock()},
config_filename=flexmock(),
config={'repositories': [], 'skip_actions': ['check']},
local_path=flexmock(),
remote_path=flexmock(),
local_borg_version=flexmock(),
repository={'path': 'repo'},
)
)
def test_run_actions_runs_extract():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.command).should_receive('execute_hook')
+3 -3
View File
@@ -2,14 +2,14 @@ from borgmatic.config import checks as module
def test_repository_enabled_for_checks_defaults_to_enabled_for_all_repositories():
enabled = module.repository_enabled_for_checks('repo.borg', consistency={})
enabled = module.repository_enabled_for_checks('repo.borg', config={})
assert enabled
def test_repository_enabled_for_checks_is_enabled_for_specified_repositories():
enabled = module.repository_enabled_for_checks(
'repo.borg', consistency={'check_repositories': ['repo.borg', 'other.borg']}
'repo.borg', config={'check_repositories': ['repo.borg', 'other.borg']}
)
assert enabled
@@ -17,7 +17,7 @@ def test_repository_enabled_for_checks_is_enabled_for_specified_repositories():
def test_repository_enabled_for_checks_is_disabled_for_other_repositories():
enabled = module.repository_enabled_for_checks(
'repo.borg', consistency={'check_repositories': ['other.borg']}
'repo.borg', config={'check_repositories': ['other.borg']}
)
assert not enabled
+58
View File
@@ -0,0 +1,58 @@
import pytest
from flexmock import flexmock
from borgmatic.config import constants as module
@pytest.mark.parametrize(
'value,expected_value',
(
('3', 3),
('0', 0),
('-3', -3),
('1234', 1234),
('true', True),
('True', True),
('false', False),
('False', False),
('thing', 'thing'),
({}, {}),
({'foo': 'bar'}, {'foo': 'bar'}),
([], []),
(['foo', 'bar'], ['foo', 'bar']),
),
)
def test_coerce_scalar_converts_value(value, expected_value):
assert module.coerce_scalar(value) == expected_value
def test_apply_constants_with_empty_constants_passes_through_value():
assert module.apply_constants(value='thing', constants={}) == 'thing'
@pytest.mark.parametrize(
'value,expected_value',
(
(None, None),
('thing', 'thing'),
('{foo}', 'bar'),
('abc{foo}', 'abcbar'),
('{foo}xyz', 'barxyz'),
('{foo}{baz}', 'barquux'),
('{int}', '3'),
('{bool}', 'True'),
(['thing', 'other'], ['thing', 'other']),
(['thing', '{foo}'], ['thing', 'bar']),
(['{foo}', '{baz}'], ['bar', 'quux']),
({'key': 'value'}, {'key': 'value'}),
({'key': '{foo}'}, {'key': 'bar'}),
(3, 3),
(True, True),
(False, False),
),
)
def test_apply_constants_makes_string_substitutions(value, expected_value):
flexmock(module).should_receive('coerce_scalar').replace_with(lambda value: value)
constants = {'foo': 'bar', 'baz': 'quux', 'int': 3, 'bool': True}
assert module.apply_constants(value, constants) == expected_value
+7 -7
View File
@@ -7,7 +7,7 @@ from borgmatic.config import generate as module
def test_schema_to_sample_configuration_generates_config_map_with_examples():
flexmock(module.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
flexmock(module.ruamel.yaml.comments).should_receive('CommentedMap').replace_with(OrderedDict)
flexmock(module).should_receive('add_comments_to_configuration_object')
schema = {
'type': 'object',
@@ -32,7 +32,7 @@ def test_schema_to_sample_configuration_generates_config_map_with_examples():
def test_schema_to_sample_configuration_generates_config_sequence_of_strings_with_example():
flexmock(module.yaml.comments).should_receive('CommentedSeq').replace_with(list)
flexmock(module.ruamel.yaml.comments).should_receive('CommentedSeq').replace_with(list)
flexmock(module).should_receive('add_comments_to_configuration_sequence')
schema = {'type': 'array', 'items': {'type': 'string'}, 'example': ['hi']}
@@ -42,7 +42,7 @@ def test_schema_to_sample_configuration_generates_config_sequence_of_strings_wit
def test_schema_to_sample_configuration_generates_config_sequence_of_maps_with_examples():
flexmock(module.yaml.comments).should_receive('CommentedSeq').replace_with(list)
flexmock(module.ruamel.yaml.comments).should_receive('CommentedSeq').replace_with(list)
flexmock(module).should_receive('add_comments_to_configuration_sequence')
flexmock(module).should_receive('add_comments_to_configuration_object')
schema = {
@@ -71,7 +71,7 @@ def test_merge_source_configuration_into_destination_inserts_map_fields():
destination_config = {'foo': 'dest1', 'bar': 'dest2'}
source_config = {'foo': 'source1', 'baz': 'source2'}
flexmock(module).should_receive('remove_commented_out_sentinel')
flexmock(module).should_receive('yaml.comments.CommentedSeq').replace_with(list)
flexmock(module).should_receive('ruamel.yaml.comments.CommentedSeq').replace_with(list)
module.merge_source_configuration_into_destination(destination_config, source_config)
@@ -82,7 +82,7 @@ def test_merge_source_configuration_into_destination_inserts_nested_map_fields()
destination_config = {'foo': {'first': 'dest1', 'second': 'dest2'}, 'bar': 'dest3'}
source_config = {'foo': {'first': 'source1'}}
flexmock(module).should_receive('remove_commented_out_sentinel')
flexmock(module).should_receive('yaml.comments.CommentedSeq').replace_with(list)
flexmock(module).should_receive('ruamel.yaml.comments.CommentedSeq').replace_with(list)
module.merge_source_configuration_into_destination(destination_config, source_config)
@@ -93,7 +93,7 @@ def test_merge_source_configuration_into_destination_inserts_sequence_fields():
destination_config = {'foo': ['dest1', 'dest2'], 'bar': ['dest3'], 'baz': ['dest4']}
source_config = {'foo': ['source1'], 'bar': ['source2', 'source3']}
flexmock(module).should_receive('remove_commented_out_sentinel')
flexmock(module).should_receive('yaml.comments.CommentedSeq').replace_with(list)
flexmock(module).should_receive('ruamel.yaml.comments.CommentedSeq').replace_with(list)
module.merge_source_configuration_into_destination(destination_config, source_config)
@@ -108,7 +108,7 @@ def test_merge_source_configuration_into_destination_inserts_sequence_of_maps():
destination_config = {'foo': [{'first': 'dest1', 'second': 'dest2'}], 'bar': 'dest3'}
source_config = {'foo': [{'first': 'source1'}, {'other': 'source2'}]}
flexmock(module).should_receive('remove_commented_out_sentinel')
flexmock(module).should_receive('yaml.comments.CommentedSeq').replace_with(list)
flexmock(module).should_receive('ruamel.yaml.comments.CommentedSeq').replace_with(list)
module.merge_source_configuration_into_destination(destination_config, source_config)
+5
View File
@@ -77,6 +77,11 @@ from borgmatic.config import normalize as module
{'bar': 'baz', 'prefix': 'foo'},
True,
),
(
{'location': {}, 'consistency': {'prefix': 'foo'}},
{'prefix': 'foo'},
True,
),
(
{},
{},
+41 -10
View File
@@ -44,6 +44,24 @@ def test_set_values_with_multiple_keys_updates_hierarchy():
assert config == {'option': {'key': 'value', 'other': 'other_value'}}
@pytest.mark.parametrize(
'schema,option_keys,expected_type',
(
({'properties': {'foo': {'type': 'array'}}}, ('foo',), 'array'),
(
{'properties': {'foo': {'properties': {'bar': {'type': 'array'}}}}},
('foo', 'bar'),
'array',
),
({'properties': {'foo': {'type': 'array'}}}, ('other',), None),
({'properties': {'foo': {'description': 'stuff'}}}, ('foo',), None),
({}, ('foo',), None),
),
)
def test_type_for_option_grabs_type_if_found_in_schema(schema, option_keys, expected_type):
assert module.type_for_option(schema, option_keys) == expected_type
@pytest.mark.parametrize(
'key,expected_key',
(
@@ -63,51 +81,64 @@ def test_strip_section_names_passes_through_key_without_section_name(key, expect
def test_parse_overrides_splits_keys_and_values():
flexmock(module).should_receive('strip_section_names').replace_with(lambda value: value)
flexmock(module).should_receive('convert_value_type').replace_with(lambda value: value)
flexmock(module).should_receive('type_for_option').and_return('string')
flexmock(module).should_receive('convert_value_type').replace_with(
lambda value, option_type: value
)
raw_overrides = ['option.my_option=value1', 'other_option=value2']
expected_result = (
(('option', 'my_option'), 'value1'),
(('other_option'), 'value2'),
)
module.parse_overrides(raw_overrides) == expected_result
module.parse_overrides(raw_overrides, schema={}) == expected_result
def test_parse_overrides_allows_value_with_equal_sign():
flexmock(module).should_receive('strip_section_names').replace_with(lambda value: value)
flexmock(module).should_receive('convert_value_type').replace_with(lambda value: value)
flexmock(module).should_receive('type_for_option').and_return('string')
flexmock(module).should_receive('convert_value_type').replace_with(
lambda value, option_type: value
)
raw_overrides = ['option=this===value']
expected_result = ((('option',), 'this===value'),)
module.parse_overrides(raw_overrides) == expected_result
module.parse_overrides(raw_overrides, schema={}) == expected_result
def test_parse_overrides_raises_on_missing_equal_sign():
flexmock(module).should_receive('strip_section_names').replace_with(lambda value: value)
flexmock(module).should_receive('convert_value_type').replace_with(lambda value: value)
flexmock(module).should_receive('type_for_option').and_return('string')
flexmock(module).should_receive('convert_value_type').replace_with(
lambda value, option_type: value
)
raw_overrides = ['option']
with pytest.raises(ValueError):
module.parse_overrides(raw_overrides)
module.parse_overrides(raw_overrides, schema={})
def test_parse_overrides_raises_on_invalid_override_value():
flexmock(module).should_receive('strip_section_names').replace_with(lambda value: value)
flexmock(module).should_receive('type_for_option').and_return('string')
flexmock(module).should_receive('convert_value_type').and_raise(ruamel.yaml.parser.ParserError)
raw_overrides = ['option=[in valid]']
with pytest.raises(ValueError):
module.parse_overrides(raw_overrides)
module.parse_overrides(raw_overrides, schema={})
def test_parse_overrides_allows_value_with_single_key():
flexmock(module).should_receive('strip_section_names').replace_with(lambda value: value)
flexmock(module).should_receive('convert_value_type').replace_with(lambda value: value)
flexmock(module).should_receive('type_for_option').and_return('string')
flexmock(module).should_receive('convert_value_type').replace_with(
lambda value, option_type: value
)
raw_overrides = ['option=value']
expected_result = ((('option',), 'value'),)
module.parse_overrides(raw_overrides) == expected_result
module.parse_overrides(raw_overrides, schema={}) == expected_result
def test_parse_overrides_handles_empty_overrides():
module.parse_overrides(raw_overrides=None) == ()
module.parse_overrides(raw_overrides=None, schema={}) == ()
+223
View File
@@ -0,0 +1,223 @@
import apprise
from apprise import NotifyFormat, NotifyType
from flexmock import flexmock
import borgmatic.hooks.monitor
from borgmatic.hooks import apprise as module
TOPIC = 'borgmatic-unit-testing'
def mock_apprise():
apprise_mock = flexmock(
add=lambda servers: None, notify=lambda title, body, body_format, notify_type: None
)
flexmock(apprise.Apprise).new_instances(apprise_mock)
return apprise_mock
def test_ping_monitor_adheres_dry_run():
mock_apprise().should_receive('notify').never()
module.ping_monitor(
{'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}]},
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=True,
)
def test_ping_monitor_does_not_hit_with_no_states():
mock_apprise().should_receive('notify').never()
module.ping_monitor(
{'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}], 'states': []},
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=True,
)
def test_ping_monitor_hits_fail_by_default():
mock_apprise().should_receive('notify').with_args(
title='A borgmatic FAIL event happened',
body='A borgmatic FAIL event happened',
body_format=NotifyFormat.TEXT,
notify_type=NotifyType.FAILURE,
).once()
for state in borgmatic.hooks.monitor.State:
module.ping_monitor(
{'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}]},
{},
'config.yaml',
state,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_hits_with_finish_default_config():
mock_apprise().should_receive('notify').with_args(
title='A borgmatic FINISH event happened',
body='A borgmatic FINISH event happened',
body_format=NotifyFormat.TEXT,
notify_type=NotifyType.SUCCESS,
).once()
module.ping_monitor(
{'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}], 'states': ['finish']},
{},
'config.yaml',
borgmatic.hooks.monitor.State.FINISH,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_hits_with_start_default_config():
mock_apprise().should_receive('notify').with_args(
title='A borgmatic START event happened',
body='A borgmatic START event happened',
body_format=NotifyFormat.TEXT,
notify_type=NotifyType.INFO,
).once()
module.ping_monitor(
{'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}], 'states': ['start']},
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_hits_with_fail_default_config():
mock_apprise().should_receive('notify').with_args(
title='A borgmatic FAIL event happened',
body='A borgmatic FAIL event happened',
body_format=NotifyFormat.TEXT,
notify_type=NotifyType.FAILURE,
).once()
module.ping_monitor(
{'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}], 'states': ['fail']},
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_hits_with_log_default_config():
mock_apprise().should_receive('notify').with_args(
title='A borgmatic LOG event happened',
body='A borgmatic LOG event happened',
body_format=NotifyFormat.TEXT,
notify_type=NotifyType.INFO,
).once()
module.ping_monitor(
{'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}], 'states': ['log']},
{},
'config.yaml',
borgmatic.hooks.monitor.State.LOG,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_passes_through_custom_message_title():
mock_apprise().should_receive('notify').with_args(
title='foo',
body='bar',
body_format=NotifyFormat.TEXT,
notify_type=NotifyType.FAILURE,
).once()
module.ping_monitor(
{
'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}],
'states': ['fail'],
'fail': {'title': 'foo', 'body': 'bar'},
},
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_passes_through_custom_message_body():
mock_apprise().should_receive('notify').with_args(
title='',
body='baz',
body_format=NotifyFormat.TEXT,
notify_type=NotifyType.FAILURE,
).once()
module.ping_monitor(
{
'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}],
'states': ['fail'],
'fail': {'body': 'baz'},
},
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_pings_multiple_services():
mock_apprise().should_receive('add').with_args([f'ntfys://{TOPIC}', f'ntfy://{TOPIC}']).once()
module.ping_monitor(
{
'services': [
{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'},
{'url': f'ntfy://{TOPIC}', 'label': 'ntfy'},
]
},
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_logs_info_for_no_services():
flexmock(module.logger).should_receive('info').once()
module.ping_monitor(
{'services': []},
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_logs_warning_when_notify_fails():
mock_apprise().should_receive('notify').and_return(False)
flexmock(module.logger).should_receive('warning').once()
for state in borgmatic.hooks.monitor.State:
module.ping_monitor(
{'services': [{'url': f'ntfys://{TOPIC}', 'label': 'ntfys'}]},
{},
'config.yaml',
state,
monitoring_log_level=1,
dry_run=False,
)
+11 -9
View File
@@ -1,19 +1,19 @@
[tox]
envlist = py37,py38,py39,py310,py311
env_list = py38,py39,py310,py311,py312
skip_missing_interpreters = True
skipsdist = True
minversion = 3.14.1
package = editable
min_version = 4.0
[testenv]
usedevelop = True
deps = -rtest_requirements.txt
deps =
-r test_requirements.txt
whitelist_externals =
find
sh
passenv = COVERAGE_FILE
commands =
pytest {posargs}
py38,py39,py310,py311: black --check .
black --check .
isort --check-only --settings-path setup.cfg .
flake8 borgmatic tests
codespell
@@ -27,10 +27,12 @@ commands =
pytest {posargs}
[testenv:end-to-end]
usedevelop = False
deps = -rtest_requirements.txt
package = editable
deps =
-r test_requirements.txt
pymongo==4.4.1
.
passenv = COVERAGE_FILE
pass_env = COVERAGE_FILE
commands =
pytest {posargs} --no-cov tests/end-to-end