Compare commits

..
45 Commits
Author SHA1 Message Date
Dan Helfman e80f27f922 Bump version for release. 2023-06-03 22:14:21 -07:00
Dan Helfman 1a5b3c9e4e Add Fedora schema loading fix to NEWS (#703). 2023-06-03 22:07:24 -07:00
Dan Helfman b3f70434df Fix error loading configuration schema on Fedora Linux (#703).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/702
2023-06-04 05:04:41 +00:00
Felix Kaechele c61d63b235 Use open() to test for file existance and readability
Signed-off-by: Felix Kaechele <felix@kaechele.ca>
2023-06-04 00:54:29 -04:00
Felix Kaechele ba0899660d Verify that schema path exists before returning it
Signed-off-by: Felix Kaechele <felix@kaechele.ca>
2023-06-03 23:42:20 -04:00
Felix Kaechele 15cabb93ca Drop importlib_metadata entirely
The fallback option using the dirname of the config module location
seems to be more robust in a number of cases.

Signed-off-by: Felix Kaechele <felix@kaechele.ca>
2023-06-03 23:42:20 -04:00
Felix Kaechele ce6daff12f Fix importlib.metadata.files workaround
Some distributions, such as Fedora, do not install the RECORDS file as
part of a package's dist-info. As a result importlib.metadata.files will
return None.

Use the workaround for these cases as well.

Signed-off-by: Felix Kaechele <felix@kaechele.ca>
2023-06-03 23:42:20 -04:00
Dan Helfman caf654366c Document work-around for colons in YAML strings (#708). 2023-06-03 10:19:34 -07:00
Dan Helfman 4f49b345af NEWS wording fix for clarity (#706). 2023-05-30 23:21:55 -07:00
Dan Helfman 1784ca5910 Fix "check" action error when repository and archive checks are configured but the archive check gets skipped due to the configured frequency (#704). 2023-05-30 23:19:33 -07:00
Dan Helfman 8f4cce5fa5 Make dev docs message stand out a little more. 2023-05-30 22:30:06 -07:00
Dan Helfman 518aeabb2a Document verbosity levels (#484). 2023-05-30 22:25:27 -07:00
Dan Helfman 341bd4118d Fix "--archive latest" on "list" and "info" actions only working on the first of multiple configured repositories (#706). 2023-05-30 16:53:55 -07:00
Dan Helfman b222f6a60b Mention new verbosity level to NEWS (#484). 2023-05-30 15:52:49 -07:00
Dan Helfman c0aaba6891 Add option to disable syslog output (#484).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/675
2023-05-30 20:03:56 +00:00
Soumik Dutta a7f81d538d nit changes
- help strings in borgmatic commands
- test fixes in test_logger and test_borgmatic

Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-29 01:09:00 +05:30
Soumik Dutta 3d41ed3a34 add test to check that log_file is disabled
if logging is disabled

Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 09:58:53 +05:30
Soumik Dutta 0283f9ae2a fix help string
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Soumik Dutta d556a23f97 update borgmatic tests
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Soumik Dutta f98d07e8d8 fix logger test
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Soumik Dutta 09f59ad97d disable monitoring hooks if monitoring_log_level is set to DISABLED
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Soumik Dutta 24be6272ed add test for logger
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Soumik Dutta 5a9bb4b97f update help strings
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Soumik Dutta 6a2eb1f157 make value of disabled level higher
so that no other log has higher priority

Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Soumik Dutta 99473c30a8 disable sending logs in Healthchecks
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Soumik Dutta f512d1e460 add verbosity level -2
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
2023-05-26 08:57:12 +05:30
Dan Helfman 84c21b062f Fix incorrect argument ordering (#659). 2023-05-23 16:55:40 -07:00
Dan Helfman 76138faaf3 Add integration test for mount action (#659). 2023-05-23 14:49:04 -07:00
Dan Helfman 9299841a5b Add date-based matching flags to NEWS (#659). 2023-05-23 14:30:16 -07:00
Dan Helfman 35b5c62ca6 Add Borg 2 date-based matching flags for archive selection (#659).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/661
2023-05-23 21:26:17 +00:00
Dan Helfman 05b989347c Upgrade requests test requirement (security). 2023-05-23 08:43:45 -07:00
Chirag Aggarwal 00e9bb011a test should mock out make_flags_from_arguments
Signed-off-by: Chirag Aggarwal <thechiragaggarwal@gmail.com>
2023-05-20 09:23:09 -04:00
Dan Helfman 833796d1c4 Add archive check probing logic tweak to NEWS (#688). 2023-05-17 08:48:54 -07:00
Dan Helfman e3425f48be Instead of taking the first check time found, take the maximum value (#688) 2023-05-16 10:20:52 -07:00
jetchirag a62ac42cca Merge branch 'main' into borg2-archive-flags 2023-04-27 16:57:29 +00:00
Chirag Aggarwal 68ee9687f5 Added tests for all subcommands and used black formatter
Signed-off-by: Chirag Aggarwal <thechiragaggarwal@gmail.com>
2023-04-27 22:27:23 +05:30
Chirag Aggarwal 32395e47f9 Added duplicate flags test for prune
Signed-off-by: Chirag Aggarwal <thechiragaggarwal@gmail.com>
2023-04-24 20:49:41 +05:30
Chirag Aggarwal 8aaba9bb0a Added new flags to prune test for review
Signed-off-by: Chirag Aggarwal <thechiragaggarwal@gmail.com>
2023-04-24 20:43:34 +05:30
Chirag Aggarwal 96aca4f446 Updated existing tests to use new parameters
Signed-off-by: Chirag Aggarwal <thechiragaggarwal@gmail.com>
2023-04-24 20:24:41 +05:30
Chirag Aggarwal 1ee56805f1 Merge remote-tracking branch 'upstream/master' into borg2-archive-flags 2023-04-15 17:29:20 +05:30
Chirag Aggarwal 98c6aa6443 Use Square brackets to denote version specific flag
Signed-off-by: Chirag Aggarwal <thechiragaggarwal@gmail.com>
2023-03-28 18:15:49 +05:30
Chirag Aggarwal edd79ed86c removed individual action parameters, and used make_flags_from_arguments
Signed-off-by: Chirag Aggarwal <thechiragaggarwal@gmail.com>
2023-03-28 18:10:42 +05:30
Chirag Aggarwal 4fa4fccab7 Use make_flags_from_arguments on mount; Pending test fixes
Signed-off-by: Chirag Aggarwal <thechiragaggarwal@gmail.com>
2023-03-27 23:24:17 +05:30
jetchirag ff1f4dc09c minor fixes to prune argument help text 2023-03-26 02:06:46 +05:30
jetchirag 141474ff07 Added TIMESPAN flags to match archive in various commands (Borg2 feature)
Signed-off-by: jetchirag <thechiragaggarwal@gmail.com>
2023-03-26 01:58:03 +05:30
35 changed files with 683 additions and 147 deletions
+11
View File
@@ -1,3 +1,14 @@
1.7.14
* #484: Add a new verbosity level (-2) to disable output entirely (for console, syslog, log file,
or monitoring), so not even errors are shown.
* #688: Tweak archive check probing logic to use the newest timestamp found when multiple exist.
* #659: Add Borg 2 date-based matching flags to various actions for archive selection.
* #703: Fix an error when loading the configuration schema on Fedora Linux.
* #704: Fix "check" action error when repository and archive checks are configured but the archive
check gets skipped due to the configured frequency.
* #706: Fix "--archive latest" on "list" and "info" actions that only worked on the first of
multiple configured repositories.
1.7.13
* #375: Restore particular PostgreSQL schemas from a database dump via "borgmatic restore --schema"
flag. See the documentation for more information:
+9
View File
@@ -0,0 +1,9 @@
import argparse
def update_arguments(arguments, **updates):
'''
Given an argparse.Namespace instance of command-line arguments and one or more keyword argument
updates to perform, return a copy of the arguments with those updates applied.
'''
return argparse.Namespace(**dict(vars(arguments), **updates))
+3 -2
View File
@@ -1,6 +1,7 @@
import json
import logging
import borgmatic.actions.arguments
import borgmatic.borg.info
import borgmatic.borg.rlist
import borgmatic.config.validate
@@ -29,7 +30,7 @@ def run_info(
logger.answer(
f'{repository.get("label", repository["path"])}: Displaying archive summary information'
)
info_arguments.archive = borgmatic.borg.rlist.resolve_archive_name(
archive_name = borgmatic.borg.rlist.resolve_archive_name(
repository['path'],
info_arguments.archive,
storage,
@@ -42,7 +43,7 @@ def run_info(
repository['path'],
storage,
local_borg_version,
info_arguments,
borgmatic.actions.arguments.update_arguments(info_arguments, archive=archive_name),
global_arguments,
local_path,
remote_path,
+4 -2
View File
@@ -1,6 +1,7 @@
import json
import logging
import borgmatic.actions.arguments
import borgmatic.borg.list
import borgmatic.config.validate
@@ -29,7 +30,8 @@ def run_list(
logger.answer(f'{repository.get("label", repository["path"])}: Searching archives')
elif not list_arguments.archive:
logger.answer(f'{repository.get("label", repository["path"])}: Listing archives')
list_arguments.archive = borgmatic.borg.rlist.resolve_archive_name(
archive_name = borgmatic.borg.rlist.resolve_archive_name(
repository['path'],
list_arguments.archive,
storage,
@@ -42,7 +44,7 @@ def run_list(
repository['path'],
storage,
local_borg_version,
list_arguments,
borgmatic.actions.arguments.update_arguments(list_arguments, archive=archive_name),
global_arguments,
local_path,
remote_path,
+1 -4
View File
@@ -40,10 +40,7 @@ def run_mount(
local_path,
remote_path,
),
mount_arguments.mount_point,
mount_arguments.paths,
mount_arguments.foreground,
mount_arguments.options,
mount_arguments,
storage,
local_borg_version,
global_arguments,
+1 -2
View File
@@ -44,11 +44,10 @@ def run_prune(
storage,
retention,
local_borg_version,
prune_arguments,
global_arguments,
local_path=local_path,
remote_path=remote_path,
stats=prune_arguments.stats,
list_archives=prune_arguments.list_archives,
)
borgmatic.hooks.command.execute_hook(
hooks.get('after_prune'),
+4 -4
View File
@@ -226,7 +226,7 @@ def make_check_flags(checks, archive_filter_flags):
else:
data_flags = ()
common_flags = archive_filter_flags + data_flags
common_flags = (archive_filter_flags if 'archives' in checks else ()) + data_flags
if {'repository', 'archives'}.issubset(set(checks)):
return common_flags
@@ -299,7 +299,7 @@ def probe_for_check_time(location_config, borg_repository_id, check, archives_ch
~/.borgmatic/checks/1234567890/archives/9876543210
~/.borgmatic/checks/1234567890/archives/all
... and returns the modification time of the first file found (if any). The first path
... and returns the maximum modification time of the files found (if any). The first path
represents a more specific archives check time (a check on a subset of archives), and the second
is a fallback to the last "all" archives check.
@@ -318,8 +318,8 @@ def probe_for_check_time(location_config, borg_repository_id, check, archives_ch
)
try:
return next(check_time for check_time in check_times if check_time)
except StopIteration:
return max(check_time for check_time in check_times if check_time)
except ValueError:
return None
+9 -9
View File
@@ -9,10 +9,7 @@ logger = logging.getLogger(__name__)
def mount_archive(
repository_path,
archive,
mount_point,
paths,
foreground,
options,
mount_arguments,
storage_config,
local_borg_version,
global_arguments,
@@ -36,8 +33,11 @@ def mount_archive(
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ (('--foreground',) if foreground else ())
+ (('-o', options) if options else ())
+ flags.make_flags_from_arguments(
mount_arguments,
excludes=('repository', 'archive', 'mount_point', 'paths', 'options'),
)
+ (('-o', mount_arguments.options) if mount_arguments.options else ())
+ (
(
flags.make_repository_flags(repository_path, local_borg_version)
@@ -54,14 +54,14 @@ def mount_archive(
else flags.make_repository_flags(repository_path, local_borg_version)
)
)
+ (mount_point,)
+ (tuple(paths) if paths else ())
+ (mount_arguments.mount_point,)
+ (tuple(mount_arguments.paths) if mount_arguments.paths else ())
)
borg_environment = environment.make_environment(storage_config)
# Don't capture the output when foreground mode is used so that ctrl-C can work properly.
if foreground:
if mount_arguments.foreground:
execute_command(
full_command,
output_file=DO_NOT_CAPTURE,
+8 -5
View File
@@ -53,11 +53,10 @@ def prune_archives(
storage_config,
retention_config,
local_borg_version,
prune_arguments,
global_arguments,
local_path='borg',
remote_path=None,
stats=False,
list_archives=False,
):
'''
Given dry-run flag, a local or remote repository path, a storage config dict, and a
@@ -76,16 +75,20 @@ def prune_archives(
+ (('--umask', str(umask)) if umask else ())
+ (('--log-json',) if global_arguments.log_json else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--stats',) if stats and not dry_run else ())
+ (('--stats',) if prune_arguments.stats and not dry_run else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--list',) if list_archives else ())
+ flags.make_flags_from_arguments(
prune_arguments,
excludes=('repository', 'stats', 'list_archives'),
)
+ (('--list',) if prune_arguments.list_archives else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ (('--dry-run',) if dry_run else ())
+ (tuple(extra_borg_options.split(' ')) if extra_borg_options else ())
+ flags.make_repository_flags(repository_path, local_borg_version)
)
if stats or list_archives:
if prune_arguments.stats or prune_arguments.list_archives:
output_log_level = logging.ANSWER
else:
output_log_level = logging.INFO
+123 -15
View File
@@ -152,30 +152,30 @@ def make_parsers():
'-v',
'--verbosity',
type=int,
choices=range(-1, 3),
choices=range(-2, 3),
default=0,
help='Display verbose progress to the console (from only errors to very verbose: -1, 0, 1, or 2)',
help='Display verbose progress to the console (disabled, errors only, default, some, or lots: -2, -1, 0, 1, or 2)',
)
global_group.add_argument(
'--syslog-verbosity',
type=int,
choices=range(-1, 3),
choices=range(-2, 3),
default=0,
help='Log verbose progress to syslog (from only errors to very verbose: -1, 0, 1, or 2). Ignored when console is interactive or --log-file is given',
help='Log verbose progress to syslog (disabled, errors only, default, some, or lots: -2, -1, 0, 1, or 2). Ignored when console is interactive or --log-file is given',
)
global_group.add_argument(
'--log-file-verbosity',
type=int,
choices=range(-1, 3),
choices=range(-2, 3),
default=0,
help='Log verbose progress to log file (from only errors to very verbose: -1, 0, 1, or 2). Only used when --log-file is given',
help='Log verbose progress to log file (disabled, errors only, default, some, or lots: -2, -1, 0, 1, or 2). Only used when --log-file is given',
)
global_group.add_argument(
'--monitoring-verbosity',
type=int,
choices=range(-1, 3),
choices=range(-2, 3),
default=0,
help='Log verbose progress to monitoring integrations that support logging (from only errors to very verbose: -1, 0, 1, or 2)',
help='Log verbose progress to monitoring integrations that support logging (from disabled, errors only, default, some, or lots: -2, -1, 0, 1, or 2)',
)
global_group.add_argument(
'--log-file',
@@ -259,7 +259,7 @@ def make_parsers():
'--source-repository',
'--other-repo',
metavar='KEY_REPOSITORY',
help='Path to an existing Borg repository whose key material should be reused (Borg 2.x+ only)',
help='Path to an existing Borg repository whose key material should be reused [Borg 2.x+ only]',
)
rcreate_group.add_argument(
'--repository',
@@ -268,7 +268,7 @@ def make_parsers():
rcreate_group.add_argument(
'--copy-crypt-key',
action='store_true',
help='Copy the crypt key used for authenticated encryption from the source repository, defaults to a new random key (Borg 2.x+ only)',
help='Copy the crypt key used for authenticated encryption from the source repository, defaults to a new random key [Borg 2.x+ only]',
)
rcreate_group.add_argument(
'--append-only',
@@ -291,8 +291,8 @@ def make_parsers():
transfer_parser = subparsers.add_parser(
'transfer',
aliases=SUBPARSER_ALIASES['transfer'],
help='Transfer archives from one repository to another, optionally upgrading the transferred data (Borg 2.0+ only)',
description='Transfer archives from one repository to another, optionally upgrading the transferred data (Borg 2.0+ only)',
help='Transfer archives from one repository to another, optionally upgrading the transferred data [Borg 2.0+ only]',
description='Transfer archives from one repository to another, optionally upgrading the transferred data [Borg 2.0+ only]',
add_help=False,
)
transfer_group = transfer_parser.add_argument_group('transfer arguments')
@@ -337,6 +337,26 @@ def make_parsers():
transfer_group.add_argument(
'--last', metavar='N', help='Only transfer last N archives after other filters are applied'
)
transfer_group.add_argument(
'--oldest',
metavar='TIMESPAN',
help='Transfer archives within a specified time range starting from the timestamp of the oldest archive (e.g. 7d or 12m) [Borg 2.x+ only]',
)
transfer_group.add_argument(
'--newest',
metavar='TIMESPAN',
help='Transfer archives within a time range that ends at timestamp of the newest archive and starts a specified time range ago (e.g. 7d or 12m) [Borg 2.x+ only]',
)
transfer_group.add_argument(
'--older',
metavar='TIMESPAN',
help='Transfer archives that are older than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
transfer_group.add_argument(
'--newer',
metavar='TIMESPAN',
help='Transfer archives that are newer than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
transfer_group.add_argument(
'-h', '--help', action='help', help='Show this help message and exit'
)
@@ -363,13 +383,33 @@ def make_parsers():
prune_group.add_argument(
'--list', dest='list_archives', action='store_true', help='List archives kept/pruned'
)
prune_group.add_argument(
'--oldest',
metavar='TIMESPAN',
help='Prune archives within a specified time range starting from the timestamp of the oldest archive (e.g. 7d or 12m) [Borg 2.x+ only]',
)
prune_group.add_argument(
'--newest',
metavar='TIMESPAN',
help='Prune archives within a time range that ends at timestamp of the newest archive and starts a specified time range ago (e.g. 7d or 12m) [Borg 2.x+ only]',
)
prune_group.add_argument(
'--older',
metavar='TIMESPAN',
help='Prune archives that are older than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
prune_group.add_argument(
'--newer',
metavar='TIMESPAN',
help='Prune archives that are newer than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
prune_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
compact_parser = subparsers.add_parser(
'compact',
aliases=SUBPARSER_ALIASES['compact'],
help='Compact segments to free space (Borg 1.2+, borgmatic 1.5.23+ only)',
description='Compact segments to free space (Borg 1.2+, borgmatic 1.5.23+ only)',
help='Compact segments to free space [Borg 1.2+, borgmatic 1.5.23+ only]',
description='Compact segments to free space [Borg 1.2+, borgmatic 1.5.23+ only]',
add_help=False,
)
compact_group = compact_parser.add_argument_group('compact arguments')
@@ -389,7 +429,7 @@ def make_parsers():
dest='cleanup_commits',
default=False,
action='store_true',
help='Cleanup commit-only 17-byte segment files left behind by Borg 1.1 (flag in Borg 1.2 only)',
help='Cleanup commit-only 17-byte segment files left behind by Borg 1.1 [flag in Borg 1.2 only]',
)
compact_group.add_argument(
'--threshold',
@@ -603,6 +643,34 @@ def make_parsers():
action='store_true',
help='Stay in foreground until ctrl-C is pressed',
)
mount_group.add_argument(
'--first',
metavar='N',
help='Mount first N archives after other filters are applied',
)
mount_group.add_argument(
'--last', metavar='N', help='Mount last N archives after other filters are applied'
)
mount_group.add_argument(
'--oldest',
metavar='TIMESPAN',
help='Mount archives within a specified time range starting from the timestamp of the oldest archive (e.g. 7d or 12m) [Borg 2.x+ only]',
)
mount_group.add_argument(
'--newest',
metavar='TIMESPAN',
help='Mount archives within a time range that ends at timestamp of the newest archive and starts a specified time range ago (e.g. 7d or 12m) [Borg 2.x+ only]',
)
mount_group.add_argument(
'--older',
metavar='TIMESPAN',
help='Mount archives that are older than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
mount_group.add_argument(
'--newer',
metavar='TIMESPAN',
help='Mount archives that are newer than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
mount_group.add_argument('--options', dest='options', help='Extra Borg mount options')
mount_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
@@ -694,6 +762,26 @@ def make_parsers():
rlist_group.add_argument(
'--last', metavar='N', help='List last N archives after other filters are applied'
)
rlist_group.add_argument(
'--oldest',
metavar='TIMESPAN',
help='List archives within a specified time range starting from the timestamp of the oldest archive (e.g. 7d or 12m) [Borg 2.x+ only]',
)
rlist_group.add_argument(
'--newest',
metavar='TIMESPAN',
help='List archives within a time range that ends at timestamp of the newest archive and starts a specified time range ago (e.g. 7d or 12m) [Borg 2.x+ only]',
)
rlist_group.add_argument(
'--older',
metavar='TIMESPAN',
help='List archives that are older than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
rlist_group.add_argument(
'--newer',
metavar='TIMESPAN',
help='List archives that are newer than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
rlist_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
list_parser = subparsers.add_parser(
@@ -825,6 +913,26 @@ def make_parsers():
info_group.add_argument(
'--last', metavar='N', help='Show info for last N archives after other filters are applied'
)
info_group.add_argument(
'--oldest',
metavar='TIMESPAN',
help='Show info for archives within a specified time range starting from the timestamp of the oldest archive (e.g. 7d or 12m) [Borg 2.x+ only]',
)
info_group.add_argument(
'--newest',
metavar='TIMESPAN',
help='Show info for archives within a time range that ends at timestamp of the newest archive and starts a specified time range ago (e.g. 7d or 12m) [Borg 2.x+ only]',
)
info_group.add_argument(
'--older',
metavar='TIMESPAN',
help='Show info for archives that are older than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
info_group.add_argument(
'--newer',
metavar='TIMESPAN',
help='Show info for archives that are newer than the specified time range (e.g. 7d or 12m) from the current time [Borg 2.x+ only]',
)
info_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
break_lock_parser = subparsers.add_parser(
+6 -5
View File
@@ -36,7 +36,7 @@ from borgmatic.borg import version as borg_version
from borgmatic.commands.arguments import parse_arguments
from borgmatic.config import checks, collect, convert, validate
from borgmatic.hooks import command, dispatch, monitor
from borgmatic.logger import add_custom_log_levels, configure_logging, should_do_markup
from borgmatic.logger import DISABLED, add_custom_log_levels, configure_logging, should_do_markup
from borgmatic.signals import configure_signals
from borgmatic.verbosity import verbosity_to_log_level
@@ -70,6 +70,7 @@ def run_configuration(config_filename, config, arguments):
error_repository = ''
using_primary_action = {'create', 'prune', 'compact', 'check'}.intersection(arguments)
monitoring_log_level = verbosity_to_log_level(global_arguments.monitoring_verbosity)
monitoring_hooks_are_activated = using_primary_action and monitoring_log_level != DISABLED
try:
local_borg_version = borg_version.local_borg_version(storage, local_path)
@@ -78,7 +79,7 @@ def run_configuration(config_filename, config, arguments):
return
try:
if using_primary_action:
if monitoring_hooks_are_activated:
dispatch.call_hooks(
'initialize_monitor',
hooks,
@@ -87,7 +88,7 @@ def run_configuration(config_filename, config, arguments):
monitoring_log_level,
global_arguments.dry_run,
)
if using_primary_action:
dispatch.call_hooks(
'ping_monitor',
hooks,
@@ -165,7 +166,7 @@ def run_configuration(config_filename, config, arguments):
error_repository = repository['path']
try:
if using_primary_action:
if monitoring_hooks_are_activated:
# send logs irrespective of error
dispatch.call_hooks(
'ping_monitor',
@@ -185,7 +186,7 @@ def run_configuration(config_filename, config, arguments):
if not encountered_error:
try:
if using_primary_action:
if monitoring_hooks_are_activated:
dispatch.call_hooks(
'ping_monitor',
hooks,
+15 -15
View File
@@ -605,7 +605,7 @@ properties:
List of one or more shell commands or scripts to execute
before all the actions for each repository.
example:
- echo "Starting actions."
- "echo Starting actions."
before_backup:
type: array
items:
@@ -614,7 +614,7 @@ properties:
List of one or more shell commands or scripts to execute
before creating a backup, run once per repository.
example:
- echo "Starting a backup."
- "echo Starting a backup."
before_prune:
type: array
items:
@@ -623,7 +623,7 @@ properties:
List of one or more shell commands or scripts to execute
before pruning, run once per repository.
example:
- echo "Starting pruning."
- "echo Starting pruning."
before_compact:
type: array
items:
@@ -632,7 +632,7 @@ properties:
List of one or more shell commands or scripts to execute
before compaction, run once per repository.
example:
- echo "Starting compaction."
- "echo Starting compaction."
before_check:
type: array
items:
@@ -641,7 +641,7 @@ properties:
List of one or more shell commands or scripts to execute
before consistency checks, run once per repository.
example:
- echo "Starting checks."
- "echo Starting checks."
before_extract:
type: array
items:
@@ -650,7 +650,7 @@ properties:
List of one or more shell commands or scripts to execute
before extracting a backup, run once per repository.
example:
- echo "Starting extracting."
- "echo Starting extracting."
after_backup:
type: array
items:
@@ -659,7 +659,7 @@ properties:
List of one or more shell commands or scripts to execute
after creating a backup, run once per repository.
example:
- echo "Finished a backup."
- "echo Finished a backup."
after_compact:
type: array
items:
@@ -668,7 +668,7 @@ properties:
List of one or more shell commands or scripts to execute
after compaction, run once per repository.
example:
- echo "Finished compaction."
- "echo Finished compaction."
after_prune:
type: array
items:
@@ -677,7 +677,7 @@ properties:
List of one or more shell commands or scripts to execute
after pruning, run once per repository.
example:
- echo "Finished pruning."
- "echo Finished pruning."
after_check:
type: array
items:
@@ -686,7 +686,7 @@ properties:
List of one or more shell commands or scripts to execute
after consistency checks, run once per repository.
example:
- echo "Finished checks."
- "echo Finished checks."
after_extract:
type: array
items:
@@ -695,7 +695,7 @@ properties:
List of one or more shell commands or scripts to execute
after extracting a backup, run once per repository.
example:
- echo "Finished extracting."
- "echo Finished extracting."
after_actions:
type: array
items:
@@ -704,7 +704,7 @@ properties:
List of one or more shell commands or scripts to execute
after all actions for each repository.
example:
- echo "Finished actions."
- "echo Finished actions."
on_error:
type: array
items:
@@ -715,7 +715,7 @@ properties:
"compact", or "check" action or an associated before/after
hook.
example:
- echo "Error during create/prune/compact/check."
- "echo Error during create/prune/compact/check."
before_everything:
type: array
items:
@@ -726,7 +726,7 @@ properties:
These are collected from all configuration files and then
run once before all of them (prior to all actions).
example:
- echo "Starting actions."
- "echo Starting actions."
after_everything:
type: array
items:
@@ -737,7 +737,7 @@ properties:
These are collected from all configuration files and then
run once after all of them (after any action).
example:
- echo "Completed actions."
- "echo Completed actions."
postgresql_databases:
type: array
items:
+4 -15
View File
@@ -3,11 +3,6 @@ import os
import jsonschema
import ruamel.yaml
try:
import importlib_metadata
except ModuleNotFoundError: # pragma: nocover
import importlib.metadata as importlib_metadata
import borgmatic.config
from borgmatic.config import environment, load, normalize, override
@@ -19,16 +14,10 @@ def schema_filename():
Raise FileNotFoundError when the schema path does not exist.
'''
try:
return next(
str(path.locate())
for path in importlib_metadata.files('borgmatic')
if path.match('config/schema.yaml')
)
except StopIteration:
# If the schema wasn't found in the package's files, this is probably a pip editable
# install, so try a different approach to get the schema.
return os.path.join(os.path.dirname(borgmatic.config.__file__), 'schema.yaml')
schema_path = os.path.join(os.path.dirname(borgmatic.config.__file__), 'schema.yaml')
with open(schema_path):
return schema_path
def format_json_error_path_element(path_element):
+6 -2
View File
@@ -141,6 +141,7 @@ def add_logging_level(level_name, level_number):
ANSWER = logging.WARN - 5
DISABLED = logging.CRITICAL + 10
def add_custom_log_levels(): # pragma: no cover
@@ -148,6 +149,7 @@ def add_custom_log_levels(): # pragma: no cover
Add a custom log level between WARN and INFO for user-requested answers.
'''
add_logging_level('ANSWER', ANSWER)
add_logging_level('DISABLED', DISABLED)
def configure_logging(
@@ -175,10 +177,12 @@ def configure_logging(
# Log certain log levels to console stderr and others to stdout. This supports use cases like
# grepping (non-error) output.
console_disabled = logging.NullHandler()
console_error_handler = logging.StreamHandler(sys.stderr)
console_standard_handler = logging.StreamHandler(sys.stdout)
console_handler = Multi_stream_handler(
{
logging.DISABLED: console_disabled,
logging.CRITICAL: console_error_handler,
logging.ERROR: console_error_handler,
logging.WARN: console_error_handler,
@@ -191,7 +195,7 @@ def configure_logging(
console_handler.setLevel(console_log_level)
syslog_path = None
if log_file is None:
if log_file is None and syslog_log_level != logging.DISABLED:
if os.path.exists('/dev/log'):
syslog_path = '/dev/log'
elif os.path.exists('/var/run/syslog'):
@@ -206,7 +210,7 @@ def configure_logging(
)
syslog_handler.setLevel(syslog_log_level)
handlers = (console_handler, syslog_handler)
elif log_file:
elif log_file and log_file_log_level != logging.DISABLED:
file_handler = logging.handlers.WatchedFileHandler(log_file)
file_handler.setFormatter(
logging.Formatter(
+2
View File
@@ -2,6 +2,7 @@ import logging
import borgmatic.logger
VERBOSITY_DISABLED = -2
VERBOSITY_ERROR = -1
VERBOSITY_ANSWER = 0
VERBOSITY_SOME = 1
@@ -15,6 +16,7 @@ def verbosity_to_log_level(verbosity):
borgmatic.logger.add_custom_log_levels()
return {
VERBOSITY_DISABLED: logging.DISABLED,
VERBOSITY_ERROR: logging.ERROR,
VERBOSITY_ANSWER: logging.ANSWER,
VERBOSITY_SOME: logging.INFO,
+1 -1
View File
@@ -17,6 +17,6 @@ services:
- sh
- -c
- |
echo "You can view dev docs at http://localhost:8080"
echo; echo "You can view dev docs at http://localhost:8080"; echo
depends_on:
- docs
@@ -28,6 +28,16 @@ hooks:
- umount /some/filesystem
```
If your command contains a special YAML character such as a colon, you may
need to quote the entire string (or use a [multiline
string](https://yaml-multiline.info/)) to avoid an error:
```yaml
hooks:
before_backup:
- "echo Backup: start"
```
<span class="minilink minilink-addedin">New in version 1.6.0</span> The
`before_backup` and `after_backup` hooks each run once per repository in a
configuration file. `before_backup` hooks runs right before the `create`
+9
View File
@@ -24,6 +24,15 @@ Or, for even more progress and debug spew:
borgmatic --verbosity 2
```
The full set of verbosity levels are:
* `-2`: disable output entirely <span class="minilink minilink-addedin">New in borgmatic 1.7.14</span>
* `-1`: only show errors
* `0`: default output
* `1`: some additional output (informational level)
* `2`: lots of additional output (debug level)
## Backup summary
If you're less concerned with progress during a backup, and you only want to
+1 -1
View File
@@ -61,4 +61,4 @@ LogRateLimitIntervalSec=0
# Delay start to prevent backups running during boot. Note that systemd-inhibit requires dbus and
# dbus-user-session to be installed.
ExecStartPre=sleep 1m
ExecStart=systemd-inhibit --who="borgmatic" --what="sleep:shutdown" --why="Prevent interrupting scheduled backup" /root/.local/bin/borgmatic --verbosity -1 --syslog-verbosity 1
ExecStart=systemd-inhibit --who="borgmatic" --what="sleep:shutdown" --why="Prevent interrupting scheduled backup" /root/.local/bin/borgmatic --verbosity -2 --syslog-verbosity 1
+1 -1
View File
@@ -1,6 +1,6 @@
from setuptools import find_packages, setup
VERSION = '1.7.13'
VERSION = '1.7.14'
setup(
+1 -1
View File
@@ -25,7 +25,7 @@ jsonschema==4.17.3
pytest==7.3.0
pytest-cov==4.0.0
regex; python_version >= '3.8'
requests==2.28.2
requests==2.31.0
ruamel.yaml>0.15.0,<0.18.0
toml==0.10.2; python_version >= '3.8'
typed-ast; python_version >= '3.8'
+45
View File
@@ -5,6 +5,8 @@ from flexmock import flexmock
import borgmatic.borg.info
import borgmatic.borg.list
import borgmatic.borg.mount
import borgmatic.borg.prune
import borgmatic.borg.rlist
import borgmatic.borg.transfer
import borgmatic.commands.arguments
@@ -68,6 +70,49 @@ def test_transfer_archives_command_does_not_duplicate_flags_or_raise():
)
def test_prune_archives_command_does_not_duplicate_flags_or_raise():
arguments = borgmatic.commands.arguments.parse_arguments('prune')['prune']
flexmock(borgmatic.borg.prune).should_receive('execute_command').replace_with(
assert_command_does_not_duplicate_flags
)
for argument_name in dir(arguments):
if argument_name.startswith('_'):
continue
borgmatic.borg.prune.prune_archives(
False,
'repo',
{},
{},
'2.3.4',
fuzz_argument(arguments, argument_name),
argparse.Namespace(log_json=False),
)
def test_mount_archive_command_does_not_duplicate_flags_or_raise():
arguments = borgmatic.commands.arguments.parse_arguments('mount', '--mount-point', 'tmp')[
'mount'
]
flexmock(borgmatic.borg.mount).should_receive('execute_command').replace_with(
assert_command_does_not_duplicate_flags
)
for argument_name in dir(arguments):
if argument_name.startswith('_'):
continue
borgmatic.borg.mount.mount_archive(
'repo',
'archive',
fuzz_argument(arguments, argument_name),
{},
'2.3.4',
argparse.Namespace(log_json=False),
)
def test_make_list_command_does_not_duplicate_flags_or_raise():
arguments = borgmatic.commands.arguments.parse_arguments('list')['list']
+10
View File
@@ -0,0 +1,10 @@
from borgmatic.actions import arguments as module
def test_update_arguments_copies_and_updates_without_modifying_original():
original = module.argparse.Namespace(foo=1, bar=2, baz=3)
result = module.update_arguments(original, bar=7, baz=8)
assert original == module.argparse.Namespace(foo=1, bar=2, baz=3)
assert result == module.argparse.Namespace(foo=1, bar=7, baz=8)
+3
View File
@@ -9,6 +9,9 @@ def test_run_info_does_not_raise():
flexmock(module.borgmatic.borg.rlist).should_receive('resolve_archive_name').and_return(
flexmock()
)
flexmock(module.borgmatic.actions.arguments).should_receive('update_arguments').and_return(
flexmock()
)
flexmock(module.borgmatic.borg.info).should_receive('display_archives_info')
info_arguments = flexmock(repository=flexmock(), archive=flexmock(), json=flexmock())
+3
View File
@@ -9,6 +9,9 @@ def test_run_list_does_not_raise():
flexmock(module.borgmatic.borg.rlist).should_receive('resolve_archive_name').and_return(
flexmock()
)
flexmock(module.borgmatic.actions.arguments).should_receive('update_arguments').and_return(
flexmock()
)
flexmock(module.borgmatic.borg.list).should_receive('list_archive')
list_arguments = flexmock(repository=flexmock(), archive=flexmock(), json=flexmock())
+9 -3
View File
@@ -356,12 +356,18 @@ def test_make_check_flags_with_archives_check_returns_flag():
assert flags == ('--archives-only',)
def test_make_check_flags_with_archive_filtler_flags_includes_those_flags():
def test_make_check_flags_with_archives_check_and_archive_filter_flags_includes_those_flags():
flags = module.make_check_flags(('archives',), ('--match-archives', 'sh:foo-*'))
assert flags == ('--archives-only', '--match-archives', 'sh:foo-*')
def test_make_check_flags_without_archives_check_and_with_archive_filter_flags_includes_those_flags():
flags = module.make_check_flags(('repository',), ('--match-archives', 'sh:foo-*'))
assert flags == ('--repository-only',)
def test_make_check_flags_with_data_check_returns_flag_and_implies_archives():
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
@@ -462,13 +468,13 @@ def test_read_check_time_on_missing_file_does_not_raise():
assert module.read_check_time('/path') is None
def test_probe_for_check_time_uses_first_of_multiple_check_times():
def test_probe_for_check_time_uses_maximum_of_multiple_check_times():
flexmock(module).should_receive('make_check_time_path').and_return(
'~/.borgmatic/checks/1234/archives/5678'
).and_return('~/.borgmatic/checks/1234/archives/all')
flexmock(module).should_receive('read_check_time').and_return(1).and_return(2)
assert module.probe_for_check_time(flexmock(), flexmock(), flexmock(), flexmock()) == 1
assert module.probe_for_check_time(flexmock(), flexmock(), flexmock(), flexmock()) == 2
def test_probe_for_check_time_deduplicates_identical_check_time_paths():
+50
View File
@@ -478,3 +478,53 @@ def test_display_archives_info_passes_through_arguments_to_borg(argument_name):
archive=None, json=False, prefix=None, match_archives=None, **{argument_name: 'value'}
),
)
def test_display_archives_info_with_date_based_matching_calls_borg_with_date_based_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(
('--newer', '1d', '--newest', '1y', '--older', '1m', '--oldest', '1w')
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
(
'borg',
'info',
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'--repo',
'repo',
),
output_log_level=module.borgmatic.logger.ANSWER,
borg_local_path='borg',
extra_environment=None,
)
info_arguments = flexmock(
archive=None,
json=False,
prefix=None,
match_archives=None,
newer='1d',
newest='1y',
older='1m',
oldest='1w',
)
module.display_archives_info(
repository_path='repo',
storage_config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=info_arguments,
)
+86 -45
View File
@@ -21,13 +21,11 @@ def test_mount_archive_calls_borg_with_required_flags():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(('borg', 'mount', 'repo', '/mnt'))
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive=None,
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -46,13 +44,11 @@ def test_mount_archive_with_borg_features_calls_borg_with_repository_and_match_a
('borg', 'mount', '--repo', 'repo', '--match-archives', 'archive', '/mnt')
)
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -66,13 +62,11 @@ def test_mount_archive_without_archive_calls_borg_with_repository_flags_only():
)
insert_execute_command_mock(('borg', 'mount', 'repo::archive', '/mnt'))
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -86,13 +80,13 @@ def test_mount_archive_calls_borg_with_path_flags():
)
insert_execute_command_mock(('borg', 'mount', 'repo::archive', '/mnt', 'path1', 'path2'))
mount_arguments = flexmock(
mount_point='/mnt', options=None, paths=['path1', 'path2'], foreground=False
)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=['path1', 'path2'],
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -108,13 +102,11 @@ def test_mount_archive_calls_borg_with_remote_path_flags():
('borg', 'mount', '--remote-path', 'borg1', 'repo::archive', '/mnt')
)
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -129,13 +121,11 @@ def test_mount_archive_calls_borg_with_umask_flags():
)
insert_execute_command_mock(('borg', 'mount', '--umask', '0770', 'repo::archive', '/mnt'))
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={'umask': '0770'},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -149,13 +139,11 @@ def test_mount_archive_calls_borg_with_log_json_flags():
)
insert_execute_command_mock(('borg', 'mount', '--log-json', 'repo::archive', '/mnt'))
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=True),
@@ -169,13 +157,11 @@ def test_mount_archive_calls_borg_with_lock_wait_flags():
)
insert_execute_command_mock(('borg', 'mount', '--lock-wait', '5', 'repo::archive', '/mnt'))
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={'lock_wait': '5'},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -190,13 +176,11 @@ def test_mount_archive_with_log_info_calls_borg_with_info_parameter():
insert_execute_command_mock(('borg', 'mount', '--info', 'repo::archive', '/mnt'))
insert_logging_mock(logging.INFO)
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -211,13 +195,11 @@ def test_mount_archive_with_log_debug_calls_borg_with_debug_flags():
insert_execute_command_mock(('borg', 'mount', '--debug', '--show-rc', 'repo::archive', '/mnt'))
insert_logging_mock(logging.DEBUG)
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=False)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=False,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -237,13 +219,11 @@ def test_mount_archive_calls_borg_with_foreground_parameter():
extra_environment=None,
).once()
mount_arguments = flexmock(mount_point='/mnt', options=None, paths=None, foreground=True)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_point='/mnt',
paths=None,
foreground=True,
options=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
@@ -257,13 +237,74 @@ def test_mount_archive_calls_borg_with_options_flags():
)
insert_execute_command_mock(('borg', 'mount', '-o', 'super_mount', 'repo::archive', '/mnt'))
mount_arguments = flexmock(
mount_point='/mnt', options='super_mount', paths=None, foreground=False
)
module.mount_archive(
repository_path='repo',
archive='archive',
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
)
def test_mount_archive_with_date_based_matching_calls_borg_with_date_based_flags():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(
(
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'--match-archives',
None,
)
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
(
'borg',
'mount',
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'--match-archives',
None,
'--repo',
'repo',
'/mnt',
),
borg_local_path='borg',
extra_environment=None,
)
mount_arguments = flexmock(
mount_point='/mnt',
options=None,
paths=None,
foreground=False,
options='super_mount',
newer='1d',
newest='1y',
older='1m',
oldest='1w',
)
module.mount_archive(
repository_path='repo',
archive=None,
mount_arguments=mount_arguments,
storage_config={},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
+88 -2
View File
@@ -117,6 +117,7 @@ def test_prune_archives_calls_borg_with_flags():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('repo',), logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -124,6 +125,7 @@ def test_prune_archives_calls_borg_with_flags():
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
prune_arguments=prune_arguments,
)
@@ -135,6 +137,7 @@ def test_prune_archives_with_log_info_calls_borg_with_info_flag():
insert_execute_command_mock(PRUNE_COMMAND + ('--info', 'repo'), logging.INFO)
insert_logging_mock(logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
repository_path='repo',
storage_config={},
@@ -142,6 +145,7 @@ def test_prune_archives_with_log_info_calls_borg_with_info_flag():
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
prune_arguments=prune_arguments,
)
@@ -153,6 +157,7 @@ def test_prune_archives_with_log_debug_calls_borg_with_debug_flag():
insert_execute_command_mock(PRUNE_COMMAND + ('--debug', '--show-rc', 'repo'), logging.INFO)
insert_logging_mock(logging.DEBUG)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
repository_path='repo',
storage_config={},
@@ -160,6 +165,7 @@ def test_prune_archives_with_log_debug_calls_borg_with_debug_flag():
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
prune_arguments=prune_arguments,
)
@@ -170,6 +176,7 @@ def test_prune_archives_with_dry_run_calls_borg_with_dry_run_flag():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('--dry-run', 'repo'), logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
repository_path='repo',
storage_config={},
@@ -177,6 +184,7 @@ def test_prune_archives_with_dry_run_calls_borg_with_dry_run_flag():
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
prune_arguments=prune_arguments,
)
@@ -187,6 +195,7 @@ def test_prune_archives_with_local_path_calls_borg_via_local_path():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(('borg1',) + PRUNE_COMMAND[1:] + ('repo',), logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -195,6 +204,7 @@ def test_prune_archives_with_local_path_calls_borg_via_local_path():
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
local_path='borg1',
prune_arguments=prune_arguments,
)
@@ -205,6 +215,7 @@ def test_prune_archives_with_remote_path_calls_borg_with_remote_path_flags():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('--remote-path', 'borg1', 'repo'), logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -213,6 +224,7 @@ def test_prune_archives_with_remote_path_calls_borg_with_remote_path_flags():
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
remote_path='borg1',
prune_arguments=prune_arguments,
)
@@ -223,6 +235,7 @@ def test_prune_archives_with_stats_calls_borg_with_stats_flag_and_answer_output_
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('--stats', 'repo'), module.borgmatic.logger.ANSWER)
prune_arguments = flexmock(stats=True, list_archives=False)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -230,7 +243,7 @@ def test_prune_archives_with_stats_calls_borg_with_stats_flag_and_answer_output_
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
stats=True,
prune_arguments=prune_arguments,
)
@@ -241,6 +254,7 @@ def test_prune_archives_with_files_calls_borg_with_list_flag_and_answer_output_l
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('--list', 'repo'), module.borgmatic.logger.ANSWER)
prune_arguments = flexmock(stats=False, list_archives=True)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -248,7 +262,7 @@ def test_prune_archives_with_files_calls_borg_with_list_flag_and_answer_output_l
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
list_archives=True,
prune_arguments=prune_arguments,
)
@@ -260,6 +274,7 @@ def test_prune_archives_with_umask_calls_borg_with_umask_flags():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('--umask', '077', 'repo'), logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -267,6 +282,7 @@ def test_prune_archives_with_umask_calls_borg_with_umask_flags():
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
prune_arguments=prune_arguments,
)
@@ -277,6 +293,7 @@ def test_prune_archives_with_log_json_calls_borg_with_log_json_flag():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('--log-json', 'repo'), logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -284,6 +301,7 @@ def test_prune_archives_with_log_json_calls_borg_with_log_json_flag():
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=True),
prune_arguments=prune_arguments,
)
@@ -295,6 +313,7 @@ def test_prune_archives_with_lock_wait_calls_borg_with_lock_wait_flags():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('--lock-wait', '5', 'repo'), logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -302,6 +321,7 @@ def test_prune_archives_with_lock_wait_calls_borg_with_lock_wait_flags():
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
prune_arguments=prune_arguments,
)
@@ -312,6 +332,7 @@ def test_prune_archives_with_extra_borg_options_calls_borg_with_extra_options():
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(PRUNE_COMMAND + ('--extra', '--options', 'repo'), logging.INFO)
prune_arguments = flexmock(stats=False, list_archives=False)
module.prune_archives(
dry_run=False,
repository_path='repo',
@@ -319,4 +340,69 @@ def test_prune_archives_with_extra_borg_options_calls_borg_with_extra_options():
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
prune_arguments=prune_arguments,
)
def test_prune_archives_with_date_based_matching_calls_borg_with_date_based_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module).should_receive('make_prune_flags').and_return(BASE_PRUNE_FLAGS)
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(
(
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'--match-archives',
None,
)
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
(
'borg',
'prune',
'--keep-daily',
'1',
'--keep-weekly',
'2',
'--keep-monthly',
'3',
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'--match-archives',
None,
'--repo',
'repo',
),
output_log_level=logging.INFO,
borg_local_path='borg',
extra_environment=None,
)
prune_arguments = flexmock(
stats=False, list_archives=False, newer='1d', newest='1y', older='1m', oldest='1w'
)
module.prune_archives(
dry_run=False,
repository_path='repo',
storage_config={},
retention_config=flexmock(),
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
prune_arguments=prune_arguments,
)
+43
View File
@@ -614,3 +614,46 @@ def test_list_repository_with_json_returns_borg_output():
)
== json_output
)
def test_make_rlist_command_with_date_based_matching_calls_borg_with_date_based_flags():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '1.2.3'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(
('--newer', '1d', '--newest', '1y', '--older', '1m', '--oldest', '1w')
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
command = module.make_rlist_command(
repository_path='repo',
storage_config={},
local_borg_version='1.2.3',
rlist_arguments=flexmock(
archive=None,
paths=None,
json=False,
prefix=None,
match_archives=None,
newer='1d',
newest='1y',
older='1m',
oldest='1w',
),
global_arguments=flexmock(log_json=False),
)
assert command == (
'borg',
'list',
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'repo',
)
+48
View File
@@ -430,3 +430,51 @@ def test_transfer_archives_with_source_repository_calls_borg_with_other_repo_fla
),
global_arguments=flexmock(log_json=False),
)
def test_transfer_archives_with_date_based_matching_calls_borg_with_date_based_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(
('--newer', '1d', '--newest', '1y', '--older', '1m', '--oldest', '1w')
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module).should_receive('execute_command').with_args(
(
'borg',
'transfer',
'--newer',
'1d',
'--newest',
'1y',
'--older',
'1m',
'--oldest',
'1w',
'--repo',
'repo',
),
output_log_level=module.borgmatic.logger.ANSWER,
output_file=None,
borg_local_path='borg',
extra_environment=None,
)
module.transfer_archives(
dry_run=False,
repository_path='repo',
storage_config={},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
transfer_arguments=flexmock(
archive=None,
progress=None,
source_repository='other',
newer='1d',
newest='1y',
older='1m',
oldest='1w',
),
)
+13
View File
@@ -169,6 +169,19 @@ def test_run_configuration_bails_for_monitor_finish_soft_failure():
assert results == []
def test_run_configuration_does_not_call_monitoring_hooks_if_monitoring_hooks_are_disabled():
flexmock(module).should_receive('verbosity_to_log_level').and_return(module.DISABLED)
flexmock(module.borg_version).should_receive('local_borg_version').and_return(flexmock())
flexmock(module.dispatch).should_receive('call_hooks').never()
flexmock(module).should_receive('run_actions').and_return([])
config = {'location': {'repositories': [{'path': 'foo'}]}}
arguments = {'global': flexmock(monitoring_verbosity=-2, dry_run=False), 'create': flexmock()}
results = list(module.run_configuration('test.yaml', config, arguments))
assert results == []
def test_run_configuration_logs_on_error_hook_error():
flexmock(module).should_receive('verbosity_to_log_level').and_return(logging.INFO)
flexmock(module.borg_version).should_receive('local_borg_version').and_return(flexmock())
+15 -12
View File
@@ -1,3 +1,7 @@
import os
import sys
from io import StringIO
import pytest
from flexmock import flexmock
@@ -7,22 +11,21 @@ from borgmatic.config import validate as module
def test_schema_filename_finds_schema_path():
schema_path = '/var/borgmatic/config/schema.yaml'
flexmock(module.importlib_metadata).should_receive('files').and_return(
flexmock(match=lambda path: False, locate=lambda: None),
flexmock(match=lambda path: True, locate=lambda: schema_path),
flexmock(match=lambda path: False, locate=lambda: None),
)
flexmock(os.path).should_receive('dirname').and_return('/var/borgmatic/config')
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args(schema_path).and_return(StringIO())
assert module.schema_filename() == schema_path
def test_schema_filename_with_missing_schema_path_in_package_still_finds_it_in_config_directory():
flexmock(module.importlib_metadata).should_receive('files').and_return(
flexmock(match=lambda path: False, locate=lambda: None),
flexmock(match=lambda path: False, locate=lambda: None),
)
def test_schema_filename_raises_filenotfounderror():
schema_path = '/var/borgmatic/config/schema.yaml'
assert module.schema_filename().endswith('/borgmatic/config/schema.yaml')
flexmock(os.path).should_receive('dirname').and_return('/var/borgmatic/config')
builtins = flexmock(sys.modules['builtins'])
builtins.should_receive('open').with_args(schema_path).and_raise(FileNotFoundError)
with pytest.raises(FileNotFoundError):
module.schema_filename()
def test_format_json_error_path_element_formats_array_index():
+39 -1
View File
@@ -285,6 +285,44 @@ def test_configure_logging_skips_syslog_if_interactive_console():
module.configure_logging(console_log_level=logging.INFO)
def test_configure_logging_skips_syslog_if_syslog_logging_is_disabled():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.logging).DISABLED = module.DISABLED
flexmock(module).should_receive('Multi_stream_handler').and_return(
flexmock(setFormatter=lambda formatter: None, setLevel=lambda level: None)
)
flexmock(module).should_receive('Console_color_formatter')
flexmock(module).should_receive('interactive_console').never()
flexmock(module.logging).should_receive('basicConfig').with_args(
level=logging.INFO, handlers=tuple
)
flexmock(module.os.path).should_receive('exists').with_args('/dev/log').and_return(True)
flexmock(module.logging.handlers).should_receive('SysLogHandler').never()
module.configure_logging(console_log_level=logging.INFO, syslog_log_level=logging.DISABLED)
def test_configure_logging_skips_log_file_if_log_file_logging_is_disabled():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.logging).DISABLED = module.DISABLED
flexmock(module).should_receive('Multi_stream_handler').and_return(
flexmock(setFormatter=lambda formatter: None, setLevel=lambda level: None)
)
# syslog skipped in non-interactive console if --log-file argument provided
flexmock(module).should_receive('interactive_console').and_return(False)
flexmock(module.logging).should_receive('basicConfig').with_args(
level=logging.INFO, handlers=tuple
)
flexmock(module.os.path).should_receive('exists').never()
flexmock(module.logging.handlers).should_receive('SysLogHandler').never()
flexmock(module.logging.handlers).should_receive('WatchedFileHandler').never()
module.configure_logging(
console_log_level=logging.INFO, log_file_log_level=logging.DISABLED, log_file='/tmp/logfile'
)
def test_configure_logging_to_log_file_instead_of_syslog():
flexmock(module).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.ANSWER
@@ -297,7 +335,7 @@ def test_configure_logging_to_log_file_instead_of_syslog():
flexmock(module.logging).should_receive('basicConfig').with_args(
level=logging.DEBUG, handlers=tuple
)
flexmock(module.os.path).should_receive('exists').with_args('/dev/log').and_return(True)
flexmock(module.os.path).should_receive('exists').never()
flexmock(module.logging.handlers).should_receive('SysLogHandler').never()
file_handler = logging.handlers.WatchedFileHandler('/tmp/logfile')
flexmock(module.logging.handlers).should_receive('WatchedFileHandler').with_args(
+2
View File
@@ -17,11 +17,13 @@ def insert_logging_mock(log_level):
def test_verbosity_to_log_level_maps_known_verbosity_to_log_level():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.logging).DISABLED = module.borgmatic.logger.DISABLED
assert module.verbosity_to_log_level(module.VERBOSITY_ERROR) == logging.ERROR
assert module.verbosity_to_log_level(module.VERBOSITY_ANSWER) == module.borgmatic.logger.ANSWER
assert module.verbosity_to_log_level(module.VERBOSITY_SOME) == logging.INFO
assert module.verbosity_to_log_level(module.VERBOSITY_LOTS) == logging.DEBUG
assert module.verbosity_to_log_level(module.VERBOSITY_DISABLED) == logging.DISABLED
def test_verbosity_to_log_level_maps_unknown_verbosity_to_warning_level():