Compare commits

...
71 Commits
Author SHA1 Message Date
Dan Helfman 4985b805b4 Bump version for release. 2024-11-18 20:40:51 -08:00
Dan Helfman d09b4c72a9 Fix a few remaining Pushover issues from the PR. 2024-11-18 20:32:17 -08:00
Dan HelfmanandGitHub 9807549f88 Add a Pushover monitoring hook.
Merge pull request #86 from tony1661/pushover-branch.
2024-11-18 20:17:28 -08:00
Dan Helfman 30c821120e Fix borgmatic ignoring the "BORG_RELOCATED_REPO_ACCESS_IS_OK" and "BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK" environment variables (#939). 2024-11-18 09:46:52 -08:00
Dan Helfman 13884bd448 Apply the "umask" option to all relevant actions, not just some of them (#441). 2024-11-18 09:07:29 -08:00
Antonio Fernandez 6bce4c4a0d changed version release to 1.9.2 2024-11-18 07:56:00 -05:00
Antonio Fernandez 25572c98d7 better fuction name 2024-11-18 07:52:19 -05:00
Antonio Fernandez dab0dfcb32 added test for value error 2024-11-18 07:49:53 -05:00
Dan Helfman 851c454ef0 Remove the restriction that the "extract" and "mount" actions must match a single repository (#722). 2024-11-17 21:39:59 -08:00
Dan Helfman c7a0cebaf7 Add a documentation link to NEWS (#934). 2024-11-17 12:46:58 -08:00
Dan Helfman 76cfeda290 Update the logic that probes for the borgmatic runtime directory to support more platforms and use cases (#934).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/937
2024-11-17 19:57:09 +00:00
Dan Helfman afdf831c59 Fix broken restore/bootstrap when using Borg 1.2 and a randomly named temporary directory (#934). 2024-11-17 11:55:10 -08:00
Dan Helfman 9ac3087304 Before creating a temp file in a directory, make sure the directory exists (#934). 2024-11-17 10:30:17 -08:00
Dan Helfman 7cca83b698 Log the path of the borgmatic runtime directory used (#934). 2024-11-17 10:15:58 -08:00
Dan Helfman 4b5df7117a Fix documentation type (#934). 2024-11-17 09:01:58 -08:00
Dan Helfman 57decfa4db Document the fact that the config bootstrap feature writes to the runtime directory (#934). 2024-11-16 16:45:49 -08:00
Dan Helfman b80f60a731 Create the borgmatic runtime directory if it doesn't exist (#934). 2024-11-16 16:03:18 -08:00
Dan Helfman 8f5ea95348 Fix use of borgmatic runtime directory in the restore action (#934). 2024-11-16 12:19:20 -08:00
Dan Helfman b0cad58d6c Add a dash to the prefix of the randomly named temporary directory to improve readability (#937). 2024-11-16 11:26:24 -08:00
Dan Helfman 073d6bddf6 Fix outdated comment (#934). 2024-11-16 07:26:23 -08:00
Dan Helfman 810b65589f Documentation for runtime/state directory changes (#934). 2024-11-15 22:23:49 -08:00
Dan Helfman 295bfb0c57 Update the logic that probes for the borgmatic streaming database dump, bootstrap metadata, and check state directories to support more platforms and use cases (#934). 2024-11-15 18:15:32 -08:00
Antonio Fernandez 5f3d4f9b03 final fix for true/false and 1/0 2024-11-13 15:27:25 -05:00
Antonio Fernandez 5321301708 fix for true/false to 1/0 2024-11-13 15:23:47 -05:00
Antonio Fernandez a939a66fb4 raise ValueError on priprity 2 with retry or expire 2024-11-13 15:20:52 -05:00
Dan Helfman c0721a8cad Fix misleading example for user_runtime_directory. 2024-11-13 11:17:40 -08:00
Antonio Fernandez ea47704d86 added missing tests to get 100% test coverage 2024-11-13 12:09:25 -05:00
Antonio Fernandez 61e4eeff6c converted html to a boolean and updated documentation and schema 2024-11-13 11:52:08 -05:00
Antonio Fernandez 3ab4b45041 added test for early exit when state is not in config 2024-11-13 09:07:47 -05:00
Tony Fernandez 4e1f256b48 added dryrun test case with minimum config 2024-11-12 10:15:19 -05:00
Tony Fernandez 96bb402837 fix test 2 2024-11-12 10:10:13 -05:00
Tony Fernandez 97949266b3 fix test 2024-11-12 10:09:23 -05:00
Tony Fernandez e69d2385fc better name for test 2024-11-12 10:08:13 -05:00
Tony Fernandez 6d9340ebb2 better comment blocks 2024-11-12 10:05:57 -05:00
Tony Fernandez 0441e79b41 "fail" -> "fails" 2024-11-12 10:00:34 -05:00
Tony Fernandez b1af304125 better data dict creation 2024-11-12 09:59:02 -05:00
Tony Fernandez eb8f7e0329 better description for expire and retry 2024-11-12 09:55:07 -05:00
Dan Helfman bf978f2db4 Fix missing build backend setting in pyproject.toml to allow Fedora builds (#932). 2024-11-10 17:09:10 -08:00
Dan Helfman 22776b123d Bump version for release. 2024-11-10 08:13:18 -08:00
Tony Fernandez ef66349674 small fixed for some failing tests 2024-11-08 20:02:29 -05:00
Tony Fernandez 51b885e7db added global constant for priority 2024-11-08 19:49:23 -05:00
Tony Fernandez 1781787305 better schema description for retry and expire 2024-11-08 19:43:40 -05:00
Tony Fernandez 46ebb0cebb removed redundant code 2024-11-08 18:45:46 -05:00
Tony Fernandez 3e0fa57860 removed tests that are not needed 2024-11-08 17:44:27 -05:00
Tony Fernandez 59f8722e05 better spacing for comments 2024-11-08 17:42:46 -05:00
Tony Fernandez 4ba42e8905 better wording. Added 'by default' 2024-11-08 17:41:18 -05:00
Tony Fernandez 3b79482b24 better wording 2024-11-08 17:38:58 -05:00
Tony Fernandez 7eb19cb0a7 added period 2024-11-08 17:38:20 -05:00
Tony Fernandez a4fabb8521 fix version 2024-11-08 17:37:26 -05:00
Tony Fernandez 85ea8f4f45 fix 10min in seconds 2024-11-08 15:03:38 -05:00
Tony Fernandez 290559116d better logic for priority 2024-11-08 15:01:28 -05:00
Tony Fernandez 72b27b0858 better message description in schema 2024-11-08 14:57:44 -05:00
Tony Fernandez 0fdee067c7 double space fix 2024-11-08 14:45:36 -05:00
Tony Fernandez 0dca5eeafc fix title wordwrap 2024-11-08 14:09:03 -05:00
Tony Fernandez 02ce3ba190 fix url_title word wrap 2024-11-08 14:07:49 -05:00
Tony Fernandez dc78bf4d6b fix TTL wordwrap 2024-11-08 14:06:39 -05:00
Tony Fernandez 4b7fbce291 fix sound word wrap 2024-11-08 14:05:27 -05:00
Tony Fernandez 1817b9a9ea fix wordwrap for html 2024-11-08 13:50:22 -05:00
Tony Fernandez 009055c61a device description rewrap 2024-11-08 13:48:21 -05:00
Tony Fernandez 54884da8fa priority word wrap 2024-11-08 13:46:19 -05:00
Tony Fernandez 1177385e08 fix expire description 2024-11-08 13:44:12 -05:00
Tony Fernandez a45ba8553c removed duplicate type:object 2024-11-08 13:42:19 -05:00
Tony Fernandez d7d6e30178 moved checks from hook to schema 2024-11-08 13:40:23 -05:00
Dan Helfman 56304fdcad Add NEWS entry for multiple system credentials fix (#930). 2024-11-07 20:20:41 -08:00
Dan Helfman 3f75e9931f Only support a single systemd credential by default (#930).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/930
Reviewed-by: Dan Helfman <witten@torsion.org>
2024-11-08 04:18:56 +00:00
Dan Helfman 227f475e17 Fix an error when implicitly upgrading the check state directory across filesystems (#931). 2024-11-07 19:19:56 -08:00
cvlc12 467ddd0e93 creds: Only support single credential by default 2024-11-08 00:36:24 +01:00
Dan Helfman be08e889f0 Fix the user runtime directory location on macOS (and possibly Cygwin) (#928). 2024-11-03 21:44:11 -08:00
Dan Helfman 94c8a56373 Reorder NEWS items. 2024-11-03 13:58:04 -08:00
Antonio Fernandez 94db527500 finalized support for Pushover 2024-10-30 15:43:06 -04:00
Antonio Fernandez 2849f54932 initial pushover commit 2024-10-30 11:25:26 -04:00
63 changed files with 2517 additions and 689 deletions
+23 -1
View File
@@ -1,3 +1,25 @@
1.9.2
* #441: Apply the "umask" option to all relevant actions, not just some of them.
* #722: Remove the restriction that the "extract" and "mount" actions must match a single
repository. Now they work more like other actions, where each repository is applied in turn.
* #932: Fix the missing build backend setting in pyproject.toml to allow Fedora builds.
* #934: Update the logic that probes for the borgmatic streaming database dump, bootstrap
metadata, and check state directories to support more platforms and use cases. See the
documentation for more information:
https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory
* #934: Add the "RuntimeDirectory" and "StateDirectory" options to the sample systemd service
file to support the new runtime and state directory logic.
* #939: Fix borgmatic ignoring the "BORG_RELOCATED_REPO_ACCESS_IS_OK" and
"BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK" environment variables.
* Add a Pushover monitoring hook. See the documentation for more information:
https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#pushover-hook
1.9.1
* #928: Fix the user runtime directory location on macOS (and possibly Cygwin).
* #930: Fix an error with the sample systemd service when no credentials are configured.
* #931: Fix an error when implicitly upgrading the check state directory from ~/.borgmatic to
~/.local/state/borgmatic across filesystems.
1.9.0
* #609: Fix the glob expansion of "source_directories" values to respect the "working_directory"
option.
@@ -20,6 +42,7 @@
cases.
* #902: Add loading of encrypted systemd credentials. See the documentation for more information:
https://torsion.org/borgmatic/docs/how-to/provide-your-passwords/#using-systemd-service-credentials
* #911: Add a "key change-passphrase" action to change the passphrase protecting a repository key.
* #914: Fix a confusing apparent hang when when the repository location changes, and instead
show a helpful error message.
* #915: BREAKING: Rename repository actions like "rcreate" to more explicit names like
@@ -31,7 +54,6 @@
* #919: Clarify the command-line help for the "--config" flag.
* #919: Document a policy for versioning and breaking changes:
https://torsion.org/borgmatic/docs/how-to/upgrade/#versioning-and-breaking-changes
* #911: Add a "key change-passphrase" action to change the passphrase protecting a repository key.
* #921: BREAKING: Change soft failure command hooks to skip only the current repository rather than
all repositories in the configuration file.
* #922: Replace setup.py (Python packaging metadata) with the more modern pyproject.toml.
+1
View File
@@ -66,6 +66,7 @@ borgmatic is powered by [Borg Backup](https://www.borgbackup.org/).
<a href="https://cronitor.io/"><img src="docs/static/cronitor.png" alt="Cronitor" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://cronhub.io/"><img src="docs/static/cronhub.png" alt="Cronhub" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://www.pagerduty.com/"><img src="docs/static/pagerduty.png" alt="PagerDuty" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://www.pushover.net/"><img src="docs/static/pushover.png" alt="Pushover" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://ntfy.sh/"><img src="docs/static/ntfy.png" alt="ntfy" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://grafana.com/oss/loki/"><img src="docs/static/loki.png" alt="Loki" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
<a href="https://github.com/caronc/apprise/wiki"><img src="docs/static/apprise.png" alt="Apprise" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
+47 -32
View File
@@ -6,6 +6,7 @@ import logging
import os
import pathlib
import random
import shutil
import borgmatic.borg.check
import borgmatic.borg.create
@@ -322,7 +323,7 @@ def upgrade_check_times(config, borg_repository_id):
f'Upgrading archives check times directory from {borgmatic_source_checks_path} to {borgmatic_state_checks_path}'
)
os.makedirs(borgmatic_state_path, mode=0o700, exist_ok=True)
os.rename(borgmatic_source_checks_path, borgmatic_state_checks_path)
shutil.move(borgmatic_source_checks_path, borgmatic_state_checks_path)
for check_type in ('archives', 'data'):
new_path = make_check_time_path(config, borg_repository_id, check_type, 'all')
@@ -335,12 +336,12 @@ def upgrade_check_times(config, borg_repository_id):
logger.debug(f'Upgrading archives check time file from {old_path} to {new_path}')
try:
os.rename(old_path, temporary_path)
shutil.move(old_path, temporary_path)
except FileNotFoundError:
pass
os.mkdir(old_path)
os.rename(temporary_path, new_path)
shutil.move(temporary_path, new_path)
def collect_spot_check_source_paths(
@@ -402,16 +403,22 @@ BORG_DIRECTORY_FILE_TYPE = 'd'
def collect_spot_check_archive_paths(
repository, archive, config, local_borg_version, global_arguments, local_path, remote_path
repository,
archive,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
borgmatic_runtime_directory,
):
'''
Given a repository configuration dict, the name of the latest archive, a configuration dict, the
local Borg version, global arguments as an argparse.Namespace instance, the local Borg path, and
the remote Borg path, collect the paths from the given archive (but only include files and
symlinks and exclude borgmatic runtime directories).
local Borg version, global arguments as an argparse.Namespace instance, the local Borg path, the
remote Borg path, and the borgmatic runtime directory, collect the paths from the given archive
(but only include files and symlinks and exclude borgmatic runtime directories).
'''
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
borgmatic_runtime_directory = borgmatic.config.paths.get_borgmatic_runtime_directory(config)
return tuple(
path
@@ -444,7 +451,7 @@ def compare_spot_check_hashes(
global_arguments,
local_path,
remote_path,
log_label,
log_prefix,
source_paths,
):
'''
@@ -468,7 +475,7 @@ def compare_spot_check_hashes(
if os.path.exists(os.path.join(working_directory or '', source_path))
}
logger.debug(
f'{log_label}: Sampling {sample_count} source paths (~{spot_check_config["data_sample_percentage"]}%) for spot check'
f'{log_prefix}: Sampling {sample_count} source paths (~{spot_check_config["data_sample_percentage"]}%) for spot check'
)
source_sample_paths_iterator = iter(source_sample_paths)
@@ -545,18 +552,19 @@ def spot_check(
global_arguments,
local_path,
remote_path,
borgmatic_runtime_directory,
):
'''
Given a repository dict, a loaded configuration dict, the local Borg version, global arguments
as an argparse.Namespace instance, the local Borg path, and the remote Borg path, perform a spot
check for the latest archive in the given repository.
as an argparse.Namespace instance, the local Borg path, the remote Borg path, and the borgmatic
runtime directory, perform a spot check for the latest archive in the given repository.
A spot check compares file counts and also the hashes for a random sampling of source files on
disk to those stored in the latest archive. If any differences are beyond configured tolerances,
then the check fails.
'''
log_label = f'{repository.get("label", repository["path"])}'
logger.debug(f'{log_label}: Running spot check')
log_prefix = f'{repository.get("label", repository["path"])}'
logger.debug(f'{log_prefix}: Running spot check')
try:
spot_check_config = next(
@@ -578,7 +586,7 @@ def spot_check(
local_path,
remote_path,
)
logger.debug(f'{log_label}: {len(source_paths)} total source paths for spot check')
logger.debug(f'{log_prefix}: {len(source_paths)} total source paths for spot check')
archive = borgmatic.borg.repo_list.resolve_archive_name(
repository['path'],
@@ -589,7 +597,7 @@ def spot_check(
local_path,
remote_path,
)
logger.debug(f'{log_label}: Using archive {archive} for spot check')
logger.debug(f'{log_prefix}: Using archive {archive} for spot check')
archive_paths = collect_spot_check_archive_paths(
repository,
@@ -599,8 +607,9 @@ def spot_check(
global_arguments,
local_path,
remote_path,
borgmatic_runtime_directory,
)
logger.debug(f'{log_label}: {len(archive_paths)} total archive paths for spot check')
logger.debug(f'{log_prefix}: {len(archive_paths)} total archive paths for spot check')
# Calculate the percentage delta between the source paths count and the archive paths count, and
# compare that delta to the configured count tolerance percentage.
@@ -608,10 +617,10 @@ def spot_check(
if count_delta_percentage > spot_check_config['count_tolerance_percentage']:
logger.debug(
f'{log_label}: Paths in source paths but not latest archive: {", ".join(set(source_paths) - set(archive_paths)) or "none"}'
f'{log_prefix}: Paths in source paths but not latest archive: {", ".join(set(source_paths) - set(archive_paths)) or "none"}'
)
logger.debug(
f'{log_label}: Paths in latest archive but not source paths: {", ".join(set(archive_paths) - set(source_paths)) or "none"}'
f'{log_prefix}: Paths in latest archive but not source paths: {", ".join(set(archive_paths) - set(source_paths)) or "none"}'
)
raise ValueError(
f'Spot check failed: {count_delta_percentage:.2f}% file count delta between source paths and latest archive (tolerance is {spot_check_config["count_tolerance_percentage"]}%)'
@@ -625,25 +634,25 @@ def spot_check(
global_arguments,
local_path,
remote_path,
log_label,
log_prefix,
source_paths,
)
# Error if the percentage of failing hashes exceeds the configured tolerance percentage.
logger.debug(f'{log_label}: {len(failing_paths)} non-matching spot check hashes')
logger.debug(f'{log_prefix}: {len(failing_paths)} non-matching spot check hashes')
data_tolerance_percentage = spot_check_config['data_tolerance_percentage']
failing_percentage = (len(failing_paths) / len(source_paths)) * 100
if failing_percentage > data_tolerance_percentage:
logger.debug(
f'{log_label}: Source paths with data not matching the latest archive: {", ".join(failing_paths)}'
f'{log_prefix}: Source paths with data not matching the latest archive: {", ".join(failing_paths)}'
)
raise ValueError(
f'Spot check failed: {failing_percentage:.2f}% of source paths with data not matching the latest archive (tolerance is {data_tolerance_percentage}%)'
)
logger.info(
f'{log_label}: Spot check passed with a {count_delta_percentage:.2f}% file count delta and a {failing_percentage:.2f}% file data delta'
f'{log_prefix}: Spot check passed with a {count_delta_percentage:.2f}% file count delta and a {failing_percentage:.2f}% file data delta'
)
@@ -677,7 +686,9 @@ def run_check(
**hook_context,
)
logger.info(f'{repository.get("label", repository["path"])}: Running consistency checks')
log_prefix = repository.get('label', repository['path'])
logger.info(f'{log_prefix}: Running consistency checks')
repository_id = borgmatic.borg.check.get_repository_id(
repository['path'],
config,
@@ -729,14 +740,18 @@ def run_check(
write_check_time(make_check_time_path(config, repository_id, 'extract'))
if 'spot' in checks:
spot_check(
repository,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
)
with borgmatic.config.paths.Runtime_directory(
config, log_prefix
) as borgmatic_runtime_directory:
spot_check(
repository,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
borgmatic_runtime_directory,
)
write_check_time(make_check_time_path(config, repository_id, 'spot'))
borgmatic.hooks.command.execute_hook(
+31 -24
View File
@@ -38,37 +38,44 @@ def get_config_paths(archive_name, bootstrap_arguments, global_arguments, local_
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(
{'borgmatic_source_directory': bootstrap_arguments.borgmatic_source_directory}
)
borgmatic_runtime_directory = borgmatic.config.paths.get_borgmatic_runtime_directory(
{'user_runtime_directory': bootstrap_arguments.user_runtime_directory}
)
config = make_bootstrap_config(bootstrap_arguments)
# Probe for the manifest file in multiple locations, as the default location has moved to the
# borgmatic runtime directory (which get stored as just "/borgmatic" with Borg 1.4+). But we
# still want to support reading the manifest from previously created archives as well.
for base_directory in ('borgmatic', borgmatic_runtime_directory, borgmatic_source_directory):
borgmatic_manifest_path = os.path.join(base_directory, 'bootstrap', 'manifest.json')
with borgmatic.config.paths.Runtime_directory(
{'user_runtime_directory': bootstrap_arguments.user_runtime_directory},
bootstrap_arguments.repository,
) as borgmatic_runtime_directory:
for base_directory in (
'borgmatic',
borgmatic.config.paths.make_runtime_directory_glob(borgmatic_runtime_directory),
borgmatic_source_directory,
):
borgmatic_manifest_path = 'sh:' + os.path.join(
base_directory, 'bootstrap', 'manifest.json'
)
extract_process = borgmatic.borg.extract.extract_archive(
global_arguments.dry_run,
bootstrap_arguments.repository,
archive_name,
[borgmatic_manifest_path],
config,
local_borg_version,
global_arguments,
local_path=bootstrap_arguments.local_path,
remote_path=bootstrap_arguments.remote_path,
extract_to_stdout=True,
)
manifest_json = extract_process.stdout.read()
extract_process = borgmatic.borg.extract.extract_archive(
global_arguments.dry_run,
bootstrap_arguments.repository,
archive_name,
[borgmatic_manifest_path],
config,
local_borg_version,
global_arguments,
local_path=bootstrap_arguments.local_path,
remote_path=bootstrap_arguments.remote_path,
extract_to_stdout=True,
)
manifest_json = extract_process.stdout.read()
if manifest_json:
break
else:
raise ValueError(
'Cannot read configuration paths from archive due to missing bootstrap manifest'
)
if manifest_json:
break
else:
raise ValueError(
'Cannot read configuration paths from archive due to missing bootstrap manifest'
)
try:
manifest_data = json.loads(manifest_json)
+61 -47
View File
@@ -14,15 +14,15 @@ import borgmatic.hooks.dump
logger = logging.getLogger(__name__)
def create_borgmatic_manifest(config, config_paths, dry_run):
def create_borgmatic_manifest(config, config_paths, borgmatic_runtime_directory, dry_run):
'''
Create a borgmatic manifest file to store the paths to the configuration files used to create
the archive.
Given a configuration dict, a sequence of config file paths, the borgmatic runtime directory,
and whether this is a dry run, create a borgmatic manifest file to store the paths to the
configuration files used to create the archive.
'''
if dry_run:
return
borgmatic_runtime_directory = borgmatic.config.paths.get_borgmatic_runtime_directory(config)
borgmatic_manifest_path = os.path.join(
borgmatic_runtime_directory, 'bootstrap', 'manifest.json'
)
@@ -71,54 +71,68 @@ def run_create(
global_arguments.dry_run,
**hook_context,
)
logger.info(f'{repository.get("label", repository["path"])}: Creating archive{dry_run_label}')
borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
'remove_data_source_dumps',
config,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
global_arguments.dry_run,
)
active_dumps = borgmatic.hooks.dispatch.call_hooks(
'dump_data_sources',
config,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
global_arguments.dry_run,
)
if config.get('store_config_files', True):
create_borgmatic_manifest(
log_prefix = repository.get('label', repository['path'])
logger.info(f'{log_prefix}: Creating archive{dry_run_label}')
with borgmatic.config.paths.Runtime_directory(
config, log_prefix
) as borgmatic_runtime_directory:
borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
'remove_data_source_dumps',
config,
config_paths,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
borgmatic_runtime_directory,
global_arguments.dry_run,
)
stream_processes = [process for processes in active_dumps.values() for process in processes]
active_dumps = borgmatic.hooks.dispatch.call_hooks(
'dump_data_sources',
config,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
borgmatic_runtime_directory,
global_arguments.dry_run,
)
stream_processes = [process for processes in active_dumps.values() for process in processes]
json_output = borgmatic.borg.create.create_archive(
global_arguments.dry_run,
repository['path'],
config,
config_paths,
local_borg_version,
global_arguments,
local_path=local_path,
remote_path=remote_path,
progress=create_arguments.progress,
stats=create_arguments.stats,
json=create_arguments.json,
list_files=create_arguments.list_files,
stream_processes=stream_processes,
)
if json_output:
yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))
if config.get('store_config_files', True):
create_borgmatic_manifest(
config,
config_paths,
borgmatic_runtime_directory,
global_arguments.dry_run,
)
json_output = borgmatic.borg.create.create_archive(
global_arguments.dry_run,
repository['path'],
config,
config_paths,
local_borg_version,
global_arguments,
borgmatic_runtime_directory,
local_path=local_path,
remote_path=remote_path,
progress=create_arguments.progress,
stats=create_arguments.stats,
json=create_arguments.json,
list_files=create_arguments.list_files,
stream_processes=stream_processes,
)
if json_output:
yield borgmatic.actions.json.parse_json(json_output, repository.get('label'))
borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
'remove_data_source_dumps',
config,
config_filename,
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
borgmatic_runtime_directory,
global_arguments.dry_run,
)
borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
'remove_data_source_dumps',
config,
config_filename,
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
global_arguments.dry_run,
)
borgmatic.hooks.command.execute_hook(
config.get('after_backup'),
config.get('umask'),
+113 -102
View File
@@ -91,7 +91,9 @@ def strip_path_prefix_from_extracted_dump_destination(
if not databases_directory.endswith('_databases'):
continue
os.rename(subdirectory_path, os.path.join(borgmatic_runtime_directory, databases_directory))
shutil.move(
subdirectory_path, os.path.join(borgmatic_runtime_directory, databases_directory)
)
break
@@ -106,6 +108,7 @@ def restore_single_data_source(
hook_name,
data_source,
connection_params,
borgmatic_runtime_directory,
):
'''
Given (among other things) an archive name, a data source hook name, the hostname, port,
@@ -121,9 +124,9 @@ def restore_single_data_source(
config,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
borgmatic_runtime_directory,
data_source['name'],
)[hook_name]
borgmatic_runtime_directory = borgmatic.config.paths.get_borgmatic_runtime_directory(config)
destination_path = (
tempfile.mkdtemp(dir=borgmatic_runtime_directory)
@@ -133,7 +136,7 @@ def restore_single_data_source(
try:
# Kick off a single data source extract. If using a directory format, extract to a temporary
# directory. Otheriwes extract the single dump file to stdout.
# directory. Otherwise extract the single dump file to stdout.
extract_process = borgmatic.borg.extract.extract_archive(
dry_run=global_arguments.dry_run,
repository=repository['path'],
@@ -168,6 +171,7 @@ def restore_single_data_source(
dry_run=global_arguments.dry_run,
extract_process=extract_process,
connection_params=connection_params,
borgmatic_runtime_directory=borgmatic_runtime_directory,
)
@@ -179,17 +183,17 @@ def collect_archive_data_source_names(
global_arguments,
local_path,
remote_path,
borgmatic_runtime_directory,
):
'''
Given a local or remote repository path, a resolved archive name, a configuration dict, the
local Borg version, global_arguments an argparse.Namespace, and local and remote Borg paths,
query the archive for the names of data sources it contains as dumps and return them as a dict
from hook name to a sequence of data source names.
local Borg version, global_arguments an argparse.Namespace, local and remote Borg paths, and the
borgmatic runtime directory, query the archive for the names of data sources it contains as
dumps and return them as a dict from hook name to a sequence of data source names.
'''
borgmatic_source_directory = str(
pathlib.Path(borgmatic.config.paths.get_borgmatic_source_directory(config))
)
borgmatic_runtime_directory = borgmatic.config.paths.get_borgmatic_runtime_directory(config)
# Probe for the data source dumps in multiple locations, as the default location has moved to
# the borgmatic runtime directory (which get stored as just "/borgmatic" with Borg 1.4+). But we
@@ -205,7 +209,7 @@ def collect_archive_data_source_names(
+ borgmatic.hooks.dump.make_data_source_dump_path(base_directory, '*_databases/*/*')
for base_directory in (
'borgmatic',
borgmatic_runtime_directory.lstrip('/'),
borgmatic.config.paths.make_runtime_directory_glob(borgmatic_runtime_directory),
borgmatic_source_directory.lstrip('/'),
)
],
@@ -340,109 +344,116 @@ def run_restore(
):
return
logger.info(
f'{repository.get("label", repository["path"])}: Restoring data sources from archive {restore_arguments.archive}'
)
log_prefix = repository.get('label', repository['path'])
logger.info(f'{log_prefix}: Restoring data sources from archive {restore_arguments.archive}')
borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
'remove_data_source_dumps',
config,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
global_arguments.dry_run,
)
with borgmatic.config.paths.Runtime_directory(
config, log_prefix
) as borgmatic_runtime_directory:
borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
'remove_data_source_dumps',
config,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
borgmatic_runtime_directory,
global_arguments.dry_run,
)
archive_name = borgmatic.borg.repo_list.resolve_archive_name(
repository['path'],
restore_arguments.archive,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
)
archive_data_source_names = collect_archive_data_source_names(
repository['path'],
archive_name,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
)
restore_names = find_data_sources_to_restore(
restore_arguments.data_sources, archive_data_source_names
)
found_names = set()
remaining_restore_names = {}
connection_params = {
'hostname': restore_arguments.hostname,
'port': restore_arguments.port,
'username': restore_arguments.username,
'password': restore_arguments.password,
'restore_path': restore_arguments.restore_path,
}
archive_name = borgmatic.borg.repo_list.resolve_archive_name(
repository['path'],
restore_arguments.archive,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
)
archive_data_source_names = collect_archive_data_source_names(
repository['path'],
archive_name,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
borgmatic_runtime_directory,
)
restore_names = find_data_sources_to_restore(
restore_arguments.data_sources, archive_data_source_names
)
found_names = set()
remaining_restore_names = {}
connection_params = {
'hostname': restore_arguments.hostname,
'port': restore_arguments.port,
'username': restore_arguments.username,
'password': restore_arguments.password,
'restore_path': restore_arguments.restore_path,
}
for hook_name, data_source_names in restore_names.items():
for data_source_name in data_source_names:
found_hook_name, found_data_source = get_configured_data_source(
config, archive_data_source_names, hook_name, data_source_name
)
if not found_data_source:
remaining_restore_names.setdefault(found_hook_name or hook_name, []).append(
data_source_name
for hook_name, data_source_names in restore_names.items():
for data_source_name in data_source_names:
found_hook_name, found_data_source = get_configured_data_source(
config, archive_data_source_names, hook_name, data_source_name
)
continue
found_names.add(data_source_name)
restore_single_data_source(
repository,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
archive_name,
found_hook_name or hook_name,
dict(found_data_source, **{'schemas': restore_arguments.schemas}),
connection_params,
)
if not found_data_source:
remaining_restore_names.setdefault(found_hook_name or hook_name, []).append(
data_source_name
)
continue
# For any data sources that weren't found via exact matches in the configuration, try to
# fallback to "all" entries.
for hook_name, data_source_names in remaining_restore_names.items():
for data_source_name in data_source_names:
found_hook_name, found_data_source = get_configured_data_source(
config, archive_data_source_names, hook_name, data_source_name, 'all'
)
found_names.add(data_source_name)
restore_single_data_source(
repository,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
archive_name,
found_hook_name or hook_name,
dict(found_data_source, **{'schemas': restore_arguments.schemas}),
connection_params,
borgmatic_runtime_directory,
)
if not found_data_source:
continue
# For any data sources that weren't found via exact matches in the configuration, try to
# fallback to "all" entries.
for hook_name, data_source_names in remaining_restore_names.items():
for data_source_name in data_source_names:
found_hook_name, found_data_source = get_configured_data_source(
config, archive_data_source_names, hook_name, data_source_name, 'all'
)
found_names.add(data_source_name)
data_source = copy.copy(found_data_source)
data_source['name'] = data_source_name
if not found_data_source:
continue
restore_single_data_source(
repository,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
archive_name,
found_hook_name or hook_name,
dict(data_source, **{'schemas': restore_arguments.schemas}),
connection_params,
)
found_names.add(data_source_name)
data_source = copy.copy(found_data_source)
data_source['name'] = data_source_name
borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
'remove_data_source_dumps',
config,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
global_arguments.dry_run,
)
restore_single_data_source(
repository,
config,
local_borg_version,
global_arguments,
local_path,
remote_path,
archive_name,
found_hook_name or hook_name,
dict(data_source, **{'schemas': restore_arguments.schemas}),
connection_params,
borgmatic_runtime_directory,
)
borgmatic.hooks.dispatch.call_hooks_even_if_unconfigured(
'remove_data_source_dumps',
config,
repository['path'],
borgmatic.hooks.dump.DATA_SOURCE_HOOK_NAMES,
borgmatic_runtime_directory,
global_arguments.dry_run,
)
ensure_data_sources_found(restore_names, remaining_restore_names, found_names)
+2
View File
@@ -150,6 +150,7 @@ def check_archives(
)
max_duration = check_arguments.max_duration or repository_check_config.get('max_duration')
umask = config.get('umask')
borg_environment = environment.make_environment(config)
borg_exit_codes = config.get('borg_exit_codes')
@@ -160,6 +161,7 @@ def check_archives(
+ (('--max-duration', str(max_duration)) if max_duration else ())
+ make_check_name_flags(checks, archive_filter_flags)
+ (('--remote-path', remote_path) if remote_path else ())
+ (('--umask', str(umask)) if umask else ())
+ (('--log-json',) if global_arguments.log_json else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ verbosity_flags
+2 -3
View File
@@ -504,6 +504,7 @@ def create_archive(
config_paths,
local_borg_version,
global_arguments,
borgmatic_runtime_directory,
local_path='borg',
remote_path=None,
progress=False,
@@ -524,9 +525,7 @@ def create_archive(
working_directory = borgmatic.config.paths.get_working_directory(config)
borgmatic_runtime_directories = expand_directories(
collect_borgmatic_runtime_directories(
borgmatic.config.paths.get_borgmatic_runtime_directory(config)
),
collect_borgmatic_runtime_directories(borgmatic_runtime_directory),
working_directory=working_directory,
)
+1
View File
@@ -31,6 +31,7 @@ def make_delete_command(
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ borgmatic.borg.flags.make_flags('dry-run', global_arguments.dry_run)
+ borgmatic.borg.flags.make_flags('remote-path', remote_path)
+ borgmatic.borg.flags.make_flags('umask', config.get('umask'))
+ borgmatic.borg.flags.make_flags('log-json', global_arguments.log_json)
+ borgmatic.borg.flags.make_flags('lock-wait', config.get('lock_wait'))
+ borgmatic.borg.flags.make_flags('list', delete_arguments.list_archives)
+5 -2
View File
@@ -1,3 +1,5 @@
import os
OPTION_TO_ENVIRONMENT_VARIABLE = {
'borg_base_directory': 'BORG_BASE_DIR',
'borg_config_directory': 'BORG_CONFIG_DIR',
@@ -38,8 +40,9 @@ def make_environment(config):
option_name,
environment_variable_name,
) in DEFAULT_BOOL_OPTION_TO_DOWNCASE_ENVIRONMENT_VARIABLE.items():
value = config.get(option_name)
environment[environment_variable_name] = 'yes' if value else 'no'
if os.environ.get(environment_variable_name) is None:
value = config.get(option_name)
environment[environment_variable_name] = 'yes' if value else 'no'
for (
option_name,
+1
View File
@@ -36,6 +36,7 @@ def make_info_command(
else ()
)
+ flags.make_flags('remote-path', remote_path)
+ flags.make_flags('umask', config.get('umask'))
+ flags.make_flags('log-json', global_arguments.log_json)
+ flags.make_flags('lock-wait', config.get('lock_wait'))
+ (
+2 -3
View File
@@ -34,8 +34,6 @@ def make_list_command(
and local and remote Borg paths, return a command as a tuple to list archives or paths within an
archive.
'''
lock_wait = config.get('lock_wait', None)
return (
(local_path, 'list')
+ (
@@ -49,8 +47,9 @@ def make_list_command(
else ()
)
+ flags.make_flags('remote-path', remote_path)
+ flags.make_flags('umask', config.get('umask'))
+ flags.make_flags('log-json', global_arguments.log_json)
+ flags.make_flags('lock-wait', lock_wait)
+ flags.make_flags('lock-wait', config.get('lock_wait'))
+ flags.make_flags_from_arguments(list_arguments, excludes=MAKE_FLAGS_EXCLUDES)
+ (
flags.make_repository_archive_flags(
+2
View File
@@ -64,6 +64,7 @@ def create_repository(
raise
lock_wait = config.get('lock_wait')
umask = config.get('umask')
extra_borg_options = config.get('extra_borg_options', {}).get('repo-create', '')
repo_create_command = (
@@ -84,6 +85,7 @@ def create_repository(
+ (('--log-json',) if global_arguments.log_json else ())
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--remote-path', remote_path) if remote_path else ())
+ (('--umask', str(umask)) if umask else ())
+ (tuple(extra_borg_options.split(' ')) if extra_borg_options else ())
+ flags.make_repository_flags(repository_path, local_borg_version)
)
+1
View File
@@ -36,6 +36,7 @@ def make_repo_delete_command(
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ borgmatic.borg.flags.make_flags('dry-run', global_arguments.dry_run)
+ borgmatic.borg.flags.make_flags('remote-path', remote_path)
+ borgmatic.borg.flags.make_flags('umask', config.get('umask'))
+ borgmatic.borg.flags.make_flags('log-json', global_arguments.log_json)
+ borgmatic.borg.flags.make_flags('lock-wait', config.get('lock_wait'))
+ borgmatic.borg.flags.make_flags('list', repo_delete_arguments.list_archives)
+1
View File
@@ -43,6 +43,7 @@ def display_repository_info(
else ()
)
+ flags.make_flags('remote-path', remote_path)
+ flags.make_flags('umask', config.get('umask'))
+ flags.make_flags('log-json', global_arguments.log_json)
+ flags.make_flags('lock-wait', lock_wait)
+ (('--json',) if repo_info_arguments.json else ())
+2
View File
@@ -39,6 +39,7 @@ def resolve_archive_name(
),
)
+ flags.make_flags('remote-path', remote_path)
+ flags.make_flags('umask', config.get('umask'))
+ flags.make_flags('log-json', global_arguments.log_json)
+ flags.make_flags('lock-wait', config.get('lock_wait'))
+ flags.make_flags('last', 1)
@@ -100,6 +101,7 @@ def make_repo_list_command(
else ()
)
+ flags.make_flags('remote-path', remote_path)
+ flags.make_flags('umask', config.get('umask'))
+ flags.make_flags('log-json', global_arguments.log_json)
+ flags.make_flags('lock-wait', config.get('lock_wait'))
+ (
+1
View File
@@ -30,6 +30,7 @@ def transfer_archives(
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ flags.make_flags('remote-path', remote_path)
+ flags.make_flags('umask', config.get('umask'))
+ flags.make_flags('log-json', global_arguments.log_json)
+ flags.make_flags('lock-wait', config.get('lock_wait', None))
+ (
+1 -1
View File
@@ -876,7 +876,7 @@ def make_parsers():
)
config_bootstrap_group.add_argument(
'--user-runtime-directory',
help='Path used for temporary runtime data like bootstrap metadata. Defaults to $XDG_RUNTIME_DIR or /var/run/$UID',
help='Path used for temporary runtime data like bootstrap metadata. Defaults to $XDG_RUNTIME_DIR or $TMPDIR or $TEMP or /var/run/$UID',
)
config_bootstrap_group.add_argument(
'--borgmatic-source-directory',
-3
View File
@@ -793,9 +793,6 @@ def collect_configuration_run_summary_logs(configs, config_paths, arguments):
break
try:
if 'extract' in arguments or 'mount' in arguments:
validate.guard_single_repository_selected(repository, configs)
validate.guard_configuration_contains_repository(repository, configs)
except ValueError as error:
yield from log_error_records(str(error))
+89 -20
View File
@@ -1,4 +1,8 @@
import logging
import os
import tempfile
logger = logging.getLogger(__name__)
def expand_user_in_path(path):
@@ -26,25 +30,91 @@ def get_borgmatic_source_directory(config):
return expand_user_in_path(config.get('borgmatic_source_directory') or '~/.borgmatic')
def get_borgmatic_runtime_directory(config):
'''
Given a configuration dict, get the borgmatic runtime directory used for storing temporary
runtime data like streaming database dumps and bootstrap metadata. Defaults to
$XDG_RUNTIME_DIR/./borgmatic or /run/user/$UID/./borgmatic.
TEMPORARY_DIRECTORY_PREFIX = 'borgmatic-'
The "/./" is taking advantage of a Borg feature such that the part of the path before the "/./"
does not get stored in the file path within an archive. That way, the path of the runtime
directory can change without leaving database dumps within an archive inaccessible.
class Runtime_directory:
'''
return expand_user_in_path(
os.path.join(
A Python context manager for creating and cleaning up the borgmatic runtime directory used for
storing temporary runtime data like streaming database dumps and bootstrap metadata.
Example use as a context manager:
with borgmatic.config.paths.Runtime_directory(config) as borgmatic_runtime_directory:
do_something_with(borgmatic_runtime_directory)
For the scope of that "with" statement, the runtime directory is available. Afterwards, it
automatically gets cleaned up as necessary.
'''
def __init__(self, config, log_prefix):
'''
Given a configuration dict and a log prefix, determine the borgmatic runtime directory,
creating a secure, temporary directory within it if necessary. Defaults to
$XDG_RUNTIME_DIR/./borgmatic or $RUNTIME_DIRECTORY/./borgmatic or
$TMPDIR/borgmatic-[random]/./borgmatic or $TEMP/borgmatic-[random]/./borgmatic or
/tmp/borgmatic-[random]/./borgmatic where "[random]" is a randomly generated string intended
to avoid path collisions.
If XDG_RUNTIME_DIR or RUNTIME_DIRECTORY is set and already ends in "/borgmatic", then don't
tack on a second "/borgmatic" path component.
The "/./" is taking advantage of a Borg feature such that the part of the path before the "/./"
does not get stored in the file path within an archive. That way, the path of the runtime
directory can change without leaving database dumps within an archive inaccessible.
'''
runtime_directory = (
config.get('user_runtime_directory')
or os.environ.get(
'XDG_RUNTIME_DIR',
f'/run/user/{os.getuid()}',
),
'.',
'borgmatic',
or os.environ.get('XDG_RUNTIME_DIR') # Set by PAM on Linux.
or os.environ.get('RUNTIME_DIRECTORY') # Set by systemd if configured.
)
if runtime_directory:
self.temporary_directory = None
else:
base_directory = os.environ.get('TMPDIR') or os.environ.get('TEMP') or '/tmp'
os.makedirs(base_directory, mode=0o700, exist_ok=True)
self.temporary_directory = tempfile.TemporaryDirectory(
prefix=TEMPORARY_DIRECTORY_PREFIX,
dir=base_directory,
)
runtime_directory = self.temporary_directory.name
(base_path, final_directory) = os.path.split(runtime_directory.rstrip(os.path.sep))
self.runtime_path = expand_user_in_path(
os.path.join(
base_path if final_directory == 'borgmatic' else runtime_directory, '.', 'borgmatic'
)
)
os.makedirs(self.runtime_path, mode=0o700, exist_ok=True)
logger.debug(f'{log_prefix}: Using runtime directory {os.path.normpath(self.runtime_path)}')
def __enter__(self):
'''
Return the borgmatic runtime path as a string.
'''
return self.runtime_path
def __exit__(self, exception, value, traceback):
'''
Delete any temporary directory that was created as part of initialization.
'''
if self.temporary_directory:
self.temporary_directory.cleanup()
def make_runtime_directory_glob(borgmatic_runtime_directory):
'''
Given a borgmatic runtime directory path, make a glob that would match that path, specifically
replacing any randomly generated temporary subdirectory with "*" since such a directory's name
changes on every borgmatic run.
'''
return os.path.join(
*(
'*' if subdirectory.startswith(TEMPORARY_DIRECTORY_PREFIX) else subdirectory
for subdirectory in os.path.normpath(borgmatic_runtime_directory).split(os.path.sep)
)
)
@@ -58,10 +128,9 @@ def get_borgmatic_state_directory(config):
return expand_user_in_path(
os.path.join(
config.get('user_state_directory')
or os.environ.get(
'XDG_STATE_HOME',
'~/.local/state',
),
or os.environ.get('XDG_STATE_HOME')
or os.environ.get('STATE_DIRECTORY') # Set by systemd if configured.
or '~/.local/state',
'borgmatic',
)
)
+260 -2
View File
@@ -218,8 +218,8 @@ properties:
Path for storing temporary runtime data like streaming database
dumps and bootstrap metadata. borgmatic automatically creates and
uses a "borgmatic" subdirectory here. Defaults to $XDG_RUNTIME_DIR
or /run/user/$UID.
example: /run/user/1001/borgmatic
or or $TMPDIR or $TEMP or /run/user/$UID.
example: /run/user/1001
user_state_directory:
type: string
description: |
@@ -1626,6 +1626,264 @@ properties:
example:
- start
- finish
pushover:
type: object
required: ['token', 'user']
additionalProperties: false
properties:
token:
type: string
description: |
Your application's API token.
example: 7ms6TXHpTokTou2P6x4SodDeentHRa
user:
type: string
description: |
Your user/group key (or that of your target user), viewable
when logged into your dashboard: often referred to as
USER_KEY in Pushover documentation and code examples.
example: hwRwoWsXMBWwgrSecfa9EfPey55WSN
start:
type: object
properties:
message:
type: string
description: |
Message to be sent to the user or group. If omitted
the default is the name of the state.
example: A backup job has started.
priority:
type: integer
description: |
A value of -2, -1, 0 (default), 1 or 2 that
indicates the message priority.
example: 0
expire:
type: integer
description: |
How many seconds your notification will continue
to be retried (every retry seconds). Defaults to
600. This settings only applies to priority 2
notifications.
example: 600
retry:
type: integer
description: |
The retry parameter specifies how often
(in seconds) the Pushover servers will send the
same notification to the user. Defaults to 30. This
settings only applies to priority 2 notifications.
example: 30
device:
type: string
description: |
The name of one of your devices to send just to
that device instead of all devices.
example: pixel8
html:
type: boolean
description: |
Set to True to enable HTML parsing of the message.
Set to False for plain text.
example: True
sound:
type: string
description: |
The name of a supported sound to override your
default sound choice. All options can be found
here: https://pushover.net/api#sounds
example: bike
title:
type: string
description: |
Your message's title, otherwise your app's name is
used.
example: A backup job has started.
ttl:
type: integer
description: |
The number of seconds that the message will live,
before being deleted automatically. The ttl
parameter is ignored for messages with a priority.
value of 2.
example: 3600
url:
type: string
description: |
A supplementary URL to show with your message.
example: https://pushover.net/apps/xxxxx-borgbackup
url_title:
type: string
description: |
A title for the URL specified as the url parameter,
otherwise just the URL is shown.
example: Pushover Link
finish:
type: object
properties:
message:
type: string
description: |
Message to be sent to the user or group. If omitted
the default is the name of the state.
example: A backup job has finished.
priority:
type: integer
description: |
A value of -2, -1, 0 (default), 1 or 2 that
indicates the message priority.
example: 0
expire:
type: integer
description: |
How many seconds your notification will continue
to be retried (every retry seconds). Defaults to
600. This settings only applies to priority 2
notifications.
example: 600
retry:
type: integer
description: |
The retry parameter specifies how often
(in seconds) the Pushover servers will send the
same notification to the user. Defaults to 30. This
settings only applies to priority 2 notifications.
example: 30
device:
type: string
description: |
The name of one of your devices to send just to
that device instead of all devices.
example: pixel8
html:
type: boolean
description: |
Set to True to enable HTML parsing of the message.
Set to False for plain text.
example: True
sound:
type: string
description: |
The name of a supported sound to override your
default sound choice. All options can be found
here: https://pushover.net/api#sounds
example: bike
title:
type: string
description: |
Your message's title, otherwise your app's name is
used.
example: A backup job has started.
ttl:
type: integer
description: |
The number of seconds that the message will live,
before being deleted automatically. The ttl
parameter is ignored for messages with a priority.
value of 2.
example: 3600
url:
type: string
description: |
A supplementary URL to show with your message.
example: https://pushover.net/apps/xxxxx-borgbackup
url_title:
type: string
description: |
A title for the URL specified as the url parameter,
otherwise just the URL is shown.
example: Pushover Link
fail:
type: object
properties:
message:
type: string
description: |
Message to be sent to the user or group. If omitted
the default is the name of the state.
example: A backup job has failed.
priority:
type: integer
description: |
A value of -2, -1, 0 (default), 1 or 2 that
indicates the message priority.
example: 0
expire:
type: integer
description: |
How many seconds your notification will continue
to be retried (every retry seconds). Defaults to
600. This settings only applies to priority 2
notifications.
example: 600
retry:
type: integer
description: |
The retry parameter specifies how often
(in seconds) the Pushover servers will send the
same notification to the user. Defaults to 30. This
settings only applies to priority 2 notifications.
example: 30
device:
type: string
description: |
The name of one of your devices to send just to
that device instead of all devices.
example: pixel8
html:
type: boolean
description: |
Set to True to enable HTML parsing of the message.
Set to False for plain text.
example: True
sound:
type: string
description: |
The name of a supported sound to override your
default sound choice. All options can be found
here: https://pushover.net/api#sounds
example: bike
title:
type: string
description: |
Your message's title, otherwise your app's name is
used.
example: A backup job has started.
ttl:
type: integer
description: |
The number of seconds that the message will live,
before being deleted automatically. The ttl
parameter is ignored for messages with a priority.
value of 2.
example: 3600
url:
type: string
description: |
A supplementary URL to show with your message.
example: https://pushover.net/apps/xxxxx-borgbackup
url_title:
type: string
description: |
A title for the URL specified as the url parameter,
otherwise just the URL is shown.
example: Pushover Link
states:
type: array
items:
type: string
enum:
- start
- finish
- fail
uniqueItems: true
description: |
List of one or more monitoring states to ping for: "start",
"finish", and/or "fail". Defaults to pinging for failure
only.
example:
- start
- finish
zabbix:
type: object
additionalProperties: false
-23
View File
@@ -199,26 +199,3 @@ def guard_configuration_contains_repository(repository, configurations):
if count == 0:
raise ValueError(f'Repository "{repository}" not found in configuration files')
def guard_single_repository_selected(repository, configurations):
'''
Given a repository path and a dict mapping from config filename to corresponding parsed config
dict, ensure either a single repository exists across all configuration files or a repository
path was given.
'''
if repository:
return
count = len(
tuple(
config_repository
for config in configurations.values()
for config_repository in config['repositories']
)
)
if count != 1:
raise ValueError(
"Can't determine which repository to use. Use --repository to disambiguate"
)
+2
View File
@@ -12,6 +12,7 @@ from borgmatic.hooks import (
ntfy,
pagerduty,
postgresql,
pushover,
sqlite,
uptimekuma,
zabbix,
@@ -31,6 +32,7 @@ HOOK_NAME_TO_MODULE = {
'ntfy': ntfy,
'pagerduty': pagerduty,
'postgresql_databases': postgresql,
'pushover': pushover,
'sqlite_databases': sqlite,
'uptime_kuma': uptimekuma,
'zabbix': zabbix,
+36 -28
View File
@@ -14,15 +14,11 @@ from borgmatic.hooks import dump
logger = logging.getLogger(__name__)
def make_dump_path(config, base_directory=None): # pragma: no cover
def make_dump_path(base_directory): # pragma: no cover
'''
Given a configuration dict and an optional base directory, make the corresponding dump path. If
a base directory isn't provided, use the borgmatic runtime directory.
Given a base directory, make the corresponding dump path.
'''
return dump.make_data_source_dump_path(
base_directory or borgmatic.config.paths.get_borgmatic_runtime_directory(config),
'mariadb_databases',
)
return dump.make_data_source_dump_path(base_directory, 'mariadb_databases')
SYSTEM_DATABASE_NAMES = ('information_schema', 'mysql', 'performance_schema', 'sys')
@@ -126,12 +122,12 @@ def use_streaming(databases, config, log_prefix):
return any(databases)
def dump_data_sources(databases, config, log_prefix, dry_run):
def dump_data_sources(databases, config, log_prefix, borgmatic_runtime_directory, dry_run):
'''
Dump the given MariaDB databases to a named pipe. The databases are supplied as a sequence of
dicts, one dict describing each database as per the configuration schema. Use the given
configuration dict to construct the destination path and the given log prefix in any log
entries.
borgmatic runtime directory to construct the destination path and the given log prefix in any
log entries.
Return a sequence of subprocess.Popen instances for the dump processes ready to spew to a named
pipe. But if this is a dry run, then don't actually dump anything and return an empty sequence.
@@ -142,7 +138,7 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
logger.info(f'{log_prefix}: Dumping MariaDB databases{dry_run_label}')
for database in databases:
dump_path = make_dump_path(config)
dump_path = make_dump_path(borgmatic_runtime_directory)
extra_environment = {'MYSQL_PWD': database['password']} if 'password' in database else None
dump_database_names = database_names_to_dump(
database, extra_environment, log_prefix, dry_run
@@ -185,43 +181,55 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
return [process for process in processes if process]
def remove_data_source_dumps(databases, config, log_prefix, dry_run): # pragma: no cover
def remove_data_source_dumps(
databases, config, log_prefix, borgmatic_runtime_directory, dry_run
): # pragma: no cover
'''
Remove all database dump files for this hook regardless of the given databases. Use the given
configuration dict to construct the destination path and the log prefix in any log entries. If
this is a dry run, then don't actually remove anything.
Remove all database dump files for this hook regardless of the given databases. Use the
borgmatic_runtime_directory to construct the destination path and the log prefix in any log
entries. If this is a dry run, then don't actually remove anything.
'''
dump.remove_data_source_dumps(make_dump_path(config), 'MariaDB', log_prefix, dry_run)
dump.remove_data_source_dumps(
make_dump_path(borgmatic_runtime_directory), 'MariaDB', log_prefix, dry_run
)
def make_data_source_dump_patterns(databases, config, log_prefix, name=None): # pragma: no cover
def make_data_source_dump_patterns(
databases, config, log_prefix, borgmatic_runtime_directory, name=None
): # pragma: no cover
'''
Given a sequence of configurations dicts, a configuration dict, a prefix to log with, and a
database name to match, return the corresponding glob patterns to match the database dump in an
archive.
Given a sequence of configurations dicts, a configuration dict, a prefix to log with, the
borgmatic runtime directory, and a database name to match, return the corresponding glob
patterns to match the database dump in an archive.
'''
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(make_dump_path('borgmatic'), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, 'borgmatic'), name, hostname='*'
make_dump_path(borgmatic_runtime_directory), name, hostname='*'
),
dump.make_data_source_dump_filename(make_dump_path(config), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, borgmatic_source_directory), name, hostname='*'
make_dump_path(borgmatic_source_directory), name, hostname='*'
),
)
def restore_data_source_dump(
hook_config, config, log_prefix, data_source, dry_run, extract_process, connection_params
hook_config,
config,
log_prefix,
data_source,
dry_run,
extract_process,
connection_params,
borgmatic_runtime_directory,
):
'''
Restore a database from the given extract stream. The database is supplied as a data source
configuration dict, but the given hook configuration is ignored. The given configuration dict is
used to construct the destination path, and the given log prefix is used for any log entries. If
this is a dry run, then don't actually restore anything. Trigger the given active extract
process (an instance of subprocess.Popen) to produce output to consume.
configuration dict, but the given hook configuration is ignored. The given log prefix is used
for any log entries. If this is a dry run, then don't actually restore anything. Trigger the
given active extract process (an instance of subprocess.Popen) to produce output to consume.
'''
dry_run_label = ' (dry run; not actually restoring anything)' if dry_run else ''
hostname = connection_params['hostname'] or data_source.get(
+37 -25
View File
@@ -8,15 +8,11 @@ from borgmatic.hooks import dump
logger = logging.getLogger(__name__)
def make_dump_path(config, base_directory=None): # pragma: no cover
def make_dump_path(base_directory): # pragma: no cover
'''
Given a configuration dict and an optional base directory, make the corresponding dump path. If
a base directory isn't provided, use the borgmatic runtime directory.
Given a base directory, make the corresponding dump path.
'''
return dump.make_data_source_dump_path(
base_directory or borgmatic.config.paths.get_borgmatic_runtime_directory(config),
'mongodb_databases',
)
return dump.make_data_source_dump_path(base_directory, 'mongodb_databases')
def use_streaming(databases, config, log_prefix):
@@ -27,11 +23,12 @@ def use_streaming(databases, config, log_prefix):
return any(database.get('format') != 'directory' for database in databases)
def dump_data_sources(databases, config, log_prefix, dry_run):
def dump_data_sources(databases, config, log_prefix, borgmatic_runtime_directory, dry_run):
'''
Dump the given MongoDB databases to a named pipe. The databases are supplied as a sequence of
dicts, one dict describing each database as per the configuration schema. Use the configuration
dict to construct the destination path and the given log prefix in any log entries.
dicts, one dict describing each database as per the configuration schema. Use the borgmatic
runtime directory to construct the destination path (used for the directory format and the given
log prefix in any log entries.
Return a sequence of subprocess.Popen instances for the dump processes ready to spew to a named
pipe. But if this is a dry run, then don't actually dump anything and return an empty sequence.
@@ -44,7 +41,7 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
for database in databases:
name = database['name']
dump_filename = dump.make_data_source_dump_filename(
make_dump_path(config), name, database.get('hostname')
make_dump_path(borgmatic_runtime_directory), name, database.get('hostname')
)
dump_format = database.get('format', 'archive')
@@ -94,36 +91,49 @@ def build_dump_command(database, dump_filename, dump_format):
)
def remove_data_source_dumps(databases, config, log_prefix, dry_run): # pragma: no cover
def remove_data_source_dumps(
databases, config, log_prefix, borgmatic_runtime_directory, dry_run
): # pragma: no cover
'''
Remove all database dump files for this hook regardless of the given databases. Use the log
prefix in any log entries. Use the given configuration dict to construct the destination path.
If this is a dry run, then don't actually remove anything.
Remove all database dump files for this hook regardless of the given databases. Use the
borgmatic_runtime_directory to construct the destination path and the log prefix in any log
entries. If this is a dry run, then don't actually remove anything.
'''
dump.remove_data_source_dumps(make_dump_path(config), 'MongoDB', log_prefix, dry_run)
dump.remove_data_source_dumps(
make_dump_path(borgmatic_runtime_directory), 'MongoDB', log_prefix, dry_run
)
def make_data_source_dump_patterns(databases, config, log_prefix, name=None): # pragma: no cover
def make_data_source_dump_patterns(
databases, config, log_prefix, borgmatic_runtime_directory, name=None
): # pragma: no cover
'''
Given a sequence of database configurations dicts, a configuration dict, a prefix to log with,
and a database name to match, return the corresponding glob patterns to match the database dump
in an archive.
Given a sequence of configurations dicts, a configuration dict, a prefix to log with, the
borgmatic runtime directory, and a database name to match, return the corresponding glob
patterns to match the database dump in an archive.
'''
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(make_dump_path('borgmatic'), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, 'borgmatic'), name, hostname='*'
make_dump_path(borgmatic_runtime_directory), name, hostname='*'
),
dump.make_data_source_dump_filename(make_dump_path(config), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, borgmatic_source_directory), name, hostname='*'
make_dump_path(borgmatic_source_directory), name, hostname='*'
),
)
def restore_data_source_dump(
hook_config, config, log_prefix, data_source, dry_run, extract_process, connection_params
hook_config,
config,
log_prefix,
data_source,
dry_run,
extract_process,
connection_params,
borgmatic_runtime_directory,
):
'''
Restore a database from the given extract stream. The database is supplied as a data source
@@ -137,7 +147,9 @@ def restore_data_source_dump(
'''
dry_run_label = ' (dry run; not actually restoring anything)' if dry_run else ''
dump_filename = dump.make_data_source_dump_filename(
make_dump_path(config), data_source['name'], data_source.get('hostname')
make_dump_path(borgmatic_runtime_directory),
data_source['name'],
data_source.get('hostname'),
)
restore_command = build_restore_command(
extract_process, data_source, dump_filename, connection_params
+1
View File
@@ -8,6 +8,7 @@ MONITOR_HOOK_NAMES = (
'loki',
'ntfy',
'pagerduty',
'pushover',
'uptime_kuma',
'zabbix',
)
+36 -27
View File
@@ -14,15 +14,11 @@ from borgmatic.hooks import dump
logger = logging.getLogger(__name__)
def make_dump_path(config, base_directory=None): # pragma: no cover
def make_dump_path(base_directory): # pragma: no cover
'''
Given a configuration dict and an optional base directory, make the corresponding dump path. If
a base directory isn't provided, use the borgmatic runtime directory.
Given a base directory, make the corresponding dump path.
'''
return dump.make_data_source_dump_path(
base_directory or borgmatic.config.paths.get_borgmatic_runtime_directory(config),
'mysql_databases',
)
return dump.make_data_source_dump_path(base_directory, 'mysql_databases')
SYSTEM_DATABASE_NAMES = ('information_schema', 'mysql', 'performance_schema', 'sys')
@@ -125,11 +121,12 @@ def use_streaming(databases, config, log_prefix):
return any(databases)
def dump_data_sources(databases, config, log_prefix, dry_run):
def dump_data_sources(databases, config, log_prefix, borgmatic_runtime_directory, dry_run):
'''
Dump the given MySQL/MariaDB databases to a named pipe. The databases are supplied as a sequence
of dicts, one dict describing each database as per the configuration schema. Use the given
configuration dict to construct the destination path and the given log prefix in any log entries.
borgmatic runtime directory to construct the destination path and the given log prefix in any
log entries.
Return a sequence of subprocess.Popen instances for the dump processes ready to spew to a named
pipe. But if this is a dry run, then don't actually dump anything and return an empty sequence.
@@ -140,7 +137,7 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
logger.info(f'{log_prefix}: Dumping MySQL databases{dry_run_label}')
for database in databases:
dump_path = make_dump_path(config)
dump_path = make_dump_path(borgmatic_runtime_directory)
extra_environment = {'MYSQL_PWD': database['password']} if 'password' in database else None
dump_database_names = database_names_to_dump(
database, extra_environment, log_prefix, dry_run
@@ -183,43 +180,55 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
return [process for process in processes if process]
def remove_data_source_dumps(databases, config, log_prefix, dry_run): # pragma: no cover
def remove_data_source_dumps(
databases, config, log_prefix, borgmatic_runtime_directory, dry_run
): # pragma: no cover
'''
Remove all database dump files for this hook regardless of the given databases. Use the given
configuration dict to construct the destination path and the log prefix in any log entries. If
this is a dry run, then don't actually remove anything.
Remove all database dump files for this hook regardless of the given databases. Use the
borgmatic runtime directory to construct the destination path and the log prefix in any log
entries. If this is a dry run, then don't actually remove anything.
'''
dump.remove_data_source_dumps(make_dump_path(config), 'MySQL', log_prefix, dry_run)
dump.remove_data_source_dumps(
make_dump_path(borgmatic_runtime_directory), 'MySQL', log_prefix, dry_run
)
def make_data_source_dump_patterns(databases, config, log_prefix, name=None): # pragma: no cover
def make_data_source_dump_patterns(
databases, config, log_prefix, borgmatic_runtime_directory, name=None
): # pragma: no cover
'''
Given a sequence of configurations dicts, a configuration dict, a prefix to log with, and a
database name to match, return the corresponding glob patterns to match the database dump in an
archive.
Given a sequence of configurations dicts, a configuration dict, a prefix to log with, the
borgmatic runtime directory, and a database name to match, return the corresponding glob
patterns to match the database dump in an archive.
'''
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(make_dump_path('borgmatic'), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, 'borgmatic'), name, hostname='*'
make_dump_path(borgmatic_runtime_directory), name, hostname='*'
),
dump.make_data_source_dump_filename(make_dump_path(config), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, borgmatic_source_directory), name, hostname='*'
make_dump_path(borgmatic_source_directory), name, hostname='*'
),
)
def restore_data_source_dump(
hook_config, config, log_prefix, data_source, dry_run, extract_process, connection_params
hook_config,
config,
log_prefix,
data_source,
dry_run,
extract_process,
connection_params,
borgmatic_runtime_directory,
):
'''
Restore a database from the given extract stream. The database is supplied as a data source
configuration dict, but the given hook configuration is ignored. The given configuration dict is
used to construct the destination path, and the given log prefix is used for any log entries. If
this is a dry run, then don't actually restore anything. Trigger the given active extract
process (an instance of subprocess.Popen) to produce output to consume.
configuration dict, but the given hook configuration is ignored. The given log prefix is used
for any log entries. If this is a dry run, then don't actually restore anything. Trigger the
given active extract process (an instance of subprocess.Popen) to produce output to consume.
'''
dry_run_label = ' (dry run; not actually restoring anything)' if dry_run else ''
hostname = connection_params['hostname'] or data_source.get(
+41 -29
View File
@@ -16,15 +16,11 @@ from borgmatic.hooks import dump
logger = logging.getLogger(__name__)
def make_dump_path(config, base_directory=None): # pragma: no cover
def make_dump_path(base_directory): # pragma: no cover
'''
Given a configuration dict and an optional base directory, make the corresponding dump path. If
a base directory isn't provided, use the borgmatic runtime directory.
Given a base directory, make the corresponding dump path.
'''
return dump.make_data_source_dump_path(
base_directory or borgmatic.config.paths.get_borgmatic_runtime_directory(config),
'postgresql_databases',
)
return dump.make_data_source_dump_path(base_directory, 'postgresql_databases')
def make_extra_environment(database, restore_connection_params=None):
@@ -108,12 +104,12 @@ def use_streaming(databases, config, log_prefix):
return any(database.get('format') != 'directory' for database in databases)
def dump_data_sources(databases, config, log_prefix, dry_run):
def dump_data_sources(databases, config, log_prefix, borgmatic_runtime_directory, dry_run):
'''
Dump the given PostgreSQL databases to a named pipe. The databases are supplied as a sequence of
dicts, one dict describing each database as per the configuration schema. Use the given
configuration dict to construct the destination path and the given log prefix in any log
entries.
borgmatic runtime directory to construct the destination path and the given log prefix in any
log entries.
Return a sequence of subprocess.Popen instances for the dump processes ready to spew to a named
pipe. But if this is a dry run, then don't actually dump anything and return an empty sequence.
@@ -127,7 +123,7 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
for database in databases:
extra_environment = make_extra_environment(database)
dump_path = make_dump_path(config)
dump_path = make_dump_path(borgmatic_runtime_directory)
dump_database_names = database_names_to_dump(
database, extra_environment, log_prefix, dry_run
)
@@ -210,43 +206,57 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
return processes
def remove_data_source_dumps(databases, config, log_prefix, dry_run): # pragma: no cover
def remove_data_source_dumps(
databases, config, log_prefix, borgmatic_runtime_directory, dry_run
): # pragma: no cover
'''
Remove all database dump files for this hook regardless of the given databases. Use the given
configuration dict to construct the destination path and the log prefix in any log entries. If
this is a dry run, then don't actually remove anything.
Remove all database dump files for this hook regardless of the given databases. Use the
borgmatic runtime directory to construct the destination path and the log prefix in any log
entries. If this is a dry run, then don't actually remove anything.
'''
dump.remove_data_source_dumps(make_dump_path(config), 'PostgreSQL', log_prefix, dry_run)
dump.remove_data_source_dumps(
make_dump_path(borgmatic_runtime_directory), 'PostgreSQL', log_prefix, dry_run
)
def make_data_source_dump_patterns(databases, config, log_prefix, name=None): # pragma: no cover
def make_data_source_dump_patterns(
databases, config, log_prefix, borgmatic_runtime_directory, name=None
): # pragma: no cover
'''
Given a sequence of configurations dicts, a configuration dict, a prefix to log with, and a
database name to match, return the corresponding glob patterns to match the database dump in an
archive.
Given a sequence of configurations dicts, a configuration dict, a prefix to log with, the
borgmatic runtime directory, and a database name to match, return the corresponding glob
patterns to match the database dump in an archive.
'''
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(make_dump_path('borgmatic'), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, 'borgmatic'), name, hostname='*'
make_dump_path(borgmatic_runtime_directory), name, hostname='*'
),
dump.make_data_source_dump_filename(make_dump_path(config), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, borgmatic_source_directory), name, hostname='*'
make_dump_path(borgmatic_source_directory), name, hostname='*'
),
)
def restore_data_source_dump(
hook_config, config, log_prefix, data_source, dry_run, extract_process, connection_params
hook_config,
config,
log_prefix,
data_source,
dry_run,
extract_process,
connection_params,
borgmatic_runtime_directory,
):
'''
Restore a database from the given extract stream. The database is supplied as a data source
configuration dict, but the given hook configuration is ignored. The given configuration dict is
used to construct the destination path, and the given log prefix is used for any log entries. If
this is a dry run, then don't actually restore anything. Trigger the given active extract
process (an instance of subprocess.Popen) to produce output to consume.
configuration dict, but the given hook configuration is ignored. The given borgmatic runtime
directory is used to construct the destination path (used for the directory format), and the
given log prefix is used for any log entries. If this is a dry run, then don't actually restore
anything. Trigger the given active extract process (an instance of subprocess.Popen) to produce
output to consume.
If the extract process is None, then restore the dump from the filesystem rather than from an
extract stream.
@@ -267,7 +277,9 @@ def restore_data_source_dump(
all_databases = bool(data_source['name'] == 'all')
dump_filename = dump.make_data_source_dump_filename(
make_dump_path(config), data_source['name'], data_source.get('hostname')
make_dump_path(borgmatic_runtime_directory),
data_source['name'],
data_source.get('hostname'),
)
psql_command = tuple(
shlex.quote(part) for part in shlex.split(data_source.get('psql_command') or 'psql')
+87
View File
@@ -0,0 +1,87 @@
import logging
import requests
logger = logging.getLogger(__name__)
EMERGENCY_PRIORITY = 2
def initialize_monitor(
ping_url, config, config_filename, monitoring_log_level, dry_run
): # pragma: no cover
'''
No initialization is necessary for this monitor.
'''
pass
def ping_monitor(hook_config, config, config_filename, state, monitoring_log_level, dry_run):
'''
Post a message to the configured Pushover application.
If this is a dry run, then don't actually update anything.
'''
run_states = hook_config.get('states', ['fail'])
if state.name.lower() not in run_states:
return
dry_run_label = ' (dry run; not actually updating)' if dry_run else ''
state_config = hook_config.get(state.name.lower(), {})
token = hook_config.get('token')
user = hook_config.get('user')
logger.info(f'{config_filename}: Updating Pushover{dry_run_label}')
if state_config.get('priority') == EMERGENCY_PRIORITY:
if 'expire' not in state_config:
logger.info(f'{config_filename}: Setting expire to default (10 min).')
state_config['expire'] = 600
if 'retry' not in state_config:
logger.info(f'{config_filename}: Setting retry to default (30 sec).')
state_config['retry'] = 30
else:
if 'expire' in state_config or 'retry' in state_config:
raise ValueError(
'The configuration parameters retry and expire should not be set when priority is not equal to 2. Please remove them from the configuration.'
)
state_config = {
key: (int(value) if key == 'html' else value) for key, value in state_config.items()
}
data = dict(
{
'token': token,
'user': user,
# Default to state name. Can be overwritten by state_config below.
'message': state.name.lower(),
},
**state_config,
)
if not dry_run:
logging.getLogger('urllib3').setLevel(logging.ERROR)
try:
response = requests.post(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data=data,
)
if not response.ok:
response.raise_for_status()
except requests.exceptions.RequestException as error:
logger.warning(f'{config_filename}: Pushover error: {error}')
def destroy_monitor(
ping_url_or_uuid, config, config_filename, monitoring_log_level, dry_run
): # pragma: no cover
'''
No destruction is necessary for this monitor.
'''
pass
+36 -28
View File
@@ -9,15 +9,11 @@ from borgmatic.hooks import dump
logger = logging.getLogger(__name__)
def make_dump_path(config, base_directory=None): # pragma: no cover
def make_dump_path(base_directory): # pragma: no cover
'''
Given a configuration dict and an optional base directory, make the corresponding dump path. If
a base directory isn't provided, use the borgmatic runtime directory.
Given a base directory, make the corresponding dump path.
'''
return dump.make_data_source_dump_path(
base_directory or borgmatic.config.paths.get_borgmatic_runtime_directory(config),
'sqlite_databases',
)
return dump.make_data_source_dump_path(base_directory, 'sqlite_databases')
def use_streaming(databases, config, log_prefix):
@@ -28,11 +24,11 @@ def use_streaming(databases, config, log_prefix):
return any(databases)
def dump_data_sources(databases, config, log_prefix, dry_run):
def dump_data_sources(databases, config, log_prefix, borgmatic_runtime_directory, dry_run):
'''
Dump the given SQLite databases to a named pipe. The databases are supplied as a sequence of
configuration dicts, as per the configuration schema. Use the given configuration dict to
construct the destination path and the given log prefix in any log entries.
configuration dicts, as per the configuration schema. Use the given borgmatic runtime directory
to construct the destination path and the given log prefix in any log entries.
Return a sequence of subprocess.Popen instances for the dump processes ready to spew to a named
pipe. But if this is a dry run, then don't actually dump anything and return an empty sequence.
@@ -52,7 +48,7 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
f'{log_prefix}: No SQLite database at {database_path}; an empty database will be created and dumped'
)
dump_path = make_dump_path(config)
dump_path = make_dump_path(borgmatic_runtime_directory)
dump_filename = dump.make_data_source_dump_filename(dump_path, database['name'])
if os.path.exists(dump_filename):
@@ -80,43 +76,55 @@ def dump_data_sources(databases, config, log_prefix, dry_run):
return processes
def remove_data_source_dumps(databases, config, log_prefix, dry_run): # pragma: no cover
def remove_data_source_dumps(
databases, config, log_prefix, borgmatic_runtime_directory, dry_run
): # pragma: no cover
'''
Remove the given SQLite database dumps from the filesystem. The databases are supplied as a
sequence of configuration dicts, as per the configuration schema. Use the given configuration
dict to construct the destination path and the given log prefix in any log entries. If this is a
dry run, then don't actually remove anything.
Remove all database dump files for this hook regardless of the given databases. Use the
borgmatic runtime directory to construct the destination path and the log prefix in any log
entries. If this is a dry run, then don't actually remove anything.
'''
dump.remove_data_source_dumps(make_dump_path(config), 'SQLite', log_prefix, dry_run)
dump.remove_data_source_dumps(
make_dump_path(borgmatic_runtime_directory), 'SQLite', log_prefix, dry_run
)
def make_data_source_dump_patterns(databases, config, log_prefix, name=None): # pragma: no cover
def make_data_source_dump_patterns(
databases, config, log_prefix, borgmatic_runtime_directory, name=None
): # pragma: no cover
'''
Make a pattern that matches the given SQLite databases. The databases are supplied as a sequence
of configuration dicts, as per the configuration schema.
Given a sequence of configurations dicts, a configuration dict, a prefix to log with, the
borgmatic runtime directory, and a database name to match, return the corresponding glob
patterns to match the database dump in an archive.
'''
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(make_dump_path('borgmatic'), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, 'borgmatic'), name, hostname='*'
make_dump_path(borgmatic_runtime_directory), name, hostname='*'
),
dump.make_data_source_dump_filename(make_dump_path(config), name, hostname='*'),
dump.make_data_source_dump_filename(
make_dump_path(config, borgmatic_source_directory), name, hostname='*'
make_dump_path(borgmatic_source_directory), name, hostname='*'
),
)
def restore_data_source_dump(
hook_config, config, log_prefix, data_source, dry_run, extract_process, connection_params
hook_config,
config,
log_prefix,
data_source,
dry_run,
extract_process,
connection_params,
borgmatic_runtime_directory,
):
'''
Restore a database from the given extract stream. The database is supplied as a data source
configuration dict, but the given hook configuration is ignored. The given configuration dict is
used to construct the destination path, and the given log prefix is used for any log entries. If
this is a dry run, then don't actually restore anything. Trigger the given active extract
process (an instance of subprocess.Popen) to produce output to consume.
configuration dict, but the given hook configuration is ignored. The given log prefix is used
for any log entries. If this is a dry run, then don't actually restore anything. Trigger the
given active extract process (an instance of subprocess.Popen) to produce output to consume.
'''
dry_run_label = ' (dry run; not actually restoring anything)' if dry_run else ''
database_path = connection_params['restore_path'] or data_source.get(
+37 -17
View File
@@ -63,23 +63,9 @@ dump formats, which can't stream and therefore do consume temporary disk
space. Additionally, prior to borgmatic 1.5.3, all database dumps consumed
temporary disk space.)
<span class="minilink minilink-addedin">New in version 1.9.0</span> To support
this, borgmatic creates temporary streaming database dumps within
`/run/user/$UID/borgmatic` by default (where `$UID` is the current user's ID).
To customize the `/run/user/$UID` portion of this path, set the
`user_runtime_directory` option in borgmatic's configuration. Alternatively,
set the `XDG_RUNTIME_DIR` environment variable (often already set to
`/run/user/$UID`).
<span class="minilink minilink-addedin">Prior to version 1.9.0</span>
borgmatic created temporary streaming database dumps within the `~/.borgmatic`
directory by default. At that time, the path was configurable by the
`borgmatic_source_directory` configuration option (now deprecated).
Also note that using a database hook implicitly enables the
`read_special` configuration option (even if it's disabled in your
configuration) to support this dump and restore streaming. See Limitations
below for more on this.
Also note that using a database hook implicitly enables the `read_special`
configuration option (even if it's disabled in your configuration) to support
this dump and restore streaming. See Limitations below for more on this.
Here's a more involved example that connects to remote databases:
@@ -127,6 +113,40 @@ additional customization of the options passed to database commands (when
listing databases, restoring databases, etc.).
### Runtime directory
<span class="minilink minilink-addedin">New in version 1.9.0</span> To support
streaming database dumps to Borg, borgmatic uses a runtime directory for
temporary file storage, probing the following locations (in order) to find it:
1. The `user_runtime_directory` borgmatic configuration option.
2. The `XDG_RUNTIME_DIR` environment variable, usually `/run/user/$UID`
(where `$UID` is the current user's ID), automatically set by PAM on Linux
for a user with a session.
3. <span class="minilink minilink-addedin">New in version 1.9.2</span>The
`RUNTIME_DIRECTORY` environment variable, set by systemd if
`RuntimeDirectory=borgmatic` is added to borgmatic's systemd service file.
4. <span class="minilink minilink-addedin">New in version 1.9.1</span>The
`TMPDIR` environment variable, set on macOS for a user with a session,
among other operating systems.
5. <span class="minilink minilink-addedin">New in version 1.9.1</span>The
`TEMP` environment variable, set on various systems.
6. <span class="minilink minilink-addedin">New in version 1.9.2</span>
Hard-coded `/tmp`. <span class="minilink minilink-addedin">Prior to
version 1.9.2</span>This was instead hard-coded to `/run/user/$UID`.
Regardless of the runtime directory selected, borgmatic stores its files
within a `borgmatic` subdirectory of the runtime directory. Additionally, in
the case of `TMPDIR`, `TEMP`, and the hard-coded `/tmp`, borgmatic creates a
randomly named subdirectory in an effort to reduce path collisions in shared
system temporary directories.
<span class="minilink minilink-addedin">Prior to version 1.9.0</span>
borgmatic created temporary streaming database dumps within the `~/.borgmatic`
directory by default. At that time, the path was configurable by the
`borgmatic_source_directory` configuration option (now deprecated).
### All databases
If you want to dump all databases on a host, use `all` for the database name:
@@ -240,6 +240,11 @@ this frequency logic, borgmatic records check timestamps within the
portion of this path, set the `user_state_directory` configuration option.
Alternatively, set the `XDG_STATE_HOME` environment variable.
<span class="minilink minilink-addedin">New in version 1.9.2</span>The
`STATE_DIRECTORY` environment variable also works for this purpose. It's set
by systemd if `StateDirectory=borgmatic` is added to borgmatic's systemd
service file.
<span class="minilink minilink-addedin">Prior to version 1.9.0</span>
borgmatic recorded check timestamps within the `~/.borgmatic` directory. At
that time, the path was configurable by the `borgmatic_source_directory`
+13 -6
View File
@@ -156,12 +156,19 @@ borgmatic umount --mount-point /mnt
## Extract the configuration files used to create an archive
<span class="minilink minilink-addedin">New in version 1.7.15</span> borgmatic
automatically stores all the configuration files used to create an archive
inside the archive itself. They are stored in the archive using their full
paths from the machine being backed up. This is useful in cases where you've
lost a configuration file or you want to see what configurations were used to
create a particular archive.
<span class="minilink minilink-addedin">New in version 1.7.15</span> As part
of creating a backup archive, borgmatic automatically includes all of the
configuration files used when creating it, storing them inside the archive
itself with their full paths from the machine being backed up. This is useful
in cases where you've lost a configuration file or you want to see what
configurations were used to create a particular archive.
To support this, borgmatic creates a manifest file that records the paths of
all the borgmatic configuration files stored within an archive. The file gets
written to borgmatic's runtime directory on disk and then stored within the
archive. See the [runtime directory
documentation](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory)
for how and where that happens.
To extract the configuration files from an archive, use the `config bootstrap`
action. For example:
+4 -4
View File
@@ -119,10 +119,10 @@ archive, regardless of the user who performs the backup. (Note that Borg
doesn't store the leading `/`.)
<span class="minilink minilink-addedin">With Borg version 1.2 and
earlier</span>Database dump files are stored at `/var/run/$UID/borgmatic`
(where `$UID` is the current user's ID) unless overridden by the
`user_runtime_directory` configuration option or the `XDG_STATE_HOME`
environment variable.
earlier</span>Database dump files are stored at a path dependent on the
[runtime
directory](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory)
in use at the time the archive was created.
<span class="minilink minilink-addedin">Prior to borgmatic version
1.9.0</span>Database dump files were instead stored at `~/.borgmatic` within
+71
View File
@@ -46,6 +46,7 @@ them as backups happen:
* [Healthchecks](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#healthchecks-hook)
* [ntfy](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#ntfy-hook)
* [PagerDuty](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#pagerduty-hook)
* [Pushover](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#pushover-hook)
* [Uptime Kuma](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#uptime-kuma-hook)
* [Zabbix](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#zabbix-hook)
@@ -290,6 +291,76 @@ If you have any issues with the integration, [please contact
us](https://torsion.org/borgmatic/#support-and-contributing).
## Pushover hook
<span class="minilink minilink-addedin">New in version 1.9.2</span>
[Pushover](https://pushover.net) makes it easy to get real-time notifications
on your Android, iPhone, iPad, and Desktop (Android Wear and Apple Watch,
too!).
First, create a Pushover account and login on your mobile device. Create an
Application in your Pushover dashboard.
Then, configure borgmatic with your user's unique "User Key" found in your
Pushover dashboard and the unique "API Token" from the created Application.
Here's a basic example:
```yaml
pushover:
token: 7ms6TXHpTokTou2P6x4SodDeentHRa
user: hwRwoWsXMBWwgrSecfa9EfPey55WSN
```
With this configuration, borgmatic creates a Pushover event for your service
whenever borgmatic fails, but only when any of the `create`, `prune`, `compact`,
or `check` actions are run. Note that borgmatic does not contact Pushover
when a backup starts or when it ends without error by default.
You can configure Pushover to have custom parameters declared for borgmatic's
`start`, `fail` and `finish` hooks states.
Here's a more advanced example:
```yaml
pushover:
token: 7ms6TXHpTokTou2P6x4SodDeentHRa
user: hwRwoWsXMBWwgrSecfa9EfPey55WSN
start:
message: "Backup <b>Started</b>"
priority: -2
title: "Backup Started"
html: True
ttl: 10 # Message will be deleted after 10 seconds.
fail:
message: "Backup <font color='#ff6961'>Failed</font>"
priority: 2 # Requests acknowledgement for messages.
expire: 600 # Used only for priority 2. Default is 600 seconds.
retry: 30 # Used only for priority 2. Default is 30 seconds.
device: "pixel8"
title: "Backup Failed"
html: True
sound: "siren"
url: "https://ticketing-system.example.com/login"
url_title: "Login to ticketing system"
finish:
message: "Backup <font color='#77dd77'>Finished</font>"
priority: 0
title: "Backup Finished"
html: True
ttl: 60
url: "https://ticketing-system.example.com/login"
url_title: "Login to ticketing system"
states:
- start
- finish
- fail
```
## ntfy hook
<span class="minilink minilink-addedin">New in version 1.6.3</span>
+24 -6
View File
@@ -39,9 +39,15 @@ Save your password as an encrypted credential to `/etc/credstore.encrypted/borgm
# systemd-ask-password -n | systemd-creds encrypt - /etc/credstore.encrypted/borgmatic.pw
```
Then uncomment or use the following in your configuration file:
```yaml
encryption_passcommand: "cat ${CREDENTIALS_DIRECTORY}/borgmatic.pw"
```
Note that the name `borgmatic.pw` is hardcoded in the systemd service file.
If you use multiple different passwords, save them as encrypted credentials to `/etc/credstore.encrypted/borgmatic/`, e.g.,
To use multiple different passwords, save them as encrypted credentials to `/etc/credstore.encrypted/borgmatic/`, e.g.,
```
# mkdir /etc/credstore.encrypted/borgmatic
@@ -49,17 +55,29 @@ If you use multiple different passwords, save them as encrypted credentials to `
# systemd-ask-password -n | systemd-creds encrypt --name=borgmatic_pw2 - /etc/credstore.encrypted/borgmatic/pw2
...
```
Ensure that the file names, (e.g. `backupserver1`) match the corresponding part of
the `--name` option *after* the underscore (_). The `borgmatic` folder is hardcoded in the systemd service file.
the `--name` option *after* the underscore (_), and that the part *before*
the underscore matches the directory name (e.g. `borgmatic`).
Then uncomment or use one of the following in your configuration file. Adjust `borgmatic_backupserver1`
according to the name given to the credential.
Then, uncomment the appropriate line in the systemd service file:
```yaml
encryption_passcommand: "cat ${CREDENTIALS_DIRECTORY}/borgmatic.pw"
```
# systemctl edit borgmatic.service
...
# Load multiple encrypted credentials.
LoadCredentialEncrypted=borgmatic:/etc/credstore.encrypted/borgmatic/
```
Finally, use the following in your configuration file:
```
encryption_passcommand: "cat ${CREDENTIALS_DIRECTORY}/borgmatic_backupserver1"
```
Adjust `borgmatic_backupserver1` according to the name given to the credential
and the directory set in the service file.
### Environment variable interpolation
<span class="minilink minilink-addedin">New in version 1.6.4</span> borgmatic
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

+2 -3
View File
@@ -1,6 +1,6 @@
[project]
name = "borgmatic"
version = "1.9.0"
version = "1.9.2"
authors = [
{ name="Dan Helfman", email="witten@torsion.org" },
]
@@ -37,8 +37,7 @@ Homepage = "https://torsion.org/borgmatic"
[build-system]
requires = ["setuptools>=61.0"]
# allow looking for conf in setup.{cfg,py}
#build-backend = "setuptools.build_meta"
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
include = ["borgmatic*"]
+6 -2
View File
@@ -9,11 +9,15 @@ Documentation=https://torsion.org/borgmatic/
[Service]
Type=oneshot
RuntimeDirectory=borgmatic
StateDirectory=borgmatic
# Load encrypted credentials.
LoadCredentialEncrypted=borgmatic:/etc/credstore.encrypted/borgmatic/
# Load single encrypted credential.
LoadCredentialEncrypted=borgmatic.pw
# Load multiple encrypted credentials.
# LoadCredentialEncrypted=borgmatic:/etc/credstore.encrypted/borgmatic/
# Security settings for systemd running as root, optional but recommended to improve security. You
# can disable individual settings if they cause problems for your use case. For more details, see
# the systemd manual: https://www.freedesktop.org/software/systemd/man/systemd.exec.html
+94 -15
View File
@@ -16,9 +16,6 @@ def test_get_config_paths_returns_list_of_config_paths():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/source')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/runtime')
flexmock(module).should_receive('make_bootstrap_config').and_return({})
bootstrap_arguments = flexmock(
repository='repo',
@@ -33,6 +30,12 @@ def test_get_config_paths_returns_list_of_config_paths():
dry_run=False,
)
local_borg_version = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
extract_process = flexmock(
stdout=flexmock(
read=lambda: '{"config_paths": ["/borgmatic/config.yaml"]}',
@@ -47,13 +50,61 @@ def test_get_config_paths_returns_list_of_config_paths():
) == ['/borgmatic/config.yaml']
def test_get_config_paths_probes_for_manifest():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/source')
flexmock(module).should_receive('make_bootstrap_config').and_return({})
bootstrap_arguments = flexmock(
repository='repo',
archive='archive',
ssh_command=None,
local_path='borg7',
remote_path='borg8',
borgmatic_source_directory=None,
user_runtime_directory=None,
)
global_arguments = flexmock(
dry_run=False,
)
local_borg_version = flexmock()
borgmatic_runtime_directory = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
borgmatic_runtime_directory,
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
flexmock(module.os.path).should_receive('join').with_args(
'borgmatic', 'bootstrap', 'manifest.json'
).and_return('borgmatic/bootstrap/manifest.json').once()
flexmock(module.os.path).should_receive('join').with_args(
borgmatic_runtime_directory, 'bootstrap', 'manifest.json'
).and_return('run/borgmatic/bootstrap/manifest.json').once()
flexmock(module.os.path).should_receive('join').with_args(
'/source', 'bootstrap', 'manifest.json'
).and_return('/source/bootstrap/manifest.json').once()
manifest_missing_extract_process = flexmock(
stdout=flexmock(read=lambda: None),
)
manifest_found_extract_process = flexmock(
stdout=flexmock(
read=lambda: '{"config_paths": ["/borgmatic/config.yaml"]}',
),
)
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
manifest_missing_extract_process
).and_return(manifest_missing_extract_process).and_return(manifest_found_extract_process)
assert module.get_config_paths(
'archive', bootstrap_arguments, global_arguments, local_borg_version
) == ['/borgmatic/config.yaml']
def test_get_config_paths_translates_ssh_command_argument_to_config():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/source')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/runtime')
config = flexmock()
flexmock(module).should_receive('make_bootstrap_config').and_return(config)
bootstrap_arguments = flexmock(
@@ -69,6 +120,12 @@ def test_get_config_paths_translates_ssh_command_argument_to_config():
dry_run=False,
)
local_borg_version = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
extract_process = flexmock(
stdout=flexmock(
read=lambda: '{"config_paths": ["/borgmatic/config.yaml"]}',
@@ -96,9 +153,6 @@ def test_get_config_paths_with_missing_manifest_raises_value_error():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/source')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/runtime')
flexmock(module).should_receive('make_bootstrap_config').and_return({})
bootstrap_arguments = flexmock(
repository='repo',
@@ -113,6 +167,13 @@ def test_get_config_paths_with_missing_manifest_raises_value_error():
dry_run=False,
)
local_borg_version = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
flexmock(module.os.path).should_receive('join').and_return('run/borgmatic')
extract_process = flexmock(stdout=flexmock(read=lambda: ''))
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
extract_process
@@ -128,9 +189,6 @@ def test_get_config_paths_with_broken_json_raises_value_error():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/source')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/runtime')
flexmock(module).should_receive('make_bootstrap_config').and_return({})
bootstrap_arguments = flexmock(
repository='repo',
@@ -145,6 +203,12 @@ def test_get_config_paths_with_broken_json_raises_value_error():
dry_run=False,
)
local_borg_version = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
extract_process = flexmock(
stdout=flexmock(read=lambda: '{"config_paths": ["/oops'),
)
@@ -162,9 +226,6 @@ def test_get_config_paths_with_json_missing_key_raises_value_error():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/source')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/runtime')
flexmock(module).should_receive('make_bootstrap_config').and_return({})
bootstrap_arguments = flexmock(
repository='repo',
@@ -179,6 +240,12 @@ def test_get_config_paths_with_json_missing_key_raises_value_error():
dry_run=False,
)
local_borg_version = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
extract_process = flexmock(
stdout=flexmock(read=lambda: '{}'),
)
@@ -210,6 +277,12 @@ def test_run_bootstrap_does_not_raise():
dry_run=False,
)
local_borg_version = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
extract_process = flexmock(
stdout=flexmock(
read=lambda: '{"config_paths": ["borgmatic/config.yaml"]}',
@@ -244,6 +317,12 @@ def test_run_bootstrap_translates_ssh_command_argument_to_config():
dry_run=False,
)
local_borg_version = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
extract_process = flexmock(
stdout=flexmock(
read=lambda: '{"config_paths": ["borgmatic/config.yaml"]}',
+30 -29
View File
@@ -381,7 +381,7 @@ def test_upgrade_check_times_moves_checks_from_borgmatic_source_directory_to_sta
'/home/user/.local/state/borgmatic/checks'
).and_return(False)
flexmock(module.os).should_receive('makedirs')
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
'/home/user/.borgmatic/checks', '/home/user/.local/state/borgmatic/checks'
).once()
@@ -408,13 +408,13 @@ def test_upgrade_check_times_with_checks_already_in_borgmatic_state_directory_do
'/home/user/.local/state/borgmatic/checks'
).and_return(True)
flexmock(module.os).should_receive('makedirs').never()
flexmock(module.os).should_receive('rename').never()
flexmock(module.shutil).should_receive('move').never()
flexmock(module).should_receive('make_check_time_path').and_return(
'/home/user/.local/state/borgmatic/checks/1234/archives/all'
)
flexmock(module.os.path).should_receive('isfile').and_return(False)
flexmock(module.os).should_receive('rename').never()
flexmock(module.shutil).should_receive('move').never()
flexmock(module.os).should_receive('mkdir').never()
module.upgrade_check_times(flexmock(), flexmock())
@@ -448,11 +448,11 @@ def test_upgrade_check_times_renames_old_check_paths_to_all():
flexmock(module.os.path).should_receive('isfile').with_args(
f'{base_path}/data.temp'
).and_return(False)
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
f'{base_path}/archives', f'{base_path}/archives.temp'
).once()
flexmock(module.os).should_receive('mkdir').with_args(f'{base_path}/archives').once()
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
f'{base_path}/archives.temp', f'{base_path}/archives/all'
).once()
@@ -484,11 +484,11 @@ def test_upgrade_check_times_renames_data_check_paths_when_archives_paths_are_al
flexmock(module.os.path).should_receive('isfile').with_args(f'{base_path}/data').and_return(
True
)
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
f'{base_path}/data', f'{base_path}/data.temp'
).once()
flexmock(module.os).should_receive('mkdir').with_args(f'{base_path}/data').once()
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
f'{base_path}/data.temp', f'{base_path}/data/all'
).once()
@@ -508,7 +508,7 @@ def test_upgrade_check_times_skips_already_upgraded_check_paths():
'/home/user/.local/state/borgmatic/checks/1234/archives/all'
)
flexmock(module.os.path).should_receive('isfile').and_return(False)
flexmock(module.os).should_receive('rename').never()
flexmock(module.shutil).should_receive('move').never()
flexmock(module.os).should_receive('mkdir').never()
module.upgrade_check_times(flexmock(), flexmock())
@@ -542,11 +542,11 @@ def test_upgrade_check_times_renames_stale_temporary_check_path():
flexmock(module.os.path).should_receive('isfile').with_args(
f'{base_path}/data.temp'
).and_return(False)
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
f'{base_path}/archives', f'{base_path}/archives.temp'
).and_raise(FileNotFoundError)
flexmock(module.os).should_receive('mkdir').with_args(f'{base_path}/archives').once()
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
f'{base_path}/archives.temp', f'{base_path}/archives/all'
).once()
@@ -713,9 +713,6 @@ def test_collect_spot_check_archive_paths_excludes_directories():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/home/user/.borgmatic')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/1001/borgmatic')
flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_return(
(
'f /etc/path',
@@ -732,6 +729,7 @@ def test_collect_spot_check_archive_paths_excludes_directories():
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/user/1001/borgmatic',
) == ('/etc/path', '/etc/other')
@@ -739,9 +737,6 @@ def test_collect_spot_check_archive_paths_excludes_file_in_borgmatic_runtime_dir
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/root/.borgmatic')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_return(
(
'f /etc/path',
@@ -757,6 +752,7 @@ def test_collect_spot_check_archive_paths_excludes_file_in_borgmatic_runtime_dir
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/user/0/borgmatic',
) == ('/etc/path',)
@@ -764,9 +760,6 @@ def test_collect_spot_check_archive_paths_excludes_file_in_borgmatic_source_dire
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/root/.borgmatic')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_return(
(
'f /etc/path',
@@ -782,6 +775,7 @@ def test_collect_spot_check_archive_paths_excludes_file_in_borgmatic_source_dire
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/user/0/borgmatic',
) == ('/etc/path',)
@@ -789,9 +783,6 @@ def test_collect_spot_check_archive_paths_excludes_file_in_borgmatic_runtime_dir
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/root.borgmatic')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_return(
(
'f /etc/path',
@@ -807,6 +798,7 @@ def test_collect_spot_check_archive_paths_excludes_file_in_borgmatic_runtime_dir
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/user/0/borgmatic',
) == ('/etc/path',)
@@ -887,7 +879,7 @@ def test_compare_spot_check_hashes_returns_paths_having_failing_hashes():
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
log_label='repo',
log_prefix='repo',
source_paths=('/foo', '/bar', '/baz', '/quux'),
) == ('/bar',)
@@ -928,7 +920,7 @@ def test_compare_spot_check_hashes_handles_data_sample_percentage_above_100():
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
log_label='repo',
log_prefix='repo',
source_paths=('/foo', '/bar'),
) == ('/foo', '/bar')
@@ -966,7 +958,7 @@ def test_compare_spot_check_hashes_uses_xxh64sum_command_option():
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
log_label='repo',
log_prefix='repo',
source_paths=('/foo', '/bar', '/baz', '/quux'),
) == ('/bar',)
@@ -1003,7 +995,7 @@ def test_compare_spot_check_hashes_considers_path_missing_from_archive_as_not_ma
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
log_label='repo',
log_prefix='repo',
source_paths=('/foo', '/bar', '/baz', '/quux'),
) == ('/bar',)
@@ -1039,7 +1031,7 @@ def test_compare_spot_check_hashes_considers_non_existent_path_as_not_matching()
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
log_label='repo',
log_prefix='repo',
source_paths=('/foo', '/bar', '/baz', '/quux'),
) == ('/bar',)
@@ -1086,7 +1078,7 @@ def test_compare_spot_check_hashes_with_too_many_paths_feeds_them_to_commands_in
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
log_label='repo',
log_prefix='repo',
source_paths=('/foo', '/bar', '/baz', '/quux'),
) == ('/quux',)
@@ -1129,7 +1121,7 @@ def test_compare_spot_check_hashes_uses_working_directory_to_access_source_paths
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
log_label='repo',
log_prefix='repo',
source_paths=('foo', 'bar', 'baz', 'quux'),
) == ('bar',)
@@ -1149,6 +1141,7 @@ def test_spot_check_without_spot_configuration_errors():
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1161,6 +1154,7 @@ def test_spot_check_without_any_configuration_errors():
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1181,6 +1175,7 @@ def test_spot_check_data_tolerance_percenatge_greater_than_data_sample_percentag
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1212,6 +1207,7 @@ def test_spot_check_with_count_delta_greater_than_count_tolerance_percentage_err
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1244,6 +1240,7 @@ def test_spot_check_with_failing_percentage_greater_than_data_tolerance_percenta
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1275,6 +1272,7 @@ def test_spot_check_with_high_enough_tolerances_does_not_raise():
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1362,6 +1360,9 @@ def test_run_check_runs_configured_spot_check():
flexmock(module).should_receive('make_archives_check_id').and_return(None)
flexmock(module).should_receive('filter_checks_on_frequency').and_return({'spot'})
flexmock(module.borgmatic.borg.check).should_receive('check_archives').never()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.actions.check).should_receive('spot_check').once()
flexmock(module).should_receive('make_check_time_path')
flexmock(module).should_receive('write_check_time')
+24 -15
View File
@@ -8,6 +8,9 @@ from borgmatic.actions import create as module
def test_run_create_executes_and_calls_hooks_for_configured_repository():
flexmock(module.logger).answer = lambda message: None
flexmock(module.borgmatic.config.validate).should_receive('repositories_match').never()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.borg.create).should_receive('create_archive').once()
flexmock(module).should_receive('create_borgmatic_manifest').once()
flexmock(module.borgmatic.hooks.command).should_receive('execute_hook').times(2)
@@ -44,6 +47,9 @@ def test_run_create_executes_and_calls_hooks_for_configured_repository():
def test_run_create_with_store_config_files_false_does_not_create_borgmatic_manifest():
flexmock(module.logger).answer = lambda message: None
flexmock(module.borgmatic.config.validate).should_receive('repositories_match').never()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.borg.create).should_receive('create_archive').once()
flexmock(module).should_receive('create_borgmatic_manifest').never()
flexmock(module.borgmatic.hooks.command).should_receive('execute_hook').times(2)
@@ -82,6 +88,9 @@ def test_run_create_runs_with_selected_repository():
flexmock(module.borgmatic.config.validate).should_receive(
'repositories_match'
).once().and_return(True)
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.borg.create).should_receive('create_archive').once()
flexmock(module).should_receive('create_borgmatic_manifest').once()
flexmock(module.borgmatic.hooks.command).should_receive('execute_hook').times(2)
@@ -120,6 +129,7 @@ def test_run_create_bails_if_repository_does_not_match():
flexmock(module.borgmatic.config.validate).should_receive(
'repositories_match'
).once().and_return(False)
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').never()
flexmock(module.borgmatic.borg.create).should_receive('create_archive').never()
flexmock(module).should_receive('create_borgmatic_manifest').never()
create_arguments = flexmock(
@@ -153,6 +163,9 @@ def test_run_create_produces_json():
flexmock(module.borgmatic.config.validate).should_receive(
'repositories_match'
).once().and_return(True)
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.borg.create).should_receive('create_archive').once().and_return(
flexmock()
)
@@ -191,18 +204,15 @@ def test_run_create_produces_json():
def test_create_borgmatic_manifest_creates_manifest_file():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.os.path).should_receive('join').with_args(
'/run/user/0/borgmatic', 'bootstrap', 'manifest.json'
).and_return('/run/user/0/borgmatic/bootstrap/manifest.json')
'/run/borgmatic', 'bootstrap', 'manifest.json'
).and_return('/run/borgmatic/bootstrap/manifest.json')
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.os).should_receive('makedirs').and_return(True)
flexmock(module.importlib.metadata).should_receive('version').and_return('1.0.0')
flexmock(sys.modules['builtins']).should_receive('open').with_args(
'/run/user/0/borgmatic/bootstrap/manifest.json', 'w'
'/run/borgmatic/bootstrap/manifest.json', 'w'
).and_return(
flexmock(
__enter__=lambda *args: flexmock(write=lambda *args: None, close=lambda *args: None),
@@ -211,22 +221,19 @@ def test_create_borgmatic_manifest_creates_manifest_file():
)
flexmock(module.json).should_receive('dump').and_return(True).once()
module.create_borgmatic_manifest({}, 'test.yaml', False)
module.create_borgmatic_manifest({}, 'test.yaml', '/run/borgmatic', False)
def test_create_borgmatic_manifest_creates_manifest_file_with_custom_borgmatic_runtime_directory():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/borgmatic')
flexmock(module.os.path).should_receive('join').with_args(
'/borgmatic', 'bootstrap', 'manifest.json'
).and_return('/borgmatic/bootstrap/manifest.json')
'/run/borgmatic', 'bootstrap', 'manifest.json'
).and_return('/run/borgmatic/bootstrap/manifest.json')
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.os).should_receive('makedirs').and_return(True)
flexmock(module.importlib.metadata).should_receive('version').and_return('1.0.0')
flexmock(sys.modules['builtins']).should_receive('open').with_args(
'/borgmatic/bootstrap/manifest.json', 'w'
'/run/borgmatic/bootstrap/manifest.json', 'w'
).and_return(
flexmock(
__enter__=lambda *args: flexmock(write=lambda *args: None, close=lambda *args: None),
@@ -236,9 +243,11 @@ def test_create_borgmatic_manifest_creates_manifest_file_with_custom_borgmatic_r
flexmock(module.json).should_receive('dump').and_return(True).once()
module.create_borgmatic_manifest(
{'borgmatic_runtime_directory': '/borgmatic'}, 'test.yaml', False
{'borgmatic_runtime_directory': '/borgmatic'}, 'test.yaml', '/run/borgmatic', False
)
def test_create_borgmatic_manifest_does_not_create_manifest_file_on_dry_run():
module.create_borgmatic_manifest({}, 'test.yaml', True)
flexmock(module.json).should_receive('dump').never()
module.create_borgmatic_manifest({}, 'test.yaml', '/run/borgmatic', True)
+61 -33
View File
@@ -75,10 +75,10 @@ def test_strip_path_prefix_from_extracted_dump_destination_renames_first_matchin
]
)
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
'/foo/bar/postgresql_databases', '/run/user/0/borgmatic/postgresql_databases'
).once()
flexmock(module.os).should_receive('rename').with_args(
flexmock(module.shutil).should_receive('move').with_args(
'/foo/bar/mariadb_databases', '/run/user/0/borgmatic/mariadb_databases'
).never()
@@ -87,11 +87,8 @@ def test_strip_path_prefix_from_extracted_dump_destination_renames_first_matchin
def test_restore_single_data_source_extracts_and_restores_single_file_dump():
flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks').with_args(
'make_data_source_dump_patterns', object, object, object, object
'make_data_source_dump_patterns', object, object, object, object, object
).and_return({'postgresql': flexmock()})
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.tempfile).should_receive('mkdtemp').never()
flexmock(module.borgmatic.hooks.dump).should_receive(
'convert_glob_patterns_to_borg_pattern'
@@ -110,6 +107,7 @@ def test_restore_single_data_source_extracts_and_restores_single_file_dump():
dry_run=object,
extract_process=object,
connection_params=object,
borgmatic_runtime_directory=object,
).once()
module.restore_single_data_source(
@@ -123,16 +121,14 @@ def test_restore_single_data_source_extracts_and_restores_single_file_dump():
hook_name='postgresql',
data_source={'name': 'test', 'format': 'plain'},
connection_params=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
def test_restore_single_data_source_extracts_and_restores_directory_dump():
flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks').with_args(
'make_data_source_dump_patterns', object, object, object, object
'make_data_source_dump_patterns', object, object, object, object, object
).and_return({'postgresql': flexmock()})
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.tempfile).should_receive('mkdtemp').once().and_return(
'/run/user/0/borgmatic/tmp1234'
)
@@ -153,6 +149,7 @@ def test_restore_single_data_source_extracts_and_restores_directory_dump():
dry_run=object,
extract_process=object,
connection_params=object,
borgmatic_runtime_directory='/run/borgmatic',
).once()
module.restore_single_data_source(
@@ -166,16 +163,14 @@ def test_restore_single_data_source_extracts_and_restores_directory_dump():
hook_name='postgresql',
data_source={'name': 'test', 'format': 'directory'},
connection_params=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
def test_restore_single_data_source_with_directory_dump_error_cleans_up_temporary_directory():
flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks').with_args(
'make_data_source_dump_patterns', object, object, object, object
'make_data_source_dump_patterns', object, object, object, object, object
).and_return({'postgresql': flexmock()})
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.tempfile).should_receive('mkdtemp').once().and_return(
'/run/user/0/borgmatic/tmp1234'
)
@@ -196,6 +191,7 @@ def test_restore_single_data_source_with_directory_dump_error_cleans_up_temporar
dry_run=object,
extract_process=object,
connection_params=object,
borgmatic_runtime_directory='/run/user/0/borgmatic/tmp1234',
).never()
with pytest.raises(ValueError):
@@ -210,19 +206,15 @@ def test_restore_single_data_source_with_directory_dump_error_cleans_up_temporar
hook_name='postgresql',
data_source={'name': 'test', 'format': 'directory'},
connection_params=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
def test_restore_single_data_source_with_directory_dump_and_dry_run_skips_directory_move_and_cleanup():
flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks').with_args(
'make_data_source_dump_patterns', object, object, object, object
'make_data_source_dump_patterns', object, object, object, object, object
).and_return({'postgresql': flexmock()})
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.tempfile).should_receive('mkdtemp').once().and_return(
'/run/user/0/borgmatic/tmp1234'
)
flexmock(module.tempfile).should_receive('mkdtemp').once().and_return('/run/borgmatic/tmp1234')
flexmock(module.borgmatic.hooks.dump).should_receive(
'convert_glob_patterns_to_borg_pattern'
).and_return(flexmock())
@@ -240,6 +232,7 @@ def test_restore_single_data_source_with_directory_dump_and_dry_run_skips_direct
dry_run=object,
extract_process=object,
connection_params=object,
borgmatic_runtime_directory='/run/borgmatic',
).once()
module.restore_single_data_source(
@@ -253,6 +246,7 @@ def test_restore_single_data_source_with_directory_dump_and_dry_run_skips_direct
hook_name='postgresql',
data_source={'name': 'test', 'format': 'directory'},
connection_params=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -260,9 +254,6 @@ def test_collect_archive_data_source_names_parses_archive_paths():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/root/.borgmatic')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.borgmatic.hooks.dump).should_receive('make_data_source_dump_path').and_return(
''
)
@@ -282,6 +273,7 @@ def test_collect_archive_data_source_names_parses_archive_paths():
global_arguments=flexmock(log_json=False),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
assert archive_data_source_names == {
@@ -294,9 +286,6 @@ def test_collect_archive_data_source_names_parses_archive_paths_with_different_b
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/root/.borgmatic')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.borgmatic.hooks.dump).should_receive('make_data_source_dump_path').and_return(
''
)
@@ -317,6 +306,7 @@ def test_collect_archive_data_source_names_parses_archive_paths_with_different_b
global_arguments=flexmock(log_json=False),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
assert archive_data_source_names == {
@@ -329,9 +319,6 @@ def test_collect_archive_data_source_names_parses_directory_format_archive_paths
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/root/.borgmatic')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.borgmatic.hooks.dump).should_receive('make_data_source_dump_path').and_return(
''
)
@@ -350,6 +337,7 @@ def test_collect_archive_data_source_names_parses_directory_format_archive_paths
global_arguments=flexmock(log_json=False),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
assert archive_data_source_names == {
@@ -361,9 +349,6 @@ def test_collect_archive_data_source_names_skips_bad_archive_paths():
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_source_directory'
).and_return('/root/.borgmatic')
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/run/user/0/borgmatic')
flexmock(module.borgmatic.hooks.dump).should_receive('make_data_source_dump_path').and_return(
''
)
@@ -384,6 +369,7 @@ def test_collect_archive_data_source_names_skips_bad_archive_paths():
global_arguments=flexmock(log_json=False),
local_path=flexmock(),
remote_path=flexmock(),
borgmatic_runtime_directory='/run/borgmatic',
)
assert archive_data_source_names == {
@@ -481,6 +467,13 @@ def test_run_restore_restores_each_data_source():
}
flexmock(module.borgmatic.config.validate).should_receive('repositories_match').and_return(True)
borgmatic_runtime_directory = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
borgmatic_runtime_directory
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks_even_if_unconfigured')
flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').and_return(
flexmock()
@@ -501,6 +494,7 @@ def test_run_restore_restores_each_data_source():
hook_name='postgresql_databases',
data_source={'name': 'foo', 'schemas': None},
connection_params=object,
borgmatic_runtime_directory=borgmatic_runtime_directory,
).once()
flexmock(module).should_receive('restore_single_data_source').with_args(
repository=object,
@@ -513,6 +507,7 @@ def test_run_restore_restores_each_data_source():
hook_name='postgresql_databases',
data_source={'name': 'bar', 'schemas': None},
connection_params=object,
borgmatic_runtime_directory=borgmatic_runtime_directory,
).once()
flexmock(module).should_receive('ensure_data_sources_found')
@@ -541,6 +536,12 @@ def test_run_restore_bails_for_non_matching_repository():
flexmock(module.borgmatic.config.validate).should_receive('repositories_match').and_return(
False
)
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
flexmock()
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
flexmock(module.borgmatic.hooks.dispatch).should_receive(
'call_hooks_even_if_unconfigured'
).never()
@@ -563,6 +564,13 @@ def test_run_restore_restores_data_source_configured_with_all_name():
}
flexmock(module.borgmatic.config.validate).should_receive('repositories_match').and_return(True)
borgmatic_runtime_directory = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
borgmatic_runtime_directory
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks_even_if_unconfigured')
flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').and_return(
flexmock()
@@ -599,6 +607,7 @@ def test_run_restore_restores_data_source_configured_with_all_name():
hook_name='postgresql_databases',
data_source={'name': 'foo', 'schemas': None},
connection_params=object,
borgmatic_runtime_directory=borgmatic_runtime_directory,
).once()
flexmock(module).should_receive('restore_single_data_source').with_args(
repository=object,
@@ -611,6 +620,7 @@ def test_run_restore_restores_data_source_configured_with_all_name():
hook_name='postgresql_databases',
data_source={'name': 'bar', 'schemas': None},
connection_params=object,
borgmatic_runtime_directory=borgmatic_runtime_directory,
).once()
flexmock(module).should_receive('ensure_data_sources_found')
@@ -641,6 +651,13 @@ def test_run_restore_skips_missing_data_source():
}
flexmock(module.borgmatic.config.validate).should_receive('repositories_match').and_return(True)
borgmatic_runtime_directory = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
borgmatic_runtime_directory
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks_even_if_unconfigured')
flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').and_return(
flexmock()
@@ -677,6 +694,7 @@ def test_run_restore_skips_missing_data_source():
hook_name='postgresql_databases',
data_source={'name': 'foo', 'schemas': None},
connection_params=object,
borgmatic_runtime_directory=borgmatic_runtime_directory,
).once()
flexmock(module).should_receive('restore_single_data_source').with_args(
repository=object,
@@ -689,6 +707,7 @@ def test_run_restore_skips_missing_data_source():
hook_name='postgresql_databases',
data_source={'name': 'bar', 'schemas': None},
connection_params=object,
borgmatic_runtime_directory=borgmatic_runtime_directory,
).never()
flexmock(module).should_receive('ensure_data_sources_found')
@@ -720,6 +739,13 @@ def test_run_restore_restores_data_sources_from_different_hooks():
}
flexmock(module.borgmatic.config.validate).should_receive('repositories_match').and_return(True)
borgmatic_runtime_directory = flexmock()
flexmock(module.borgmatic.config.paths).should_receive('Runtime_directory').and_return(
borgmatic_runtime_directory
)
flexmock(module.borgmatic.config.paths).should_receive(
'make_runtime_directory_glob'
).replace_with(lambda path: path)
flexmock(module.borgmatic.hooks.dispatch).should_receive('call_hooks_even_if_unconfigured')
flexmock(module.borgmatic.borg.repo_list).should_receive('resolve_archive_name').and_return(
flexmock()
@@ -749,6 +775,7 @@ def test_run_restore_restores_data_sources_from_different_hooks():
hook_name='postgresql_databases',
data_source={'name': 'foo', 'schemas': None},
connection_params=object,
borgmatic_runtime_directory=borgmatic_runtime_directory,
).once()
flexmock(module).should_receive('restore_single_data_source').with_args(
repository=object,
@@ -761,6 +788,7 @@ def test_run_restore_restores_data_sources_from_different_hooks():
hook_name='mysql_databases',
data_source={'name': 'bar', 'schemas': None},
connection_params=object,
borgmatic_runtime_directory=borgmatic_runtime_directory,
).once()
flexmock(module).should_receive('ensure_data_sources_found')
+25
View File
@@ -703,6 +703,31 @@ def test_check_archives_with_remote_path_passes_through_to_borg():
)
def test_check_archives_with_umask_passes_through_to_borg():
checks = {'repository'}
config = {'umask': '077'}
flexmock(module).should_receive('make_check_name_flags').with_args(checks, ()).and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
insert_execute_command_mock(('borg', 'check', '--umask', '077', 'repo'))
module.check_archives(
repository_path='repo',
config=config,
local_borg_version='1.2.3',
check_arguments=flexmock(
progress=None,
repair=None,
only_checks=None,
force=None,
match_archives=None,
max_duration=None,
),
global_arguments=flexmock(log_json=False),
checks=checks,
archive_filter_flags=(),
)
def test_check_archives_with_log_json_passes_through_to_borg():
checks = {'repository'}
config = {}
+17 -51
View File
@@ -1453,9 +1453,6 @@ def test_create_archive_calls_borg_with_parameters():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1483,6 +1480,7 @@ def test_create_archive_calls_borg_with_parameters():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
)
@@ -1490,9 +1488,6 @@ def test_create_archive_calls_borg_with_environment():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1521,6 +1516,7 @@ def test_create_archive_calls_borg_with_environment():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
)
@@ -1528,9 +1524,6 @@ def test_create_archive_with_log_info_calls_borg_with_info_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1559,6 +1552,7 @@ def test_create_archive_with_log_info_calls_borg_with_info_parameter():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
)
@@ -1566,9 +1560,6 @@ def test_create_archive_with_log_info_and_json_suppresses_most_borg_output():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1595,6 +1586,7 @@ def test_create_archive_with_log_info_and_json_suppresses_most_borg_output():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
json=True,
)
@@ -1603,9 +1595,6 @@ def test_create_archive_with_log_debug_calls_borg_with_debug_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1634,6 +1623,7 @@ def test_create_archive_with_log_debug_calls_borg_with_debug_parameter():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
)
@@ -1641,9 +1631,6 @@ def test_create_archive_with_log_debug_and_json_suppresses_most_borg_output():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1670,6 +1657,7 @@ def test_create_archive_with_log_debug_and_json_suppresses_most_borg_output():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
json=True,
)
@@ -1680,9 +1668,6 @@ def test_create_archive_with_stats_and_dry_run_calls_borg_without_stats():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create', '--dry-run'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1711,6 +1696,7 @@ def test_create_archive_with_stats_and_dry_run_calls_borg_without_stats():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
stats=True,
)
@@ -1719,9 +1705,6 @@ def test_create_archive_with_working_directory_calls_borg_with_working_directory
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1752,6 +1735,7 @@ def test_create_archive_with_working_directory_calls_borg_with_working_directory
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
)
@@ -1759,9 +1743,6 @@ def test_create_archive_with_exit_codes_calls_borg_using_them():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1791,6 +1772,7 @@ def test_create_archive_with_exit_codes_calls_borg_using_them():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
)
@@ -1798,9 +1780,6 @@ def test_create_archive_with_stats_calls_borg_with_stats_parameter_and_answer_ou
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1828,6 +1807,7 @@ def test_create_archive_with_stats_calls_borg_with_stats_parameter_and_answer_ou
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
stats=True,
)
@@ -1836,9 +1816,6 @@ def test_create_archive_with_files_calls_borg_with_answer_output_log_level():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(
@@ -1871,6 +1848,7 @@ def test_create_archive_with_files_calls_borg_with_answer_output_log_level():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
list_files=True,
)
@@ -1879,9 +1857,6 @@ def test_create_archive_with_progress_and_log_info_calls_borg_with_progress_para
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1910,6 +1885,7 @@ def test_create_archive_with_progress_and_log_info_calls_borg_with_progress_para
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
progress=True,
)
@@ -1918,9 +1894,6 @@ def test_create_archive_with_progress_calls_borg_with_progress_parameter():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -1948,6 +1921,7 @@ def test_create_archive_with_progress_calls_borg_with_progress_parameter():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
progress=True,
)
@@ -1957,9 +1931,6 @@ def test_create_archive_with_progress_and_stream_processes_calls_borg_with_progr
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
processes = flexmock()
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(
@@ -2009,6 +1980,7 @@ def test_create_archive_with_progress_and_stream_processes_calls_borg_with_progr
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
progress=True,
stream_processes=processes,
)
@@ -2018,9 +1990,6 @@ def test_create_archive_with_json_calls_borg_with_json_flag():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -2046,6 +2015,7 @@ def test_create_archive_with_json_calls_borg_with_json_flag():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
json=True,
)
@@ -2056,9 +2026,6 @@ def test_create_archive_with_stats_and_json_calls_borg_without_stats_flag():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -2084,6 +2051,7 @@ def test_create_archive_with_stats_and_json_calls_borg_without_stats_flag():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
json=True,
stats=True,
)
@@ -2095,9 +2063,6 @@ def test_create_archive_calls_borg_with_working_directory():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('expand_directories').and_return(())
flexmock(module.borgmatic.config.paths).should_receive(
'get_borgmatic_runtime_directory'
).and_return('/var/run/0/borgmatic')
flexmock(module).should_receive('collect_borgmatic_runtime_directories').and_return([])
flexmock(module).should_receive('make_base_create_command').and_return(
(('borg', 'create'), REPO_ARCHIVE_WITH_PATHS, flexmock(), flexmock())
@@ -2128,6 +2093,7 @@ def test_create_archive_calls_borg_with_working_directory():
config_paths=['/tmp/test.yaml'],
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
borgmatic_runtime_directory='/borgmatic/run',
)
+23
View File
@@ -100,6 +100,29 @@ def test_make_delete_command_includes_remote_path():
assert command == ('borg', 'delete', '--remote-path', 'borg1', 'repo')
def test_make_delete_command_includes_umask():
flexmock(module.borgmatic.borg.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.borgmatic.borg.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.borgmatic.borg.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
('repo',)
)
command = module.make_delete_command(
repository={'path': 'repo'},
config={'umask': '077'},
local_borg_version='1.2.3',
delete_arguments=flexmock(list_archives=False, force=0, match_archives=None, archive=None),
global_arguments=flexmock(dry_run=False, log_json=False),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'delete', '--umask', '077', 'repo')
def test_make_delete_command_includes_log_json():
flexmock(module.borgmatic.borg.flags).should_receive('make_flags').and_return(())
flexmock(module.borgmatic.borg.flags).should_receive('make_flags').with_args(
+23 -1
View File
@@ -1,3 +1,5 @@
from flexmock import flexmock
from borgmatic.borg import environment as module
@@ -8,21 +10,24 @@ def test_make_environment_with_passcommand_should_set_environment():
def test_make_environment_with_passphrase_should_set_environment():
flexmock(module.os.environ).should_receive('get').and_return(None)
environment = module.make_environment({'encryption_passphrase': 'pass'})
assert environment.get('BORG_PASSPHRASE') == 'pass'
def test_make_environment_with_ssh_command_should_set_environment():
flexmock(module.os.environ).should_receive('get').and_return(None)
environment = module.make_environment({'ssh_command': 'ssh -C'})
assert environment.get('BORG_RSH') == 'ssh -C'
def test_make_environment_without_configuration_sets_certain_environment_variables():
flexmock(module.os.environ).should_receive('get').and_return(None)
environment = module.make_environment({})
# borgmatic always sets this Borg environment variable.
# Default environment variables.
assert environment == {
'BORG_EXIT_CODES': 'modern',
'BORG_RELOCATED_REPO_ACCESS_IS_OK': 'no',
@@ -30,30 +35,47 @@ def test_make_environment_without_configuration_sets_certain_environment_variabl
}
def test_make_environment_without_configuration_does_not_set_certain_environment_variables_if_already_set():
flexmock(module.os.environ).should_receive('get').with_args(
'BORG_RELOCATED_REPO_ACCESS_IS_OK'
).and_return('yup')
flexmock(module.os.environ).should_receive('get').with_args(
'BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK'
).and_return('nah')
environment = module.make_environment({})
assert environment == {'BORG_EXIT_CODES': 'modern'}
def test_make_environment_with_relocated_repo_access_true_should_set_environment_yes():
flexmock(module.os.environ).should_receive('get').and_return(None)
environment = module.make_environment({'relocated_repo_access_is_ok': True})
assert environment.get('BORG_RELOCATED_REPO_ACCESS_IS_OK') == 'yes'
def test_make_environment_with_relocated_repo_access_false_should_set_environment_no():
flexmock(module.os.environ).should_receive('get').and_return(None)
environment = module.make_environment({'relocated_repo_access_is_ok': False})
assert environment.get('BORG_RELOCATED_REPO_ACCESS_IS_OK') == 'no'
def test_make_environment_check_i_know_what_i_am_doing_true_should_set_environment_YES():
flexmock(module.os.environ).should_receive('get').and_return(None)
environment = module.make_environment({'check_i_know_what_i_am_doing': True})
assert environment.get('BORG_CHECK_I_KNOW_WHAT_I_AM_DOING') == 'YES'
def test_make_environment_check_i_know_what_i_am_doing_false_should_set_environment_NO():
flexmock(module.os.environ).should_receive('get').and_return(None)
environment = module.make_environment({'check_i_know_what_i_am_doing': False})
assert environment.get('BORG_CHECK_I_KNOW_WHAT_I_AM_DOING') == 'NO'
def test_make_environment_with_integer_variable_value():
flexmock(module.os.environ).should_receive('get').and_return(None)
environment = module.make_environment({'borg_files_cache_ttl': 40})
assert environment.get('BORG_FILES_CACHE_TTL') == '40'
+23
View File
@@ -203,6 +203,29 @@ def test_make_info_command_with_remote_path_passes_through_to_command():
assert command == ('borg', 'info', '--remote-path', 'borg1', '--repo', 'repo')
def test_make_info_command_with_umask_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '2.3.4'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
command = module.make_info_command(
repository_path='repo',
config={'umask': '077'},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
info_arguments=flexmock(archive=None, json=False, prefix=None, match_archives=None),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'info', '--umask', '077', '--repo', 'repo')
def test_make_info_command_with_log_json_passes_through_to_command():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_flags').with_args('log-json', True).and_return(
+22 -2
View File
@@ -94,7 +94,9 @@ def test_make_list_command_includes_json():
def test_make_list_command_includes_log_json():
flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(('--log-json',))
flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(()).and_return(
('--log-json',)
)
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
@@ -110,7 +112,7 @@ def test_make_list_command_includes_log_json():
def test_make_list_command_includes_lock_wait():
flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(
flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(()).and_return(
('--lock-wait', '5')
)
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
@@ -203,6 +205,24 @@ def test_make_list_command_includes_remote_path():
assert command == ('borg', 'list', '--remote-path', 'borg2', 'repo')
def test_make_list_command_includes_umask():
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
command = module.make_list_command(
repository_path='repo',
config={'umask': '077'},
local_borg_version='1.2.3',
list_arguments=flexmock(archive=None, paths=None, json=False),
global_arguments=flexmock(log_json=False),
)
assert command == ('borg', 'list', '--umask', '077', 'repo')
def test_make_list_command_includes_short():
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(('--short',))
+21
View File
@@ -438,6 +438,27 @@ def test_create_repository_with_remote_path_calls_borg_with_remote_path_flag():
)
def test_create_repository_with_umask_calls_borg_with_umask_flag():
insert_repo_info_command_not_found_mock()
insert_repo_create_command_mock(REPO_CREATE_COMMAND + ('--umask', '077', '--repo', 'repo'))
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
'repo',
)
)
module.create_repository(
dry_run=False,
repository_path='repo',
config={'umask': '077'},
local_borg_version='2.3.4',
global_arguments=flexmock(log_json=False),
encryption_mode='repokey',
)
def test_create_repository_with_extra_borg_options_calls_borg_with_extra_options():
insert_repo_info_command_not_found_mock()
insert_repo_create_command_mock(
+23
View File
@@ -141,6 +141,29 @@ def test_make_repo_delete_command_includes_remote_path():
assert command == ('borg', 'repo-delete', '--remote-path', 'borg1', 'repo')
def test_make_repo_delete_command_includes_umask():
flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
flexmock(module.borgmatic.borg.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.borgmatic.borg.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.borgmatic.borg.flags).should_receive('make_repository_flags').and_return(
('repo',)
)
command = module.make_repo_delete_command(
repository={'path': 'repo'},
config={'umask': '077'},
local_borg_version='1.2.3',
repo_delete_arguments=flexmock(list_archives=False, force=0),
global_arguments=flexmock(dry_run=False, log_json=False),
local_path='borg',
remote_path=None,
)
assert command == ('borg', 'repo-delete', '--umask', '077', 'repo')
def test_make_repo_delete_command_includes_log_json():
flexmock(module.borgmatic.borg.feature).should_receive('available').and_return(True)
flexmock(module.borgmatic.borg.flags).should_receive('make_flags').and_return(())
+84
View File
@@ -11,6 +11,9 @@ def test_display_repository_info_calls_borg_with_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -49,6 +52,9 @@ def test_display_repository_info_without_borg_features_calls_borg_with_info_sub_
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(False)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -82,6 +88,9 @@ def test_display_repository_info_with_log_info_calls_borg_with_info_flag():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -120,6 +129,9 @@ def test_display_repository_info_with_log_info_and_json_suppresses_most_borg_out
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -153,6 +165,9 @@ def test_display_repository_info_with_log_debug_calls_borg_with_debug_flag():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -192,6 +207,9 @@ def test_display_repository_info_with_log_debug_and_json_suppresses_most_borg_ou
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -225,6 +243,9 @@ def test_display_repository_info_with_json_calls_borg_with_json_flag():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -257,6 +278,9 @@ def test_display_repository_info_with_local_path_calls_borg_via_local_path():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -296,6 +320,9 @@ def test_display_repository_info_with_exit_codes_calls_borg_using_them():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -335,6 +362,9 @@ def test_display_repository_info_with_remote_path_calls_borg_with_remote_path_fl
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -370,10 +400,58 @@ def test_display_repository_info_with_remote_path_calls_borg_with_remote_path_fl
)
def test_display_repository_info_with_umask_calls_borg_with_umask_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
'repo',
)
)
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'repo-info', '--umask', '077', '--json', '--repo', 'repo'),
extra_environment=None,
working_directory=None,
borg_local_path='borg',
borg_exit_codes=None,
).and_return('[]')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
('borg', 'repo-info', '--umask', '077', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
extra_environment=None,
working_directory=None,
borg_local_path='borg',
borg_exit_codes=None,
)
module.display_repository_info(
repository_path='repo',
config={'umask': '077'},
local_borg_version='2.3.4',
repo_info_arguments=flexmock(json=False),
global_arguments=flexmock(log_json=False),
remote_path=None,
)
def test_display_repository_info_with_log_json_calls_borg_with_log_json_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_flags').with_args('log-json', True).and_return(
('--log-json',)
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -413,6 +491,9 @@ def test_display_repository_info_with_lock_wait_calls_borg_with_lock_wait_flags(
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
config = {'lock_wait': 5}
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', str(value)) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
@@ -451,6 +532,9 @@ def test_display_repository_info_calls_borg_with_working_directory():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.feature).should_receive('available').and_return(True)
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(
(
'--repo',
+50 -3
View File
@@ -180,6 +180,30 @@ def test_resolve_archive_name_with_remote_path_calls_borg_with_remote_path_flags
)
def test_resolve_archive_name_with_umask_calls_borg_with_umask_flags():
expected_archive = 'archive-name'
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'list', '--umask', '077') + BORG_LIST_LATEST_ARGUMENTS,
extra_environment=None,
working_directory=None,
borg_local_path='borg',
borg_exit_codes=None,
).and_return(expected_archive + '\n')
assert (
module.resolve_archive_name(
'repo',
'latest',
config={'umask': '077'},
local_borg_version='1.2.3',
global_arguments=flexmock(log_json=False),
)
== expected_archive
)
def test_resolve_archive_name_without_archives_raises():
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -455,9 +479,9 @@ def test_make_repo_list_command_includes_local_path():
def test_make_repo_list_command_includes_remote_path():
flexmock(module.flags).should_receive('make_flags').and_return(
('--remote-path', 'borg2')
).and_return(()).and_return(())
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '1.2.3'
).and_return(())
@@ -478,6 +502,29 @@ def test_make_repo_list_command_includes_remote_path():
assert command == ('borg', 'list', '--remote-path', 'borg2', 'repo')
def test_make_repo_list_command_includes_umask():
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_match_archives_flags').with_args(
None, None, '1.2.3'
).and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
command = module.make_repo_list_command(
repository_path='repo',
config={'umask': '077'},
local_borg_version='1.2.3',
repo_list_arguments=flexmock(
archive=None, paths=None, json=False, prefix=None, match_archives=None
),
global_arguments=flexmock(log_json=False),
)
assert command == ('borg', 'list', '--umask', '077', 'repo')
def test_make_repo_list_command_transforms_prefix_into_match_archives():
flexmock(module.flags).should_receive('make_flags').and_return(()).and_return(()).and_return(
('--match-archives', 'sh:foo*')
+33
View File
@@ -334,6 +334,39 @@ def test_transfer_archives_with_remote_path_calls_borg_with_remote_path_flags():
)
def test_transfer_archives_with_umask_calls_borg_with_umask_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module.flags).should_receive('make_flags').replace_with(
lambda name, value: (f'--{name}', value) if value else ()
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_flags_from_arguments').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command').with_args(
('borg', 'transfer', '--umask', '077', '--repo', 'repo'),
output_log_level=module.borgmatic.logger.ANSWER,
output_file=None,
extra_environment=None,
working_directory=None,
borg_local_path='borg',
borg_exit_codes=None,
)
module.transfer_archives(
dry_run=False,
repository_path='repo',
config={'umask': '077'},
local_borg_version='2.3.4',
transfer_arguments=flexmock(
archive=None, progress=None, match_archives=None, source_repository=None
),
global_arguments=flexmock(log_json=False),
)
def test_transfer_archives_with_log_json_calls_borg_with_log_json_flags():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
-2
View File
@@ -1397,7 +1397,6 @@ def test_collect_configuration_run_summary_executes_hooks_for_create():
def test_collect_configuration_run_summary_logs_info_for_success_with_extract():
flexmock(module.validate).should_receive('guard_single_repository_selected')
flexmock(module.validate).should_receive('guard_configuration_contains_repository')
flexmock(module).should_receive('run_configuration').and_return([])
arguments = {'extract': flexmock(repository='repo')}
@@ -1429,7 +1428,6 @@ def test_collect_configuration_run_summary_logs_extract_with_repository_error():
def test_collect_configuration_run_summary_logs_info_for_success_with_mount():
flexmock(module.validate).should_receive('guard_single_repository_selected')
flexmock(module.validate).should_receive('guard_configuration_contains_repository')
flexmock(module).should_receive('run_configuration').and_return([])
arguments = {'mount': flexmock(repository='repo')}
+135 -21
View File
@@ -1,3 +1,4 @@
import pytest
from flexmock import flexmock
from borgmatic.config import paths as module
@@ -33,36 +34,141 @@ def test_get_borgmatic_source_directory_without_config_option_uses_default():
assert module.get_borgmatic_source_directory({}) == '~/.borgmatic'
def test_get_borgmatic_runtime_directory_uses_config_option():
def test_runtime_directory_uses_config_option():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os).should_receive('makedirs')
config = {'user_runtime_directory': '/run', 'borgmatic_source_directory': '/nope'}
assert (
module.get_borgmatic_runtime_directory(
{'user_runtime_directory': '/tmp', 'borgmatic_source_directory': '/nope'}
)
== '/tmp/./borgmatic'
with module.Runtime_directory(config, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/run/./borgmatic'
def test_runtime_directory_uses_config_option_without_adding_duplicate_borgmatic_subdirectory():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os).should_receive('makedirs')
config = {'user_runtime_directory': '/run/borgmatic', 'borgmatic_source_directory': '/nope'}
with module.Runtime_directory(config, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/run/./borgmatic'
def test_runtime_directory_falls_back_to_xdg_runtime_dir():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').with_args('XDG_RUNTIME_DIR').and_return(
'/run'
)
flexmock(module.os).should_receive('makedirs')
with module.Runtime_directory({}, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/run/./borgmatic'
def test_get_borgmatic_runtime_directory_falls_back_to_environment_variable():
def test_runtime_directory_falls_back_to_xdg_runtime_dir_without_adding_duplicate_borgmatic_subdirectory():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').with_args(
'XDG_RUNTIME_DIR', object
).and_return('/tmp')
flexmock(module.os.environ).should_receive('get').with_args('XDG_RUNTIME_DIR').and_return(
'/run/borgmatic'
)
flexmock(module.os).should_receive('makedirs')
assert module.get_borgmatic_runtime_directory({}) == '/tmp/./borgmatic'
with module.Runtime_directory({}, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/run/./borgmatic'
def test_get_borgmatic_runtime_directory_defaults_to_hard_coded_path():
def test_runtime_directory_falls_back_to_runtime_directory():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').and_return('/run/user/0')
flexmock(module.os).should_receive('getuid').and_return(0)
flexmock(module.os.environ).should_receive('get').with_args('XDG_RUNTIME_DIR').and_return(None)
flexmock(module.os.environ).should_receive('get').with_args('RUNTIME_DIRECTORY').and_return(
'/run'
)
flexmock(module.os).should_receive('makedirs')
assert module.get_borgmatic_runtime_directory({}) == '/run/user/0/./borgmatic'
with module.Runtime_directory({}, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/run/./borgmatic'
def test_runtime_directory_falls_back_to_runtime_directory_without_adding_duplicate_borgmatic_subdirectory():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').with_args('XDG_RUNTIME_DIR').and_return(None)
flexmock(module.os.environ).should_receive('get').with_args('RUNTIME_DIRECTORY').and_return(
'/run/borgmatic'
)
flexmock(module.os).should_receive('makedirs')
with module.Runtime_directory({}, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/run/./borgmatic'
def test_runtime_directory_falls_back_to_tmpdir_and_adds_temporary_subdirectory_that_get_cleaned_up():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').with_args('XDG_RUNTIME_DIR').and_return(None)
flexmock(module.os.environ).should_receive('get').with_args('RUNTIME_DIRECTORY').and_return(
None
)
flexmock(module.os.environ).should_receive('get').with_args('TMPDIR').and_return('/run')
temporary_directory = flexmock(name='/run/borgmatic-1234')
temporary_directory.should_receive('cleanup').once()
flexmock(module.tempfile).should_receive('TemporaryDirectory').with_args(
prefix='borgmatic-', dir='/run'
).and_return(temporary_directory)
flexmock(module.os).should_receive('makedirs')
with module.Runtime_directory({}, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/run/borgmatic-1234/./borgmatic'
def test_runtime_directory_falls_back_to_temp_and_adds_temporary_subdirectory_that_get_cleaned_up():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').with_args('XDG_RUNTIME_DIR').and_return(None)
flexmock(module.os.environ).should_receive('get').with_args('RUNTIME_DIRECTORY').and_return(
None
)
flexmock(module.os.environ).should_receive('get').with_args('TMPDIR').and_return(None)
flexmock(module.os.environ).should_receive('get').with_args('TEMP').and_return('/run')
temporary_directory = flexmock(name='/run/borgmatic-1234')
temporary_directory.should_receive('cleanup').once()
flexmock(module.tempfile).should_receive('TemporaryDirectory').with_args(
prefix='borgmatic-', dir='/run'
).and_return(temporary_directory)
flexmock(module.os).should_receive('makedirs')
with module.Runtime_directory({}, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/run/borgmatic-1234/./borgmatic'
def test_runtime_directory_falls_back_to_hard_coded_tmp_path_and_adds_temporary_subdirectory_that_get_cleaned_up():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').with_args('XDG_RUNTIME_DIR').and_return(None)
flexmock(module.os.environ).should_receive('get').with_args('RUNTIME_DIRECTORY').and_return(
None
)
flexmock(module.os.environ).should_receive('get').with_args('TMPDIR').and_return(None)
flexmock(module.os.environ).should_receive('get').with_args('TEMP').and_return(None)
temporary_directory = flexmock(name='/tmp/borgmatic-1234')
temporary_directory.should_receive('cleanup').once()
flexmock(module.tempfile).should_receive('TemporaryDirectory').with_args(
prefix='borgmatic-', dir='/tmp'
).and_return(temporary_directory)
flexmock(module.os).should_receive('makedirs')
with module.Runtime_directory({}, 'prefix') as borgmatic_runtime_directory:
assert borgmatic_runtime_directory == '/tmp/borgmatic-1234/./borgmatic'
@pytest.mark.parametrize(
'borgmatic_runtime_directory,expected_glob',
(
('/foo/bar/baz/./borgmatic', 'foo/bar/baz/borgmatic'),
('/foo/borgmatic/baz/./borgmatic', 'foo/borgmatic/baz/borgmatic'),
('/foo/borgmatic-jti8idds/./borgmatic', 'foo/*/borgmatic'),
),
)
def test_make_runtime_directory_glob(borgmatic_runtime_directory, expected_glob):
assert module.make_runtime_directory_glob(borgmatic_runtime_directory) == expected_glob
def test_get_borgmatic_state_directory_uses_config_option():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').never()
assert (
module.get_borgmatic_state_directory(
@@ -72,16 +178,24 @@ def test_get_borgmatic_state_directory_uses_config_option():
)
def test_get_borgmatic_state_directory_falls_back_to_environment_variable():
def test_get_borgmatic_state_directory_falls_back_to_xdg_state_home():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').with_args(
'XDG_STATE_HOME', object
).and_return('/tmp')
flexmock(module.os.environ).should_receive('get').with_args('XDG_STATE_HOME').and_return('/tmp')
assert module.get_borgmatic_state_directory({}) == '/tmp/borgmatic'
def test_get_borgmatic_state_directory_falls_back_to_state_directory():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').with_args('XDG_STATE_HOME').and_return(None)
flexmock(module.os.environ).should_receive('get').with_args('STATE_DIRECTORY').and_return(
'/tmp'
)
assert module.get_borgmatic_state_directory({}) == '/tmp/borgmatic'
def test_get_borgmatic_state_directory_defaults_to_hard_coded_path():
flexmock(module).should_receive('expand_user_in_path').replace_with(lambda path: path)
flexmock(module.os.environ).should_receive('get').and_return('/root/.local/state')
assert module.get_borgmatic_state_directory({}) == '/root/.local/state/borgmatic'
flexmock(module.os.environ).should_receive('get').and_return(None)
assert module.get_borgmatic_state_directory({}) == '~/.local/state/borgmatic'
-29
View File
@@ -210,32 +210,3 @@ def test_guard_configuration_contains_repository_errors_when_repository_does_not
repository='nope',
configurations={'config.yaml': {'repositories': ['repo', 'repo2']}},
)
def test_guard_single_repository_selected_raises_when_multiple_repositories_configured_and_none_selected():
with pytest.raises(ValueError):
module.guard_single_repository_selected(
repository=None,
configurations={'config.yaml': {'repositories': ['repo', 'repo2']}},
)
def test_guard_single_repository_selected_does_not_raise_when_single_repository_configured_and_none_selected():
module.guard_single_repository_selected(
repository=None,
configurations={'config.yaml': {'repositories': ['repo']}},
)
def test_guard_single_repository_selected_does_not_raise_when_no_repositories_configured_and_one_selected():
module.guard_single_repository_selected(
repository='repo',
configurations={'config.yaml': {'repositories': []}},
)
def test_guard_single_repository_selected_does_not_raise_when_repositories_configured_and_one_selected():
module.guard_single_repository_selected(
repository='repo',
configurations={'config.yaml': {'repositories': ['repo', 'repo2']}},
)
+35 -6
View File
@@ -73,7 +73,12 @@ def test_dump_data_sources_dumps_each_database():
dry_run_label=object,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_dump_data_sources_dumps_with_password():
@@ -94,7 +99,9 @@ def test_dump_data_sources_dumps_with_password():
dry_run_label=object,
).and_return(process).once()
assert module.dump_data_sources([database], {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
[database], {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_dumps_all_databases_at_once():
@@ -112,7 +119,9 @@ def test_dump_data_sources_dumps_all_databases_at_once():
dry_run_label=object,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_dumps_all_databases_separately_when_format_configured():
@@ -132,7 +141,12 @@ def test_dump_data_sources_dumps_all_databases_separately_when_format_configured
dry_run_label=object,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_database_names_to_dump_runs_mariadb_with_list_options():
@@ -434,7 +448,9 @@ def test_dump_data_sources_errors_for_missing_all_databases():
flexmock(module).should_receive('database_names_to_dump').and_return(())
with pytest.raises(ValueError):
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False)
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
def test_dump_data_sources_does_not_error_for_missing_all_databases_with_dry_run():
@@ -445,7 +461,12 @@ def test_dump_data_sources_does_not_error_for_missing_all_databases_with_dry_run
)
flexmock(module).should_receive('database_names_to_dump').and_return(())
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=True) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=True
)
== []
)
def test_restore_data_source_dump_runs_mariadb_to_restore():
@@ -473,6 +494,7 @@ def test_restore_data_source_dump_runs_mariadb_to_restore():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -501,6 +523,7 @@ def test_restore_data_source_dump_runs_mariadb_with_options():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -531,6 +554,7 @@ def test_restore_data_source_dump_runs_non_default_mariadb_with_options():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -568,6 +592,7 @@ def test_restore_data_source_dump_runs_mariadb_with_hostname_and_port():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -596,6 +621,7 @@ def test_restore_data_source_dump_runs_mariadb_with_username_and_password():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -645,6 +671,7 @@ def test_restore_data_source_dump_with_connection_params_uses_connection_params_
'username': 'cliusername',
'password': 'clipassword',
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -696,6 +723,7 @@ def test_restore_data_source_dump_without_connection_params_uses_restore_params_
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -717,4 +745,5 @@ def test_restore_data_source_dump_with_dry_run_skips_restore():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
+40 -7
View File
@@ -41,7 +41,12 @@ def test_dump_data_sources_runs_mongodump_for_each_database():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_dump_data_sources_with_dry_run_skips_mongodump():
@@ -53,7 +58,12 @@ def test_dump_data_sources_with_dry_run_skips_mongodump():
flexmock(module.dump).should_receive('create_named_pipe_for_dump').never()
flexmock(module).should_receive('execute_command').never()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=True) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=True
)
== []
)
def test_dump_data_sources_runs_mongodump_with_hostname_and_port():
@@ -82,7 +92,9 @@ def test_dump_data_sources_runs_mongodump_with_hostname_and_port():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_runs_mongodump_with_username_and_password():
@@ -120,7 +132,9 @@ def test_dump_data_sources_runs_mongodump_with_username_and_password():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_runs_mongodump_with_directory_format():
@@ -137,7 +151,12 @@ def test_dump_data_sources_runs_mongodump_with_directory_format():
shell=True,
).and_return(flexmock()).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== []
)
def test_dump_data_sources_runs_mongodump_with_options():
@@ -163,7 +182,9 @@ def test_dump_data_sources_runs_mongodump_with_options():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_runs_mongodumpall_for_all_databases():
@@ -181,7 +202,9 @@ def test_dump_data_sources_runs_mongodumpall_for_all_databases():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_build_dump_command_with_username_injection_attack_gets_escaped():
@@ -218,6 +241,7 @@ def test_restore_data_source_dump_runs_mongorestore():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -257,6 +281,7 @@ def test_restore_data_source_dump_runs_mongorestore_with_hostname_and_port():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -304,6 +329,7 @@ def test_restore_data_source_dump_runs_mongorestore_with_username_and_password()
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -359,6 +385,7 @@ def test_restore_data_source_dump_with_connection_params_uses_connection_params_
'username': 'cliusername',
'password': 'clipassword',
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -414,6 +441,7 @@ def test_restore_data_source_dump_without_connection_params_uses_restore_params_
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -443,6 +471,7 @@ def test_restore_data_source_dump_runs_mongorestore_with_options():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -480,6 +509,7 @@ def test_restore_databases_dump_runs_mongorestore_with_schemas():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -509,6 +539,7 @@ def test_restore_data_source_dump_runs_psql_for_all_database_dump():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -532,6 +563,7 @@ def test_restore_data_source_dump_with_dry_run_skips_restore():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -560,4 +592,5 @@ def test_restore_data_source_dump_without_extract_process_restores_from_disk():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
+35 -6
View File
@@ -73,7 +73,12 @@ def test_dump_data_sources_dumps_each_database():
dry_run_label=object,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_dump_data_sources_dumps_with_password():
@@ -94,7 +99,9 @@ def test_dump_data_sources_dumps_with_password():
dry_run_label=object,
).and_return(process).once()
assert module.dump_data_sources([database], {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
[database], {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_dumps_all_databases_at_once():
@@ -112,7 +119,9 @@ def test_dump_data_sources_dumps_all_databases_at_once():
dry_run_label=object,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_dumps_all_databases_separately_when_format_configured():
@@ -132,7 +141,12 @@ def test_dump_data_sources_dumps_all_databases_separately_when_format_configured
dry_run_label=object,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_database_names_to_dump_runs_mysql_with_list_options():
@@ -432,7 +446,9 @@ def test_dump_data_sources_errors_for_missing_all_databases():
flexmock(module).should_receive('database_names_to_dump').and_return(())
with pytest.raises(ValueError):
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False)
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
def test_dump_data_sources_does_not_error_for_missing_all_databases_with_dry_run():
@@ -443,7 +459,12 @@ def test_dump_data_sources_does_not_error_for_missing_all_databases_with_dry_run
)
flexmock(module).should_receive('database_names_to_dump').and_return(())
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=True) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=True
)
== []
)
def test_restore_data_source_dump_runs_mysql_to_restore():
@@ -471,6 +492,7 @@ def test_restore_data_source_dump_runs_mysql_to_restore():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -499,6 +521,7 @@ def test_restore_data_source_dump_runs_mysql_with_options():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -527,6 +550,7 @@ def test_restore_data_source_dump_runs_non_default_mysql_with_options():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -564,6 +588,7 @@ def test_restore_data_source_dump_runs_mysql_with_hostname_and_port():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -592,6 +617,7 @@ def test_restore_data_source_dump_runs_mysql_with_username_and_password():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -641,6 +667,7 @@ def test_restore_data_source_dump_with_connection_params_uses_connection_params_
'username': 'cliusername',
'password': 'clipassword',
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -692,6 +719,7 @@ def test_restore_data_source_dump_without_connection_params_uses_restore_params_
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -713,4 +741,5 @@ def test_restore_data_source_dump_with_dry_run_skips_restore():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
+60 -12
View File
@@ -251,7 +251,12 @@ def test_dump_data_sources_runs_pg_dump_for_each_database():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_dump_data_sources_raises_when_no_database_names_to_dump():
@@ -261,7 +266,9 @@ def test_dump_data_sources_raises_when_no_database_names_to_dump():
flexmock(module).should_receive('database_names_to_dump').and_return(())
with pytest.raises(ValueError):
module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False)
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
def test_dump_data_sources_does_not_raise_when_no_database_names_to_dump():
@@ -270,7 +277,9 @@ def test_dump_data_sources_does_not_raise_when_no_database_names_to_dump():
flexmock(module).should_receive('make_dump_path').and_return('')
flexmock(module).should_receive('database_names_to_dump').and_return(())
module.dump_data_sources(databases, {}, 'test.yaml', dry_run=True) == []
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=True
) == []
def test_dump_data_sources_with_duplicate_dump_skips_pg_dump():
@@ -287,7 +296,12 @@ def test_dump_data_sources_with_duplicate_dump_skips_pg_dump():
flexmock(module.dump).should_receive('create_named_pipe_for_dump').never()
flexmock(module).should_receive('execute_command').never()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== []
)
def test_dump_data_sources_with_dry_run_skips_pg_dump():
@@ -304,7 +318,12 @@ def test_dump_data_sources_with_dry_run_skips_pg_dump():
flexmock(module.dump).should_receive('create_named_pipe_for_dump').never()
flexmock(module).should_receive('execute_command').never()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=True) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=True
)
== []
)
def test_dump_data_sources_runs_pg_dump_with_hostname_and_port():
@@ -340,7 +359,9 @@ def test_dump_data_sources_runs_pg_dump_with_hostname_and_port():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_runs_pg_dump_with_username_and_password():
@@ -376,7 +397,9 @@ def test_dump_data_sources_runs_pg_dump_with_username_and_password():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_with_username_injection_attack_gets_escaped():
@@ -412,7 +435,9 @@ def test_dump_data_sources_with_username_injection_attack_gets_escaped():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_runs_pg_dump_with_directory_format():
@@ -443,7 +468,12 @@ def test_dump_data_sources_runs_pg_dump_with_directory_format():
extra_environment={'PGSSLMODE': 'disable'},
).and_return(flexmock()).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== []
)
def test_dump_data_sources_runs_pg_dump_with_options():
@@ -476,7 +506,9 @@ def test_dump_data_sources_runs_pg_dump_with_options():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_runs_pg_dumpall_for_all_databases():
@@ -498,7 +530,9 @@ def test_dump_data_sources_runs_pg_dumpall_for_all_databases():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_dump_data_sources_runs_non_default_pg_dump():
@@ -532,7 +566,9 @@ def test_dump_data_sources_runs_non_default_pg_dump():
run_to_completion=False,
).and_return(process).once()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == [process]
assert module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
) == [process]
def test_restore_data_source_dump_runs_pg_restore():
@@ -584,6 +620,7 @@ def test_restore_data_source_dump_runs_pg_restore():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -646,6 +683,7 @@ def test_restore_data_source_dump_runs_pg_restore_with_hostname_and_port():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -706,6 +744,7 @@ def test_restore_data_source_dump_runs_pg_restore_with_username_and_password():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -785,6 +824,7 @@ def test_restore_data_source_dump_with_connection_params_uses_connection_params_
'username': 'cliusername',
'password': 'clipassword',
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -864,6 +904,7 @@ def test_restore_data_source_dump_without_connection_params_uses_restore_params_
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -925,6 +966,7 @@ def test_restore_data_source_dump_runs_pg_restore_with_options():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -964,6 +1006,7 @@ def test_restore_data_source_dump_runs_psql_for_all_database_dump():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1008,6 +1051,7 @@ def test_restore_data_source_dump_runs_psql_for_plain_database_dump():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1077,6 +1121,7 @@ def test_restore_data_source_dump_runs_non_default_pg_restore_and_psql():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1101,6 +1146,7 @@ def test_restore_data_source_dump_with_dry_run_skips_restore():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1153,6 +1199,7 @@ def test_restore_data_source_dump_without_extract_process_restores_from_disk():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -1209,4 +1256,5 @@ def test_restore_data_source_dump_with_schemas_restores_schemas():
'username': None,
'password': None,
},
borgmatic_runtime_directory='/run/borgmatic',
)
+522
View File
@@ -0,0 +1,522 @@
import pytest
from flexmock import flexmock
import borgmatic.hooks.monitor
from borgmatic.hooks import pushover as module
def test_ping_monitor_config_with_minimum_config_fail_state_backup_successfully_send_to_pushover():
'''
This test should be the minimum working configuration. The "message"
should be auto populated with the default value which is the state name.
'''
hook_config = {'token': 'ksdjfwoweijfvwoeifvjmwghagy92', 'user': '983hfe0of902lkjfa2amanfgui'}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'fail',
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_config_with_minimum_config_start_state_backup_not_send_to_pushover_exit_early():
'''
This test should exit early since the hook config does not specify the
'start' state. Only the 'fail' state is enabled by default.
'''
hook_config = {'token': 'ksdjfwoweijfvwoeifvjmwghagy92', 'user': '983hfe0of902lkjfa2amanfgui'}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').never()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_start_state_backup_default_message_successfully_send_to_pushover():
'''
This test should send a notification to Pushover on backup start
since the state has been configured. It should default to sending
the name of the state as the 'message' since it is not
explicitly declared in the state config.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'start',
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_start_state_backup_custom_message_successfully_send_to_pushover():
'''
This test should send a notification to Pushover on backup start
since the state has been configured. It should send a custom
'message' since it is explicitly declared in the state config.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
'start': {'message': 'custom start message'},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'custom start message',
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_start_state_backup_default_message_with_priority_emergency_uses_expire_and_retry_defaults():
'''
This simulates priority level 2 being set but expiry and retry are
not declared. This should set retry and expiry to their defaults.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
'start': {'priority': 2},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'start',
'priority': 2,
'retry': 30,
'expire': 600,
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_start_state_backup_default_message_with_priority_emergency_declared_with_expire_no_retry_success():
'''
This simulates priority level 2 and expiry being set but retry is
not declared. This should set retry to the default.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
'start': {'priority': 2, 'expire': 600},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'start',
'priority': 2,
'retry': 30,
'expire': 600,
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_start_state_backup_default_message_with_priority_emergency_declared_no_expire_with_retry_success():
'''
This simulates priority level 2 and retry being set but expire is
not declared. This should set expire to the default.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
'start': {'priority': 2, 'retry': 30},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'start',
'priority': 2,
'retry': 30,
'expire': 600,
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_start_state_backup_default_message_with_priority_high_declared_expire_and_retry_raises():
'''
This simulates priority level 1, retry and expiry being set. Since expire
and retry are only used for priority level 2, they should not be included
in the request sent to Pushover. This test verifies that a ValueError is
raised.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
'start': {'priority': 1, 'expire': 30, 'retry': 30},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').never()
with pytest.raises(ValueError):
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_start_state_backup_based_on_documentation_advanced_example_success():
'''
Here is a test of what is provided in the monitor-your-backups.md file
as an 'advanced example'. This test runs the start state.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
'start': {
'message': 'Backup <b>Started</b>',
'priority': -2,
'title': 'Backup Started',
'html': 1,
'ttl': 10,
},
'fail': {
'message': 'Backup <font color="#ff6961">Failed</font>',
'priority': 2,
'expire': 600,
'retry': 30,
'device': 'pixel8',
'title': 'Backup Failed',
'html': 1,
'sound': 'siren',
'url': 'https://ticketing-system.example.com/login',
'url_title': 'Login to ticketing system',
},
'finish': {
'message': 'Backup <font color="#77dd77">Finished</font>',
'priority': 0,
'title': 'Backup Finished',
'html': 1,
'ttl': 60,
'url': 'https://ticketing-system.example.com/login',
'url_title': 'Login to ticketing system',
},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'Backup <b>Started</b>',
'priority': -2,
'title': 'Backup Started',
'html': 1,
'ttl': 10,
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_fail_state_backup_based_on_documentation_advanced_example_success():
'''
Here is a test of what is provided in the monitor-your-backups.md file
as an 'advanced example'. This test runs the fail state.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
'start': {
'message': 'Backup <b>Started</b>',
'priority': -2,
'title': 'Backup Started',
'html': 1,
'ttl': 10,
},
'fail': {
'message': 'Backup <font color="#ff6961">Failed</font>',
'priority': 2,
'expire': 600,
'retry': 30,
'device': 'pixel8',
'title': 'Backup Failed',
'html': 1,
'sound': 'siren',
'url': 'https://ticketing-system.example.com/login',
'url_title': 'Login to ticketing system',
},
'finish': {
'message': 'Backup <font color="#77dd77">Finished</font>',
'priority': 0,
'title': 'Backup Finished',
'html': 1,
'ttl': 60,
'url': 'https://ticketing-system.example.com/login',
'url_title': 'Login to ticketing system',
},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'Backup <font color="#ff6961">Failed</font>',
'priority': 2,
'expire': 600,
'retry': 30,
'device': 'pixel8',
'title': 'Backup Failed',
'html': 1,
'sound': 'siren',
'url': 'https://ticketing-system.example.com/login',
'url_title': 'Login to ticketing system',
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_finish_state_backup_based_on_documentation_advanced_example_success():
'''
Here is a test of what is provided in the monitor-your-backups.md file
as an 'advanced example'. This test runs the finish state.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'states': {'start', 'fail', 'finish'},
'start': {
'message': 'Backup <b>Started</b>',
'priority': -2,
'title': 'Backup Started',
'html': 1,
'ttl': 10,
},
'fail': {
'message': 'Backup <font color="#ff6961">Failed</font>',
'priority': 2,
'expire': 600,
'retry': 30,
'device': 'pixel8',
'title': 'Backup Failed',
'html': 1,
'sound': 'siren',
'url': 'https://ticketing-system.example.com/login',
'url_title': 'Login to ticketing system',
},
'finish': {
'message': 'Backup <font color="#77dd77">Finished</font>',
'priority': 0,
'title': 'Backup Finished',
'html': 1,
'ttl': 60,
'url': 'https://ticketing-system.example.com/login',
'url_title': 'Login to ticketing system',
},
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'Backup <font color="#77dd77">Finished</font>',
'priority': 0,
'title': 'Backup Finished',
'html': 1,
'ttl': 60,
'url': 'https://ticketing-system.example.com/login',
'url_title': 'Login to ticketing system',
},
).and_return(flexmock(ok=True)).once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.FINISH,
monitoring_log_level=1,
dry_run=False,
)
def test_ping_monitor_config_with_minimum_config_fail_state_backup_successfully_send_to_pushover_dryrun():
'''
This test should be the minimum working configuration. The "message"
should be auto populated with the default value which is the state name.
'''
hook_config = {'token': 'ksdjfwoweijfvwoeifvjmwghagy92', 'user': '983hfe0of902lkjfa2amanfgui'}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').and_return(flexmock(ok=True)).never()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=True,
)
def test_ping_monitor_config_incorrect_state_exit_early():
'''
This test should exit early since the start state is not declared in the configuration.
'''
hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
}
flexmock(module.logger).should_receive('warning').never()
flexmock(module.requests).should_receive('post').and_return(flexmock(ok=True)).never()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.START,
monitoring_log_level=1,
dry_run=True,
)
def test_ping_monitor_push_post_error_exits_early():
'''
This test simulates the Pushover servers not responding with a 200 OK. We
should raise for status and warn then exit.
'''
hook_config = hook_config = {
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
}
push_response = flexmock(ok=False)
push_response.should_receive('raise_for_status').and_raise(
module.requests.ConnectionError
).once()
flexmock(module.requests).should_receive('post').with_args(
'https://api.pushover.net/1/messages.json',
headers={'Content-type': 'application/x-www-form-urlencoded'},
data={
'token': 'ksdjfwoweijfvwoeifvjmwghagy92',
'user': '983hfe0of902lkjfa2amanfgui',
'message': 'fail',
},
).and_return(push_response).once()
flexmock(module.logger).should_receive('warning').once()
module.ping_monitor(
hook_config,
{},
'config.yaml',
borgmatic.hooks.monitor.State.FAIL,
monitoring_log_level=1,
dry_run=False,
)
+53 -19
View File
@@ -18,15 +18,20 @@ def test_use_streaming_false_for_no_databases():
def test_dump_data_sources_logs_and_skips_if_dump_already_exists():
databases = [{'path': '/path/to/database', 'name': 'database'}]
flexmock(module).should_receive('make_dump_path').and_return('/path/to/dump')
flexmock(module).should_receive('make_dump_path').and_return('/run/borgmatic')
flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return(
'/path/to/dump/database'
'/run/borgmatic/database'
)
flexmock(module.os.path).should_receive('exists').and_return(True)
flexmock(module.dump).should_receive('create_named_pipe_for_dump').never()
flexmock(module).should_receive('execute_command').never()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== []
)
def test_dump_data_sources_dumps_each_database():
@@ -36,9 +41,9 @@ def test_dump_data_sources_dumps_each_database():
]
processes = [flexmock(), flexmock()]
flexmock(module).should_receive('make_dump_path').and_return('/path/to/dump')
flexmock(module).should_receive('make_dump_path').and_return('/run/borgmatic')
flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return(
'/path/to/dump/database'
'/run/borgmatic/database'
)
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.dump).should_receive('create_named_pipe_for_dump')
@@ -46,7 +51,12 @@ def test_dump_data_sources_dumps_each_database():
processes[1]
)
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_dump_data_sources_with_path_injection_attack_gets_escaped():
@@ -55,9 +65,9 @@ def test_dump_data_sources_with_path_injection_attack_gets_escaped():
]
processes = [flexmock()]
flexmock(module).should_receive('make_dump_path').and_return('/path/to/dump')
flexmock(module).should_receive('make_dump_path').and_return('/run/borgmatic')
flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return(
'/path/to/dump/database'
'/run/borgmatic/database'
)
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.dump).should_receive('create_named_pipe_for_dump')
@@ -67,13 +77,18 @@ def test_dump_data_sources_with_path_injection_attack_gets_escaped():
"'/path/to/database1; naughty-command'",
'.dump',
'>',
'/path/to/dump/database',
'/run/borgmatic/database',
),
shell=True,
run_to_completion=False,
).and_return(processes[0])
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_dump_data_sources_with_non_existent_path_warns_and_dumps_database():
@@ -82,16 +97,21 @@ def test_dump_data_sources_with_non_existent_path_warns_and_dumps_database():
]
processes = [flexmock()]
flexmock(module).should_receive('make_dump_path').and_return('/path/to/dump')
flexmock(module).should_receive('make_dump_path').and_return('/run/borgmatic')
flexmock(module.logger).should_receive('warning').once()
flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return(
'/path/to/dump/database'
'/run/borgmatic'
)
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.dump).should_receive('create_named_pipe_for_dump')
flexmock(module).should_receive('execute_command').and_return(processes[0])
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_dump_data_sources_with_name_all_warns_and_dumps_all_databases():
@@ -100,32 +120,42 @@ def test_dump_data_sources_with_name_all_warns_and_dumps_all_databases():
]
processes = [flexmock()]
flexmock(module).should_receive('make_dump_path').and_return('/path/to/dump')
flexmock(module).should_receive('make_dump_path').and_return('/run/borgmatic')
flexmock(module.logger).should_receive(
'warning'
).twice() # once for the name=all, once for the non-existent path
flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return(
'/path/to/dump/database'
'/run/borgmatic/database'
)
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.dump).should_receive('create_named_pipe_for_dump')
flexmock(module).should_receive('execute_command').and_return(processes[0])
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=False) == processes
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=False
)
== processes
)
def test_dump_data_sources_does_not_dump_if_dry_run():
databases = [{'path': '/path/to/database', 'name': 'database'}]
flexmock(module).should_receive('make_dump_path').and_return('/path/to/dump')
flexmock(module).should_receive('make_dump_path').and_return('/run/borgmatic')
flexmock(module.dump).should_receive('make_data_source_dump_filename').and_return(
'/path/to/dump/database'
'/run/borgmatic'
)
flexmock(module.os.path).should_receive('exists').and_return(False)
flexmock(module.dump).should_receive('create_named_pipe_for_dump').never()
flexmock(module).should_receive('execute_command').never()
assert module.dump_data_sources(databases, {}, 'test.yaml', dry_run=True) == []
assert (
module.dump_data_sources(
databases, {}, 'test.yaml', borgmatic_runtime_directory='/run/borgmatic', dry_run=True
)
== []
)
def test_restore_data_source_dump_restores_database():
@@ -152,6 +182,7 @@ def test_restore_data_source_dump_restores_database():
dry_run=False,
extract_process=extract_process,
connection_params={'restore_path': None},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -181,6 +212,7 @@ def test_restore_data_source_dump_with_connection_params_uses_connection_params_
dry_run=False,
extract_process=extract_process,
connection_params={'restore_path': 'cli/path/to/database'},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -210,6 +242,7 @@ def test_restore_data_source_dump_without_connection_params_uses_restore_params_
dry_run=False,
extract_process=extract_process,
connection_params={'restore_path': None},
borgmatic_runtime_directory='/run/borgmatic',
)
@@ -228,4 +261,5 @@ def test_restore_data_source_dump_does_not_restore_database_if_dry_run():
dry_run=True,
extract_process=extract_process,
connection_params={'restore_path': None},
borgmatic_runtime_directory='/run/borgmatic',
)