Compare commits

..
26 Commits
Author SHA1 Message Date
Dan Helfman 731e8d7c37 Bump version for release. 2026-04-05 19:57:22 -07:00
Dan Helfman 3bcf592d53 Fix the "spot" check, "extract" check, and all uses of the archive name "latest" to respect the "match_archives" and "archives_name_format" options (#1268). 2026-04-05 09:58:54 -07:00
Dan Helfman 9bb5791e9f Fix a traceback in the "spot" check with Borg 2. 2026-04-04 21:48:10 -07:00
Dan Helfman 8a60fb6398 Fix broken CI build (#1264). 2026-04-04 18:02:30 -07:00
Dan Helfman 9db2bb2b54 Improve performance of the "info" and "repo-list" actions by eliminating a second "borg info" call that supports a "no matching archives" warning (#1264). 2026-04-03 20:57:25 -07:00
Dan Helfman 87bfd6e97f Fix end-to-end tests. 2026-03-30 12:37:42 -07:00
Dan Helfman 52f9442377 More requirements updates. 2026-03-30 11:29:15 -07:00
Dan Helfman af841e0c89 Upgrade Pygments. 2026-03-30 11:17:27 -07:00
Dan Helfman ed8320c1bb Upgrade requests. 2026-03-30 11:16:21 -07:00
Dan Helfman 26b3a03721 Fix for the Loki monitoring hook not respecting the monitoring verbosity (#1257). 2026-03-30 11:13:55 -07:00
Dan Helfman 52234c47e6 Fix a regression in which running borgmatic with no arguments and no configuration files doesn't error as expected (#1286). 2026-03-22 20:10:46 -07:00
Dan Helfman b74b6aa18d Remove Nuitka's --show-modules flag, as it's mostly for debugging (#1266). 2026-03-21 13:26:43 -07:00
Dan Helfman 0380ecd8fb Add stand-alone borgmatic binary to documentation (#1266). 2026-03-21 12:53:33 -07:00
Dan Helfman 32b7d1a0f7 Add a stand-alone borgmatic binary to the release downloads to serve as another way to install borgmatic (#1266). 2026-03-21 11:10:14 -07:00
Dan Helfman da873c09f8 Update test requirements. 2026-03-21 11:05:04 -07:00
Dan Helfman d612d398e7 When Borg exits with a warning exit code, show a description of it, so you don't have to lookup the code. 2026-03-15 21:38:27 -07:00
Dan Helfman 1301bec702 Fix missing words in upgrade documentation. 2026-03-14 14:01:24 -07:00
Dan Helfman c7fc68a49a Fix typo. 2026-03-14 13:31:26 -07:00
Dan Helfman c66e29906e Attempt to fix documentation build. 2026-03-13 16:41:50 -07:00
Dan Helfman 475389a094 Move project tracker from Gitea to Forgejo. 2026-03-13 16:28:09 -07:00
Dan Helfman 9f59bf2827 Attempt to get workflow working. 2026-03-13 13:16:08 -07:00
Dan Helfman f4e9569297 Add Forgejo actions metadata. 2026-03-13 12:48:58 -07:00
Dan Helfman f43c2f7130 Add dedicated documenation installation link in Getting Started. 2026-03-08 12:29:14 -07:00
Dan Helfman 889b599d55 Switch the default borgmatic installation method from pipx to uv, as uv is faster and used for borgmatic development. 2026-03-07 21:13:53 -08:00
Dan Helfman 53791d4dc9 Fix key for documentation page. 2026-03-07 20:02:53 -08:00
Dan Helfman ee58adb4eb Split out borgmatic installation documentation to its own page, so it's easier to find. 2026-03-07 19:27:04 -08:00
41 changed files with 702 additions and 415 deletions
@@ -1,5 +1,5 @@
name: build
run-name: ${{ gitea.actor }} is building
run-name: ${{ forgejo.actor }} is building
on:
push:
branches: [main]
@@ -17,7 +17,7 @@ jobs:
docs:
needs: [test]
runs-on: host
if: gitea.event_name == 'push'
if: forgejo.event_name == 'push'
env:
IMAGE_NAME: projects.torsion.org/borgmatic-collective/borgmatic:docs
+1 -1
View File
@@ -9,7 +9,7 @@ Python project using setuptools.
Please do not use AI agents to modify this codebase. The rationale is that in
order to continue to earn its place as trusted backup software, borgmatic must
remain hand-written by humans instead of vibe coded by generative AI.
remain handwritten by humans instead of vibe coded by generative AI.
Additionally, if LLMs were to perform a sizeable chunk of the feature
development on this codebase, then human borgmatic developers would lose their
+22
View File
@@ -1,3 +1,25 @@
2.1.4
* #1266: Add a stand-alone borgmatic Linux binary to the release downloads to serve as another way
to install borgmatic. Consider this binary a beta feature.
* #1286: Fix a regression in which running borgmatic with no arguments and no configuration files
doesn't error as expected.
* #1257: Fix for the Loki monitoring hook not respecting the monitoring verbosity.
* #1264: Improve performance of the "info" and "repo-list" actions by eliminating a second "borg
info" call that supports a "no matching archives" warning. The warning still occurs; it's just
done now without the extra "borg info" call.
* #1268: Fix the "spot" check, "extract" check, and all uses of the archive name "latest" to
respect the "match_archives" and "archives_name_format" options. This means that borgmatic now
uses the "latest" archive that also matches those options instead of the latest archive overall.
* When Borg exits with a warning exit code, show a description of it, so you don't have to lookup
the code.
* Split out borgmatic installation documentation to its own page, so it's easier to find.
* Switch the default borgmatic installation method from pipx to uv, as uv is faster and used for
borgmatic development. If you'd like to switch, see the documentation for more information:
https://torsion.org/borgmatic/how-to/upgrade/
* Move the project tracker from Gitea to Forgejo.
* Fix a regression in which borgmatic didn't show an error message when run with no configuration.
* Fix a traceback in the "spot" check with Borg 2.
2.1.3
* #1175: Add a "files_changed" option for customizing Borg's file modification detection.
* #1175: Add a "msgpack_version_check" option to prevent Borg from validating msgpack's version.
+5 -4
View File
@@ -94,10 +94,11 @@ borgmatic is powered by [Borg Backup](https://www.borgbackup.org/).
## Getting started
Your first step is to [install and configure
borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/).
Your first steps are to
[install](https://torsion.org/borgmatic/how-to/install-borgmatic/) and
[configure borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/).
For additional documentation, check out the links above (left panel on wide screens)
For additional documentation, check out the links on the top/left
for <a href="https://torsion.org/borgmatic/#documentation">borgmatic how-to and
reference guides</a>.
@@ -183,4 +184,4 @@ Thanks to all borgmatic contributors! There are multiple ways to contribute to
this project, so the following includes those who have fixed bugs, contributed
features, *or* filed tickets.
{% include borgmatic/contributors.html %}
{% include borgmatic/contributors.html %}
+19
View File
@@ -0,0 +1,19 @@
.
apprise==1.9.9
attrs==26.1.0
certifi==2026.2.25
charset-normalizer==3.4.6
click==8.3.1
idna==3.11
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
markdown==3.10.2
oauthlib==3.3.1
packaging==26.0
pyyaml==6.0.3
referencing==0.37.0
requests==2.33.1
requests-oauthlib==2.0.0
rpds-py==0.30.0
ruamel-yaml==0.19.1
urllib3==2.6.3
+2 -2
View File
@@ -605,7 +605,7 @@ def compare_spot_check_hashes(
local_borg_version,
global_arguments,
list_paths=source_sample_paths_subset,
path_format='{xxh64}{path}{linktarget}',
path_format='{xxh64}{path}{linktarget}{target}',
local_path=local_path,
remote_path=remote_path,
):
@@ -614,7 +614,7 @@ def compare_spot_check_hashes(
# Borg can't get hashes of stored hard links. So if this is a hard link path (and not
# deemed as the "original" by Borg), then skip hashing of it.
if entry['linktarget']:
if entry.get('linktarget') or entry.get('target'):
source_hashes.pop(os.path.join('/', entry['path']), None)
continue
+4 -2
View File
@@ -391,7 +391,7 @@ def create_archive(
borg_exit_codes = config.get('borg_exit_codes')
if stream_processes:
return '\n'.join(
output = '\n'.join(
execute_command_with_processes(
create_flags + create_positional_arguments,
stream_processes,
@@ -403,9 +403,10 @@ def create_archive(
borg_exit_codes=borg_exit_codes,
)
)
return output if json else None
if output_log_level is None:
return '\n'.join(
output = '\n'.join(
execute_command_and_capture_output(
create_flags + create_positional_arguments,
working_directory=working_directory,
@@ -414,6 +415,7 @@ def create_archive(
borg_exit_codes=borg_exit_codes,
)
)
return output if json else None
execute_command(
create_flags + create_positional_arguments,
+10 -20
View File
@@ -1,5 +1,4 @@
import itertools
import json
import logging
import re
@@ -135,27 +134,18 @@ def make_match_archives_flags( # noqa: PLR0911
return ('--glob-archives', f'{derived_match_archives}')
def warn_for_aggressive_archive_flags(json_command, json_output):
def warn_for_aggressive_archive_flags(command, output_lines):
'''
Given a JSON archives command and the resulting JSON string output from running it, parse the
JSON and warn if the command used an archive flag but the output indicates zero archives were
found.
Given an archives command and the resulting output lines from running it, warn if the command
used an archive flag but the output indicates zero archives were found.
'''
archive_flags_used = {'--glob-archives', '--match-archives'}.intersection(set(json_command))
if not archive_flags_used:
return
try:
if len(json.loads(json_output)['archives']) == 0:
logger.warning('An archive filter was applied, but no matching archives were found.')
logger.warning(
'Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration.',
)
except json.JSONDecodeError as error:
logger.debug(f'Cannot parse JSON output from archive command: {error}')
except (TypeError, KeyError):
logger.debug('Cannot parse JSON output from archive command: No "archives" key found')
if {'--glob-archives', '--match-archives'}.intersection(set(command)) and len(
tuple(line for line in output_lines if not line.startswith('terminating with '))
) == 0:
logger.warning('An archive filter was applied, but no matching archives were found.')
logger.warning(
'Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration.',
)
def omit_flag(arguments, flag):
+16 -16
View File
@@ -5,7 +5,7 @@ import shlex
import borgmatic.config.paths
import borgmatic.logger
from borgmatic.borg import environment, feature, flags
from borgmatic.execute import execute_command, execute_command_and_capture_output
from borgmatic.execute import execute_command_and_capture_output
logger = logging.getLogger(__name__)
@@ -103,9 +103,21 @@ def display_archives_info(
borg_exit_codes = config.get('borg_exit_codes')
working_directory = borgmatic.config.paths.get_working_directory(config)
json_info = '\n'.join(
if info_arguments.json:
return '\n'.join(
execute_command_and_capture_output(
json_command,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
borg_exit_codes=borg_exit_codes,
)
)
output_lines = tuple(
execute_command_and_capture_output(
json_command,
main_command,
output_log_level=logging.ANSWER,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
@@ -113,18 +125,6 @@ def display_archives_info(
)
)
if info_arguments.json:
return json_info
flags.warn_for_aggressive_archive_flags(json_command, json_info)
execute_command(
main_command,
output_log_level=logging.ANSWER,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
borg_exit_codes=borg_exit_codes,
)
flags.warn_for_aggressive_archive_flags(main_command, output_lines)
return None
+23 -16
View File
@@ -6,7 +6,7 @@ import shlex
import borgmatic.config.paths
import borgmatic.logger
from borgmatic.borg import environment, feature, flags
from borgmatic.execute import execute_command, execute_command_and_capture_output
from borgmatic.execute import execute_command_and_capture_output
logger = logging.getLogger(__name__)
@@ -79,6 +79,13 @@ def get_latest_archive(
*flags.make_flags('umask', config.get('umask')),
*('--log-json',),
*flags.make_flags('lock-wait', config.get('lock_wait')),
*(
flags.make_match_archives_flags(
config.get('match_archives'),
config.get('archive_name_format'),
local_borg_version,
)
),
*(
flags.make_flags('consider-checkpoints', consider_checkpoints)
if not feature.available(feature.Feature.REPO_LIST, local_borg_version)
@@ -219,9 +226,21 @@ def list_repository(
working_directory = borgmatic.config.paths.get_working_directory(config)
borg_exit_codes = config.get('borg_exit_codes')
json_listing = '\n'.join(
if repo_list_arguments.json:
return '\n'.join(
execute_command_and_capture_output(
json_command,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
borg_exit_codes=borg_exit_codes,
)
)
output_lines = tuple(
execute_command_and_capture_output(
json_command,
main_command,
output_log_level=logging.ANSWER,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
@@ -229,18 +248,6 @@ def list_repository(
)
)
if repo_list_arguments.json:
return json_listing
flags.warn_for_aggressive_archive_flags(json_command, json_listing)
execute_command(
main_command,
output_log_level=logging.ANSWER,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
borg_exit_codes=borg_exit_codes,
)
flags.warn_for_aggressive_archive_flags(main_command, output_lines)
return None
+14 -10
View File
@@ -1044,16 +1044,17 @@ def exit_with_help_link(): # pragma: no cover
sys.exit(1)
def check_and_show_help_on_no_args(configs):
def check_and_show_help_on_no_args(configs, schema):
'''
Given a dict of configuration filename to corresponding parsed configuration, check if the
borgmatic command is run without any arguments. If the configuration option "default_actions" is
set to False, show the help message. Otherwise, trigger the default backup behavior.
Given a dict of configuration filename to corresponding parsed configuration and the
configuration schema as a dict, check if the borgmatic command was run without any arguments. If
the configuration option "default_actions" is set to False, then show the help message an exit.
'''
if len(sys.argv) == 1: # No arguments provided
if len(sys.argv) == 1: # No arguments provided.
default_actions = any(config.get('default_actions', True) for config in configs.values())
if not default_actions:
parse_arguments('--help')
if configs and not default_actions:
parse_arguments(schema, '--help')
sys.exit(0)
@@ -1122,7 +1123,7 @@ def main(extra_summary_logs=()): # pragma: no cover
exit_with_help_link()
except SystemExit as error:
if error.code == 0:
raise error
raise
configure_logging(logging.CRITICAL)
logger.critical(f"Error parsing arguments: {' '.join(sys.argv)}")
@@ -1150,8 +1151,7 @@ def main(extra_summary_logs=()): # pragma: no cover
resolve_env=global_arguments.resolve_env and not arguments.get('validate'),
)
# Use the helper function to check and show help on no arguments, passing the preloaded configs
check_and_show_help_on_no_args(configs)
check_and_show_help_on_no_args(configs, schema)
configuration_parse_errors = (
(max(log.levelno for log in parse_logs) >= logging.CRITICAL) if parse_logs else False
@@ -1202,3 +1202,7 @@ def main(extra_summary_logs=()): # pragma: no cover
)
display_summary(summary_logs, log_json)
if __name__ == '__main__': # pragma: no cover
main()
+3 -3
View File
@@ -1098,9 +1098,9 @@ properties:
default_actions:
type: boolean
description: |
Whether to apply default actions (create, prune, compact and check)
when no arguments are supplied to the borgmatic command. If set to
false, borgmatic displays the help message instead.
Whether to run default actions (create, prune, compact, and check)
when no arguments are given on the command line. If set to false,
borgmatic displays the help message instead.
example: true
skip_actions:
type: array
+75 -54
View File
@@ -43,6 +43,18 @@ def command_is_borg(command, borg_local_path):
return bool(borg_local_path and parsed_command[0] == borg_local_path)
BORG_EXIT_CODE_TO_DESCRIPTION = {
100: 'File changed while we backed it up',
101: 'Include pattern never matched',
102: 'General backup issue',
103: 'File type or inode changed while we backed it up',
104: 'Backup OS issue',
105: 'Backup permission issue',
106: 'Backup IO issue',
107: 'Backup file not found',
}
def interpret_exit_code(command, exit_code, borg_local_path=None, borg_exit_codes=None): # noqa: PLR0911
'''
Return an Exit_status value (e.g. SUCCESS, ERROR, or WARNING) based on interpreting the given
@@ -56,46 +68,47 @@ def interpret_exit_code(command, exit_code, borg_local_path=None, borg_exit_code
if exit_code == 0:
return Exit_status.SUCCESS
if command_is_borg(command, borg_local_path):
# First try looking for the exit code in the borg_exit_codes configuration.
for entry in borg_exit_codes or ():
if entry.get('code') == exit_code:
treat_as = entry.get('treat_as')
if not command_is_borg(command, borg_local_path):
return Exit_status.ERROR
if treat_as == 'error':
logger.error(
f'Treating exit code {exit_code} as an error, as per configuration',
)
return Exit_status.ERROR
description = BORG_EXIT_CODE_TO_DESCRIPTION.get(exit_code)
description_parenthetical = f' ({description})' if description else ''
if treat_as == 'warning':
logger.warning(
f'Treating exit code {exit_code} as a warning, as per configuration',
)
return Exit_status.WARNING
# First try looking for the exit code in the borg_exit_codes configuration.
for entry in borg_exit_codes or ():
if entry.get('code') == exit_code:
treat_as = entry.get('treat_as')
# If the exit code doesn't have explicit configuration, then fall back to the default
# behavior of treating Borg errors as errors and some Borg warnings as errors.
if exit_code in BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS:
logger.error(
f'Treating exit code {exit_code} as an error, as per borgmatic defaults',
)
return Exit_status.ERROR
return (
Exit_status.ERROR
if (
exit_code < 0
or (
exit_code >= BORG_ERROR_EXIT_CODE_START
and exit_code <= BORG_ERROR_EXIT_CODE_END
if treat_as == 'error':
logger.error(
f'Treating exit code {exit_code}{description_parenthetical} as an error, as per configuration',
)
)
else Exit_status.WARNING
return Exit_status.ERROR
if treat_as == 'warning':
logger.warning(
f'Treating exit code {exit_code}{description_parenthetical} as a warning, as per configuration',
)
return Exit_status.WARNING
# If the exit code doesn't have explicit configuration, then fall back to the default
# behavior of treating Borg errors as errors and some Borg warnings as errors.
if exit_code in BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS:
logger.error(
f'Treating exit code {exit_code}{description_parenthetical} as an error, as per borgmatic defaults',
)
return Exit_status.ERROR
return Exit_status.ERROR
if exit_code < 0 or (
exit_code >= BORG_ERROR_EXIT_CODE_START and exit_code <= BORG_ERROR_EXIT_CODE_END
):
return Exit_status.ERROR
logger.warning(
f'Treating exit code {exit_code}{description_parenthetical} as a warning, as per borgmatic defaults',
)
return Exit_status.WARNING
def command_for_process(process):
@@ -301,8 +314,9 @@ def log_buffer_lines(
'''
Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
Process_metadata instance, a requested output log level for stdout, Borg's local path, and
whether to capture stderr, read and log any ready output lines from the buffers. Additionally,
if the log level is None for any log record, then yield those log messages for capture.
whether to capture stderr, read and log any ready output lines from the buffers. Additionally,
for any log records with a log level that meets or exceeds the output log level, yield those log
messages for capture.
This function just does one "turn of the crank" of logging buffer output. It is intended to be
called repeatedly to continue to process buffers.
@@ -352,7 +366,11 @@ def log_buffer_lines(
last_lines=process_metadatas[reader.process].last_lines,
)
if log_record.levelno is None and process_metadatas[reader.process].capture:
if (
log_record.levelno is None
or output_log_level is None
or log_record.levelno >= output_log_level
) and process_metadatas[reader.process].capture:
yield log_record.getMessage()
@@ -411,8 +429,8 @@ def log_remaining_buffer_lines(
Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
Process_metadata instance, a requested output log level for stdout, Borg's local path, and
whether to capture stderr, drain and log any remaining output lines from the buffers until
they're empty. Additionally, if the log level is None for any log record, then yield those log
messages for capture.
they're empty. Additionally, for any log records with a log level that meets or exceeds the
output log level, yield those log messages for capture.
'''
for output_buffer, reader in buffer_readers.items():
if not reader.process:
@@ -432,7 +450,11 @@ def log_remaining_buffer_lines(
),
)
if log_record.levelno is None and process_metadatas[reader.process].capture:
if (
log_record.levelno is None
or output_log_level is None
or log_record.levelno >= output_log_level
) and process_metadatas[reader.process].capture:
yield log_record.getMessage()
@@ -615,6 +637,7 @@ def execute_command(
def execute_command_and_capture_output(
full_command,
output_log_level=None,
input_file=None,
capture_stderr=False,
shell=False,
@@ -629,13 +652,15 @@ def execute_command_and_capture_output(
output (stdout) as a generator that yields one line at a time. The generator must be consumed in
order for the called command to execute.
If an input file descriptor is given, then pipe it to the command's stdin. If capture stderr is
True, then capture stderr in addition to stdout. If shell is True, execute the command within a
shell. If an environment variables dict is given, then pass it into the command. If a working
directory is given, use that as the present working directory when running the command. If a
Borg local path is given, and the command matches it (regardless of arguments), treat exit code
1 as a warning instead of an error. But if Borg exit codes are given as a sequence of exit code
configuration dicts, then use that configuration to decide what's an error and what's a warning.
If an output log level is given, then instead of suppressing log output, also output the
captured lines at the given log level. If an input file descriptor is given, then pipe it to
the command's stdin. If capture stderr is True, then capture stderr in addition to stdout. If
shell is True, execute the command within a shell. If an environment variables dict is given,
then pass it into the command. If a working directory is given, use that as the present working
directory when running the command. If a Borg local path is given, and the command matches it
(regardless of arguments), treat exit code 1 as a warning instead of an error. But if Borg exit
codes are given as a sequence of exit code configuration dicts, then use that configuration to
decide what's an error and what's a warning.
Raise subprocesses.CalledProcessError if an error occurs while running the command.
'''
@@ -668,17 +693,15 @@ def execute_command_and_capture_output(
return
with borgmatic.logger.Log_prefix(None): # Log command output without any prefix.
captured_lines = log_outputs(
yield from log_outputs(
(process,),
(input_file,),
None,
output_log_level,
borg_local_path,
borg_exit_codes,
capture_stderr=capture_stderr,
)
yield from captured_lines
def execute_command_with_processes(
full_command,
@@ -741,12 +764,10 @@ def execute_command_with_processes(
raise
with borgmatic.logger.Log_prefix(None): # Log command output without any prefix.
captured_lines = log_outputs(
yield from log_outputs(
(*processes, command_process),
(input_file, output_file),
output_log_level,
borg_local_path,
borg_exit_codes,
)
yield from captured_lines
+13 -8
View File
@@ -88,16 +88,17 @@ class Loki_log_handler(logging.Handler):
A log handler that sends logs to Loki.
'''
def __init__(self, url, send_logs, dry_run):
def __init__(self, url, send_logs, log_level, dry_run):
'''
Given a URL to send logs to, whether all borgmatic logs should be sent (or just explicitly
added messages from this hook), and whether this is a dry run, create an instance of
Loki_log_buffer.
added messages from this hook), the log level to use (influencing which logs get sent), and
whether this is a dry run, create an instance of Loki_log_buffer.
'''
super().__init__()
self.buffer = Loki_log_buffer(url, dry_run)
self.send_logs = send_logs
self.setLevel(log_level)
def emit(self, record):
'''
@@ -138,7 +139,7 @@ def initialize_monitor(hook_config, config, config_filename, monitoring_log_leve
Add a handler to the root logger to regularly send the logs to Loki.
'''
url = hook_config.get('url')
loki = Loki_log_handler(url, hook_config.get('send_logs', False), dry_run)
loki = Loki_log_handler(url, hook_config.get('send_logs', False), monitoring_log_level, dry_run)
for key, value in hook_config.get('labels').items():
if value == '__hostname':
@@ -150,7 +151,9 @@ def initialize_monitor(hook_config, config, config_filename, monitoring_log_leve
else:
loki.add_label(key, value)
logging.getLogger().addHandler(loki)
global_logger = logging.getLogger()
global_logger.addHandler(loki)
global_logger.setLevel(min(handler.level for handler in global_logger.handlers))
def ping_monitor(hook_config, config, config_filename, state, monitoring_log_level, dry_run):
@@ -166,9 +169,11 @@ def destroy_monitor(hook_config, config, monitoring_log_level, dry_run):
'''
Remove the monitor handler that was added to the root logger.
'''
logger = logging.getLogger()
global_logger = logging.getLogger()
for handler in tuple(logger.handlers):
for handler in tuple(global_logger.handlers):
if isinstance(handler, Loki_log_handler):
handler.flush()
logger.removeHandler(handler)
global_logger.removeHandler(handler)
global_logger.setLevel(min(handler.level for handler in global_logger.handlers))
+1 -1
View File
@@ -14,7 +14,7 @@ import requests
def list_merged_pulls(url):
'''
Given a Gitea or GitHub API endpoint URL for pull requests, fetch and return the corresponding
Given a Forgejo or GitHub API endpoint URL for pull requests, fetch and return the corresponding
JSON for all such merged pull requests.
'''
response = requests.get(f'{url}?state=closed', headers={'Accept': 'application/json', 'Content-Type': 'application/json'})
+1 -1
View File
@@ -236,7 +236,7 @@ Compose.
Please do not use AI agents to modify this codebase. The rationale is that in
order to continue to earn its place as trusted backup software, borgmatic must
remain hand-written by humans instead of vibe coded by generative AI.
remain handwritten by humans instead of vibe coded by generative AI.
Additionally, if LLMs were to perform a sizeable chunk of the feature
development on this codebase, then human borgmatic developers would lose their
+107
View File
@@ -0,0 +1,107 @@
---
title: 📥 How to install borgmatic
eleventyNavigation:
key: 📥 Install borgmatic
parent: How-to guides
order: -1
---
### Prerequisites
Before installing borgmatic, first [install
Borg](https://borgbackup.readthedocs.io/en/stable/installation.html), at least
version 1.1. (borgmatic does not install Borg automatically so as to avoid
conflicts with existing Borg installations.)
Then, [install uv](https://docs.astral.sh/uv/getting-started/installation/) as
the root user (with `sudo`) to make installing borgmatic easier without
impacting other Python applications on your system. For Debian, there is a
[third-party package for
uv](https://dario.griffo.io/posts/how-to-install-uv-debian/). On Ubuntu, there
is a [snap package](https://snapcraft.io/install/astral-uv/ubuntu). On Arch, you
can just install the `python-uv` package.
### Root install
If you want borgmatic to run with privileged access so it can backup your system
files, then install borgmatic as the root user by running the following
commands:
```bash
sudo uv tool update-shell
sudo uv tool install borgmatic
```
Check whether this worked with:
```bash
sudo su -
borgmatic --version
```
If borgmatic is properly installed, that should output your borgmatic version.
And if you'd also like `sudo borgmatic` to work as well, keep reading!
### Non-root install
If you only want to run borgmatic as a non-root user (without privileged file
access) *or* you want to make `sudo borgmatic` work so borgmatic runs as root,
then install borgmatic as a non-root user by running the following commands as
that user:
```bash
uv tool update-shell
uv tool install borgmatic
```
This should work even if you've also installed borgmatic as the root user.
Check whether this worked with:
```bash
borgmatic --version
```
If borgmatic is properly installed, that should output your borgmatic version.
You can also try `sudo borgmatic --version` if you intend to run borgmatic
with `sudo`. If that doesn't work, you may need to update your [sudoers
`secure_path` option](https://wiki.archlinux.org/title/Sudo).
### Other ways to install
Besides the approaches described above, there are several other options for
installing borgmatic:
#### <span data-pagefind-weight="7.0">Docker / Podman</span>
* [container image with scheduled backups](https://github.com/borgmatic-collective/docker-borgmatic) (+ Docker Compose files)
* [container image with multi-arch and Docker CLI support](https://github.com/modem7/docker-borgmatic)
#### Operating system packages
* [Debian](https://tracker.debian.org/pkg/borgmatic)
* [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
* [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
* [Gentoo](https://packages.gentoo.org/packages/app-backup/borgmatic)
* [Arch Linux](https://archlinux.org/packages/extra/any/borgmatic/)
* [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=borgmatic)
* [OpenBSD](https://openports.pl/path/sysutils/borgmatic)
* [openSUSE](https://software.opensuse.org/package/borgmatic)
* [macOS (via Homebrew)](https://formulae.brew.sh/formula/borgmatic)
* [macOS (via MacPorts)](https://ports.macports.org/port/borgmatic/)
* [NixOS](https://search.nixos.org/packages?channel=unstable&show=borgmatic&query=borgmatic)
#### Etc.
* [stand-alone Linux binary](https://projects.torsion.org/borgmatic-collective/borgmatic/releases) (This is a beta feature!)
* [Ansible role](https://github.com/borgbase/ansible-role-borgbackup)
* [pipx](https://pipx.pypa.io/stable/)
## Next steps
* [Set up backups](https://torsion.org/borgmatic/how-to/set-up-backups/)
+6 -96
View File
@@ -1,104 +1,14 @@
---
title: 📥 How to set up backups
title: 📋 How to set up backups
eleventyNavigation:
key: 📥 Set up backups
key: 📋 Set up backups
parent: How-to guides
order: 0
---
To install borgmatic, first [install
Borg](https://borgbackup.readthedocs.io/en/stable/installation.html), at least
version 1.1. (borgmatic does not install Borg automatically so as to avoid
conflicts with existing Borg installations.)
Then, [install pipx](https://pypa.github.io/pipx/installation/) as the root
user (with `sudo`) to make installing borgmatic easier without impacting other
Python applications on your system. If you have trouble installing pipx with
pip, then you can install a system package instead. E.g. on Ubuntu or Debian,
run:
```bash
sudo apt update
sudo apt install pipx
```
### Root install
If you want to run borgmatic on a schedule with privileged access to your
files, then you should install borgmatic as the root user by running the
following commands:
```bash
sudo pipx ensurepath
sudo pipx install borgmatic
```
Check whether this worked with:
```bash
sudo su -
borgmatic --version
```
If borgmatic is properly installed, that should output your borgmatic version.
And if you'd also like `sudo borgmatic` to work, keep reading!
### Non-root install
If you only want to run borgmatic as a non-root user (without privileged file
access) *or* you want to make `sudo borgmatic` work so borgmatic runs as root,
then install borgmatic as a non-root user by running the following commands as
that user:
```bash
pipx ensurepath
pipx install borgmatic
```
This should work even if you've also installed borgmatic as the root user.
Check whether this worked with:
```bash
borgmatic --version
```
If borgmatic is properly installed, that should output your borgmatic version.
You can also try `sudo borgmatic --version` if you intend to run borgmatic
with `sudo`. If that doesn't work, you may need to update your [sudoers
`secure_path` option](https://wiki.archlinux.org/title/Sudo).
### Other ways to install
Besides the approaches described above, there are several other options for
installing borgmatic:
#### <span data-pagefind-weight="7.0">Docker / Podman</span>
* [container image with scheduled backups](https://github.com/borgmatic-collective/docker-borgmatic) (+ Docker Compose files)
* [container image with multi-arch and Docker CLI support](https://github.com/modem7/docker-borgmatic)
#### Operating system packages
* [Debian](https://tracker.debian.org/pkg/borgmatic)
* [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
* [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
* [Gentoo](https://packages.gentoo.org/packages/app-backup/borgmatic)
* [Arch Linux](https://archlinux.org/packages/extra/any/borgmatic/)
* [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=borgmatic)
* [OpenBSD](https://openports.pl/path/sysutils/borgmatic)
* [openSUSE](https://software.opensuse.org/package/borgmatic)
* [macOS (via Homebrew)](https://formulae.brew.sh/formula/borgmatic)
* [macOS (via MacPorts)](https://ports.macports.org/port/borgmatic/)
* [NixOS](https://search.nixos.org/packages?channel=unstable&show=borgmatic&query=borgmatic)
#### Etc.
* [Ansible role](https://github.com/borgbase/ansible-role-borgbackup)
* [uv tool install](https://docs.astral.sh/uv/)
Start by [installing
borgmatic](https://torsion.org/borgmatic/how-to/install-borgmatic/) if you haven't
already.
## Hosting providers
@@ -322,7 +232,7 @@ If you're using systemd instead of cron to run jobs, you can still configure
borgmatic to run automatically.
(If you installed borgmatic from [Other ways to
install](https://torsion.org/borgmatic/how-to/set-up-backups/#other-ways-to-install),
install](https://torsion.org/borgmatic/how-to/install-borgmatic/#other-ways-to-install),
you may already have borgmatic systemd service and timer files. If so, you may
be able to skip some of the steps below.)
+37 -8
View File
@@ -5,10 +5,25 @@ eleventyNavigation:
parent: How-to guides
order: 14
---
In general, all you should need to do to upgrade borgmatic if you've
[installed it with
pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation)
is to run the following:
In general, all you should need to do to upgrade borgmatic if you've [installed
it with uv](https://docs.astral.sh/uv/) is to run the following:
```bash
sudo uv tool upgrade borgmatic
```
Omit `sudo` if you installed borgmatic as a non-root user. And if you
installed borgmatic *both* as root and as a non-root user, you'll need to
upgrade each installation independently.
### Upgrading from other installation methods
#### pipx
If you have borgmatic installed with
[pipx](https://pipx.pypa.io/latest/installation/), and you'd like to continue
using pipx, then you can upgrade borgmatic with:
```bash
sudo pipx upgrade borgmatic
@@ -18,16 +33,30 @@ Omit `sudo` if you installed borgmatic as a non-root user. And if you
installed borgmatic *both* as root and as a non-root user, you'll need to
upgrade each installation independently.
But if you'd like to switch your borgmatic install from pipx to
[uv](https://docs.astral.sh/uv/), uninstall borgmatic with pipx (`sudo pipx
uninstall borgmatic`) and then [install borgmatic with
uv](https://torsion.org/borgmatic/how-to/install-borgmatic/).
Either one should work just fine. uv is just faster than pipx and also used for
borgmatic
[development](https://torsion.org/borgmatic/how-to/develop-on-borgmatic/).
#### pip install
If you originally installed borgmatic with `sudo pip3 install --user`, you can
uninstall it first with `sudo pip3 uninstall borgmatic` and then [install it
again with
pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation),
uv](https://torsion.org/borgmatic/how-to/install-borgmatic/),
which should better isolate borgmatic from your other Python applications.
But if you [installed borgmatic without pipx or
pip3](https://torsion.org/borgmatic/how-to/set-up-backups/#other-ways-to-install),
then your upgrade method may be different.
#### Etc.
If you installed borgmatic [some other
way](https://torsion.org/borgmatic/how-to/install-borgmatic/#other-ways-to-install),
then your upgrade method may be different.
### Upgrading your configuration
@@ -97,4 +97,3 @@ option for limiting the archives used for the `check` action was a separate
`prefix` in the `consistency` section. Both of these options are deprecated in
favor of the auto-matching behavior (or `match_archives`/`--match-archives`)
in newer versions of borgmatic.
@@ -12,12 +12,11 @@ us today such as: Telegram, Discord, Slack, Amazon SNS, Gotify, etc."
Depending on how you installed borgmatic, it may not have come with Apprise.
For instance, if you originally [installed borgmatic with
pipx](https://torsion.org/borgmatic/how-to/set-up-backups/),
uv](https://torsion.org/borgmatic/how-to/install-borgmatic/),
run the following to install Apprise so borgmatic can use it:
```bash
sudo pipx uninstall borgmatic
sudo pipx install borgmatic[Apprise]
sudo uv tool install borgmatic[Apprise]
```
Omit `sudo` if borgmatic is installed as a non-root user.
+3 -2
View File
@@ -1,6 +1,6 @@
[project]
name = "borgmatic"
version = "2.1.3"
version = "2.1.4"
authors = [
{ name="Dan Helfman", email="witten@torsion.org" },
]
@@ -44,7 +44,7 @@ namespaces = false
[tool.pytest.ini_options]
testpaths = "tests"
addopts = "--cov-report term-missing:skip-covered --cov=borgmatic --no-cov-on-fail --cov-fail-under=100 --ignore=tests/end-to-end"
addopts = "--cov-report term-missing:skip-covered --cov=borgmatic --no-cov-on-fail --cov-fail-under=100 --ignore=tests/end-to-end --timeout=120"
[tool.ruff]
line-length = 100
@@ -100,6 +100,7 @@ ignore = [
"S404", # subprocess import
"SIM115", # open() without context manager
"SIM905", # split() on literal string
"TRY004", # type check and raise something other than TypeError
]
[tool.ruff.lint.flake8-quotes]
+15
View File
@@ -43,6 +43,16 @@ twine upload -r pypi --username __token__ "$wheel_path"
scripts/export-docs-from-image
docs_path=dist/borgmatic-docs.tar.gz
# Build stand-alone binary.
uv venv --clear binary
source binary/bin/activate
uv pip install -r binary_requirements.txt
nuitka --mode=onefile --enable-plugin=upx --include-package-data=borgmatic --include-data-dir=borgmatic.egg-info=borgmatic.egg-info --include-package=borgmatic.hooks --include-package=apprise --no-deployment-flag=self-execution borgmatic/commands/borgmatic.py
deactivate
rm -fr binary borgmatic.build borgmatic.dist
standalone_binary_path="dist/borgmatic-${version}-binary-linux-glibc-x86_64"
mv borgmatic.bin "$standalone_binary_path"
# Set release changelogs on projects.torsion.org and GitHub.
release_changelog="$(cat NEWS | sed '/^$/q' | grep -v '^\S')"
escaped_release_changelog="$(echo "$release_changelog" | sed -z 's/\n/\\n/g' | sed -z 's/\"/\\"/g')"
@@ -68,6 +78,11 @@ curl --silent --request POST \
--header "Authorization: token $projects_token" \
--header "Accept: application/json" \
--form attachment=@"$docs_path"
curl --silent --request POST \
"https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/releases/$release_id/assets?name=$(basename $standalone_binary_path)" \
--header "Authorization: token $projects_token" \
--header "Accept: application/json" \
--form attachment=@"$standalone_binary_path"
github-release create --token="$github_token" --owner=witten --repo=borgmatic --tag="$version" --target_commit="main" \
--name="borgmatic $version" --body="$release_changelog"
+16 -15
View File
@@ -1,27 +1,28 @@
apprise==1.9.5
attrs==25.4.0
certifi==2025.10.5
charset-normalizer==3.4.4
apprise==1.9.9
attrs==26.1.0
certifi==2026.2.25
charset-normalizer==3.4.6
click>=8.1.8
codespell==2.4.1
coverage==7.11.0
flexmock==0.12.2
codespell==2.4.2
coverage==7.13.5
flexmock==0.13.0
idna==3.11
iniconfig==2.3.0
jsonschema==4.25.1
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
markdown==3.9
markdown==3.10.2
oauthlib==3.3.1
packaging==25.0
packaging==26.0
pluggy==1.6.0
pygments==2.19.2
pytest==8.4.2
pytest-cov==7.0.0
pygments==2.20.0
pytest==9.0.2
pytest-cov==7.1.0
pytest-timeout==2.4.0
pyyaml>5.0.0
referencing==0.37.0
requests==2.32.5
requests==2.33.1
requests-oauthlib==2.0.0
rpds-py==0.28.0
rpds-py==0.30.0
ruamel-yaml>0.15.0
typing-extensions==4.15.0
urllib3==2.6.3
+1 -1
View File
@@ -44,7 +44,7 @@ services:
MONGO_INITDB_ROOT_PASSWORD: test2
command: docker-entrypoint.sh --port=27018
tests:
image: docker.io/alpine:3.22
image: docker.io/alpine:3.23
environment:
TEST_CONTAINER: true
volumes:
@@ -69,6 +69,7 @@ class Background_web_server:
self.thread.join()
FINISH = 1
START_AND_FINISH = 2
START_LOG_AND_FINISH = 3
@@ -94,7 +95,7 @@ START_LOG_AND_FINISH = 3
),
(
'loki:\n url: http://localhost:12345/loki/api/v1/push\n labels:\n app: borgmatic\n send_logs: true',
START_AND_FINISH,
FINISH,
),
(
'loki:\n url: http://localhost:12345/loki/api/v1/push\n labels:\n app: borgmatic',
+2 -5
View File
@@ -35,7 +35,7 @@ def assert_command_does_not_duplicate_flags(command, *args, **kwargs):
if '--json' in command:
return '{}'
return None
return ''
def fuzz_argument(arguments, argument_name):
@@ -159,10 +159,7 @@ def test_make_repo_list_command_does_not_duplicate_flags_or_raise():
def test_display_archives_info_command_does_not_duplicate_flags_or_raise():
arguments = borgmatic.commands.arguments.parse_arguments({}, 'info')['info']
flexmock(borgmatic.borg.info).should_receive('execute_command_and_capture_output').replace_with(
assert_command_does_not_duplicate_flags,
)
flexmock(borgmatic.borg.info).should_receive('execute_command').replace_with(
assert_command_does_not_duplicate_flags,
lambda command, *args, **kwargs: iter((assert_command_does_not_duplicate_flags(command),)),
)
for argument_name in dir(arguments):
+20 -11
View File
@@ -8,7 +8,7 @@ from borgmatic.hooks.monitoring import loki as module
def test_loki_log_handler_raw_with_send_logs_posts_to_server_after_buffer_full():
handler = module.Loki_log_handler(flexmock(), send_logs=True, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=True, log_level=10, dry_run=False)
flexmock(module.requests).should_receive('post').and_return(
flexmock(raise_for_status=lambda: ''),
).once()
@@ -18,7 +18,7 @@ def test_loki_log_handler_raw_with_send_logs_posts_to_server_after_buffer_full()
def test_loki_log_handler_raw_without_send_logs_posts_to_server_without_buffering():
handler = module.Loki_log_handler(flexmock(), send_logs=False, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=False, log_level=10, dry_run=False)
flexmock(module.requests).should_receive('post').and_return(
flexmock(raise_for_status=lambda: ''),
).times(3)
@@ -28,7 +28,7 @@ def test_loki_log_handler_raw_without_send_logs_posts_to_server_without_bufferin
def test_loki_log_handler_raw_post_failure_does_not_raise():
handler = module.Loki_log_handler(flexmock(), send_logs=True, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=True, log_level=10, dry_run=False)
flexmock(module.requests).should_receive('post').and_return(
flexmock(raise_for_status=lambda: (_ for _ in ()).throw(requests.RequestException())),
).once()
@@ -46,7 +46,10 @@ def test_initialize_monitor_replaces_labels():
'labels': {'hostname': '__hostname', 'config': '__config', 'config_full': '__config_path'},
}
config_filename = '/mock/path/test.yaml'
module.initialize_monitor(hook_config, flexmock(), config_filename, flexmock(), dry_run=False)
flexmock(module.logging.getLogger()).should_receive('setLevel')
module.initialize_monitor(
hook_config, flexmock(), config_filename, monitoring_log_level=10, dry_run=False
)
for handler in tuple(logging.getLogger().handlers):
if isinstance(handler, module.Loki_log_handler):
@@ -63,11 +66,12 @@ def test_initialize_monitor_adds_log_handler():
Assert that calling initialize_monitor adds our logger to the root logger.
'''
hook_config = {'url': 'http://localhost:3100/loki/api/v1/push', 'labels': {'app': 'borgmatic'}}
flexmock(module.logging.getLogger()).should_receive('setLevel')
module.initialize_monitor(
hook_config,
flexmock(),
config_filename='test.yaml',
monitoring_log_level=flexmock(),
monitoring_log_level=10,
dry_run=True,
)
@@ -99,7 +103,10 @@ def test_ping_monitor_sends_log_message():
flexmock(module.requests).should_receive('post').replace_with(post)
module.initialize_monitor(hook_config, flexmock(), config_filename, flexmock(), dry_run=False)
flexmock(module.logging.getLogger()).should_receive('setLevel')
module.initialize_monitor(
hook_config, flexmock(), config_filename, monitoring_log_level=10, dry_run=False
)
module.ping_monitor(
hook_config,
flexmock(),
@@ -108,7 +115,7 @@ def test_ping_monitor_sends_log_message():
flexmock(),
dry_run=False,
)
module.destroy_monitor(hook_config, flexmock(), flexmock(), dry_run=False)
module.destroy_monitor(hook_config, flexmock(), monitoring_log_level=10, dry_run=False)
assert post_called
@@ -121,9 +128,11 @@ def test_destroy_monitor_removes_log_handler():
config_filename = 'test.yaml'
flexmock(module.requests).should_receive('post').never()
module.initialize_monitor(hook_config, flexmock(), config_filename, flexmock(), dry_run=False)
module.destroy_monitor(hook_config, flexmock(), flexmock(), dry_run=False)
flexmock(module.logging.getLogger()).should_receive('setLevel')
module.initialize_monitor(
hook_config, flexmock(), config_filename, monitoring_log_level=10, dry_run=False
)
module.destroy_monitor(hook_config, flexmock(), monitoring_log_level=10, dry_run=False)
for handler in tuple(logging.getLogger().handlers):
if isinstance(handler, module.Loki_log_handler):
raise AssertionError()
assert not isinstance(handler, module.Loki_log_handler)
+24 -33
View File
@@ -105,18 +105,15 @@ def test_log_outputs_logs_each_line_separately():
(),
).and_return((there_process.stdout,))
assert (
tuple(
module.log_outputs(
(hi_process, there_process),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
assert tuple(
module.log_outputs(
(hi_process, there_process),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
== ()
)
) == ('there',)
def test_log_outputs_logs_stderr_as_error():
@@ -140,18 +137,15 @@ def test_log_outputs_logs_stderr_as_error():
(),
).and_return((echo_process.stdout, echo_process.stderr))
assert (
tuple(
module.log_outputs(
(echo_process,),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
assert tuple(
module.log_outputs(
(echo_process,),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
== ()
)
) == ('error',)
def test_log_outputs_skips_logs_for_process_with_none_stdout():
@@ -180,18 +174,15 @@ def test_log_outputs_skips_logs_for_process_with_none_stdout():
(),
).and_return((there_process.stdout,))
assert (
tuple(
module.log_outputs(
(hi_process, there_process),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
assert tuple(
module.log_outputs(
(hi_process, there_process),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
== ()
)
) == ('there',)
def test_log_outputs_returns_output_without_logging_for_output_log_level_none():
+37
View File
@@ -1473,6 +1473,43 @@ def test_compare_spot_check_hashes_considers_symlink_path_as_not_matching():
) == ('/bar',)
def test_compare_spot_check_hashes_considers_borg_2_symlink_path_as_not_matching():
flexmock(module.random).should_receive('SystemRandom').and_return(
flexmock(sample=lambda population, count: population[:count]),
)
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(
None,
)
flexmock(module.os.path).should_receive('exists').and_return(True)
flexmock(module.os.path).should_receive('islink').with_args('/foo').and_return(False)
flexmock(module.os.path).should_receive('islink').with_args('/bar').and_return(True)
flexmock(module.borgmatic.execute).should_receive(
'execute_command_and_capture_output',
).with_args(('xxh64sum', '/foo'), working_directory=None).and_yield('hash1 /foo')
flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
{'xxh64': 'hash1', 'path': 'foo', 'target': ''},
{'xxh64': 'hash2', 'path': 'bar', 'target': ''},
)
assert module.compare_spot_check_hashes(
repository={'path': 'repo'},
archive='archive',
config={
'checks': [
{
'name': 'spot',
'data_sample_percentage': 50,
},
],
},
local_borg_version=flexmock(),
global_arguments=flexmock(),
local_path=flexmock(),
remote_path=flexmock(),
source_paths=('/foo', '/bar', '/baz', '/quux'),
) == ('/bar',)
def test_compare_spot_check_hashes_considers_non_existent_path_as_not_matching():
flexmock(module.random).should_receive('SystemRandom').and_return(
flexmock(sample=lambda population, count: population[:count]),
+15 -18
View File
@@ -246,10 +246,10 @@ def test_make_match_archives_flags_accepts_default_archive_name_format():
)
def test_warn_for_aggressive_archive_flags_without_archive_flags_bails():
def test_warn_for_aggressive_archive_flags_without_archive_flags_does_not_warn():
flexmock(module.logger).should_receive('warning').never()
module.warn_for_aggressive_archive_flags(('borg', '--do-stuff'), '{}')
module.warn_for_aggressive_archive_flags(('borg', '--do-stuff'), ())
def test_warn_for_aggressive_archive_flags_with_glob_archives_and_zero_archives_warns():
@@ -257,7 +257,7 @@ def test_warn_for_aggressive_archive_flags_with_glob_archives_and_zero_archives_
module.warn_for_aggressive_archive_flags(
('borg', '--glob-archives', 'foo*'),
'{"archives": []}',
(),
)
@@ -266,7 +266,16 @@ def test_warn_for_aggressive_archive_flags_with_match_archives_and_zero_archives
module.warn_for_aggressive_archive_flags(
('borg', '--match-archives', 'foo*'),
'{"archives": []}',
(),
)
def test_warn_for_aggressive_archive_flags_with_match_archives_and_just_exit_code_warns():
flexmock(module.logger).should_receive('warning').twice()
module.warn_for_aggressive_archive_flags(
('borg', '--match-archives', 'foo*'),
('terminating with success status, rc 0',),
)
@@ -275,7 +284,7 @@ def test_warn_for_aggressive_archive_flags_with_glob_archives_and_one_archive_do
module.warn_for_aggressive_archive_flags(
('borg', '--glob-archives', 'foo*'),
'{"archives": [{"name": "foo"]}',
('this is an archive line',),
)
@@ -284,22 +293,10 @@ def test_warn_for_aggressive_archive_flags_with_match_archives_and_one_archive_d
module.warn_for_aggressive_archive_flags(
('borg', '--match-archives', 'foo*'),
'{"archives": [{"name": "foo"]}',
('this is an archive line',),
)
def test_warn_for_aggressive_archive_flags_with_glob_archives_and_invalid_json_does_not_warn():
flexmock(module.logger).should_receive('warning').never()
module.warn_for_aggressive_archive_flags(('borg', '--glob-archives', 'foo*'), '{"archives": [}')
def test_warn_for_aggressive_archive_flags_with_glob_archives_and_json_missing_archives_does_not_warn():
flexmock(module.logger).should_receive('warning').never()
module.warn_for_aggressive_archive_flags(('borg', '--glob-archives', 'foo*'), '{}')
def test_omit_flag_removes_flag_from_arguments():
assert module.omit_flag(('borg', 'create', '--flag', '--other'), '--flag') == (
'borg',
+4 -13
View File
@@ -565,15 +565,14 @@ def test_make_info_command_with_date_based_matching_passes_through_to_command():
)
def test_display_archives_info_calls_two_commands():
def test_display_archives_info_calls_borg_command():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('make_info_command')
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command_and_capture_output').and_yield().once()
flexmock(module).should_receive('execute_command_and_capture_output').and_yield('').once()
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').once()
module.display_archives_info(
repository_path='repo',
@@ -592,7 +591,6 @@ def test_display_archives_info_with_json_calls_json_command_only():
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command_and_capture_output').and_yield('{}')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags').never()
flexmock(module).should_receive('execute_command').never()
assert (
module.display_archives_info(
@@ -616,20 +614,13 @@ def test_display_archives_info_calls_borg_with_working_directory():
)
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
full_command=object,
output_log_level=int,
environment=object,
working_directory='/working/dir',
borg_local_path=object,
borg_exit_codes=object,
).and_yield().once()
).and_yield('').once()
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
full_command=object,
output_log_level=object,
environment=object,
working_directory='/working/dir',
borg_local_path=object,
borg_exit_codes=object,
).once()
module.display_archives_info(
repository_path='repo',
+49 -12
View File
@@ -115,6 +115,7 @@ def test_get_latest_archive_calls_borg_with_flags():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -144,6 +145,7 @@ def test_get_latest_archive_with_log_info_calls_borg_without_info_flag():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -174,6 +176,7 @@ def test_get_latest_archive_with_log_debug_calls_borg_without_debug_flag():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -204,6 +207,7 @@ def test_get_latest_archive_with_local_path_calls_borg_via_local_path():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -234,6 +238,7 @@ def test_get_latest_archive_with_exit_codes_calls_borg_using_them():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -267,6 +272,7 @@ def test_get_latest_archive_with_remote_path_calls_borg_with_remote_path_flags()
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -300,6 +306,7 @@ def test_get_latest_archive_with_umask_calls_borg_with_umask_flags():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -329,6 +336,7 @@ def test_get_latest_archive_without_archives_raises():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
@@ -358,6 +366,7 @@ def test_get_latest_archive_with_lock_wait_calls_borg_with_lock_wait_flags():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -380,6 +389,38 @@ def test_get_latest_archive_with_lock_wait_calls_borg_with_lock_wait_flags():
)
def test_get_latest_archive_with_match_archives_calls_borg_with_match_archives_flags():
expected_archive = {'name': 'archive-name', 'id': 'd34db33f'}
flexmock(module.feature).should_receive('available').and_return(False)
flexmock(module.flags).should_receive('make_flags').and_return(())
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(
('--match-archives', 'foo')
)
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
('borg', 'list', '--log-json', '--match-archives', 'foo', *BORG_LIST_LATEST_ARGUMENTS),
environment=None,
working_directory=None,
borg_local_path='borg',
borg_exit_codes=None,
).and_yield(json.dumps({'archives': [expected_archive]}))
assert (
module.get_latest_archive(
'repo',
config={'match_archives': 'foo'},
local_borg_version='1.2.3',
global_arguments=flexmock(),
)
== expected_archive
)
def test_get_latest_archive_calls_borg_with_list_extra_borg_options():
expected_archive = {'name': 'archive-name', 'id': 'd34db33f'}
flexmock(module.feature).should_receive('available').and_return(False)
@@ -387,6 +428,7 @@ def test_get_latest_archive_calls_borg_with_list_extra_borg_options():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -424,6 +466,7 @@ def test_get_latest_archive_with_feature_available_calls_borg_with_repo_list_ext
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -464,6 +507,7 @@ def test_get_latest_archive_with_consider_checkpoints_calls_borg_with_consider_c
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -497,6 +541,7 @@ def test_get_latest_archive_with_consider_checkpoints_and_feature_available_call
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('--repo', 'repo'))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
@@ -527,6 +572,7 @@ def test_get_latest_archive_calls_borg_with_working_directory():
flexmock(module.flags).should_receive('make_flags').with_args('last', 1).and_return(
('--last', '1')
)
flexmock(module.flags).should_receive('make_match_archives_flags').and_return(())
flexmock(module.flags).should_receive('make_repository_flags').and_return(('repo',))
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(
@@ -1108,7 +1154,7 @@ def test_make_repo_list_command_with_match_archives_calls_borg_with_match_archiv
assert command == ('borg', 'list', '--log-json', '--match-archives', 'foo-*', 'repo')
def test_list_repository_calls_two_commands():
def test_list_repository_calls_borg_command():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module.logging).ANSWER = module.borgmatic.logger.ANSWER
flexmock(module).should_receive('make_repo_list_command')
@@ -1116,7 +1162,6 @@ def test_list_repository_calls_two_commands():
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command_and_capture_output').and_yield('').once()
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').once()
module.list_repository(
repository_path='repo',
@@ -1127,14 +1172,13 @@ def test_list_repository_calls_two_commands():
)
def test_list_repository_with_json_calls_json_command_only():
def test_list_repository_with_json_calls_borg_json_command_only():
flexmock(module.borgmatic.logger).should_receive('add_custom_log_levels')
flexmock(module).should_receive('make_repo_list_command')
flexmock(module.environment).should_receive('make_environment')
flexmock(module.borgmatic.config.paths).should_receive('get_working_directory').and_return(None)
flexmock(module).should_receive('execute_command_and_capture_output').and_yield('{}')
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags').never()
flexmock(module).should_receive('execute_command').never()
assert (
module.list_repository(
@@ -1206,20 +1250,13 @@ def test_list_repository_calls_borg_with_working_directory():
)
flexmock(module).should_receive('execute_command_and_capture_output').with_args(
full_command=object,
output_log_level=int,
environment=object,
working_directory='/working/dir',
borg_local_path=object,
borg_exit_codes=object,
).and_yield('').once()
flexmock(module.flags).should_receive('warn_for_aggressive_archive_flags')
flexmock(module).should_receive('execute_command').with_args(
full_command=object,
output_log_level=object,
environment=object,
working_directory='/working/dir',
borg_local_path=object,
borg_exit_codes=object,
).once()
module.list_repository(
repository_path='repo',
+24 -7
View File
@@ -2582,23 +2582,40 @@ def test_collect_configuration_run_summary_logs_outputs_merged_json_results():
def test_check_and_show_help_on_no_args_shows_help_when_no_args_and_default_actions_false():
flexmock(module.sys).should_receive('argv').and_return(['borgmatic'])
flexmock(module).should_receive('parse_arguments').with_args('--help').once()
flexmock(module).should_receive('parse_arguments').with_args({}, '--help').once()
flexmock(module.sys).should_receive('exit').with_args(0).once()
module.check_and_show_help_on_no_args({'test.yaml': {'default_actions': False}})
module.check_and_show_help_on_no_args(
configs={'test.yaml': {'default_actions': False}}, schema={}
)
def test_check_and_show_help_on_no_args_does_not_show_help_when_no_args_and_no_configurations():
flexmock(module.sys).should_receive('argv').and_return(['borgmatic'])
flexmock(module).should_receive('parse_arguments').never()
flexmock(module.sys).should_receive('exit').never()
module.check_and_show_help_on_no_args(configs={}, schema={})
def test_check_and_show_help_on_no_args_does_not_show_help_when_no_args_and_default_actions_true():
flexmock(module.sys).should_receive('argv').and_return(['borgmatic'])
flexmock(module).should_receive('parse_arguments').never()
flexmock(module.sys).should_receive('exit').never()
module.check_and_show_help_on_no_args({'test.yaml': {'default_actions': True}})
module.check_and_show_help_on_no_args(
configs={'test.yaml': {'default_actions': True}}, schema={}
)
def test_check_and_show_help_on_no_args_does_not_show_help_when_args_provided():
flexmock(module.sys).should_receive('argv').and_return(['borgmatic', '--create'])
flexmock(module).should_receive('parse_arguments').never()
flexmock(module.sys).should_receive('exit').never()
module.check_and_show_help_on_no_args({'test.yaml': {'default_actions': False}})
module.check_and_show_help_on_no_args(
configs={'test.yaml': {'default_actions': False}}, schema={}
)
def test_check_and_show_help_on_no_args_with_no_default_actions_in_all_configs():
@@ -2611,10 +2628,10 @@ def test_check_and_show_help_on_no_args_with_no_default_actions_in_all_configs()
}
# Expect help to be shown
flexmock(module).should_receive('parse_arguments').with_args('--help').once()
flexmock(module).should_receive('parse_arguments').with_args({}, '--help').once()
flexmock(module.sys).should_receive('exit').with_args(0).once()
module.check_and_show_help_on_no_args(configs)
module.check_and_show_help_on_no_args(configs=configs, schema={})
def test_check_and_show_help_on_no_args_with_conflicting_configs():
@@ -2630,7 +2647,7 @@ def test_check_and_show_help_on_no_args_with_conflicting_configs():
flexmock(module).should_receive('parse_arguments').never()
flexmock(module.sys).should_receive('exit').never()
module.check_and_show_help_on_no_args(configs)
module.check_and_show_help_on_no_args(configs=configs, schema={})
def test_get_singular_option_value_with_conflicting_values_exits():
-15
View File
@@ -47,11 +47,6 @@ from borgmatic.config import normalize as module
{'prefix': 'foo'},
True,
),
(
{'location': {'prefix': 'foo'}, 'consistency': {'prefix': 'foo'}},
{'prefix': 'foo'},
True,
),
(
{'location': {'prefix': 'foo'}, 'consistency': {'bar': 'baz'}},
{'prefix': 'foo', 'bar': 'baz'},
@@ -62,11 +57,6 @@ from borgmatic.config import normalize as module
{'umask': 'foo'},
True,
),
(
{'storage': {'umask': 'foo'}, 'hooks': {'umask': 'foo'}},
{'umask': 'foo'},
True,
),
(
{'storage': {'umask': 'foo'}, 'hooks': {'bar': 'baz'}},
{'umask': 'foo', 'bar': 'baz'},
@@ -288,11 +278,6 @@ def test_normalize_commands_moves_individual_command_hooks_to_unified_commands(
{'checks': [{'name': 'archives'}]},
True,
),
(
{'checks': ['archives']},
{'checks': [{'name': 'archives'}]},
True,
),
(
{'numeric_owner': False},
{'numeric_ids': False},
+5 -10
View File
@@ -50,10 +50,7 @@ def test_loki_log_buffer_json_serializes_log_lines():
assert json.loads(buffer.to_request())['streams'][0]['values'][0][1] == 'Some test log line'
def test_loki_log_handler_add_label_gets_labels():
'''
Assert that adding labels works.
'''
def test_loki_log_buffer_add_label_gets_labels():
buffer = module.Loki_log_buffer(flexmock(), dry_run=False)
buffer.add_label('test', 'label')
@@ -64,22 +61,20 @@ def test_loki_log_handler_add_label_gets_labels():
def test_loki_log_handler_emit_with_send_logs_records_log_message():
handler = module.Loki_log_handler(flexmock(), send_logs=True, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=True, log_level=10, dry_run=False)
flexmock(handler).should_receive('raw').once()
handler.emit(flexmock(getMessage=lambda: 'Some test log line'))
def test_loki_log_handler_emit_without_send_logs_skips_log_message():
handler = module.Loki_log_handler(flexmock(), send_logs=False, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=False, log_level=10, dry_run=False)
flexmock(handler).should_receive('raw').never()
handler.emit(flexmock(getMessage=lambda: 'Some test log line'))
def test_loki_log_handler_flush_with_empty_buffer_does_not_raise():
'''
Test that flushing an empty buffer does indeed nothing.
'''
handler = module.Loki_log_handler(flexmock(), send_logs=False, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=False, log_level=10, dry_run=False)
handler.flush()
+122 -24
View File
@@ -31,8 +31,6 @@ def test_command_is_borg_matches_local_path_to_command(command, borg_local_path,
(False, 1, None, module.Exit_status.ERROR),
(True, 1, None, module.Exit_status.WARNING),
(False, 100, None, module.Exit_status.ERROR),
(False, 2, None, module.Exit_status.ERROR),
(True, 2, None, module.Exit_status.ERROR),
(True, 100, None, module.Exit_status.WARNING),
(False, 0, None, module.Exit_status.SUCCESS),
(True, 0, None, module.Exit_status.SUCCESS),
@@ -405,7 +403,7 @@ def test_log_buffer_lines_with_ready_buffer_and_running_process_handles_each_log
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -437,7 +435,7 @@ def test_log_buffer_lines_with_ready_buffer_and_capture_process_yields_each_line
) == ('message', 'message')
def test_log_buffer_lines_with_ready_buffer_and_log_level_and_capture_process_does_not_yield_each_line():
def test_log_buffer_lines_with_ready_buffer_and_same_log_level_and_capture_process_yields_each_line():
process = flexmock(poll=lambda: None, stderr=flexmock(), args=flexmock())
buffer_readers = {
flexmock(): module.Buffer_reader(lines=iter((('hi', 'there'),)), process=process)
@@ -448,7 +446,31 @@ def test_log_buffer_lines_with_ready_buffer_and_log_level_and_capture_process_do
).and_return(list(buffer_readers.keys()), [], [])
flexmock(module).should_receive('parse_log_line').and_return(flexmock())
flexmock(module).should_receive('handle_log_record').and_return(
flexmock(levelno=10, getMessage=lambda: 'message')
flexmock(levelno=module.logging.INFO, getMessage=lambda: 'message')
).twice()
assert tuple(
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
) == ('message', 'message')
def test_log_buffer_lines_with_ready_buffer_and_higher_log_level_and_capture_process_does_not_yield_each_line():
process = flexmock(poll=lambda: None, stderr=flexmock(), args=flexmock())
buffer_readers = {
flexmock(): module.Buffer_reader(lines=iter((('hi', 'there'),)), process=process)
}
process_metadatas = {process: module.Process_metadata(last_lines=[], capture=True)}
flexmock(module.select).should_receive('select').with_args(
buffer_readers.keys(), [], []
).and_return(list(buffer_readers.keys()), [], [])
flexmock(module).should_receive('parse_log_line').and_return(flexmock())
flexmock(module).should_receive('handle_log_record').and_return(
flexmock(levelno=module.logging.DEBUG, getMessage=lambda: 'message')
).twice()
assert (
@@ -456,7 +478,7 @@ def test_log_buffer_lines_with_ready_buffer_and_log_level_and_capture_process_do
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -464,6 +486,30 @@ def test_log_buffer_lines_with_ready_buffer_and_log_level_and_capture_process_do
)
def test_log_buffer_lines_with_ready_buffer_and_lower_log_level_and_capture_process_yields_each_line():
process = flexmock(poll=lambda: None, stderr=flexmock(), args=flexmock())
buffer_readers = {
flexmock(): module.Buffer_reader(lines=iter((('hi', 'there'),)), process=process)
}
process_metadatas = {process: module.Process_metadata(last_lines=[], capture=True)}
flexmock(module.select).should_receive('select').with_args(
buffer_readers.keys(), [], []
).and_return(list(buffer_readers.keys()), [], [])
flexmock(module).should_receive('parse_log_line').and_return(flexmock())
flexmock(module).should_receive('handle_log_record').and_return(
flexmock(levelno=module.logging.ERROR, getMessage=lambda: 'message')
).twice()
assert tuple(
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
) == ('message', 'message')
def test_log_buffer_lines_with_ready_buffer_and_finished_process_vents_other_processes():
process_stdout = flexmock()
process = flexmock(poll=lambda: 0, stdout=process_stdout, stderr=flexmock(), args=flexmock())
@@ -487,7 +533,7 @@ def test_log_buffer_lines_with_ready_buffer_and_finished_process_vents_other_pro
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -524,7 +570,7 @@ def test_log_buffer_lines_with_ready_buffer_and_finished_process_does_not_vent_o
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -550,7 +596,7 @@ def test_log_buffer_lines_with_ready_eof_buffer_and_running_process_skips_it():
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -573,7 +619,7 @@ def test_log_buffer_lines_with_ready_buffer_with_empty_line_skips_it():
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -603,7 +649,7 @@ def test_log_buffer_lines_with_multiple_ready_buffers_and_running_processes_hand
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -631,7 +677,7 @@ def test_log_buffer_lines_with_multiple_ready_buffers_from_same_running_process_
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -659,7 +705,7 @@ def test_log_buffer_lines_with_ready_stderr_buffer_and_running_process_elevates_
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -687,7 +733,7 @@ def test_log_buffer_lines_with_ready_stdout_buffer_and_running_process_does_not_
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -715,7 +761,7 @@ def test_log_buffer_lines_with_ready_stderr_buffer_and_capture_stderr_does_not_e
module.log_buffer_lines(
buffer_readers=buffer_readers,
process_metadatas=process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
capture_stderr=True,
)
@@ -995,7 +1041,7 @@ def test_log_remaining_buffer_lines_without_reader_process_bails():
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -1022,7 +1068,7 @@ def test_log_remaining_buffer_lines_logs_each_line():
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -1053,7 +1099,7 @@ def test_log_remaining_buffer_lines_with_multiple_buffers_logs_lines_from_each()
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -1088,7 +1134,7 @@ def test_log_remaining_buffer_lines_with_stderr_buffer_elevates_stderr():
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -1120,7 +1166,7 @@ def test_log_remaining_buffer_lines_with_stderr_buffer_and_capture_stderr_does_n
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
capture_stderr=True,
)
@@ -1149,13 +1195,13 @@ def test_log_remaining_buffer_lines_with_capture_process_yields_each_line():
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
) == ('message', 'message')
def test_log_remaining_buffer_lines_with_log_level_and_capture_process_does_not_yield_each_line():
def test_log_remaining_buffer_lines_with_same_log_level_and_capture_process_yields_each_line():
process = flexmock(stderr=flexmock(), args=flexmock())
buffer_readers = {
flexmock(): module.Buffer_reader(
@@ -1168,7 +1214,33 @@ def test_log_remaining_buffer_lines_with_log_level_and_capture_process_does_not_
line=str, log_level=object, elevate_stderr=False, borg_local_path=object, command=object
).and_return(flexmock()).twice()
flexmock(module).should_receive('handle_log_record').and_return(
flexmock(levelno=10, getMessage=lambda: 'message')
flexmock(levelno=module.logging.INFO, getMessage=lambda: 'message')
).twice()
assert tuple(
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
) == ('message', 'message')
def test_log_remaining_buffer_lines_with_higher_log_level_and_capture_process_does_not_yield_each_line():
process = flexmock(stderr=flexmock(), args=flexmock())
buffer_readers = {
flexmock(): module.Buffer_reader(
lines=(('hi', 'there'),),
process=process,
)
}
process_metadatas = {process: module.Process_metadata(last_lines=[], capture=True)}
flexmock(module).should_receive('parse_log_line').with_args(
line=str, log_level=object, elevate_stderr=False, borg_local_path=object, command=object
).and_return(flexmock()).twice()
flexmock(module).should_receive('handle_log_record').and_return(
flexmock(levelno=module.logging.DEBUG, getMessage=lambda: 'message')
).twice()
assert (
@@ -1176,7 +1248,7 @@ def test_log_remaining_buffer_lines_with_log_level_and_capture_process_does_not_
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=flexmock(),
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
)
@@ -1184,6 +1256,32 @@ def test_log_remaining_buffer_lines_with_log_level_and_capture_process_does_not_
)
def test_log_remaining_buffer_lines_with_lower_log_level_and_capture_process_does_yields_each_line():
process = flexmock(stderr=flexmock(), args=flexmock())
buffer_readers = {
flexmock(): module.Buffer_reader(
lines=(('hi', 'there'),),
process=process,
)
}
process_metadatas = {process: module.Process_metadata(last_lines=[], capture=True)}
flexmock(module).should_receive('parse_log_line').with_args(
line=str, log_level=object, elevate_stderr=False, borg_local_path=object, command=object
).and_return(flexmock()).twice()
flexmock(module).should_receive('handle_log_record').and_return(
flexmock(levelno=module.logging.ERROR, getMessage=lambda: 'message')
).twice()
assert tuple(
module.log_remaining_buffer_lines(
buffer_readers,
process_metadatas,
output_log_level=module.logging.INFO,
borg_local_path=flexmock(),
)
) == ('message', 'message')
def test_mask_command_secrets_masks_password_flag_value():
assert module.mask_command_secrets(('cooldb', '--username', 'bob', '--password', 'pass')) == (
'cooldb',