Compare commits

...
111 Commits
Author SHA1 Message Date
Dan Helfman dbc3fc356c Fix schema example for "archive_username" (#1317). 2026-07-20 23:04:51 -07:00
Dan Helfman e428d74258 Fix the (unreleased) "archive_hostname" option / "--archive-hostname" flag to pass an environment variable to Borg instead of a flag. Also add an "archive_username" option and corresponding "--archive-username" flag to override the "{user}" plaecholder (#1317). 2026-07-20 18:57:11 -07:00
Dan Helfman d454f3a3af Fix an error from the "diff" action when exclude options are configured (#1327). 2026-07-19 18:59:03 -07:00
Dan Helfman 5825af9fc1 Fix shadowing of built-in function (#1331). 2026-07-16 12:28:31 -07:00
Dan Helfman eb5c9e6bf3 Fix the "repo-create" action to more surgically suppress Borg "Repository does not exist" logs and avoid inadvertently suppressing other error logs (#1331). 2026-07-16 11:58:10 -07:00
Dan Helfman b41f4cfa0d Add the "CAP_FOWNER" capability to "CapabilityBoundingSet" in the sample systemd service, so that Borg can open source files without changing file access times (#1334). 2026-07-13 16:08:52 -07:00
Dan Helfman eae3341b01 Fix the "--progress" flag on the "compact" action to actually update the progress of segment compaction (#1333). 2026-07-13 15:58:35 -07:00
Dan Helfman 42743273d8 Tweak NEWS entry to mention Borg version limitation (#1317). 2026-07-13 14:04:07 -07:00
Dan Helfman 07d5a0129c For the MariaDB and MySQL hooks, add "events", "routines", and "tablespaces" options for disabling dumping of scheduled events, stored routines, and tablespaces, respectively (#1324). 2026-06-17 16:26:01 -07:00
Dan Helfman b14425b2af Consume "--archive-hostname" flag and "archive_hostname" option in pretty much all actions (#1317). 2026-06-17 14:57:42 -07:00
Dan Helfman f768abfaa0 Update the documentation to reflect the fact that Hetzner has ended its referral program. 2026-06-15 17:13:02 -07:00
Dan Helfman a73e28155c Add an "--archive-hostname" flag and corresponding "archive_hostname" option for overriding the hostname used when creating an archive (#1317). 2026-06-15 13:48:02 -07:00
Dan Helfman beaeea25b3 Fix for the "restore" action sometimes failing to find a database dump that was dumped with a default port (#1322). 2026-06-15 11:56:04 -07:00
Dan Helfman fa099b8471 Add support for the "--quick-stats" flag and the "quick_statistics" option to the "prune" action (#1309). 2026-06-11 15:57:52 -07:00
Dan Helfman 501f12dce3 Fix the ZFS hook's overzealous unmounting of snapshot paths when a source dataset is at "/" (#1319). 2026-06-10 16:15:32 -07:00
Dan Helfman d74b340024 Fix a bug in which the "compact" action does not pass a compact threshold of zero to Borg. 2026-06-06 15:31:29 -07:00
Dan Helfman 219282b5fa Some NEWS reordering. 2026-06-01 16:02:30 -07:00
Dan Helfman 4af31db99c Bump version for release. 2026-06-01 15:52:45 -07:00
Dan Helfman 7cd9e6b361 Add a minimal stand-alone borgmatic binary in addition to the standard one (#1314). 2026-06-01 15:51:01 -07:00
Dan Helfman 4ee057a909 Experimental "browse" action, a console UI for browsing your backups (#1312).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1312
2026-06-01 20:09:24 +00:00
Dan Helfman 3e7e7879bc Remove some default system commands that aren't particular useful for this app. 2026-06-01 12:59:52 -07:00
Dan Helfman d29cfaf227 Use full URL for image. 2026-06-01 12:47:53 -07:00
Dan Helfman 23a8d665d6 Add a screenshot to the browse docs. 2026-06-01 12:44:32 -07:00
Dan Helfman 2a026a4842 Stop using Ruff "preview" rules, as CI has an older version of Ruff. 2026-06-01 12:42:02 -07:00
Dan Helfman 16c8f81a5a Attempted CI test fixes. Also spelling. 2026-06-01 12:24:57 -07:00
Dan Helfman 1670ba2aeb Add missing tests. 2026-06-01 11:57:49 -07:00
Dan Helfman 5accda1a65 Complete tests. 2026-06-01 11:54:28 -07:00
Dan Helfman ec8e52944c More tests! 2026-05-31 20:30:59 -07:00
Dan Helfman 30ca942f76 Improve performance wen loading file previews. 2026-05-31 18:55:56 -07:00
Dan Helfman 9748a0298a Add icon to configuration files list border title. 2026-05-30 23:11:41 -07:00
Dan Helfman dc91f81a39 Fix loading race conditions. 2026-05-30 22:22:06 -07:00
Dan Helfman 64ceebb2c8 Replace cross-thread calls with signals. 2026-05-30 21:12:02 -07:00
Dan Helfman 7439d7cb8f Docstrings. 2026-05-30 16:58:29 -07:00
Dan Helfman f832a361ed More tests. 2026-05-30 13:58:04 -07:00
Dan Helfman 3a4bdfb3c5 Additional tests and docstrings. 2026-05-30 11:27:59 -07:00
Dan Helfman ed955c2a73 Add additional tests. 2026-05-29 19:56:17 -07:00
Dan Helfman 174aa3d522 Additional tests. Also fix a potential race that loses loaded archive paths for display. 2026-05-29 16:59:00 -07:00
Dan Helfman 5a74b3d081 Fix broken test. 2026-05-29 11:57:13 -07:00
Dan Helfman a876ce2058 Yet more tests. 2026-05-29 11:52:02 -07:00
Dan Helfman e82c1bb195 More tests and refactoring. 2026-05-28 20:00:47 -07:00
Dan Helfman 16061f4f6d Fix tests after recent refactoring. 2026-05-28 17:15:13 -07:00
Dan Helfman 59e6e61786 Merge branch 'main' into browse-action 2026-05-28 11:53:23 -07:00
Dan Helfman b7d0f006d6 MariaDB systemd database documentation clarifications (#1303). 2026-05-28 11:50:25 -07:00
Dan Helfman 6869a5d5f2 Additional tests and docstrings. 2026-05-27 22:40:13 -07:00
Dan Helfman 9da6d0f90a Split out panels.py into multiple separate source files. 2026-05-26 23:06:36 -07:00
Dan Helfman ac65e65302 Merge get_archive_path() into get_archive_paths(). 2026-05-26 22:53:04 -07:00
Dan Helfman 8820a1eeab Faster exit for the browse action. 2026-05-26 12:01:55 -07:00
Dan Helfman 3abe025a80 Better performance when bulk populating directory lists. 2026-05-26 11:38:42 -07:00
Dan Helfman b979e0356d Fix display of ".." while loading is still underway in the browse action. 2026-05-25 17:30:06 -07:00
Dan Helfman ea0ed8345a Performance: Switch browse action to list archive files only once per archive. 2026-05-25 16:27:36 -07:00
Dan Helfman 4b8a6d1a34 More browse tests. 2026-05-24 19:22:50 -07:00
Dan Helfman 60201541e0 Add Textual to stand-alone binary to support the browse action. 2026-05-24 10:16:36 -07:00
Dan Helfman bfd79a5500 Refuse to preview named pipes. 2026-05-23 23:05:28 -07:00
Dan Helfman 43e995a4f3 Binary versus text file detection for borgmatic browse. 2026-05-23 22:50:51 -07:00
Dan Helfman 94a2d68198 Switch from a Static to a RichLog for file preview so we can get scrolling. 2026-05-23 19:20:24 -07:00
Dan Helfman c9ba0762f0 Add docstring. 2026-05-23 18:31:24 -07:00
Dan Helfman 13031a4ce4 Upgrade/add test requirements. 2026-05-22 20:03:32 -07:00
Dan Helfman 9a2e853ea7 Merge branch 'main' into browse-action 2026-05-22 20:01:50 -07:00
Dan Helfman b1f42eb233 Upgrade dependencies. 2026-05-22 20:00:40 -07:00
Dan Helfman 7e7c3c273e Update the Apprise monitoring hook's "url" option to support loading credentials with the "{credential ...}" syntax (#1308). 2026-05-22 15:25:42 -07:00
Dan Helfman 704db9c9d7 Add a "--quick-stats" flag and corresponding "quick_statistics" option (#1309). 2026-05-22 14:12:46 -07:00
Dan Helfman 61d86c55b4 Add missing Rich library to test requirements. 2026-05-22 13:44:38 -07:00
Dan Helfman ecedccadee Yet another NEWS wording tweak. 2026-05-22 13:43:44 -07:00
Dan Helfman 006825838f NEWS tweak. 2026-05-22 13:39:45 -07:00
Dan Helfman 1102bf3787 Merge branch 'main' into browse-action 2026-05-22 13:37:44 -07:00
Dan Helfman 220c76b3c0 Formatting. 2026-05-22 13:37:24 -07:00
Dan Helfman 9aea0b1d90 Fix broken tests. 2026-05-22 12:19:34 -07:00
Dan Helfman b6ab326bb4 Switch to PEP 639 license format. 2026-05-22 10:26:12 -07:00
Dan Helfman c4170bb126 Several browse action unit and integration tests. 2026-05-21 23:23:19 -07:00
Dan Helfman 9935d0ef8d Linting and more file refactoring for the browse action. 2026-05-21 17:01:07 -07:00
Dan Helfman ec11e8817d Code formatting. 2026-05-21 16:08:10 -07:00
Dan Helfman 73222e7b5b Factor out make_next_panel() out of browse action widgets into a plain function. 2026-05-21 15:50:34 -07:00
Dan Helfman fd98f4fdd1 Browse action documentation tweaks. 2026-05-21 14:10:49 -07:00
Dan Helfman 16e743d8d9 More browse action option list keys. 2026-05-21 11:41:44 -07:00
Dan Helfman e93fe4cc15 Browse action refactoring into separate files. 2026-05-21 11:33:05 -07:00
Dan Helfman da0698f6e5 Browse action logging improvements. 2026-05-21 10:08:42 -07:00
Dan Helfman 5075c1e539 Skip the browse action's configuration list if there's only one config file. 2026-05-20 23:10:29 -07:00
Dan Helfman 4b4851e78a Add browse action documentation. 2026-05-20 22:57:52 -07:00
Dan Helfman 8e91af6820 Make Textual an optional dependency. 2026-05-20 22:17:19 -07:00
Dan Helfman 9be1698762 Add syntax highlighting to previews. 2026-05-20 21:00:16 -07:00
Dan Helfman 9310b42d9f Add a few more browse action bindings. 2026-05-20 20:16:30 -07:00
Dan Helfman d1d757a5d1 Fix focus on file preview widgets so key bindings for them work in the browse action. 2026-05-20 20:01:03 -07:00
Dan Helfman 3f4a87517b Fix browse action option list highlights. 2026-05-20 19:40:47 -07:00
Dan Helfman a2dcd4747f Get rid of automatic preview panels in the browse action; they're just too slow and annoying in real-world use. 2026-05-20 17:10:00 -07:00
Dan Helfman 0396a89f72 Various browse action improvements. 2026-05-20 16:38:51 -07:00
Dan Helfman f170ef19df Switch to a generalized "carousel" for the borgmatic browse panels. 2026-05-17 16:10:58 -07:00
Dan Helfman 9729809f6f Add the first level of file loading. 2026-05-16 23:04:30 -07:00
Dan Helfman dfb3c0830e Convert browse action UI from a tree to a series of option lists. 2026-05-16 22:06:39 -07:00
Dan Helfman b22a068a60 Put browse action archive files into an actual tree hierarchy. 2026-05-14 23:26:09 -07:00
Dan Helfman 1286ccce46 Browse action MVC refactoring into multiple files. 2026-05-14 22:53:59 -07:00
Dan Helfman e339ce2fd1 Initial work on a browse action. 2026-05-14 22:24:06 -07:00
Dan Helfman aaf5812a3f Cosmetic function argument ordering fix (#1256). 2026-05-12 22:13:05 -07:00
Dan Helfman 47a6691886 Upgrade dependencies. 2026-05-12 15:29:56 -07:00
Dan Helfman 596d59ef60 Fix a race condition in which borgmatic sometimes swallows Borg error output without logging it (#1256). 2026-05-12 15:28:31 -07:00
Dan Helfman 9839b3dada Fix end-to-end database tests (#1303). 2026-05-09 11:05:28 -07:00
Dan Helfman 411685280c For MariaDB/MySQL, dump events, routines, and tablespaces. For MariaDB, dump a subset of system data when dumping "mysql"/"all" (#1303). 2026-05-08 09:38:13 -07:00
Dan Helfman cce679248f Clarify "umask" option semantics in regards to integer and octal (#1305). 2026-05-06 14:57:43 -07:00
Dan Helfman 68c9516424 Upgrade Apprise dependency. 2026-05-02 19:08:24 -07:00
Dan Helfman a508cabe3f Update the KeePassXC credential hook to support KeePassXC's secret service integration. 2026-04-29 12:52:27 -07:00
Dan Helfman 0e1659bd73 Expand the "patterns_from" and "exclude_from" options to support paths containing tildes and globs (#1301). 2026-04-27 10:13:21 -07:00
Dan Helfman ad61ad356e Update the documentation to use Pagefind's component-based search UI. 2026-04-26 09:10:07 -07:00
Dan Helfman 5c7d03910b Fix the "source_directories_must_exist" option to support source directories relative to a "working_directory" (#1300). 2026-04-25 13:49:55 -07:00
Dan Helfman 468af1de0b Fix the PostgreSQL database hook to properly parse "*options" values containing quoted spaces. 2026-04-24 19:24:44 -07:00
Dan Helfman 524d8263a7 Upgrade dependencies. 2026-04-24 17:04:29 -07:00
Dan Helfman 1d2aea0951 Add a development script for upgrading pinned dependencies. 2026-04-24 11:54:37 -07:00
Dan Helfman 961ff7c724 For the MariaDB and MySQL database hooks, escape quotes in passwords when the "password_transport" option is "pipe". 2026-04-24 10:24:45 -07:00
Dan Helfman 3e80056956 Add projects.torsion.org reply by email to NEWS. 2026-04-20 21:52:22 -07:00
Dan Helfman d8f558ce0d Add the NEWS changelog file to release tarball (#1298). 2026-04-20 15:42:17 -07:00
Dan Helfman 9da75fdc58 Use pytest tmp_path fixture instead of hardcoded /tmp paths (#1296).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1296
2026-04-20 18:10:02 +00:00
Dan Helfman ac9c8bb644 Fix build of standalone binary. 2026-04-20 10:28:57 -07:00
Sirio Balmelli 0b34ef0e1a tests: use pytest tmp_path fixture instead of hardcoded /tmp paths
tmp_path is a built-in pytest fixture providing a unique per-test tempdir.

Hardcoded /tmp paths break when multiple builders run tests concurrently
(e.g. Nix parallel builds): leftover or colliding files cause spurious failures.

Signed-off-by: Sirio Balmelli <sirio@b-ad.ch>
2026-04-15 16:35:41 +02:00
138 changed files with 5756 additions and 464 deletions
+1
View File
@@ -1,4 +1,5 @@
# This file only applies to the source dist tarball, not the built wheel.
include NEWS
include borgmatic/config/schema.yaml
graft docs
graft sample
+56
View File
@@ -1,3 +1,59 @@
2.1.7.dev0
* #1309: Add support for the "--quick-stats" flag and the "quick_statistics" option to the "prune"
action. Borg >= 1.4.5 and < 2 only.
* #1317: Add an "archive_hostname" option and a corresponding "--archive-hostname" flag for
overriding the hostname used for the "{hostname}" placeholder in the "archive_name_format"
option. Also add an "archive_username" option and corresponding "--archive-username" flag to
override the "{user}" plaecholder. Both options/flags are Borg 1.4.5+ only.
* #1319: Fix the ZFS hook's overzealous unmounting of snapshot paths when a source dataset is at
"/".
* #1322: Fix for the "restore" action sometimes failing to find a database dump that was dumped
with a default port.
* #1324: For the MariaDB and MySQL hooks, add "events", "routines", and "tablespaces" options for
disabling dumping of scheduled events, stored routines, and tablespaces, respectively.
* #1327: Fix an error from the "diff" action when exclude options are configured.
* #1331: Fix the "repo-create" action to more surgically suppress Borg "Repository does not exist"
logs and avoid inadvertently suppressing other error logs.
* #1333: Fix the "--progress" flag on the "compact" action to actually update the progress of
segment compaction.
* #1334: Add the "CAP_FOWNER" capability to "CapabilityBoundingSet" in the sample systemd service,
so that Borg can open source files without changing file access times.
* Fix a bug in which the "compact" action does not pass a compact threshold of zero to Borg.
2.1.6
* #1256: Fix a race condition in which borgmatic sometimes swallows Borg error output without
logging it.
* #1300: Fix the "source_directories_must_exist" option to support source directories relative to a
"working_directory".
* #1301: Expand the "patterns_from" and "exclude_from" options to support paths containing tildes
and globs.
* #1303: For the MariaDB and MySQL database hooks, include events, routines, and tablespaces when
dumping a database.
* #1303: For the MariaDB hook, include only a subset of system data when dumping the "mysql" system
database (or "all" databases), so the dump is actually restorable. See the documentation for more
information: https://torsion.org/borgmatic/reference/configuration/data-sources/mariadb/
* #1308: Update the Apprise monitoring hook's "url" option to support loading credentials with the
"{credential ...}" syntax. See the documentation for more information:
https://torsion.org/borgmatic/reference/configuration/credentials/
* #1309: Add a "--quick-stats" flag and corresponding "quick_statistics" option for showing only
abbreviated statistics for the "create" action. Borg 1.4.5+ only.
* #1314: Add a minimal stand-alone borgmatic binary in addition to the standard one. The minimal
binary omits support for the Apprise monitoring hook.
* Add an experimental "browse" action providing a console UI for browsing your backups. See the
documentation for more information:
https://torsion.org/borgmatic/how-to/inspect-your-backups/#browsing-backups
* Update the KeePassXC credential hook to support KeePassXC's secret service integration. See the
documentation for more information:
https://torsion.org/borgmatic/reference/configuration/credentials/keepassxc/
* Add the NEWS changelog file to release tarball (#1298).
* Enable reply by email on projects.torsion.org, so replies to notification emails get posted as
comments on tickets.
* For the MariaDB and MySQL database hooks, escape quotes in passwords when the
"password_transport" option is "pipe".
* Add a development script for upgrading pinned dependencies.
* Fix the PostgreSQL database hook to properly parse "*options" values containing quoted spaces.
* Update the documentation to use Pagefind's component-based search UI.
2.1.5
* #1229: Document the permissions needed for the PostgreSQL database hook:
https://torsion.org/borgmatic/reference/configuration/data-sources/postgresql/
+5 -5
View File
@@ -106,16 +106,16 @@ reference guides</a>.
## Hosting providers
Need somewhere to store your encrypted off-site backups? The following hosting
providers include specific support for Borg/borgmatic—and fund borgmatic
development and hosting when you use these referral links to sign up:
provider includes specific support for Borg/borgmatic—and funds borgmatic
development and hosting when you use this referral links to sign up:
<ul>
<li class="referral"><a href="https://www.borgbase.com/?utm_source=borgmatic">BorgBase</a>: Borg hosting service with support for monitoring, 2FA, and append-only repos</li>
<li class="referral"><a href="https://hetzner.cloud/?ref=v9dOJ98Ic9I8">Hetzner</a>: A "storage box" that includes support for Borg</li>
</ul>
Additionally, rsync.net has a compatible storage offering, but does not fund
borgmatic development or hosting.
Additionally, Hetzner and rsync\.net have compatible storage offerings, but do
not fund borgmatic development or hosting.
## Support and contributing
+19
View File
@@ -0,0 +1,19 @@
.
apprise
attrs
certifi
charset-normalizer
click
idna
jsonschema
jsonschema-specifications
markdown
oauthlib
packaging
pyyaml
referencing
requests
requests-oauthlib
rpds-py
ruamel-yaml
urllib3
+21 -19
View File
@@ -1,19 +1,21 @@
.
apprise==1.9.9
attrs==26.1.0
certifi==2026.2.25
charset-normalizer==3.4.6
click==8.3.1
idna==3.11
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
markdown==3.10.2
oauthlib==3.3.1
packaging==26.0
pyyaml==6.0.3
referencing==0.37.0
requests==2.33.1
requests-oauthlib==2.0.0
rpds-py==0.30.0
ruamel-yaml==0.19.1
urllib3==2.6.3
# This file was autogenerated by uv via the following command:
# uv pip compile --annotation-style line binary_requirements.in -o binary_requirements.txt
apprise==1.10.0 # via -r binary_requirements.in
attrs==26.1.0 # via jsonschema, referencing, -r binary_requirements.in
. # via -r binary_requirements.in
certifi==2026.5.20 # via apprise, requests, -r binary_requirements.in
charset-normalizer==3.4.7 # via requests, -r binary_requirements.in
click==8.4.1 # via apprise, -r binary_requirements.in
idna==3.16 # via requests, -r binary_requirements.in
jsonschema==4.26.0 # via borgmatic, -r binary_requirements.in
jsonschema-specifications==2025.9.1 # via jsonschema, -r binary_requirements.in
markdown==3.10.2 # via apprise, -r binary_requirements.in
oauthlib==3.3.1 # via requests-oauthlib, -r binary_requirements.in
packaging==26.2 # via borgmatic, -r binary_requirements.in
pyyaml==6.0.3 # via apprise, -r binary_requirements.in
referencing==0.37.0 # via jsonschema, jsonschema-specifications, -r binary_requirements.in
requests==2.34.2 # via apprise, borgmatic, requests-oauthlib, -r binary_requirements.in
requests-oauthlib==2.0.0 # via apprise, -r binary_requirements.in
rpds-py==0.30.0 # via jsonschema, referencing, -r binary_requirements.in
ruamel-yaml==0.19.1 # via borgmatic, -r binary_requirements.in
urllib3==2.7.0 # via requests, -r binary_requirements.in
+108
View File
@@ -0,0 +1,108 @@
import signal
import textual.app
import textual.binding
import textual.widgets
import borgmatic.actions.browse.carousel
import borgmatic.actions.browse.configuration_files_list
import borgmatic.actions.browse.logs
import borgmatic.actions.browse.repositories_list
class Browse_app(textual.app.App):
'''
The main app / entry point for the browse action UI.
'''
BINDINGS = (
textual.binding.Binding(key='q', action='quit', description='quit'),
textual.binding.Binding(key='v', action='toggle_logs', description='view logs'),
textual.binding.Binding(
key='c', action='command_palette', description='commands', show=False
),
)
COMMAND_PALETTE_BINDING = 'c'
CSS = '''
.panel {
border: round $primary;
border-title-color: $text-primary;
width: 100%;
height: 100%;
}
#logs {
width: 100%;
height: 50%;
display: none;
}
'''
def __init__(self, configs):
self.configs = configs
super().__init__()
def get_system_commands(self, screen): # pragma: no cover
'''
Remove the screenshot system command because it produces broken screenshots. (Emoji
weirdness, etc.) Also remove minimize and maximize because they don't accomplish much with
out particular layout.
'''
yield from (
command
for command in super().get_system_commands(screen)
if command.title not in {'Screenshot', 'Minimize', 'Maximize'}
)
def compose(self):
'''
Compose a UI consisting of:
* a header with the application name
* a carousel container that contains the main UI panels
* a logs panel where Python logs show up (panel hidden by default)
* a footer with available keys listed
'''
yield textual.widgets.Header()
yield borgmatic.actions.browse.carousel.Carousel(
[
borgmatic.actions.browse.configuration_files_list.Configuration_files_list(
self.configs
)
]
if len(self.configs) > 1
else [
borgmatic.actions.browse.repositories_list.Repositories_list(
next(iter(self.configs.values()))
)
]
)
logs_panel = borgmatic.actions.browse.logs.Logs()
yield logs_panel
yield textual.widgets.Footer()
borgmatic.actions.browse.logs.log_to_widget(logs_panel)
def on_mount(self):
'''
Set the application title, which ends up in the header.
'''
self.title = 'borgmatic browse'
def action_toggle_logs(self):
'''
Toggle the show/hide status of the logs panel.
'''
logs_panel = self.query_one('#logs')
logs_panel.styles.display = 'none' if logs_panel.styles.display == 'block' else 'block'
def exit(self): # pragma: no cover
'''
Exit the application. But first raise a SIGTERM (handled in borgmatic/signals.py) to
encourage a fast exit by killing any ongoing Borg subprocesses.
'''
signal.raise_signal(signal.SIGTERM)
super().exit()
+143
View File
@@ -0,0 +1,143 @@
import argparse
import collections
import enum
import json
import logging
import binaryornot.helpers
import borgmatic.borg.extract
import borgmatic.borg.list
import borgmatic.borg.repo_list
import borgmatic.borg.version
logger = logging.getLogger(__name__)
class Path_type(enum.Enum):
DIRECTORY = 'd'
LINK = 'l'
PIPE = 'p'
FILE = '-'
# A data structure capturing a path stored in a Borg archive.
Archive_path = collections.namedtuple(
'Archive_path',
('path_type', 'file_path', 'link_target'),
)
def get_repository_archives(config, repository):
'''
Given a configuration dict and a repository dict, return a list of the repository's archives,
one dict per archive.
'''
with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
logger.info('Listing repository')
repo_list_arguments = argparse.Namespace(
repository=repository['path'],
short=None,
format=None,
json=True,
prefix=None,
match_archives=None,
sort_by=None,
first=None,
last=None,
)
global_arguments = argparse.Namespace()
local_path = config.get('local_path', 'borg')
remote_path = config.get('remote_path')
local_borg_version = borgmatic.borg.version.local_borg_version(config, local_path)
return json.loads(
borgmatic.borg.repo_list.list_repository(
repository['path'],
config,
local_borg_version,
repo_list_arguments,
global_arguments,
local_path,
remote_path,
)
)
def get_archive_paths(config, repository, archive_name):
'''
Given a configuration dict, a repository dict, and an archive name in that repository, get a
generator of files, directories, symlinks, etc. found in the archive, each as an Archive_path
instance.
'''
with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
logger.info(f'Listing archive {archive_name}')
global_arguments = argparse.Namespace()
local_path = config.get('local_path', 'borg')
remote_path = config.get('remote_path')
local_borg_version = borgmatic.borg.version.local_borg_version(config, local_path)
return (
Archive_path(
path_data['type'],
path_data['path'],
path_data.get('linktarget'),
)
for path_data in borgmatic.borg.list.capture_archive_listing(
repository['path'],
archive_name,
config,
local_borg_version,
global_arguments,
local_path=local_path,
remote_path=remote_path,
)
)
READLINES_HINT_BYTES = 100000
TRUNCATION_MESSAGE = '[... truncated for display ...]'
def get_archive_file_content(config, repository, archive_name, file_path):
'''
Given a configuration dict, a repository dict, an archive name in that repository, and a file
path in that archive, return the file's contents or None if the file can't be loaded, e.g.
because it's binary or can't be decoded.
If the file is too large, then truncate the returned content.
'''
with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
logger.info(f'Getting archive content of file {file_path}')
local_path = config.get('local_path', 'borg')
remote_path = config.get('remote_path')
lines = borgmatic.borg.extract.extract_archive(
dry_run=False,
repository=repository['path'],
archive=archive_name,
paths=(file_path,),
config=config,
local_borg_version=borgmatic.borg.version.local_borg_version(config, local_path),
global_arguments=argparse.Namespace(),
local_path=local_path,
remote_path=remote_path,
destination_path=None,
strip_components=None,
extract_to_stdout=True,
).stdout.readlines(READLINES_HINT_BYTES)
content = b''.join(lines)
if binaryornot.helpers.is_binary_string(content):
return None
try:
return (
content.decode()
if len(content) < READLINES_HINT_BYTES
else f'{content.decode()}\n{TRUNCATION_MESSAGE}'
)
except UnicodeDecodeError:
return None
+89
View File
@@ -0,0 +1,89 @@
import textual.widgets
import borgmatic.actions.browse.bindings
import borgmatic.actions.browse.loading
import borgmatic.actions.browse.workers
class Archives_list(textual.widgets.OptionList):
'''
A widget for selecting a single Borg archive from among the archives in a repository. The item
selection event is handled in a Carousel instance, the parent widget of an Archives_list.
'''
BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
def __init__(self, config, repository):
'''
Given a configuration dict and a repository dict, prepare to load the archives from the
repository for eventual display in this widget. Actual loading kicks off in on_mount()
below.
'''
self.config = config
self.repository = repository
super().__init__(classes='panel')
self.border_title = '📚 archives'
self.highlighted_option_changed = False
self.archive_loaded = borgmatic.actions.browse.workers.Archive_loaded(
self, 'archive loaded'
)
self.loading_timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
def on_mount(self):
'''
When this widget gets mounted in the DOM, subscribe to archive loaded events so that we can
find out about archives as they load. Also start loading archives from the repository.
Loading is started *after* subscribing to the archive loaded signal so that there's not a
gap where we might miss out on signal publishes.
'''
self.archive_loaded.subscribe(self, self.on_archive_loaded)
borgmatic.actions.browse.workers.add_repository_archives(
self.app,
archive_loaded=self.archive_loaded,
config=self.config,
repository=self.repository,
)
def on_archive_loaded(self, archive_name):
'''
When an archive loads, add it as an option to this archives list. But if we get a
signal that all path loading is complete, stop and remove our loading indicator.
'''
if archive_name is borgmatic.actions.browse.workers.LOADING_DONE:
self.loading_timer.stop()
self.remove_option('loading-indicator')
return
label_pieces = (
(archive_name, '[dim](latest)[/dim]') if len(self.options) == 1 else (archive_name,)
)
highlighted_option = self.highlighted_option
loading_indicator = self.get_option('loading-indicator')
self.remove_option('loading-indicator')
self.add_options(
(
textual.widgets.option_list.Option(' '.join(label_pieces), id=archive_name),
loading_indicator,
),
)
# Retain the highlighted option position even as other options load around it.
self.highlighted = (
self.get_option_index(highlighted_option.id)
if highlighted_option and self.highlighted_option_changed
else 0
)
def on_option_list_option_highlighted(self, event):
'''
When the highlighted option changes, record that fact. This flag is consumed in
borgmatic.actions.browse.workers.add_repository_archives() in order to retain the
highlighted option even as other options load around it.
'''
if self.highlighted not in {None, 0}:
self.highlighted_option_changed = True
+22
View File
@@ -0,0 +1,22 @@
import textual.binding
import textual.widgets
OPTION_LIST_BINDINGS = (
*textual.widgets.OptionList.BINDINGS,
textual.binding.Binding(
key='up,k', action='cursor_up', description='scroll up', show=True, priority=True
),
textual.binding.Binding(
key='down,j', action='cursor_down', description='scroll down', show=True, priority=True
),
textual.binding.Binding(
key='pageup', action='page_up', description='page up', show=True, priority=True
),
textual.binding.Binding(
key='pagedown', action='page_down', description='page down', show=True, priority=True
),
textual.binding.Binding(
key='enter', action='select', description='select', show=True, priority=True
),
textual.binding.Binding(key='right,l', action='select', description='select', show=False),
)
+153
View File
@@ -0,0 +1,153 @@
import os
import textual.binding
import textual.containers
import borgmatic.actions.browse.archive
import borgmatic.actions.browse.archives_list
import borgmatic.actions.browse.configuration_files_list
import borgmatic.actions.browse.directory_list
import borgmatic.actions.browse.file_preview
import borgmatic.actions.browse.icons
import borgmatic.actions.browse.repositories_list
def make_next_panel(focused_panel, option_id):
'''
Given a focused panel widget and the selected option ID, return the next panel corresponding to
that selection. This is the mechanism by which the user can successively drill down from
configuration file to repository to archive to root directory to non-root directory or file.
If the particular option ID on the focused panel doesn't have a supported next panel, then
return None.
'''
if isinstance(
focused_panel, borgmatic.actions.browse.configuration_files_list.Configuration_files_list
):
return borgmatic.actions.browse.repositories_list.Repositories_list(
config=focused_panel.configs[option_id]
)
if isinstance(focused_panel, borgmatic.actions.browse.repositories_list.Repositories_list):
return borgmatic.actions.browse.archives_list.Archives_list(
config=focused_panel.config, repository=focused_panel.repositories[option_id]
)
if isinstance(focused_panel, borgmatic.actions.browse.archives_list.Archives_list):
return borgmatic.actions.browse.directory_list.Directory_list(
config=focused_panel.config, repository=focused_panel.repository, archive_name=option_id
)
if isinstance(focused_panel, borgmatic.actions.browse.directory_list.Directory_list):
option = focused_panel.get_option(option_id)
if option.prompt.startswith(
borgmatic.actions.browse.icons.PATH_TYPE_ICONS[
borgmatic.actions.browse.archive.Path_type.DIRECTORY.value
]
):
return borgmatic.actions.browse.directory_list.Directory_list(
focused_panel.config,
focused_panel.repository,
focused_panel.archive_name,
path_loaded=focused_panel.path_loaded,
path_components=(*focused_panel.path_components, option_id),
)
if option.prompt.startswith(
borgmatic.actions.browse.icons.PATH_TYPE_ICONS[
borgmatic.actions.browse.archive.Path_type.FILE.value
]
):
return borgmatic.actions.browse.file_preview.File_preview(
focused_panel.config,
focused_panel.repository,
focused_panel.archive_name,
file_path=os.path.sep.join((*focused_panel.path_components, option_id)),
)
return None
class Carousel(textual.containers.Horizontal):
BINDINGS = (
textual.binding.Binding(
key='left,h', action='previous', description='previous', priority=True
),
)
def __init__(self, panels):
self.panels = panels
self.focused_panel = panels[0]
super().__init__()
def compose(self):
'''
Compose with each of the contained panels and focus the first one.
'''
yield from self.panels
self.focused_panel.focus()
def action_previous(self):
'''
Make the previous panel into the focused panel.
'''
previous_panel_index = self.panels.index(self.focused_panel) - 1
if previous_panel_index < 0:
return
self.focused_panel.styles.display = 'none'
self.focused_panel = self.panels[previous_panel_index]
self.focused_panel.styles.display = 'block'
self.focused_panel.focus()
def action_next(self, option_id):
'''
Hide the current focused panel and create the next one.
'''
next_panel_index = self.panels.index(self.focused_panel) + 1
if next_panel_index < len(self.panels):
next_panel = self.panels[next_panel_index]
next_panel.styles.display = 'block'
else:
next_panel = make_next_panel(self.focused_panel, option_id)
if next_panel is None:
self.notify('Cannot display this content', severity='warning')
return
self.panels.append(next_panel)
next_panel.highlighted = 0
self.focused_panel.styles.display = 'none'
self.focused_panel = next_panel
self.focused_panel.focus()
self.mount(self.focused_panel)
def on_option_list_option_highlighted(self, event):
'''
The highlighted option has changed, so truncate any next panels.
'''
next_panel_index = self.panels.index(self.focused_panel) + 1
del self.panels[next_panel_index:]
def on_option_list_option_selected(self, event):
'''
An option has been selected, so advance to the next panel—unless the option selected is
"..", in which case go to the previous panel.
'''
if (
event.option_list != self.focused_panel or event.option_id == 'loading-indicator'
): # pragma: no cover
return
if event.option_id == '..':
self.action_previous()
else:
self.action_next(event.option_id)
@@ -0,0 +1,33 @@
import os
import textual.widgets
import borgmatic.actions.browse.bindings
class Configuration_files_list(textual.widgets.OptionList):
'''
A widget for selecting a single borgmatic configuration file from among available configuration
files. The item selection event is handled in a Carousel instance, the parent widget of an
Configuration_files_list.
'''
BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
def __init__(self, configs):
'''
Given a dict mapping from configuration path to corresponding configuration dict, add each
configuration path as an option to this widget.
'''
self.configs = configs
home_directory = os.path.expanduser('~')
super().__init__(
*(
textual.widgets.option_list.Option(unexpanded_path, id=config_path)
for config_path in configs
for unexpanded_path in (config_path.replace(home_directory, '~'),)
),
classes='panel',
)
self.border_title = '📄 configuration files'
+211
View File
@@ -0,0 +1,211 @@
import os
import textual.widgets
import borgmatic.actions.browse.bindings
import borgmatic.actions.browse.icons
import borgmatic.actions.browse.loading
import borgmatic.actions.browse.workers
def get_relative_archive_path_components(archive_path, current_directory_path_components):
'''
Given an Archive_path instance and a tuple of path components for the currently browsed
directory, get the path components as a tuple for the archive path relative to that directory.
For instance, given an archive path with a path of 'foo/bar/baz/quux.txt' and current
directory path components of ('foo', 'bar'), return ('baz', 'quux.txt').
If the archive path is not actually relative to the current directory, return None.
'''
archive_path_components = tuple(archive_path.file_path.split(os.path.sep))
if not current_directory_path_components:
return archive_path_components
# If the loaded path doesn't match this directory list's own path, then we don't care about
# it for purposes of displaying this particular directory.
if (
tuple(archive_path_components[: len(current_directory_path_components)])
!= current_directory_path_components
):
return None
# Strip off the portion of the archive path that matches the directory list's own path.
return archive_path_components[len(current_directory_path_components) :]
def make_directory_list_option(archive_path, relative_path_components):
'''
Given an Archive_path instance and a tuple of relative path components for it, make a
textual.widgets.option_list.Option for the path. Use an the icon based on whether this looks
like a terminal filename or a directory.
'''
pieces = (
borgmatic.actions.browse.icons.PATH_TYPE_ICONS.get(
archive_path.path_type if len(relative_path_components) == 1 else 'd', ''
),
relative_path_components[0],
) + (('', archive_path.link_target) if archive_path.link_target else ())
return textual.widgets.option_list.Option(
prompt=' '.join(pieces), id=relative_path_components[0]
)
def add_archive_paths(
directory_list,
config,
repository,
archive_name,
archive_paths,
):
'''
Given a DirectoryList instance, a configuration dict, a repository dict, an archive name, and a
sequence of ArchivePath instances, add the paths to the directory list as options, sorting and
deduplicating the resulting directory list's options.
After all of this reshuffling, make sure the original highlighted option remains highlighted.
'''
highlighted_option = directory_list.highlighted_option
original_options_count = len(directory_list.options)
sorted_options = sorted(
(
*directory_list.options,
*(
make_directory_list_option(archive_path, relative_path_components)
for archive_path in archive_paths
for relative_path_components in (
get_relative_archive_path_components(
archive_path,
directory_list.path_components,
),
)
if relative_path_components
if relative_path_components[0] not in directory_list._id_to_option
),
),
# The loading indicator "option" always goes to the bottom.
key=lambda option: ((option.id == 'loading-indicator'), option.prompt),
)
# If there aren't actually any options to add (due to deduplication), bail.
if len(sorted_options) == original_options_count:
return
# Retain the highlighted option position even as other options load around it.
directory_list.set_options(sorted_options)
directory_list.highlighted = (
directory_list.get_option_index(highlighted_option.id)
if highlighted_option and directory_list.highlighted_option_changed
else 0
)
class Directory_list(textual.widgets.OptionList):
'''
A widget for selecting a path from among the contents of a particular directory in a Borg
archive. The item selection event is handled in a Carousel instance, the parent widget of a
Directory_list.
'''
BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
def __init__(self, config, repository, archive_name, path_loaded=None, path_components=None):
'''
Given a configuration dict, a repository dict, an archive name, an optional
Archive_path_loaded instance for signalling new paths as they load, and an optional tuple of
path components indicating this directory's position in the backed up filesystem, prepare to
load paths from the archive for eventual display in this widget. Actual loading kicks off in
on_mount() below.
'''
self.config = config
self.repository = repository
self.archive_name = archive_name
self.path_components = path_components or ()
self.highlighted_option_changed = False
super().__init__(classes='panel')
self.border_title = ' '.join(
(
'📁',
os.path.sep.join(self.path_components)
if self.path_components
else f'{archive_name}',
)
)
if self.path_components:
self.add_option(
textual.widgets.option_list.Option(
'📁 ..',
id='..',
),
)
self.path_loaded = path_loaded or borgmatic.actions.browse.workers.Archive_path_loaded(
self, 'archive path loaded'
)
if not self.path_loaded.complete:
self.timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
def on_mount(self):
'''
When this widget gets mounted in the DOM, subscribe to path loaded events so that we can
find out about relevant archive paths as they load. And if this is a root directory list,
start loading paths from the archive. If this is a non-root directory list, add any already
loaded archive paths to this widget as options.
Loading is started *after* subscribing to path loaded signals so that there's not a gap
where we might miss out on any paths.
'''
self.path_loaded.subscribe(self, self.on_archive_path_loaded)
if self.path_components:
add_archive_paths(
directory_list=self,
config=self.config,
repository=self.repository,
archive_name=self.archive_name,
archive_paths=borgmatic.actions.browse.workers.get_paths(
self.path_loaded.path_hierarchy, self.path_components
),
)
else:
borgmatic.actions.browse.workers.load_archive_paths(
self.app,
path_loaded=self.path_loaded,
config=self.config,
repository=self.repository,
archive_name=self.archive_name,
)
def on_archive_path_loaded(self, data):
'''
When an archive path loads, add it as an option to this directory list. But if we get a
signal that all path loading is complete, stop and remove our loading indicator.
'''
if data is borgmatic.actions.browse.workers.LOADING_DONE:
self.timer.stop()
self.remove_option('loading-indicator')
return
add_archive_paths(
directory_list=self,
config=self.config,
repository=self.repository,
archive_name=self.archive_name,
archive_paths=(data,),
)
def on_option_list_option_highlighted(self, event):
'''
When the highlighted option changes, record that fact. This flag is consumed in
add_archive_paths() in order to retain the highlighted option even as other options load
around it.
'''
if self.highlighted not in {None, 0}:
self.highlighted_option_changed = True
+87
View File
@@ -0,0 +1,87 @@
import logging
import rich.syntax
import textual.binding
import textual.widgets
import borgmatic.actions.browse.loading
import borgmatic.actions.browse.workers
logger = logging.getLogger('__name__')
class File_preview(textual.widgets.RichLog):
'''
A widget for extracting and previewing the contents of a file stored in a Borg archive.
'''
BINDINGS = (
*textual.widgets.RichLog.BINDINGS,
textual.binding.Binding(
key='up,k', action='scroll_up', description='scroll up', show=True, priority=True
),
textual.binding.Binding(
key='down,j', action='scroll_down', description='scroll down', show=True, priority=True
),
textual.binding.Binding(
key='pageup', action='page_up', description='page up', show=True, priority=True
),
textual.binding.Binding(
key='pagedown', action='page_down', description='page down', show=True, priority=True
),
)
def __init__(self, config, repository, archive_name, file_path):
'''
Given a configuration dict, a repository dict, an archive name, and the path of a file in
the archive, prepare to load the file's contents for eventual display in this widget. Actual
loading kicks off in on_mount() below.
'''
self.config = config
self.repository = repository
self.archive_name = archive_name
self.file_path = file_path
super().__init__(classes='panel')
self.border_title = f'📄 {self.file_path} preview'
self.auto_scroll = False
self.file_preview_loaded = borgmatic.actions.browse.workers.File_preview_loaded(
self, 'file preview loaded'
)
self.loading_timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
def on_mount(self):
'''
When this widget gets mounted in the DOM, subscribe to archive loaded events so that we can
find out about archives as they load. Also start loading file contents from the archive.
Loading is started *after* subscribing to the file preview loaded signal so that there's not
a gap where we might miss out on signal publishes.
'''
self.file_preview_loaded.subscribe(self, self.on_file_preview_loaded)
borgmatic.actions.browse.workers.load_file_preview(
self.app,
file_preview_loaded=self.file_preview_loaded,
config=self.config,
repository=self.repository,
archive_name=self.archive_name,
file_path=self.file_path,
)
def on_file_preview_loaded(self, file_contents):
'''
When a file loads, write its contents (syntax highlighted) to this file preview widget.
'''
self.loading_timer.stop()
self.clear()
if file_contents is None:
self.write('Cannot display a preview for this file')
else:
# Only pass the file path and not its contents to guess_lexer(). Passing the contents is
# more accurate, but also much slower.
self.write(
rich.syntax.Syntax(file_contents, rich.syntax.Syntax.guess_lexer(self.file_path))
)
+8
View File
@@ -0,0 +1,8 @@
import borgmatic.actions.browse.archive
PATH_TYPE_ICONS = {
borgmatic.actions.browse.archive.Path_type.DIRECTORY.value: '📁',
borgmatic.actions.browse.archive.Path_type.LINK.value: '🔗',
borgmatic.actions.browse.archive.Path_type.PIPE.value: '🚰',
borgmatic.actions.browse.archive.Path_type.FILE.value: '📄',
}
+54
View File
@@ -0,0 +1,54 @@
import contextlib
import functools
import logging
import textual.widgets
import textual.widgets.option_list
LOADING_DOT_INTERVAL_SECONDS = 0.3
logger = logging.getLogger('__name__')
def update_inline_loading_indicator(widget):
'''
Given a textual.widgets.OptionList or a textual.widgets.RichLog instance, animate the existing
loading indicator inside it.
'''
if isinstance(widget, textual.widgets.OptionList):
with contextlib.suppress(textual.widgets.option_list.OptionDoesNotExist):
widget.replace_option_prompt(
'loading-indicator',
(str(widget.get_option('loading-indicator').prompt) + '.').replace('....', ''),
)
elif isinstance(widget, textual.widgets.RichLog):
with contextlib.suppress(IndexError):
loading_message = str(widget.lines[0].text)
widget.clear()
widget.write((loading_message + '.').replace('....', ''))
else:
raise ValueError(f'Unsupported widget type: {type(widget)}')
LOADING_MESSAGE = '⏳ loading...'
def add_inline_loading_indicator(widget):
'''
Given a textual.widgets.OptionList or a textual.widgets.RichLog instance, add a loading
indicator to it.
'''
if isinstance(widget, textual.widgets.OptionList):
loading_option = textual.widgets.option_list.Option(LOADING_MESSAGE, id='loading-indicator')
widget.add_option(loading_option)
widget.highlighted = None
elif isinstance(widget, textual.widgets.RichLog):
widget.write(LOADING_MESSAGE)
else:
raise ValueError(f'Unsupported widget type: {type(widget)}')
return widget.set_interval(
LOADING_DOT_INTERVAL_SECONDS,
functools.partial(update_inline_loading_indicator, widget),
)
+102
View File
@@ -0,0 +1,102 @@
import contextlib
import logging
import textual._context
import textual.widgets
import textual.worker
import borgmatic.logger
class Rich_color_formatter(logging.Formatter):
'''
A Python logging formatter that formats log records with Rich-compatible color markup according
to their levels.
'''
def __init__(self, *args, **kwargs):
self.prefix = None
super().__init__(
'{prefix}{message}',
*args,
style='{',
**kwargs,
)
def format(self, record):
'''
Given a log record, format it with Rich-compatibe color markup corresponding to its log
level.
'''
borgmatic.logger.add_custom_log_levels()
color = {
logging.CRITICAL: 'bright_red',
logging.ERROR: 'bright_red',
logging.WARNING: 'bright_yellow',
logging.ANSWER: 'bright_magenta',
logging.INFO: 'bright_green',
logging.DEBUG: 'bright_cyan',
}.get(record.levelno)
record.prefix = f'{self.prefix}: ' if self.prefix else ''
return f'[{color}]{super().format(record)}[/{color}]'
class Browse_log_handler(logging.Handler):
'''
A Python log handler that writes any log records to a logging widget.
'''
def __init__(self, logs_widget):
'''
Given a logs widget, save it for use below.
'''
self.logs_widget = logs_widget
super().__init__()
def emit(self, record):
'''
Given a log record, format it and log it to the logs widgets. This works whether or not the
logging is happening in the main thread.
'''
message = self.format(record)
try:
textual.worker.get_current_worker()
self.logs_widget.app.call_from_thread(self.logs_widget.write, message)
except (RuntimeError, textual.worker.NoActiveWorker):
with contextlib.suppress(textual._context.NoActiveAppError):
self.logs_widget.write(message)
def log_to_widget(logs_widget):
'''
Given a Textual RichLog logs widget, add a log handler and formatter that logs to it. Also
remove the default borgmatic console log handler so it doesn't try to log all over our UI.
'''
handler = Browse_log_handler(logs_widget)
handler.setFormatter(Rich_color_formatter())
logger = logging.getLogger()
logger.setLevel(min(handler.level for handler in logger.handlers))
logger.addHandler(handler)
with contextlib.suppress(StopIteration):
console_handler = next(
handler
for handler in logging.getLogger().handlers
if isinstance(handler, borgmatic.logger.Multi_stream_handler)
)
logger.removeHandler(console_handler)
class Logs(textual.widgets.RichLog):
'''
A widget for viewing borgmatic logs in realtime. The log level is determined by borgmatic's
current verbosity level.
'''
def __init__(self):
super().__init__(markup=True, id='logs', classes='panel')
self.border_title = '🪵 logs'
@@ -0,0 +1,30 @@
import textual.widgets
import borgmatic.actions.browse.bindings
class Repositories_list(textual.widgets.OptionList):
'''
A widget for selecting a single Borg repository from among the repositories in a borgmatic
configuration file. The item selection event is handled in a Carousel instance, the parent
widget of a Repositories_list.
'''
BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
def __init__(self, config):
'''
Given a configuration dict, populate the repositories in this widget.
'''
self.config = config
self.repositories = config['repositories']
super().__init__(
*(
textual.widgets.option_list.Option(label, id=index)
for index, repository in enumerate(self.repositories)
for label in (repository.get('label', repository.get('path')),)
),
classes='panel',
)
self.border_title = '📦 repositories'
+29
View File
@@ -0,0 +1,29 @@
import logging
def run_browse(
diff_arguments,
global_arguments,
configs,
):
'''
Run the "browse" action for the given borgmatic configurations. This launches a console UI.
Raise ValueError if the Textual library (a prerequisite for this action) can't be imported.
'''
if not configs:
return
logging.getLogger('asyncio').setLevel(logging.WARNING)
try:
import textual # noqa: F401, PLC0415
except ImportError: # pragma: no cover
raise ValueError(
'Unable to import the Textual library for the browse action; try installing "borgmatic[browse]"'
)
import borgmatic.actions.browse.app # noqa: PLC0415
app = borgmatic.actions.browse.app.Browse_app(configs)
app.run()
+192
View File
@@ -0,0 +1,192 @@
import logging
import os
import textual
import textual.signal
import borgmatic.actions.browse.archive
logger = logging.getLogger('__name__')
LOADING_DONE = object()
class Archive_loaded(textual.signal.Signal):
'''
A signal that publishes when each subsequent archive is loaded from a repository, intended for
consumption in widgets that display archives as they are loaded. This signal also publishes
when loading is complete.
Each subscribed callback call includes the archive as an archive name string. Given the lack of
other identifying information (configuration file, repository), there should be a separate
Archive_loaded instance per repository.
'''
@textual.work(thread=True)
def add_repository_archives(browse_app, archive_loaded, config, repository):
'''
Given a running Browse_app instance, an Archive_loaded instance, a configuration dict, and a
repository dict, load a list of the archives from the repository and add them as options in the
archives list. Reverse the order so the most recent archive is first.
This function runs in a separate thread from the main UI. When loading is complete, publish a
loading done signal.
'''
archives_data = borgmatic.actions.browse.archive.get_repository_archives(config, repository)
# Reverse the archives, so the common case of accessing the latest archive is easy because it's
# at the top.
for archive in reversed(archives_data['archives']):
archive_loaded.publish(archive['archive'])
archive_loaded.publish(LOADING_DONE)
def record_path(archive_path, hierarchy, path_components):
'''
Given an Archive_path instance, a dict capturing a filesystem hierarchy of paths, and a tuple of
path components for the archive path, set the archive path into the hierarchy data structure.
For instance, if given an archive path and path components representing "foo/bar/baz.txt",
produce a hierarchy that looks like:
{'foo': {'bar': {'baz.txt': Archive_path('-', 'foo/bar/baz.txt', '')}}}
Note that the hierarchy is modified in place, so any existing paths there are retained.
'''
if len(path_components) == 1:
hierarchy[path_components[0]] = {} if archive_path.path_type == 'd' else archive_path
return
record_path(archive_path, hierarchy.setdefault(path_components[0], {}), path_components[1:])
def get_paths(hierarchy, path_components, full_path_components=None):
'''
Given a dict capturing a filesystem hierarchy of paths (or a subset thereof), a tuple of path
components for a directory path relative to the hierarchy root, and an optional tuple of
*absolute* path components for the same path (if different), return a generator of the
contained file and directory Archive_path instances from the hierarchy.
For instance, given the following hierarchy:
{'foo': {'bar': {'baz.txt': Archive_path('-', 'foo/bar/baz.txt', ''), 'quux': {}}}}
... and path components of ('foo', 'bar'), return a generator with the following:
* Archive_path('-', 'foo/bar/baz.txt', '')
* Archive_path('d', 'foo/bar/quux', '')
The given absolute path components are use to construct directory paths like that last archive
path.
'''
if full_path_components is None:
full_path_components = path_components
if len(path_components) == 1:
try:
return (
archive_path
if isinstance(archive_path, borgmatic.actions.browse.archive.Archive_path)
else borgmatic.actions.browse.archive.Archive_path(
'd', os.path.join(*full_path_components, component), ''
)
for component, archive_path in hierarchy[path_components[0]].items()
)
except KeyError:
raise ValueError(f'Unknown file or directory: {path_components[0]}')
try:
return get_paths(hierarchy[path_components[0]], path_components[1:], full_path_components)
except KeyError:
raise ValueError(f'Unknown directory: {path_components[0]}')
class Archive_path_loaded(textual.signal.Signal):
'''
A signal that publishes when each subsequent path is loaded from an archive, intended for
consumption in widgets that display paths as they are loaded. This signal also tracks the
complete filesystem hierarchy seen thus far, so new widgets that get created after loading has
started can "catch up" with existing known paths. Lastly, this signal publishes and tracks when
loading is complete.
Each subscrided callback call includes the loaded path as an Archive_path instance. There is
intended to be a separate Archive_path_loaded instance per archive, but that instance should be
shared among several different widgets for the same archive for performance reasons.
'''
def __init__(self, owner, name):
self.path_hierarchy = {}
self.complete = False
super().__init__(owner, name)
def publish(self, archive_path):
'''
Publish the given archive path to subscribers and record its path locally. But if the
archive path is actually LOADING_DONE, then record loading as complete.
'''
super().publish(archive_path)
if archive_path is LOADING_DONE:
self.complete = True
else:
record_path(
archive_path, self.path_hierarchy, archive_path.file_path.split(os.path.sep)
)
@textual.work(thread=True)
def load_archive_paths(browse_app, path_loaded, config, repository, archive_name):
'''
Given a running Browse_app instance, an Archive_path_loaded instance, a configuration dict, a
repository dict, and an archive name, load the paths in this archive and publish each one via
the Archive_path_loaded signal, so interested widgets can subscribe. Also send a "loading done"
signal when loading completes.
This function runs in a separate thread from the main UI. When loading is complete, publish a
loading done signal.
'''
for archive_path in borgmatic.actions.browse.archive.get_archive_paths(
config, repository, archive_name
):
path_loaded.publish(archive_path)
path_loaded.publish(LOADING_DONE)
class File_preview_loaded(textual.signal.Signal):
'''
A signal that publishes when file contents are loaded from an archive, intended for consumption
in widgets that display loaded files. This signal also publishes when loading is complete.
Each published callback includes a the file's contents as a string. Given the lack of other
identifying information (configuration file, repository, archive), there should be a separate
Archive_loaded instance per previewed file.
'''
@textual.work(thread=True)
def load_file_preview(
browse_app,
file_preview_loaded,
config,
repository,
archive_name,
file_path,
):
'''
Given a running Browse_app instance, a File_preview_loaded instance, a configuration dict, a
repository dict, an archive name, and the path of a file in that archive, load the contents of
the file and write it into the given file preview widget.
This function runs in a separate thread from the main UI. When loading is complete, publish a
loading done signal.
'''
file_contents = borgmatic.actions.browse.archive.get_archive_file_content(
config, repository, archive_name, file_path
)
file_preview_loaded.publish(file_contents)
+1 -1
View File
@@ -383,7 +383,7 @@ def collect_spot_check_source_paths(
# Omit "progress" because it interferes with "list_details".
config=dict(config, progress=False, list_details=True),
patterns=borgmatic.actions.pattern.process_patterns(
borgmatic.actions.pattern.collect_patterns(config)
borgmatic.actions.pattern.collect_patterns(config, working_directory)
+ tuple(
borgmatic.borg.pattern.Pattern(
config_path,
+1 -1
View File
@@ -45,7 +45,7 @@ def run_create(
with borgmatic.config.paths.Runtime_directory(config) as borgmatic_runtime_directory:
patterns = pattern.process_patterns(
pattern.collect_patterns(config),
pattern.collect_patterns(config, working_directory),
config,
working_directory,
borgmatic_runtime_directory,
+3 -2
View File
@@ -21,10 +21,11 @@ def run_diff(
# Only process patterns if only_patterns flag is set
if diff_arguments.only_patterns:
working_directory = borgmatic.config.paths.get_working_directory(config)
processed_patterns = borgmatic.actions.pattern.process_patterns(
(*borgmatic.actions.pattern.collect_patterns(config),),
(*borgmatic.actions.pattern.collect_patterns(config, working_directory),),
config,
borgmatic.config.paths.get_working_directory(config),
working_directory,
)
else:
processed_patterns = None
+8 -5
View File
@@ -34,10 +34,11 @@ def parse_pattern(pattern_line, default_style=borgmatic.borg.pattern.Pattern_sty
)
def collect_patterns(config):
def collect_patterns(config, working_directory):
'''
Given a configuration dict, produce a single sequence of patterns comprised of the configured
source directories, patterns, excludes, pattern files, and exclude files.
Given a configuration dict and the working directory, produce a single sequence of patterns
comprised of the configured source directories, patterns, excludes, pattern files, and exclude
files.
The idea is that Borg has all these different ways of specifying includes, excludes, source
directories, etc., but we'd like to collapse them all down to one common format (patterns) for
@@ -68,7 +69,8 @@ def collect_patterns(config):
+ tuple(
parse_pattern(pattern_line.strip())
for filename in config.get('patterns_from', ())
for pattern_line in open(filename, encoding='utf-8')
for expanded_path in expand_directory(filename, working_directory)
for pattern_line in open(expanded_path, encoding='utf-8')
if not pattern_line.lstrip().startswith('#')
if pattern_line.strip()
)
@@ -78,7 +80,8 @@ def collect_patterns(config):
borgmatic.borg.pattern.Pattern_style.FNMATCH,
)
for filename in config.get('exclude_from', ())
for exclude_line in open(filename, encoding='utf-8')
for expanded_path in expand_directory(filename, working_directory)
for exclude_line in open(expanded_path, encoding='utf-8')
if not exclude_line.lstrip().startswith('#')
if exclude_line.strip()
)
+4 -2
View File
@@ -30,10 +30,12 @@ def run_recreate(
else:
logger.answer(f'Recreating repository{dry_run_label}')
working_directory = borgmatic.config.paths.get_working_directory(config)
# Collect and process patterns.
processed_patterns = borgmatic.actions.pattern.process_patterns(
(
*borgmatic.actions.pattern.collect_patterns(config),
*borgmatic.actions.pattern.collect_patterns(config, working_directory),
# Also add borgmatic-specific paths, so they don't get excluded from the recreated
# archive. Note that this doesn't currently work for archives created with Borg 1.2 or
# below.
@@ -42,7 +44,7 @@ def run_recreate(
),
),
config,
borgmatic.config.paths.get_working_directory(config),
working_directory,
)
archive = borgmatic.borg.repo_list.resolve_archive_name(
+1 -1
View File
@@ -539,7 +539,7 @@ def run_restore(
with borgmatic.config.paths.Runtime_directory(config) as borgmatic_runtime_directory:
patterns = borgmatic.actions.pattern.process_patterns(
borgmatic.actions.pattern.collect_patterns(config),
borgmatic.actions.pattern.collect_patterns(config, working_directory),
config,
working_directory,
)
+1 -2
View File
@@ -29,7 +29,6 @@ def run_arbitrary_borg(
command.
'''
borgmatic.logger.add_custom_log_levels()
lock_wait = config.get('lock_wait', None)
try:
options = options[1:] if options[0] == '--' else options
@@ -53,7 +52,7 @@ def run_arbitrary_borg(
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ flags.make_flags('remote-path', remote_path)
+ flags.make_flags('lock-wait', lock_wait)
+ flags.make_flags('lock-wait', config.get('lock_wait'))
+ command_options
)
+2 -2
View File
@@ -21,8 +21,8 @@ def break_lock(
argparse.Namespace of global arguments, and optional local and remote Borg paths, break any
repository and cache locks leftover from Borg aborting.
'''
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get('break_lock', '')
full_command = (
+2 -2
View File
@@ -24,8 +24,8 @@ def change_passphrase(
based on an interactive prompt.
'''
borgmatic.logger.add_custom_log_levels()
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get('key_change_passphrase', '')
full_command = (
+1 -1
View File
@@ -20,7 +20,7 @@ def make_archive_filter_flags(local_borg_version, config, checks, check_argument
flag. And if "prefix" is set in configuration and "archives" is in checks, then include a
"--match-archives" flag.
'''
check_last = config.get('check_last', None)
check_last = config.get('check_last')
prefix = config.get('prefix')
if 'archives' in checks or 'data' in checks:
+5 -4
View File
@@ -3,7 +3,7 @@ import shlex
import borgmatic.config.paths
from borgmatic.borg import environment, feature, flags
from borgmatic.execute import execute_command
from borgmatic.execute import DO_NOT_CAPTURE, execute_command
logger = logging.getLogger(__name__)
@@ -22,8 +22,8 @@ def compact_segments(
Given dry-run flag, a local or remote repository path, a configuration dict, and the local Borg
version, compact the segments in a repository.
'''
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get('compact', '')
threshold = config.get('compact_threshold')
@@ -35,7 +35,7 @@ def compact_segments(
+ (('--lock-wait', str(lock_wait)) if lock_wait else ())
+ (('--progress',) if config.get('progress') else ())
+ (('--cleanup-commits',) if cleanup_commits else ())
+ (('--threshold', str(threshold)) if threshold else ())
+ (('--threshold', str(threshold)) if threshold is not None else ())
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
+ (
@@ -54,6 +54,7 @@ def compact_segments(
execute_command(
full_command,
output_log_level=logging.INFO,
output_file=DO_NOT_CAPTURE if config.get('progress') else None,
environment=environment.make_environment(config),
working_directory=borgmatic.config.paths.get_working_directory(config),
borg_local_path=local_path,
+19 -10
View File
@@ -180,20 +180,22 @@ def make_base_create_command( # noqa: PLR0912
open pattern file handle).
'''
if config.get('source_directories_must_exist', True):
borgmatic.borg.pattern.check_all_root_patterns_exist(patterns)
borgmatic.borg.pattern.check_all_root_patterns_exist(
patterns, borgmatic.config.paths.get_working_directory(config)
)
patterns_file = borgmatic.borg.pattern.write_patterns_file(
patterns,
borgmatic_runtime_directory,
)
checkpoint_interval = config.get('checkpoint_interval', None)
checkpoint_volume = config.get('checkpoint_volume', None)
chunker_params = config.get('chunker_params', None)
compression = config.get('compression', None)
upload_rate_limit = config.get('upload_rate_limit', None)
upload_buffer_size = config.get('upload_buffer_size', None)
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
checkpoint_interval = config.get('checkpoint_interval')
checkpoint_volume = config.get('checkpoint_volume')
chunker_params = config.get('chunker_params')
compression = config.get('compression')
upload_rate_limit = config.get('upload_rate_limit')
upload_buffer_size = config.get('upload_buffer_size')
umask = config.get('umask')
lock_wait = config.get('lock_wait')
list_filter_flags = flags.make_list_filter_flags(local_borg_version, dry_run)
files_changed = config.get('files_changed')
files_cache = config.get('files_cache')
@@ -372,7 +374,9 @@ def create_archive(
if json:
output_log_level = None
elif config.get('list_details') or (config.get('statistics') and not dry_run):
elif config.get('list_details') or (
(config.get('statistics') or config.get('quick_statistics')) and not dry_run
):
output_log_level = logging.ANSWER
else:
output_log_level = logging.INFO
@@ -384,6 +388,11 @@ def create_archive(
create_flags += (
(('--info',) if logger.getEffectiveLevel() == logging.INFO and not json else ())
+ (('--stats',) if config.get('statistics') and not json and not dry_run else ())
+ (
('--quick-stats',)
if config.get('quick_statistics') and not json and not dry_run
else ()
)
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) and not json else ())
+ (('--progress',) if config.get('progress') else ())
+ (('--json',) if json else ())
+1 -3
View File
@@ -31,8 +31,7 @@ def diff(
'''
borgmatic.logger.add_custom_log_levels()
lock_wait = config.get('lock_wait', None)
exclude_flags = flags.make_exclude_flags(config)
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get('diff', '')
if diff_arguments.only_patterns:
@@ -63,7 +62,6 @@ def diff(
if patterns_file and diff_arguments.only_patterns
else ()
)
+ exclude_flags
+ numeric_ids_flags
+ (('--same-chunker-params',) if diff_arguments.same_chunker_params else ())
+ (('--sort-by', ','.join(diff_arguments.sort_keys)) if diff_arguments.sort_keys else ())
+2
View File
@@ -4,6 +4,8 @@ import borgmatic.borg.passcommand
import borgmatic.hooks.credential.parse
OPTION_TO_ENVIRONMENT_VARIABLE = {
'archive_hostname': 'BORG_HOSTNAME',
'archive_username': 'BORG_USERNAME',
'borg_base_directory': 'BORG_BASE_DIR',
'borg_config_directory': 'BORG_CONFIG_DIR',
'borg_cache_directory': 'BORG_CACHE_DIR',
+2 -2
View File
@@ -29,8 +29,8 @@ def export_key(
Raise FileExistsError if a path is given but it already exists on disk.
'''
borgmatic.logger.add_custom_log_levels()
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
working_directory = borgmatic.config.paths.get_working_directory(config)
extra_borg_options = config.get('extra_borg_options', {}).get('key_export', '')
+2 -2
View File
@@ -33,8 +33,8 @@ def export_tar_archive(
If the destination path is "-", then stream the output to stdout instead of to a file.
'''
borgmatic.logger.add_custom_log_levels()
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get('export_tar', '')
full_command = (
+2 -2
View File
@@ -104,8 +104,8 @@ def extract_archive(
If extract to stdout is True, then start the extraction streaming to stdout, and return that
extract process as an instance of subprocess.Popen.
'''
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get('extract', '')
if feature.available(feature.Feature.NUMERIC_IDS, local_borg_version):
+2 -2
View File
@@ -27,8 +27,8 @@ def import_key(
Raise ValueError if the path is given and it does not exist.
'''
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
working_directory = borgmatic.config.paths.get_working_directory(config)
extra_borg_options = config.get('extra_borg_options', {}).get('key_import', '')
+3 -3
View File
@@ -115,10 +115,10 @@ def capture_archive_listing(
Given a local or remote repository path, an archive name, a configuration dict, the local Borg
version, global arguments as an argparse.Namespace, the archive paths (or Borg patterns) in
which to list files, the Borg path format indicating keys to include in the output, and local
and remote Borg paths, capture the output of listing that archive and return it as a sequence of
dicts, one per path.
and remote Borg paths, capture the output of listing that archive and return it as a generator
of dicts, one per path.
'''
return tuple(
return (
json.loads(entry)
for entry in execute_command_and_capture_output(
make_list_command(
+2 -2
View File
@@ -24,8 +24,8 @@ def mount_archive(
dict, the local Borg version, global arguments as an argparse.Namespace instance, and optional
local and remote Borg paths, mount the archive onto the mount point.
'''
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get('mount', '')
full_command = (
+4 -4
View File
@@ -88,16 +88,16 @@ def write_patterns_file(patterns, borgmatic_runtime_directory, patterns_file=Non
return patterns_file
def check_all_root_patterns_exist(patterns):
def check_all_root_patterns_exist(patterns, working_directory):
'''
Given a sequence of Pattern instances, check that all root pattern paths exist. If any don't,
raise an exception.
Given a sequence of Pattern instances and the current working directory, check that all root
pattern paths exist. If any don't, raise an exception.
'''
missing_paths = [
pattern.path
for pattern in patterns
if pattern.type == Pattern_type.ROOT
if not os.path.exists(pattern.path)
if not os.path.exists(os.path.join(working_directory or '', pattern.path))
]
if missing_paths:
+17 -4
View File
@@ -65,8 +65,8 @@ def prune_archives(
archives according to the retention policy specified in that configuration.
'''
borgmatic.logger.add_custom_log_levels()
umask = config.get('umask', None)
lock_wait = config.get('lock_wait', None)
umask = config.get('umask')
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get('prune', '')
full_command = (
@@ -83,10 +83,23 @@ def prune_archives(
and not feature.available(feature.Feature.NO_PRUNE_STATS, local_borg_version)
else ()
)
+ (
('--quick-stats',)
if config.get('quick_statistics')
and not dry_run
and not feature.available(feature.Feature.NO_PRUNE_STATS, local_borg_version)
else ()
)
+ (('--info',) if logger.getEffectiveLevel() == logging.INFO else ())
+ flags.make_flags_from_arguments(
prune_arguments,
excludes=('repository', 'match_archives', 'statistics', 'list_details'),
excludes=(
'repository',
'match_archives',
'statistics',
'quick_statistics',
'list_details',
),
)
+ (('--list',) if config.get('list_details') else ())
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) else ())
@@ -95,7 +108,7 @@ def prune_archives(
+ flags.make_repository_flags(repository_path, local_borg_version)
)
if config.get('statistics') or config.get('list_details'):
if config.get('statistics') or config.get('quick_statistics') or config.get('list_details'):
output_log_level = logging.ANSWER
else:
output_log_level = logging.INFO
+4 -4
View File
@@ -28,14 +28,14 @@ def recreate_archive(
arguments, optional local and remote Borg paths, executes the recreate command with the given
arguments.
'''
lock_wait = config.get('lock_wait', None)
lock_wait = config.get('lock_wait')
exclude_flags = flags.make_exclude_flags(config)
compression = config.get('compression', None)
chunker_params = config.get('chunker_params', None)
compression = config.get('compression')
chunker_params = config.get('chunker_params')
extra_borg_options = config.get('extra_borg_options', {}).get('recreate', '')
# Available recompress MODES: "if-different", "always", "never" (default)
recompress = config.get('recompress', None)
recompress = config.get('recompress')
# Write patterns to a temporary file and use that file with --patterns-from.
patterns_file = write_patterns_file(
+16 -11
View File
@@ -5,6 +5,7 @@ import shlex
import subprocess
import borgmatic.config.paths
import borgmatic.logger
from borgmatic.borg import environment, feature, flags, repo_info
from borgmatic.execute import DO_NOT_CAPTURE, execute_command
@@ -40,17 +41,21 @@ def create_repository(
Raise subprocess.CalledProcessError if "borg info" returns an error exit code.
'''
try:
info_data = json.loads(
repo_info.display_repository_info(
repository_path,
config,
local_borg_version,
argparse.Namespace(json=True),
global_arguments,
local_path,
remote_path,
),
)
# Suppress Borg's "repository does not exist" error log, so the user isn't confused by
# seeing an error during successful repository creation.
with borgmatic.logger.Logs_suppressed(msgid='Repository.DoesNotExist'):
info_data = json.loads(
repo_info.display_repository_info(
repository_path,
config,
local_borg_version,
argparse.Namespace(json=True),
global_arguments,
local_path,
remote_path,
),
)
repository_encryption_mode = info_data.get('encryption', {}).get('mode')
if repository_encryption_mode != encryption_mode:
+1 -2
View File
@@ -24,7 +24,7 @@ def display_repository_info(
information for the Borg repository or return JSON summary information.
'''
borgmatic.logger.add_custom_log_levels()
lock_wait = config.get('lock_wait', None)
lock_wait = config.get('lock_wait')
extra_borg_options = config.get('extra_borg_options', {}).get(
'repo_info' if feature.available(feature.Feature.REPO_INFO, local_borg_version) else 'info',
'',
@@ -37,7 +37,6 @@ def display_repository_info(
if feature.available(feature.Feature.REPO_INFO, local_borg_version)
else ('info',)
)
+ (('--critical',) if repo_info_arguments.json else ())
+ (
('--info',)
if logger.getEffectiveLevel() == logging.INFO and not repo_info_arguments.json
+34 -6
View File
@@ -31,9 +31,10 @@ ACTION_ALIASES = {
'transfer': [],
'break-lock': [],
'key': [],
'borg': [],
'recreate': [],
'diff': [],
'browse': [],
'borg': [],
}
@@ -324,7 +325,7 @@ def make_argument_description(schema, flag_name):
' To specify a different list element, replace the "[0]" with another array index ("[1]", "[2]", etc.).',
)
if example and schema_type in ('array', 'object'): # noqa: PLR6201
if example and schema_type in ('array', 'object'):
example_buffer = io.StringIO()
yaml = ruamel.yaml.YAML(typ='safe')
yaml.default_flow_style = True
@@ -804,6 +805,13 @@ def make_parsers(schema, unparsed_arguments): # noqa: PLR0915
action='store_true',
help='Display statistics of the pruned archive [Borg 1 only]',
)
prune_group.add_argument(
'--quick-stats',
dest='quick_statistics',
default=None,
action='store_true',
help='Display statistics of the pruned archive, skipping repository-wide "All archives" and chunk index statistics [Borg >= 1.4.5 and < 2 only]',
)
prune_group.add_argument(
'--list',
dest='list_details',
@@ -896,6 +904,13 @@ def make_parsers(schema, unparsed_arguments): # noqa: PLR0915
action='store_true',
help='Display statistics of archive',
)
create_group.add_argument(
'--quick-stats',
dest='quick_statistics',
default=None,
action='store_true',
help='Display statistics of archive, skipping repository-wide "All archives" and chunk index statistics [Borg 1.4.5+ only]',
)
create_group.add_argument(
'--list',
'--files',
@@ -1997,8 +2012,9 @@ def make_parsers(schema, unparsed_arguments): # noqa: PLR0915
diff_parser = action_parsers.add_parser(
'diff',
aliases=ACTION_ALIASES['diff'],
help='This command finds differences (file contents, user/group/mode) between archives',
description='This command finds differences (file contents, user/group/mode) between archives',
help='Find differences (file contents, user/group/mode) between archives',
description='Find differences (file contents, user/group/mode) between archives',
add_help=False,
)
diff_group = diff_parser.add_argument_group('diff arguments')
diff_group.add_argument(
@@ -2035,6 +2051,18 @@ def make_parsers(schema, unparsed_arguments): # noqa: PLR0915
action='store_true',
help='Run the diff according to borgmatic configured patterns (ie do not diff entire archives)',
)
diff_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
browse_parser = action_parsers.add_parser(
'browse',
aliases=ACTION_ALIASES['browse'],
help='Browse repositories, archives, and files in a console UI',
description='Browse repositories, archives, and files in a console UI',
add_help=False,
)
browse_group = browse_parser.add_argument_group('browse arguments')
browse_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
borg_parser = action_parsers.add_parser(
'borg',
aliases=ACTION_ALIASES['borg'],
@@ -2107,7 +2135,7 @@ def parse_arguments(schema, *unparsed_arguments):
)
if (
('list' in arguments and 'repo-info' in arguments and arguments['list'].json) # noqa: PLR0916
('list' in arguments and 'repo-info' in arguments and arguments['list'].json)
or ('list' in arguments and 'info' in arguments and arguments['list'].json)
or ('repo-info' in arguments and 'info' in arguments and arguments['repo-info'].json)
):
@@ -2125,7 +2153,7 @@ def parse_arguments(schema, *unparsed_arguments):
'With the repo-list action, only one of --prefix or --match-archives flags can be used.',
)
if 'info' in arguments and ( # noqa: PLR0916
if 'info' in arguments and (
(arguments['info'].archive and arguments['info'].prefix)
or (arguments['info'].archive and arguments['info'].match_archives)
or (arguments['info'].prefix and arguments['info'].match_archives)
+10 -2
View File
@@ -12,6 +12,7 @@ import ruamel.yaml
import borgmatic.actions.borg
import borgmatic.actions.break_lock
import borgmatic.actions.browse.run
import borgmatic.actions.change_passphrase
import borgmatic.actions.check
import borgmatic.actions.compact
@@ -211,7 +212,7 @@ def run_configuration(config_filename, config, config_paths, arguments): # noqa
f"Skipping {'/'.join(skip_actions)} action{'s' if len(skip_actions) > 1 else ''} due to configured skip_actions",
)
try: # noqa: PLR1702
try:
with (
Monitoring_hooks(config_filename, config, arguments, global_arguments),
borgmatic.hooks.command.Before_after_hooks(
@@ -825,8 +826,8 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
try:
if 'bootstrap' in arguments:
try:
# No configuration file is needed for bootstrap.
local_borg_version = borg_version.local_borg_version(
# No configuration file is needed for bootstrap.
{},
arguments['bootstrap'].local_path,
)
@@ -897,6 +898,13 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
return
if 'browse' in arguments:
borgmatic.actions.browse.run.run_browse(
arguments['browse'],
arguments['global'],
configs,
)
except (
CalledProcessError,
ValueError,
+89 -6
View File
@@ -489,6 +489,9 @@ properties:
description: |
Umask used for when executing Borg or calling hooks. Defaults to
0077 for Borg or the umask that borgmatic is run with for hooks.
Even though this value is a YAML integer, borgmatic interprets it as
octal. YAML's "0o"-prefixed octal notation is not currently
supported.
example: 0077
lock_wait:
type: integer
@@ -518,6 +521,21 @@ properties:
If match_archives is not specified, borgmatic defaults to deriving
the match_archives value from archive_name_format.
example: "sh:{hostname}-*"
archive_hostname:
type: string
description: |
Hostname to use for the "{hostname}" placeholder in
"archive_name_format", "match_archives", etc. Defaults to the system
hostname. (This option is supported for Borg 1.4.5+ only.)
example: example.org
archive_username:
type: string
description: |
Username to use for the "{user}" placeholder in
"archive_name_format", "match_archives", etc. Defaults to the
username of the user running borgmatic. (This option is supported
for Borg 1.4.5+ only.)
example: backup_user
file_list_format:
type: string
description: |
@@ -1088,6 +1106,16 @@ properties:
Corresponds to the "--stats" flag on those actions. Defaults to
false.
example: true
quick_statistics:
type: boolean
description: |
Display statistics for an archive when running supported actions,
skipping the repository-wide "All archives" and chunk index
statistics to save some time. Corresponds to the "--quick-stats"
flag on those actions. Defaults to false. (This option is supported
for Borg 1.4.5+ only for the "create" action and for Borg >= 1.4.5
and < Borg 2 for the "prune" action.)
example: true
list_details:
type: boolean
description: |
@@ -1128,8 +1156,9 @@ properties:
- info
- break-lock
- key
- borg
- diff
- browse
- borg
description: |
List of one or more actions to skip running for this configuration
file, even if specified on the command-line (explicitly or
@@ -1345,8 +1374,9 @@ properties:
- info
- break-lock
- key
- borg
- diff
- browse
- borg
description: |
List of actions for which the commands will be
run. Defaults to running for all actions.
@@ -1411,8 +1441,9 @@ properties:
- info
- break-lock
- key
- borg
- diff
- browse
- borg
description: |
Only trigger the hook when borgmatic is run with
particular actions listed here. Defaults to
@@ -1826,6 +1857,27 @@ properties:
client and restore server. The default varies based on
the MariaDB version.
example: false
events:
type: boolean
description: |
Whether to include scheduled events within the dump.
Disable if your database user doesn't have the
permissions to dump events. Defaults to true.
example: false
routines:
type: boolean
description: |
Whether to include stored routines within the dump.
Disable if your user database doesn't have the
permissions to dump routines. Defaults to true.
example: false
tablespaces:
type: boolean
description: |
Whether to include tablespaces within the dump. Disable
if your database user doesn't have the permissions to
dump tablespaces. Defaults to true.
example: false
mariadb_dump_command:
type: string
description: |
@@ -2034,6 +2086,27 @@ properties:
client and restore server. The default varies based on
the MySQL installation.
example: false
events:
type: boolean
description: |
Whether to include scheduled events within the dump.
Disable if your database user doesn't have the
permissions to dump events. Defaults to true.
example: false
routines:
type: boolean
description: |
Whether to include stored routines within the dump.
Disable if your database user doesn't have the
permissions to dump routines. Defaults to true.
example: false
tablespaces:
type: boolean
description: |
Whether to include tablespaces within the dump. Disable
if your database user doesn't have the permissions to
dump tablespaces. Defaults to true.
example: false
mysql_dump_command:
type: string
description: |
@@ -2822,7 +2895,9 @@ properties:
properties:
url:
type: string
description: URL of this Apprise service.
description: |
URL of this Apprise service. Supports the
"{credential ...}" syntax.
example: "gotify://hostname/token"
label:
type: string
@@ -3309,24 +3384,32 @@ properties:
description: |
Command to use instead of "keepassxc-cli".
example: /usr/local/bin/keepassxc-cli
secret_tool_command:
type: string
description: |
Command to use instead of "secret-tool".
example: /usr/local/bin/secret-tool
ask_for_password:
type: boolean
description: |
Whether keepassxc-cli should prompt the user for a password.
Disabling this is only really useful if you're unlocking
your KeePassXC database with a key file instead of a
password. Defaults to true.
password. Ignored when using KeePassXC's secret service
integration. Defaults to true.
example: false
key_file:
type: string
description: |
Path to a key file for unlocking the KeePassXC database.
Ignored when using KeePassXC's secret service integration.
example: /path/to/keyfile
yubikey:
type: string
description: |
YubiKey slot and optional serial number used to access the
KeePassXC database. The format is "<slot[:serial]>", where:
KeePassXC database. Ignored when using KeePassXC's secret
service integration. The format is "<slot[:serial]>", where:
* <slot> is the YubiKey slot number (e.g., `1` or `2`).
* <serial> (optional) is the YubiKey's serial number (e.g.,
`7370001`).
+34 -7
View File
@@ -157,6 +157,7 @@ def borg_json_log_line_to_record(line, log_level):
levelno=log_level,
created=log_data.get('time'),
msg=log_data.get('message'),
msgid=log_data.get('msgid'),
levelname=logging.getLevelName(log_level),
name=log_data.get('name'),
)
@@ -167,6 +168,7 @@ def borg_json_log_line_to_record(line, log_level):
levelno=borg_log_level,
created=log_data.get('time'),
msg=log_data.get('message'),
msgid=log_data.get('msgid'),
levelname=log_data.get('levelname'),
name=log_data.get('name'),
)
@@ -178,6 +180,7 @@ def borg_json_log_line_to_record(line, log_level):
levelno=log_level,
created=time.time(),
msg=f'{log_data.get("status")} {log_data.get("path")}',
msgid=log_data.get('msgid'),
levelname=logging.getLevelName(log_level),
name='borg.file_status',
)
@@ -372,13 +375,21 @@ def log_buffer_lines(
yield log_record.getMessage()
def raise_for_process_errors(buffer_readers, process_metadatas, borg_local_path, borg_exit_codes):
def raise_for_process_errors(
buffer_readers,
process_metadatas,
output_log_level,
borg_local_path,
borg_exit_codes,
capture_stderr=False,
):
'''
Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
Process_metadata instance, Borg's local path, a sequence of exit code configuration dicts, check
the given processes for error or warning exit codes. If found, vent or kill any running
processes. In the case of an error exit code, raise. In the case of warning, return
Exit_status.WARNING. Otherwise, return None.
Process_metadata instance, a requested output log level for stdout, Borg's local path, a
sequence of exit code configuration dicts, and whether to capture stderr, check the given
processes for error or warning exit codes. If found, vent or kill any running processes and
drain any remaining buffer lines. In the case of an error exit code, raise. In the case of
warning, return Exit_status.WARNING. Otherwise, return None.
'''
result_status = None
@@ -404,6 +415,13 @@ def raise_for_process_errors(buffer_readers, process_metadatas, borg_local_path,
result_status = Exit_status.WARNING
continue
# Drain and log remaining buffer lines, so no output gets lost. But swallow captured output,
# since we're raising below instead of yielding captured lines.
tuple(
log_remaining_buffer_lines(
buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
)
)
last_lines = process_metadatas[process].last_lines
# If an error occurs, include its output in the raised exception so that we don't
@@ -429,6 +447,9 @@ def log_remaining_buffer_lines(
whether to capture stderr, drain and log any remaining output lines from the buffers until
they're empty. Additionally, for any log records with a log level the same as the output log
level, yield those log messages for capture.
The main difference between this function and log_buffer_lines() is that this one doesn't just
do one "turn of the crank" of logging buffer output; it completely drains any remaining output.
'''
for output_buffer, reader in buffer_readers.items():
if not reader.process:
@@ -446,6 +467,7 @@ def log_remaining_buffer_lines(
borg_local_path=borg_local_path,
command=reader.process.args,
),
last_lines=process_metadatas[reader.process].last_lines,
)
if (
@@ -494,7 +516,7 @@ def log_outputs(
for buffer in output_buffers_for_process(process, exclude_stdouts)
}
# Log output lines for each process until they all exit.
# Log output lines for each process until they all exit or one errors.
while True:
yield from log_buffer_lines(
buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
@@ -502,7 +524,12 @@ def log_outputs(
if (
raise_for_process_errors(
buffer_readers, process_metadatas, borg_local_path, borg_exit_codes
buffer_readers,
process_metadatas,
output_log_level,
borg_local_path,
borg_exit_codes,
capture_stderr,
)
== Exit_status.WARNING
):
+17 -2
View File
@@ -7,19 +7,34 @@ import borgmatic.execute
logger = logging.getLogger(__name__)
SECRET_SERVICE_DATABASE_PATH = 'secret-service'
def load_credential(hook_config, config, credential_parameters):
'''
Given the hook configuration dict, the configuration dict, and a credential parameters tuple
containing a KeePassXC database path and an attribute name to load, run keepassxc-cli to fetch
the corresponding KeePassXC credential and return it.
the corresponding KeePassXC credential and return it. Or use secret-tool if the database path
is "secret-service", indicating that KeePassXC's secret service integration should be used
instead.
Raise ValueError if keepassxc-cli can't retrieve the credential.
Raise ValueError if keepassxc-cli or secret-tool can't retrieve the credential.
'''
try:
(database_path, attribute_name) = credential_parameters
except ValueError:
raise ValueError(f'Invalid KeePassXC credential parameters: {credential_parameters}')
if database_path == SECRET_SERVICE_DATABASE_PATH:
command = (
*shlex.split((hook_config or {}).get('secret_tool_command', 'secret-tool')),
*('lookup', 'Path', attribute_name),
)
return '\n'.join(borgmatic.execute.execute_command_and_capture_output(command)).rstrip(
os.linesep
)
expanded_database_path = os.path.expanduser(database_path)
if not os.path.exists(expanded_database_path):
+67 -26
View File
@@ -1,4 +1,3 @@
import copy
import logging
import os
import re
@@ -67,7 +66,9 @@ def make_defaults_file_options(username=None, password=None, defaults_extra_file
Do not use the returned value for multiple different command invocations. That will not work
because each pipe is "used up" once read.
'''
escaped_password = None if password is None else password.replace('\\', '\\\\')
escaped_password = (
None if password is None else password.replace('\\', '\\\\').replace('"', '\\"')
)
values = '\n'.join(
(
@@ -106,6 +107,10 @@ def make_defaults_file_options(username=None, password=None, defaults_extra_file
return (f'--defaults-extra-file=/dev/fd/{read_file_descriptor}',)
EXCLUDED_SYSTEM_DATABASE_NAMES = ('information_schema', 'performance_schema', 'sys')
SYSTEM_DATABASE_NAME = 'mysql'
def database_names_to_dump(database, config, username, password, environment, dry_run):
'''
Given a requested database config, a configuration dict, a database username and password, an
@@ -164,17 +169,18 @@ def database_names_to_dump(database, config, username, password, environment, dr
working_directory=borgmatic.config.paths.get_working_directory(config),
)
# Dumping system databases directly doesn't work; too much gets dumped (including virtual tables
# that MariaDB recreates on startup) and so the dump isn't restorable. Therefore several system
# databases are excluded here. But also see below where select system tables from the "mysql"
# system table are dumped via the "--system" flag.
return tuple(
show_name.strip()
for show_name in show_lines
if show_name not in SYSTEM_DATABASE_NAMES
if show_name not in EXCLUDED_SYSTEM_DATABASE_NAMES
if not skip_names or show_name not in skip_names
)
SYSTEM_DATABASE_NAMES = ('information_schema', 'mysql', 'performance_schema', 'sys')
def execute_dump_command(
database,
config,
@@ -235,8 +241,12 @@ def execute_dump_command(
+ (('--user', username) if username and password_transport == 'environment' else ())
+ (('--ssl',) if database.get('tls') is True else ())
+ (('--skip-ssl',) if database.get('tls') is False else ())
+ (('--events',) if database.get('events', True) else ())
+ (('--routines',) if database.get('routines', True) else ())
+ (('--all-tablespaces',) if database.get('tablespaces', True) else ())
+ (('--system=users,udfs,servers',) if SYSTEM_DATABASE_NAME in database_names else ())
+ ('--databases',)
+ database_names
+ tuple(name for name in database_names if name != SYSTEM_DATABASE_NAME)
+ ('--result-file', dump_filename)
)
@@ -323,6 +333,7 @@ def dump_data_sources(
raise ValueError('Cannot find any MariaDB databases to dump.')
# Database dumps to individual files.
if database['name'] == 'all' and database.get('format'):
for database_name in dump_database_names:
dumps_metadata.append(
@@ -335,8 +346,7 @@ def dump_data_sources(
database.get('container'),
)
)
renamed_database = copy.copy(database)
renamed_database['name'] = database_name
renamed_database = dict(database, name=database_name)
processes.append(
execute_dump_command(
renamed_database,
@@ -350,6 +360,7 @@ def dump_data_sources(
dry_run_label,
),
)
# Database dumps all to one file.
else:
dumps_metadata.append(
borgmatic.actions.restore.Dump(
@@ -414,7 +425,7 @@ def make_data_source_dump_patterns(
port=None,
container=None,
label=None,
): # pragma: no cover
):
'''
Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
and a database name to match, return the corresponding glob patterns to match the database dump
@@ -423,24 +434,54 @@ def make_data_source_dump_patterns(
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'), name, hostname, port, container, label
*(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'), name, hostname, port, container, label
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_runtime_directory),
name,
hostname,
port,
container,
label,
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_source_directory),
name,
hostname,
port,
container,
label,
),
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_runtime_directory),
name,
hostname,
port,
container,
label,
*(
(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'),
name,
hostname,
port=None,
container=container,
label=label,
),
)
if port == get_default_port(databases, config)
else ()
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_source_directory),
name,
hostname,
port,
container,
label,
*(
(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'),
name,
hostname,
port=get_default_port(databases, config),
container=container,
label=label,
),
)
if port is None
else ()
),
)
+47 -17
View File
@@ -216,7 +216,7 @@ def make_data_source_dump_patterns(
port=None,
container=None,
label=None,
): # pragma: no cover
):
'''
Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
and a database name to match, return the corresponding glob patterns to match the database dump
@@ -225,24 +225,54 @@ def make_data_source_dump_patterns(
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'), name, hostname, port, container, label
*(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'), name, hostname, port, container, label
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_runtime_directory),
name,
hostname,
port,
container,
label,
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_source_directory),
name,
hostname,
port,
container,
label,
),
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_runtime_directory),
name,
hostname,
port,
container,
label,
*(
(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'),
name,
hostname,
port=None,
container=container,
label=label,
),
)
if port == get_default_port(databases, config)
else ()
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_source_directory),
name,
hostname,
port,
container,
label,
*(
(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'),
name,
hostname,
port=get_default_port(databases, config),
container=container,
label=label,
),
)
if port is None
else ()
),
)
+51 -20
View File
@@ -1,4 +1,3 @@
import copy
import logging
import os
import shlex
@@ -166,6 +165,9 @@ def execute_dump_command(
+ (('--user', username) if username and password_transport == 'environment' else ())
+ (('--ssl',) if database.get('tls') is True else ())
+ (('--skip-ssl',) if database.get('tls') is False else ())
+ (('--events',) if database.get('events', True) else ())
+ (('--routines',) if database.get('routines', True) else ())
+ (('--all-tablespaces',) if database.get('tablespaces', True) else ())
+ ('--databases',)
+ database_names
+ ('--result-file', dump_filename)
@@ -266,8 +268,7 @@ def dump_data_sources(
database.get('container'),
)
)
renamed_database = copy.copy(database)
renamed_database['name'] = database_name
renamed_database = dict(database, name=database_name)
processes.append(
execute_dump_command(
renamed_database,
@@ -345,7 +346,7 @@ def make_data_source_dump_patterns(
port=None,
container=None,
label=None,
): # pragma: no cover
):
'''
Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
and a database name to match, return the corresponding glob patterns to match the database dump
@@ -354,24 +355,54 @@ def make_data_source_dump_patterns(
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'), name, hostname, port, container, label
*(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'), name, hostname, port, container, label
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_runtime_directory),
name,
hostname,
port,
container,
label,
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_source_directory),
name,
hostname,
port,
container,
label,
),
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_runtime_directory),
name,
hostname,
port,
container,
label,
*(
(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'),
name,
hostname,
port=None,
container=container,
label=label,
),
)
if port == get_default_port(databases, config)
else ()
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_source_directory),
name,
hostname,
port,
container,
label,
*(
(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'),
name,
hostname,
port=get_default_port(databases, config),
container=container,
label=label,
),
)
if port is None
else ()
),
)
+55 -21
View File
@@ -100,7 +100,11 @@ def database_names_to_dump(database, config, environment, dry_run):
if 'username' in database
else ()
)
+ (tuple(database['list_options'].split(' ')) if 'list_options' in database else ())
+ (
tuple(shlex.quote(part) for part in shlex.split(database['list_options']))
if 'list_options' in database
else ()
)
)
logger.debug('Querying for "all" PostgreSQL databases to dump')
list_lines = execute_command_and_capture_output(
@@ -226,7 +230,7 @@ def dump_data_sources(
+ (('--compress', shlex.quote(str(compression))) if compression is not None else ())
+ (('--file', shlex.quote(dump_filename)) if dump_format == 'directory' else ())
+ (
tuple(shlex.quote(option) for option in database['options'].split(' '))
tuple(shlex.quote(part) for part in shlex.split(database['options']))
if 'options' in database
else ()
)
@@ -306,7 +310,7 @@ def make_data_source_dump_patterns(
port=None,
container=None,
label=None,
): # pragma: no cover
):
'''
Given a sequence of configurations dicts, a configuration dict, the borgmatic runtime directory,
and a database name to match, return the corresponding glob patterns to match the database dump
@@ -315,24 +319,54 @@ def make_data_source_dump_patterns(
borgmatic_source_directory = borgmatic.config.paths.get_borgmatic_source_directory(config)
return (
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'), name, hostname, port, container, label
*(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'), name, hostname, port, container, label
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_runtime_directory),
name,
hostname,
port,
container,
label,
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_source_directory),
name,
hostname,
port,
container,
label,
),
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_runtime_directory),
name,
hostname,
port,
container,
label,
*(
(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'),
name,
hostname,
port=None,
container=container,
label=label,
),
)
if port == get_default_port(databases, config)
else ()
),
dump.make_data_source_dump_filename(
make_dump_path(borgmatic_source_directory),
name,
hostname,
port,
container,
label,
*(
(
dump.make_data_source_dump_filename(
make_dump_path('borgmatic'),
name,
hostname,
port=get_default_port(databases, config),
container=container,
label=label,
),
)
if port is None
else ()
),
)
@@ -393,7 +427,7 @@ def restore_data_source_dump(
+ (('--username', username) if username else ())
+ (('--dbname', data_source['name']) if not all_databases else ())
+ (
tuple(data_source['analyze_options'].split(' '))
tuple(shlex.quote(part) for part in shlex.split(data_source['analyze_options']))
if 'analyze_options' in data_source
else ()
)
@@ -414,7 +448,7 @@ def restore_data_source_dump(
+ (('--username', username) if username else ())
+ (('--no-owner',) if data_source.get('no_owner', False) else ())
+ (
tuple(data_source['restore_options'].split(' '))
tuple(shlex.quote(part) for part in shlex.split(data_source['restore_options']))
if 'restore_options' in data_source
else ()
)
+42 -26
View File
@@ -403,41 +403,57 @@ def remove_data_source_dumps(hook_config, config, borgmatic_runtime_directory, p
snapshot_dataset_names = {
full_snapshot_name.split('@')[0] for full_snapshot_name in full_snapshot_names
}
hash_to_dataset_mount_point = {}
# Make a map from mount point hash to the corresponding (dataset name, mount point) tuple.
for dataset_name, mount_point in dataset_name_to_mount_point.items():
mount_point_hash = hashlib.shake_256(mount_point.encode('utf-8')).hexdigest(
MOUNT_POINT_HASH_LENGTH
)
hash_to_dataset_mount_point[mount_point_hash] = (dataset_name, mount_point)
for snapshots_directory in glob.glob(snapshots_glob):
if not os.path.isdir(snapshots_directory):
continue
for dataset_name, mount_point in dataset_name_to_mount_point.items():
snapshot_mount_path = os.path.join(snapshots_directory, mount_point.lstrip(os.path.sep))
# Get the dataset and mount point corresponding to the hash found in this snapshot directory
# path. If none is found, bail.
try:
(dataset_name, mount_point) = hash_to_dataset_mount_point[
os.path.basename(snapshots_directory)
]
except KeyError:
continue
# If this dataset name does not correspond to a known snapshot, then this is probably
# just a "shadow" of a nested dataset and therefore there's nothing to unmount.
if not os.path.isdir(snapshot_mount_path) or dataset_name not in snapshot_dataset_names:
snapshot_mount_path = os.path.join(snapshots_directory, mount_point.lstrip(os.path.sep))
# If this dataset name doesn't correspond to a known snapshot, then this is probably
# just a "shadow" of a nested dataset and therefore there's nothing to unmount.
if not os.path.isdir(snapshot_mount_path) or dataset_name not in snapshot_dataset_names:
continue
# This might fail if the path is already mounted, but we swallow errors here since we'll
# do another recursive delete below. The point of doing it here is that we don't want to
# try to unmount a non-mounted directory (which *will* fail), and probing for whether a
# directory is mounted is tough to do in a cross-platform way.
if not dry_run:
shutil.rmtree(snapshot_mount_path, ignore_errors=True)
# If the delete was successful, that means there's nothing to unmount.
if not os.path.isdir(snapshot_mount_path):
continue
# This might fail if the path is already mounted, but we swallow errors here since we'll
# do another recursive delete below. The point of doing it here is that we don't want to
# try to unmount a non-mounted directory (which *will* fail), and probing for whether a
# directory is mounted is tough to do in a cross-platform way.
if not dry_run:
shutil.rmtree(snapshot_mount_path, ignore_errors=True)
logger.debug(f'Unmounting ZFS snapshot at {snapshot_mount_path}{dry_run_label}')
# If the delete was successful, that means there's nothing to unmount.
if not os.path.isdir(snapshot_mount_path):
continue
logger.debug(f'Unmounting ZFS snapshot at {snapshot_mount_path}{dry_run_label}')
if not dry_run:
try:
unmount_snapshot(umount_command, snapshot_mount_path)
except FileNotFoundError:
logger.debug(f'Could not find "{umount_command}" command')
return
except subprocess.CalledProcessError as error:
logger.debug(error)
continue
if not dry_run:
try:
unmount_snapshot(umount_command, snapshot_mount_path)
except FileNotFoundError:
logger.debug(f'Could not find "{umount_command}" command')
return
except subprocess.CalledProcessError as error:
logger.debug(error)
continue
if not dry_run:
shutil.rmtree(snapshot_mount_path, ignore_errors=True)
+10 -2
View File
@@ -1,6 +1,7 @@
import logging
import operator
import borgmatic.hooks.credential.parse
import borgmatic.hooks.monitoring.logs
import borgmatic.hooks.monitoring.monitor
@@ -44,7 +45,9 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
import apprise # noqa: PLC0415
from apprise import NotifyFormat, NotifyType # noqa: PLC0415
except ImportError: # pragma: no cover
logger.warning('Unable to import Apprise in monitoring hook')
logger.warning(
'Unable to import Apprise in its monitoring hook; try installing "borgmatic[Apprise]"'
)
return
state_to_notify_type = {
@@ -76,7 +79,12 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
logger.info(f'Pinging Apprise services: {labels_string}{dry_run_string}')
apprise_object = apprise.Apprise()
apprise_object.add(list(map(operator.itemgetter('url'), hook_config.get('services'))))
apprise_object.add(
[
borgmatic.hooks.credential.parse.resolve_credential(service['url'], config)
for service in hook_config.get('services')
]
)
if dry_run:
return
+89 -1
View File
@@ -154,6 +154,8 @@ def log_record_to_json(record):
'''
Given a logging.LogRecord, return it as a JSON-encoded string containing relevant attributes.
'''
message_id = getattr(record, 'msgid', None)
return json.dumps(
dict(
type='log_message',
@@ -162,6 +164,7 @@ def log_record_to_json(record):
levelname=record.levelname,
name=record.name,
)
| ({'msgid': message_id} if message_id is not None else {})
)
@@ -169,7 +172,7 @@ class Json_formatter(logging.Formatter):
def __init__(self, fmt='{message}', *args, style='{', **kwargs):
super().__init__(*args, fmt=fmt, style=style, **kwargs)
def format(self, record): # noqa: PLR6301
def format(self, record):
return log_record_to_json(record)
@@ -336,6 +339,91 @@ class Log_prefix:
set_log_prefix(self.original_prefix)
class Log_exclude_filter(logging.Filter):
'''
A Python log filter that omits log records matching given attributes.
'''
def __init__(self, name, filter_attributes):
'''
Given a unique name for this filter and a dict of attributes to filter on, set the filter
name and save the attributes for use below.
'''
self.filter_attributes = filter_attributes
super().__init__(name)
def filter(self, log_record):
'''
Given a log record, return False (indicating the record should be omitted) if the record's
attributes match any of the saved filter attributes. Return True (indicating do not omit)
otherwise.
'''
for attribute_name, value in self.filter_attributes.items():
if getattr(log_record, attribute_name, None) == value:
return False
return True
def add_log_exclude_filter(name, filter_attributes):
'''
Given a unique filter name and a dict of attributes to filter on, create a log exclude filter
with them and add the filter to each log handler.
'''
for handler in logging.getLogger().handlers:
handler.addFilter(Log_exclude_filter(name, filter_attributes))
def remove_log_exclude_filter(name):
'''
Given a unique filter name, remove matching filters from each log handler.
'''
for handler in logging.getLogger().handlers:
for exclude_filter in handler.filters:
if getattr(exclude_filter, 'name', None) == name:
handler.removeFilter(exclude_filter)
class Logs_suppressed:
'''
A Python context manager for temporarily adding a log filter that suppresses requested log
records for the duration of the context manager.
Example use:
with borgmatic.logger.Logs_suppressed(msgid='Repository.DoesNotExist'):
do_something_that_logs()
For the scope of that "with" statement, any records logged with the given message ID are
filtered out of the log output. "msgid" is just an example; any logging.LogRecord attributes
(standard or custom) can be passed in to filter on.
Multiple instances of this context manager with different filter attributes can be in use at
once.
'''
def __init__(self, **filter_attributes):
'''
Given the desired log record filter attributes as keyword arguments, save them for use below.
'''
self.filter_attributes = filter_attributes
def __enter__(self):
'''
Create a log filter with the saved filter attributes and add the filter to every logging
handler, so that they filter out the desired log records.
'''
add_log_exclude_filter(name=str(id(self)), filter_attributes=self.filter_attributes)
def __exit__(self, exception_type, exception, traceback):
'''
Remove the previously added filter from every logging handler.
'''
remove_log_exclude_filter(name=str(id(self)))
class Delayed_logging_handler(logging.handlers.BufferingHandler):
'''
A logging handler that buffers logs and doesn't flush them until explicitly flushed (after
+1 -1
View File
@@ -5,7 +5,7 @@ RUN apk add --no-cache py3-pip py3-ruamel.yaml py3-ruamel.yaml.clib
RUN pip install --break-system-packages --no-cache /app && borgmatic config generate && borgmatic config generate --destination /etc/borgmatic --split && chmod +r /etc/borgmatic/*.yaml
RUN mkdir /command-line \
&& borgmatic --help > /command-line/global.txt \
&& for action in repo-create transfer create prune compact check delete extract config "config bootstrap" "config generate" "config validate" "config show" export-tar mount umount repo-delete restore repo-list list repo-info info break-lock "key export" "key import" "key change-passphrase" recreate diff borg; do \
&& for action in repo-create transfer create prune compact check delete extract config "config bootstrap" "config generate" "config validate" "config show" export-tar mount umount repo-delete restore repo-list list repo-info info break-lock "key export" "key import" "key change-passphrase" recreate diff browse borg; do \
borgmatic $action --help > /command-line/${action/ /-}.txt; done
RUN /app/docs/fetch-contributors >> /contributors.html
+1
View File
@@ -23,6 +23,7 @@
{% if feedTitle and feedUrl %}
<link rel="alternate" href="{{ feedUrl }}" title="{{ feedTitle }}" type="application/atom+xml">
{% endif %}
{{ head_additions | safe }}
</head>
<body>
@@ -1,5 +1,5 @@
Here's the command-line help for this action in the [most recent version of
borgmatic](https://projects.torsion.org/borgmatic-collective/borgmatic/releases).
If you're using an older version, some of these flags may not work, and you
should instead run the action with `--help` to see the flags specific to your
borgmatic version.
If you're using an older version, some of these flags may not work (or the action
may be missing entirely). You should instead run the action with `--help` to see
the flags and actions specific to your borgmatic version.
+9 -2
View File
@@ -31,7 +31,7 @@ changes work:
```bash
cd borgmatic
uv tool update-shell
uv tool install --editable .
uv tool install --editable .[dev]
```
Or to work on the [Apprise
@@ -39,7 +39,14 @@ hook](https://torsion.org/borgmatic/reference/configuration/monitoring/apprise/)
change that last line to:
```bash
uv tool install --editable .[Apprise]
uv tool install --editable .[dev,Apprise]
```
Or to work on the [browse
action](https://torsion.org/borgmatic/reference/command-line/actions/browse/):
```bash
uv tool install --editable .[dev,browse]
```
To get oriented with the borgmatic source code, have a look at the [source
+49
View File
@@ -135,3 +135,52 @@ is, patterns are _not_ taken into consideration. If you require this, supply the
See the [Borg](https://borgbackup.readthedocs.io/en/stable/usage/diff.html)
documentation for information on output format, what is compared, and more.
## Browsing backups
<span class="minilink minilink-addedin">New in version 2.1.6</span> <span
class="minilink minilink-addedin">Experimental feature</span> borgmatic has an
experimental console UI for browsing your repositories, archives, and files.
Here's what it looks like:
<img src="https://torsion.org/borgmatic/static/browse.png" alt="borgmatic browse screenshot" style="width: 100%">
This feature is not intended to be a general-purpose Borg UI with every
borgmatic feature, but rather it's for use cases like quickly looking at the
contents of your backups when you're feeling too lazy to type out a full
borgmatic command-line.
Depending on how you installed borgmatic, it may not have come with the
necessary Python libraries to support the browse action. (borgmatic's
[stand-alone
binary](https://projects.torsion.org/borgmatic-collective/borgmatic/releases)
does not currently include them.) If you originally [installed borgmatic with
uv](https://torsion.org/borgmatic/how-to/install-borgmatic/), run the following
to install the libraries needed for the browse action:
```bash
sudo uv tool install borgmatic[browse]
```
Omit `sudo` if borgmatic is installed as a non-root user.
Once the libraries are installed, run the following to access the browse action:
```bash
borgmatic browse
```
This launches a console UI where you can select a borgmatic configuration file
(if there's more than one), select a Borg repository, select an archive in that
repository, and even browse the backed up files in that archive.
Use the keyboard or the mouse to navigate the UI. The footer at the bottom of
the screen shows some of the available keys. Logs show up directly in the UI at
the [selected
verbosity](https://torsion.org/borgmatic/reference/command-line/logging/),
although logs are hidden by default.
Please [provide
feedback](https://torsion.org/borgmatic/#support-and-contributing) if you find
this feature useful—or even if you don't, but would like it to become useful.
+1 -1
View File
@@ -98,7 +98,7 @@ installing borgmatic:
#### Etc.
* [stand-alone Linux binary](https://projects.torsion.org/borgmatic-collective/borgmatic/releases) (This is a beta feature!)
* [stand-alone Linux binary](https://projects.torsion.org/borgmatic-collective/borgmatic/releases) (This is a beta feature! The minimal binary omits [Apprise support](https://torsion.org/borgmatic/reference/configuration/monitoring/apprise/).)
* [Ansible role](https://github.com/borgbase/ansible-role-borgbackup)
* [pipx](https://pipx.pypa.io/stable/)
+4 -5
View File
@@ -13,16 +13,15 @@ already.
## Hosting providers
Need somewhere to store your encrypted off-site backups? The following hosting
providers include specific support for Borg/borgmatic—and fund borgmatic
development and hosting when you use these referral links to sign up:
provider includes specific support for Borg/borgmatic—and funds borgmatic
development and hosting when you use this referral links to sign up:
<ul>
<li class="referral"><a href="https://www.borgbase.com/?utm_source=borgmatic">BorgBase</a>: Borg hosting service with support for monitoring, 2FA, and append-only repos</li>
<li class="referral"><a href="https://hetzner.cloud/?ref=v9dOJ98Ic9I8">Hetzner</a>: A "storage box" that includes support for Borg</li>
</ul>
Additionally, rsync.net has a compatible storage offering, but does not fund
borgmatic development or hosting.
Additionally, Hetzner and rsync\.net have compatible storage offerings, but do
not fund borgmatic development or hosting.
## Configuration
@@ -0,0 +1,17 @@
---
title: browse
eleventyNavigation:
key: browse
parent: 🎬 Actions
---
<span class="minilink minilink-addedin">Experimental feature</span> {% include snippet/command-line/sample.md %}
```
{% include borgmatic/command-line/browse.txt %}
```
## Related documentation
* [Inspect your backups](https://torsion.org/borgmatic/how-to/inspect-your-backups/)
@@ -43,14 +43,15 @@ postgresql_databases:
```
### Custom command
### Custom commands
You can also optionally override the `keepassxc-cli` command that borgmatic calls to load
passwords:
You can also optionally override the `keepassxc-cli` or `secret-tool` commands
that borgmatic call to load passwords:
```yaml
keepassxc:
keepassxc_cli_command: /usr/local/bin/keepassxc-cli
secret_tool_command: /usr/local/bin/secret-tool
```
Another example:
@@ -98,3 +99,30 @@ keepassxc:
The value here is the YubiKey slot number (e.g., `1` or `2`) and optional serial
number (e.g., `7370001`) used to access the KeePassXC database. Join the two
values with a colon, but omit the colon if you're leaving out the serial number.
### Secret service integration
<span class="minilink minilink-addedin">New in version 2.1.6</span> borgmatic
supports [KeePassXC's secret service
integration](https://keepassxc.org/docs/KeePassXC_UserGuide#_secret_service_integration)
that integrates with the [freedesktop secret service
API](https://specifications.freedesktop.org/secret-service/latest/) and allows
clients like borgmatic to access your passwords.
To use this feature from borgmatic, specify `secret-service` instead of a
KeePassXC database path when calling this credential hook. For instance:
```yaml
encryption_passphrase: "{credential keepassxc secret-service borgmatic}"
```
With this in place, borgmatic runs libsecret's `secret-tool` command to retrieve
the password titled "borgmatic" on demand. KeePassXC may then prompt you to
approve the password access, depending on how you've configured it.
One benefit of using the KeePassXC's secret service integration like this is
that you don't have to type a KeePassXC database passhprase (or use a keyfile)
whenever borgmatic accesses your passwords. Instead, you can configure
KeePassXC to prompt you to approve or deny each access. Or you can even
pre-approve password access to support automated borgmatic runs.
@@ -16,10 +16,51 @@ mariadb_databases:
```
### Full configuration
## System databases
<span class="minilink minilink-addedin">New in version 2.1.6</span> When dumping
["all"
databases](https://torsion.org/borgmatic/how-to/backup-your-databases/#all-databases),
borgmatic excludes most data coming from [MariaDB system
databases](https://mariadb.com/docs/server/reference/system-tables),
because much of it is populated on MariaDB startup and thus not restorable (or
just unnecessary to backup).
The system data that borgmatic does include in these dumps are: users, roles,
grants, user-defined functions, and remote servers—all from the `mysql` system
database. This omits all other data from the `mysql` database, which includes
index and table statistics, time zones, and installed server plugins. It also
excludes the separate `information_schema`, `performance_schema`, and `sys`
system databases. This is not currently configurable.
Within a Borg archive, you can find this data stored in a dump named `mysql`—the
name of the system table this data comes from. And if you'd like to dump this
data without having to dump "all" databases, then you can configure a database
named `mysql` in your borgmatic configuration. For example:
```yaml
mariadb_databases:
- name: mysql
```
Even in this case though, only the subset of system data described above is
included in the dump.
<span class="minilink minilink-addedin">Prior to version 2.1.6</span> Dumps of
"all" databases excluded system databases and all of their data. Additionally,
explicitly dumping `mysql` was treated like any other database—and thus wasn't
easily restorable.
## Full configuration
{% include snippet/configuration/sample.md %}
```yaml
{% include borgmatic/mariadb_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -14,10 +14,15 @@ mongodb_databases:
```
### Full configuration
## Full configuration
{% include snippet/configuration/sample.md %}
```yaml
{% include borgmatic/mongodb_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -21,3 +21,8 @@ mysql_databases:
```yaml
{% include borgmatic/mysql_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -76,3 +76,8 @@ roles](https://www.postgresql.org/docs/current/predefined-roles.html) and
```yaml
{% include borgmatic/postgresql_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -22,3 +22,8 @@ sqlite_databases:
```yaml
{% include borgmatic/sqlite_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -5,9 +5,9 @@ eleventyNavigation:
parent: 🚨 Monitoring
---
<span class="minilink minilink-addedin">New in version 1.8.4</span>
[Apprise](https://github.com/caronc/apprise/wiki) is a local notification library
[Apprise](https://appriseit.com/) is a local notification library
that "allows you to send a notification to almost all of the most popular
[notification services](https://github.com/caronc/apprise/wiki) available to
[notification services](https://appriseit.com/services/) available to
us today such as: Telegram, Discord, Slack, Amazon SNS, Gotify, etc."
Depending on how you installed borgmatic, it may not have come with Apprise.
@@ -22,7 +22,7 @@ sudo uv tool install borgmatic[Apprise]
Omit `sudo` if borgmatic is installed as a non-root user.
Once Apprise is installed, configure borgmatic to notify one or more [Apprise
services](https://github.com/caronc/apprise/wiki). For example:
services](https://appriseit.com/services/). For example:
```yaml
apprise:
+40 -6
View File
@@ -1,14 +1,48 @@
---
title: Search the documentation
eleventyExcludeFromCollections: true
templateEngineOverride: md
head_additions: >
<link href="/borgmatic/pagefind/pagefind-component-ui.css" rel="stylesheet">
<script src="/borgmatic/pagefind/pagefind-component-ui.js" type="module"></script>
<style>
@media (prefers-color-scheme: light) {
:root {
--pf-summary-font-size: 14px;
--pf-result-title-font-size: 16px;
--pf-result-excerpt-font-size: 15px;
}
}
@media (prefers-color-scheme: dark) {
:root {
--pf-summary-font-size: 14px;
--pf-result-title-font-size: 16px;
--pf-result-excerpt-font-size: 15px;
}
}
.pf-result .pf-result-title, .pf-heading-link {
font-weight: 600 !important;
}
.pf-result .pf-result-excerpt mark, .pf-heading-excerpt mark {
background-color: yellow !important;
}
</style>
---
<link href="/borgmatic/pagefind/pagefind-ui.css" rel="stylesheet">
<script src="/borgmatic/pagefind/pagefind-ui.js"></script>
<div id="search"></div>
<script>
<pagefind-config bundle-path="/borgmatic/pagefind/" base-url="/borgmatic/"></pagefind-config>
<p><pagefind-input placeholder="Search"></pagefind-input></p>
<p><pagefind-summary></pagefind-summary></p>
<p><pagefind-results></pagefind-results></p>
<script type="module">
const manager = window.PagefindComponents.getInstanceManager();
const instance = manager.getInstance('default');
window.addEventListener('DOMContentLoaded', (event) => {
let search = new PagefindUI({ element: '#search', showSubResults: true, autofocus: true });
let url_parameters = new URLSearchParams(window.location.search);
search.triggerSearch(url_parameters.get('query'));
instance.triggerSearch(url_parameters.get('query'));
});
</script>
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

+7 -4
View File
@@ -1,17 +1,18 @@
[project]
name = "borgmatic"
version = "2.1.5"
version = "2.1.7.dev0"
authors = [
{ name="Dan Helfman", email="witten@torsion.org" },
]
description = "Simple, configuration-driven backup software for servers and workstations"
license = "GPL-3.0-or-later"
license-files = ["LICENSE"]
readme = "README.md"
requires-python = ">=3.9"
classifiers=[
"Development Status :: 5 - Production/Stable",
"Environment :: Console",
"Intended Audience :: System Administrators",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"Programming Language :: Python",
"Topic :: Security :: Cryptography",
"Topic :: System :: Archiving :: Backup",
@@ -30,6 +31,8 @@ validate-borgmatic-config = "borgmatic.commands.validate_config:main"
[project.optional-dependencies]
Apprise = ["apprise"]
browse = ["textual", "binaryornot"]
browse-dev = ["textual-dev"]
[project.urls]
Homepage = "https://torsion.org/borgmatic"
@@ -44,7 +47,7 @@ namespaces = false
[tool.pytest.ini_options]
testpaths = "tests"
addopts = "--cov-report term-missing:skip-covered --cov=borgmatic --no-cov-on-fail --cov-fail-under=100 --ignore=tests/end-to-end --timeout=120"
addopts = "--cov-report term-missing:skip-covered --cov=borgmatic --no-cov-on-fail --cov-fail-under=100 --ignore=tests/end-to-end --timeout=120 --asyncio-mode=auto"
[tool.ruff]
line-length = 100
@@ -54,7 +57,7 @@ exclude = ["*.*/*"]
quote-style = "preserve"
[tool.ruff.lint]
preview = true
preview = false
extend-select = [
"A", # flake8-builtins: builtin shadowing
"B", # flake8-bugbear: bugs and design problems
+1 -1
View File
@@ -60,7 +60,7 @@ ProtectSystem=full
# includes, for instance, programs to snapshot filesystems (e.g. ZFS, LVM, and
# Btrfs). But rather than commenting this out entirely, one workaround may be to
# add "CAP_SYS_ADMIN".
CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW
CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_FOWNER CAP_NET_RAW
# Lower CPU and I/O priority.
Nice=19
+14 -5
View File
@@ -43,15 +43,19 @@ twine upload -r pypi --username __token__ "$wheel_path"
scripts/export-docs-from-image
docs_path=dist/borgmatic-docs.tar.gz
# Build stand-alone binary.
uv venv --clear binary
# Build stand-alone binaries.
uv venv --python 3.13 --clear binary
source binary/bin/activate
uv pip install -r binary_requirements.txt
uv pip install -r binary_requirements.txt nuitka[onefile]
nuitka --mode=onefile --enable-plugin=upx --include-package-data=borgmatic --include-data-dir=borgmatic.egg-info=borgmatic.egg-info --include-package=borgmatic.hooks --include-package=apprise --no-deployment-flag=self-execution borgmatic/commands/borgmatic.py
deactivate
rm -fr binary borgmatic.build borgmatic.dist
rm -fr borgmatic.build borgmatic.dist
standalone_binary_path="dist/borgmatic-${version}-binary-linux-glibc-x86_64"
mv borgmatic.bin "$standalone_binary_path"
nuitka --mode=onefile --enable-plugin=upx --include-package-data=borgmatic --include-data-dir=borgmatic.egg-info=borgmatic.egg-info --include-package=borgmatic.hooks --no-deployment-flag=self-execution borgmatic/commands/borgmatic.py
rm -fr binary borgmatic.build borgmatic.dist
deactivate
standalone_minimal_binary_path="dist/borgmatic-${version}-minimal-binary-linux-glibc-x86_64"
mv borgmatic.bin "$standalone_minimal_binary_path"
# Set release changelogs on projects.torsion.org and GitHub.
release_changelog="$(cat NEWS | sed '/^$/q' | grep -v '^\S')"
@@ -83,6 +87,11 @@ curl --silent --request POST \
--header "Authorization: token $projects_token" \
--header "Accept: application/json" \
--form attachment=@"$standalone_binary_path"
curl --silent --request POST \
"https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/releases/$release_id/assets?name=$(basename $standalone_minimal_binary_path)" \
--header "Authorization: token $projects_token" \
--header "Accept: application/json" \
--form attachment=@"$standalone_minimal_binary_path"
github-release create --token="$github_token" --owner=witten --repo=borgmatic --tag="$version" --target_commit="main" \
--name="borgmatic $version" --body="$release_changelog"
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
set -eou pipefail
uv pip compile --quiet --upgrade --annotation-style line test_requirements.in -o test_requirements.txt
uv pip compile --quiet --upgrade --annotation-style line binary_requirements.in -o binary_requirements.txt
# Carry forward certain packages with version ranges instead of absolute pins. This is because these
# packages are a pain to compile, and pre-built wheels available on pypi may not be available for
# pinned versions on particular platforms.
while IFS= read -r package_line; do
package_name=$(echo "$package_line" | cut -d ">" -f 1)
sed -i "s/^$package_name==.*/$package_line/" test_requirements.txt
done <<< $(grep ">" test_requirements.in)
sed -i "s/was autogenerated/was (mostly) autogenerated/" test_requirements.txt
+32
View File
@@ -0,0 +1,32 @@
apprise
attrs
binaryornot
certifi
charset-normalizer
click>=8.1.8
codespell
coverage
flexmock
idna
iniconfig
jsonschema
jsonschema-specifications
markdown
oauthlib
packaging
pluggy
pygments
pytest
pytest-asyncio
pytest-cov
pytest-timeout
pyyaml>5.0.0
referencing
requests
requests-oauthlib
rich
rpds-py
ruamel-yaml>0.15.0
textual
typing-extensions
urllib3
+37 -25
View File
@@ -1,28 +1,40 @@
apprise==1.9.9
attrs==26.1.0
certifi==2026.2.25
charset-normalizer==3.4.6
# This file was (mostly) autogenerated by uv via the following command:
# uv pip compile --annotation-style line test_requirements.in -o test_requirements.txt
apprise==1.10.0 # via -r test_requirements.in
attrs==26.1.0 # via jsonschema, referencing, -r test_requirements.in
binaryornot==0.6.0 # via -r test_requirements.in
certifi==2026.5.20 # via apprise, requests, -r test_requirements.in
charset-normalizer==3.4.7 # via requests, -r test_requirements.in
click>=8.1.8
codespell==2.4.2
coverage==7.13.5
flexmock==0.13.0
idna==3.11
iniconfig==2.3.0
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
markdown==3.10.2
oauthlib==3.3.1
packaging==26.0
pluggy==1.6.0
pygments==2.20.0
pytest==9.0.2
pytest-cov==7.1.0
pytest-timeout==2.4.0
codespell==2.4.2 # via -r test_requirements.in
coverage==7.14.0 # via pytest-cov, -r test_requirements.in
flexmock==0.13.0 # via -r test_requirements.in
idna==3.16 # via requests, -r test_requirements.in
iniconfig==2.3.0 # via pytest, -r test_requirements.in
jsonschema==4.26.0 # via -r test_requirements.in
jsonschema-specifications==2025.9.1 # via jsonschema, -r test_requirements.in
linkify-it-py==2.1.0 # via markdown-it-py
markdown==3.10.2 # via apprise, -r test_requirements.in
markdown-it-py==4.2.0 # via mdit-py-plugins, rich, textual
mdit-py-plugins==0.6.1 # via textual
mdurl==0.1.2 # via markdown-it-py
oauthlib==3.3.1 # via requests-oauthlib, -r test_requirements.in
packaging==26.2 # via pytest, -r test_requirements.in
platformdirs==4.9.6 # via textual
pluggy==1.6.0 # via pytest, pytest-cov, -r test_requirements.in
pygments==2.20.0 # via pytest, rich, textual, -r test_requirements.in
pytest==9.0.3 # via pytest-asyncio, pytest-cov, pytest-timeout, -r test_requirements.in
pytest-asyncio==1.3.0 # via -r test_requirements.in
pytest-cov==7.1.0 # via -r test_requirements.in
pytest-timeout==2.4.0 # via -r test_requirements.in
pyyaml>5.0.0
referencing==0.37.0
requests==2.33.1
requests-oauthlib==2.0.0
rpds-py==0.30.0
referencing==0.37.0 # via jsonschema, jsonschema-specifications, -r test_requirements.in
requests==2.34.2 # via apprise, requests-oauthlib, -r test_requirements.in
requests-oauthlib==2.0.0 # via apprise, -r test_requirements.in
rich==15.0.0 # via textual, -r test_requirements.in
rpds-py==0.30.0 # via jsonschema, referencing, -r test_requirements.in
ruamel-yaml>0.15.0
typing-extensions==4.15.0
urllib3==2.6.3
textual==8.2.7 # via -r test_requirements.in
typing-extensions==4.15.0 # via textual, -r test_requirements.in
uc-micro-py==2.0.0 # via linkify-it-py
urllib3==2.7.0 # via requests, -r test_requirements.in
@@ -57,11 +57,15 @@ mariadb_databases:
hostname: mariadb
username: root
password: test
# Needed for restoring users to the system database, because we're not actually dropping
# users first. (Dropping root and mariadb.sys would cause problems.)
restore_options: --force
- name: all
{mariadb_mysql_dump_format_option}
hostname: mariadb
username: root
password: test
restore_options: --force
mysql_databases:
- name: test
hostname: not-actually-mysql
@@ -132,6 +136,7 @@ mariadb_databases:
restore_port: 3307
restore_username: root
restore_password: test2
restore_options: --force
mysql_databases:
- name: test
container: not-actually-mysql
@@ -201,6 +206,7 @@ mariadb_databases:
restore_port: 3307
restore_username: root
restore_password: test2
restore_options: --force
mysql_databases:
- name: test
hostname: not-actually-mysql
@@ -0,0 +1,68 @@
from flexmock import flexmock
import borgmatic.actions.browse.app
import borgmatic.actions.browse.configuration_files_list
import borgmatic.actions.browse.logs
import borgmatic.actions.browse.repositories_list
async def test_browse_app_with_multiple_configs_uses_configuration_files_list():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test():
app.query_one(selector='Header')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].configs == app.configs
app.query_one(selector='Logs')
app.query_one(selector='Footer')
async def test_browse_app_with_one_config_uses_repositories_list():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test():
app.query_one(selector='Header')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[0].config == app.configs['test1.yaml']
app.query_one(selector='Logs')
app.query_one(selector='Footer')
async def test_browse_app_key_toggles_logs_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
logs_panel = app.query_one('#logs')
await pilot.press('v')
await pilot.pause()
assert logs_panel.styles.display == 'block'
@@ -0,0 +1,86 @@
import textual.app
import textual.widgets.option_list
from flexmock import flexmock
from borgmatic.actions.browse import archives_list as module
async def test_archives_list_on_mount_does_not_raise():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.workers).should_receive('add_repository_archives')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
flexmock(archives_list.archive_loaded).should_receive('subscribe')
async with textual.app.App().run_test():
archives_list.on_mount()
def test_archives_list_on_archive_loaded_with_loading_done_removes_loading_indicator():
loading_timer = flexmock()
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).and_return(loading_timer)
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
flexmock(loading_timer).should_receive('stop')
flexmock(archives_list).should_receive('remove_option').with_args('loading-indicator').once()
flexmock(archives_list).should_receive('add_options').never()
archives_list.on_archive_loaded(module.borgmatic.actions.browse.workers.LOADING_DONE)
def test_archives_list_on_archive_loaded_adds_archive_name():
loading_timer = flexmock()
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).and_return(loading_timer)
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
flexmock(loading_timer).should_receive('stop').never()
loading_indicator = flexmock()
flexmock(archives_list).should_receive('get_option').and_return(loading_indicator)
flexmock(archives_list).should_receive('remove_option').with_args('loading-indicator').once()
flexmock(archives_list).should_receive('add_options').once()
archives_list.on_archive_loaded('archive')
def test_archives_list_on_option_list_option_highlighted_with_highlighted_none_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
archives_list.highlighted = None
archives_list.on_option_list_option_highlighted(event=flexmock())
assert archives_list.highlighted_option_changed is False
def test_archives_list_on_option_list_option_highlighted_with_highlighted_zero_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
archives_list.highlighted = 0
archives_list.on_option_list_option_highlighted(event=flexmock())
assert archives_list.highlighted_option_changed is False
def test_archives_list_on_option_list_option_highlighted_with_existing_option_and_highlighted_zero_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
archives_list.add_option(textual.widgets.option_list.Option('zero', id='zero'))
archives_list.highlighted = 0
archives_list.on_option_list_option_highlighted(event=flexmock())
assert archives_list.highlighted_option_changed is False
def test_archives_list_on_option_list_option_highlighted_with_highlighted_non_zero_marks_it_changed():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
archives_list.add_option(textual.widgets.option_list.Option('zero', id='zero'))
archives_list.add_option(textual.widgets.option_list.Option('one', id='one'))
archives_list.highlighted = 1
archives_list.on_option_list_option_highlighted(event=flexmock())
assert archives_list.highlighted_option_changed is True
@@ -0,0 +1,522 @@
import pytest
import textual.widgets.option_list
from flexmock import flexmock
import borgmatic.actions.browse.app
import borgmatic.actions.browse.archive
import borgmatic.actions.browse.carousel
import borgmatic.actions.browse.loading
import borgmatic.actions.browse.logs
import borgmatic.actions.browse.workers
from borgmatic.actions.browse import carousel as module
def test_make_next_panel_with_configuration_files_list_returns_repositories_list():
configs = {'test.yaml': {'repositories': [{'path': 'test.borg'}]}}
repositories_list = module.make_next_panel(
focused_panel=borgmatic.actions.browse.configuration_files_list.Configuration_files_list(
configs
),
option_id='test.yaml',
)
assert isinstance(
repositories_list, borgmatic.actions.browse.repositories_list.Repositories_list
)
assert repositories_list.config == configs['test.yaml']
def test_make_next_panel_with_repositories_list_returns_archives_list():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('add_repository_archives')
flexmock(borgmatic.actions.browse.archives_list.Archives_list).should_receive('app').and_return(
flexmock()
)
archives_list = module.make_next_panel(
focused_panel=borgmatic.actions.browse.repositories_list.Repositories_list(config),
option_id=0,
)
assert isinstance(archives_list, borgmatic.actions.browse.archives_list.Archives_list)
assert archives_list.config == config
assert archives_list.repository == config['repositories'][0]
def test_make_next_panel_with_archives_list_returns_directory_list():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('add_repository_archives')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.archives_list.Archives_list).should_receive('app').and_return(
flexmock()
)
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.make_next_panel(
focused_panel=borgmatic.actions.browse.archives_list.Archives_list(
config, config['repositories'][0]
),
option_id='archive',
)
assert isinstance(directory_list, borgmatic.actions.browse.directory_list.Directory_list)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
def test_make_next_panel_with_root_directory_list_and_selected_directory_option_returns_new_directory_list():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.workers).should_receive('Archive_path_loaded').replace_with(
flexmock(
path_hierarchy={
'etc': {},
},
complete=False,
),
)
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config, config['repositories'][0], 'archive'
)
flexmock(focused_panel).should_receive('get_option').and_return(flexmock(prompt='📁 etc'))
directory_list = module.make_next_panel(focused_panel=focused_panel, option_id='etc')
assert isinstance(directory_list, borgmatic.actions.browse.directory_list.Directory_list)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
assert directory_list.archive_name == 'archive'
assert directory_list.path_components == ('etc',)
def test_make_next_panel_with_non_root_directory_list_and_selected_directory_option_returns_new_directory_list():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.workers).should_receive('Archive_path_loaded').replace_with(
flexmock(
path_hierarchy={
'etc': {
'borgmatic': {},
},
},
complete=False,
),
)
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config,
config['repositories'][0],
'archive',
path_components=('etc',),
)
flexmock(focused_panel).should_receive('get_option').and_return(flexmock(prompt='📁 borgmatic'))
directory_list = module.make_next_panel(focused_panel=focused_panel, option_id='borgmatic')
assert isinstance(directory_list, borgmatic.actions.browse.directory_list.Directory_list)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
assert directory_list.archive_name == 'archive'
assert directory_list.path_components == ('etc', 'borgmatic')
def test_make_next_panel_with_root_directory_list_and_selected_file_option_returns_new_file_preview():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.workers).should_receive('load_file_preview')
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(borgmatic.actions.browse.file_preview.File_preview).should_receive('app').and_return(
flexmock()
)
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config, config['repositories'][0], 'archive'
)
flexmock(focused_panel).should_receive('get_option').and_return(
flexmock(prompt='📄 config.yaml')
)
directory_list = module.make_next_panel(focused_panel=focused_panel, option_id='config.yaml')
assert isinstance(directory_list, borgmatic.actions.browse.file_preview.File_preview)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
assert directory_list.archive_name == 'archive'
assert directory_list.file_path == 'config.yaml'
def test_make_next_panel_with_non_root_directory_list_and_selected_file_option_returns_new_file_preview():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.workers).should_receive('load_file_preview')
flexmock(borgmatic.actions.browse.workers).should_receive('Archive_path_loaded').replace_with(
flexmock(
path_hierarchy={
'etc': {
'borgmatic': {
'config.yaml': borgmatic.actions.browse.archive.Archive_path(
'-', 'config.yaml', ''
),
},
},
},
complete=False,
),
)
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(borgmatic.actions.browse.file_preview.File_preview).should_receive('app').and_return(
flexmock()
)
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config,
config['repositories'][0],
'archive',
path_components=('etc', 'borgmatic'),
)
flexmock(focused_panel).should_receive('get_option').and_return(
flexmock(prompt='📄 config.yaml')
)
directory_list = module.make_next_panel(focused_panel=focused_panel, option_id='config.yaml')
assert isinstance(directory_list, borgmatic.actions.browse.file_preview.File_preview)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
assert directory_list.archive_name == 'archive'
assert directory_list.file_path == 'etc/borgmatic/config.yaml'
def test_make_next_panel_with_unsupported_focused_panel_returns_none():
assert module.make_next_panel(focused_panel=flexmock(), option_id='hmmm') is None
@pytest.mark.parametrize('icon', ('🔗', '🚰', '🐙'))
def test_make_next_panel_with_directory_list_and_unsupported_selected_option_returns_none(icon):
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config, config['repositories'][0], 'archive'
)
flexmock(focused_panel).should_receive('get_option').and_return(
flexmock(prompt=f'{icon} config.yaml')
)
assert module.make_next_panel(focused_panel=focused_panel, option_id='config.yaml') is None
async def test_carousel_previous_action_with_multiple_configs_does_not_raise():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('left')
async def test_carousel_previous_action_with_one_config_does_not_raise():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('left')
async def test_carousel_next_action_with_multiple_configs_advances_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'none'
assert isinstance(
carousel.panels[1], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[1].styles.display == 'block'
assert carousel.panels[1].highlighted == 0
assert app.focused == carousel.panels[1]
async def test_carousel_next_action_with_one_config_advances_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
flexmock(borgmatic.actions.browse.workers).should_receive('add_repository_archives')
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[0].styles.display == 'none'
assert isinstance(carousel.panels[1], borgmatic.actions.browse.archives_list.Archives_list)
assert carousel.panels[1].styles.display == 'block'
assert carousel.panels[1].highlighted == 0
assert app.focused == carousel.panels[1]
async def test_carousel_next_action_with_no_next_panel_does_not_advance():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
flexmock(borgmatic.actions.browse.workers).should_receive('add_repository_archives')
flexmock(module).should_receive('make_next_panel')
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[0].styles.display == 'block'
assert app.focused == carousel.panels[0]
async def test_carousel_next_action_and_previous_action_returns_to_original_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('enter')
await pilot.press('left')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 0
assert isinstance(
carousel.panels[1], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[1].styles.display == 'none'
assert app.focused == carousel.panels[0]
async def test_carousel_next_action_and_previous_action_and_next_action_reuses_next_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
flexmock(borgmatic.actions.browse.carousel).should_call('make_next_panel').once()
async with app.run_test() as pilot:
await pilot.press('enter')
await pilot.press('left')
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'none'
assert isinstance(
carousel.panels[1], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[1].styles.display == 'block'
assert carousel.panels[1].highlighted == 0
assert app.focused == carousel.panels[1]
async def test_carousel_next_action_with_multiple_configs_and_no_next_panel_does_not_advance():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
flexmock(module).should_receive('make_next_panel').and_return(None)
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 0
assert app.focused == carousel.panels[0]
async def test_carousel_next_action_and_previous_action_and_down_truncates_next_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('enter')
await pilot.press('left')
await pilot.press('down')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 1
assert app.focused == carousel.panels[0]
async def test_carousel_down_does_not_raise():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('down')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 1
assert app.focused == carousel.panels[0]
async def test_carousel_up_does_not_raise():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('down')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 1
assert app.focused == carousel.panels[0]
async def test_carousel_next_action_and_select_dot_dot_returns_to_original_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
carousel.panels[1].options[0] = textual.widgets.option_list.Option('..', id='..')
await pilot.press('enter')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 0
assert isinstance(
carousel.panels[1], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[1].styles.display == 'none'
assert app.focused == carousel.panels[0]
@@ -0,0 +1,34 @@
from flexmock import flexmock
from borgmatic.actions.browse import configuration_files_list as module
def test_configuration_files_list_adds_config_paths_as_options():
flexmock(module.os.path).should_receive('expanduser').and_return('/home/user')
configuration_files_list = module.Configuration_files_list(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
assert len(configuration_files_list.options) == 2
assert configuration_files_list.options[0].prompt == 'test1.yaml'
assert configuration_files_list.options[0].id == 'test1.yaml'
assert configuration_files_list.options[1].prompt == 'test2.yaml'
assert configuration_files_list.options[1].id == 'test2.yaml'
def test_configuration_files_list_collapses_home_directory_in_config_path_option():
flexmock(module.os.path).should_receive('expanduser').and_return('/home/user')
configuration_files_list = module.Configuration_files_list(
configs={
'/home/user/test.yaml': {'repositories': [{'path': '/home/user/test.borg'}]},
}
)
assert len(configuration_files_list.options) == 1
assert configuration_files_list.options[0].prompt == '~/test.yaml'
assert configuration_files_list.options[0].id == '/home/user/test.yaml'
@@ -0,0 +1,287 @@
import textual.widgets
import textual.widgets.option_list
from flexmock import flexmock
from borgmatic.actions.browse import directory_list as module
def test_add_archive_paths_with_only_duplicate_paths_bails():
directory_list = textual.widgets.OptionList()
directory_list.path_components = ('etc',)
directory_list.add_option(textual.widgets.option_list.Option('foo', id='foo'))
directory_list.add_option(textual.widgets.option_list.Option('bar', id='bar'))
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(directory_list).should_receive('set_options').never()
module.add_archive_paths(
directory_list=directory_list,
config=config,
repository=config['repositories'][0],
archive_name='archive',
archive_paths=(
flexmock(path_type='-', file_path='etc/foo/one.txt', link_target=''),
flexmock(path_type='-', file_path='etc/foo/two.txt', link_target=''),
),
)
def test_add_archive_paths_adds_ands_sorts_and_filters_and_deduplicates():
directory_list = textual.widgets.OptionList()
directory_list.path_components = ('etc',)
directory_list.add_option(textual.widgets.option_list.Option('📄 foo', id='foo'))
directory_list.add_option(textual.widgets.option_list.Option('📄 bar', id='bar'))
directory_list.highlighted = 1
directory_list.highlighted_option_changed = True
config = {'repositories': [{'path': 'test.borg'}]}
module.add_archive_paths(
directory_list=directory_list,
config=config,
repository=config['repositories'][0],
archive_name='archive',
archive_paths=(
flexmock(path_type='d', file_path='etc/quux', link_target=''),
flexmock(path_type='-', file_path='etc/foo', link_target=''),
flexmock(path_type='-', file_path='etc/baz', link_target=''),
flexmock(path_type='d', file_path='root/nope', link_target=''),
flexmock(path_type='d', file_path='etc/other', link_target=''),
),
)
assert len(directory_list.options) == 5
assert directory_list.options[0].prompt == '📁 other'
assert directory_list.options[0].id == 'other'
assert directory_list.options[1].prompt == '📁 quux'
assert directory_list.options[1].id == 'quux'
assert directory_list.options[2].prompt == '📄 bar'
assert directory_list.options[2].id == 'bar'
assert directory_list.options[3].prompt == '📄 baz'
assert directory_list.options[3].id == 'baz'
assert directory_list.options[4].prompt == '📄 foo'
assert directory_list.options[4].id == 'foo'
assert directory_list.highlighted == 2
def test_add_archive_paths_highlights_first_option_if_highlight_has_not_changed():
directory_list = textual.widgets.OptionList()
directory_list.path_components = ('etc',)
directory_list.add_option(textual.widgets.option_list.Option('📄 foo', id='foo'))
directory_list.add_option(textual.widgets.option_list.Option('📄 bar', id='bar'))
directory_list.highlighted = None
directory_list.highlighted_option_changed = False
config = {'repositories': [{'path': 'test.borg'}]}
module.add_archive_paths(
directory_list=directory_list,
config=config,
repository=config['repositories'][0],
archive_name='archive',
archive_paths=(flexmock(path_type='-', file_path='etc/baz', link_target=''),),
)
assert len(directory_list.options) == 3
assert directory_list.options[0].prompt == '📄 bar'
assert directory_list.options[0].id == 'bar'
assert directory_list.options[1].prompt == '📄 baz'
assert directory_list.options[1].id == 'baz'
assert directory_list.options[2].prompt == '📄 foo'
assert directory_list.options[2].id == 'foo'
assert directory_list.highlighted == 0
def test_add_archive_paths_retains_loading_indicator_at_bottom():
directory_list = textual.widgets.OptionList()
directory_list.path_components = ('etc',)
directory_list.add_option(textual.widgets.option_list.Option('📄 foo', id='foo'))
directory_list.add_option(textual.widgets.option_list.Option('📄 bar', id='bar'))
directory_list.add_option(
textual.widgets.option_list.Option('loading!!!', id='loading-indicator')
)
directory_list.highlighted = 0
directory_list.highlighted_option_changed = True
config = {'repositories': [{'path': 'test.borg'}]}
module.add_archive_paths(
directory_list=directory_list,
config=config,
repository=config['repositories'][0],
archive_name='archive',
archive_paths=(flexmock(path_type='-', file_path='etc/baz', link_target=''),),
)
assert len(directory_list.options) == 4
assert directory_list.options[0].prompt == '📄 bar'
assert directory_list.options[0].id == 'bar'
assert directory_list.options[1].prompt == '📄 baz'
assert directory_list.options[1].id == 'baz'
assert directory_list.options[2].prompt == '📄 foo'
assert directory_list.options[2].id == 'foo'
assert directory_list.options[3].prompt == 'loading!!!'
assert directory_list.options[3].id == 'loading-indicator'
assert directory_list.highlighted == 2
def test_directory_list_with_root_directory_adds_loading_indicator():
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).once()
directory_list = module.Directory_list(
config=flexmock(), repository=flexmock(), archive_name='archive'
)
assert directory_list.border_title == '📁 archive'
assert len(directory_list.options) == 0
assert not directory_list.path_loaded.complete
def test_directory_list_with_already_complete_loading_skips_loading_indicator():
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).never()
flexmock(module.borgmatic.actions.browse.workers).should_receive('load_archive_paths').never()
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=True),
path_components=('etc',),
)
assert directory_list.border_title == '📁 etc'
assert len(directory_list.options) == 1
assert directory_list.options[0].prompt == '📁 ..'
assert directory_list.options[0].id == '..'
def test_directory_list_on_mount_with_root_directory_loads_archive_paths():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(module.borgmatic.actions.browse.workers).should_receive('load_archive_paths').once()
flexmock(module).should_receive('add_archive_paths').never()
directory_list = module.Directory_list(
config=flexmock(), repository=flexmock(), archive_name='archive'
)
flexmock(directory_list.path_loaded).should_receive('subscribe')
directory_list.on_mount()
def test_directory_list_on_mount_with_non_root_directory_adds_archive_paths():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(module.borgmatic.actions.browse.workers).should_receive('load_archive_paths').never()
flexmock(module).should_receive('add_archive_paths').once()
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
flexmock(directory_list.path_loaded).should_receive('subscribe')
directory_list.on_mount()
def test_on_archive_path_loaded_with_loading_done_signal_removes_loading_indicator():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(module).should_receive('add_archive_paths').never()
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
directory_list.timer = flexmock(stop=lambda: None)
flexmock(directory_list).should_receive('remove_option').once()
directory_list.on_archive_path_loaded(data=module.borgmatic.actions.browse.workers.LOADING_DONE)
def test_on_archive_path_loaded_with_path_loaded_signal_adds_archive_path():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(module).should_receive('add_archive_paths').once()
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
flexmock(directory_list).should_receive('remove_option').never()
directory_list.on_archive_path_loaded(data=flexmock())
def test_directory_list_on_option_list_option_highlighted_with_highlighted_none_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
directory_list.highlighted = None
directory_list.on_option_list_option_highlighted(event=flexmock())
assert directory_list.highlighted_option_changed is False
def test_directory_list_on_option_list_option_highlighted_with_highlighted_zero_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
directory_list.add_option(textual.widgets.option_list.Option('zero', id='zero'))
directory_list.highlighted = 0
directory_list.on_option_list_option_highlighted(event=flexmock())
assert directory_list.highlighted_option_changed is False
def test_directory_list_on_option_list_option_highlighted_with_highlighted_non_zero_marks_it_changed():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
directory_list.add_option(textual.widgets.option_list.Option('zero', id='zero'))
directory_list.add_option(textual.widgets.option_list.Option('one', id='one'))
directory_list.highlighted = 1
directory_list.on_option_list_option_highlighted(event=flexmock())
assert directory_list.highlighted_option_changed is True
@@ -0,0 +1,53 @@
import textual.widgets.option_list
from flexmock import flexmock
from borgmatic.actions.browse import file_preview as module
def test_file_preview_does_not_raise():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
module.File_preview(
config=flexmock(), repository=flexmock(), archive_name='archive', file_path='foo/bar.txt'
)
async def test_file_preview_on_mount_does_not_raise():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.workers).should_receive('load_file_preview')
file_preview = module.File_preview(
config=flexmock(), repository=flexmock(), archive_name='archive', file_path='foo.txt'
)
flexmock(file_preview.file_preview_loaded).should_receive('subscribe')
async with textual.app.App().run_test():
file_preview.on_mount()
def test_file_preview_on_file_preview_loaded_with_none_file_contents_displays_error():
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).and_return(flexmock(stop=lambda: None))
file_preview = module.File_preview(
config=flexmock(), repository=flexmock(), archive_name='archive', file_path='foo.txt'
)
flexmock(file_preview).should_receive('write').with_args(
'Cannot display a preview for this file'
).once()
file_preview.on_file_preview_loaded(None)
def test_file_preview_on_file_preview_loaded_with_file_contents_displays_contents():
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).and_return(flexmock(stop=lambda: None))
file_preview = module.File_preview(
config=flexmock(), repository=flexmock(), archive_name='archive', file_path='foo.txt'
)
flexmock(file_preview).should_receive('write').with_args(
'Cannot display a preview for this file'
).never()
flexmock(file_preview).should_receive('write').once()
file_preview.on_file_preview_loaded('hi')
@@ -0,0 +1,97 @@
import pytest
import textual.app
import textual.widgets
from flexmock import flexmock
from borgmatic.actions.browse import loading as module
def test_update_inline_loading_indicator_with_option_list_adds_a_dot():
widget = textual.widgets.OptionList()
widget.add_option(textual.widgets.option_list.Option('HOLD.', id='loading-indicator'))
module.update_inline_loading_indicator(widget)
assert len(widget.options) == 1
assert widget.options[0].prompt == 'HOLD..'
def test_update_inline_loading_indicator_with_option_list_wraps_dots_beyond_three():
widget = textual.widgets.OptionList()
widget.add_option(textual.widgets.option_list.Option('HOLD...', id='loading-indicator'))
module.update_inline_loading_indicator(widget)
assert len(widget.options) == 1
assert widget.options[0].prompt == 'HOLD'
def test_update_inline_loading_indicator_with_option_list_and_missing_indicator_does_not_raise():
widget = textual.widgets.OptionList()
module.update_inline_loading_indicator(widget)
assert len(widget.options) == 0
async def test_update_inline_loading_indicator_with_rich_log_adds_a_dot():
async with textual.app.App().run_test():
widget = textual.widgets.RichLog()
widget._size_known = True
widget.write('HOLD.')
module.update_inline_loading_indicator(widget)
assert str(widget.lines[0].text) == 'HOLD..'
async def test_update_inline_loading_indicator_with_rich_log_wraps_dots_beyond_three():
async with textual.app.App().run_test():
widget = textual.widgets.RichLog()
widget._size_known = True
widget.write('HOLD...')
module.update_inline_loading_indicator(widget)
assert str(widget.lines[0].text) == 'HOLD'
def test_update_inline_loading_indicator_with_rich_log_and_missing_indicator_does_not_raise():
widget = textual.widgets.RichLog()
module.update_inline_loading_indicator(widget)
assert len(widget.lines) == 0
def test_update_inline_loading_indicator_with_unsupported_widget_type_raises():
with pytest.raises(ValueError):
module.update_inline_loading_indicator(flexmock())
def test_add_inline_loading_indicator_with_option_list_adds_loading_indicator_option():
widget = textual.widgets.OptionList()
flexmock(widget).should_receive('set_interval')
module.add_inline_loading_indicator(widget)
assert len(widget.options) == 1
assert widget.options[0].prompt == module.LOADING_MESSAGE
assert widget.options[0].id == 'loading-indicator'
assert widget.highlighted is None
async def test_add_inline_loading_indicator_with_rich_log_writes_loading_indicator_text():
async with textual.app.App().run_test():
widget = textual.widgets.RichLog()
widget._size_known = True
flexmock(widget).should_receive('set_interval')
module.add_inline_loading_indicator(widget)
assert str(widget.lines[0].text) == module.LOADING_MESSAGE
def test_add_inline_loading_indicator_with_unsupported_widget_type_raises():
with pytest.raises(ValueError):
module.add_inline_loading_indicator(flexmock())
@@ -0,0 +1,37 @@
import contextlib
from flexmock import flexmock
from borgmatic.actions.browse import logs as module
def test_log_to_widget_adds_our_handler_and_removes_default_handler():
default_handler = module.borgmatic.logger.Multi_stream_handler({})
root_logger = module.logging.getLogger()
root_logger.addHandler(default_handler)
browse_log_handler = None
try:
module.log_to_widget(flexmock())
with contextlib.suppress(StopIteration):
browse_log_handler = next(
handler
for handler in root_logger.handlers
if isinstance(handler, module.Browse_log_handler)
)
assert browse_log_handler
finally:
if browse_log_handler:
root_logger.removeHandler(browse_log_handler)
assert not any(
handler
for handler in root_logger.handlers
if isinstance(handler, module.borgmatic.logger.Multi_stream_handler)
)
def test_logs_does_not_raise():
module.Logs()
@@ -0,0 +1,12 @@
from borgmatic.actions.browse import repositories_list as module
def test_repositories_list_populates_options():
config = {'repositories': [{'path': 'test1.borg'}, {'path': 'test2.borg', 'label': 'two'}]}
repositories_list = module.Repositories_list(config=config)
assert len(repositories_list.options) == 2
assert repositories_list.options[0].prompt == 'test1.borg'
assert repositories_list.options[0].id == 0
assert repositories_list.options[1].prompt == 'two'
assert repositories_list.options[1].id == 1
@@ -0,0 +1,19 @@
from flexmock import flexmock
from borgmatic.actions.browse import workers as module
def test_archive_path_loaded_publish_records_complete():
signal = module.Archive_path_loaded(owner=flexmock(), name='Bob')
signal.publish(module.LOADING_DONE)
assert signal.complete
def test_archive_path_loaded_publish_records_published_path():
archive_path = module.borgmatic.actions.browse.archive.Archive_path('-', 'foo/bar.txt', '')
signal = module.Archive_path_loaded(owner=flexmock(), name='Bob')
signal.publish(archive_path)
assert signal.path_hierarchy == {'foo': {'bar.txt': archive_path}}
assert not signal.complete
+12
View File
@@ -57,6 +57,8 @@ def test_make_rename_command_includes_log_debug():
def test_make_rename_command_includes_dry_run():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=True,
repository_name='repo',
@@ -79,6 +81,8 @@ def test_make_rename_command_includes_dry_run():
def test_make_rename_command_includes_remote_path():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
@@ -102,6 +106,8 @@ def test_make_rename_command_includes_remote_path():
def test_make_rename_command_includes_umask():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
@@ -125,6 +131,8 @@ def test_make_rename_command_includes_umask():
def test_make_rename_command_includes_log_json():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
@@ -140,6 +148,8 @@ def test_make_rename_command_includes_log_json():
def test_make_rename_command_includes_lock_wait():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
@@ -163,6 +173,8 @@ def test_make_rename_command_includes_lock_wait():
def test_make_rename_command_includes_extra_borg_options():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
+2 -2
View File
@@ -315,10 +315,10 @@ def test_write_configuration_with_already_existing_file_raises():
module.write_configuration('config.yaml', 'config: yaml')
def test_write_configuration_with_already_existing_file_and_overwrite_does_not_raise():
def test_write_configuration_with_already_existing_file_and_overwrite_does_not_raise(tmp_path):
flexmock(os.path).should_receive('exists').and_return(True)
module.write_configuration('/tmp/config.yaml', 'config: yaml', overwrite=True)
module.write_configuration(str(tmp_path / 'config.yaml'), 'config: yaml', overwrite=True)
def test_write_configuration_with_already_existing_directory_does_not_raise():
+3 -3
View File
@@ -222,11 +222,11 @@ def test_parse_configuration_merges_include():
assert logs == []
def test_parse_configuration_raises_for_missing_config_file():
def test_parse_configuration_raises_for_missing_config_file(tmp_path):
with pytest.raises(FileNotFoundError):
module.parse_configuration(
'/tmp/config.yaml',
'/tmp/schema.yaml',
str(tmp_path / 'nonexistent' / 'config.yaml'),
str(tmp_path / 'nonexistent' / 'schema.yaml'),
arguments={'global': flexmock()},
)
+37
View File
@@ -557,6 +557,43 @@ def test_log_outputs_with_unfinished_process_re_polls():
)
def test_log_outputs_includes_error_output_when_output_spans_multiple_chunks():
flexmock(module.logger).should_receive('log')
flexmock(module).should_receive('interpret_exit_code').and_return(module.Exit_status.ERROR)
flexmock(module).should_receive('command_for_process').and_return('python')
process = subprocess.Popen(
[
sys.executable,
'-c',
(
'import os, sys; '
f'os.write(sys.stdout.fileno(), b"x" * {module.READ_CHUNK_SIZE + 10}); '
'os.write(sys.stdout.fileno(), b"\\nERROR: critical failure"); '
'os.close(1); '
'os._exit(2)'
),
],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
flexmock(module).should_receive('output_buffers_for_process').and_return((process.stdout,))
with pytest.raises(subprocess.CalledProcessError) as error:
tuple(
module.log_outputs(
(process,),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
)
assert error.value.output
assert 'ERROR: critical failure' in error.value.output
def test_read_lines_uses_system_locale_when_decoding_output():
flexmock(module.locale).should_receive('getpreferredencoding').and_return('ISO-8859-1')
process = subprocess.Popen(['echo', b'\xc4pple'], stdout=subprocess.PIPE)
+90
View File
@@ -0,0 +1,90 @@
from flexmock import flexmock
from borgmatic.actions.browse import archive as module
def test_get_repository_archives_does_not_raise():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.repo_list).should_receive('list_repository').and_return('{}')
assert module.get_repository_archives(config, config['repositories'][0]) == {}
def test_get_archive_paths_returns_each_as_archive_path_with_metadata():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
{'path': 'foo.txt', 'type': '-', 'linktarget': ''},
{'path': 'bar.txt', 'type': 'l', 'linktarget': 'foo.txt'},
{'path': 'etc', 'type': 'd', 'linktarget': ''},
)
assert tuple(module.get_archive_paths(config, config['repositories'][0], 'archive')) == (
module.Archive_path('-', 'foo.txt', ''),
module.Archive_path('l', 'bar.txt', 'foo.txt'),
module.Archive_path('d', 'etc', ''),
)
def test_get_archive_file_content_with_binary_file_bails():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
flexmock(stdout=flexmock(readlines=lambda hint: [b'foo\n', b'bar\n'])),
)
flexmock(module.binaryornot.helpers).should_receive('is_binary_string').and_return(True)
assert (
module.get_archive_file_content(config, config['repositories'][0], 'archive', 'etc/foo.txt')
is None
)
def test_get_archive_file_content_decodes_content():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
flexmock(stdout=flexmock(readlines=lambda hint: [b'foo\n', b'bar\n'])),
)
flexmock(module.binaryornot.helpers).should_receive('is_binary_string').and_return(False)
assert (
module.get_archive_file_content(config, config['repositories'][0], 'archive', 'etc/foo.txt')
== 'foo\nbar\n'
)
def test_get_archive_file_content_with_large_file_truncates_content():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module).READLINES_HINT_BYTES = 3
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
flexmock(stdout=flexmock(readlines=lambda hint: [b'foo\n', b'bar\n'])),
)
flexmock(module.binaryornot.helpers).should_receive('is_binary_string').and_return(False)
assert (
module.get_archive_file_content(config, config['repositories'][0], 'archive', 'etc/foo.txt')
== f'foo\nbar\n\n{module.TRUNCATION_MESSAGE}'
)
def test_get_archive_file_content_with_unicode_decode_error_does_not_raise():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
flexmock(stdout=flexmock(readlines=lambda hint: [b'foo\n', b'\xc3\n'])),
)
flexmock(module.binaryornot.helpers).should_receive('is_binary_string').and_return(False)
assert (
module.get_archive_file_content(config, config['repositories'][0], 'archive', 'etc/foo.txt')
is None
)
@@ -0,0 +1,94 @@
from flexmock import flexmock
from borgmatic.actions.browse import directory_list as module
def test_get_relative_archive_path_components_strips_off_current_directory():
assert module.get_relative_archive_path_components(
flexmock(file_path='foo/bar/baz/quux.txt'), ('foo', 'bar')
) == ('baz', 'quux.txt')
def test_get_relative_archive_path_components_with_root_current_directory_strips_off_nothing():
assert module.get_relative_archive_path_components(
flexmock(file_path='foo/bar/baz/quux.txt'), ()
) == ('foo', 'bar', 'baz', 'quux.txt')
def test_get_relative_archive_path_components_with_non_matching_paths_returns_none():
assert (
module.get_relative_archive_path_components(
flexmock(file_path='foo/bar/baz/quux.txt'), ('etc',)
)
is None
)
def test_make_directory_list_option_with_file_path_makes_file_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='-', file_path='foo/bar/baz.txt', link_target=''), ('baz.txt',)
)
assert option.prompt == '📄 baz.txt'
assert option.id == 'baz.txt'
def test_make_directory_list_option_with_directory_path_makes_directory_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='d', file_path='foo/bar/baz', link_target=''), ('baz',)
)
assert option.prompt == '📁 baz'
assert option.id == 'baz'
def test_make_directory_list_option_with_contained_file_path_makes_directory_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='d', file_path='foo/bar/baz.txt', link_target=''),
(
'bar',
'baz.txt',
),
)
assert option.prompt == '📁 bar'
assert option.id == 'bar'
def test_make_directory_list_option_with_link_path_makes_link_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='l', file_path='foo/bar/baz.txt', link_target='quux.txt'), ('baz.txt',)
)
assert option.prompt == '🔗 baz.txt → quux.txt'
assert option.id == 'baz.txt'
def test_make_directory_list_option_with_pipe_path_makes_pipe_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='p', file_path='foo/bar/baz.txt', link_target=''), ('baz.txt',)
)
assert option.prompt == '🚰 baz.txt'
assert option.id == 'baz.txt'
def test_make_directory_list_option_with_unknown_path_makes_unknown_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='wtf', file_path='foo/bar/baz.txt', link_target=''), ('baz.txt',)
)
assert option.prompt == '❓ baz.txt'
assert option.id == 'baz.txt'
+87
View File
@@ -0,0 +1,87 @@
from flexmock import flexmock
from borgmatic.actions.browse import logs as module
def test_rich_color_formatter_format_colors_log_record_based_on_level():
formatted = module.Rich_color_formatter().format(
module.logging.makeLogRecord(
dict(
levelno=module.logging.ERROR,
levelname='ERROR',
msg='oh no',
)
),
)
assert formatted == '[bright_red]oh no[/bright_red]'
def test_rich_color_formatter_format_includes_prefix():
formatter = module.Rich_color_formatter()
formatter.prefix = 'sup'
formatted = formatter.format(
module.logging.makeLogRecord(
dict(
levelno=module.logging.ERROR,
levelname='ERROR',
msg='oh no',
),
),
)
assert formatted == '[bright_red]sup: oh no[/bright_red]'
def test_browse_log_handler_emit_from_worker_thread_calls_write_in_main_thread():
flexmock(module.textual.worker).should_receive('get_current_worker')
logs_widget = flexmock(write=lambda message: None, app=flexmock())
flexmock(logs_widget.app).should_receive('call_from_thread').with_args(
logs_widget.write, 'hi'
).once()
module.Browse_log_handler(logs_widget).emit(
module.logging.makeLogRecord(
dict(
levelno=module.logging.DEBUG,
levelname='DEBUG',
msg='hi',
),
),
)
def test_browse_log_handler_emit_from_main_thread_calls_write_directly():
flexmock(module.textual.worker).should_receive('get_current_worker').and_raise(RuntimeError)
logs_widget = flexmock(write=lambda message: None, app=flexmock())
flexmock(logs_widget.app).should_receive('call_from_thread').never()
flexmock(logs_widget).should_receive('write').with_args('hi').once()
module.Browse_log_handler(logs_widget).emit(
module.logging.makeLogRecord(
dict(
levelno=module.logging.DEBUG,
levelname='DEBUG',
msg='hi',
),
),
)
def test_browse_log_handler_emit_from_main_thread_with_no_active_app_does_not_raise():
flexmock(module.textual.worker).should_receive('get_current_worker').and_raise(RuntimeError)
logs_widget = flexmock(write=lambda message: None, app=flexmock())
flexmock(logs_widget.app).should_receive('call_from_thread').never()
flexmock(logs_widget).should_receive('write').with_args('hi').and_raise(
module.textual._context.NoActiveAppError
)
module.Browse_log_handler(logs_widget).emit(
module.logging.makeLogRecord(
dict(
levelno=module.logging.DEBUG,
levelname='DEBUG',
msg='hi',
),
),
)

Some files were not shown because too many files have changed in this diff Show More