Compare commits

...
132 Commits
Author SHA1 Message Date
Dan Helfman 4af31db99c Bump version for release. 2026-06-01 15:52:45 -07:00
Dan Helfman 7cd9e6b361 Add a minimal stand-alone borgmatic binary in addition to the standard one (#1314). 2026-06-01 15:51:01 -07:00
Dan Helfman 4ee057a909 Experimental "browse" action, a console UI for browsing your backups (#1312).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1312
2026-06-01 20:09:24 +00:00
Dan Helfman 3e7e7879bc Remove some default system commands that aren't particular useful for this app. 2026-06-01 12:59:52 -07:00
Dan Helfman d29cfaf227 Use full URL for image. 2026-06-01 12:47:53 -07:00
Dan Helfman 23a8d665d6 Add a screenshot to the browse docs. 2026-06-01 12:44:32 -07:00
Dan Helfman 2a026a4842 Stop using Ruff "preview" rules, as CI has an older version of Ruff. 2026-06-01 12:42:02 -07:00
Dan Helfman 16c8f81a5a Attempted CI test fixes. Also spelling. 2026-06-01 12:24:57 -07:00
Dan Helfman 1670ba2aeb Add missing tests. 2026-06-01 11:57:49 -07:00
Dan Helfman 5accda1a65 Complete tests. 2026-06-01 11:54:28 -07:00
Dan Helfman ec8e52944c More tests! 2026-05-31 20:30:59 -07:00
Dan Helfman 30ca942f76 Improve performance wen loading file previews. 2026-05-31 18:55:56 -07:00
Dan Helfman 9748a0298a Add icon to configuration files list border title. 2026-05-30 23:11:41 -07:00
Dan Helfman dc91f81a39 Fix loading race conditions. 2026-05-30 22:22:06 -07:00
Dan Helfman 64ceebb2c8 Replace cross-thread calls with signals. 2026-05-30 21:12:02 -07:00
Dan Helfman 7439d7cb8f Docstrings. 2026-05-30 16:58:29 -07:00
Dan Helfman f832a361ed More tests. 2026-05-30 13:58:04 -07:00
Dan Helfman 3a4bdfb3c5 Additional tests and docstrings. 2026-05-30 11:27:59 -07:00
Dan Helfman ed955c2a73 Add additional tests. 2026-05-29 19:56:17 -07:00
Dan Helfman 174aa3d522 Additional tests. Also fix a potential race that loses loaded archive paths for display. 2026-05-29 16:59:00 -07:00
Dan Helfman 5a74b3d081 Fix broken test. 2026-05-29 11:57:13 -07:00
Dan Helfman a876ce2058 Yet more tests. 2026-05-29 11:52:02 -07:00
Dan Helfman e82c1bb195 More tests and refactoring. 2026-05-28 20:00:47 -07:00
Dan Helfman 16061f4f6d Fix tests after recent refactoring. 2026-05-28 17:15:13 -07:00
Dan Helfman 59e6e61786 Merge branch 'main' into browse-action 2026-05-28 11:53:23 -07:00
Dan Helfman b7d0f006d6 MariaDB systemd database documentation clarifications (#1303). 2026-05-28 11:50:25 -07:00
Dan Helfman 6869a5d5f2 Additional tests and docstrings. 2026-05-27 22:40:13 -07:00
Dan Helfman 9da6d0f90a Split out panels.py into multiple separate source files. 2026-05-26 23:06:36 -07:00
Dan Helfman ac65e65302 Merge get_archive_path() into get_archive_paths(). 2026-05-26 22:53:04 -07:00
Dan Helfman 8820a1eeab Faster exit for the browse action. 2026-05-26 12:01:55 -07:00
Dan Helfman 3abe025a80 Better performance when bulk populating directory lists. 2026-05-26 11:38:42 -07:00
Dan Helfman b979e0356d Fix display of ".." while loading is still underway in the browse action. 2026-05-25 17:30:06 -07:00
Dan Helfman ea0ed8345a Performance: Switch browse action to list archive files only once per archive. 2026-05-25 16:27:36 -07:00
Dan Helfman 4b8a6d1a34 More browse tests. 2026-05-24 19:22:50 -07:00
Dan Helfman 60201541e0 Add Textual to stand-alone binary to support the browse action. 2026-05-24 10:16:36 -07:00
Dan Helfman bfd79a5500 Refuse to preview named pipes. 2026-05-23 23:05:28 -07:00
Dan Helfman 43e995a4f3 Binary versus text file detection for borgmatic browse. 2026-05-23 22:50:51 -07:00
Dan Helfman 94a2d68198 Switch from a Static to a RichLog for file preview so we can get scrolling. 2026-05-23 19:20:24 -07:00
Dan Helfman c9ba0762f0 Add docstring. 2026-05-23 18:31:24 -07:00
Dan Helfman 13031a4ce4 Upgrade/add test requirements. 2026-05-22 20:03:32 -07:00
Dan Helfman 9a2e853ea7 Merge branch 'main' into browse-action 2026-05-22 20:01:50 -07:00
Dan Helfman b1f42eb233 Upgrade dependencies. 2026-05-22 20:00:40 -07:00
Dan Helfman 7e7c3c273e Update the Apprise monitoring hook's "url" option to support loading credentials with the "{credential ...}" syntax (#1308). 2026-05-22 15:25:42 -07:00
Dan Helfman 704db9c9d7 Add a "--quick-stats" flag and corresponding "quick_statistics" option (#1309). 2026-05-22 14:12:46 -07:00
Dan Helfman 61d86c55b4 Add missing Rich library to test requirements. 2026-05-22 13:44:38 -07:00
Dan Helfman ecedccadee Yet another NEWS wording tweak. 2026-05-22 13:43:44 -07:00
Dan Helfman 006825838f NEWS tweak. 2026-05-22 13:39:45 -07:00
Dan Helfman 1102bf3787 Merge branch 'main' into browse-action 2026-05-22 13:37:44 -07:00
Dan Helfman 220c76b3c0 Formatting. 2026-05-22 13:37:24 -07:00
Dan Helfman 9aea0b1d90 Fix broken tests. 2026-05-22 12:19:34 -07:00
Dan Helfman b6ab326bb4 Switch to PEP 639 license format. 2026-05-22 10:26:12 -07:00
Dan Helfman c4170bb126 Several browse action unit and integration tests. 2026-05-21 23:23:19 -07:00
Dan Helfman 9935d0ef8d Linting and more file refactoring for the browse action. 2026-05-21 17:01:07 -07:00
Dan Helfman ec11e8817d Code formatting. 2026-05-21 16:08:10 -07:00
Dan Helfman 73222e7b5b Factor out make_next_panel() out of browse action widgets into a plain function. 2026-05-21 15:50:34 -07:00
Dan Helfman fd98f4fdd1 Browse action documentation tweaks. 2026-05-21 14:10:49 -07:00
Dan Helfman 16e743d8d9 More browse action option list keys. 2026-05-21 11:41:44 -07:00
Dan Helfman e93fe4cc15 Browse action refactoring into separate files. 2026-05-21 11:33:05 -07:00
Dan Helfman da0698f6e5 Browse action logging improvements. 2026-05-21 10:08:42 -07:00
Dan Helfman 5075c1e539 Skip the browse action's configuration list if there's only one config file. 2026-05-20 23:10:29 -07:00
Dan Helfman 4b4851e78a Add browse action documentation. 2026-05-20 22:57:52 -07:00
Dan Helfman 8e91af6820 Make Textual an optional dependency. 2026-05-20 22:17:19 -07:00
Dan Helfman 9be1698762 Add syntax highlighting to previews. 2026-05-20 21:00:16 -07:00
Dan Helfman 9310b42d9f Add a few more browse action bindings. 2026-05-20 20:16:30 -07:00
Dan Helfman d1d757a5d1 Fix focus on file preview widgets so key bindings for them work in the browse action. 2026-05-20 20:01:03 -07:00
Dan Helfman 3f4a87517b Fix browse action option list highlights. 2026-05-20 19:40:47 -07:00
Dan Helfman a2dcd4747f Get rid of automatic preview panels in the browse action; they're just too slow and annoying in real-world use. 2026-05-20 17:10:00 -07:00
Dan Helfman 0396a89f72 Various browse action improvements. 2026-05-20 16:38:51 -07:00
Dan Helfman f170ef19df Switch to a generalized "carousel" for the borgmatic browse panels. 2026-05-17 16:10:58 -07:00
Dan Helfman 9729809f6f Add the first level of file loading. 2026-05-16 23:04:30 -07:00
Dan Helfman dfb3c0830e Convert browse action UI from a tree to a series of option lists. 2026-05-16 22:06:39 -07:00
Dan Helfman b22a068a60 Put browse action archive files into an actual tree hierarchy. 2026-05-14 23:26:09 -07:00
Dan Helfman 1286ccce46 Browse action MVC refactoring into multiple files. 2026-05-14 22:53:59 -07:00
Dan Helfman e339ce2fd1 Initial work on a browse action. 2026-05-14 22:24:06 -07:00
Dan Helfman aaf5812a3f Cosmetic function argument ordering fix (#1256). 2026-05-12 22:13:05 -07:00
Dan Helfman 47a6691886 Upgrade dependencies. 2026-05-12 15:29:56 -07:00
Dan Helfman 596d59ef60 Fix a race condition in which borgmatic sometimes swallows Borg error output without logging it (#1256). 2026-05-12 15:28:31 -07:00
Dan Helfman 9839b3dada Fix end-to-end database tests (#1303). 2026-05-09 11:05:28 -07:00
Dan Helfman 411685280c For MariaDB/MySQL, dump events, routines, and tablespaces. For MariaDB, dump a subset of system data when dumping "mysql"/"all" (#1303). 2026-05-08 09:38:13 -07:00
Dan Helfman cce679248f Clarify "umask" option semantics in regards to integer and octal (#1305). 2026-05-06 14:57:43 -07:00
Dan Helfman 68c9516424 Upgrade Apprise dependency. 2026-05-02 19:08:24 -07:00
Dan Helfman a508cabe3f Update the KeePassXC credential hook to support KeePassXC's secret service integration. 2026-04-29 12:52:27 -07:00
Dan Helfman 0e1659bd73 Expand the "patterns_from" and "exclude_from" options to support paths containing tildes and globs (#1301). 2026-04-27 10:13:21 -07:00
Dan Helfman ad61ad356e Update the documentation to use Pagefind's component-based search UI. 2026-04-26 09:10:07 -07:00
Dan Helfman 5c7d03910b Fix the "source_directories_must_exist" option to support source directories relative to a "working_directory" (#1300). 2026-04-25 13:49:55 -07:00
Dan Helfman 468af1de0b Fix the PostgreSQL database hook to properly parse "*options" values containing quoted spaces. 2026-04-24 19:24:44 -07:00
Dan Helfman 524d8263a7 Upgrade dependencies. 2026-04-24 17:04:29 -07:00
Dan Helfman 1d2aea0951 Add a development script for upgrading pinned dependencies. 2026-04-24 11:54:37 -07:00
Dan Helfman 961ff7c724 For the MariaDB and MySQL database hooks, escape quotes in passwords when the "password_transport" option is "pipe". 2026-04-24 10:24:45 -07:00
Dan Helfman 3e80056956 Add projects.torsion.org reply by email to NEWS. 2026-04-20 21:52:22 -07:00
Dan Helfman d8f558ce0d Add the NEWS changelog file to release tarball (#1298). 2026-04-20 15:42:17 -07:00
Dan Helfman 9da75fdc58 Use pytest tmp_path fixture instead of hardcoded /tmp paths (#1296).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1296
2026-04-20 18:10:02 +00:00
Dan Helfman ac9c8bb644 Fix build of standalone binary. 2026-04-20 10:28:57 -07:00
Dan Helfman d085fc2398 Bump version for release. 2026-04-20 09:46:54 -07:00
Dan Helfman 40d2d521a7 Fix the ZFS hook to properly unmount snapshots for empty datasets (#1295). 2026-04-18 11:43:58 -07:00
Sirio Balmelli 0b34ef0e1a tests: use pytest tmp_path fixture instead of hardcoded /tmp paths
tmp_path is a built-in pytest fixture providing a unique per-test tempdir.

Hardcoded /tmp paths break when multiple builders run tests concurrently
(e.g. Nix parallel builds): leftover or colliding files cause spurious failures.

Signed-off-by: Sirio Balmelli <sirio@b-ad.ch>
2026-04-15 16:35:41 +02:00
Dan Helfman 3f34d0848e Modify NEWS entry. 2026-04-13 15:09:04 -07:00
Dan Helfman 54289e3ee0 Fix documentation typo (#1229). 2026-04-13 10:51:32 -07:00
Dan Helfman 6eea2d5323 Document the permissions needed for the PostgreSQL database hook (#1229). 2026-04-13 10:40:10 -07:00
Dan Helfman 0ca5333fd4 Add Loki documentation link to NEWS (#1289). 2026-04-13 09:22:22 -07:00
Dan Helfman 462e1392da Add Loki mTLS to NEWS and remove unnecesary log (#1289). 2026-04-13 09:16:15 -07:00
Dan Helfman 71e2762aa7 Add mTLS support for Loki monitoring hook (#1289).
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1293
Reviewed-by: Dan Helfman <witten@torsion.org>
2026-04-13 16:07:56 +00:00
Maxime Hamon edfa708fa3 Add mTLS support for Loki monitoring hook 2026-04-12 17:57:01 +02:00
Dan Helfman aee16e32e2 Fix a regression in which SSH warnings from remote repositories broke the "spot" check (#1294). 2026-04-10 20:28:14 -07:00
Dan Helfman f3ae04225d Fix a "source directories do not exist" regression when configuration paths are relative symlinks and the bootstrap data source hook is enabled (#1292). 2026-04-06 16:03:39 -07:00
Dan Helfman 3f70cf0b29 Add Borgmatic Director UI link. 2026-04-06 09:10:36 -07:00
Dan Helfman 731e8d7c37 Bump version for release. 2026-04-05 19:57:22 -07:00
Dan Helfman 3bcf592d53 Fix the "spot" check, "extract" check, and all uses of the archive name "latest" to respect the "match_archives" and "archives_name_format" options (#1268). 2026-04-05 09:58:54 -07:00
Dan Helfman 9bb5791e9f Fix a traceback in the "spot" check with Borg 2. 2026-04-04 21:48:10 -07:00
Dan Helfman 8a60fb6398 Fix broken CI build (#1264). 2026-04-04 18:02:30 -07:00
Dan Helfman 9db2bb2b54 Improve performance of the "info" and "repo-list" actions by eliminating a second "borg info" call that supports a "no matching archives" warning (#1264). 2026-04-03 20:57:25 -07:00
Dan Helfman 87bfd6e97f Fix end-to-end tests. 2026-03-30 12:37:42 -07:00
Dan Helfman 52f9442377 More requirements updates. 2026-03-30 11:29:15 -07:00
Dan Helfman af841e0c89 Upgrade Pygments. 2026-03-30 11:17:27 -07:00
Dan Helfman ed8320c1bb Upgrade requests. 2026-03-30 11:16:21 -07:00
Dan Helfman 26b3a03721 Fix for the Loki monitoring hook not respecting the monitoring verbosity (#1257). 2026-03-30 11:13:55 -07:00
Dan Helfman 52234c47e6 Fix a regression in which running borgmatic with no arguments and no configuration files doesn't error as expected (#1286). 2026-03-22 20:10:46 -07:00
Dan Helfman b74b6aa18d Remove Nuitka's --show-modules flag, as it's mostly for debugging (#1266). 2026-03-21 13:26:43 -07:00
Dan Helfman 0380ecd8fb Add stand-alone borgmatic binary to documentation (#1266). 2026-03-21 12:53:33 -07:00
Dan Helfman 32b7d1a0f7 Add a stand-alone borgmatic binary to the release downloads to serve as another way to install borgmatic (#1266). 2026-03-21 11:10:14 -07:00
Dan Helfman da873c09f8 Update test requirements. 2026-03-21 11:05:04 -07:00
Dan Helfman d612d398e7 When Borg exits with a warning exit code, show a description of it, so you don't have to lookup the code. 2026-03-15 21:38:27 -07:00
Dan Helfman 1301bec702 Fix missing words in upgrade documentation. 2026-03-14 14:01:24 -07:00
Dan Helfman c7fc68a49a Fix typo. 2026-03-14 13:31:26 -07:00
Dan Helfman c66e29906e Attempt to fix documentation build. 2026-03-13 16:41:50 -07:00
Dan Helfman 475389a094 Move project tracker from Gitea to Forgejo. 2026-03-13 16:28:09 -07:00
Dan Helfman 9f59bf2827 Attempt to get workflow working. 2026-03-13 13:16:08 -07:00
Dan Helfman f4e9569297 Add Forgejo actions metadata. 2026-03-13 12:48:58 -07:00
Dan Helfman f43c2f7130 Add dedicated documenation installation link in Getting Started. 2026-03-08 12:29:14 -07:00
Dan Helfman 889b599d55 Switch the default borgmatic installation method from pipx to uv, as uv is faster and used for borgmatic development. 2026-03-07 21:13:53 -08:00
Dan Helfman 53791d4dc9 Fix key for documentation page. 2026-03-07 20:02:53 -08:00
Dan Helfman ee58adb4eb Split out borgmatic installation documentation to its own page, so it's easier to find. 2026-03-07 19:27:04 -08:00
143 changed files with 5280 additions and 648 deletions
@@ -1,5 +1,5 @@
name: build
run-name: ${{ gitea.actor }} is building
run-name: ${{ forgejo.actor }} is building
on:
push:
branches: [main]
@@ -17,7 +17,7 @@ jobs:
docs:
needs: [test]
runs-on: host
if: gitea.event_name == 'push'
if: forgejo.event_name == 'push'
env:
IMAGE_NAME: projects.torsion.org/borgmatic-collective/borgmatic:docs
+1 -1
View File
@@ -9,7 +9,7 @@ Python project using setuptools.
Please do not use AI agents to modify this codebase. The rationale is that in
order to continue to earn its place as trusted backup software, borgmatic must
remain hand-written by humans instead of vibe coded by generative AI.
remain handwritten by humans instead of vibe coded by generative AI.
Additionally, if LLMs were to perform a sizeable chunk of the feature
development on this codebase, then human borgmatic developers would lose their
+1
View File
@@ -1,4 +1,5 @@
# This file only applies to the source dist tarball, not the built wheel.
include NEWS
include borgmatic/config/schema.yaml
graft docs
graft sample
+67
View File
@@ -1,3 +1,70 @@
2.1.6
* #1256: Fix a race condition in which borgmatic sometimes swallows Borg error output without
logging it.
* #1300: Fix the "source_directories_must_exist" option to support source directories relative to a
"working_directory".
* #1301: Expand the "patterns_from" and "exclude_from" options to support paths containing tildes
and globs.
* #1303: For the MariaDB and MySQL database hooks, include events, routines, and tablespaces when
dumping a database.
* #1303: For the MariaDB hook, include only a subset of system data when dumping the "mysql" system
database (or "all" databases), so the dump is actually restorable. See the documentation for more
information: https://torsion.org/borgmatic/reference/configuration/data-sources/mariadb/
* #1314: Add a minimal stand-alone borgmatic binary in addition to the standard one. The minimal
binary omits support for the Apprise monitoring hook.
* Add an experimental "browse" action providing a console UI for browsing your backups. See the
documentation for more information:
https://torsion.org/borgmatic/how-to/inspect-your-backups/#browsing-backups
* #1308: Update the Apprise monitoring hook's "url" option to support loading credentials with the
"{credential ...}" syntax. See the documentation for more information:
https://torsion.org/borgmatic/reference/configuration/credentials/
* #1309: Add a "--quick-stats" flag and corresponding "quick_statistics" option for showing only
abbreviated statistics for the "create" action. Borg 1.4.5+ only.
* Update the KeePassXC credential hook to support KeePassXC's secret service integration. See the
documentation for more information:
https://torsion.org/borgmatic/reference/configuration/credentials/keepassxc/
* Add the NEWS changelog file to release tarball (#1298).
* Enable reply by email on projects.torsion.org, so replies to notification emails get posted as
comments on tickets.
* For the MariaDB and MySQL database hooks, escape quotes in passwords when the
"password_transport" option is "pipe".
* Add a development script for upgrading pinned dependencies.
* Fix the PostgreSQL database hook to properly parse "*options" values containing quoted spaces.
* Update the documentation to use Pagefind's component-based search UI.
2.1.5
* #1229: Document the permissions needed for the PostgreSQL database hook:
https://torsion.org/borgmatic/reference/configuration/data-sources/postgresql/
* #1289: Add mutual TLS support for the Loki monitoring hook. See the documentation for more
information: https://torsion.org/borgmatic/reference/configuration/monitoring/loki/
* #1292: Fix a "source directories do not exist" regression when configuration paths are relative
symlinks and the bootstrap data source hook is enabled.
* #1294: Fix a regression in which SSH warnings from remote repositories broke the "spot" check
and other actions as well.
* #1295: Fix the ZFS hook to properly unmount snapshots for empty datasets.
2.1.4
* #1266: Add a stand-alone borgmatic Linux binary to the release downloads to serve as another way
to install borgmatic. Consider this binary a beta feature.
* #1286: Fix a regression in which running borgmatic with no arguments and no configuration files
doesn't error as expected.
* #1257: Fix for the Loki monitoring hook not respecting the monitoring verbosity.
* #1264: Improve performance of the "info" and "repo-list" actions by eliminating a second "borg
info" call that supports a "no matching archives" warning. The warning still occurs; it's just
done now without the extra "borg info" call.
* #1268: Fix the "spot" check, "extract" check, and all uses of the archive name "latest" to
respect the "match_archives" and "archives_name_format" options. This means that borgmatic now
uses the "latest" archive that also matches those options instead of the latest archive overall.
* When Borg exits with a warning exit code, show a description of it, so you don't have to look up
the code.
* Split out borgmatic installation documentation to its own page, so it's easier to find.
* Switch the default borgmatic installation method from pipx to uv, as uv is faster and used for
borgmatic development. If you'd like to switch, see the documentation for more information:
https://torsion.org/borgmatic/how-to/upgrade/
* Move the project tracker from Gitea to Forgejo.
* Fix a regression in which borgmatic didn't show an error message when run with no configuration.
* Fix a traceback in the "spot" check with Borg 2.
2.1.3
* #1175: Add a "files_changed" option for customizing Borg's file modification detection.
* #1175: Add a "msgpack_version_check" option to prevent Borg from validating msgpack's version.
+5 -4
View File
@@ -94,10 +94,11 @@ borgmatic is powered by [Borg Backup](https://www.borgbackup.org/).
## Getting started
Your first step is to [install and configure
borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/).
Your first steps are to
[install](https://torsion.org/borgmatic/how-to/install-borgmatic/) and
[configure borgmatic](https://torsion.org/borgmatic/how-to/set-up-backups/).
For additional documentation, check out the links above (left panel on wide screens)
For additional documentation, check out the links on the top/left
for <a href="https://torsion.org/borgmatic/#documentation">borgmatic how-to and
reference guides</a>.
@@ -183,4 +184,4 @@ Thanks to all borgmatic contributors! There are multiple ways to contribute to
this project, so the following includes those who have fixed bugs, contributed
features, *or* filed tickets.
{% include borgmatic/contributors.html %}
{% include borgmatic/contributors.html %}
+19
View File
@@ -0,0 +1,19 @@
.
apprise
attrs
certifi
charset-normalizer
click
idna
jsonschema
jsonschema-specifications
markdown
oauthlib
packaging
pyyaml
referencing
requests
requests-oauthlib
rpds-py
ruamel-yaml
urllib3
+21
View File
@@ -0,0 +1,21 @@
# This file was autogenerated by uv via the following command:
# uv pip compile --annotation-style line binary_requirements.in -o binary_requirements.txt
apprise==1.10.0 # via -r binary_requirements.in
attrs==26.1.0 # via jsonschema, referencing, -r binary_requirements.in
. # via -r binary_requirements.in
certifi==2026.5.20 # via apprise, requests, -r binary_requirements.in
charset-normalizer==3.4.7 # via requests, -r binary_requirements.in
click==8.4.1 # via apprise, -r binary_requirements.in
idna==3.16 # via requests, -r binary_requirements.in
jsonschema==4.26.0 # via borgmatic, -r binary_requirements.in
jsonschema-specifications==2025.9.1 # via jsonschema, -r binary_requirements.in
markdown==3.10.2 # via apprise, -r binary_requirements.in
oauthlib==3.3.1 # via requests-oauthlib, -r binary_requirements.in
packaging==26.2 # via borgmatic, -r binary_requirements.in
pyyaml==6.0.3 # via apprise, -r binary_requirements.in
referencing==0.37.0 # via jsonschema, jsonschema-specifications, -r binary_requirements.in
requests==2.34.2 # via apprise, borgmatic, requests-oauthlib, -r binary_requirements.in
requests-oauthlib==2.0.0 # via apprise, -r binary_requirements.in
rpds-py==0.30.0 # via jsonschema, referencing, -r binary_requirements.in
ruamel-yaml==0.19.1 # via borgmatic, -r binary_requirements.in
urllib3==2.7.0 # via requests, -r binary_requirements.in
+108
View File
@@ -0,0 +1,108 @@
import signal
import textual.app
import textual.binding
import textual.widgets
import borgmatic.actions.browse.carousel
import borgmatic.actions.browse.configuration_files_list
import borgmatic.actions.browse.logs
import borgmatic.actions.browse.repositories_list
class Browse_app(textual.app.App):
'''
The main app / entry point for the browse action UI.
'''
BINDINGS = (
textual.binding.Binding(key='q', action='quit', description='quit'),
textual.binding.Binding(key='v', action='toggle_logs', description='view logs'),
textual.binding.Binding(
key='c', action='command_palette', description='commands', show=False
),
)
COMMAND_PALETTE_BINDING = 'c'
CSS = '''
.panel {
border: round $primary;
border-title-color: $text-primary;
width: 100%;
height: 100%;
}
#logs {
width: 100%;
height: 50%;
display: none;
}
'''
def __init__(self, configs):
self.configs = configs
super().__init__()
def get_system_commands(self, screen): # pragma: no cover
'''
Remove the screenshot system command because it produces broken screenshots. (Emoji
weirdness, etc.) Also remove minimize and maximize because they don't accomplish much with
out particular layout.
'''
yield from (
command
for command in super().get_system_commands(screen)
if command.title not in {'Screenshot', 'Minimize', 'Maximize'}
)
def compose(self):
'''
Compose a UI consisting of:
* a header with the application name
* a carousel container that contains the main UI panels
* a logs panel where Python logs show up (panel hidden by default)
* a footer with available keys listed
'''
yield textual.widgets.Header()
yield borgmatic.actions.browse.carousel.Carousel(
[
borgmatic.actions.browse.configuration_files_list.Configuration_files_list(
self.configs
)
]
if len(self.configs) > 1
else [
borgmatic.actions.browse.repositories_list.Repositories_list(
next(iter(self.configs.values()))
)
]
)
logs_panel = borgmatic.actions.browse.logs.Logs()
yield logs_panel
yield textual.widgets.Footer()
borgmatic.actions.browse.logs.log_to_widget(logs_panel)
def on_mount(self):
'''
Set the application title, which ends up in the header.
'''
self.title = 'borgmatic browse'
def action_toggle_logs(self):
'''
Toggle the show/hide status of the logs panel.
'''
logs_panel = self.query_one('#logs')
logs_panel.styles.display = 'none' if logs_panel.styles.display == 'block' else 'block'
def exit(self): # pragma: no cover
'''
Exit the application. But first raise a SIGTERM (handled in borgmatic/signals.py) to
encourage a fast exit by killing any ongoing Borg subprocesses.
'''
signal.raise_signal(signal.SIGTERM)
super().exit()
+143
View File
@@ -0,0 +1,143 @@
import argparse
import collections
import enum
import json
import logging
import binaryornot.helpers
import borgmatic.borg.extract
import borgmatic.borg.list
import borgmatic.borg.repo_list
import borgmatic.borg.version
logger = logging.getLogger(__name__)
class Path_type(enum.Enum):
DIRECTORY = 'd'
LINK = 'l'
PIPE = 'p'
FILE = '-'
# A data structure capturing a path stored in a Borg archive.
Archive_path = collections.namedtuple(
'Archive_path',
('path_type', 'file_path', 'link_target'),
)
def get_repository_archives(config, repository):
'''
Given a configuration dict and a repository dict, return a list of the repository's archives,
one dict per archive.
'''
with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
logger.info('Listing repository')
repo_list_arguments = argparse.Namespace(
repository=repository['path'],
short=None,
format=None,
json=True,
prefix=None,
match_archives=None,
sort_by=None,
first=None,
last=None,
)
global_arguments = argparse.Namespace()
local_path = config.get('local_path', 'borg')
remote_path = config.get('remote_path')
local_borg_version = borgmatic.borg.version.local_borg_version(config, local_path)
return json.loads(
borgmatic.borg.repo_list.list_repository(
repository['path'],
config,
local_borg_version,
repo_list_arguments,
global_arguments,
local_path,
remote_path,
)
)
def get_archive_paths(config, repository, archive_name):
'''
Given a configuration dict, a repository dict, and an archive name in that repository, get a
generator of files, directories, symlinks, etc. found in the archive, each as an Archive_path
instance.
'''
with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
logger.info(f'Listing archive {archive_name}')
global_arguments = argparse.Namespace()
local_path = config.get('local_path', 'borg')
remote_path = config.get('remote_path')
local_borg_version = borgmatic.borg.version.local_borg_version(config, local_path)
return (
Archive_path(
path_data['type'],
path_data['path'],
path_data.get('linktarget'),
)
for path_data in borgmatic.borg.list.capture_archive_listing(
repository['path'],
archive_name,
config,
local_borg_version,
global_arguments,
local_path=local_path,
remote_path=remote_path,
)
)
READLINES_HINT_BYTES = 100000
TRUNCATION_MESSAGE = '[... truncated for display ...]'
def get_archive_file_content(config, repository, archive_name, file_path):
'''
Given a configuration dict, a repository dict, an archive name in that repository, and a file
path in that archive, return the file's contents or None if the file can't be loaded, e.g.
because it's binary or can't be decoded.
If the file is too large, then truncate the returned content.
'''
with borgmatic.logger.Log_prefix(repository.get('label', repository['path'])):
logger.info(f'Getting archive content of file {file_path}')
local_path = config.get('local_path', 'borg')
remote_path = config.get('remote_path')
lines = borgmatic.borg.extract.extract_archive(
dry_run=False,
repository=repository['path'],
archive=archive_name,
paths=(file_path,),
config=config,
local_borg_version=borgmatic.borg.version.local_borg_version(config, local_path),
global_arguments=argparse.Namespace(),
local_path=local_path,
remote_path=remote_path,
destination_path=None,
strip_components=None,
extract_to_stdout=True,
).stdout.readlines(READLINES_HINT_BYTES)
content = b''.join(lines)
if binaryornot.helpers.is_binary_string(content):
return None
try:
return (
content.decode()
if len(content) < READLINES_HINT_BYTES
else f'{content.decode()}\n{TRUNCATION_MESSAGE}'
)
except UnicodeDecodeError:
return None
+89
View File
@@ -0,0 +1,89 @@
import textual.widgets
import borgmatic.actions.browse.bindings
import borgmatic.actions.browse.loading
import borgmatic.actions.browse.workers
class Archives_list(textual.widgets.OptionList):
'''
A widget for selecting a single Borg archive from among the archives in a repository. The item
selection event is handled in a Carousel instance, the parent widget of an Archives_list.
'''
BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
def __init__(self, config, repository):
'''
Given a configuration dict and a repository dict, prepare to load the archives from the
repository for eventual display in this widget. Actual loading kicks off in on_mount()
below.
'''
self.config = config
self.repository = repository
super().__init__(classes='panel')
self.border_title = '📚 archives'
self.highlighted_option_changed = False
self.archive_loaded = borgmatic.actions.browse.workers.Archive_loaded(
self, 'archive loaded'
)
self.loading_timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
def on_mount(self):
'''
When this widget gets mounted in the DOM, subscribe to archive loaded events so that we can
find out about archives as they load. Also start loading archives from the repository.
Loading is started *after* subscribing to the archive loaded signal so that there's not a
gap where we might miss out on signal publishes.
'''
self.archive_loaded.subscribe(self, self.on_archive_loaded)
borgmatic.actions.browse.workers.add_repository_archives(
self.app,
archive_loaded=self.archive_loaded,
config=self.config,
repository=self.repository,
)
def on_archive_loaded(self, archive_name):
'''
When an archive loads, add it as an option to this archives list. But if we get a
signal that all path loading is complete, stop and remove our loading indicator.
'''
if archive_name is borgmatic.actions.browse.workers.LOADING_DONE:
self.loading_timer.stop()
self.remove_option('loading-indicator')
return
label_pieces = (
(archive_name, '[dim](latest)[/dim]') if len(self.options) == 1 else (archive_name,)
)
highlighted_option = self.highlighted_option
loading_indicator = self.get_option('loading-indicator')
self.remove_option('loading-indicator')
self.add_options(
(
textual.widgets.option_list.Option(' '.join(label_pieces), id=archive_name),
loading_indicator,
),
)
# Retain the highlighted option position even as other options load around it.
self.highlighted = (
self.get_option_index(highlighted_option.id)
if highlighted_option and self.highlighted_option_changed
else 0
)
def on_option_list_option_highlighted(self, event):
'''
When the highlighted option changes, record that fact. This flag is consumed in
borgmatic.actions.browse.workers.add_repository_archives() in order to retain the
highlighted option even as other options load around it.
'''
if self.highlighted not in {None, 0}:
self.highlighted_option_changed = True
+22
View File
@@ -0,0 +1,22 @@
import textual.binding
import textual.widgets
OPTION_LIST_BINDINGS = (
*textual.widgets.OptionList.BINDINGS,
textual.binding.Binding(
key='up,k', action='cursor_up', description='scroll up', show=True, priority=True
),
textual.binding.Binding(
key='down,j', action='cursor_down', description='scroll down', show=True, priority=True
),
textual.binding.Binding(
key='pageup', action='page_up', description='page up', show=True, priority=True
),
textual.binding.Binding(
key='pagedown', action='page_down', description='page down', show=True, priority=True
),
textual.binding.Binding(
key='enter', action='select', description='select', show=True, priority=True
),
textual.binding.Binding(key='right,l', action='select', description='select', show=False),
)
+153
View File
@@ -0,0 +1,153 @@
import os
import textual.binding
import textual.containers
import borgmatic.actions.browse.archive
import borgmatic.actions.browse.archives_list
import borgmatic.actions.browse.configuration_files_list
import borgmatic.actions.browse.directory_list
import borgmatic.actions.browse.file_preview
import borgmatic.actions.browse.icons
import borgmatic.actions.browse.repositories_list
def make_next_panel(focused_panel, option_id):
'''
Given a focused panel widget and the selected option ID, return the next panel corresponding to
that selection. This is the mechanism by which the user can successively drill down from
configuration file to repository to archive to root directory to non-root directory or file.
If the particular option ID on the focused panel doesn't have a supported next panel, then
return None.
'''
if isinstance(
focused_panel, borgmatic.actions.browse.configuration_files_list.Configuration_files_list
):
return borgmatic.actions.browse.repositories_list.Repositories_list(
config=focused_panel.configs[option_id]
)
if isinstance(focused_panel, borgmatic.actions.browse.repositories_list.Repositories_list):
return borgmatic.actions.browse.archives_list.Archives_list(
config=focused_panel.config, repository=focused_panel.repositories[option_id]
)
if isinstance(focused_panel, borgmatic.actions.browse.archives_list.Archives_list):
return borgmatic.actions.browse.directory_list.Directory_list(
config=focused_panel.config, repository=focused_panel.repository, archive_name=option_id
)
if isinstance(focused_panel, borgmatic.actions.browse.directory_list.Directory_list):
option = focused_panel.get_option(option_id)
if option.prompt.startswith(
borgmatic.actions.browse.icons.PATH_TYPE_ICONS[
borgmatic.actions.browse.archive.Path_type.DIRECTORY.value
]
):
return borgmatic.actions.browse.directory_list.Directory_list(
focused_panel.config,
focused_panel.repository,
focused_panel.archive_name,
path_loaded=focused_panel.path_loaded,
path_components=(*focused_panel.path_components, option_id),
)
if option.prompt.startswith(
borgmatic.actions.browse.icons.PATH_TYPE_ICONS[
borgmatic.actions.browse.archive.Path_type.FILE.value
]
):
return borgmatic.actions.browse.file_preview.File_preview(
focused_panel.config,
focused_panel.repository,
focused_panel.archive_name,
file_path=os.path.sep.join((*focused_panel.path_components, option_id)),
)
return None
class Carousel(textual.containers.Horizontal):
BINDINGS = (
textual.binding.Binding(
key='left,h', action='previous', description='previous', priority=True
),
)
def __init__(self, panels):
self.panels = panels
self.focused_panel = panels[0]
super().__init__()
def compose(self):
'''
Compose with each of the contained panels and focus the first one.
'''
yield from self.panels
self.focused_panel.focus()
def action_previous(self):
'''
Make the previous panel into the focused panel.
'''
previous_panel_index = self.panels.index(self.focused_panel) - 1
if previous_panel_index < 0:
return
self.focused_panel.styles.display = 'none'
self.focused_panel = self.panels[previous_panel_index]
self.focused_panel.styles.display = 'block'
self.focused_panel.focus()
def action_next(self, option_id):
'''
Hide the current focused panel and create the next one.
'''
next_panel_index = self.panels.index(self.focused_panel) + 1
if next_panel_index < len(self.panels):
next_panel = self.panels[next_panel_index]
next_panel.styles.display = 'block'
else:
next_panel = make_next_panel(self.focused_panel, option_id)
if next_panel is None:
self.notify('Cannot display this content', severity='warning')
return
self.panels.append(next_panel)
next_panel.highlighted = 0
self.focused_panel.styles.display = 'none'
self.focused_panel = next_panel
self.focused_panel.focus()
self.mount(self.focused_panel)
def on_option_list_option_highlighted(self, event):
'''
The highlighted option has changed, so truncate any next panels.
'''
next_panel_index = self.panels.index(self.focused_panel) + 1
del self.panels[next_panel_index:]
def on_option_list_option_selected(self, event):
'''
An option has been selected, so advance to the next panel—unless the option selected is
"..", in which case go to the previous panel.
'''
if (
event.option_list != self.focused_panel or event.option_id == 'loading-indicator'
): # pragma: no cover
return
if event.option_id == '..':
self.action_previous()
else:
self.action_next(event.option_id)
@@ -0,0 +1,33 @@
import os
import textual.widgets
import borgmatic.actions.browse.bindings
class Configuration_files_list(textual.widgets.OptionList):
'''
A widget for selecting a single borgmatic configuration file from among available configuration
files. The item selection event is handled in a Carousel instance, the parent widget of an
Configuration_files_list.
'''
BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
def __init__(self, configs):
'''
Given a dict mapping from configuration path to corresponding configuration dict, add each
configuration path as an option to this widget.
'''
self.configs = configs
home_directory = os.path.expanduser('~')
super().__init__(
*(
textual.widgets.option_list.Option(unexpanded_path, id=config_path)
for config_path in configs
for unexpanded_path in (config_path.replace(home_directory, '~'),)
),
classes='panel',
)
self.border_title = '📄 configuration files'
+211
View File
@@ -0,0 +1,211 @@
import os
import textual.widgets
import borgmatic.actions.browse.bindings
import borgmatic.actions.browse.icons
import borgmatic.actions.browse.loading
import borgmatic.actions.browse.workers
def get_relative_archive_path_components(archive_path, current_directory_path_components):
'''
Given an Archive_path instance and a tuple of path components for the currently browsed
directory, get the path components as a tuple for the archive path relative to that directory.
For instance, given an archive path with a path of 'foo/bar/baz/quux.txt' and current
directory path components of ('foo', 'bar'), return ('baz', 'quux.txt').
If the archive path is not actually relative to the current directory, return None.
'''
archive_path_components = tuple(archive_path.file_path.split(os.path.sep))
if not current_directory_path_components:
return archive_path_components
# If the loaded path doesn't match this directory list's own path, then we don't care about
# it for purposes of displaying this particular directory.
if (
tuple(archive_path_components[: len(current_directory_path_components)])
!= current_directory_path_components
):
return None
# Strip off the portion of the archive path that matches the directory list's own path.
return archive_path_components[len(current_directory_path_components) :]
def make_directory_list_option(archive_path, relative_path_components):
'''
Given an Archive_path instance and a tuple of relative path components for it, make a
textual.widgets.option_list.Option for the path. Use an the icon based on whether this looks
like a terminal filename or a directory.
'''
pieces = (
borgmatic.actions.browse.icons.PATH_TYPE_ICONS.get(
archive_path.path_type if len(relative_path_components) == 1 else 'd', ''
),
relative_path_components[0],
) + (('', archive_path.link_target) if archive_path.link_target else ())
return textual.widgets.option_list.Option(
prompt=' '.join(pieces), id=relative_path_components[0]
)
def add_archive_paths(
directory_list,
config,
repository,
archive_name,
archive_paths,
):
'''
Given a DirectoryList instance, a configuration dict, a repository dict, an archive name, and a
sequence of ArchivePath instances, add the paths to the directory list as options, sorting and
deduplicating the resulting directory list's options.
After all of this reshuffling, make sure the original highlighted option remains highlighted.
'''
highlighted_option = directory_list.highlighted_option
original_options_count = len(directory_list.options)
sorted_options = sorted(
(
*directory_list.options,
*(
make_directory_list_option(archive_path, relative_path_components)
for archive_path in archive_paths
for relative_path_components in (
get_relative_archive_path_components(
archive_path,
directory_list.path_components,
),
)
if relative_path_components
if relative_path_components[0] not in directory_list._id_to_option
),
),
# The loading indicator "option" always goes to the bottom.
key=lambda option: ((option.id == 'loading-indicator'), option.prompt),
)
# If there aren't actually any options to add (due to deduplication), bail.
if len(sorted_options) == original_options_count:
return
# Retain the highlighted option position even as other options load around it.
directory_list.set_options(sorted_options)
directory_list.highlighted = (
directory_list.get_option_index(highlighted_option.id)
if highlighted_option and directory_list.highlighted_option_changed
else 0
)
class Directory_list(textual.widgets.OptionList):
'''
A widget for selecting a path from among the contents of a particular directory in a Borg
archive. The item selection event is handled in a Carousel instance, the parent widget of a
Directory_list.
'''
BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
def __init__(self, config, repository, archive_name, path_loaded=None, path_components=None):
'''
Given a configuration dict, a repository dict, an archive name, an optional
Archive_path_loaded instance for signalling new paths as they load, and an optional tuple of
path components indicating this directory's position in the backed up filesystem, prepare to
load paths from the archive for eventual display in this widget. Actual loading kicks off in
on_mount() below.
'''
self.config = config
self.repository = repository
self.archive_name = archive_name
self.path_components = path_components or ()
self.highlighted_option_changed = False
super().__init__(classes='panel')
self.border_title = ' '.join(
(
'📁',
os.path.sep.join(self.path_components)
if self.path_components
else f'{archive_name}',
)
)
if self.path_components:
self.add_option(
textual.widgets.option_list.Option(
'📁 ..',
id='..',
),
)
self.path_loaded = path_loaded or borgmatic.actions.browse.workers.Archive_path_loaded(
self, 'archive path loaded'
)
if not self.path_loaded.complete:
self.timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
def on_mount(self):
'''
When this widget gets mounted in the DOM, subscribe to path loaded events so that we can
find out about relevant archive paths as they load. And if this is a root directory list,
start loading paths from the archive. If this is a non-root directory list, add any already
loaded archive paths to this widget as options.
Loading is started *after* subscribing to path loaded signals so that there's not a gap
where we might miss out on any paths.
'''
self.path_loaded.subscribe(self, self.on_archive_path_loaded)
if self.path_components:
add_archive_paths(
directory_list=self,
config=self.config,
repository=self.repository,
archive_name=self.archive_name,
archive_paths=borgmatic.actions.browse.workers.get_paths(
self.path_loaded.path_hierarchy, self.path_components
),
)
else:
borgmatic.actions.browse.workers.load_archive_paths(
self.app,
path_loaded=self.path_loaded,
config=self.config,
repository=self.repository,
archive_name=self.archive_name,
)
def on_archive_path_loaded(self, data):
'''
When an archive path loads, add it as an option to this directory list. But if we get a
signal that all path loading is complete, stop and remove our loading indicator.
'''
if data is borgmatic.actions.browse.workers.LOADING_DONE:
self.timer.stop()
self.remove_option('loading-indicator')
return
add_archive_paths(
directory_list=self,
config=self.config,
repository=self.repository,
archive_name=self.archive_name,
archive_paths=(data,),
)
def on_option_list_option_highlighted(self, event):
'''
When the highlighted option changes, record that fact. This flag is consumed in
add_archive_paths() in order to retain the highlighted option even as other options load
around it.
'''
if self.highlighted not in {None, 0}:
self.highlighted_option_changed = True
+87
View File
@@ -0,0 +1,87 @@
import logging
import rich.syntax
import textual.binding
import textual.widgets
import borgmatic.actions.browse.loading
import borgmatic.actions.browse.workers
logger = logging.getLogger('__name__')
class File_preview(textual.widgets.RichLog):
'''
A widget for extracting and previewing the contents of a file stored in a Borg archive.
'''
BINDINGS = (
*textual.widgets.RichLog.BINDINGS,
textual.binding.Binding(
key='up,k', action='scroll_up', description='scroll up', show=True, priority=True
),
textual.binding.Binding(
key='down,j', action='scroll_down', description='scroll down', show=True, priority=True
),
textual.binding.Binding(
key='pageup', action='page_up', description='page up', show=True, priority=True
),
textual.binding.Binding(
key='pagedown', action='page_down', description='page down', show=True, priority=True
),
)
def __init__(self, config, repository, archive_name, file_path):
'''
Given a configuration dict, a repository dict, an archive name, and the path of a file in
the archive, prepare to load the file's contents for eventual display in this widget. Actual
loading kicks off in on_mount() below.
'''
self.config = config
self.repository = repository
self.archive_name = archive_name
self.file_path = file_path
super().__init__(classes='panel')
self.border_title = f'📄 {self.file_path} preview'
self.auto_scroll = False
self.file_preview_loaded = borgmatic.actions.browse.workers.File_preview_loaded(
self, 'file preview loaded'
)
self.loading_timer = borgmatic.actions.browse.loading.add_inline_loading_indicator(self)
def on_mount(self):
'''
When this widget gets mounted in the DOM, subscribe to archive loaded events so that we can
find out about archives as they load. Also start loading file contents from the archive.
Loading is started *after* subscribing to the file preview loaded signal so that there's not
a gap where we might miss out on signal publishes.
'''
self.file_preview_loaded.subscribe(self, self.on_file_preview_loaded)
borgmatic.actions.browse.workers.load_file_preview(
self.app,
file_preview_loaded=self.file_preview_loaded,
config=self.config,
repository=self.repository,
archive_name=self.archive_name,
file_path=self.file_path,
)
def on_file_preview_loaded(self, file_contents):
'''
When a file loads, write its contents (syntax highlighted) to this file preview widget.
'''
self.loading_timer.stop()
self.clear()
if file_contents is None:
self.write('Cannot display a preview for this file')
else:
# Only pass the file path and not its contents to guess_lexer(). Passing the contents is
# more accurate, but also much slower.
self.write(
rich.syntax.Syntax(file_contents, rich.syntax.Syntax.guess_lexer(self.file_path))
)
+8
View File
@@ -0,0 +1,8 @@
import borgmatic.actions.browse.archive
PATH_TYPE_ICONS = {
borgmatic.actions.browse.archive.Path_type.DIRECTORY.value: '📁',
borgmatic.actions.browse.archive.Path_type.LINK.value: '🔗',
borgmatic.actions.browse.archive.Path_type.PIPE.value: '🚰',
borgmatic.actions.browse.archive.Path_type.FILE.value: '📄',
}
+54
View File
@@ -0,0 +1,54 @@
import contextlib
import functools
import logging
import textual.widgets
import textual.widgets.option_list
LOADING_DOT_INTERVAL_SECONDS = 0.3
logger = logging.getLogger('__name__')
def update_inline_loading_indicator(widget):
'''
Given a textual.widgets.OptionList or a textual.widgets.RichLog instance, animate the existing
loading indicator inside it.
'''
if isinstance(widget, textual.widgets.OptionList):
with contextlib.suppress(textual.widgets.option_list.OptionDoesNotExist):
widget.replace_option_prompt(
'loading-indicator',
(str(widget.get_option('loading-indicator').prompt) + '.').replace('....', ''),
)
elif isinstance(widget, textual.widgets.RichLog):
with contextlib.suppress(IndexError):
loading_message = str(widget.lines[0].text)
widget.clear()
widget.write((loading_message + '.').replace('....', ''))
else:
raise ValueError(f'Unsupported widget type: {type(widget)}')
LOADING_MESSAGE = '⏳ loading...'
def add_inline_loading_indicator(widget):
'''
Given a textual.widgets.OptionList or a textual.widgets.RichLog instance, add a loading
indicator to it.
'''
if isinstance(widget, textual.widgets.OptionList):
loading_option = textual.widgets.option_list.Option(LOADING_MESSAGE, id='loading-indicator')
widget.add_option(loading_option)
widget.highlighted = None
elif isinstance(widget, textual.widgets.RichLog):
widget.write(LOADING_MESSAGE)
else:
raise ValueError(f'Unsupported widget type: {type(widget)}')
return widget.set_interval(
LOADING_DOT_INTERVAL_SECONDS,
functools.partial(update_inline_loading_indicator, widget),
)
+102
View File
@@ -0,0 +1,102 @@
import contextlib
import logging
import textual._context
import textual.widgets
import textual.worker
import borgmatic.logger
class Rich_color_formatter(logging.Formatter):
'''
A Python logging formatter that formats log records with Rich-compatible color markup according
to their levels.
'''
def __init__(self, *args, **kwargs):
self.prefix = None
super().__init__(
'{prefix}{message}',
*args,
style='{',
**kwargs,
)
def format(self, record):
'''
Given a log record, format it with Rich-compatibe color markup corresponding to its log
level.
'''
borgmatic.logger.add_custom_log_levels()
color = {
logging.CRITICAL: 'bright_red',
logging.ERROR: 'bright_red',
logging.WARNING: 'bright_yellow',
logging.ANSWER: 'bright_magenta',
logging.INFO: 'bright_green',
logging.DEBUG: 'bright_cyan',
}.get(record.levelno)
record.prefix = f'{self.prefix}: ' if self.prefix else ''
return f'[{color}]{super().format(record)}[/{color}]'
class Browse_log_handler(logging.Handler):
'''
A Python log handler that writes any log records to a logging widget.
'''
def __init__(self, logs_widget):
'''
Given a logs widget, save it for use below.
'''
self.logs_widget = logs_widget
super().__init__()
def emit(self, record):
'''
Given a log record, format it and log it to the logs widgets. This works whether or not the
logging is happening in the main thread.
'''
message = self.format(record)
try:
textual.worker.get_current_worker()
self.logs_widget.app.call_from_thread(self.logs_widget.write, message)
except (RuntimeError, textual.worker.NoActiveWorker):
with contextlib.suppress(textual._context.NoActiveAppError):
self.logs_widget.write(message)
def log_to_widget(logs_widget):
'''
Given a Textual RichLog logs widget, add a log handler and formatter that logs to it. Also
remove the default borgmatic console log handler so it doesn't try to log all over our UI.
'''
handler = Browse_log_handler(logs_widget)
handler.setFormatter(Rich_color_formatter())
logger = logging.getLogger()
logger.setLevel(min(handler.level for handler in logger.handlers))
logger.addHandler(handler)
with contextlib.suppress(StopIteration):
console_handler = next(
handler
for handler in logging.getLogger().handlers
if isinstance(handler, borgmatic.logger.Multi_stream_handler)
)
logger.removeHandler(console_handler)
class Logs(textual.widgets.RichLog):
'''
A widget for viewing borgmatic logs in realtime. The log level is determined by borgmatic's
current verbosity level.
'''
def __init__(self):
super().__init__(markup=True, id='logs', classes='panel')
self.border_title = '🪵 logs'
@@ -0,0 +1,30 @@
import textual.widgets
import borgmatic.actions.browse.bindings
class Repositories_list(textual.widgets.OptionList):
'''
A widget for selecting a single Borg repository from among the repositories in a borgmatic
configuration file. The item selection event is handled in a Carousel instance, the parent
widget of a Repositories_list.
'''
BINDINGS = borgmatic.actions.browse.bindings.OPTION_LIST_BINDINGS
def __init__(self, config):
'''
Given a configuration dict, populate the repositories in this widget.
'''
self.config = config
self.repositories = config['repositories']
super().__init__(
*(
textual.widgets.option_list.Option(label, id=index)
for index, repository in enumerate(self.repositories)
for label in (repository.get('label', repository.get('path')),)
),
classes='panel',
)
self.border_title = '📦 repositories'
+29
View File
@@ -0,0 +1,29 @@
import logging
def run_browse(
diff_arguments,
global_arguments,
configs,
):
'''
Run the "browse" action for the given borgmatic configurations. This launches a console UI.
Raise ValueError if the Textual library (a prerequisite for this action) can't be imported.
'''
if not configs:
return
logging.getLogger('asyncio').setLevel(logging.WARNING)
try:
import textual # noqa: F401, PLC0415
except ImportError: # pragma: no cover
raise ValueError(
'Unable to import the Textual library for the browse action; try installing "borgmatic[browse]"'
)
import borgmatic.actions.browse.app # noqa: PLC0415
app = borgmatic.actions.browse.app.Browse_app(configs)
app.run()
+192
View File
@@ -0,0 +1,192 @@
import logging
import os
import textual
import textual.signal
import borgmatic.actions.browse.archive
logger = logging.getLogger('__name__')
LOADING_DONE = object()
class Archive_loaded(textual.signal.Signal):
'''
A signal that publishes when each subsequent archive is loaded from a repository, intended for
consumption in widgets that display archives as they are loaded. This signal also publishes
when loading is complete.
Each subscribed callback call includes the archive as an archive name string. Given the lack of
other identifying information (configuration file, repository), there should be a separate
Archive_loaded instance per repository.
'''
@textual.work(thread=True)
def add_repository_archives(browse_app, archive_loaded, config, repository):
'''
Given a running Browse_app instance, an Archive_loaded instance, a configuration dict, and a
repository dict, load a list of the archives from the repository and add them as options in the
archives list. Reverse the order so the most recent archive is first.
This function runs in a separate thread from the main UI. When loading is complete, publish a
loading done signal.
'''
archives_data = borgmatic.actions.browse.archive.get_repository_archives(config, repository)
# Reverse the archives, so the common case of accessing the latest archive is easy because it's
# at the top.
for archive in reversed(archives_data['archives']):
archive_loaded.publish(archive['archive'])
archive_loaded.publish(LOADING_DONE)
def record_path(archive_path, hierarchy, path_components):
'''
Given an Archive_path instance, a dict capturing a filesystem hierarchy of paths, and a tuple of
path components for the archive path, set the archive path into the hierarchy data structure.
For instance, if given an archive path and path components representing "foo/bar/baz.txt",
produce a hierarchy that looks like:
{'foo': {'bar': {'baz.txt': Archive_path('-', 'foo/bar/baz.txt', '')}}}
Note that the hierarchy is modified in place, so any existing paths there are retained.
'''
if len(path_components) == 1:
hierarchy[path_components[0]] = {} if archive_path.path_type == 'd' else archive_path
return
record_path(archive_path, hierarchy.setdefault(path_components[0], {}), path_components[1:])
def get_paths(hierarchy, path_components, full_path_components=None):
'''
Given a dict capturing a filesystem hierarchy of paths (or a subset thereof), a tuple of path
components for a directory path relative to the hierarchy root, and an optional tuple of
*absolute* path components for the same path (if different), return a generator of the
contained file and directory Archive_path instances from the hierarchy.
For instance, given the following hierarchy:
{'foo': {'bar': {'baz.txt': Archive_path('-', 'foo/bar/baz.txt', ''), 'quux': {}}}}
... and path components of ('foo', 'bar'), return a generator with the following:
* Archive_path('-', 'foo/bar/baz.txt', '')
* Archive_path('d', 'foo/bar/quux', '')
The given absolute path components are use to construct directory paths like that last archive
path.
'''
if full_path_components is None:
full_path_components = path_components
if len(path_components) == 1:
try:
return (
archive_path
if isinstance(archive_path, borgmatic.actions.browse.archive.Archive_path)
else borgmatic.actions.browse.archive.Archive_path(
'd', os.path.join(*full_path_components, component), ''
)
for component, archive_path in hierarchy[path_components[0]].items()
)
except KeyError:
raise ValueError(f'Unknown file or directory: {path_components[0]}')
try:
return get_paths(hierarchy[path_components[0]], path_components[1:], full_path_components)
except KeyError:
raise ValueError(f'Unknown directory: {path_components[0]}')
class Archive_path_loaded(textual.signal.Signal):
'''
A signal that publishes when each subsequent path is loaded from an archive, intended for
consumption in widgets that display paths as they are loaded. This signal also tracks the
complete filesystem hierarchy seen thus far, so new widgets that get created after loading has
started can "catch up" with existing known paths. Lastly, this signal publishes and tracks when
loading is complete.
Each subscrided callback call includes the loaded path as an Archive_path instance. There is
intended to be a separate Archive_path_loaded instance per archive, but that instance should be
shared among several different widgets for the same archive for performance reasons.
'''
def __init__(self, owner, name):
self.path_hierarchy = {}
self.complete = False
super().__init__(owner, name)
def publish(self, archive_path):
'''
Publish the given archive path to subscribers and record its path locally. But if the
archive path is actually LOADING_DONE, then record loading as complete.
'''
super().publish(archive_path)
if archive_path is LOADING_DONE:
self.complete = True
else:
record_path(
archive_path, self.path_hierarchy, archive_path.file_path.split(os.path.sep)
)
@textual.work(thread=True)
def load_archive_paths(browse_app, path_loaded, config, repository, archive_name):
'''
Given a running Browse_app instance, an Archive_path_loaded instance, a configuration dict, a
repository dict, and an archive name, load the paths in this archive and publish each one via
the Archive_path_loaded signal, so interested widgets can subscribe. Also send a "loading done"
signal when loading completes.
This function runs in a separate thread from the main UI. When loading is complete, publish a
loading done signal.
'''
for archive_path in borgmatic.actions.browse.archive.get_archive_paths(
config, repository, archive_name
):
path_loaded.publish(archive_path)
path_loaded.publish(LOADING_DONE)
class File_preview_loaded(textual.signal.Signal):
'''
A signal that publishes when file contents are loaded from an archive, intended for consumption
in widgets that display loaded files. This signal also publishes when loading is complete.
Each published callback includes a the file's contents as a string. Given the lack of other
identifying information (configuration file, repository, archive), there should be a separate
Archive_loaded instance per previewed file.
'''
@textual.work(thread=True)
def load_file_preview(
browse_app,
file_preview_loaded,
config,
repository,
archive_name,
file_path,
):
'''
Given a running Browse_app instance, a File_preview_loaded instance, a configuration dict, a
repository dict, an archive name, and the path of a file in that archive, load the contents of
the file and write it into the given file preview widget.
This function runs in a separate thread from the main UI. When loading is complete, publish a
loading done signal.
'''
file_contents = borgmatic.actions.browse.archive.get_archive_file_content(
config, repository, archive_name, file_path
)
file_preview_loaded.publish(file_contents)
+3 -3
View File
@@ -383,7 +383,7 @@ def collect_spot_check_source_paths(
# Omit "progress" because it interferes with "list_details".
config=dict(config, progress=False, list_details=True),
patterns=borgmatic.actions.pattern.process_patterns(
borgmatic.actions.pattern.collect_patterns(config)
borgmatic.actions.pattern.collect_patterns(config, working_directory)
+ tuple(
borgmatic.borg.pattern.Pattern(
config_path,
@@ -605,7 +605,7 @@ def compare_spot_check_hashes(
local_borg_version,
global_arguments,
list_paths=source_sample_paths_subset,
path_format='{xxh64}{path}{linktarget}',
path_format='{xxh64}{path}{linktarget}{target}',
local_path=local_path,
remote_path=remote_path,
):
@@ -614,7 +614,7 @@ def compare_spot_check_hashes(
# Borg can't get hashes of stored hard links. So if this is a hard link path (and not
# deemed as the "original" by Borg), then skip hashing of it.
if entry['linktarget']:
if entry.get('linktarget') or entry.get('target'):
source_hashes.pop(os.path.join('/', entry['path']), None)
continue
+1 -1
View File
@@ -45,7 +45,7 @@ def run_create(
with borgmatic.config.paths.Runtime_directory(config) as borgmatic_runtime_directory:
patterns = pattern.process_patterns(
pattern.collect_patterns(config),
pattern.collect_patterns(config, working_directory),
config,
working_directory,
borgmatic_runtime_directory,
+3 -2
View File
@@ -21,10 +21,11 @@ def run_diff(
# Only process patterns if only_patterns flag is set
if diff_arguments.only_patterns:
working_directory = borgmatic.config.paths.get_working_directory(config)
processed_patterns = borgmatic.actions.pattern.process_patterns(
(*borgmatic.actions.pattern.collect_patterns(config),),
(*borgmatic.actions.pattern.collect_patterns(config, working_directory),),
config,
borgmatic.config.paths.get_working_directory(config),
working_directory,
)
else:
processed_patterns = None
+8 -5
View File
@@ -34,10 +34,11 @@ def parse_pattern(pattern_line, default_style=borgmatic.borg.pattern.Pattern_sty
)
def collect_patterns(config):
def collect_patterns(config, working_directory):
'''
Given a configuration dict, produce a single sequence of patterns comprised of the configured
source directories, patterns, excludes, pattern files, and exclude files.
Given a configuration dict and the working directory, produce a single sequence of patterns
comprised of the configured source directories, patterns, excludes, pattern files, and exclude
files.
The idea is that Borg has all these different ways of specifying includes, excludes, source
directories, etc., but we'd like to collapse them all down to one common format (patterns) for
@@ -68,7 +69,8 @@ def collect_patterns(config):
+ tuple(
parse_pattern(pattern_line.strip())
for filename in config.get('patterns_from', ())
for pattern_line in open(filename, encoding='utf-8')
for expanded_path in expand_directory(filename, working_directory)
for pattern_line in open(expanded_path, encoding='utf-8')
if not pattern_line.lstrip().startswith('#')
if pattern_line.strip()
)
@@ -78,7 +80,8 @@ def collect_patterns(config):
borgmatic.borg.pattern.Pattern_style.FNMATCH,
)
for filename in config.get('exclude_from', ())
for exclude_line in open(filename, encoding='utf-8')
for expanded_path in expand_directory(filename, working_directory)
for exclude_line in open(expanded_path, encoding='utf-8')
if not exclude_line.lstrip().startswith('#')
if exclude_line.strip()
)
+4 -2
View File
@@ -30,10 +30,12 @@ def run_recreate(
else:
logger.answer(f'Recreating repository{dry_run_label}')
working_directory = borgmatic.config.paths.get_working_directory(config)
# Collect and process patterns.
processed_patterns = borgmatic.actions.pattern.process_patterns(
(
*borgmatic.actions.pattern.collect_patterns(config),
*borgmatic.actions.pattern.collect_patterns(config, working_directory),
# Also add borgmatic-specific paths, so they don't get excluded from the recreated
# archive. Note that this doesn't currently work for archives created with Borg 1.2 or
# below.
@@ -42,7 +44,7 @@ def run_recreate(
),
),
config,
borgmatic.config.paths.get_working_directory(config),
working_directory,
)
archive = borgmatic.borg.repo_list.resolve_archive_name(
+1 -1
View File
@@ -539,7 +539,7 @@ def run_restore(
with borgmatic.config.paths.Runtime_directory(config) as borgmatic_runtime_directory:
patterns = borgmatic.actions.pattern.process_patterns(
borgmatic.actions.pattern.collect_patterns(config),
borgmatic.actions.pattern.collect_patterns(config, working_directory),
config,
working_directory,
)
+15 -4
View File
@@ -180,7 +180,9 @@ def make_base_create_command( # noqa: PLR0912
open pattern file handle).
'''
if config.get('source_directories_must_exist', True):
borgmatic.borg.pattern.check_all_root_patterns_exist(patterns)
borgmatic.borg.pattern.check_all_root_patterns_exist(
patterns, borgmatic.config.paths.get_working_directory(config)
)
patterns_file = borgmatic.borg.pattern.write_patterns_file(
patterns,
@@ -372,7 +374,9 @@ def create_archive(
if json:
output_log_level = None
elif config.get('list_details') or (config.get('statistics') and not dry_run):
elif config.get('list_details') or (
(config.get('statistics') or config.get('quick_statistics')) and not dry_run
):
output_log_level = logging.ANSWER
else:
output_log_level = logging.INFO
@@ -384,6 +388,11 @@ def create_archive(
create_flags += (
(('--info',) if logger.getEffectiveLevel() == logging.INFO and not json else ())
+ (('--stats',) if config.get('statistics') and not json and not dry_run else ())
+ (
('--quick-stats',)
if config.get('quick_statistics') and not json and not dry_run
else ()
)
+ (('--debug', '--show-rc') if logger.isEnabledFor(logging.DEBUG) and not json else ())
+ (('--progress',) if config.get('progress') else ())
+ (('--json',) if json else ())
@@ -391,7 +400,7 @@ def create_archive(
borg_exit_codes = config.get('borg_exit_codes')
if stream_processes:
return '\n'.join(
output = '\n'.join(
execute_command_with_processes(
create_flags + create_positional_arguments,
stream_processes,
@@ -403,9 +412,10 @@ def create_archive(
borg_exit_codes=borg_exit_codes,
)
)
return output if json else None
if output_log_level is None:
return '\n'.join(
output = '\n'.join(
execute_command_and_capture_output(
create_flags + create_positional_arguments,
working_directory=working_directory,
@@ -414,6 +424,7 @@ def create_archive(
borg_exit_codes=borg_exit_codes,
)
)
return output if json else None
execute_command(
create_flags + create_positional_arguments,
+10 -20
View File
@@ -1,5 +1,4 @@
import itertools
import json
import logging
import re
@@ -135,27 +134,18 @@ def make_match_archives_flags( # noqa: PLR0911
return ('--glob-archives', f'{derived_match_archives}')
def warn_for_aggressive_archive_flags(json_command, json_output):
def warn_for_aggressive_archive_flags(command, output_lines):
'''
Given a JSON archives command and the resulting JSON string output from running it, parse the
JSON and warn if the command used an archive flag but the output indicates zero archives were
found.
Given an archives command and the resulting output lines from running it, warn if the command
used an archive flag but the output indicates zero archives were found.
'''
archive_flags_used = {'--glob-archives', '--match-archives'}.intersection(set(json_command))
if not archive_flags_used:
return
try:
if len(json.loads(json_output)['archives']) == 0:
logger.warning('An archive filter was applied, but no matching archives were found.')
logger.warning(
'Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration.',
)
except json.JSONDecodeError as error:
logger.debug(f'Cannot parse JSON output from archive command: {error}')
except (TypeError, KeyError):
logger.debug('Cannot parse JSON output from archive command: No "archives" key found')
if {'--glob-archives', '--match-archives'}.intersection(set(command)) and len(
tuple(line for line in output_lines if not line.startswith('terminating with '))
) == 0:
logger.warning('An archive filter was applied, but no matching archives were found.')
logger.warning(
'Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration.',
)
def omit_flag(arguments, flag):
+16 -16
View File
@@ -5,7 +5,7 @@ import shlex
import borgmatic.config.paths
import borgmatic.logger
from borgmatic.borg import environment, feature, flags
from borgmatic.execute import execute_command, execute_command_and_capture_output
from borgmatic.execute import execute_command_and_capture_output
logger = logging.getLogger(__name__)
@@ -103,9 +103,21 @@ def display_archives_info(
borg_exit_codes = config.get('borg_exit_codes')
working_directory = borgmatic.config.paths.get_working_directory(config)
json_info = '\n'.join(
if info_arguments.json:
return '\n'.join(
execute_command_and_capture_output(
json_command,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
borg_exit_codes=borg_exit_codes,
)
)
output_lines = tuple(
execute_command_and_capture_output(
json_command,
main_command,
output_log_level=logging.ANSWER,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
@@ -113,18 +125,6 @@ def display_archives_info(
)
)
if info_arguments.json:
return json_info
flags.warn_for_aggressive_archive_flags(json_command, json_info)
execute_command(
main_command,
output_log_level=logging.ANSWER,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
borg_exit_codes=borg_exit_codes,
)
flags.warn_for_aggressive_archive_flags(main_command, output_lines)
return None
+3 -3
View File
@@ -115,10 +115,10 @@ def capture_archive_listing(
Given a local or remote repository path, an archive name, a configuration dict, the local Borg
version, global arguments as an argparse.Namespace, the archive paths (or Borg patterns) in
which to list files, the Borg path format indicating keys to include in the output, and local
and remote Borg paths, capture the output of listing that archive and return it as a sequence of
dicts, one per path.
and remote Borg paths, capture the output of listing that archive and return it as a generator
of dicts, one per path.
'''
return tuple(
return (
json.loads(entry)
for entry in execute_command_and_capture_output(
make_list_command(
+4 -4
View File
@@ -88,16 +88,16 @@ def write_patterns_file(patterns, borgmatic_runtime_directory, patterns_file=Non
return patterns_file
def check_all_root_patterns_exist(patterns):
def check_all_root_patterns_exist(patterns, working_directory):
'''
Given a sequence of Pattern instances, check that all root pattern paths exist. If any don't,
raise an exception.
Given a sequence of Pattern instances and the current working directory, check that all root
pattern paths exist. If any don't, raise an exception.
'''
missing_paths = [
pattern.path
for pattern in patterns
if pattern.type == Pattern_type.ROOT
if not os.path.exists(pattern.path)
if not os.path.exists(os.path.join(working_directory or '', pattern.path))
]
if missing_paths:
+23 -16
View File
@@ -6,7 +6,7 @@ import shlex
import borgmatic.config.paths
import borgmatic.logger
from borgmatic.borg import environment, feature, flags
from borgmatic.execute import execute_command, execute_command_and_capture_output
from borgmatic.execute import execute_command_and_capture_output
logger = logging.getLogger(__name__)
@@ -79,6 +79,13 @@ def get_latest_archive(
*flags.make_flags('umask', config.get('umask')),
*('--log-json',),
*flags.make_flags('lock-wait', config.get('lock_wait')),
*(
flags.make_match_archives_flags(
config.get('match_archives'),
config.get('archive_name_format'),
local_borg_version,
)
),
*(
flags.make_flags('consider-checkpoints', consider_checkpoints)
if not feature.available(feature.Feature.REPO_LIST, local_borg_version)
@@ -219,9 +226,21 @@ def list_repository(
working_directory = borgmatic.config.paths.get_working_directory(config)
borg_exit_codes = config.get('borg_exit_codes')
json_listing = '\n'.join(
if repo_list_arguments.json:
return '\n'.join(
execute_command_and_capture_output(
json_command,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
borg_exit_codes=borg_exit_codes,
)
)
output_lines = tuple(
execute_command_and_capture_output(
json_command,
main_command,
output_log_level=logging.ANSWER,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
@@ -229,18 +248,6 @@ def list_repository(
)
)
if repo_list_arguments.json:
return json_listing
flags.warn_for_aggressive_archive_flags(json_command, json_listing)
execute_command(
main_command,
output_log_level=logging.ANSWER,
environment=environment.make_environment(config),
working_directory=working_directory,
borg_local_path=local_path,
borg_exit_codes=borg_exit_codes,
)
flags.warn_for_aggressive_archive_flags(main_command, output_lines)
return None
+27 -6
View File
@@ -31,9 +31,10 @@ ACTION_ALIASES = {
'transfer': [],
'break-lock': [],
'key': [],
'borg': [],
'recreate': [],
'diff': [],
'browse': [],
'borg': [],
}
@@ -324,7 +325,7 @@ def make_argument_description(schema, flag_name):
' To specify a different list element, replace the "[0]" with another array index ("[1]", "[2]", etc.).',
)
if example and schema_type in ('array', 'object'): # noqa: PLR6201
if example and schema_type in ('array', 'object'):
example_buffer = io.StringIO()
yaml = ruamel.yaml.YAML(typ='safe')
yaml.default_flow_style = True
@@ -896,6 +897,13 @@ def make_parsers(schema, unparsed_arguments): # noqa: PLR0915
action='store_true',
help='Display statistics of archive',
)
create_group.add_argument(
'--quick-stats',
dest='quick_statistics',
default=None,
action='store_true',
help='Display statistics of archive, skipping repository-wide "All archives" and chunk index statistics [Borg 1.4.5+ only]',
)
create_group.add_argument(
'--list',
'--files',
@@ -1997,8 +2005,9 @@ def make_parsers(schema, unparsed_arguments): # noqa: PLR0915
diff_parser = action_parsers.add_parser(
'diff',
aliases=ACTION_ALIASES['diff'],
help='This command finds differences (file contents, user/group/mode) between archives',
description='This command finds differences (file contents, user/group/mode) between archives',
help='Find differences (file contents, user/group/mode) between archives',
description='Find differences (file contents, user/group/mode) between archives',
add_help=False,
)
diff_group = diff_parser.add_argument_group('diff arguments')
diff_group.add_argument(
@@ -2035,6 +2044,18 @@ def make_parsers(schema, unparsed_arguments): # noqa: PLR0915
action='store_true',
help='Run the diff according to borgmatic configured patterns (ie do not diff entire archives)',
)
diff_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
browse_parser = action_parsers.add_parser(
'browse',
aliases=ACTION_ALIASES['browse'],
help='Browse repositories, archives, and files in a console UI',
description='Browse repositories, archives, and files in a console UI',
add_help=False,
)
browse_group = browse_parser.add_argument_group('browse arguments')
browse_group.add_argument('-h', '--help', action='help', help='Show this help message and exit')
borg_parser = action_parsers.add_parser(
'borg',
aliases=ACTION_ALIASES['borg'],
@@ -2107,7 +2128,7 @@ def parse_arguments(schema, *unparsed_arguments):
)
if (
('list' in arguments and 'repo-info' in arguments and arguments['list'].json) # noqa: PLR0916
('list' in arguments and 'repo-info' in arguments and arguments['list'].json)
or ('list' in arguments and 'info' in arguments and arguments['list'].json)
or ('repo-info' in arguments and 'info' in arguments and arguments['repo-info'].json)
):
@@ -2125,7 +2146,7 @@ def parse_arguments(schema, *unparsed_arguments):
'With the repo-list action, only one of --prefix or --match-archives flags can be used.',
)
if 'info' in arguments and ( # noqa: PLR0916
if 'info' in arguments and (
(arguments['info'].archive and arguments['info'].prefix)
or (arguments['info'].archive and arguments['info'].match_archives)
or (arguments['info'].prefix and arguments['info'].match_archives)
+24 -12
View File
@@ -12,6 +12,7 @@ import ruamel.yaml
import borgmatic.actions.borg
import borgmatic.actions.break_lock
import borgmatic.actions.browse.run
import borgmatic.actions.change_passphrase
import borgmatic.actions.check
import borgmatic.actions.compact
@@ -211,7 +212,7 @@ def run_configuration(config_filename, config, config_paths, arguments): # noqa
f"Skipping {'/'.join(skip_actions)} action{'s' if len(skip_actions) > 1 else ''} due to configured skip_actions",
)
try: # noqa: PLR1702
try:
with (
Monitoring_hooks(config_filename, config, arguments, global_arguments),
borgmatic.hooks.command.Before_after_hooks(
@@ -825,8 +826,8 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
try:
if 'bootstrap' in arguments:
try:
# No configuration file is needed for bootstrap.
local_borg_version = borg_version.local_borg_version(
# No configuration file is needed for bootstrap.
{},
arguments['bootstrap'].local_path,
)
@@ -897,6 +898,13 @@ def collect_highlander_action_summary_logs(configs, arguments, configuration_par
return
if 'browse' in arguments:
borgmatic.actions.browse.run.run_browse(
arguments['browse'],
arguments['global'],
configs,
)
except (
CalledProcessError,
ValueError,
@@ -1044,16 +1052,17 @@ def exit_with_help_link(): # pragma: no cover
sys.exit(1)
def check_and_show_help_on_no_args(configs):
def check_and_show_help_on_no_args(configs, schema):
'''
Given a dict of configuration filename to corresponding parsed configuration, check if the
borgmatic command is run without any arguments. If the configuration option "default_actions" is
set to False, show the help message. Otherwise, trigger the default backup behavior.
Given a dict of configuration filename to corresponding parsed configuration and the
configuration schema as a dict, check if the borgmatic command was run without any arguments. If
the configuration option "default_actions" is set to False, then show the help message an exit.
'''
if len(sys.argv) == 1: # No arguments provided
if len(sys.argv) == 1: # No arguments provided.
default_actions = any(config.get('default_actions', True) for config in configs.values())
if not default_actions:
parse_arguments('--help')
if configs and not default_actions:
parse_arguments(schema, '--help')
sys.exit(0)
@@ -1122,7 +1131,7 @@ def main(extra_summary_logs=()): # pragma: no cover
exit_with_help_link()
except SystemExit as error:
if error.code == 0:
raise error
raise
configure_logging(logging.CRITICAL)
logger.critical(f"Error parsing arguments: {' '.join(sys.argv)}")
@@ -1150,8 +1159,7 @@ def main(extra_summary_logs=()): # pragma: no cover
resolve_env=global_arguments.resolve_env and not arguments.get('validate'),
)
# Use the helper function to check and show help on no arguments, passing the preloaded configs
check_and_show_help_on_no_args(configs)
check_and_show_help_on_no_args(configs, schema)
configuration_parse_errors = (
(max(log.levelno for log in parse_logs) >= logging.CRITICAL) if parse_logs else False
@@ -1202,3 +1210,7 @@ def main(extra_summary_logs=()): # pragma: no cover
)
display_summary(summary_logs, log_json)
if __name__ == '__main__': # pragma: no cover
main()
+52 -9
View File
@@ -489,6 +489,9 @@ properties:
description: |
Umask used for when executing Borg or calling hooks. Defaults to
0077 for Borg or the umask that borgmatic is run with for hooks.
Even though this value is a YAML integer, borgmatic interprets it as
octal. YAML's "0o"-prefixed octal notation is not currently
supported.
example: 0077
lock_wait:
type: integer
@@ -1088,6 +1091,15 @@ properties:
Corresponds to the "--stats" flag on those actions. Defaults to
false.
example: true
quick_statistics:
type: boolean
description: |
Display statistics for an archive when running supported actions,
skipping the repository-wide "All archives" and chunk index
statistics to save some time. Corresponds to the "--quick-stats"
flag on those actions. Defaults to false. (This option is supported
for Borg 1.4.5+ only.)
example: true
list_details:
type: boolean
description: |
@@ -1098,9 +1110,9 @@ properties:
default_actions:
type: boolean
description: |
Whether to apply default actions (create, prune, compact and check)
when no arguments are supplied to the borgmatic command. If set to
false, borgmatic displays the help message instead.
Whether to run default actions (create, prune, compact, and check)
when no arguments are given on the command line. If set to false,
borgmatic displays the help message instead.
example: true
skip_actions:
type: array
@@ -1128,8 +1140,9 @@ properties:
- info
- break-lock
- key
- borg
- diff
- browse
- borg
description: |
List of one or more actions to skip running for this configuration
file, even if specified on the command-line (explicitly or
@@ -1345,8 +1358,9 @@ properties:
- info
- break-lock
- key
- borg
- diff
- browse
- borg
description: |
List of actions for which the commands will be
run. Defaults to running for all actions.
@@ -1411,8 +1425,9 @@ properties:
- info
- break-lock
- key
- borg
- diff
- browse
- borg
description: |
Only trigger the hook when borgmatic is run with
particular actions listed here. Defaults to
@@ -2822,7 +2837,9 @@ properties:
properties:
url:
type: string
description: URL of this Apprise service.
description: |
URL of this Apprise service. Supports the
"{credential ...}" syntax.
example: "gotify://hostname/token"
label:
type: string
@@ -3098,6 +3115,24 @@ properties:
Grafana Loki log URL to notify when a backup begins,
ends, or fails.
example: "http://localhost:3100/loki/api/v1/push"
tls:
type: object
additionalProperties: false
properties:
cert_path:
type: string
description: |
Path to a PEM client certificate file for mutual
TLS authentication.
example: /etc/borgmatic/loki-client.crt
key_path:
type: string
description: |
Path to a PEM private key file for the client
certificate.
example: /etc/borgmatic/loki-client.key
description: |
TLS options for mutual TLS (mTLS) authentication with Loki.
labels:
type: object
additionalProperties:
@@ -3291,24 +3326,32 @@ properties:
description: |
Command to use instead of "keepassxc-cli".
example: /usr/local/bin/keepassxc-cli
secret_tool_command:
type: string
description: |
Command to use instead of "secret-tool".
example: /usr/local/bin/secret-tool
ask_for_password:
type: boolean
description: |
Whether keepassxc-cli should prompt the user for a password.
Disabling this is only really useful if you're unlocking
your KeePassXC database with a key file instead of a
password. Defaults to true.
password. Ignored when using KeePassXC's secret service
integration. Defaults to true.
example: false
key_file:
type: string
description: |
Path to a key file for unlocking the KeePassXC database.
Ignored when using KeePassXC's secret service integration.
example: /path/to/keyfile
yubikey:
type: string
description: |
YubiKey slot and optional serial number used to access the
KeePassXC database. The format is "<slot[:serial]>", where:
KeePassXC database. Ignored when using KeePassXC's secret
service integration. The format is "<slot[:serial]>", where:
* <slot> is the YubiKey slot number (e.g., `1` or `2`).
* <serial> (optional) is the YubiKey's serial number (e.g.,
`7370001`).
+102 -61
View File
@@ -43,6 +43,18 @@ def command_is_borg(command, borg_local_path):
return bool(borg_local_path and parsed_command[0] == borg_local_path)
BORG_EXIT_CODE_TO_DESCRIPTION = {
100: 'File changed while we backed it up',
101: 'Include pattern never matched',
102: 'General backup issue',
103: 'File type or inode changed while we backed it up',
104: 'Backup OS issue',
105: 'Backup permission issue',
106: 'Backup IO issue',
107: 'Backup file not found',
}
def interpret_exit_code(command, exit_code, borg_local_path=None, borg_exit_codes=None): # noqa: PLR0911
'''
Return an Exit_status value (e.g. SUCCESS, ERROR, or WARNING) based on interpreting the given
@@ -56,46 +68,47 @@ def interpret_exit_code(command, exit_code, borg_local_path=None, borg_exit_code
if exit_code == 0:
return Exit_status.SUCCESS
if command_is_borg(command, borg_local_path):
# First try looking for the exit code in the borg_exit_codes configuration.
for entry in borg_exit_codes or ():
if entry.get('code') == exit_code:
treat_as = entry.get('treat_as')
if not command_is_borg(command, borg_local_path):
return Exit_status.ERROR
if treat_as == 'error':
logger.error(
f'Treating exit code {exit_code} as an error, as per configuration',
)
return Exit_status.ERROR
description = BORG_EXIT_CODE_TO_DESCRIPTION.get(exit_code)
description_parenthetical = f' ({description})' if description else ''
if treat_as == 'warning':
logger.warning(
f'Treating exit code {exit_code} as a warning, as per configuration',
)
return Exit_status.WARNING
# First try looking for the exit code in the borg_exit_codes configuration.
for entry in borg_exit_codes or ():
if entry.get('code') == exit_code:
treat_as = entry.get('treat_as')
# If the exit code doesn't have explicit configuration, then fall back to the default
# behavior of treating Borg errors as errors and some Borg warnings as errors.
if exit_code in BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS:
logger.error(
f'Treating exit code {exit_code} as an error, as per borgmatic defaults',
)
return Exit_status.ERROR
return (
Exit_status.ERROR
if (
exit_code < 0
or (
exit_code >= BORG_ERROR_EXIT_CODE_START
and exit_code <= BORG_ERROR_EXIT_CODE_END
if treat_as == 'error':
logger.error(
f'Treating exit code {exit_code}{description_parenthetical} as an error, as per configuration',
)
)
else Exit_status.WARNING
return Exit_status.ERROR
if treat_as == 'warning':
logger.warning(
f'Treating exit code {exit_code}{description_parenthetical} as a warning, as per configuration',
)
return Exit_status.WARNING
# If the exit code doesn't have explicit configuration, then fall back to the default
# behavior of treating Borg errors as errors and some Borg warnings as errors.
if exit_code in BORG_WARNING_EXIT_CODES_TREATED_AS_ERRORS:
logger.error(
f'Treating exit code {exit_code}{description_parenthetical} as an error, as per borgmatic defaults',
)
return Exit_status.ERROR
return Exit_status.ERROR
if exit_code < 0 or (
exit_code >= BORG_ERROR_EXIT_CODE_START and exit_code <= BORG_ERROR_EXIT_CODE_END
):
return Exit_status.ERROR
logger.warning(
f'Treating exit code {exit_code}{description_parenthetical} as a warning, as per borgmatic defaults',
)
return Exit_status.WARNING
def command_for_process(process):
@@ -301,8 +314,9 @@ def log_buffer_lines(
'''
Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
Process_metadata instance, a requested output log level for stdout, Borg's local path, and
whether to capture stderr, read and log any ready output lines from the buffers. Additionally,
if the log level is None for any log record, then yield those log messages for capture.
whether to capture stderr, read and log any ready output lines from the buffers. Additionally,
for any log records with a log level the same as the output log level, yield those log messages
for capture.
This function just does one "turn of the crank" of logging buffer output. It is intended to be
called repeatedly to continue to process buffers.
@@ -352,17 +366,27 @@ def log_buffer_lines(
last_lines=process_metadatas[reader.process].last_lines,
)
if log_record.levelno is None and process_metadatas[reader.process].capture:
if (
log_record.levelno is None or log_record.levelno == output_log_level
) and process_metadatas[reader.process].capture:
yield log_record.getMessage()
def raise_for_process_errors(buffer_readers, process_metadatas, borg_local_path, borg_exit_codes):
def raise_for_process_errors(
buffer_readers,
process_metadatas,
output_log_level,
borg_local_path,
borg_exit_codes,
capture_stderr=False,
):
'''
Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
Process_metadata instance, Borg's local path, a sequence of exit code configuration dicts, check
the given processes for error or warning exit codes. If found, vent or kill any running
processes. In the case of an error exit code, raise. In the case of warning, return
Exit_status.WARNING. Otherwise, return None.
Process_metadata instance, a requested output log level for stdout, Borg's local path, a
sequence of exit code configuration dicts, and whether to capture stderr, check the given
processes for error or warning exit codes. If found, vent or kill any running processes and
drain any remaining buffer lines. In the case of an error exit code, raise. In the case of
warning, return Exit_status.WARNING. Otherwise, return None.
'''
result_status = None
@@ -388,6 +412,13 @@ def raise_for_process_errors(buffer_readers, process_metadatas, borg_local_path,
result_status = Exit_status.WARNING
continue
# Drain and log remaining buffer lines, so no output gets lost. But swallow captured output,
# since we're raising below instead of yielding captured lines.
tuple(
log_remaining_buffer_lines(
buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
)
)
last_lines = process_metadatas[process].last_lines
# If an error occurs, include its output in the raised exception so that we don't
@@ -411,8 +442,11 @@ def log_remaining_buffer_lines(
Given a dict from buffer object to Buffer_reader, a dict from subprocess.Popen() instance to
Process_metadata instance, a requested output log level for stdout, Borg's local path, and
whether to capture stderr, drain and log any remaining output lines from the buffers until
they're empty. Additionally, if the log level is None for any log record, then yield those log
messages for capture.
they're empty. Additionally, for any log records with a log level the same as the output log
level, yield those log messages for capture.
The main difference between this function and log_buffer_lines() is that this one doesn't just
do one "turn of the crank" of logging buffer output; it completely drains any remaining output.
'''
for output_buffer, reader in buffer_readers.items():
if not reader.process:
@@ -430,9 +464,12 @@ def log_remaining_buffer_lines(
borg_local_path=borg_local_path,
command=reader.process.args,
),
last_lines=process_metadatas[reader.process].last_lines,
)
if log_record.levelno is None and process_metadatas[reader.process].capture:
if (
log_record.levelno is None or log_record.levelno == output_log_level
) and process_metadatas[reader.process].capture:
yield log_record.getMessage()
@@ -476,7 +513,7 @@ def log_outputs(
for buffer in output_buffers_for_process(process, exclude_stdouts)
}
# Log output lines for each process until they all exit.
# Log output lines for each process until they all exit or one errors.
while True:
yield from log_buffer_lines(
buffer_readers, process_metadatas, output_log_level, borg_local_path, capture_stderr
@@ -484,7 +521,12 @@ def log_outputs(
if (
raise_for_process_errors(
buffer_readers, process_metadatas, borg_local_path, borg_exit_codes
buffer_readers,
process_metadatas,
output_log_level,
borg_local_path,
borg_exit_codes,
capture_stderr,
)
== Exit_status.WARNING
):
@@ -615,6 +657,7 @@ def execute_command(
def execute_command_and_capture_output(
full_command,
output_log_level=None,
input_file=None,
capture_stderr=False,
shell=False,
@@ -629,13 +672,15 @@ def execute_command_and_capture_output(
output (stdout) as a generator that yields one line at a time. The generator must be consumed in
order for the called command to execute.
If an input file descriptor is given, then pipe it to the command's stdin. If capture stderr is
True, then capture stderr in addition to stdout. If shell is True, execute the command within a
shell. If an environment variables dict is given, then pass it into the command. If a working
directory is given, use that as the present working directory when running the command. If a
Borg local path is given, and the command matches it (regardless of arguments), treat exit code
1 as a warning instead of an error. But if Borg exit codes are given as a sequence of exit code
configuration dicts, then use that configuration to decide what's an error and what's a warning.
If an output log level is given, then instead of suppressing log output, also output the
captured lines at the given log level. If an input file descriptor is given, then pipe it to
the command's stdin. If capture stderr is True, then capture stderr in addition to stdout. If
shell is True, execute the command within a shell. If an environment variables dict is given,
then pass it into the command. If a working directory is given, use that as the present working
directory when running the command. If a Borg local path is given, and the command matches it
(regardless of arguments), treat exit code 1 as a warning instead of an error. But if Borg exit
codes are given as a sequence of exit code configuration dicts, then use that configuration to
decide what's an error and what's a warning.
Raise subprocesses.CalledProcessError if an error occurs while running the command.
'''
@@ -668,17 +713,15 @@ def execute_command_and_capture_output(
return
with borgmatic.logger.Log_prefix(None): # Log command output without any prefix.
captured_lines = log_outputs(
yield from log_outputs(
(process,),
(input_file,),
None,
output_log_level,
borg_local_path,
borg_exit_codes,
capture_stderr=capture_stderr,
)
yield from captured_lines
def execute_command_with_processes(
full_command,
@@ -741,12 +784,10 @@ def execute_command_with_processes(
raise
with borgmatic.logger.Log_prefix(None): # Log command output without any prefix.
captured_lines = log_outputs(
yield from log_outputs(
(*processes, command_process),
(input_file, output_file),
output_log_level,
borg_local_path,
borg_exit_codes,
)
yield from captured_lines
+17 -2
View File
@@ -7,19 +7,34 @@ import borgmatic.execute
logger = logging.getLogger(__name__)
SECRET_SERVICE_DATABASE_PATH = 'secret-service'
def load_credential(hook_config, config, credential_parameters):
'''
Given the hook configuration dict, the configuration dict, and a credential parameters tuple
containing a KeePassXC database path and an attribute name to load, run keepassxc-cli to fetch
the corresponding KeePassXC credential and return it.
the corresponding KeePassXC credential and return it. Or use secret-tool if the database path
is "secret-service", indicating that KeePassXC's secret service integration should be used
instead.
Raise ValueError if keepassxc-cli can't retrieve the credential.
Raise ValueError if keepassxc-cli or secret-tool can't retrieve the credential.
'''
try:
(database_path, attribute_name) = credential_parameters
except ValueError:
raise ValueError(f'Invalid KeePassXC credential parameters: {credential_parameters}')
if database_path == SECRET_SERVICE_DATABASE_PATH:
command = (
*shlex.split((hook_config or {}).get('secret_tool_command', 'secret-tool')),
*('lookup', 'Path', attribute_name),
)
return '\n'.join(borgmatic.execute.execute_command_and_capture_output(command)).rstrip(
os.linesep
)
expanded_database_path = os.path.expanduser(database_path)
if not os.path.exists(expanded_database_path):
+6 -3
View File
@@ -28,17 +28,20 @@ def resolve_config_path_symlinks(path):
Given a path, resolve and yield each successive symlink until the final non-symlink target. If
the given path isn't a symlink, then just yield it.
The purpose of this is to ensure that configuration files that are behind a symbolic link (or
several) actually get backed up.
Raise ValueError if we have to follow too many symlinks without getting to the final target.
'''
original_path = path
original_path = os.path.normpath(path)
for _ in range(MAXIMUM_CONFIG_SYMLINKS_TO_FOLLOW):
yield os.path.abspath(path)
yield path
if not os.path.islink(path):
return
path = os.readlink(path)
path = os.path.normpath(os.path.join(os.path.dirname(path), os.readlink(path)))
raise ValueError(f'Too many symlinks to follow for configuration path: {original_path}')
+18 -10
View File
@@ -1,4 +1,3 @@
import copy
import logging
import os
import re
@@ -67,7 +66,9 @@ def make_defaults_file_options(username=None, password=None, defaults_extra_file
Do not use the returned value for multiple different command invocations. That will not work
because each pipe is "used up" once read.
'''
escaped_password = None if password is None else password.replace('\\', '\\\\')
escaped_password = (
None if password is None else password.replace('\\', '\\\\').replace('"', '\\"')
)
values = '\n'.join(
(
@@ -106,6 +107,10 @@ def make_defaults_file_options(username=None, password=None, defaults_extra_file
return (f'--defaults-extra-file=/dev/fd/{read_file_descriptor}',)
EXCLUDED_SYSTEM_DATABASE_NAMES = ('information_schema', 'performance_schema', 'sys')
SYSTEM_DATABASE_NAME = 'mysql'
def database_names_to_dump(database, config, username, password, environment, dry_run):
'''
Given a requested database config, a configuration dict, a database username and password, an
@@ -164,17 +169,18 @@ def database_names_to_dump(database, config, username, password, environment, dr
working_directory=borgmatic.config.paths.get_working_directory(config),
)
# Dumping system databases directly doesn't work; too much gets dumped (including virtual tables
# that MariaDB recreates on startup) and so the dump isn't restorable. Therefore several system
# databases are excluded here. But also see below where select system tables from the "mysql"
# system table are dumped via the "--system" flag.
return tuple(
show_name.strip()
for show_name in show_lines
if show_name not in SYSTEM_DATABASE_NAMES
if show_name not in EXCLUDED_SYSTEM_DATABASE_NAMES
if not skip_names or show_name not in skip_names
)
SYSTEM_DATABASE_NAMES = ('information_schema', 'mysql', 'performance_schema', 'sys')
def execute_dump_command(
database,
config,
@@ -235,8 +241,9 @@ def execute_dump_command(
+ (('--user', username) if username and password_transport == 'environment' else ())
+ (('--ssl',) if database.get('tls') is True else ())
+ (('--skip-ssl',) if database.get('tls') is False else ())
+ ('--databases',)
+ database_names
+ ('--databases', '--events', '--routines', '--all-tablespaces')
+ (('--system=users,udfs,servers',) if SYSTEM_DATABASE_NAME in database_names else ())
+ tuple(name for name in database_names if name != SYSTEM_DATABASE_NAME)
+ ('--result-file', dump_filename)
)
@@ -323,6 +330,7 @@ def dump_data_sources(
raise ValueError('Cannot find any MariaDB databases to dump.')
# Database dumps to individual files.
if database['name'] == 'all' and database.get('format'):
for database_name in dump_database_names:
dumps_metadata.append(
@@ -335,8 +343,7 @@ def dump_data_sources(
database.get('container'),
)
)
renamed_database = copy.copy(database)
renamed_database['name'] = database_name
renamed_database = dict(database, name=database_name)
processes.append(
execute_dump_command(
renamed_database,
@@ -350,6 +357,7 @@ def dump_data_sources(
dry_run_label,
),
)
# Database dumps all to one file.
else:
dumps_metadata.append(
borgmatic.actions.restore.Dump(
+2 -4
View File
@@ -1,4 +1,3 @@
import copy
import logging
import os
import shlex
@@ -166,7 +165,7 @@ def execute_dump_command(
+ (('--user', username) if username and password_transport == 'environment' else ())
+ (('--ssl',) if database.get('tls') is True else ())
+ (('--skip-ssl',) if database.get('tls') is False else ())
+ ('--databases',)
+ ('--databases', '--events', '--routines', '--all-tablespaces')
+ database_names
+ ('--result-file', dump_filename)
)
@@ -266,8 +265,7 @@ def dump_data_sources(
database.get('container'),
)
)
renamed_database = copy.copy(database)
renamed_database['name'] = database_name
renamed_database = dict(database, name=database_name)
processes.append(
execute_dump_command(
renamed_database,
+8 -4
View File
@@ -100,7 +100,11 @@ def database_names_to_dump(database, config, environment, dry_run):
if 'username' in database
else ()
)
+ (tuple(database['list_options'].split(' ')) if 'list_options' in database else ())
+ (
tuple(shlex.quote(part) for part in shlex.split(database['list_options']))
if 'list_options' in database
else ()
)
)
logger.debug('Querying for "all" PostgreSQL databases to dump')
list_lines = execute_command_and_capture_output(
@@ -226,7 +230,7 @@ def dump_data_sources(
+ (('--compress', shlex.quote(str(compression))) if compression is not None else ())
+ (('--file', shlex.quote(dump_filename)) if dump_format == 'directory' else ())
+ (
tuple(shlex.quote(option) for option in database['options'].split(' '))
tuple(shlex.quote(part) for part in shlex.split(database['options']))
if 'options' in database
else ()
)
@@ -393,7 +397,7 @@ def restore_data_source_dump(
+ (('--username', username) if username else ())
+ (('--dbname', data_source['name']) if not all_databases else ())
+ (
tuple(data_source['analyze_options'].split(' '))
tuple(shlex.quote(part) for part in shlex.split(data_source['analyze_options']))
if 'analyze_options' in data_source
else ()
)
@@ -414,7 +418,7 @@ def restore_data_source_dump(
+ (('--username', username) if username else ())
+ (('--no-owner',) if data_source.get('no_owner', False) else ())
+ (
tuple(data_source['restore_options'].split(' '))
tuple(shlex.quote(part) for part in shlex.split(data_source['restore_options']))
if 'restore_options' in data_source
else ()
)
+22 -16
View File
@@ -2,6 +2,7 @@ import collections
import glob
import hashlib
import logging
import operator
import os
import shutil
import subprocess
@@ -123,7 +124,8 @@ def get_datasets_to_backup(zfs_command, patterns):
def get_all_dataset_mount_points(zfs_command):
'''
Given a ZFS command to run, return all ZFS datasets as a sequence of sorted mount points.
Given a ZFS command to run, return a dict from ZFS dataset name to mount point (reverse sorted
by mount point).
'''
list_lines = borgmatic.execute.execute_command_and_capture_output(
(
@@ -133,19 +135,23 @@ def get_all_dataset_mount_points(zfs_command):
'-t',
'filesystem',
'-o',
'mountpoint',
'name,mountpoint',
),
close_fds=True,
)
return tuple(
return dict(
sorted(
{
mount_point
(
(dataset_name, mount_point)
for line in list_lines
for mount_point in (line.rstrip(),)
for (dataset_name, mount_point) in (line.rstrip().split('\t'),)
if mount_point != 'none'
},
),
key=operator.itemgetter(1),
# Reversing the sorted datasets ensures that we unmount the longer mount point paths of
# child datasets before the shorter mount point paths of parent datasets.
reverse=True,
),
)
@@ -376,7 +382,8 @@ def remove_data_source_dumps(hook_config, config, borgmatic_runtime_directory, p
zfs_command = hook_config.get('zfs_command', 'zfs')
try:
dataset_mount_points = get_all_dataset_mount_points(zfs_command)
dataset_name_to_mount_point = get_all_dataset_mount_points(zfs_command)
full_snapshot_names = get_all_snapshots(zfs_command)
except FileNotFoundError:
logger.debug(f'Could not find "{zfs_command}" command')
return
@@ -393,19 +400,20 @@ def remove_data_source_dumps(hook_config, config, borgmatic_runtime_directory, p
)
logger.debug(f'Looking for snapshots to remove in {snapshots_glob}{dry_run_label}')
umount_command = hook_config.get('umount_command', 'umount')
snapshot_dataset_names = {
full_snapshot_name.split('@')[0] for full_snapshot_name in full_snapshot_names
}
for snapshots_directory in glob.glob(snapshots_glob):
if not os.path.isdir(snapshots_directory):
continue
# Reversing the sorted datasets ensures that we unmount the longer mount point paths of
# child datasets before the shorter mount point paths of parent datasets.
for mount_point in reversed(dataset_mount_points):
for dataset_name, mount_point in dataset_name_to_mount_point.items():
snapshot_mount_path = os.path.join(snapshots_directory, mount_point.lstrip(os.path.sep))
# If the snapshot mount path is empty, this is probably just a "shadow" of a nested
# dataset and therefore there's nothing to unmount.
if not os.path.isdir(snapshot_mount_path) or not os.listdir(snapshot_mount_path):
# If this dataset name does not correspond to a known snapshot, then this is probably
# just a "shadow" of a nested dataset and therefore there's nothing to unmount.
if not os.path.isdir(snapshot_mount_path) or dataset_name not in snapshot_dataset_names:
continue
# This might fail if the path is already mounted, but we swallow errors here since we'll
@@ -435,8 +443,6 @@ def remove_data_source_dumps(hook_config, config, borgmatic_runtime_directory, p
shutil.rmtree(snapshot_mount_path, ignore_errors=True)
# Destroy snapshots.
full_snapshot_names = get_all_snapshots(zfs_command)
for full_snapshot_name in full_snapshot_names:
# Only destroy snapshots that borgmatic actually created!
if not full_snapshot_name.split('@')[-1].startswith(BORGMATIC_SNAPSHOT_PREFIX):
+10 -2
View File
@@ -1,6 +1,7 @@
import logging
import operator
import borgmatic.hooks.credential.parse
import borgmatic.hooks.monitoring.logs
import borgmatic.hooks.monitoring.monitor
@@ -44,7 +45,9 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
import apprise # noqa: PLC0415
from apprise import NotifyFormat, NotifyType # noqa: PLC0415
except ImportError: # pragma: no cover
logger.warning('Unable to import Apprise in monitoring hook')
logger.warning(
'Unable to import Apprise in its monitoring hook; try installing "borgmatic[Apprise]"'
)
return
state_to_notify_type = {
@@ -76,7 +79,12 @@ def ping_monitor(hook_config, config, config_filename, state, monitoring_log_lev
logger.info(f'Pinging Apprise services: {labels_string}{dry_run_string}')
apprise_object = apprise.Apprise()
apprise_object.add(list(map(operator.itemgetter('url'), hook_config.get('services'))))
apprise_object.add(
[
borgmatic.hooks.credential.parse.resolve_credential(service['url'], config)
for service in hook_config.get('services')
]
)
if dry_run:
return
+39 -10
View File
@@ -27,9 +27,16 @@ class Loki_log_buffer:
adding labels to the log stream and takes care of communication with Loki.
'''
def __init__(self, url, dry_run):
def __init__(self, url, dry_run, tls_cert_path=None, tls_key_path=None):
'''
Given a Loki URL, a dry run flag, and optional TLS certificate and key paths for mTLS authentication,
create an instance of Loki_log_buffer.
'''
self.url = url
self.dry_run = dry_run
self.tls_cert_path = tls_cert_path
self.tls_key_path = tls_key_path
self.root = {'streams': [{'stream': {}, 'values': []}]}
def add_value(self, value):
@@ -77,6 +84,7 @@ class Loki_log_buffer:
'Content-Type': 'application/json',
'User-Agent': 'borgmatic',
},
cert=(self.tls_cert_path, self.tls_key_path) if self.tls_cert_path else None,
)
result.raise_for_status()
except requests.RequestException:
@@ -88,16 +96,19 @@ class Loki_log_handler(logging.Handler):
A log handler that sends logs to Loki.
'''
def __init__(self, url, send_logs, dry_run):
def __init__(self, url, send_logs, log_level, dry_run, tls_cert_path=None, tls_key_path=None):
'''
Given a URL to send logs to, whether all borgmatic logs should be sent (or just explicitly
added messages from this hook), and whether this is a dry run, create an instance of
Loki_log_buffer.
added messages from this hook), the log level to use (influencing which logs get sent), and
whether this is a dry run, create an instance of Loki_log_buffer.
'''
super().__init__()
self.buffer = Loki_log_buffer(url, dry_run)
self.buffer = Loki_log_buffer(
url, dry_run, tls_cert_path=tls_cert_path, tls_key_path=tls_key_path
)
self.send_logs = send_logs
self.setLevel(log_level)
def emit(self, record):
'''
@@ -138,7 +149,21 @@ def initialize_monitor(hook_config, config, config_filename, monitoring_log_leve
Add a handler to the root logger to regularly send the logs to Loki.
'''
url = hook_config.get('url')
loki = Loki_log_handler(url, hook_config.get('send_logs', False), dry_run)
tls = hook_config.get('tls', {})
if bool(tls.get('cert_path')) != bool(tls.get('key_path')):
raise ValueError(
'Invalid Loki TLS configuration: cert_path and key_path must both be set or both be unset'
)
loki = Loki_log_handler(
url,
hook_config.get('send_logs', False),
monitoring_log_level,
dry_run,
tls_cert_path=tls.get('cert_path'),
tls_key_path=tls.get('key_path'),
)
for key, value in hook_config.get('labels').items():
if value == '__hostname':
@@ -150,7 +175,9 @@ def initialize_monitor(hook_config, config, config_filename, monitoring_log_leve
else:
loki.add_label(key, value)
logging.getLogger().addHandler(loki)
global_logger = logging.getLogger()
global_logger.addHandler(loki)
global_logger.setLevel(min(handler.level for handler in global_logger.handlers))
def ping_monitor(hook_config, config, config_filename, state, monitoring_log_level, dry_run):
@@ -166,9 +193,11 @@ def destroy_monitor(hook_config, config, monitoring_log_level, dry_run):
'''
Remove the monitor handler that was added to the root logger.
'''
logger = logging.getLogger()
global_logger = logging.getLogger()
for handler in tuple(logger.handlers):
for handler in tuple(global_logger.handlers):
if isinstance(handler, Loki_log_handler):
handler.flush()
logger.removeHandler(handler)
global_logger.removeHandler(handler)
global_logger.setLevel(min(handler.level for handler in global_logger.handlers))
+1 -1
View File
@@ -169,7 +169,7 @@ class Json_formatter(logging.Formatter):
def __init__(self, fmt='{message}', *args, style='{', **kwargs):
super().__init__(*args, fmt=fmt, style=style, **kwargs)
def format(self, record): # noqa: PLR6301
def format(self, record):
return log_record_to_json(record)
+1 -1
View File
@@ -5,7 +5,7 @@ RUN apk add --no-cache py3-pip py3-ruamel.yaml py3-ruamel.yaml.clib
RUN pip install --break-system-packages --no-cache /app && borgmatic config generate && borgmatic config generate --destination /etc/borgmatic --split && chmod +r /etc/borgmatic/*.yaml
RUN mkdir /command-line \
&& borgmatic --help > /command-line/global.txt \
&& for action in repo-create transfer create prune compact check delete extract config "config bootstrap" "config generate" "config validate" "config show" export-tar mount umount repo-delete restore repo-list list repo-info info break-lock "key export" "key import" "key change-passphrase" recreate diff borg; do \
&& for action in repo-create transfer create prune compact check delete extract config "config bootstrap" "config generate" "config validate" "config show" export-tar mount umount repo-delete restore repo-list list repo-info info break-lock "key export" "key import" "key change-passphrase" recreate diff browse borg; do \
borgmatic $action --help > /command-line/${action/ /-}.txt; done
RUN /app/docs/fetch-contributors >> /contributors.html
+1
View File
@@ -23,6 +23,7 @@
{% if feedTitle and feedUrl %}
<link rel="alternate" href="{{ feedUrl }}" title="{{ feedTitle }}" type="application/atom+xml">
{% endif %}
{{ head_additions | safe }}
</head>
<body>
@@ -1,5 +1,5 @@
Here's the command-line help for this action in the [most recent version of
borgmatic](https://projects.torsion.org/borgmatic-collective/borgmatic/releases).
If you're using an older version, some of these flags may not work, and you
should instead run the action with `--help` to see the flags specific to your
borgmatic version.
If you're using an older version, some of these flags may not work (or the action
may be missing entirely). You should instead run the action with `--help` to see
the flags and actions specific to your borgmatic version.
+1 -1
View File
@@ -14,7 +14,7 @@ import requests
def list_merged_pulls(url):
'''
Given a Gitea or GitHub API endpoint URL for pull requests, fetch and return the corresponding
Given a Forgejo or GitHub API endpoint URL for pull requests, fetch and return the corresponding
JSON for all such merged pull requests.
'''
response = requests.get(f'{url}?state=closed', headers={'Accept': 'application/json', 'Content-Type': 'application/json'})
+10 -3
View File
@@ -31,7 +31,7 @@ changes work:
```bash
cd borgmatic
uv tool update-shell
uv tool install --editable .
uv tool install --editable .[dev]
```
Or to work on the [Apprise
@@ -39,7 +39,14 @@ hook](https://torsion.org/borgmatic/reference/configuration/monitoring/apprise/)
change that last line to:
```bash
uv tool install --editable .[Apprise]
uv tool install --editable .[dev,Apprise]
```
Or to work on the [browse
action](https://torsion.org/borgmatic/reference/command-line/actions/browse/):
```bash
uv tool install --editable .[dev,browse]
```
To get oriented with the borgmatic source code, have a look at the [source
@@ -236,7 +243,7 @@ Compose.
Please do not use AI agents to modify this codebase. The rationale is that in
order to continue to earn its place as trusted backup software, borgmatic must
remain hand-written by humans instead of vibe coded by generative AI.
remain handwritten by humans instead of vibe coded by generative AI.
Additionally, if LLMs were to perform a sizeable chunk of the feature
development on this codebase, then human borgmatic developers would lose their
+49
View File
@@ -135,3 +135,52 @@ is, patterns are _not_ taken into consideration. If you require this, supply the
See the [Borg](https://borgbackup.readthedocs.io/en/stable/usage/diff.html)
documentation for information on output format, what is compared, and more.
## Browsing backups
<span class="minilink minilink-addedin">New in version 2.1.6</span> <span
class="minilink minilink-addedin">Experimental feature</span> borgmatic has an
experimental console UI for browsing your repositories, archives, and files.
Here's what it looks like:
<img src="https://torsion.org/borgmatic/static/browse.png" alt="borgmatic browse screenshot" style="width: 100%">
This feature is not intended to be a general-purpose Borg UI with every
borgmatic feature, but rather it's for use cases like quickly looking at the
contents of your backups when you're feeling too lazy to type out a full
borgmatic command-line.
Depending on how you installed borgmatic, it may not have come with the
necessary Python libraries to support the browse action. (borgmatic's
[stand-alone
binary](https://projects.torsion.org/borgmatic-collective/borgmatic/releases)
does not currently include them.) If you originally [installed borgmatic with
uv](https://torsion.org/borgmatic/how-to/install-borgmatic/), run the following
to install the libraries needed for the browse action:
```bash
sudo uv tool install borgmatic[browse]
```
Omit `sudo` if borgmatic is installed as a non-root user.
Once the libraries are installed, run the following to access the browse action:
```bash
borgmatic browse
```
This launches a console UI where you can select a borgmatic configuration file
(if there's more than one), select a Borg repository, select an archive in that
repository, and even browse the backed up files in that archive.
Use the keyboard or the mouse to navigate the UI. The footer at the bottom of
the screen shows some of the available keys. Logs show up directly in the UI at
the [selected
verbosity](https://torsion.org/borgmatic/reference/command-line/logging/),
although logs are hidden by default.
Please [provide
feedback](https://torsion.org/borgmatic/#support-and-contributing) if you find
this feature useful—or even if you don't, but would like it to become useful.
+108
View File
@@ -0,0 +1,108 @@
---
title: 📥 How to install borgmatic
eleventyNavigation:
key: 📥 Install borgmatic
parent: How-to guides
order: -1
---
### Prerequisites
Before installing borgmatic, first [install
Borg](https://borgbackup.readthedocs.io/en/stable/installation.html), at least
version 1.1. (borgmatic does not install Borg automatically so as to avoid
conflicts with existing Borg installations.)
Then, [install uv](https://docs.astral.sh/uv/getting-started/installation/) as
the root user (with `sudo`) to make installing borgmatic easier without
impacting other Python applications on your system. For Debian, there is a
[third-party package for
uv](https://dario.griffo.io/posts/how-to-install-uv-debian/). On Ubuntu, there
is a [snap package](https://snapcraft.io/install/astral-uv/ubuntu). On Arch, you
can just install the `python-uv` package.
### Root install
If you want borgmatic to run with privileged access so it can backup your system
files, then install borgmatic as the root user by running the following
commands:
```bash
sudo uv tool update-shell
sudo uv tool install borgmatic
```
Check whether this worked with:
```bash
sudo su -
borgmatic --version
```
If borgmatic is properly installed, that should output your borgmatic version.
And if you'd also like `sudo borgmatic` to work as well, keep reading!
### Non-root install
If you only want to run borgmatic as a non-root user (without privileged file
access) *or* you want to make `sudo borgmatic` work so borgmatic runs as root,
then install borgmatic as a non-root user by running the following commands as
that user:
```bash
uv tool update-shell
uv tool install borgmatic
```
This should work even if you've also installed borgmatic as the root user.
Check whether this worked with:
```bash
borgmatic --version
```
If borgmatic is properly installed, that should output your borgmatic version.
You can also try `sudo borgmatic --version` if you intend to run borgmatic
with `sudo`. If that doesn't work, you may need to update your [sudoers
`secure_path` option](https://wiki.archlinux.org/title/Sudo).
### Other ways to install
Besides the approaches described above, there are several other options for
installing borgmatic:
#### <span data-pagefind-weight="7.0">Docker / Podman</span>
* [container image with scheduled backups](https://github.com/borgmatic-collective/docker-borgmatic) (+ Docker Compose files)
* [container image with multi-arch and Docker CLI support](https://github.com/modem7/docker-borgmatic)
* [Borgmatic Director UI](https://github.com/SpeedbitsInfinityTools/borgmatic-ui-community)
#### Operating system packages
* [Debian](https://tracker.debian.org/pkg/borgmatic)
* [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
* [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
* [Gentoo](https://packages.gentoo.org/packages/app-backup/borgmatic)
* [Arch Linux](https://archlinux.org/packages/extra/any/borgmatic/)
* [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=borgmatic)
* [OpenBSD](https://openports.pl/path/sysutils/borgmatic)
* [openSUSE](https://software.opensuse.org/package/borgmatic)
* [macOS (via Homebrew)](https://formulae.brew.sh/formula/borgmatic)
* [macOS (via MacPorts)](https://ports.macports.org/port/borgmatic/)
* [NixOS](https://search.nixos.org/packages?channel=unstable&show=borgmatic&query=borgmatic)
#### Etc.
* [stand-alone Linux binary](https://projects.torsion.org/borgmatic-collective/borgmatic/releases) (This is a beta feature! The minimal binary omits [Apprise support](https://torsion.org/borgmatic/reference/configuration/monitoring/apprise/).)
* [Ansible role](https://github.com/borgbase/ansible-role-borgbackup)
* [pipx](https://pipx.pypa.io/stable/)
## Next steps
* [Set up backups](https://torsion.org/borgmatic/how-to/set-up-backups/)
+6 -96
View File
@@ -1,104 +1,14 @@
---
title: 📥 How to set up backups
title: 📋 How to set up backups
eleventyNavigation:
key: 📥 Set up backups
key: 📋 Set up backups
parent: How-to guides
order: 0
---
To install borgmatic, first [install
Borg](https://borgbackup.readthedocs.io/en/stable/installation.html), at least
version 1.1. (borgmatic does not install Borg automatically so as to avoid
conflicts with existing Borg installations.)
Then, [install pipx](https://pypa.github.io/pipx/installation/) as the root
user (with `sudo`) to make installing borgmatic easier without impacting other
Python applications on your system. If you have trouble installing pipx with
pip, then you can install a system package instead. E.g. on Ubuntu or Debian,
run:
```bash
sudo apt update
sudo apt install pipx
```
### Root install
If you want to run borgmatic on a schedule with privileged access to your
files, then you should install borgmatic as the root user by running the
following commands:
```bash
sudo pipx ensurepath
sudo pipx install borgmatic
```
Check whether this worked with:
```bash
sudo su -
borgmatic --version
```
If borgmatic is properly installed, that should output your borgmatic version.
And if you'd also like `sudo borgmatic` to work, keep reading!
### Non-root install
If you only want to run borgmatic as a non-root user (without privileged file
access) *or* you want to make `sudo borgmatic` work so borgmatic runs as root,
then install borgmatic as a non-root user by running the following commands as
that user:
```bash
pipx ensurepath
pipx install borgmatic
```
This should work even if you've also installed borgmatic as the root user.
Check whether this worked with:
```bash
borgmatic --version
```
If borgmatic is properly installed, that should output your borgmatic version.
You can also try `sudo borgmatic --version` if you intend to run borgmatic
with `sudo`. If that doesn't work, you may need to update your [sudoers
`secure_path` option](https://wiki.archlinux.org/title/Sudo).
### Other ways to install
Besides the approaches described above, there are several other options for
installing borgmatic:
#### <span data-pagefind-weight="7.0">Docker / Podman</span>
* [container image with scheduled backups](https://github.com/borgmatic-collective/docker-borgmatic) (+ Docker Compose files)
* [container image with multi-arch and Docker CLI support](https://github.com/modem7/docker-borgmatic)
#### Operating system packages
* [Debian](https://tracker.debian.org/pkg/borgmatic)
* [Ubuntu](https://launchpad.net/ubuntu/+source/borgmatic)
* [Fedora](https://bodhi.fedoraproject.org/updates/?search=borgmatic)
* [Gentoo](https://packages.gentoo.org/packages/app-backup/borgmatic)
* [Arch Linux](https://archlinux.org/packages/extra/any/borgmatic/)
* [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=borgmatic)
* [OpenBSD](https://openports.pl/path/sysutils/borgmatic)
* [openSUSE](https://software.opensuse.org/package/borgmatic)
* [macOS (via Homebrew)](https://formulae.brew.sh/formula/borgmatic)
* [macOS (via MacPorts)](https://ports.macports.org/port/borgmatic/)
* [NixOS](https://search.nixos.org/packages?channel=unstable&show=borgmatic&query=borgmatic)
#### Etc.
* [Ansible role](https://github.com/borgbase/ansible-role-borgbackup)
* [uv tool install](https://docs.astral.sh/uv/)
Start by [installing
borgmatic](https://torsion.org/borgmatic/how-to/install-borgmatic/) if you haven't
already.
## Hosting providers
@@ -322,7 +232,7 @@ If you're using systemd instead of cron to run jobs, you can still configure
borgmatic to run automatically.
(If you installed borgmatic from [Other ways to
install](https://torsion.org/borgmatic/how-to/set-up-backups/#other-ways-to-install),
install](https://torsion.org/borgmatic/how-to/install-borgmatic/#other-ways-to-install),
you may already have borgmatic systemd service and timer files. If so, you may
be able to skip some of the steps below.)
+37 -8
View File
@@ -5,10 +5,25 @@ eleventyNavigation:
parent: How-to guides
order: 14
---
In general, all you should need to do to upgrade borgmatic if you've
[installed it with
pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation)
is to run the following:
In general, all you should need to do to upgrade borgmatic if you've [installed
it with uv](https://docs.astral.sh/uv/) is to run the following:
```bash
sudo uv tool upgrade borgmatic
```
Omit `sudo` if you installed borgmatic as a non-root user. And if you
installed borgmatic *both* as root and as a non-root user, you'll need to
upgrade each installation independently.
### Upgrading from other installation methods
#### pipx
If you have borgmatic installed with
[pipx](https://pipx.pypa.io/latest/installation/), and you'd like to continue
using pipx, then you can upgrade borgmatic with:
```bash
sudo pipx upgrade borgmatic
@@ -18,16 +33,30 @@ Omit `sudo` if you installed borgmatic as a non-root user. And if you
installed borgmatic *both* as root and as a non-root user, you'll need to
upgrade each installation independently.
But if you'd like to switch your borgmatic install from pipx to
[uv](https://docs.astral.sh/uv/), uninstall borgmatic with pipx (`sudo pipx
uninstall borgmatic`) and then [install borgmatic with
uv](https://torsion.org/borgmatic/how-to/install-borgmatic/).
Either one should work just fine. uv is just faster than pipx and also used for
borgmatic
[development](https://torsion.org/borgmatic/how-to/develop-on-borgmatic/).
#### pip install
If you originally installed borgmatic with `sudo pip3 install --user`, you can
uninstall it first with `sudo pip3 uninstall borgmatic` and then [install it
again with
pipx](https://torsion.org/borgmatic/how-to/set-up-backups/#installation),
uv](https://torsion.org/borgmatic/how-to/install-borgmatic/),
which should better isolate borgmatic from your other Python applications.
But if you [installed borgmatic without pipx or
pip3](https://torsion.org/borgmatic/how-to/set-up-backups/#other-ways-to-install),
then your upgrade method may be different.
#### Etc.
If you installed borgmatic [some other
way](https://torsion.org/borgmatic/how-to/install-borgmatic/#other-ways-to-install),
then your upgrade method may be different.
### Upgrading your configuration
@@ -0,0 +1,17 @@
---
title: browse
eleventyNavigation:
key: browse
parent: 🎬 Actions
---
<span class="minilink minilink-addedin">Experimental feature</span> {% include snippet/command-line/sample.md %}
```
{% include borgmatic/command-line/browse.txt %}
```
## Related documentation
* [Inspect your backups](https://torsion.org/borgmatic/how-to/inspect-your-backups/)
@@ -97,4 +97,3 @@ option for limiting the archives used for the `check` action was a separate
`prefix` in the `consistency` section. Both of these options are deprecated in
favor of the auto-matching behavior (or `match_archives`/`--match-archives`)
in newer versions of borgmatic.
@@ -43,14 +43,15 @@ postgresql_databases:
```
### Custom command
### Custom commands
You can also optionally override the `keepassxc-cli` command that borgmatic calls to load
passwords:
You can also optionally override the `keepassxc-cli` or `secret-tool` commands
that borgmatic call to load passwords:
```yaml
keepassxc:
keepassxc_cli_command: /usr/local/bin/keepassxc-cli
secret_tool_command: /usr/local/bin/secret-tool
```
Another example:
@@ -98,3 +99,30 @@ keepassxc:
The value here is the YubiKey slot number (e.g., `1` or `2`) and optional serial
number (e.g., `7370001`) used to access the KeePassXC database. Join the two
values with a colon, but omit the colon if you're leaving out the serial number.
### Secret service integration
<span class="minilink minilink-addedin">New in version 2.1.6</span> borgmatic
supports [KeePassXC's secret service
integration](https://keepassxc.org/docs/KeePassXC_UserGuide#_secret_service_integration)
that integrates with the [freedesktop secret service
API](https://specifications.freedesktop.org/secret-service/latest/) and allows
clients like borgmatic to access your passwords.
To use this feature from borgmatic, specify `secret-service` instead of a
KeePassXC database path when calling this credential hook. For instance:
```yaml
encryption_passphrase: "{credential keepassxc secret-service borgmatic}"
```
With this in place, borgmatic runs libsecret's `secret-tool` command to retrieve
the password titled "borgmatic" on demand. KeePassXC may then prompt you to
approve the password access, depending on how you've configured it.
One benefit of using the KeePassXC's secret service integration like this is
that you don't have to type a KeePassXC database passhprase (or use a keyfile)
whenever borgmatic accesses your passwords. Instead, you can configure
KeePassXC to prompt you to approve or deny each access. Or you can even
pre-approve password access to support automated borgmatic runs.
@@ -16,10 +16,51 @@ mariadb_databases:
```
### Full configuration
## System databases
<span class="minilink minilink-addedin">New in version 2.1.6</span> When dumping
["all"
databases](https://torsion.org/borgmatic/how-to/backup-your-databases/#all-databases),
borgmatic excludes most data coming from [MariaDB system
databases](https://mariadb.com/docs/server/reference/system-tables),
because much of it is populated on MariaDB startup and thus not restorable (or
just unnecessary to backup).
The system data that borgmatic does include in these dumps are: users, roles,
grants, user-defined functions, and remote servers—all from the `mysql` system
database. This omits all other data from the `mysql` database, which includes
index and table statistics, time zones, and installed server plugins. It also
excludes the separate `information_schema`, `performance_schema`, and `sys`
system databases. This is not currently configurable.
Within a Borg archive, you can find this data stored in a dump named `mysql`—the
name of the system table this data comes from. And if you'd like to dump this
data without having to dump "all" databases, then you can configure a database
named `mysql` in your borgmatic configuration. For example:
```yaml
mariadb_databases:
- name: mysql
```
Even in this case though, only the subset of system data described above is
included in the dump.
<span class="minilink minilink-addedin">Prior to version 2.1.6</span> Dumps of
"all" databases excluded system databases and all of their data. Additionally,
explicitly dumping `mysql` was treated like any other database—and thus wasn't
easily restorable.
## Full configuration
{% include snippet/configuration/sample.md %}
```yaml
{% include borgmatic/mariadb_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -14,10 +14,15 @@ mongodb_databases:
```
### Full configuration
## Full configuration
{% include snippet/configuration/sample.md %}
```yaml
{% include borgmatic/mongodb_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -21,3 +21,8 @@ mysql_databases:
```yaml
{% include borgmatic/mysql_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -13,6 +13,61 @@ postgresql_databases:
- name: users
```
See below for the full set of configuration options available, including
hostname, PostgreSQL username, password, etc.
## Permissions
### Dumping
In order to dump your database as part of creating a backup, the PostgreSQL user
performing the dump needs relevant permissions. A common way to accomplish this
is to connect as the PostgreSQL superuser, usually `postgres`. However, if you'd
like to connect as a non-superuser, that user will need permissions to:
* connect to the database
* read tables and sequences
Here is one way to do that with PostgreSQL 14+:
```sql
GRANT CONNECT ON DATABASE example_database TO database_user;
GRANT pg_read_all_data TO database_user;
```
And here is an alternate way to accomplish something similar that limits read access
to a particular schema instead of the whole cluster. Replace "public" with the
name of the schema you're using:
```sql
GRANT CONNECT ON DATABASE example_database TO database_user;
GRANT USAGE ON SCHEMA public TO database_user;
-- Grant read privileges on all current and future tables in the schema.
GRANT SELECT ON ALL TABLES IN SCHEMA public TO database_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO database_user;
-- Grant read privileges on all current and future indexes in the schema.
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO database_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT USAGE, SELECT ON SEQUENCES TO database_user;
```
### Restoring
If you also want this user to be able to restore your database, and you're not
restoring as the PostgreSQL superuser, then you'll need to grant write and
`ANALYZE` permissions as well. For instance:
```sql
GRANT pg_write_all_data TO database_user;
GRANT pg_maintain TO database_user;
```
Or you can perform schema-level grants if you prefer.
For more information, see the PostgreSQL documentation on [PostgreSQL predefined
roles](https://www.postgresql.org/docs/current/predefined-roles.html) and
[privileges](https://www.postgresql.org/docs/current/ddl-priv.html).
## Full configuration
@@ -21,3 +76,8 @@ postgresql_databases:
```yaml
{% include borgmatic/postgresql_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -22,3 +22,8 @@ sqlite_databases:
```yaml
{% include borgmatic/sqlite_databases.yaml %}
```
## Related documentation
* [How to backup your databases](https://torsion.org/borgmatic/how-to/backup-your-databases/)
@@ -5,25 +5,24 @@ eleventyNavigation:
parent: 🚨 Monitoring
---
<span class="minilink minilink-addedin">New in version 1.8.4</span>
[Apprise](https://github.com/caronc/apprise/wiki) is a local notification library
[Apprise](https://appriseit.com/) is a local notification library
that "allows you to send a notification to almost all of the most popular
[notification services](https://github.com/caronc/apprise/wiki) available to
[notification services](https://appriseit.com/services/) available to
us today such as: Telegram, Discord, Slack, Amazon SNS, Gotify, etc."
Depending on how you installed borgmatic, it may not have come with Apprise.
For instance, if you originally [installed borgmatic with
pipx](https://torsion.org/borgmatic/how-to/set-up-backups/),
uv](https://torsion.org/borgmatic/how-to/install-borgmatic/),
run the following to install Apprise so borgmatic can use it:
```bash
sudo pipx uninstall borgmatic
sudo pipx install borgmatic[Apprise]
sudo uv tool install borgmatic[Apprise]
```
Omit `sudo` if borgmatic is installed as a non-root user.
Once Apprise is installed, configure borgmatic to notify one or more [Apprise
services](https://github.com/caronc/apprise/wiki). For example:
services](https://appriseit.com/services/). For example:
```yaml
apprise:
@@ -89,3 +89,34 @@ for more information.
<span class="minilink minilink-addedin">New in version 2.0.0</span>Set the
defaults for these flags in your borgmatic configuration via the
`monitoring_verbosity`, `list`, and `statistics` options.
### Mutual TLS authentication
<span class="minilink minilink-addedin">New in version 2.1.5</span> Since Loki
does not come with a built-in authentication layer
[(doc)](https://grafana.com/docs/loki/latest/operations/authentication/), this
feature is typically used alongside a reverse proxy (such as
[nginx](https://docs.nginx.com/waf/configure/secure-mtls/) or
[Traefik](https://doc.traefik.io/traefik/reference/routing-configuration/http/tls/tls-options/#client-authentication-mtls))
that handles mTLS termination.
If your setup is configured for mTLS authentication, you can provide a client
certificate and private key:
```yaml
loki:
url: https://loki.fqdn/loki/api/v1/push
labels:
app: borgmatic
tls:
cert_path: /etc/borgmatic/loki-client.crt
key_path: /etc/borgmatic/loki-client.key
```
Both `cert_path` and `key_path` must be
[PEM-encoded](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail). They are
passed directly to the underlying HTTP client, so the standard mutual TLS
handshake is performed for every request borgmatic sends to Loki.
+40 -6
View File
@@ -1,14 +1,48 @@
---
title: Search the documentation
eleventyExcludeFromCollections: true
templateEngineOverride: md
head_additions: >
<link href="/borgmatic/pagefind/pagefind-component-ui.css" rel="stylesheet">
<script src="/borgmatic/pagefind/pagefind-component-ui.js" type="module"></script>
<style>
@media (prefers-color-scheme: light) {
:root {
--pf-summary-font-size: 14px;
--pf-result-title-font-size: 16px;
--pf-result-excerpt-font-size: 15px;
}
}
@media (prefers-color-scheme: dark) {
:root {
--pf-summary-font-size: 14px;
--pf-result-title-font-size: 16px;
--pf-result-excerpt-font-size: 15px;
}
}
.pf-result .pf-result-title, .pf-heading-link {
font-weight: 600 !important;
}
.pf-result .pf-result-excerpt mark, .pf-heading-excerpt mark {
background-color: yellow !important;
}
</style>
---
<link href="/borgmatic/pagefind/pagefind-ui.css" rel="stylesheet">
<script src="/borgmatic/pagefind/pagefind-ui.js"></script>
<div id="search"></div>
<script>
<pagefind-config bundle-path="/borgmatic/pagefind/" base-url="/borgmatic/"></pagefind-config>
<p><pagefind-input placeholder="Search"></pagefind-input></p>
<p><pagefind-summary></pagefind-summary></p>
<p><pagefind-results></pagefind-results></p>
<script type="module">
const manager = window.PagefindComponents.getInstanceManager();
const instance = manager.getInstance('default');
window.addEventListener('DOMContentLoaded', (event) => {
let search = new PagefindUI({ element: '#search', showSubResults: true, autofocus: true });
let url_parameters = new URLSearchParams(window.location.search);
search.triggerSearch(url_parameters.get('query'));
instance.triggerSearch(url_parameters.get('query'));
});
</script>
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

+8 -4
View File
@@ -1,17 +1,18 @@
[project]
name = "borgmatic"
version = "2.1.3"
version = "2.1.6"
authors = [
{ name="Dan Helfman", email="witten@torsion.org" },
]
description = "Simple, configuration-driven backup software for servers and workstations"
license = "GPL-3.0-or-later"
license-files = ["LICENSE"]
readme = "README.md"
requires-python = ">=3.9"
classifiers=[
"Development Status :: 5 - Production/Stable",
"Environment :: Console",
"Intended Audience :: System Administrators",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"Programming Language :: Python",
"Topic :: Security :: Cryptography",
"Topic :: System :: Archiving :: Backup",
@@ -30,6 +31,8 @@ validate-borgmatic-config = "borgmatic.commands.validate_config:main"
[project.optional-dependencies]
Apprise = ["apprise"]
browse = ["textual", "binaryornot"]
browse-dev = ["textual-dev"]
[project.urls]
Homepage = "https://torsion.org/borgmatic"
@@ -44,7 +47,7 @@ namespaces = false
[tool.pytest.ini_options]
testpaths = "tests"
addopts = "--cov-report term-missing:skip-covered --cov=borgmatic --no-cov-on-fail --cov-fail-under=100 --ignore=tests/end-to-end"
addopts = "--cov-report term-missing:skip-covered --cov=borgmatic --no-cov-on-fail --cov-fail-under=100 --ignore=tests/end-to-end --timeout=120 --asyncio-mode=auto"
[tool.ruff]
line-length = 100
@@ -54,7 +57,7 @@ exclude = ["*.*/*"]
quote-style = "preserve"
[tool.ruff.lint]
preview = true
preview = false
extend-select = [
"A", # flake8-builtins: builtin shadowing
"B", # flake8-bugbear: bugs and design problems
@@ -100,6 +103,7 @@ ignore = [
"S404", # subprocess import
"SIM115", # open() without context manager
"SIM905", # split() on literal string
"TRY004", # type check and raise something other than TypeError
]
[tool.ruff.lint.flake8-quotes]
+24
View File
@@ -43,6 +43,20 @@ twine upload -r pypi --username __token__ "$wheel_path"
scripts/export-docs-from-image
docs_path=dist/borgmatic-docs.tar.gz
# Build stand-alone binaries.
uv venv --python 3.13 --clear binary
source binary/bin/activate
uv pip install -r binary_requirements.txt nuitka[onefile]
nuitka --mode=onefile --enable-plugin=upx --include-package-data=borgmatic --include-data-dir=borgmatic.egg-info=borgmatic.egg-info --include-package=borgmatic.hooks --include-package=apprise --no-deployment-flag=self-execution borgmatic/commands/borgmatic.py
rm -fr borgmatic.build borgmatic.dist
standalone_binary_path="dist/borgmatic-${version}-binary-linux-glibc-x86_64"
mv borgmatic.bin "$standalone_binary_path"
nuitka --mode=onefile --enable-plugin=upx --include-package-data=borgmatic --include-data-dir=borgmatic.egg-info=borgmatic.egg-info --include-package=borgmatic.hooks --no-deployment-flag=self-execution borgmatic/commands/borgmatic.py
rm -fr binary borgmatic.build borgmatic.dist
deactivate
standalone_minimal_binary_path="dist/borgmatic-${version}-minimal-binary-linux-glibc-x86_64"
mv borgmatic.bin "$standalone_minimal_binary_path"
# Set release changelogs on projects.torsion.org and GitHub.
release_changelog="$(cat NEWS | sed '/^$/q' | grep -v '^\S')"
escaped_release_changelog="$(echo "$release_changelog" | sed -z 's/\n/\\n/g' | sed -z 's/\"/\\"/g')"
@@ -68,6 +82,16 @@ curl --silent --request POST \
--header "Authorization: token $projects_token" \
--header "Accept: application/json" \
--form attachment=@"$docs_path"
curl --silent --request POST \
"https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/releases/$release_id/assets?name=$(basename $standalone_binary_path)" \
--header "Authorization: token $projects_token" \
--header "Accept: application/json" \
--form attachment=@"$standalone_binary_path"
curl --silent --request POST \
"https://projects.torsion.org/api/v1/repos/borgmatic-collective/borgmatic/releases/$release_id/assets?name=$(basename $standalone_minimal_binary_path)" \
--header "Authorization: token $projects_token" \
--header "Accept: application/json" \
--form attachment=@"$standalone_minimal_binary_path"
github-release create --token="$github_token" --owner=witten --repo=borgmatic --tag="$version" --target_commit="main" \
--name="borgmatic $version" --body="$release_changelog"
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
set -eou pipefail
uv pip compile --quiet --upgrade --annotation-style line test_requirements.in -o test_requirements.txt
uv pip compile --quiet --upgrade --annotation-style line binary_requirements.in -o binary_requirements.txt
# Carry forward certain packages with version ranges instead of absolute pins. This is because these
# packages are a pain to compile, and pre-built wheels available on pypi may not be available for
# pinned versions on particular platforms.
while IFS= read -r package_line; do
package_name=$(echo "$package_line" | cut -d ">" -f 1)
sed -i "s/^$package_name==.*/$package_line/" test_requirements.txt
done <<< $(grep ">" test_requirements.in)
sed -i "s/was autogenerated/was (mostly) autogenerated/" test_requirements.txt
+32
View File
@@ -0,0 +1,32 @@
apprise
attrs
binaryornot
certifi
charset-normalizer
click>=8.1.8
codespell
coverage
flexmock
idna
iniconfig
jsonschema
jsonschema-specifications
markdown
oauthlib
packaging
pluggy
pygments
pytest
pytest-asyncio
pytest-cov
pytest-timeout
pyyaml>5.0.0
referencing
requests
requests-oauthlib
rich
rpds-py
ruamel-yaml>0.15.0
textual
typing-extensions
urllib3
+37 -24
View File
@@ -1,27 +1,40 @@
apprise==1.9.5
attrs==25.4.0
certifi==2025.10.5
charset-normalizer==3.4.4
# This file was (mostly) autogenerated by uv via the following command:
# uv pip compile --annotation-style line test_requirements.in -o test_requirements.txt
apprise==1.10.0 # via -r test_requirements.in
attrs==26.1.0 # via jsonschema, referencing, -r test_requirements.in
binaryornot==0.6.0 # via -r test_requirements.in
certifi==2026.5.20 # via apprise, requests, -r test_requirements.in
charset-normalizer==3.4.7 # via requests, -r test_requirements.in
click>=8.1.8
codespell==2.4.1
coverage==7.11.0
flexmock==0.12.2
idna==3.11
iniconfig==2.3.0
jsonschema==4.25.1
jsonschema-specifications==2025.9.1
markdown==3.9
oauthlib==3.3.1
packaging==25.0
pluggy==1.6.0
pygments==2.19.2
pytest==8.4.2
pytest-cov==7.0.0
codespell==2.4.2 # via -r test_requirements.in
coverage==7.14.0 # via pytest-cov, -r test_requirements.in
flexmock==0.13.0 # via -r test_requirements.in
idna==3.16 # via requests, -r test_requirements.in
iniconfig==2.3.0 # via pytest, -r test_requirements.in
jsonschema==4.26.0 # via -r test_requirements.in
jsonschema-specifications==2025.9.1 # via jsonschema, -r test_requirements.in
linkify-it-py==2.1.0 # via markdown-it-py
markdown==3.10.2 # via apprise, -r test_requirements.in
markdown-it-py==4.2.0 # via mdit-py-plugins, rich, textual
mdit-py-plugins==0.6.1 # via textual
mdurl==0.1.2 # via markdown-it-py
oauthlib==3.3.1 # via requests-oauthlib, -r test_requirements.in
packaging==26.2 # via pytest, -r test_requirements.in
platformdirs==4.9.6 # via textual
pluggy==1.6.0 # via pytest, pytest-cov, -r test_requirements.in
pygments==2.20.0 # via pytest, rich, textual, -r test_requirements.in
pytest==9.0.3 # via pytest-asyncio, pytest-cov, pytest-timeout, -r test_requirements.in
pytest-asyncio==1.3.0 # via -r test_requirements.in
pytest-cov==7.1.0 # via -r test_requirements.in
pytest-timeout==2.4.0 # via -r test_requirements.in
pyyaml>5.0.0
referencing==0.37.0
requests==2.32.5
requests-oauthlib==2.0.0
rpds-py==0.28.0
referencing==0.37.0 # via jsonschema, jsonschema-specifications, -r test_requirements.in
requests==2.34.2 # via apprise, requests-oauthlib, -r test_requirements.in
requests-oauthlib==2.0.0 # via apprise, -r test_requirements.in
rich==15.0.0 # via textual, -r test_requirements.in
rpds-py==0.30.0 # via jsonschema, referencing, -r test_requirements.in
ruamel-yaml>0.15.0
typing-extensions==4.15.0
urllib3==2.6.3
textual==8.2.7 # via -r test_requirements.in
typing-extensions==4.15.0 # via textual, -r test_requirements.in
uc-micro-py==2.0.0 # via linkify-it-py
urllib3==2.7.0 # via requests, -r test_requirements.in
+1 -1
View File
@@ -44,7 +44,7 @@ services:
MONGO_INITDB_ROOT_PASSWORD: test2
command: docker-entrypoint.sh --port=27018
tests:
image: docker.io/alpine:3.22
image: docker.io/alpine:3.23
environment:
TEST_CONTAINER: true
volumes:
@@ -57,11 +57,15 @@ mariadb_databases:
hostname: mariadb
username: root
password: test
# Needed for restoring users to the system database, because we're not actually dropping
# users first. (Dropping root and mariadb.sys would cause problems.)
restore_options: --force
- name: all
{mariadb_mysql_dump_format_option}
hostname: mariadb
username: root
password: test
restore_options: --force
mysql_databases:
- name: test
hostname: not-actually-mysql
@@ -132,6 +136,7 @@ mariadb_databases:
restore_port: 3307
restore_username: root
restore_password: test2
restore_options: --force
mysql_databases:
- name: test
container: not-actually-mysql
@@ -201,6 +206,7 @@ mariadb_databases:
restore_port: 3307
restore_username: root
restore_password: test2
restore_options: --force
mysql_databases:
- name: test
hostname: not-actually-mysql
@@ -69,6 +69,7 @@ class Background_web_server:
self.thread.join()
FINISH = 1
START_AND_FINISH = 2
START_LOG_AND_FINISH = 3
@@ -94,7 +95,7 @@ START_LOG_AND_FINISH = 3
),
(
'loki:\n url: http://localhost:12345/loki/api/v1/push\n labels:\n app: borgmatic\n send_logs: true',
START_AND_FINISH,
FINISH,
),
(
'loki:\n url: http://localhost:12345/loki/api/v1/push\n labels:\n app: borgmatic',
@@ -0,0 +1,68 @@
from flexmock import flexmock
import borgmatic.actions.browse.app
import borgmatic.actions.browse.configuration_files_list
import borgmatic.actions.browse.logs
import borgmatic.actions.browse.repositories_list
async def test_browse_app_with_multiple_configs_uses_configuration_files_list():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test():
app.query_one(selector='Header')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].configs == app.configs
app.query_one(selector='Logs')
app.query_one(selector='Footer')
async def test_browse_app_with_one_config_uses_repositories_list():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test():
app.query_one(selector='Header')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[0].config == app.configs['test1.yaml']
app.query_one(selector='Logs')
app.query_one(selector='Footer')
async def test_browse_app_key_toggles_logs_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
logs_panel = app.query_one('#logs')
await pilot.press('v')
await pilot.pause()
assert logs_panel.styles.display == 'block'
@@ -0,0 +1,86 @@
import textual.app
import textual.widgets.option_list
from flexmock import flexmock
from borgmatic.actions.browse import archives_list as module
async def test_archives_list_on_mount_does_not_raise():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.workers).should_receive('add_repository_archives')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
flexmock(archives_list.archive_loaded).should_receive('subscribe')
async with textual.app.App().run_test():
archives_list.on_mount()
def test_archives_list_on_archive_loaded_with_loading_done_removes_loading_indicator():
loading_timer = flexmock()
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).and_return(loading_timer)
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
flexmock(loading_timer).should_receive('stop')
flexmock(archives_list).should_receive('remove_option').with_args('loading-indicator').once()
flexmock(archives_list).should_receive('add_options').never()
archives_list.on_archive_loaded(module.borgmatic.actions.browse.workers.LOADING_DONE)
def test_archives_list_on_archive_loaded_adds_archive_name():
loading_timer = flexmock()
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).and_return(loading_timer)
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
flexmock(loading_timer).should_receive('stop').never()
loading_indicator = flexmock()
flexmock(archives_list).should_receive('get_option').and_return(loading_indicator)
flexmock(archives_list).should_receive('remove_option').with_args('loading-indicator').once()
flexmock(archives_list).should_receive('add_options').once()
archives_list.on_archive_loaded('archive')
def test_archives_list_on_option_list_option_highlighted_with_highlighted_none_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
archives_list.highlighted = None
archives_list.on_option_list_option_highlighted(event=flexmock())
assert archives_list.highlighted_option_changed is False
def test_archives_list_on_option_list_option_highlighted_with_highlighted_zero_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
archives_list.highlighted = 0
archives_list.on_option_list_option_highlighted(event=flexmock())
assert archives_list.highlighted_option_changed is False
def test_archives_list_on_option_list_option_highlighted_with_existing_option_and_highlighted_zero_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
archives_list.add_option(textual.widgets.option_list.Option('zero', id='zero'))
archives_list.highlighted = 0
archives_list.on_option_list_option_highlighted(event=flexmock())
assert archives_list.highlighted_option_changed is False
def test_archives_list_on_option_list_option_highlighted_with_highlighted_non_zero_marks_it_changed():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
archives_list = module.Archives_list(config=flexmock(), repository=flexmock())
archives_list.add_option(textual.widgets.option_list.Option('zero', id='zero'))
archives_list.add_option(textual.widgets.option_list.Option('one', id='one'))
archives_list.highlighted = 1
archives_list.on_option_list_option_highlighted(event=flexmock())
assert archives_list.highlighted_option_changed is True
@@ -0,0 +1,522 @@
import pytest
import textual.widgets.option_list
from flexmock import flexmock
import borgmatic.actions.browse.app
import borgmatic.actions.browse.archive
import borgmatic.actions.browse.carousel
import borgmatic.actions.browse.loading
import borgmatic.actions.browse.logs
import borgmatic.actions.browse.workers
from borgmatic.actions.browse import carousel as module
def test_make_next_panel_with_configuration_files_list_returns_repositories_list():
configs = {'test.yaml': {'repositories': [{'path': 'test.borg'}]}}
repositories_list = module.make_next_panel(
focused_panel=borgmatic.actions.browse.configuration_files_list.Configuration_files_list(
configs
),
option_id='test.yaml',
)
assert isinstance(
repositories_list, borgmatic.actions.browse.repositories_list.Repositories_list
)
assert repositories_list.config == configs['test.yaml']
def test_make_next_panel_with_repositories_list_returns_archives_list():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('add_repository_archives')
flexmock(borgmatic.actions.browse.archives_list.Archives_list).should_receive('app').and_return(
flexmock()
)
archives_list = module.make_next_panel(
focused_panel=borgmatic.actions.browse.repositories_list.Repositories_list(config),
option_id=0,
)
assert isinstance(archives_list, borgmatic.actions.browse.archives_list.Archives_list)
assert archives_list.config == config
assert archives_list.repository == config['repositories'][0]
def test_make_next_panel_with_archives_list_returns_directory_list():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('add_repository_archives')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.archives_list.Archives_list).should_receive('app').and_return(
flexmock()
)
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.make_next_panel(
focused_panel=borgmatic.actions.browse.archives_list.Archives_list(
config, config['repositories'][0]
),
option_id='archive',
)
assert isinstance(directory_list, borgmatic.actions.browse.directory_list.Directory_list)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
def test_make_next_panel_with_root_directory_list_and_selected_directory_option_returns_new_directory_list():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.workers).should_receive('Archive_path_loaded').replace_with(
flexmock(
path_hierarchy={
'etc': {},
},
complete=False,
),
)
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config, config['repositories'][0], 'archive'
)
flexmock(focused_panel).should_receive('get_option').and_return(flexmock(prompt='📁 etc'))
directory_list = module.make_next_panel(focused_panel=focused_panel, option_id='etc')
assert isinstance(directory_list, borgmatic.actions.browse.directory_list.Directory_list)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
assert directory_list.archive_name == 'archive'
assert directory_list.path_components == ('etc',)
def test_make_next_panel_with_non_root_directory_list_and_selected_directory_option_returns_new_directory_list():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.workers).should_receive('Archive_path_loaded').replace_with(
flexmock(
path_hierarchy={
'etc': {
'borgmatic': {},
},
},
complete=False,
),
)
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config,
config['repositories'][0],
'archive',
path_components=('etc',),
)
flexmock(focused_panel).should_receive('get_option').and_return(flexmock(prompt='📁 borgmatic'))
directory_list = module.make_next_panel(focused_panel=focused_panel, option_id='borgmatic')
assert isinstance(directory_list, borgmatic.actions.browse.directory_list.Directory_list)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
assert directory_list.archive_name == 'archive'
assert directory_list.path_components == ('etc', 'borgmatic')
def test_make_next_panel_with_root_directory_list_and_selected_file_option_returns_new_file_preview():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.workers).should_receive('load_file_preview')
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(borgmatic.actions.browse.file_preview.File_preview).should_receive('app').and_return(
flexmock()
)
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config, config['repositories'][0], 'archive'
)
flexmock(focused_panel).should_receive('get_option').and_return(
flexmock(prompt='📄 config.yaml')
)
directory_list = module.make_next_panel(focused_panel=focused_panel, option_id='config.yaml')
assert isinstance(directory_list, borgmatic.actions.browse.file_preview.File_preview)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
assert directory_list.archive_name == 'archive'
assert directory_list.file_path == 'config.yaml'
def test_make_next_panel_with_non_root_directory_list_and_selected_file_option_returns_new_file_preview():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.workers).should_receive('load_file_preview')
flexmock(borgmatic.actions.browse.workers).should_receive('Archive_path_loaded').replace_with(
flexmock(
path_hierarchy={
'etc': {
'borgmatic': {
'config.yaml': borgmatic.actions.browse.archive.Archive_path(
'-', 'config.yaml', ''
),
},
},
},
complete=False,
),
)
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(borgmatic.actions.browse.file_preview.File_preview).should_receive('app').and_return(
flexmock()
)
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config,
config['repositories'][0],
'archive',
path_components=('etc', 'borgmatic'),
)
flexmock(focused_panel).should_receive('get_option').and_return(
flexmock(prompt='📄 config.yaml')
)
directory_list = module.make_next_panel(focused_panel=focused_panel, option_id='config.yaml')
assert isinstance(directory_list, borgmatic.actions.browse.file_preview.File_preview)
assert directory_list.config == config
assert directory_list.repository == config['repositories'][0]
assert directory_list.archive_name == 'archive'
assert directory_list.file_path == 'etc/borgmatic/config.yaml'
def test_make_next_panel_with_unsupported_focused_panel_returns_none():
assert module.make_next_panel(focused_panel=flexmock(), option_id='hmmm') is None
@pytest.mark.parametrize('icon', ('🔗', '🚰', '🐙'))
def test_make_next_panel_with_directory_list_and_unsupported_selected_option_returns_none(icon):
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(borgmatic.actions.browse.workers).should_receive('load_archive_paths')
flexmock(borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
focused_panel = borgmatic.actions.browse.directory_list.Directory_list(
config, config['repositories'][0], 'archive'
)
flexmock(focused_panel).should_receive('get_option').and_return(
flexmock(prompt=f'{icon} config.yaml')
)
assert module.make_next_panel(focused_panel=focused_panel, option_id='config.yaml') is None
async def test_carousel_previous_action_with_multiple_configs_does_not_raise():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('left')
async def test_carousel_previous_action_with_one_config_does_not_raise():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('left')
async def test_carousel_next_action_with_multiple_configs_advances_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'none'
assert isinstance(
carousel.panels[1], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[1].styles.display == 'block'
assert carousel.panels[1].highlighted == 0
assert app.focused == carousel.panels[1]
async def test_carousel_next_action_with_one_config_advances_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
flexmock(borgmatic.actions.browse.workers).should_receive('add_repository_archives')
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[0].styles.display == 'none'
assert isinstance(carousel.panels[1], borgmatic.actions.browse.archives_list.Archives_list)
assert carousel.panels[1].styles.display == 'block'
assert carousel.panels[1].highlighted == 0
assert app.focused == carousel.panels[1]
async def test_carousel_next_action_with_no_next_panel_does_not_advance():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
flexmock(borgmatic.actions.browse.workers).should_receive('add_repository_archives')
flexmock(module).should_receive('make_next_panel')
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[0].styles.display == 'block'
assert app.focused == carousel.panels[0]
async def test_carousel_next_action_and_previous_action_returns_to_original_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('enter')
await pilot.press('left')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 0
assert isinstance(
carousel.panels[1], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[1].styles.display == 'none'
assert app.focused == carousel.panels[0]
async def test_carousel_next_action_and_previous_action_and_next_action_reuses_next_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
flexmock(borgmatic.actions.browse.carousel).should_call('make_next_panel').once()
async with app.run_test() as pilot:
await pilot.press('enter')
await pilot.press('left')
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'none'
assert isinstance(
carousel.panels[1], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[1].styles.display == 'block'
assert carousel.panels[1].highlighted == 0
assert app.focused == carousel.panels[1]
async def test_carousel_next_action_with_multiple_configs_and_no_next_panel_does_not_advance():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
flexmock(module).should_receive('make_next_panel').and_return(None)
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 0
assert app.focused == carousel.panels[0]
async def test_carousel_next_action_and_previous_action_and_down_truncates_next_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('enter')
await pilot.press('left')
await pilot.press('down')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 1
assert app.focused == carousel.panels[0]
async def test_carousel_down_does_not_raise():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('down')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 1
assert app.focused == carousel.panels[0]
async def test_carousel_up_does_not_raise():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('down')
carousel = app.query_one(selector='Carousel')
assert len(carousel.panels) == 1
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 1
assert app.focused == carousel.panels[0]
async def test_carousel_next_action_and_select_dot_dot_returns_to_original_panel():
app = borgmatic.actions.browse.app.Browse_app(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
flexmock(borgmatic.actions.browse.logs).should_receive('log_to_widget')
async with app.run_test() as pilot:
await pilot.press('enter')
carousel = app.query_one(selector='Carousel')
carousel.panels[1].options[0] = textual.widgets.option_list.Option('..', id='..')
await pilot.press('enter')
assert len(carousel.panels) == 2
assert isinstance(
carousel.panels[0],
borgmatic.actions.browse.configuration_files_list.Configuration_files_list,
)
assert carousel.panels[0].styles.display == 'block'
assert carousel.panels[0].highlighted == 0
assert isinstance(
carousel.panels[1], borgmatic.actions.browse.repositories_list.Repositories_list
)
assert carousel.panels[1].styles.display == 'none'
assert app.focused == carousel.panels[0]
@@ -0,0 +1,34 @@
from flexmock import flexmock
from borgmatic.actions.browse import configuration_files_list as module
def test_configuration_files_list_adds_config_paths_as_options():
flexmock(module.os.path).should_receive('expanduser').and_return('/home/user')
configuration_files_list = module.Configuration_files_list(
configs={
'test1.yaml': {'repositories': [{'path': 'test1.borg'}]},
'test2.yaml': {'repositories': [{'path': 'test2.borg'}]},
}
)
assert len(configuration_files_list.options) == 2
assert configuration_files_list.options[0].prompt == 'test1.yaml'
assert configuration_files_list.options[0].id == 'test1.yaml'
assert configuration_files_list.options[1].prompt == 'test2.yaml'
assert configuration_files_list.options[1].id == 'test2.yaml'
def test_configuration_files_list_collapses_home_directory_in_config_path_option():
flexmock(module.os.path).should_receive('expanduser').and_return('/home/user')
configuration_files_list = module.Configuration_files_list(
configs={
'/home/user/test.yaml': {'repositories': [{'path': '/home/user/test.borg'}]},
}
)
assert len(configuration_files_list.options) == 1
assert configuration_files_list.options[0].prompt == '~/test.yaml'
assert configuration_files_list.options[0].id == '/home/user/test.yaml'
@@ -0,0 +1,287 @@
import textual.widgets
import textual.widgets.option_list
from flexmock import flexmock
from borgmatic.actions.browse import directory_list as module
def test_add_archive_paths_with_only_duplicate_paths_bails():
directory_list = textual.widgets.OptionList()
directory_list.path_components = ('etc',)
directory_list.add_option(textual.widgets.option_list.Option('foo', id='foo'))
directory_list.add_option(textual.widgets.option_list.Option('bar', id='bar'))
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(directory_list).should_receive('set_options').never()
module.add_archive_paths(
directory_list=directory_list,
config=config,
repository=config['repositories'][0],
archive_name='archive',
archive_paths=(
flexmock(path_type='-', file_path='etc/foo/one.txt', link_target=''),
flexmock(path_type='-', file_path='etc/foo/two.txt', link_target=''),
),
)
def test_add_archive_paths_adds_ands_sorts_and_filters_and_deduplicates():
directory_list = textual.widgets.OptionList()
directory_list.path_components = ('etc',)
directory_list.add_option(textual.widgets.option_list.Option('📄 foo', id='foo'))
directory_list.add_option(textual.widgets.option_list.Option('📄 bar', id='bar'))
directory_list.highlighted = 1
directory_list.highlighted_option_changed = True
config = {'repositories': [{'path': 'test.borg'}]}
module.add_archive_paths(
directory_list=directory_list,
config=config,
repository=config['repositories'][0],
archive_name='archive',
archive_paths=(
flexmock(path_type='d', file_path='etc/quux', link_target=''),
flexmock(path_type='-', file_path='etc/foo', link_target=''),
flexmock(path_type='-', file_path='etc/baz', link_target=''),
flexmock(path_type='d', file_path='root/nope', link_target=''),
flexmock(path_type='d', file_path='etc/other', link_target=''),
),
)
assert len(directory_list.options) == 5
assert directory_list.options[0].prompt == '📁 other'
assert directory_list.options[0].id == 'other'
assert directory_list.options[1].prompt == '📁 quux'
assert directory_list.options[1].id == 'quux'
assert directory_list.options[2].prompt == '📄 bar'
assert directory_list.options[2].id == 'bar'
assert directory_list.options[3].prompt == '📄 baz'
assert directory_list.options[3].id == 'baz'
assert directory_list.options[4].prompt == '📄 foo'
assert directory_list.options[4].id == 'foo'
assert directory_list.highlighted == 2
def test_add_archive_paths_highlights_first_option_if_highlight_has_not_changed():
directory_list = textual.widgets.OptionList()
directory_list.path_components = ('etc',)
directory_list.add_option(textual.widgets.option_list.Option('📄 foo', id='foo'))
directory_list.add_option(textual.widgets.option_list.Option('📄 bar', id='bar'))
directory_list.highlighted = None
directory_list.highlighted_option_changed = False
config = {'repositories': [{'path': 'test.borg'}]}
module.add_archive_paths(
directory_list=directory_list,
config=config,
repository=config['repositories'][0],
archive_name='archive',
archive_paths=(flexmock(path_type='-', file_path='etc/baz', link_target=''),),
)
assert len(directory_list.options) == 3
assert directory_list.options[0].prompt == '📄 bar'
assert directory_list.options[0].id == 'bar'
assert directory_list.options[1].prompt == '📄 baz'
assert directory_list.options[1].id == 'baz'
assert directory_list.options[2].prompt == '📄 foo'
assert directory_list.options[2].id == 'foo'
assert directory_list.highlighted == 0
def test_add_archive_paths_retains_loading_indicator_at_bottom():
directory_list = textual.widgets.OptionList()
directory_list.path_components = ('etc',)
directory_list.add_option(textual.widgets.option_list.Option('📄 foo', id='foo'))
directory_list.add_option(textual.widgets.option_list.Option('📄 bar', id='bar'))
directory_list.add_option(
textual.widgets.option_list.Option('loading!!!', id='loading-indicator')
)
directory_list.highlighted = 0
directory_list.highlighted_option_changed = True
config = {'repositories': [{'path': 'test.borg'}]}
module.add_archive_paths(
directory_list=directory_list,
config=config,
repository=config['repositories'][0],
archive_name='archive',
archive_paths=(flexmock(path_type='-', file_path='etc/baz', link_target=''),),
)
assert len(directory_list.options) == 4
assert directory_list.options[0].prompt == '📄 bar'
assert directory_list.options[0].id == 'bar'
assert directory_list.options[1].prompt == '📄 baz'
assert directory_list.options[1].id == 'baz'
assert directory_list.options[2].prompt == '📄 foo'
assert directory_list.options[2].id == 'foo'
assert directory_list.options[3].prompt == 'loading!!!'
assert directory_list.options[3].id == 'loading-indicator'
assert directory_list.highlighted == 2
def test_directory_list_with_root_directory_adds_loading_indicator():
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).once()
directory_list = module.Directory_list(
config=flexmock(), repository=flexmock(), archive_name='archive'
)
assert directory_list.border_title == '📁 archive'
assert len(directory_list.options) == 0
assert not directory_list.path_loaded.complete
def test_directory_list_with_already_complete_loading_skips_loading_indicator():
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).never()
flexmock(module.borgmatic.actions.browse.workers).should_receive('load_archive_paths').never()
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=True),
path_components=('etc',),
)
assert directory_list.border_title == '📁 etc'
assert len(directory_list.options) == 1
assert directory_list.options[0].prompt == '📁 ..'
assert directory_list.options[0].id == '..'
def test_directory_list_on_mount_with_root_directory_loads_archive_paths():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(module.borgmatic.actions.browse.workers).should_receive('load_archive_paths').once()
flexmock(module).should_receive('add_archive_paths').never()
directory_list = module.Directory_list(
config=flexmock(), repository=flexmock(), archive_name='archive'
)
flexmock(directory_list.path_loaded).should_receive('subscribe')
directory_list.on_mount()
def test_directory_list_on_mount_with_non_root_directory_adds_archive_paths():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(module.borgmatic.actions.browse.workers).should_receive('load_archive_paths').never()
flexmock(module).should_receive('add_archive_paths').once()
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
flexmock(directory_list.path_loaded).should_receive('subscribe')
directory_list.on_mount()
def test_on_archive_path_loaded_with_loading_done_signal_removes_loading_indicator():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(module).should_receive('add_archive_paths').never()
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
directory_list.timer = flexmock(stop=lambda: None)
flexmock(directory_list).should_receive('remove_option').once()
directory_list.on_archive_path_loaded(data=module.borgmatic.actions.browse.workers.LOADING_DONE)
def test_on_archive_path_loaded_with_path_loaded_signal_adds_archive_path():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
flexmock(module).should_receive('add_archive_paths').once()
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
flexmock(directory_list).should_receive('remove_option').never()
directory_list.on_archive_path_loaded(data=flexmock())
def test_directory_list_on_option_list_option_highlighted_with_highlighted_none_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
directory_list.highlighted = None
directory_list.on_option_list_option_highlighted(event=flexmock())
assert directory_list.highlighted_option_changed is False
def test_directory_list_on_option_list_option_highlighted_with_highlighted_zero_marks_it_unchanged():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
directory_list.add_option(textual.widgets.option_list.Option('zero', id='zero'))
directory_list.highlighted = 0
directory_list.on_option_list_option_highlighted(event=flexmock())
assert directory_list.highlighted_option_changed is False
def test_directory_list_on_option_list_option_highlighted_with_highlighted_non_zero_marks_it_changed():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.directory_list.Directory_list).should_receive(
'app'
).and_return(flexmock())
directory_list = module.Directory_list(
config=flexmock(),
repository=flexmock(),
archive_name='archive',
path_loaded=flexmock(complete=False, path_hierarchy={'etc': {}}),
path_components=('etc',),
)
directory_list.add_option(textual.widgets.option_list.Option('zero', id='zero'))
directory_list.add_option(textual.widgets.option_list.Option('one', id='one'))
directory_list.highlighted = 1
directory_list.on_option_list_option_highlighted(event=flexmock())
assert directory_list.highlighted_option_changed is True
@@ -0,0 +1,53 @@
import textual.widgets.option_list
from flexmock import flexmock
from borgmatic.actions.browse import file_preview as module
def test_file_preview_does_not_raise():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
module.File_preview(
config=flexmock(), repository=flexmock(), archive_name='archive', file_path='foo/bar.txt'
)
async def test_file_preview_on_mount_does_not_raise():
flexmock(module.borgmatic.actions.browse.loading).should_receive('add_inline_loading_indicator')
flexmock(module.borgmatic.actions.browse.workers).should_receive('load_file_preview')
file_preview = module.File_preview(
config=flexmock(), repository=flexmock(), archive_name='archive', file_path='foo.txt'
)
flexmock(file_preview.file_preview_loaded).should_receive('subscribe')
async with textual.app.App().run_test():
file_preview.on_mount()
def test_file_preview_on_file_preview_loaded_with_none_file_contents_displays_error():
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).and_return(flexmock(stop=lambda: None))
file_preview = module.File_preview(
config=flexmock(), repository=flexmock(), archive_name='archive', file_path='foo.txt'
)
flexmock(file_preview).should_receive('write').with_args(
'Cannot display a preview for this file'
).once()
file_preview.on_file_preview_loaded(None)
def test_file_preview_on_file_preview_loaded_with_file_contents_displays_contents():
flexmock(module.borgmatic.actions.browse.loading).should_receive(
'add_inline_loading_indicator'
).and_return(flexmock(stop=lambda: None))
file_preview = module.File_preview(
config=flexmock(), repository=flexmock(), archive_name='archive', file_path='foo.txt'
)
flexmock(file_preview).should_receive('write').with_args(
'Cannot display a preview for this file'
).never()
flexmock(file_preview).should_receive('write').once()
file_preview.on_file_preview_loaded('hi')
@@ -0,0 +1,97 @@
import pytest
import textual.app
import textual.widgets
from flexmock import flexmock
from borgmatic.actions.browse import loading as module
def test_update_inline_loading_indicator_with_option_list_adds_a_dot():
widget = textual.widgets.OptionList()
widget.add_option(textual.widgets.option_list.Option('HOLD.', id='loading-indicator'))
module.update_inline_loading_indicator(widget)
assert len(widget.options) == 1
assert widget.options[0].prompt == 'HOLD..'
def test_update_inline_loading_indicator_with_option_list_wraps_dots_beyond_three():
widget = textual.widgets.OptionList()
widget.add_option(textual.widgets.option_list.Option('HOLD...', id='loading-indicator'))
module.update_inline_loading_indicator(widget)
assert len(widget.options) == 1
assert widget.options[0].prompt == 'HOLD'
def test_update_inline_loading_indicator_with_option_list_and_missing_indicator_does_not_raise():
widget = textual.widgets.OptionList()
module.update_inline_loading_indicator(widget)
assert len(widget.options) == 0
async def test_update_inline_loading_indicator_with_rich_log_adds_a_dot():
async with textual.app.App().run_test():
widget = textual.widgets.RichLog()
widget._size_known = True
widget.write('HOLD.')
module.update_inline_loading_indicator(widget)
assert str(widget.lines[0].text) == 'HOLD..'
async def test_update_inline_loading_indicator_with_rich_log_wraps_dots_beyond_three():
async with textual.app.App().run_test():
widget = textual.widgets.RichLog()
widget._size_known = True
widget.write('HOLD...')
module.update_inline_loading_indicator(widget)
assert str(widget.lines[0].text) == 'HOLD'
def test_update_inline_loading_indicator_with_rich_log_and_missing_indicator_does_not_raise():
widget = textual.widgets.RichLog()
module.update_inline_loading_indicator(widget)
assert len(widget.lines) == 0
def test_update_inline_loading_indicator_with_unsupported_widget_type_raises():
with pytest.raises(ValueError):
module.update_inline_loading_indicator(flexmock())
def test_add_inline_loading_indicator_with_option_list_adds_loading_indicator_option():
widget = textual.widgets.OptionList()
flexmock(widget).should_receive('set_interval')
module.add_inline_loading_indicator(widget)
assert len(widget.options) == 1
assert widget.options[0].prompt == module.LOADING_MESSAGE
assert widget.options[0].id == 'loading-indicator'
assert widget.highlighted is None
async def test_add_inline_loading_indicator_with_rich_log_writes_loading_indicator_text():
async with textual.app.App().run_test():
widget = textual.widgets.RichLog()
widget._size_known = True
flexmock(widget).should_receive('set_interval')
module.add_inline_loading_indicator(widget)
assert str(widget.lines[0].text) == module.LOADING_MESSAGE
def test_add_inline_loading_indicator_with_unsupported_widget_type_raises():
with pytest.raises(ValueError):
module.add_inline_loading_indicator(flexmock())
@@ -0,0 +1,37 @@
import contextlib
from flexmock import flexmock
from borgmatic.actions.browse import logs as module
def test_log_to_widget_adds_our_handler_and_removes_default_handler():
default_handler = module.borgmatic.logger.Multi_stream_handler({})
root_logger = module.logging.getLogger()
root_logger.addHandler(default_handler)
browse_log_handler = None
try:
module.log_to_widget(flexmock())
with contextlib.suppress(StopIteration):
browse_log_handler = next(
handler
for handler in root_logger.handlers
if isinstance(handler, module.Browse_log_handler)
)
assert browse_log_handler
finally:
if browse_log_handler:
root_logger.removeHandler(browse_log_handler)
assert not any(
handler
for handler in root_logger.handlers
if isinstance(handler, module.borgmatic.logger.Multi_stream_handler)
)
def test_logs_does_not_raise():
module.Logs()
@@ -0,0 +1,12 @@
from borgmatic.actions.browse import repositories_list as module
def test_repositories_list_populates_options():
config = {'repositories': [{'path': 'test1.borg'}, {'path': 'test2.borg', 'label': 'two'}]}
repositories_list = module.Repositories_list(config=config)
assert len(repositories_list.options) == 2
assert repositories_list.options[0].prompt == 'test1.borg'
assert repositories_list.options[0].id == 0
assert repositories_list.options[1].prompt == 'two'
assert repositories_list.options[1].id == 1
@@ -0,0 +1,19 @@
from flexmock import flexmock
from borgmatic.actions.browse import workers as module
def test_archive_path_loaded_publish_records_complete():
signal = module.Archive_path_loaded(owner=flexmock(), name='Bob')
signal.publish(module.LOADING_DONE)
assert signal.complete
def test_archive_path_loaded_publish_records_published_path():
archive_path = module.borgmatic.actions.browse.archive.Archive_path('-', 'foo/bar.txt', '')
signal = module.Archive_path_loaded(owner=flexmock(), name='Bob')
signal.publish(archive_path)
assert signal.path_hierarchy == {'foo': {'bar.txt': archive_path}}
assert not signal.complete
+2 -5
View File
@@ -35,7 +35,7 @@ def assert_command_does_not_duplicate_flags(command, *args, **kwargs):
if '--json' in command:
return '{}'
return None
return ''
def fuzz_argument(arguments, argument_name):
@@ -159,10 +159,7 @@ def test_make_repo_list_command_does_not_duplicate_flags_or_raise():
def test_display_archives_info_command_does_not_duplicate_flags_or_raise():
arguments = borgmatic.commands.arguments.parse_arguments({}, 'info')['info']
flexmock(borgmatic.borg.info).should_receive('execute_command_and_capture_output').replace_with(
assert_command_does_not_duplicate_flags,
)
flexmock(borgmatic.borg.info).should_receive('execute_command').replace_with(
assert_command_does_not_duplicate_flags,
lambda command, *args, **kwargs: iter((assert_command_does_not_duplicate_flags(command),)),
)
for argument_name in dir(arguments):
+12
View File
@@ -57,6 +57,8 @@ def test_make_rename_command_includes_log_debug():
def test_make_rename_command_includes_dry_run():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=True,
repository_name='repo',
@@ -79,6 +81,8 @@ def test_make_rename_command_includes_dry_run():
def test_make_rename_command_includes_remote_path():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
@@ -102,6 +106,8 @@ def test_make_rename_command_includes_remote_path():
def test_make_rename_command_includes_umask():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
@@ -125,6 +131,8 @@ def test_make_rename_command_includes_umask():
def test_make_rename_command_includes_log_json():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
@@ -140,6 +148,8 @@ def test_make_rename_command_includes_log_json():
def test_make_rename_command_includes_lock_wait():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
@@ -163,6 +173,8 @@ def test_make_rename_command_includes_lock_wait():
def test_make_rename_command_includes_extra_borg_options():
insert_logging_mock(logging.WARNING)
command = module.make_rename_command(
dry_run=False,
repository_name='repo',
+2 -2
View File
@@ -315,10 +315,10 @@ def test_write_configuration_with_already_existing_file_raises():
module.write_configuration('config.yaml', 'config: yaml')
def test_write_configuration_with_already_existing_file_and_overwrite_does_not_raise():
def test_write_configuration_with_already_existing_file_and_overwrite_does_not_raise(tmp_path):
flexmock(os.path).should_receive('exists').and_return(True)
module.write_configuration('/tmp/config.yaml', 'config: yaml', overwrite=True)
module.write_configuration(str(tmp_path / 'config.yaml'), 'config: yaml', overwrite=True)
def test_write_configuration_with_already_existing_directory_does_not_raise():
+3 -3
View File
@@ -222,11 +222,11 @@ def test_parse_configuration_merges_include():
assert logs == []
def test_parse_configuration_raises_for_missing_config_file():
def test_parse_configuration_raises_for_missing_config_file(tmp_path):
with pytest.raises(FileNotFoundError):
module.parse_configuration(
'/tmp/config.yaml',
'/tmp/schema.yaml',
str(tmp_path / 'nonexistent' / 'config.yaml'),
str(tmp_path / 'nonexistent' / 'schema.yaml'),
arguments={'global': flexmock()},
)
+21 -12
View File
@@ -8,7 +8,7 @@ from borgmatic.hooks.monitoring import loki as module
def test_loki_log_handler_raw_with_send_logs_posts_to_server_after_buffer_full():
handler = module.Loki_log_handler(flexmock(), send_logs=True, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=True, log_level=10, dry_run=False)
flexmock(module.requests).should_receive('post').and_return(
flexmock(raise_for_status=lambda: ''),
).once()
@@ -18,7 +18,7 @@ def test_loki_log_handler_raw_with_send_logs_posts_to_server_after_buffer_full()
def test_loki_log_handler_raw_without_send_logs_posts_to_server_without_buffering():
handler = module.Loki_log_handler(flexmock(), send_logs=False, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=False, log_level=10, dry_run=False)
flexmock(module.requests).should_receive('post').and_return(
flexmock(raise_for_status=lambda: ''),
).times(3)
@@ -28,7 +28,7 @@ def test_loki_log_handler_raw_without_send_logs_posts_to_server_without_bufferin
def test_loki_log_handler_raw_post_failure_does_not_raise():
handler = module.Loki_log_handler(flexmock(), send_logs=True, dry_run=False)
handler = module.Loki_log_handler(flexmock(), send_logs=True, log_level=10, dry_run=False)
flexmock(module.requests).should_receive('post').and_return(
flexmock(raise_for_status=lambda: (_ for _ in ()).throw(requests.RequestException())),
).once()
@@ -46,7 +46,10 @@ def test_initialize_monitor_replaces_labels():
'labels': {'hostname': '__hostname', 'config': '__config', 'config_full': '__config_path'},
}
config_filename = '/mock/path/test.yaml'
module.initialize_monitor(hook_config, flexmock(), config_filename, flexmock(), dry_run=False)
flexmock(module.logging.getLogger()).should_receive('setLevel')
module.initialize_monitor(
hook_config, flexmock(), config_filename, monitoring_log_level=10, dry_run=False
)
for handler in tuple(logging.getLogger().handlers):
if isinstance(handler, module.Loki_log_handler):
@@ -63,11 +66,12 @@ def test_initialize_monitor_adds_log_handler():
Assert that calling initialize_monitor adds our logger to the root logger.
'''
hook_config = {'url': 'http://localhost:3100/loki/api/v1/push', 'labels': {'app': 'borgmatic'}}
flexmock(module.logging.getLogger()).should_receive('setLevel')
module.initialize_monitor(
hook_config,
flexmock(),
config_filename='test.yaml',
monitoring_log_level=flexmock(),
monitoring_log_level=10,
dry_run=True,
)
@@ -86,7 +90,7 @@ def test_ping_monitor_sends_log_message():
config_filename = 'test.yaml'
post_called = False
def post(url, data, timeout, headers):
def post(url, data, timeout, headers, **kwargs):
nonlocal post_called
post_called = True
@@ -99,7 +103,10 @@ def test_ping_monitor_sends_log_message():
flexmock(module.requests).should_receive('post').replace_with(post)
module.initialize_monitor(hook_config, flexmock(), config_filename, flexmock(), dry_run=False)
flexmock(module.logging.getLogger()).should_receive('setLevel')
module.initialize_monitor(
hook_config, flexmock(), config_filename, monitoring_log_level=10, dry_run=False
)
module.ping_monitor(
hook_config,
flexmock(),
@@ -108,7 +115,7 @@ def test_ping_monitor_sends_log_message():
flexmock(),
dry_run=False,
)
module.destroy_monitor(hook_config, flexmock(), flexmock(), dry_run=False)
module.destroy_monitor(hook_config, flexmock(), monitoring_log_level=10, dry_run=False)
assert post_called
@@ -121,9 +128,11 @@ def test_destroy_monitor_removes_log_handler():
config_filename = 'test.yaml'
flexmock(module.requests).should_receive('post').never()
module.initialize_monitor(hook_config, flexmock(), config_filename, flexmock(), dry_run=False)
module.destroy_monitor(hook_config, flexmock(), flexmock(), dry_run=False)
flexmock(module.logging.getLogger()).should_receive('setLevel')
module.initialize_monitor(
hook_config, flexmock(), config_filename, monitoring_log_level=10, dry_run=False
)
module.destroy_monitor(hook_config, flexmock(), monitoring_log_level=10, dry_run=False)
for handler in tuple(logging.getLogger().handlers):
if isinstance(handler, module.Loki_log_handler):
raise AssertionError()
assert not isinstance(handler, module.Loki_log_handler)
+53 -22
View File
@@ -105,18 +105,15 @@ def test_log_outputs_logs_each_line_separately():
(),
).and_return((there_process.stdout,))
assert (
tuple(
module.log_outputs(
(hi_process, there_process),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
assert tuple(
module.log_outputs(
(hi_process, there_process),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
== ()
)
) == ('there',)
def test_log_outputs_logs_stderr_as_error():
@@ -180,18 +177,15 @@ def test_log_outputs_skips_logs_for_process_with_none_stdout():
(),
).and_return((there_process.stdout,))
assert (
tuple(
module.log_outputs(
(hi_process, there_process),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
assert tuple(
module.log_outputs(
(hi_process, there_process),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
== ()
)
) == ('there',)
def test_log_outputs_returns_output_without_logging_for_output_log_level_none():
@@ -563,6 +557,43 @@ def test_log_outputs_with_unfinished_process_re_polls():
)
def test_log_outputs_includes_error_output_when_output_spans_multiple_chunks():
flexmock(module.logger).should_receive('log')
flexmock(module).should_receive('interpret_exit_code').and_return(module.Exit_status.ERROR)
flexmock(module).should_receive('command_for_process').and_return('python')
process = subprocess.Popen(
[
sys.executable,
'-c',
(
'import os, sys; '
f'os.write(sys.stdout.fileno(), b"x" * {module.READ_CHUNK_SIZE + 10}); '
'os.write(sys.stdout.fileno(), b"\\nERROR: critical failure"); '
'os.close(1); '
'os._exit(2)'
),
],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
flexmock(module).should_receive('output_buffers_for_process').and_return((process.stdout,))
with pytest.raises(subprocess.CalledProcessError) as error:
tuple(
module.log_outputs(
(process,),
exclude_stdouts=(),
output_log_level=logging.INFO,
borg_local_path='borg',
borg_exit_codes=None,
)
)
assert error.value.output
assert 'ERROR: critical failure' in error.value.output
def test_read_lines_uses_system_locale_when_decoding_output():
flexmock(module.locale).should_receive('getpreferredencoding').and_return('ISO-8859-1')
process = subprocess.Popen(['echo', b'\xc4pple'], stdout=subprocess.PIPE)
+90
View File
@@ -0,0 +1,90 @@
from flexmock import flexmock
from borgmatic.actions.browse import archive as module
def test_get_repository_archives_does_not_raise():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.repo_list).should_receive('list_repository').and_return('{}')
assert module.get_repository_archives(config, config['repositories'][0]) == {}
def test_get_archive_paths_returns_each_as_archive_path_with_metadata():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.list).should_receive('capture_archive_listing').and_yield(
{'path': 'foo.txt', 'type': '-', 'linktarget': ''},
{'path': 'bar.txt', 'type': 'l', 'linktarget': 'foo.txt'},
{'path': 'etc', 'type': 'd', 'linktarget': ''},
)
assert tuple(module.get_archive_paths(config, config['repositories'][0], 'archive')) == (
module.Archive_path('-', 'foo.txt', ''),
module.Archive_path('l', 'bar.txt', 'foo.txt'),
module.Archive_path('d', 'etc', ''),
)
def test_get_archive_file_content_with_binary_file_bails():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
flexmock(stdout=flexmock(readlines=lambda hint: [b'foo\n', b'bar\n'])),
)
flexmock(module.binaryornot.helpers).should_receive('is_binary_string').and_return(True)
assert (
module.get_archive_file_content(config, config['repositories'][0], 'archive', 'etc/foo.txt')
is None
)
def test_get_archive_file_content_decodes_content():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
flexmock(stdout=flexmock(readlines=lambda hint: [b'foo\n', b'bar\n'])),
)
flexmock(module.binaryornot.helpers).should_receive('is_binary_string').and_return(False)
assert (
module.get_archive_file_content(config, config['repositories'][0], 'archive', 'etc/foo.txt')
== 'foo\nbar\n'
)
def test_get_archive_file_content_with_large_file_truncates_content():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module).READLINES_HINT_BYTES = 3
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
flexmock(stdout=flexmock(readlines=lambda hint: [b'foo\n', b'bar\n'])),
)
flexmock(module.binaryornot.helpers).should_receive('is_binary_string').and_return(False)
assert (
module.get_archive_file_content(config, config['repositories'][0], 'archive', 'etc/foo.txt')
== f'foo\nbar\n\n{module.TRUNCATION_MESSAGE}'
)
def test_get_archive_file_content_with_unicode_decode_error_does_not_raise():
config = {'repositories': [{'path': 'test.borg'}]}
flexmock(module.borgmatic.logger).should_receive('Log_prefix').and_return(flexmock())
flexmock(module.borgmatic.borg.version).should_receive('local_borg_version').and_return('3.0')
flexmock(module.borgmatic.borg.extract).should_receive('extract_archive').and_return(
flexmock(stdout=flexmock(readlines=lambda hint: [b'foo\n', b'\xc3\n'])),
)
flexmock(module.binaryornot.helpers).should_receive('is_binary_string').and_return(False)
assert (
module.get_archive_file_content(config, config['repositories'][0], 'archive', 'etc/foo.txt')
is None
)
@@ -0,0 +1,94 @@
from flexmock import flexmock
from borgmatic.actions.browse import directory_list as module
def test_get_relative_archive_path_components_strips_off_current_directory():
assert module.get_relative_archive_path_components(
flexmock(file_path='foo/bar/baz/quux.txt'), ('foo', 'bar')
) == ('baz', 'quux.txt')
def test_get_relative_archive_path_components_with_root_current_directory_strips_off_nothing():
assert module.get_relative_archive_path_components(
flexmock(file_path='foo/bar/baz/quux.txt'), ()
) == ('foo', 'bar', 'baz', 'quux.txt')
def test_get_relative_archive_path_components_with_non_matching_paths_returns_none():
assert (
module.get_relative_archive_path_components(
flexmock(file_path='foo/bar/baz/quux.txt'), ('etc',)
)
is None
)
def test_make_directory_list_option_with_file_path_makes_file_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='-', file_path='foo/bar/baz.txt', link_target=''), ('baz.txt',)
)
assert option.prompt == '📄 baz.txt'
assert option.id == 'baz.txt'
def test_make_directory_list_option_with_directory_path_makes_directory_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='d', file_path='foo/bar/baz', link_target=''), ('baz',)
)
assert option.prompt == '📁 baz'
assert option.id == 'baz'
def test_make_directory_list_option_with_contained_file_path_makes_directory_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='d', file_path='foo/bar/baz.txt', link_target=''),
(
'bar',
'baz.txt',
),
)
assert option.prompt == '📁 bar'
assert option.id == 'bar'
def test_make_directory_list_option_with_link_path_makes_link_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='l', file_path='foo/bar/baz.txt', link_target='quux.txt'), ('baz.txt',)
)
assert option.prompt == '🔗 baz.txt → quux.txt'
assert option.id == 'baz.txt'
def test_make_directory_list_option_with_pipe_path_makes_pipe_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='p', file_path='foo/bar/baz.txt', link_target=''), ('baz.txt',)
)
assert option.prompt == '🚰 baz.txt'
assert option.id == 'baz.txt'
def test_make_directory_list_option_with_unknown_path_makes_unknown_option():
flexmock(module.textual.widgets.option_list).should_receive('Option').replace_with(flexmock)
option = module.make_directory_list_option(
flexmock(path_type='wtf', file_path='foo/bar/baz.txt', link_target=''), ('baz.txt',)
)
assert option.prompt == '❓ baz.txt'
assert option.id == 'baz.txt'

Some files were not shown because too many files have changed in this diff Show More