peter/borgmatic
1
0
Fork 0
mirror of https://github.com/borgmatic-collective/borgmatic.git synced 2026-08-01 14:13:01 +02:00
Code Issues Packages Projects Releases Wiki Activity

Adjust Btrfs snapshot paths so that Borg 1.x gets file cache hits when backing them up (#1206).

Browse Source 
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1206
This commit is contained in:
Dan Helfman
2025-12-21 01:10:33 +00:00
parent 1b471f5346 18a1d4f00b
commit 406b533b15
5 changed files with 126 additions and 174 deletions
Download Patch File Download Diff File Expand all files Collapse all files
NEWS
+4 -2
View File
@@ -2,6 +2,8 @@
* #1054: Allow the Btrfs hook to create and delete snapshots even when running
as a non-root user. See the documentation for more information:
https://torsion.org/borgmatic/reference/configuration/data-sources/btrfs/#non-root-user
* #1122: To prevent the user from inadvertently excluding the "bootstrap" action's manifest, always
error and exit when the borgmatic runtime directory overlaps with the configured excludes.
* #1179: Add a "file_list_format" option for setting the "list" action's output format and an
"archive_list_format" option for setting the "repo-list" action's format.
* #1192: Fix for over-aggressive deduplication of source directories that contain the borgmatic
@@ -24,8 +26,8 @@
* #1203: Fix that errors and exits when the borgmatic runtime directory is partially excluded by
configured excludes. Previously, borgmatic only errored when the runtime directory was completely
excluded.
* #1122: To prevent the user from inadvertently excluding the "bootstrap" action's manifest, always
error and exit when the borgmatic runtime directory overlaps with the configured excludes.
* #1206: Adjust Btrfs snapshot paths so that Borg 1.x gets file cache hits when backing them up,
improving performance.
* Update the sample systemd timer with a shorter random delay when catching up on a missed run.
2.0.12
borgmatic/hooks/data_source/btrfs.py
+6 -6
View File
@@ -171,7 +171,7 @@ def get_subvolumes(btrfs_command, patterns):
return tuple(sorted(subvolumes, key=lambda subvolume: subvolume.path))
BORGMATIC_SNAPSHOT_PREFIX = '.borgmatic-snapshot-'
BORGMATIC_SNAPSHOT_PREFIX = '.borgmatic-snapshot'
def make_snapshot_path(subvolume_path):
@@ -180,7 +180,7 @@ def make_snapshot_path(subvolume_path):
'''
return os.path.join(
subvolume_path,
f'{BORGMATIC_SNAPSHOT_PREFIX}{os.getpid()}',
f'{BORGMATIC_SNAPSHOT_PREFIX}',
# Included so that the snapshot ends up in the Borg archive at the "original" subvolume path.
) + subvolume_path.rstrip(os.path.sep)
@@ -193,16 +193,16 @@ def make_snapshot_exclude_pattern(subvolume_path): # pragma: no cover
directory within the snapshot itself. For instance, if you have a Btrfs subvolume at /mnt and
make a snapshot of it at:
/mnt/.borgmatic-snapshot-1234/mnt
/mnt/.borgmatic-snapshot/mnt
... then the snapshot itself will have an empty directory at:
/mnt/.borgmatic-snapshot-1234/mnt/.borgmatic-snapshot-1234
/mnt/.borgmatic-snapshot/mnt/.borgmatic-snapshot
So to prevent that from ending up in the Borg archive, this function produces an exclude pattern
to exclude that path.
'''
snapshot_directory = f'{BORGMATIC_SNAPSHOT_PREFIX}{os.getpid()}'
snapshot_directory = f'{BORGMATIC_SNAPSHOT_PREFIX}'
return borgmatic.borg.pattern.Pattern(
os.path.join(
@@ -235,7 +235,7 @@ def make_borg_snapshot_pattern(subvolume_path, pattern):
rewritten_path = initial_caret + os.path.join(
subvolume_path,
f'{BORGMATIC_SNAPSHOT_PREFIX}{os.getpid()}',
f'{BORGMATIC_SNAPSHOT_PREFIX}',
# Use the Borg 1.4+ "slashdot" hack to prevent the snapshot path prefix from getting
# included in the archive—but only if there's not already a slashdot hack present in the
# pattern.
docs/reference/configuration/data-sources/btrfs.md
+15 -17
View File
@@ -60,7 +60,7 @@ Additionally, borgmatic rewrites the snapshot file paths so that they appear at
their original subvolume locations in a Borg archive. For instance, if your
subvolume path is `/var/subvolume`, then the snapshotted files will appear in an
archive at `/var/subvolume` as well—even if borgmatic has to mount the snapshot
somewhere in `/var/subvolume/.borgmatic-snapshot-1234/` to perform the backup.
somewhere in `/var/subvolume/.borgmatic-snapshot/` to perform the backup.
<span class="minilink minilink-addedin">With Borg version 1.2 and
earlier</span>Snapshotted files are instead stored at a path dependent on the
@@ -70,26 +70,24 @@ temporary snapshot directory in use at the time the archive was created, as Borg
## Performance
<span class="minilink minilink-addedin">With Borg version 1.x</span> Because of
the way that Btrfs snapshot paths change from one borgmatic invocation to the
next, the [Borg file
<span class="minilink minilink-addedin">New in borgmatic version 2.0.13, with
Borg version 1.x</span> borgmatic uses consistent snapshot paths between
invocations, so Btrfs snapshots are cached correctly. No configuration is
necessary.
<span class="minilink minilink-addedin">Prior to borgmatic version 2.0.13, with
Borg version 1.x</span> Because of the way that Btrfs snapshot paths change from
one borgmatic invocation to the next, the [Borg file
cache](https://borgbackup.readthedocs.io/en/stable/internals/data-structures.html#cache)
will never get cache hits on snapshotted files. This makes backing up Btrfs
never gets cache hits on snapshotted files. This makes backing up Btrfs
snapshots a little slower than non-snapshotted files that have consistent paths.
**It is also not possible to mitigate cache misses**, as the Btrfs hook uses
snapshot paths which change between borgmatic invocations, and the snapshots
are located outside the [runtime
directory](https://torsion.org/borgmatic/reference/configuration/runtime-directory/),
contrary to
[ZFS](https://torsion.org/borgmatic/reference/configuration/data-sources/zfs/#performance)
and
[LVM](https://torsion.org/borgmatic/reference/configuration/data-sources/lvm/#performance).
If this is an issue for you, upgrade to borgmatic to 2.0.13+.
<span class="minilink minilink-addedin">With Borg version 2.x</span> Even
snapshotted files should get cache hits, because Borg 2.x is smarter about how
it looks up file paths in its cache—it constructs the cache key with the path
*as it's seen in the archive* (which is consistent across runs) rather than the
full absolute source path (which changes).
snapshotted files get cache hits, because Borg 2.x is smarter about how it looks
up file paths in its cache—it constructs the cache key with the path *as it's
seen in the archive* (which is consistent across runs) rather than the full
absolute source path (which changes).
## systemd settings
tests/integration/hooks/data_source/test_btrfs.py
+7 -8
View File
@@ -24,16 +24,15 @@ def test_dump_data_sources_snapshots_each_subvolume_and_updates_patterns():
module.Subvolume('/mnt/subvol2', contained_patterns=(Pattern('/mnt/subvol2'),)),
),
)
flexmock(module.os).should_receive('getpid').and_return(1234)
flexmock(module).should_receive('snapshot_subvolume').with_args(
'btrfs',
'/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).once()
flexmock(module).should_receive('snapshot_subvolume').with_args(
'btrfs',
'/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).once()
assert (
@@ -50,19 +49,19 @@ def test_dump_data_sources_snapshots_each_subvolume_and_updates_patterns():
assert patterns == [
Pattern(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2/.borgmatic-snapshot-1234',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
Pattern('/foo'),
Pattern('/mnt/subvol1/.borgmatic-snapshot-1234/./mnt/subvol1'),
Pattern('/mnt/subvol1/.borgmatic-snapshot-1234/./mnt/subvol1/.cache', Pattern_type.EXCLUDE),
Pattern('/mnt/subvol2/.borgmatic-snapshot-1234/./mnt/subvol2'),
Pattern('/mnt/subvol1/.borgmatic-snapshot/./mnt/subvol1'),
Pattern('/mnt/subvol1/.borgmatic-snapshot/./mnt/subvol1/.cache', Pattern_type.EXCLUDE),
Pattern('/mnt/subvol2/.borgmatic-snapshot/./mnt/subvol2'),
]
assert config == {
'btrfs': {},
tests/unit/hooks/data_source/test_btrfs.py
+94 -141
View File
@@ -331,16 +331,14 @@ def test_get_subvolumes_skips_non_config_patterns():
@pytest.mark.parametrize(
'subvolume_path,expected_snapshot_path',
(
('/foo/bar', '/foo/bar/.borgmatic-snapshot-1234/foo/bar'),
('/', '/.borgmatic-snapshot-1234'),
('/foo/bar', '/foo/bar/.borgmatic-snapshot/foo/bar'),
('/', '/.borgmatic-snapshot'),
),
)
def test_make_snapshot_path_includes_stripped_subvolume_path(
subvolume_path,
expected_snapshot_path,
):
flexmock(module.os).should_receive('getpid').and_return(1234)
assert module.make_snapshot_path(subvolume_path) == expected_snapshot_path
@@ -350,14 +348,14 @@ def test_make_snapshot_path_includes_stripped_subvolume_path(
(
'/foo/bar',
Pattern('/foo/bar/baz'),
Pattern('/foo/bar/.borgmatic-snapshot-1234/./foo/bar/baz'),
Pattern('/foo/bar/.borgmatic-snapshot/./foo/bar/baz'),
),
('/foo/bar', Pattern('/foo/bar'), Pattern('/foo/bar/.borgmatic-snapshot-1234/./foo/bar')),
('/foo/bar', Pattern('/foo/bar'), Pattern('/foo/bar/.borgmatic-snapshot/./foo/bar')),
(
'/foo/bar',
Pattern('^/foo/bar', Pattern_type.INCLUDE, Pattern_style.REGULAR_EXPRESSION),
Pattern(
'^/foo/bar/.borgmatic-snapshot-1234/./foo/bar',
'^/foo/bar/.borgmatic-snapshot/./foo/bar',
Pattern_type.INCLUDE,
Pattern_style.REGULAR_EXPRESSION,
),
@@ -366,17 +364,17 @@ def test_make_snapshot_path_includes_stripped_subvolume_path(
'/foo/bar',
Pattern('/foo/bar', Pattern_type.INCLUDE, Pattern_style.REGULAR_EXPRESSION),
Pattern(
'/foo/bar/.borgmatic-snapshot-1234/./foo/bar',
'/foo/bar/.borgmatic-snapshot/./foo/bar',
Pattern_type.INCLUDE,
Pattern_style.REGULAR_EXPRESSION,
),
),
('/', Pattern('/foo'), Pattern('/.borgmatic-snapshot-1234/./foo')),
('/', Pattern('/'), Pattern('/.borgmatic-snapshot-1234/./')),
('/', Pattern('/foo'), Pattern('/.borgmatic-snapshot/./foo')),
('/', Pattern('/'), Pattern('/.borgmatic-snapshot/./')),
(
'/foo/bar',
Pattern('/foo/bar/./baz'),
Pattern('/foo/bar/.borgmatic-snapshot-1234/foo/bar/./baz'),
Pattern('/foo/bar/.borgmatic-snapshot/foo/bar/./baz'),
),
),
)
@@ -385,8 +383,6 @@ def test_make_borg_snapshot_pattern_includes_slashdot_hack_and_stripped_pattern_
pattern,
expected_pattern,
):
flexmock(module.os).should_receive('getpid').and_return(1234)
assert module.make_borg_snapshot_pattern(subvolume_path, pattern) == expected_pattern
@@ -400,26 +396,26 @@ def test_dump_data_sources_snapshots_each_subvolume_and_replaces_patterns():
),
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol2').and_return(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
)
flexmock(module).should_receive('snapshot_subvolume').with_args(
'btrfs',
'/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).once()
flexmock(module).should_receive('snapshot_subvolume').with_args(
'btrfs',
'/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).once()
flexmock(module).should_receive('make_snapshot_exclude_pattern').with_args(
'/mnt/subvol1',
).and_return(
Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
@@ -428,7 +424,7 @@ def test_dump_data_sources_snapshots_each_subvolume_and_replaces_patterns():
'/mnt/subvol2',
).and_return(
Pattern(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2/.borgmatic-snapshot-1234',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
@@ -436,16 +432,16 @@ def test_dump_data_sources_snapshots_each_subvolume_and_replaces_patterns():
flexmock(module).should_receive('make_borg_snapshot_pattern').with_args(
'/mnt/subvol1',
object,
).and_return(Pattern('/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1'))
).and_return(Pattern('/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1'))
flexmock(module).should_receive('make_borg_snapshot_pattern').with_args(
'/mnt/subvol2',
object,
).and_return(Pattern('/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2'))
).and_return(Pattern('/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2'))
flexmock(module.borgmatic.hooks.data_source.config).should_receive('replace_pattern').with_args(
object,
Pattern('/mnt/subvol1'),
module.borgmatic.borg.pattern.Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
source=module.borgmatic.borg.pattern.Pattern_source.HOOK,
),
).once()
@@ -453,14 +449,14 @@ def test_dump_data_sources_snapshots_each_subvolume_and_replaces_patterns():
object,
Pattern('/mnt/subvol2'),
module.borgmatic.borg.pattern.Pattern(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
source=module.borgmatic.borg.pattern.Pattern_source.HOOK,
),
).once()
flexmock(module.borgmatic.hooks.data_source.config).should_receive('inject_pattern').with_args(
object,
module.borgmatic.borg.pattern.Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
@@ -468,7 +464,7 @@ def test_dump_data_sources_snapshots_each_subvolume_and_replaces_patterns():
flexmock(module.borgmatic.hooks.data_source.config).should_receive('inject_pattern').with_args(
object,
module.borgmatic.borg.pattern.Pattern(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2/.borgmatic-snapshot-1234',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
@@ -498,18 +494,18 @@ def test_dump_data_sources_uses_custom_btrfs_command_in_commands():
(module.Subvolume('/mnt/subvol1', contained_patterns=(Pattern('/mnt/subvol1'),)),),
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
)
flexmock(module).should_receive('snapshot_subvolume').with_args(
'/usr/local/bin/btrfs',
'/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).once()
flexmock(module).should_receive('make_snapshot_exclude_pattern').with_args(
'/mnt/subvol1',
).and_return(
Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
@@ -517,19 +513,19 @@ def test_dump_data_sources_uses_custom_btrfs_command_in_commands():
flexmock(module).should_receive('make_borg_snapshot_pattern').with_args(
'/mnt/subvol1',
object,
).and_return(Pattern('/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1'))
).and_return(Pattern('/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1'))
flexmock(module.borgmatic.hooks.data_source.config).should_receive('replace_pattern').with_args(
object,
Pattern('/mnt/subvol1'),
module.borgmatic.borg.pattern.Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
source=module.borgmatic.borg.pattern.Pattern_source.HOOK,
),
).once()
flexmock(module.borgmatic.hooks.data_source.config).should_receive('inject_pattern').with_args(
object,
module.borgmatic.borg.pattern.Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
@@ -565,18 +561,18 @@ def test_dump_data_sources_with_findmnt_command_warns():
(module.Subvolume('/mnt/subvol1', contained_patterns=(Pattern('/mnt/subvol1'),)),),
).once()
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
)
flexmock(module).should_receive('snapshot_subvolume').with_args(
'btrfs',
'/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).once()
flexmock(module).should_receive('make_snapshot_exclude_pattern').with_args(
'/mnt/subvol1',
).and_return(
Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
@@ -584,19 +580,19 @@ def test_dump_data_sources_with_findmnt_command_warns():
flexmock(module).should_receive('make_borg_snapshot_pattern').with_args(
'/mnt/subvol1',
object,
).and_return(Pattern('/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1'))
).and_return(Pattern('/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1'))
flexmock(module.borgmatic.hooks.data_source.config).should_receive('replace_pattern').with_args(
object,
Pattern('/mnt/subvol1'),
module.borgmatic.borg.pattern.Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
source=module.borgmatic.borg.pattern.Pattern_source.HOOK,
),
).once()
flexmock(module.borgmatic.hooks.data_source.config).should_receive('inject_pattern').with_args(
object,
module.borgmatic.borg.pattern.Pattern(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1/.borgmatic-snapshot',
Pattern_type.NO_RECURSE,
Pattern_style.FNMATCH,
),
@@ -628,7 +624,7 @@ def test_dump_data_sources_with_dry_run_skips_snapshot_and_patterns_update():
(module.Subvolume('/mnt/subvol1', contained_patterns=(Pattern('/mnt/subvol1'),)),),
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
)
flexmock(module).should_receive('snapshot_subvolume').never()
flexmock(module).should_receive('make_snapshot_exclude_pattern').never()
@@ -684,91 +680,59 @@ def test_remove_data_source_dumps_deletes_snapshots():
),
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/./mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/./mnt/subvol1',
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol2').and_return(
'/mnt/subvol2/.borgmatic-snapshot-1234/./mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/./mnt/subvol2',
)
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol1/.borgmatic-*/mnt/subvol1')
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol2/.borgmatic-*/mnt/subvol2')
flexmock(module.glob).should_receive('glob').with_args(
'/mnt/subvol1/.borgmatic-*/mnt/subvol1',
).and_return(
(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
),
('/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',),
)
flexmock(module.glob).should_receive('glob').with_args(
'/mnt/subvol2/.borgmatic-*/mnt/subvol2',
).and_return(
(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
),
('/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',),
)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
).and_return(False)
flexmock(module).should_receive('delete_snapshot').with_args(
'btrfs',
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).once()
flexmock(module).should_receive('delete_snapshot').with_args(
'btrfs',
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).once()
flexmock(module).should_receive('delete_snapshot').with_args(
'btrfs',
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
).once()
flexmock(module).should_receive('delete_snapshot').with_args(
'btrfs',
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
).never()
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-5678',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-5678',
'/mnt/subvol2/.borgmatic-snapshot',
).and_return(True)
flexmock(module.shutil).should_receive('rmtree').with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234',
'/mnt/subvol1/.borgmatic-snapshot',
).once()
flexmock(module.shutil).should_receive('rmtree').with_args(
'/mnt/subvol1/.borgmatic-snapshot-5678',
'/mnt/subvol2/.borgmatic-snapshot',
).once()
flexmock(module.shutil).should_receive('rmtree').with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234',
).once()
flexmock(module.shutil).should_receive('rmtree').with_args(
'/mnt/subvol2/.borgmatic-snapshot-5678',
).never()
module.remove_data_source_dumps(
hook_config=config['btrfs'],
@@ -846,50 +810,50 @@ def test_remove_data_source_dumps_with_dry_run_skips_deletes():
),
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/./mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/./mnt/subvol1',
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol2').and_return(
'/mnt/subvol2/.borgmatic-snapshot-1234/./mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/./mnt/subvol2',
)
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol1/.borgmatic-*/mnt/subvol1')
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol2/.borgmatic-*/mnt/subvol2')
flexmock(module.glob).should_receive('glob').with_args(
'/mnt/subvol1/.borgmatic-*/mnt/subvol1',
).and_return(
(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
),
)
flexmock(module.glob).should_receive('glob').with_args(
'/mnt/subvol2/.borgmatic-*/mnt/subvol2',
).and_return(
(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
),
)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).and_return(False)
flexmock(module).should_receive('delete_snapshot').never()
flexmock(module.shutil).should_receive('rmtree').never()
@@ -931,21 +895,21 @@ def test_remove_data_source_without_snapshots_skips_deletes():
),
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/./mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/./mnt/subvol1',
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol2').and_return(
'/mnt/subvol2/.borgmatic-snapshot-1234/./mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/./mnt/subvol2',
)
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol1/.borgmatic-*/mnt/subvol1')
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol2/.borgmatic-*/mnt/subvol2')
flexmock(module.glob).should_receive('glob').and_return(())
@@ -971,50 +935,50 @@ def test_remove_data_source_dumps_with_delete_snapshot_file_not_found_error_bail
),
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/./mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/./mnt/subvol1',
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol2').and_return(
'/mnt/subvol2/.borgmatic-snapshot-1234/./mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/./mnt/subvol2',
)
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol1/.borgmatic-*/mnt/subvol1')
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol2/.borgmatic-*/mnt/subvol2')
flexmock(module.glob).should_receive('glob').with_args(
'/mnt/subvol1/.borgmatic-*/mnt/subvol1',
).and_return(
(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
),
)
flexmock(module.glob).should_receive('glob').with_args(
'/mnt/subvol2/.borgmatic-*/mnt/subvol2',
).and_return(
(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
),
)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).and_return(False)
flexmock(module).should_receive('delete_snapshot').and_raise(FileNotFoundError)
flexmock(module.shutil).should_receive('rmtree').never()
@@ -1037,50 +1001,50 @@ def test_remove_data_source_dumps_with_delete_snapshot_called_process_error_bail
),
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol1').and_return(
'/mnt/subvol1/.borgmatic-snapshot-1234/./mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/./mnt/subvol1',
)
flexmock(module).should_receive('make_snapshot_path').with_args('/mnt/subvol2').and_return(
'/mnt/subvol2/.borgmatic-snapshot-1234/./mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/./mnt/subvol2',
)
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol1/.borgmatic-*/mnt/subvol1')
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/mnt/subvol2/.borgmatic-*/mnt/subvol2')
flexmock(module.glob).should_receive('glob').with_args(
'/mnt/subvol1/.borgmatic-*/mnt/subvol1',
).and_return(
(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
),
)
flexmock(module.glob).should_receive('glob').with_args(
'/mnt/subvol2/.borgmatic-*/mnt/subvol2',
).and_return(
(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
),
)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-1234/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol1/.borgmatic-snapshot-5678/mnt/subvol1',
'/mnt/subvol1/.borgmatic-snapshot/mnt/subvol1',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-1234/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).and_return(True)
flexmock(module.os.path).should_receive('isdir').with_args(
'/mnt/subvol2/.borgmatic-snapshot-5678/mnt/subvol2',
'/mnt/subvol2/.borgmatic-snapshot/mnt/subvol2',
).and_return(False)
flexmock(module).should_receive('delete_snapshot').and_raise(
module.subprocess.CalledProcessError(1, 'command', 'error'),
@@ -1103,40 +1067,29 @@ def test_remove_data_source_dumps_with_root_subvolume_skips_duplicate_removal():
)
flexmock(module).should_receive('make_snapshot_path').with_args('/').and_return(
'/.borgmatic-snapshot-1234',
'/.borgmatic-snapshot',
)
flexmock(module.borgmatic.config.paths).should_receive(
'replace_temporary_subdirectory_with_glob',
).with_args(
'/.borgmatic-snapshot-1234',
'/.borgmatic-snapshot',
temporary_directory_prefix=module.BORGMATIC_SNAPSHOT_PREFIX,
).and_return('/.borgmatic-*')
flexmock(module.glob).should_receive('glob').with_args('/.borgmatic-*').and_return(
('/.borgmatic-snapshot-1234', '/.borgmatic-snapshot-5678'),
('/.borgmatic-snapshot', '/.borgmatic-snapshot'),
)
flexmock(module.os.path).should_receive('isdir').with_args(
'/.borgmatic-snapshot-1234'
).and_return(
True,
).and_return(False)
flexmock(module.os.path).should_receive('isdir').with_args(
'/.borgmatic-snapshot-5678'
).and_return(
True,
flexmock(module.os.path).should_receive('isdir').with_args('/.borgmatic-snapshot').and_return(
True
).and_return(False)
flexmock(module).should_receive('delete_snapshot').with_args(
'btrfs', '/.borgmatic-snapshot-1234'
).once()
flexmock(module).should_receive('delete_snapshot').with_args(
'btrfs', '/.borgmatic-snapshot-5678'
'btrfs', '/.borgmatic-snapshot'
).once()
flexmock(module.os.path).should_receive('isdir').with_args('').and_return(False)
flexmock(module.shutil).should_receive('rmtree').never()
module.remove_data_source_dumps(