test: Optionally use deterministic topology mappings

Due to the salting being applied to Python's `hash()` (see [1]), the
hash of an object is not stable across different Python processes. As
a result, neither are the topology mappings generated by networkx.

The upside of this is randomness that we get better test coverage over
time, especially on physical devices where the underlying hardware
could differ between ports, for example.

On the other hand, it can be very frustrating to track down a bug when
locial nodes are suffled around between phyical nodes on each
invocation of a test case.

Therefore, use a random seed by default, but allow the user to specify
a fixed value if they so choose. We then add a meta test that logs the
seed being used for the current suite - which means we can simply
copy&paste that value to rerun the suite (or single test) with the
same topology mappings.

[1]: https://docs.python.org/3/reference/datamodel.html#object.__hash__
This commit is contained in:
Tobias Waldekranz
2024-05-03 14:44:54 +02:00
committed by Joachim Wiberg
parent 753f3806a1
commit a2906acdf7
3 changed files with 18 additions and 0 deletions
+1
View File
@@ -1,4 +1,5 @@
---
- case: meta/reproducible.py
- case: meta/wait.py
- name: Misc tests
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env python3
import os
import infamy
with infamy.Test() as test:
with test.step("$PYTHONHASHSEED is set"):
seed = os.environ.get("PYTHONHASHSEED")
if seed == None:
print("$PYTHONHASHSEED must be set in order to create a reproducible test environment")
test.fail()
else:
print(f"Specify PYTHONHASHSEED={seed} to reproduce this test environment")
test.succeed()
+1
View File
@@ -157,6 +157,7 @@ if [ "$containerize" ]; then
--cap-add=NET_RAW \
--cap-add=NET_ADMIN \
--device=/dev/net/tun \
--env PYTHONHASHSEED=${PYTHONHASHSEED:-$(shuf -i 0-$(((1 << 32) - 1)) -n 1)} \
--env VIRTUAL_ENV_DISABLE_PROMPT=yes \
--env PROMPT_DIRTRIM="$ixdir" \
--env PS1='\e[1m$(date "+%H:%M:%S") \h\e[0m:\W # ' \