Commit Graph
31 Commits
Author SHA1 Message Date
Joachim Wiberg a3d30eb32f test/infamy: minor, whitespace only
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-28 21:35:31 +02:00
Mattias Walström 8740b2eb23 test: infamy: Add a timeout when starting each test if the DUTs is not reachable
This to cover if you run one test, but the DUTs are still booting.
2025-08-26 14:38:42 +02:00
Tobias Waldekranz da39855cec test: infamy: Allow tests to refine the topology matching process
To run LAG tests on hardware, we have to make sure that the chosen
links are running at the same speed.  We do not want to require a
specific link speed from the logical topology, as the real requirement
is merely that the two (or however many) links are of the _same_
speed, not any _particular_ speed.

Rather than having to complicate the topology matcher, let it take
care of the common pattern of matching requirements to provided
features, then let tests pass custom node/edge matchers for the
complicated cases.
2025-02-12 23:22:09 +01:00
Joachim WibergandTobias Waldekranz f9b155b49f test: infamy: use decorator to reduce boilerplate for test args
Example use:

    @infamy.test_argument("--mode", help="one of 'static' or 'lacp'")
    class TestArgs(infamy.ArgumentParser):
        pass

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-12 23:21:08 +01:00
Mattias Walström b2db59f3bc infamy: Add support for parameters to tests
All test added parameters is required and no default behaviour is allowed.
2025-01-10 21:40:42 +01:00
Tobias Waldekranz f3f313bb98 infamy: Fix default transport selection
Using `sys.argv[0]` does not quite achieve the effect we want, namely
that any (test-case, $PYTHONHASHSEED) tuple should always default to
the same transport.

For example, the following two examples should always use the same
transport:

    $ make PYTHONHASHSEED=1337 test-sh
    infamy0:test$ ./case/ietf_system/hostname/test.py

    $ make PYTHONHASHSEED=1337 test-sh
    infamy0:test$ cd ./case/ietf_system/hostname
    infamy0:hostname$ ./test.py

But, since their argv[0]'s are different, they don't.

Therefore use the _last two components_ of argv[0]'s full path
instead. This should:

- Spread the allocation over an entire suite run, since the
  penultimate component is usually the test name.

- Keep the allocation stable when test code is modified (which is
  common during debugging of failing tests)

- Avoid instabilities stemming from the local storage location (i.e.,
  home directory names etc.)
2025-01-08 10:14:46 +01:00
Tobias Waldekranz 343400c5fe test: infamy: Default to attaching to DUTs via the "mgmt" port
This convention is now well established, so it seems silly to have to
repeat it in every test.
2025-01-08 10:14:46 +01:00
Mattias Walström 35a909edf0 test: infamy: Make possible to override username and password
This is now possible for SSH/RESTCONF/NETCONF, most useful with
SSH and adding/modify users.
2024-09-17 08:27:23 +02:00
Joachim Wiberg f071dacf8b test: add __str__ to Device class for reading device name
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-09-05 07:04:58 +02:00
Joachim Wiberg a69f1c9272 test: autodetect available transports from .config
With the addition of minimal defconfigs without RESTCONF support, detect
transport when 'make test' and 'make test-sh' are called.

We set the new variable INFAMY_TRANSPORT so Infamy can detect protocol
also when in interactive test mode ('make test-sh').

Issue #597

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-09-05 07:04:58 +02:00
Joachim Wiberg 5e03b20bb2 test: minor, cleanup & pep-8 fixes
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-09-05 07:04:58 +02:00
Ahmed KaricandMattias Walström 46ad19793b test: refactor test environment 2024-09-03 12:13:32 +02:00
Mattias Walström dc4c38ede6 test: Adapt tests/infamy for the new test config
Instead of copy factoy config to reset between tests,
use the new test-config.cfg.

Change the logic in the factory config since the default values have changed.
2024-08-30 17:40:02 +02:00
Mattias Walström d27153ea95 Refactor randomization of transport protocol
Simplify the logic and add a parameter to the command line parser
instead of environment variable.
2024-06-27 15:43:06 +02:00
Mattias Walström df7dab5951 infamy: Randomize tests for restconf/netconf 2024-06-27 15:43:06 +02:00
Mattias Walström 29f56eb384 Restconf, netconf: Implement full feature compatibility
Abstract functions and common functions are located in transport.py
Some quirks has been required due to how rousette works,
these functions has been added to the abstract class

Can be removed when rousette behaves as expected.
2024-06-27 15:43:06 +02:00
Joachim Wiberg e5820e1e55 test: new method get_password() in Env
Allow tests to fetch expected factory default password of devices.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-06-25 17:22:36 +02:00
Joachim Wiberg fb424709d7 test: minor, fix pep-8 warnings from pylint
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-06-25 17:22:36 +02:00
Mattias Walström 23585486de test: Add backend to run tests with restconf
Very simple tests work (hostname.py), more complex
tests (static_routing.py) failes due to the lack of
model prefix, for example. infix-ip:ipv4 instead of just
ipv4.

We need to fix this before you send your configuration, run
it through libyang for example and get the full model-name.
2024-06-04 13:38:03 +02:00
Tobias Waldekranz e65625e32f test: Add support for attaching to devices via SSH
Support the familiar subprocess.run API, and runsh from netns.py.

This should only be used for debugging, or in the rare instances when
we want to test functionality that is not available via NETCONF.
2024-05-20 16:21:06 +02:00
Tobias Waldekranz 772019db22 test: env: Add support for querying global attributes
This will let other Infamy submodules tune their behavior based on the
particular test system they are running on.

Attributes are defined by creating graph-global attributes with an
"ix_" prefix.
2024-05-20 16:21:06 +02:00
Tobias Waldekranz 7d6792a638 test: Always log logical-to-physical mappings
For tests using a logical topology, log the resulting mapping to the
physical one. This way, we always have a record of the entire
environment.
2024-05-20 16:21:06 +02:00
Tobias WaldekranzandJoachim Wiberg 753f3806a1 test: Improve efficiency of logical to physical topology matching
- Present the topologies to networkx as multigraphs, rather than
  expaning each port of the record shape as a separate node.

- Create a topology mapping by locating a subgraph monomorphism of the
  logical topology in the physical ditto.

This cuts down the time of finding a mapping by several orders of
magnitude. Example:

Before:

time python3 test/infamy/topology.py \
     	     test/virt/quad/topology.dot.in \
	     test/infamy/topologies/ring-4-duts.dot >/dev/null

real    13m1,213s
user    13m0,112s
sys     0m0,732s

After:

time python3 test/infamy/topology.py \
     	     test/virt/quad/topology.dot.in \
	     test/infamy/topologies/ring-4-duts.dot >/dev/null

real    0m0,153s
user    0m0,128s
sys     0m0,024s
2024-05-03 14:44:54 +02:00
Mattias Walström c1309da10b infamy: Add support to copy running to startup
Also add support for rebooting unit.
2024-02-23 14:01:23 +01:00
Mattias WalströmandTobias Waldekranz b29df19478 infamy: Add support for devices with other default password 2023-12-01 15:58:27 +01:00
Tobias WaldekranzandJoachim Wiberg 5be61a7013 test/case: Add ietf_system/upgrade
Verifies that RAUC bundles can be installed using NETCONF.

We choose to not add this to the "all" suite for now, as the qeneth
topologies do not run with full disk emulation.

To test locally, configure your "make run" instance to use the UEFI
loader, launch `make run`, and then run:

    make INFIX_TESTS=test/case/ietf_system/upgrade.py test-run
2023-06-30 15:33:10 +02:00
Tobias WaldekranzandJoachim Wiberg 60beca8820 test/infamy: Ensure that environment always has an ltop attribute
This fixes an issue where attaching to a device in an environment
without a logical topology would trigger an exception.
2023-06-30 15:33:10 +02:00
Tobias WaldekranzandJoachim Wiberg aa8e11a58b test/infamy: Create an infamy specific topology object
Create topology objects that are tailored to represent topologies on
the expected format. This let's us make convenient accessors for
things like the controller node, infix devices, getting the ports used
to reach a certain device from another one, etc.
2023-06-14 17:40:16 +02:00
Joachim Wiberg 9e5cf13522 test: debug gh action
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2023-06-09 11:54:23 +02:00
Joachim Wiberg 0da9606644 test: infamy: raise exception if dut mgmt ip cannot be found
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2023-06-09 11:54:23 +02:00
Tobias WaldekranzandJoachim Wiberg 0fc720d48a test/infamy: Initial add of python based test library 2023-06-08 11:10:31 +02:00