Commit Graph
195 Commits
Author SHA1 Message Date
Joachim Wiberg a0a4051c8a test/infamy: fix xpath_to_uri() to handle multiple predicates
The xpath_to_uri() method only processed the first predicate in XPath
expressions with multiple [key='value'] patterns.  Each re.sub() call
was performed on the original xpath instead of the result the previous
substitutions, causing subsequent predicates to be ignored.

Example XPath that would fail:

    /infix-firewall:firewall/zone[name='untrusted']/interface[.='e2']

Would incorrectly convert to:

    /infix-firewall:firewall/zone[name='untrusted']/interface=e2

Instead of the correct RESTCONF URL:

    /infix-firewall:firewall/zone=untrusted/interface=e2

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-28 22:02:58 +02:00
Joachim Wiberg 9d9e099cdb test/infamy: slight improvement to tap.py::Test.__exit__()
Before this change:

  ok 2 - Configure basic end-device firewall
  not ok 3 - Verify unused interface assigned to default zone
  # Exiting (2025-09-25 11:33:00)
  # Traceback (most recent call last):
  #   File "/home/jocke/src/x-misc/test/./case/infix_firewall/basic/test.py", line 127, in <module>
  #     assert unused_if not in public_zone["interface"], \
  #            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  # AssertionError: Unused interface e4 should be in default zone 'public', got interfaces: ['e2', 'e3', 'e5', 'e7', 'e8']

After this change:

  ok 2 - Configure basic end-device firewall
  not ok 3 - Verify unused interface assigned to default zone
  # Exiting (2025-09-25 11:35:00)
  #   File "/home/jocke/src/x-misc/test/./case/infix_firewall/basic/test.py", line 127, in <module>
  #     assert unused_if not in public_zone["interface"], \
  #            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  # Unused interface e4 should be in default zone 'public', got interfaces: ['e2', 'e3', 'e5', 'e7', 'e8']

Slightly shorter and arguably easier to read for a non-pythonic human.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-28 21:35:32 +02:00
Joachim Wiberg a9430070d2 test/infamy: add optional msg support to tap.py::Test.fail()
A very common pattern in our tests is:

  if (condition):
    print(f"the condition failed with {output}")
    test.fail()

This change allows us to write:

  if (condition):
    test.fail(f"the condition failed with {output}")

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-28 21:35:32 +02:00
Joachim Wiberg a3d30eb32f test/infamy: minor, whitespace only
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-28 21:35:31 +02:00
Joachim Wiberg 9fe661f422 test/infamy: silence schema download in restconf backend
NETCONF backedn already silenced.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-15 06:40:38 +02:00
Joachim Wiberg 52beeb5814 test/infamy: refactor Furl class to support list of needles
Also, add some documentation to lower barrier of entry/use.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-15 06:40:35 +02:00
Mattias Walström dce9a9c53d test: Add new sanity check tests
One to verify it is the correct version on the duts and one
that check that it is the correct bootorder.
2025-09-05 16:34:11 +02:00
Mattias Walström 7683466424 test: infamy: Silence reachability check
This was to verbose.
2025-08-26 14:38:42 +02:00
Mattias Walström 8740b2eb23 test: infamy: Add a timeout when starting each test if the DUTs is not reachable
This to cover if you run one test, but the DUTs are still booting.
2025-08-26 14:38:42 +02:00
Ahmed Karic 2cbba5cf1f test: Cleanup topologies 2025-05-13 16:58:46 +02:00
Mattias Walström cfbdce1b90 Change timezone test to check timezone in operational instead of checking time
Libyang is smart, it "fixes" the timezone for you, this was overriden by a
hack in netconf.py and restconf.py, but what you really want is to see
in operational what the current timezone is.
2025-04-30 11:51:45 +02:00
Joachim Wiberg 91f0094048 test: add missing path to container images
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-24 22:31:18 +01:00
Joachim Wiberg 417d48f015 test: update topology to new syntax
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-24 21:39:35 +01:00
Tobias Waldekranz fb394db981 test: lag_*: Ensure that links making up a LAG are compatible
Do not allow the topology matcher to setup a LAG where one link is
running at 10G while the other one runs at 1G, since the bond will
never use the 1G port as long as there the 10G link is up.
2025-02-12 23:22:10 +01:00
Tobias Waldekranz da39855cec test: infamy: Allow tests to refine the topology matching process
To run LAG tests on hardware, we have to make sure that the chosen
links are running at the same speed.  We do not want to require a
specific link speed from the logical topology, as the real requirement
is merely that the two (or however many) links are of the _same_
speed, not any _particular_ speed.

Rather than having to complicate the topology matcher, let it take
care of the common pattern of matching requirements to provided
features, then let tests pass custom node/edge matchers for the
complicated cases.
2025-02-12 23:22:09 +01:00
Joachim WibergandTobias Waldekranz a90a39e8d8 test: infamy: allow easy debugging of differences in config data
Unclear if the libyang merge operation actually does it's job of merging
all the data properly in the restconf backend.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-12 23:21:08 +01:00
Joachim WibergandTobias Waldekranz dc5c7732d7 test: infamy: return None if port doesn't exist
Follow-up to e4535aa, allowing tests to be even simpler.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-12 23:21:08 +01:00
Joachim WibergandTobias Waldekranz 5bf9200880 test: infamy: drop debug message breaking tap output
When first starting up the YANG models are downloaded.  Initially all
were printed on the test console, so this was later changed to become
a sort of progress output.  However, that broke tap output, causing a
missing leading '#' for the final "YANG models downloaded." message.

Thist patch drops the output entirely from the netconf transport, it
was never added to the restconf transport.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-12 23:21:08 +01:00
Joachim WibergandTobias Waldekranz f9b155b49f test: infamy: use decorator to reduce boilerplate for test args
Example use:

    @infamy.test_argument("--mode", help="one of 'static' or 'lacp'")
    class TestArgs(infamy.ArgumentParser):
        pass

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-12 23:21:08 +01:00
Joachim WibergandTobias Waldekranz 9782ac9afa test: update virt topologies, add lag
This commit adds two SVGs for previewing the virtual test topologies.

quad: fix a lot of copy-paste mistakes, rename nodes and ports to their
      correct names.  Replace shell comments with C++ style comments,
      not all tools, e.g., dotty, support shell comments.  The neato
      absolute positioning has also been fixed to make it possible to
      see all the edges, to facilitate this the used ports have been
      swapped around a abit.  Colors are used according to the schema
      agreed on for tests.  Finally, a second link for lag tests have
      been added between dut2 and dut3.

dual: coloring similar to quad, update to new edge attribute matcher.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-12 23:21:08 +01:00
Ahmed Karic af1f173497 test: Verify LLDP admin status 2025-02-12 10:41:46 +01:00
Ahmed Karic b3ad9e5693 test/infamy: add option to set interface up by default
Introduce a parameter in IsolatedMacVlans to set interfaces up by
default. This removes the need to manually bring the interface up
inside the addip function.
2025-01-27 15:48:44 +01:00
Ahmed Karic 3e0b816351 test: Match on edge attributes when mapping topologies
Implemented logic to check if the system supports required services.
Logical edges with 'requires' fields are mapped only if physical edges
provide corresponding services. If not, the test is skipped.
This ensures that tests only run when the required services are available
in the test system configuration.

Fixes #654
2025-01-24 11:51:55 +01:00
Tobias Waldekranz 1d13c5ea9e test: infamy: Generalize netconf_syn() to test for any TCP service
Being able to check for a TCP based service is generally useful, not
only for NETCONF. Therefore, break it out to a separate function that
allows any port to be queried.
2025-01-22 20:16:47 +01:00
Tobias Waldekranz 19c1f49fd3 test: Whitespace cleanup 2025-01-22 15:53:41 +01:00
Mattias Walström b2db59f3bc infamy: Add support for parameters to tests
All test added parameters is required and no default behaviour is allowed.
2025-01-10 21:40:42 +01:00
Tobias Waldekranz f3f313bb98 infamy: Fix default transport selection
Using `sys.argv[0]` does not quite achieve the effect we want, namely
that any (test-case, $PYTHONHASHSEED) tuple should always default to
the same transport.

For example, the following two examples should always use the same
transport:

    $ make PYTHONHASHSEED=1337 test-sh
    infamy0:test$ ./case/ietf_system/hostname/test.py

    $ make PYTHONHASHSEED=1337 test-sh
    infamy0:test$ cd ./case/ietf_system/hostname
    infamy0:hostname$ ./test.py

But, since their argv[0]'s are different, they don't.

Therefore use the _last two components_ of argv[0]'s full path
instead. This should:

- Spread the allocation over an entire suite run, since the
  penultimate component is usually the test name.

- Keep the allocation stable when test code is modified (which is
  common during debugging of failing tests)

- Avoid instabilities stemming from the local storage location (i.e.,
  home directory names etc.)
2025-01-08 10:14:46 +01:00
Tobias Waldekranz e4535aa856 test: infamy: Make it easier to resolve mapped port names
The somewhat clumsy construct of resolving a logical port name to its
physical counterpart has been with us from Infamy's inception:

    _, physical_port = env.ltop.xlate(logical_node, logical_port)

This is needlessly verbose. Since devices already have access to the
mappings which applies to it, make them easy to access:

    target = env.attach("target")

    # Get physical port names via the subscript operator
    target["data0"]
2025-01-08 10:14:46 +01:00
Tobias Waldekranz 343400c5fe test: infamy: Default to attaching to DUTs via the "mgmt" port
This convention is now well established, so it seems silly to have to
repeat it in every test.
2025-01-08 10:14:46 +01:00
Mattias Walström da260fabf9 infamy: restconf: Add support to delete individual entries in leaf-lists 2024-12-19 14:42:17 +01:00
Mattias WalströmandGitHub a8545e3369 Merge pull request #876 from kernelkit/change-boot-order
Change boot order
2024-12-19 11:23:45 +01:00
Tobias WaldekranzandGitHub 1f2973d93f Merge pull request #874 from kernelkit/br-refactor
Bridge refactor
2024-12-19 10:29:05 +01:00
Mattias Walström e00737ef78 test: Refactor upgrade test
* Use new cleanup in infamy to ensure the boot order gets restored
* Use new boot order in operational and use RPC to set boot order
  to remove all SSH commands.
2024-12-19 10:14:15 +01:00
Mattias Walström a90294fe08 test: infamy: Add optional test cleanup
If a test need to clean up itself disregarding test status,
it can add an optional test-cleanup phase.

This is useful if the test in some circomstances leave the device in a
bad state. This can be used to restore the unit to its ordinary state.
2024-12-19 10:14:15 +01:00
Tobias Waldekranz 77215aeb87 infamy: furl: Handle connection resets
Furl answers the question "is this URL serving the content I expect?"

Sometimes, we might be asking a server that in the middle of being
reconfigured, is crashing, is buggy, etc. - in which case it might RST
the connection. In that case the answer to the question is "False" and
not "BURN IT ALL DOWN!!"
2024-12-19 09:33:45 +01:00
Ahmed Karic 03cab6d1e4 Verify MAC multicast
Fixes #649
2024-12-18 14:55:00 +01:00
Mattias Walström 734958b89d test: restconf: Remove extra / when delete xpath 2024-12-11 12:23:02 +01:00
Mattias Walström 397568f1c8 infamy: restconf: Do not percent encode ':' 2024-12-11 10:13:52 +01:00
Mattias Walström a2c3919c0f test: restconf: Remove extra / when getting datastore 2024-12-11 10:13:52 +01:00
Mattias WalströmandGitHub 1db039d6b4 Merge pull request #846 from kernelkit/fix-upgrade-test
Update upgrade test
2024-12-06 00:30:22 +01:00
Mattias Walström 85603eb53d Refactor file_server to run in a netns 2024-12-05 21:24:52 +01:00
Tobias Waldekranz 967388395f test: netns: Avoid infinite hangs on exceptions in .call()s
If the callable threw an exception, then the process running inside
the netns would exit before sending back a value on the tx pipe, which
meant that `rx.recv()` would hang in the original process.

Make sure that we catch any exceptions, and send that instead of the
result over the pipe in those scenarios. Then in the original process,
re-raise the exception.
2024-12-05 13:10:42 +01:00
Joachim Wiberg 997310f9a2 test: fix xpath_delete() over NETCONF
Fixes the following issue when deleting /infix-services:mdns/interfaces,
which works fine over RESTCONF:

Traceback (most recent call last):
  File "/home/jocke/src/infix/test/./case/infix_services/mdns_allow_deny/test.py", line 142, in <module>
    dut.delete_xpath("/infix-services:mdns/interfaces")
  File "/home/jocke/src/infix/test/infamy/netconf.py", line 418, in delete_xpath
    oldd = mod.parse_data_dict(old, no_state=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.infix/venv/lib/python3.11/site-packages/libyang/schema.py", line 215, in parse_data_dict
    return dict_to_dnode(
           ^^^^^^^^^^^^^^
  File "/root/.infix/venv/lib/python3.11/site-packages/libyang/data.py", line 1277, in dict_to_dnode
    result.root().validate(
  File "/root/.infix/venv/lib/python3.11/site-packages/libyang/data.py", line 430, in validate
    self.validate_all(no_state, validate_present)
  File "/root/.infix/venv/lib/python3.11/site-packages/libyang/data.py", line 449, in validate_all
    raise self.context.error("validation failed")
libyang.util.LibyangError: validation failed:
    Invalid leafref value "e3" - no target instance "/if:interfaces/if:interface/if:name" with the same value.

The patch skips validation of the diff in favor of checking the result
of libyang.xpath_del().

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-12-04 12:57:04 +01:00
Joachim Wiberg bf23f53770 test: minor, pep8 fixes
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-12-04 12:57:04 +01:00
Mattias Walström 7334d0e54b test: add test for ntp client 2024-12-03 16:29:28 +01:00
Tobias Waldekranz 6d18ba4c39 test: infamy: Unify {NET,REST}CONF operational data formatting
Replace the existing remove-module-names-pass in the RESTCONF
transport -- which only worked for top-level containers -- with
libyang's standard `.print_dict()` used by the NETCONF transport
implementation.

As a bouns, the implementation of `.get_iface()` is now transport
agnostic.
2024-12-02 12:42:28 +01:00
Ahmed Karic 03a2cf7b3b test/infamy: Initialize DHCP server interface parameter 2024-11-25 18:05:51 +01:00
Ahmed Karic 289534e0d5 test/infamy: Fix adding ip address inside IsolatedMacVlans 2024-11-25 18:05:51 +01:00
Ahmed Karic d81b20ca0b test/infamy: Enhance route verification
- Ensure _exist_route function performs complete checks without early
    returns.
  - Add support for verifying 'active' state in route checks.
2024-11-25 18:05:51 +01:00
Mattias Walström 66ae02363e infamy: Stability fix for getting OSPF status
Always return {} if fail to get OSPF status, it may not have started
yet.
2024-11-25 14:25:26 +01:00