Commit Graph
466 Commits
Author SHA1 Message Date
Richard Alpe 39de797b10 test: specify test specification in top suite
Signed-off-by: Richard Alpe <richard@bit42.se>
2025-01-21 15:02:08 +01:00
Richard Alpe f0c57da4ab test: bump 9pm submodule
Signed-off-by: Richard Alpe <richard@bit42.se>
2025-01-21 15:02:05 +01:00
Richard Alpe 254c922254 test: add git to docker container
Intended to be used when fetching version information for the test
report.

Signed-off-by: Richard Alpe <richard@bit42.se>
2025-01-21 14:00:12 +01:00
Mattias Walström 4782cee201 Add support for simple VXLAN tunnels
This is just a first step to add VXLAN tunnels, much more remains.
Implemented right now is as GRE, local, remote and a VNI.

Also refactor gre_basic and gre_bridged to generic tunnel tests,
the same test now tests VXLAN as well.
2025-01-14 14:31:21 +01:00
Mattias Walström 907401f6f2 generate_spec: Allow to run script manually and show all output 2025-01-14 13:43:59 +01:00
Mattias Walström 9a831217e4 test-specification: Add support for python f-strings in step and description
This is only availible for tests that require paramters. If the test have a
parameter --data the description and test steps can contain {data} which will
be replaced with tha actual data of the parameter data in the test specification.
2025-01-14 10:54:05 +01:00
Joachim WibergandMattias Walström 1aab75d86a test: spec: remove .tmp files when done
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-01-13 20:16:37 +01:00
Mattias Walström 12b6146551 Add new generated specification files 2025-01-10 21:42:10 +01:00
Mattias Walström 29031be4e9 test-specification: Generate test specification from suite file
The suite can also contain meta data for the test specification,
One meta data that is implemented is title, which is used when the
same test code is used twice with parameters.

Also if a test case should not be present in the test specification
for some case. You can add specification: False as meta data.
2025-01-10 21:42:10 +01:00
Mattias Walström b2db59f3bc infamy: Add support for parameters to tests
All test added parameters is required and no default behaviour is allowed.
2025-01-10 21:40:42 +01:00
Tobias Waldekranz 2ec226640e yanger: Move ietf-{interfaces,routing} implementations to separate modules
Should this have been two commits: yes.

Should we spend time on splitting them after the fact because yours
truly dropped the ball: no.
2025-01-10 10:12:22 +01:00
Tobias Waldekranz a0e4e813dd yanger: Move ietf-ospf implementation to separate module 2025-01-10 09:57:40 +01:00
Tobias Waldekranz 8812ae7052 yanger: Move ietf-hardware implementation to separate module 2025-01-10 09:57:40 +01:00
Tobias Waldekranz 28bf2d6c3c yanger: Move infix-containers implementation to separate module 2025-01-09 23:59:20 +01:00
Tobias Waldekranz dadf0e2d16 yanger: Start the modularization process with ietf-system
The yanger source grown significantly, code is disorganized, the
namespace is all over the place, etc.

Take a stab at remedying the situation by splitting out the individual
YANG models in separate Python modules, starting with ietf-system.
2025-01-09 23:59:20 +01:00
Tobias Waldekranz f3f313bb98 infamy: Fix default transport selection
Using `sys.argv[0]` does not quite achieve the effect we want, namely
that any (test-case, $PYTHONHASHSEED) tuple should always default to
the same transport.

For example, the following two examples should always use the same
transport:

    $ make PYTHONHASHSEED=1337 test-sh
    infamy0:test$ ./case/ietf_system/hostname/test.py

    $ make PYTHONHASHSEED=1337 test-sh
    infamy0:test$ cd ./case/ietf_system/hostname
    infamy0:hostname$ ./test.py

But, since their argv[0]'s are different, they don't.

Therefore use the _last two components_ of argv[0]'s full path
instead. This should:

- Spread the allocation over an entire suite run, since the
  penultimate component is usually the test name.

- Keep the allocation stable when test code is modified (which is
  common during debugging of failing tests)

- Avoid instabilities stemming from the local storage location (i.e.,
  home directory names etc.)
2025-01-08 10:14:46 +01:00
Tobias Waldekranz 3ef40031dd test: bridge_stp_basic: Add 2025-01-08 10:14:46 +01:00
Tobias Waldekranz 44b60956a8 test: upgrade: Add .gitignore for package symlink 2025-01-08 10:14:46 +01:00
Tobias Waldekranz e4535aa856 test: infamy: Make it easier to resolve mapped port names
The somewhat clumsy construct of resolving a logical port name to its
physical counterpart has been with us from Infamy's inception:

    _, physical_port = env.ltop.xlate(logical_node, logical_port)

This is needlessly verbose. Since devices already have access to the
mappings which applies to it, make them easy to access:

    target = env.attach("target")

    # Get physical port names via the subscript operator
    target["data0"]
2025-01-08 10:14:46 +01:00
Tobias Waldekranz 343400c5fe test: infamy: Default to attaching to DUTs via the "mgmt" port
This convention is now well established, so it seems silly to have to
repeat it in every test.
2025-01-08 10:14:46 +01:00
Joachim WibergandMattias Walström c099d887af Add support for GRE tunnels
Basic support GRE and GRETAP tunnels both for IPv4 and IPv6.
Limited support right now, only possible to configure local
ip address and remote ip address.
2025-01-07 15:33:37 +01:00
Ahmed KaricandMattias Walström e29a721571 test: move ssh_key_authentication to infix_services 2024-12-19 14:42:17 +01:00
Mattias Walström 389fc96794 test: Verify ssh server config
Fixes #441
2024-12-19 14:42:17 +01:00
Mattias Walström da260fabf9 infamy: restconf: Add support to delete individual entries in leaf-lists 2024-12-19 14:42:17 +01:00
Mattias WalströmandGitHub a8545e3369 Merge pull request #876 from kernelkit/change-boot-order
Change boot order
2024-12-19 11:23:45 +01:00
Tobias WaldekranzandGitHub 1f2973d93f Merge pull request #874 from kernelkit/br-refactor
Bridge refactor
2024-12-19 10:29:05 +01:00
Mattias Walström e00737ef78 test: Refactor upgrade test
* Use new cleanup in infamy to ensure the boot order gets restored
* Use new boot order in operational and use RPC to set boot order
  to remove all SSH commands.
2024-12-19 10:14:15 +01:00
Mattias Walström a90294fe08 test: infamy: Add optional test cleanup
If a test need to clean up itself disregarding test status,
it can add an optional test-cleanup phase.

This is useful if the test in some circomstances leave the device in a
bad state. This can be used to restore the unit to its ordinary state.
2024-12-19 10:14:15 +01:00
Tobias Waldekranz 77215aeb87 infamy: furl: Handle connection resets
Furl answers the question "is this URL serving the content I expect?"

Sometimes, we might be asking a server that in the middle of being
reconfigured, is crashing, is buggy, etc. - in which case it might RST
the connection. In that case the answer to the question is "False" and
not "BURN IT ALL DOWN!!"
2024-12-19 09:33:45 +01:00
Mattias Walström ff8669e735 Add support to set configure with RPC and read it from operational datastore 2024-12-18 17:44:40 +01:00
Ahmed Karic 03cab6d1e4 Verify MAC multicast
Fixes #649
2024-12-18 14:55:00 +01:00
Ahmed Karic dc2ced8e09 test/docker: Add scapy for verifying MAC multicast
'msend' only supports IP multicast
2024-12-16 16:31:40 +01:00
Mattias Walström a4ef38fbee test: upgrade: Add some debug print to see whats happen over time 2024-12-13 20:32:07 +01:00
Mattias Walström 5608e9457d Move operational data about software state from confd => yanger
Statd is now responsible for operational, move last big part
(infix-system-software) to yanger.
2024-12-13 20:32:07 +01:00
Mattias Walström 734958b89d test: restconf: Remove extra / when delete xpath 2024-12-11 12:23:02 +01:00
Mattias Walström 397568f1c8 infamy: restconf: Do not percent encode ':' 2024-12-11 10:13:52 +01:00
Mattias Walström a2c3919c0f test: restconf: Remove extra / when getting datastore 2024-12-11 10:13:52 +01:00
Mattias WalströmandGitHub 1db039d6b4 Merge pull request #846 from kernelkit/fix-upgrade-test
Update upgrade test
2024-12-06 00:30:22 +01:00
Mattias Walström f023304823 Update upgrade test to not mess up boot order
Works on both virtual and physical target
2024-12-05 21:24:52 +01:00
Mattias Walström 85603eb53d Refactor file_server to run in a netns 2024-12-05 21:24:52 +01:00
Tobias Waldekranz 967388395f test: netns: Avoid infinite hangs on exceptions in .call()s
If the callable threw an exception, then the process running inside
the netns would exit before sending back a value on the tx pipe, which
meant that `rx.recv()` would hang in the original process.

Make sure that we catch any exceptions, and send that instead of the
result over the pipe in those scenarios. Then in the original process,
re-raise the exception.
2024-12-05 13:10:42 +01:00
Tobias Waldekranz d16ad7eedc build: Define build id and version in one place
Before this change, setting `GIT_VERSION` in `make`'s environment was
intended to allow the user to specify a custom build id.  As it turns
out, `test/test.mk` had duplicated the logic from
`board/common/post-build.sh` to unconditionally override any value set
in it.  Because of the way `make` handles variables, where assignments
to variables inherited from the environment are exported back to
it[^1], this would mean that `test.mk` would always clobber any value
set by the user.  Furthermore, there is also this file called
`buildroot/package/git/git.mk`, which also (reasonably) has opinions
about what the proper value of `GIT_VERSION` should be.  In summary,
this was a bit of a mess.

Therefore: Make sure that there is one single place where we determine
the build id and version, and make sure that those variables are
scoped under the `INFIX_` prefix to avoid clashing with any other
component.

[^1]: https://www.gnu.org/software/make/manual/html_node/Environment.html
2024-12-05 10:29:21 +01:00
Tobias Waldekranz e635e3e720 test: Remove dead code 2024-12-04 23:00:19 +01:00
Joachim Wiberg 3576450982 test: verify mDNS allow/deny interfaces
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-12-04 12:57:04 +01:00
Joachim Wiberg bb987cfe9e test: minor, spelling
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-12-04 12:57:04 +01:00
Joachim Wiberg 997310f9a2 test: fix xpath_delete() over NETCONF
Fixes the following issue when deleting /infix-services:mdns/interfaces,
which works fine over RESTCONF:

Traceback (most recent call last):
  File "/home/jocke/src/infix/test/./case/infix_services/mdns_allow_deny/test.py", line 142, in <module>
    dut.delete_xpath("/infix-services:mdns/interfaces")
  File "/home/jocke/src/infix/test/infamy/netconf.py", line 418, in delete_xpath
    oldd = mod.parse_data_dict(old, no_state=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.infix/venv/lib/python3.11/site-packages/libyang/schema.py", line 215, in parse_data_dict
    return dict_to_dnode(
           ^^^^^^^^^^^^^^
  File "/root/.infix/venv/lib/python3.11/site-packages/libyang/data.py", line 1277, in dict_to_dnode
    result.root().validate(
  File "/root/.infix/venv/lib/python3.11/site-packages/libyang/data.py", line 430, in validate
    self.validate_all(no_state, validate_present)
  File "/root/.infix/venv/lib/python3.11/site-packages/libyang/data.py", line 449, in validate_all
    raise self.context.error("validation failed")
libyang.util.LibyangError: validation failed:
    Invalid leafref value "e3" - no target instance "/if:interfaces/if:interface/if:name" with the same value.

The patch skips validation of the diff in favor of checking the result
of libyang.xpath_del().

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-12-04 12:57:04 +01:00
Joachim Wiberg bf23f53770 test: minor, pep8 fixes
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-12-04 12:57:04 +01:00
Mattias Walström a74eb58000 test: Add unit test for 'show cli' 2024-12-03 16:29:28 +01:00
Mattias Walström 7334d0e54b test: add test for ntp client 2024-12-03 16:29:28 +01:00
Tobias Waldekranz 784b391467 test: reproducible: Log software versions
Make sure that we keep a record of the versions and boot source used
by all devices under test when running the full suite - in case we
ever suspect that there is a mismatch between the intended test image
and the actual one.

Since this test now needs to communicate with the devices, make sure
that we run the wait test first.
2024-12-02 12:42:28 +01:00