Let's drop the leading IETF or Infix prefixes from tests. Initially the
idea was to mimnic the YANG models, but it's difficult to navigate and
does not provide any real benefit to developers or end-users.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
To reduce the duplication of effort between 9pm and the Infamy framework
this change consolidates the move from local 'infamy: title:' extension
to 9pm 'name:'.
For the parameterized tunnel tests we leverage the 9pm dynamic test-spec
variable, which looks for a correspodning <case>.adoc instead of static
Readme.adoc, when generating the test report.
Each test documentation should cover all aspects of the test, much like
the usage text of a UNIX program. To this end, the tunnel test docs are
now more spelled out, including all invariants.
Some refactoring of these tests were also necessary, e.g., replacing any
reserved Python keywords like 'type', and other PEP-8 fixes.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The suite can also contain meta data for the test specification,
One meta data that is implemented is title, which is used when the
same test code is used twice with parameters.
Also if a test case should not be present in the test specification
for some case. You can add specification: False as meta data.
Make sure that we keep a record of the versions and boot source used
by all devices under test when running the full suite - in case we
ever suspect that there is a mismatch between the intended test image
and the actual one.
Since this test now needs to communicate with the devices, make sure
that we run the wait test first.
Due to the salting being applied to Python's `hash()` (see [1]), the
hash of an object is not stable across different Python processes. As
a result, neither are the topology mappings generated by networkx.
The upside of this is randomness that we get better test coverage over
time, especially on physical devices where the underlying hardware
could differ between ports, for example.
On the other hand, it can be very frustrating to track down a bug when
locial nodes are suffled around between phyical nodes on each
invocation of a test case.
Therefore, use a random seed by default, but allow the user to specify
a fixed value if they so choose. We then add a meta test that logs the
seed being used for the current suite - which means we can simply
copy&paste that value to rerun the suite (or single test) with the
same topology mappings.
[1]: https://docs.python.org/3/reference/datamodel.html#object.__hash__
- New helper class for container testing
- New helper class to urllib, Furl
Due to extremely weak Python-fu in the undersigned, this patch changes
the __init__.py file to add new helper classes for container tests.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This test use NETCONF to enable/disable SSDP, it then verifies that the
DUT responds to, or does not respond to, SSDP discover messages.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When running in the cloud, without KVM emulation, together with many
other tenants, it can take a very long time to boot the instances.
Therefore, add a meta test that will wait for all devices in the
physical topology to come up before exiting. This means that the
actual test cases that follow can assume that devices will answer
NETCONF requests.
