Break up the --test parameter into either:
--capture: Write system command output to a local directory for later
consumption by...
--replay: Which can replay a previous capture
Rename --wrap-commands => --cmd-prefix
This should be a non functional change. The idea is to use the topdoc
dir from 9pm when generating the test report.
Signed-off-by: Richard Alpe <richard@bit42.se>
Existing tests relied on manually captured system data from various
points in Infix's history, making them hard to extend as we add new
features.
Move to a new model where we:
To update a unit test:
1. Use Infamy tests to setup a DUT in a known state
2. Use yanger's wrapper mode to capture the system state needed to
produce operational data for a set of YANG models
3. Use cli-pretty to create the expected output for a set of show
commands
When running unit tests, we can then use the captured data to verify
that yanger and cli-pretty works as expected without the need for a
running Infix system - just like before. The difference is that it is
now much easier to capture a new snapshot when the system evolves.
Ensure that all flavors of GRE tunnels can be configured. In addition
to the increased test coverage, it will also be useful as a basis for
collecting representative system state for CLI testing.
Ensure that VLANs can be configured on VLAN filtering bridges. In
addition to the increased test coverage, it will also be useful as a
basis for collecting representative system state for CLI testing.
This is just a first step to add VXLAN tunnels, much more remains.
Implemented right now is as GRE, local, remote and a VNI.
Also refactor gre_basic and gre_bridged to generic tunnel tests,
the same test now tests VXLAN as well.
This is only availible for tests that require paramters. If the test have a
parameter --data the description and test steps can contain {data} which will
be replaced with tha actual data of the parameter data in the test specification.
The suite can also contain meta data for the test specification,
One meta data that is implemented is title, which is used when the
same test code is used twice with parameters.
Also if a test case should not be present in the test specification
for some case. You can add specification: False as meta data.
The yanger source grown significantly, code is disorganized, the
namespace is all over the place, etc.
Take a stab at remedying the situation by splitting out the individual
YANG models in separate Python modules, starting with ietf-system.
Using `sys.argv[0]` does not quite achieve the effect we want, namely
that any (test-case, $PYTHONHASHSEED) tuple should always default to
the same transport.
For example, the following two examples should always use the same
transport:
$ make PYTHONHASHSEED=1337 test-sh
infamy0:test$ ./case/ietf_system/hostname/test.py
$ make PYTHONHASHSEED=1337 test-sh
infamy0:test$ cd ./case/ietf_system/hostname
infamy0:hostname$ ./test.py
But, since their argv[0]'s are different, they don't.
Therefore use the _last two components_ of argv[0]'s full path
instead. This should:
- Spread the allocation over an entire suite run, since the
penultimate component is usually the test name.
- Keep the allocation stable when test code is modified (which is
common during debugging of failing tests)
- Avoid instabilities stemming from the local storage location (i.e.,
home directory names etc.)
The somewhat clumsy construct of resolving a logical port name to its
physical counterpart has been with us from Infamy's inception:
_, physical_port = env.ltop.xlate(logical_node, logical_port)
This is needlessly verbose. Since devices already have access to the
mappings which applies to it, make them easy to access:
target = env.attach("target")
# Get physical port names via the subscript operator
target["data0"]
Basic support GRE and GRETAP tunnels both for IPv4 and IPv6.
Limited support right now, only possible to configure local
ip address and remote ip address.
* Use new cleanup in infamy to ensure the boot order gets restored
* Use new boot order in operational and use RPC to set boot order
to remove all SSH commands.
If a test need to clean up itself disregarding test status,
it can add an optional test-cleanup phase.
This is useful if the test in some circomstances leave the device in a
bad state. This can be used to restore the unit to its ordinary state.
Furl answers the question "is this URL serving the content I expect?"
Sometimes, we might be asking a server that in the middle of being
reconfigured, is crashing, is buggy, etc. - in which case it might RST
the connection. In that case the answer to the question is "False" and
not "BURN IT ALL DOWN!!"
Mattias Walström