test,doc,board: Infamy Made Easy

As of 4768cae6, we can use conserver/telnet to connect to the docker0 bridge IP:90XX, where XX is DUTXX. This was a great UX improvement and this patch set aims to further the experience by: - allow running Infamy (infix-test container) completely rootless¹ - reduce the expsed port range 50->10 (can be improved further) - use 'podman --publish-all' ports, which allocates ten random ports for the exposed Qemu telnet ports - add Quick Start Guide to doc/testing.md This restores the possiblity of running multiple "make test-sh" instances, e.g., when multiple users share the same server. The 'console' script included in this commit uses 'podman inspect' to find the port number for a given DUT, and optional instance. It takes either the console/dut number (1-N), or the dut name from the topology. Also in this commit: - set hostname for easy identification (for console script) - set conatainer --name to hostname - adjust workdir from buildroot to infix/test - simplify PS1 and add time to prompt (when did the test finish?) When running 'make test-sh' the prompt now tells you which instance you are running, e.g., infmay10. Calling 'console 1 infamy10' connects to console 1 (port 9001) on the infamy10 instance. For more information, see the Quick Start Guide in doc/testing.md. It shows how to use the new helper scripts: console and shell. Fix #227 _______ ¹) devs using podman, instead of docker, with slirp4netns installed, can run the infix-test container completely rootless. We like this, and as of today podman is our recommendation. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2026-07-22 01:13:00 +02:00 · 2024-02-25 19:49:27 +01:00
parent 00c2c0c608
commit 5386ec9a3a
8 changed files with 371 additions and 69 deletions
@@ -22,12 +22,14 @@ $(config):
 	@echo "'make <board>_defconfig' before building an image."
 	@exit 1

-
 %: | buildroot/Makefile
 	@+$(call bmake,$@)

+# Workaround, see board/x86_64/board.mk
+test:
+	@+$(call bmake,$@)
+
 buildroot/Makefile:
 	@git submodule update --init

-.PHONY: all check
-
+.PHONY: all check test
@@ -10,16 +10,14 @@ test-env = $(test-dir)/env \
 test-env-qeneth = $(call test-env,-q $(test-dir)/virt/quad,$(1))
 test-env-run    = $(call test-env,-C -t $(BINARIES_DIR)/qemu.dot,$(1))

-.PHONY: test-%
-
 test-unit:
 	$(test-dir)/env $(test-dir)/9pm/9pm.py $(UNIT_TESTS)

-test-qeneth:
+test test-qeneth:
 	$(call test-env-qeneth,\
 		$(BR2_EXTERNAL_INFIX_PATH)/test/9pm/9pm.py \
 			$(INFIX_TESTS))
-test-qeneth-sh:
+test-sh test-qeneth-sh:
 	$(call test-env-qeneth,/bin/sh)

 test-run: | ~/.infix-test-venv
@@ -34,3 +32,5 @@ test-run-play: | ~/.infix-test-venv

 ~/.infix-test-venv:
 	$(test-dir)/docker/init-venv.sh $(test-dir)/docker/pip-requirements.txt
+
+.PHONY: test-unit test test-sh test-qeneth test-qeneth-sh test-run test-run-sh test-run-play
@@ -11,11 +11,18 @@ traffic at various points.
 TL;DR
 -----

-    make x86_64_defconfig
-    make
-    make test-qeneth
+Build Infix and run the test suite on a set of virtual Infix nodes.

-Runs the test suite on a set of virtual Infix nodes.
+    $ make x86_64_defconfig
+    $ make
+    $ make test
+
+To run a subset of tests, e.g., only the DHCP client tests:
+
+    $ make test INFIX_TESTS=case/infix_dhcp/all.yaml
+
+> **Note:** see the below section [Quick Start Guide][] for how to
+> connect to the test system, debug and develop tests, and more.


 Tenets
@@ -121,15 +128,17 @@ against an Infix image started from `make run`. When the instance is
 running, you can open a separate terminal and run `make test-run`, to
 run the subset of the test suite that can be mapped to it.

-Both `test-qeneth` and `test-run` targets have a respective target
-with a `-sh` suffix.  These can be used to start an interactive
-session in the reproducible environment, which is usually much easier
-to work with during a debugging session.
+Both `make test` and `make test-run` targets have a respective target
+with a `-sh` suffix.  These can be used to start an interactive session
+in the reproducible environment, which is usually much easier to work
+with during a debugging session.

-Inside of the reproducible environment, a wrapper for Qeneth is
-automatically created that will run it from the running network's
-directory.  E.g., running a plain `qeneth status` inside a `make
-test-qeneth-sh` environment will show the expected status information.
+Inside the reproducible environment, a wrapper for Qeneth is created for
+the running network's directory.  E.g., calling `qeneth status` inside a
+`make test-sh` environment show the expected status information.
+
+> **Note:** for more information and help writing a test, see the below
+> [Quick Start Guide][].


 Physical and Logical Topologies
@@ -274,6 +283,133 @@ it offline, enable STP on all DUTs, and then verify that the resulting
 spanning tree matches the expected one.


+Quick Start Guide
+-----------------
+
+When developing a test, instead of blindly coding in Python and running
+`make test` over and over, start a test shell:
+
+    $ make test-sh
+    Info: Generating topology
+    Info: Generating node YAML
+    Info: Generating executables
+    Info: Launching dut1
+    Info: Launching dut2
+    Info: Launching dut3
+    Info: Launching dut4
+    11:42:52 infamy0:test #
+
+It takes a little while to start up, but then we have a shell prompt
+inside the container running Infamy.  It's a very limited environment,
+but it has enough to easily run single tests, connect to the virtual
+devices, and *step* your code.  Let's run a test:
+
+    11:42:53 infamy0:test # ./case/infix_dhcp/dhcp_basic.py
+
+### Connecting to Infamy
+
+The test system runs in a Docker container, so to get a shell prompt in
+*another terminal* you need to connect to that container.  Infamy comes
+with a helper script for this:
+
+    $ ./test/shell
+
+By default it connect to the latest started Infamy instance.  If you for
+some reason run multiple instances of Infamy the `shell` script takes an
+optional argument "system", which is the hostname of the container you
+want to connect to:
+
+    $ ./test/shell infamy2
+
+### Connecting to a DUT
+
+All DUTs in a virtual Infamy topology are emulated in Qemu instances
+managed by qeneth.  If you want to watch what's happening on one of the
+target systems, e.g., tail a log file or run `tcpdump` during the test,
+there is another helper script in Infamy for this:
+
+    $ ./test/console 1
+    Trying 127.0.0.1...
+    Connected to 127.0.0.1.
+    
+    Infix — a Network Operating System v23.11.0-226-g0c144da-dirt (console)
+    infix-00-00-00 login: admin
+    Password:
+    .-------.
+    |  . .  | Infix -- a Network Operating System
+    |-. v .-| https://kernelkit.github.io
+    '-'---'-'
+    
+    Run the command 'cli' for interactive OAM
+    
+    admin@infix-00-00-00:~$
+
+From here we can observe `dut1` freely while running tests.
+
+> The `console` script uses `telnet` to connect to a port forwarded to
+> `localhost` by the Docker container.  To exit Telnet, use Ctrl-] and
+> then 'q' followed by enter.  This can be customized in `~/.telnetrc`
+
+Like the `shell` script, `console` takes an optional "system" argument
+in case you run multiple instances of Infamy:
+
+    $ ./test/console 1 infamy2
+
+You can also connect to the console of a DUT from within a `shell`:
+
+    $ ./test/shell
+    11:42:54 infamy0:test # qeneth status
+    11:42:54 infamy0:test # qeneth console dut1
+    login: admin
+    password: *****
+    admin@infix-00-00-00:~$
+
+> **Note:** disconnect from the qeneth console by using bringing up the
+> old Telnet "menu" using Ctrl-], compared to standard Telnet this is
+> the BusyBox version so you press 'e' + enter instead of 'q' to quit.
+
+
+### Debug a Test
+
+First, add a Python `breakpoint()` to your test and run it from your
+`make test-sh` Infamy instance:
+
+    11:42:58 infamy0:test # ./case/infix_dhcp/dhcp_basic.py
+    # Starting (2024-02-10 11:42:59)
+    # Probing dut1 on port d1a for IPv6LL mgmt address ...
+    # Connecting to mgmt IP fe80::ff:fe00:0%d1a:830 ...
+    ok 1 - Initialize
+    > /home/jocke/src/infix/test/case/infix_dhcp/dhcp_basic.py(44)<module>()
+    (Pdb)
+
+You are now in the Python debugger, Pdb.
+
+### Debug a Test in a MacVLAN
+
+Most tests use some sort of network connection to the DUTs.  From a test
+shell you cannot see or debug that connection by default.  So you need
+to employ a little trick.
+
+Open another terminal, and start a test shell in your Infamy instance:
+
+    $ ./test/shell
+    11:43:19 infamy0:test #
+
+MacVLAN interfaces created by Infamy run in another network namespace,
+to enter it we can look for a sleeper process:
+
+    11:43:20 infamy0:test # nsenter -n U -t $(pidof sleep infinity)
+
+You are now one step further down:
+
+    infamy0:/home/jocke/src/infix/test# ip -br a
+    lo               UNKNOWN        127.0.0.1/8 ::1/128
+    iface@if7        UP             10.0.0.1/24 fe80::38d0:88ff:fe77:b7cd/64
+
+You can now freely debug the network activity of your test and the
+responses from the DUT.
+
+
 [9PM]:    https://github.com/rical/9pm
 [Qeneth]: https://github.com/wkz/qeneth
 [TAP]:    https://testanything.org/
@@ -0,0 +1,35 @@
+# Set $testdir before sourcing these envs -*-shell-script-*-
+# shellcheck disable=SC2034,SC2154
+
+# Current container image
+INFIX_TEST=ghcr.io/kernelkit/infix-test:1.2
+
+ixdir=$(readlink -f "$testdir/..")
+logdir=$(readlink -f "$testdir/.log")
+envdir="$HOME/.infix-test-venv"
+qeneth="$testdir/qeneth/qeneth"
+
+#
+# Figure out available container runner
+#
+runner()
+{
+    if which podman >/dev/null; then
+	runner=podman
+    elif which docker >/dev/null; then
+	runner=docker
+    else
+	echo "Error: Neither podman or docker is installed, we recommend podman."
+	exit 1
+    fi
+
+    echo "$runner"
+}
+
+#
+# Returns the latest started infamy container
+#
+infamy()
+{
+    podman ps -f name='infamy.*' --format '{{.Names}}' |tail -1
+}
@@ -0,0 +1,86 @@
+#!/bin/sh
+# Attach to console of a Qemu device (DUT) running in Infamy
+#
+# You can override Ctrl-] in your ~/.telnetrc
+#
+#     DEFAULT
+#     	environ define USER root
+#     	set escape "^X"
+#
+# If you're unhappy with your telnet experience, Debian/Ubuntu
+# systems have two telnet packages, Netkit and GNU Inetutils.
+#
+testdir=$(dirname "$(readlink -f "$0")")
+. "$testdir/.env"
+
+usage()
+{
+    echo "Usage:"
+    echo "       console DUT [SYSTEM]"
+    echo "Example:"
+    echo "       console 1                  # attach to DUT1 on infamy0"
+    echo "       console dut3 infamy1       # attach to 'dut2' on infamy1"
+    echo "       console cisco2 infamy2     # attach to DUT 'cisco2' on infamy2"
+    exit 1
+}
+
+topo()
+{
+    dir=$(podman inspect "$1" 2>/dev/null | jq -re '.[].Args | index("-q") as $index | .[$index+1]')
+    dot="$dir/topology.dot.in"
+    [ -f "$dot" ] && echo "$dot"
+}
+
+list()
+{
+    dot=$(topo "$1")
+    if [ -z "$dot" ]; then
+	echo ", cannot even find $1"
+	return
+    fi
+
+    echo ", available DUTs:"
+    gvpr 'N [$.qn_console] { print($.name); }' "$dot"
+}
+
+find()
+{
+    dot=$(topo "$2")
+    [ -n "$dot" ] || return
+
+    gvpr '
+        N [$.name == "'"$1"'"] {
+            printf("%s\n", $.qn_console);
+        }
+	' "$dot"
+}
+
+dut=$1
+if [ -z "$dut" ]; then
+    usage
+    exit 1
+fi
+
+sys=$2
+[ -n "$sys" ] || sys=$(infamy)
+
+if test "$dut" -eq "$dut" 2>/dev/null; then
+    port=$((9000 + dut))
+else
+    port=$(find "$dut" "$sys")
+    if [ -z "$port" ]; then
+	printf "Cannot find port for %s on %s" "$dut" "$sys"
+	list "$sys"
+	exit 1
+    fi
+fi
+
+portmap=$($(runner) port "$sys" $port)
+port=${portmap#0.0.0.0:}
+
+if [ -z "$port" ]; then
+    echo "Cannot find DUT$dut telnet port $port on $sys (port map $portmap)"
+    exit 1
+fi
+
+telnet 127.0.0.1 "$port"
@@ -1,5 +1,5 @@
-Checklist for Updating Docker Image
-===================================
+Checklist for Updating Infamy Docker Image
+==========================================

 This directory holds the Dockerfile and any extras needed to build and
 update the infix-test container image used for the test system.
@@ -18,7 +18,7 @@ the image, e.g., with missing Alpine packages.

        docker build -t ghcr.io/kernelkit/infix-test:0.4 .

- 3. Update the `test/env` file to use the new version
+ 3. Update the `test/.env` file to use the new version
 4. Verify your new image works properly (remember to remove your `~/.infix-test-venv`)
 5. Send PR to co-maintainer for review

@@ -1,35 +1,8 @@
 #!/bin/sh
-
 set -e

-INFIX_TEST=ghcr.io/kernelkit/infix-test:1.2
-
-start_topology()
-{
-    local qenethdir="$1"
-    local imgfile="$2"
-
-    [ "$qenethdir" ] || { true && return; }
-
-    rm -rf $envdir/qeneth
-    cp -a "$qenethdir" $envdir/qeneth
-    if [ "$imgfile" ]; then
-	local imgname=$(basename "$imgfile")
-	imgfile=$(readlink -f "$imgfile")
-
-	ln -sf "$imgfile" $envdir/qeneth/"$imgname"
-    fi
-
-    (cd $envdir/qeneth/ && $qeneth generate && $qeneth start)
-    INFAMY_ARGS="$INFAMY_ARGS $envdir/qeneth/topology.dot"
-
-    cat <<EOF >"$envdir/bin/qeneth"
-#!/bin/sh
-set -x
-cd $envdir/qeneth && exec $testdir/qeneth/qeneth "\$@"
-EOF
-    chmod +x "$envdir/bin/qeneth"
-}
+testdir=$(dirname "$(readlink -f "$0")")
+. "$testdir/.env"

 usage()
 {
@@ -74,11 +47,58 @@ usage: test/env [<OPTS>]    -f <IMAGE> -q <QENETH-DIR> <COMMAND> [<ARGS>...]
 EOF
 }

-testdir=$(dirname $(readlink -f "$0"))
-ixdir=$(readlink -f "$testdir/..")
-logdir=$(readlink -f "$testdir/.log")
-envdir="$HOME/.infix-test-venv"
-qeneth="$testdir/qeneth/qeneth"
+start_topology()
+{
+    qenethdir="$1"
+    imgfile="$2"
+
+    [ "$qenethdir" ] || { true && return; }
+
+    rm -rf "$envdir/qeneth"
+    cp -a "$qenethdir" "$envdir/qeneth"
+    if [ "$imgfile" ]; then
+	imgname="$(basename "$imgfile")"
+	imgfile=$(readlink -f "$imgfile")
+
+	ln -sf "$imgfile" "$envdir/qeneth/$imgname"
+    fi
+
+    (cd "$envdir/qeneth/" && $qeneth generate && $qeneth start)
+    INFAMY_ARGS="$INFAMY_ARGS $envdir/qeneth/topology.dot"
+
+    cat <<EOF >"$envdir/bin/qeneth"
+#!/bin/sh
+set -x
+cd $envdir/qeneth && exec $testdir/qeneth/qeneth "\$@"
+EOF
+    chmod +x "$envdir/bin/qeneth"
+}
+
+name()
+{
+    nm=infamy
+    id=0
+
+    names=$(podman ps -f name='infamy.*' --format '{{.Names}}')
+    while true; do
+	name="$nm$id"
+	unset hit
+	for n in $names; do
+	    if [ "$name" = "$n" ]; then
+		hit=true
+		break;
+	    fi
+	done
+
+	if [ -n "$hit" ]; then
+	    id=$((id + 1))
+	    continue
+	fi
+	break;
+    done
+
+    echo "$name"
+}

 # Global options
 containerize=yes
@@ -107,31 +127,33 @@ while getopts "Cf:hKp:q:t:" opt; do
 	t)
 	    topology="$OPTARG"
 	    ;;
+	*)
+	    >&2 echo "Unknown option -$opt"
+	    usage
+	    exit 1
+	    ;;
    esac
 done

 if [ "$containerize" ]; then
-    if which podman >/dev/null; then
-	runner=podman
-    elif which docker >/dev/null; then
-	runner=docker
-    else
-	echo "Error: Neither podman nor docker is installed"
-	exit 1
-    fi
-
-    exec $runner run \
+    # shellcheck disable=SC2016
+        exec $(runner) run \
 	 --cap-add=NET_ADMIN \
 	 --device=/dev/net/tun \
-	 --expose 9000-9050 \
+	 --env VIRTUAL_ENV_DISABLE_PROMPT=yes \
+	 --env PROMPT_DIRTRIM="$ixdir" \
+	 --env PS1='\e[1m$(date "+%H:%M:%S") \h\e[0m:\W # ' \
+	 --expose 9001-9010 --publish-all \
+	 --hostname "$(name)" \
+	 --name "$(name)" \
 	 --interactive \
 	 --rm \
 	 --security-opt seccomp=unconfined \
 	 --sysctl net.ipv6.conf.all.disable_ipv6=0 \
 	 --tty \
 	 --volume "$ixdir":"$ixdir" \
-	 --workdir $(pwd) \
-	 $kvm \
+	 --workdir "$ixdir/test" \
+	 "$kvm" \
 	 $INFIX_TEST \
 	 "$0" -C "$@"
 else
@@ -150,6 +172,7 @@ if [ $# -lt 1 ]; then
    usage && exit 1
 fi

+# shellcheck disable=SC1091
 . "$envdir/bin/activate"
 export PYTHONPATH="$testdir"
 INFAMY_ARGS="$INFAMY_ARGS -y $envdir/yangdir"
@@ -0,0 +1,20 @@
+#!/bin/sh
+# Enter shell in container running Infamy
+
+testdir=$(dirname "$(readlink -f "$0")")
+. "$testdir/.env"
+
+usage()
+{
+    echo "Usage:"
+    echo "       shell [SYSTEM]"
+    echo "Example:"
+    echo "       shell              # Open shell in infamy0"
+    echo "       shell infamy2      # Open shell in infamy2"
+    exit 1
+}
+
+sys=$1
+[ -n "$sys" ] || sys=$(infamy)
+
+$(runner) exec -it --workdir "$ixdir/test" "$sys" ./env -C sh