Simplify the code a bit by passing the test file to the runner
function instead of checking it in every caller.
Signed-off-by: Richard Alpe <richard@bit42.se>
As of 4768cae6, we can use conserver/telnet to connect to the docker0
bridge IP:90XX, where XX is DUTXX. This was a great UX improvement
and this patch set aims to further the experience by:
- allow running Infamy (infix-test container) completely rootless¹
- reduce the expsed port range 50->10 (can be improved further)
- use 'podman --publish-all' ports, which allocates ten random
ports for the exposed Qemu telnet ports
- add Quick Start Guide to doc/testing.md
This restores the possiblity of running multiple "make test-sh"
instances, e.g., when multiple users share the same server.
The 'console' script included in this commit uses 'podman inspect' to
find the port number for a given DUT, and optional instance. It takes
either the console/dut number (1-N), or the dut name from the topology.
Also in this commit:
- set hostname for easy identification (for console script)
- set conatainer --name to hostname
- adjust workdir from buildroot to infix/test
- simplify PS1 and add time to prompt (when did the test finish?)
When running 'make test-sh' the prompt now tells you which instance you
are running, e.g., infmay10. Calling 'console 1 infamy10' connects to
console 1 (port 9001) on the infamy10 instance.
For more information, see the Quick Start Guide in doc/testing.md. It
shows how to use the new helper scripts: console and shell.
Fix#227
_______
¹) devs using podman, instead of docker, with slirp4netns installed, can
run the infix-test container completely rootless. We like this, and
as of today podman is our recommendation.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
- New helper class for container testing
- New helper class to urllib, Furl
Due to extremely weak Python-fu in the undersigned, this patch changes
the __init__.py file to add new helper classes for container tests.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
- Fix most of what pylint complains about
- Simplify constructs and add missing whitespace
- Simplify output of YANG model downloader
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This patch handles a corner case we for some reason have not run into
yet. With the new container operational data it was triggered almost
immediately, so let's add some basic guards around it and return None
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Add a helper that lets you call a Python function (or lambda) in the
context of a network samespace.
Example:
with infamy.IsolatedMacVlan("eth0") as ns:
res = ns.call(lambda: subprocess.run(["ip", "link"],
capture_output=True))
print(res.stdout, f"\n[exitcode:{res.returncode}]")
The call to subprocess.run will be executed in the context of the
network namespace, and thus only list "lo" and the "iface"
macvlan.
The return value of the function passed to ns.call() is passed back
over a multiprocessing.Pipe, which requires that the object be
"picklable". This is true for most objects (even more complex ones
like the CompletedProcess object seen in the example above). Some
notable exceptions are things like file objects, sockets, etc.
The setns(2) syscall is unfortunately not available in current
versions of Python shipped with neither Ubuntu nor Alpine at the time
of this writing. Therefore, shoot from the hip, assume that we're
running on an x86_64 CPU, and just hotwire the syscall directly with
some constants we found in a dumpster.
This is an example of how to create a package that downloads an OCI
archive and installs it to /lib/oci
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This patch adds the optional 'user name' argument to the 'copy' command,
enabling using curl scp/sftp protocols for fetching and uploading files.
Changes to cfg_adjust() allow saving and referencing files in the user's
home directory. Useful for both fetching upgrade bundles and container
images.
The 'dir' command now lists files in both $HOME and /cfg.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This patch adds a new klish symbol, shell@infix, which performs proper
droprivs before calling the configured shell.
Fix#298: track user id so shell command gets correct user
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Some customers don't want unprivileged CLI users to be able to exit to
a shell. The shell is currently hard-coded to bash, but should be one
of the ietf-system shells, configurable --with-shell=foo.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This refactor is basically only an extra step after
copy source-ds -> dest-ds
to also
copy source-file -> dest-file
Which in the case of factory-config and startup-config is relevant. We
want to update the startup datastore, in case of additional commands,
but also the file /cfg/startup-config.cfg in case of consecutive reboot.
Fix#259 copy factory-config startup-config
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Sending SIGTERM to conmon is not a safe shutdown of a podman container.
To handle gracefully handle shutdown, restarting and provide an orderly
start of dependencies, we use the Finit sysv trick via container script
wrapper to call 'podman stop foo'.
However, since podman does not support syslog as output for containers
we employ an old FIFO trick with another program, k8s-logger, to allow
logs to reach syslog. Please note that k8s-logger must have properly
started before we call `podman start` -- this makes us fully dependent
on the 'container' wrapper script. Hence the documentation update.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Because sysrepo callbacks are threaded and factory-default RPC is called
from a separate subscription (to prevent blocking), we cannot prevent
ietf-intefaces.c from being called before infix-containers.c, regardless
of the priority we set for our subscriptions.
When assigning a physical network interface this becomes a bit of a pain
during factory-default RPC since the physical interface is hidden from
the host network namespace.
So, when applying factory to running, we check each interface if it was
a container-network previously, if so we call on the container script in
the exit of the current dagger generation to move the interface back to
the host netns.
This affects all other functions that assume interfaces only live in the
host netns. To that end a set of new helper functions have been added
to wrap iproute2 commands in nsenter when the interface lives elsewhere.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The interface name is already a unique qualifier, so we can simplify CNI
network naming to ease the burden when debugging.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
"Sloppy" because, for some reason, we don't get sysrepo update callbacks
for presence containers. I.e., when calling 'set container-network'
compared to 'edit container-network'.
Yes, I've gone over klish-plugins-sysrepo with a fine-toothed comb to
see if it's the culprit, but no, it seems to be sysrepo. Hence the
defaulting of the type also in the change path.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Review found quite a few strings that could be given an obvious pattern
or specialized sub-type, e.g. inet:ip-address, to restrict the input
data validation.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Due to the deprecation of CNI i Podman[1], and its upcoming removal in
Podman v5.0, we decided to cut our losses and modify all documentation
and models to become backend neutral. We will continue to use CNI as
long as possible -- it gives us host interfaces, for instance, that is
not part of netawark yet, it seems.
Also, sync mount and volume documentation with YANG model changes.
[1]: https://blog.podman.io/2023/11/cni-deprecation-and-removal-from-podman-5-0/
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
After discussions in the development team, and an external demo, we
decided to rename file mounts to just mounts and adopt established
nomenclature.
With this change we now allow mounting any type of file or directory
from the host to the container, for regular files/directories a bind
mount is used. We also introduce 'source' and 'type', to support
mounting of host files and directories, the destination path has been
renamed to 'target'. The mount cousin, 'volume', which basically is a
specialized mount, has also been updated to use 'target'.
The 'source' and 'content' are now hidden behind a YANG choice to ensure
users can only use them the intended way (either/or not both).
Initial models had 'destination' (path) as mandatory, with this change
we can reintroduce that to ensure this proper configurations.
The documentation in both YANG model and markdown has been updated,
significantly, to provide lots of examples.
Please note, glob style mounts have not been tested, at all.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When container configurations are not saved to startup-config and the
user reboot the system (or lose power) we will have lingering active
containers cached on persistent storage.
This patch adds a cleanup() function to the post hook, right before
we try to launch any new containers. This callback must be run for
all cases, even if the running-config has no containers configured.
Also, a minor amendment to volume pruning.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
1. add support for extracting, and finding the directory holding the
index.json metadata file in, OCI images. An archive in Infix is either
a .tar or .tar.gz archive, which we need to unpack for this version of
podman to be able to load them.
2. new RPC '/infix-containers:oci-load' calls 'container load' of an
OCI archive (tarball), optionally gzipped. The resulting image uses
the directory name of the unpacked tarball, so the container script
offers a way to retag the image after loading it.
First class citizens in container transport are docker:// and the OCI
family of URI:s. A docker:// URL, or a local docker-archive:path, is
assumed to be well formed, in which case we leave it up to podman to
handle.
Note: 'podman import' does not fully understand OCI formats. It drops
ENTRYPOINT and COMMAND, while 'podman load' handles things a lot
better. Only letdown is it does not support nameing the image.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Not all customers want Container support in their Ínfix builds. This
patch adds build-time support for detecting if podman is enabled in
the configuration.
- Klish container commands have been broken out to their own xml file
- infix-interfaces have been given a 'containers' feature
- ietf-interfces.c enables 'containers' feature on an #ifdef
- infix-container.c is only built if --enable-containers
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
- Rename container/container -> containers/container
- Collapse containers-state into containers
- Complete refactor of network/host-network for improved UX. A new
must expression ensure mutual exclusion between host and networks
- Rename entrypoint -> command, for consistency (overrides ENTRYPOINT)
- Move nodes around a bit for more logical placement in model
- Clean out all traces of a container on removal
- Rename container image_id -> image-id for consistency
The active queue is for tracking currently running containers, ensuring
only relevant changes are applied, and also for container image upgrade,
at the end of which these scripts are re-run to activate the new image
Also, fix misplaced '-f' when pruning unused volumes.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Brief overview of configuration support. Need to add admin-exec level
commands, but since operational and actions still are missing, we keep
it out until after the code review.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
- drop debug logs
- no need to restart upgraded containers, they restart automatically
- fix ordering in volume prune (when containers are removed)
- skip directories in container activation (inbox -> execd queue)
- Fix 'container pull IMG creds USER[:PASS]' bug, extra '=' variable
assignment inside infix.xml
- Fix 'container run IMG CMD ARGS' to actually put the CMD in
ENTRYPOINT and append ARGS to image. The podman run and create
commands are *not* behaving the same way
- Rename 'attach' to 'exec', execute command inside an running container
- Add 'container [shell | connect]' to start shell in -- " -- " -- " --
- Add 'container [stat | cleanup | usage stats]' commands
- Add in="tty" to commands thay may end up being interactive
- Split <ACTION> line into multiple lines for readability
- Fix container shift logic:
root@infix-00-00-00:/> show container
/usr/sbin/container: line 361: shift: shift count out of range
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Also, update container YANG model descriptions and ensure 'path' is
mandatory for each list element and attach a regexp pattern to fix
the mountpoint to an absolute path inside the container.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
In the container configuration context:
edit file ntp.conf
set path /etc/ntp.conf
set content
The last command opens a text editor where you can paste the contents
of the file. This is stored base64 encoded in the datastore as well
as in JSON/XML.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This patch adds a new CLI command 'container upgrade foo', where 'foo'
is the name of the container. If more than one container use the same
image, multiple upgrades must be done because a container runs not on
the 'image:tag' but on the hash of the 'image:tag' it was created from.
Rename "done" queue to "active", since we want to recreate an active
container after fetching an updated base image.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This commit adds support for writable container volumes. A volume, when
compared to a mount, is a writable directory created when the container
starts and is automatically rsync'ed with the underlying directory it
is mounted on on first use. A mount otoh does not rsync so it results
only in an empty directory inside the container.
The podman/docker mount feature will be used later to bind mount single
files or sharing directories from the host, e.g., /sys/class/leds/ to
one of more containers.
Note: unused volumes are automatically pruned. Hence, a volume
currently cannot be moved to another container.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This patch allows containers created with "podman create" to surivce
rebots, and even reconfigurations at runtime, as long as their config
has not changed. Allowing some level of persistence in the writable
layer given to containers.
For even more persistency, support for volumes (they sync with dir in
container on first use) and mounts (creates empty writable dirs in the
container) must be added. A third option, a variant of mounts, is to
allow a leaf-list "file", where binary content can be added and then
mounted into the container, e.g.
/run/containers/files/$name/file.ext -> container://$name/etc/file.ext
None of this is available yet, but seems likely we need to add in the
very near term.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
