Commit Graph
1582 Commits
Author SHA1 Message Date
Joachim Wiberg 4b2acdedc5 .github: switch regression test to self-hosted runner
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 11:46:44 +01:00
Joachim Wiberg 0741d2a655 .github: simplify PR template slightly add reminder of YANG changes
[skip ci]

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 10:39:22 +01:00
Joachim Wiberg e38063c0ac Minor update/clarification now that content has shifted downwards
[skip ci]

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 10:27:19 +01:00
Joachim Wiberg a2bc3bef61 .github: reduce number of caches required by CI builds
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 10:08:20 +01:00
Joachim Wiberg eceeb8c59f confd: fix cni_popen() when building without container support
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg d12d66b013 package/klish-plugin-sysrepo: display default value in help text
When available, show the default value of a leaf node after the
description when issuing 'help leaf-node'.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg e73742504d Add new intro blurb from homepage to README
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg 6cde353029 Drop unicode from tagline, does not survive syslog
The tagline is read by Finit at boot, from /etc/os-release, and
logged when changing runlevel.  The unicode charachter does not
survive the trip via syslog to log file.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg a9e78aa329 patches/iproute2: backport ip address 'accept symbolic names'
This change completes the backport of symbolic names for ip address
protocol field:

    ip addr add IFADDR proto 5
=>
    ip addr add IFADDR proto dhcp

Ref:
https://lore.kernel.org/netdev/20230602155419.8958-1-stephen@networkplumber.org/

Fix #286

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg 7a52f49c7d execd: fix printf format specifier for systemf()
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg ba76103c7f execd: refactor to autotools
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg b0b1a79aa5 statd: refactor to autotools, same as confd
Fix issue #299

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg 5762a73c45 package/confd: reduce sysrepo default log level
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim Wiberg 4909a9e553 patches: backport finit silence udevd by default
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-27 08:14:41 +01:00
Joachim WibergandTobias Waldekranz 83821def3b confd: always create bridge with default_pvid 0
Fix #310
2024-02-26 16:36:29 +01:00
Richard AlpeandJoachim Wiberg f1ab05a86f yanger: remove stray variables and brackets (cosmetic)
Signed-off-by: Richard Alpe <richard@bit42.se>
2024-02-26 13:01:06 +01:00
Richard AlpeandJoachim Wiberg 412ee33f09 yanger: fix indentation (cosmetic)
Signed-off-by: Richard Alpe <richard@bit42.se>
2024-02-26 13:01:06 +01:00
Richard AlpeandJoachim Wiberg 0f3433e9a0 yanger: remove stray semicolons (cosmetic)
Signed-off-by: Richard Alpe <richard@bit42.se>
2024-02-26 13:01:06 +01:00
Richard AlpeandJoachim Wiberg 807b113d4e yanger: refactor test backend
Simplify the code a bit by passing the test file to the runner
function instead of checking it in every caller.

Signed-off-by: Richard Alpe <richard@bit42.se>
2024-02-26 13:01:06 +01:00
Richard AlpeandJoachim Wiberg 5f25c063d5 yanger: introduce main function
Signed-off-by: Richard Alpe <richard@bit42.se>
2024-02-26 13:01:06 +01:00
Joachim Wiberg 5038cd47e5 .github: proposed pull request template/checklist
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 5386ec9a3a test,doc,board: Infamy Made Easy
As of 4768cae6, we can use conserver/telnet to connect to the docker0
bridge IP:90XX, where XX is DUTXX.  This was a great UX improvement
and this patch set aims to further the experience by:

 - allow running Infamy (infix-test container) completely rootless¹
 - reduce the expsed port range 50->10 (can be improved further)
 - use 'podman --publish-all' ports, which allocates ten random
   ports for the exposed Qemu telnet ports
 - add Quick Start Guide to doc/testing.md

This restores the possiblity of running multiple "make test-sh"
instances, e.g., when multiple users share the same server.

The 'console' script included in this commit uses 'podman inspect' to
find the port number for a given DUT, and optional instance.  It takes
either the console/dut number (1-N), or the dut name from the topology.

Also in this commit:
 - set hostname for easy identification (for console script)
 - set conatainer --name to hostname
 - adjust workdir from buildroot to infix/test
 - simplify PS1 and add time to prompt (when did the test finish?)

When running 'make test-sh' the prompt now tells you which instance you
are running, e.g., infmay10.  Calling 'console 1 infamy10' connects to
console 1 (port 9001) on the infamy10 instance.

For more information, see the Quick Start Guide in doc/testing.md.  It
shows how to use the new helper scripts: console and shell.

Fix #227

_______
¹) devs using podman, instead of docker, with slirp4netns installed, can
   run the infix-test container completely rootless.  We like this, and
   as of today podman is our recommendation.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 00c2c0c608 test/case: new test, container connected using physical interface
Verifies that a regular/physical port/interface can be handed over to a
container.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 4856705803 test/case: new test, plain bridge + veth connection for container
Vanilla bridge and veth pair managed entirely by Infix.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg a38f5902c8 test/case: new test, verify connectivity behind docker0 masquerade
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg ec4c1b1d70 test/case: new test, basic container test
- New helper class for container testing
 - New helper class to urllib, Furl

Due to extremely weak Python-fu in the undersigned, this patch changes
the __init__.py file to add new helper classes for container tests.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg bdbbfcb50f test/infamy: improved logging for connect + factory-default
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg f8b4a858bc test/infamy: pep-8 cleanup and simplification
- Fix most of what pylint complains about
 - Simplify constructs and add missing whitespace
 - Simplify output of YANG model downloader

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 3da045dd40 test/infamy: handle empty NETCONF reply from remote
This patch handles a corner case we for some reason have not run into
yet.  With the new container operational data it was triggered almost
immediately, so let's add some basic guards around it and return None

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 72a86301f0 test/infamy: save location data for Device and add address() method
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Tobias WaldekranzandJoachim Wiberg fcc9fe84f4 test/infamy: Support calling arbitrary functions from within a netns
Add a helper that lets you call a Python function (or lambda) in the
context of a network samespace.

Example:

    with infamy.IsolatedMacVlan("eth0") as ns:
        res = ns.call(lambda: subprocess.run(["ip", "link"],
	                                     capture_output=True))
	print(res.stdout, f"\n[exitcode:{res.returncode}]")

    The call to subprocess.run will be executed in the context of the
    network namespace, and thus only list "lo" and the "iface"
    macvlan.

The return value of the function passed to ns.call() is passed back
over a multiprocessing.Pipe, which requires that the object be
"picklable". This is true for most objects (even more complex ones
like the CompletedProcess object seen in the example above). Some
notable exceptions are things like file objects, sockets, etc.

The setns(2) syscall is unfortunately not available in current
versions of Python shipped with neither Ubuntu nor Alpine at the time
of this writing. Therefore, shoot from the hip, assume that we're
running on an x86_64 CPU, and just hotwire the syscall directly with
some constants we found in a dumpster.
2024-02-25 19:49:27 +01:00
Joachim Wiberg 9ce763662f package/curios-httpd: new package, bundle a demo container
This is an example of how to create a package that downloads an OCI
archive and installs it to /lib/oci

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 257636d997 package/klish: bump for __ptype -> xml param.name fix
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 25c31ce218 src/klish-plugin-infix: improve log message on failed copy
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg bcdd311cb8 src/klish-plugin-infix: support for copying remote files with auth
This patch adds the optional 'user name' argument to the 'copy' command,
enabling using curl scp/sftp protocols for fetching and uploading files.

Changes to cfg_adjust() allow saving and referencing files in the user's
home directory.  Useful for both fetching upgrade bundles and container
images.

The 'dir' command now lists files in both $HOME and /cfg.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 3159d6c042 src/klish-plugin-infix: new symbol shell@infix with user tracking
This patch adds a new klish symbol, shell@infix, which performs proper
droprivs before calling the configured shell.

Fix #298: track user id so shell command gets correct user

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 8354997104 src/klish-plugin-infix: break out shell to optional shell.xml
Some customers don't want unprivileged CLI users to be able to exit to
a shell.  The shell is currently hard-coded to bash, but should be one
of the ietf-system shells, configurable --with-shell=foo.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg b0a0b31e68 src/klish-plugin-infix: refactor copy@infix, allow file -> file copy
This refactor is basically only an extra step after

   copy source-ds -> dest-ds

to also

   copy source-file -> dest-file

Which in the case of factory-config and startup-config is relevant.  We
want to update the startup datastore, in case of additional commands,
but also the file /cfg/startup-config.cfg in case of consecutive reboot.

Fix #259 copy factory-config startup-config

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 686f0bec03 confd: refactor, relocate container network code to cni.[ch]
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 360d3b322d confd: use podman stop/start to prevent container corruption
Sending SIGTERM to conmon is not a safe shutdown of a podman container.
To handle gracefully handle shutdown, restarting and provide an orderly
start of dependencies, we use the Finit sysv trick via container script
wrapper to call 'podman stop foo'.

However, since podman does not support syslog as output for containers
we employ an old FIFO trick with another program, k8s-logger, to allow
logs to reach syslog.  Please note that k8s-logger must have properly
started before we call `podman start` -- this makes us fully dependent
on the 'container' wrapper script.  Hence the documentation update.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 6fa1eb9e00 package/k8s-logger: new package
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg a530db9ae5 board/common: confine containers in their own parent cgroup
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 337f39a74d board/common: quiet flag and check if running for start/stop
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 3b5c0248d7 confd: handle factory-default RPC better wrt. container networks
Because sysrepo callbacks are threaded and factory-default RPC is called
from a separate subscription (to prevent blocking), we cannot prevent
ietf-intefaces.c from being called before infix-containers.c, regardless
of the priority we set for our subscriptions.

When assigning a physical network interface this becomes a bit of a pain
during factory-default RPC since the physical interface is hidden from
the host network namespace.

So, when applying factory to running, we check each interface if it was
a container-network previously, if so we call on the container script in
the exit of the current dagger generation to move the interface back to
the host netns.

This affects all other functions that assume interfaces only live in the
host netns.  To that end a set of new helper functions have been added
to wrap iproute2 commands in nsenter when the interface lives elsewhere.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 168fb24957 confd: handle building without containers
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg ec3e3b3322 confd: minor refactor, split hook, improve log messages
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 67bb7e5928 confd: simplify container network naming
The interface name is already a unique qualifier, so we can simplify CNI
network naming to ease the burden when debugging.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg 214ca82bd6 confd: allow host container interfaces without IP address
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg a05a3609c5 confd: add sloppy inference of container-network type
"Sloppy" because, for some reason, we don't get sysrepo update callbacks
for presence containers.  I.e., when calling 'set container-network'
compared to 'edit container-network'.

Yes, I've gone over klish-plugins-sysrepo with a fine-toothed comb to
see if it's the culprit, but no, it seems to be sysrepo.  Hence the
defaulting of the type also in the change path.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00
Joachim Wiberg e4aa36dfe7 confd: tighten up container data validation in YANG model
Review found quite a few strings that could be given an obvious pattern
or specialized sub-type, e.g. inet:ip-address, to restrict the input
data validation.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2024-02-25 19:49:27 +01:00