This is just a first step to add VXLAN tunnels, much more remains.
Implemented right now is as GRE, local, remote and a VNI.
Also refactor gre_basic and gre_bridged to generic tunnel tests,
the same test now tests VXLAN as well.
The yanger source grown significantly, code is disorganized, the
namespace is all over the place, etc.
Take a stab at remedying the situation by splitting out the individual
YANG models in separate Python modules, starting with ietf-system.
Rather than explicitly defining the sources, specify the package names
and let poetry figure out the details. Otherwise, subpackages are not
included, which we want to start using in yanger.
When interactively running, for exampe, yanger interactively from the
source tree, we end up with compiled files in our working tree. Make
sure to ignore these.
These are also provided by other packages (sysrepo, netopeer, etc.)
but having them in here allows us to lint all of our own models
without the need for a build tree.
Basic support GRE and GRETAP tunnels both for IPv4 and IPv6.
Limited support right now, only possible to configure local
ip address and remote ip address.
Instead of running initctl reload, which also prematurely enables
services which are not supposed to be started until we have run
through the netdag and evolved to the next generation.
Handle the original use-case of disabling zeroconf is managed by
asking finit to stop the service immediately in the exit action, and
deferring the cleanup of the service until we call `initctl reload`
after the init action of the new generation.
In ye olde confd, we basically had `50-init.ip` - and that was fine.
Then came containers, VLANs, bridges, etc.; and with it: a whole
forest of setup/teardown scripts - and it was not fine anymore.
For example: if found an init script running at 61, you would have to
grep the source tree and look at _all_ other call sites to get a sense
of what runs before and/or after it.
Therefore: create enums for the init/exit actions where the order is
explicitly stated, and use those for all interface scripts.
Clean up bridge-port config generation to align with the bridge upper
code.
- Remove lots of unnecessary guards that we model guaratees are always
met (existance of containers and leafs). This reduces indentation
depth and increases readability, IMHO.
- Split the generation into smaller pieces, each with a clear purpose.
- Try to include all edge cases around the tricky subject of PVID
migrations, with the actions running in either exit or init, as
appropriate.
TL;DR:
> New code, all bugs removed ;)
Incremental reconfiguration of bridge related settings had lots of
ordering issues.
This change tries to remedy that by:
- Introducing the concept of "snippets": Instead of having to run C
code in the correct order (which is doomed to fail since teardowns
typically need to run in reverse), collect related settings in a
snippet. Later on, we can then concatenate snippets into a larger
file in the proper order.
- Refactoring the bridge setup to take advantage of snippets. Now we
can call out to functions that _both_ generate the VLAN config _and_
sets some bridge global options, for example. This gets us out of
having to pass back settings to the main generation function.
- Ensuring that deleted objects are always removed - in the proper
order - in the exit action of the previous generation. This was
previously not true for VLANs, fix#869.
- Generating `mcd`:s config orthogonally from the bridge setup. Since
we need to keep track of _all_ bridges, figure out when VLAN uppers
exist etc., it becomes _very_ messy to generate the configuration
from the diff of a single interface. There were lots of cases here
where disabling the querier service on one bridge would disable the
whole daemon, thereby disabling the service for other bridges that
still had it enabled.
Fact: ietf-interface.c has become too unwieldy.
Therefore, split it, roughly based on the YANG structure, into
multiple files.
There is much more to do here, like cleaning up the naming (which is
all over the place), moving more system commands into dagger, etc. But
I wanted to keep this change to a strict code reshuffle.
Previously, the generation number was only increased for successful
evolutions. This led to a confusing state in /run/net where multiple
<N>-ABANDONED-<date> directories could exist, with the same value of
N.
Move to a scheme where the generation increments every time we evolve,
even if the evolution fails. If multiple consecutive evolutions fail,
we can now unabiguously determine the order in which they were
executed.
When reconfiguring the system many times (seen during regression
testing), the RAM usage of keeping all generations around starts to
add up.
A tmpfs file, no matter how small, will always consume at least one
full page (4k). So one generation of tiny setup scripts and log files
can end up locking a 1 MB (!) of memory.
Therefore: Once we're done with a generation, archive it in a
compressed tarball, which typically fits inside a single 4k
page. Limit the total number of archives to 10. This should give full
visibility into the history of a system in all normal situations,
while still allowing hammering systems with reconfig cycles in testing
scenarios.
Use container for interface allow/deny and for reflector settings.
Hopefully there will be more settings in the future, e.g., control
reflector interfaces independently.
Fix#678
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Introduce a new approach to dtb-building:
Old way: `rsync` an overlay containing our dt source to the kernel,
and then build via buildroot's `_INTREE_DTS_NAME`. This does not work
in recent kernels (at least since 6.12), which requires all dtb's to
be explicitly listed in a `Makefile`.
New way: Create a separate package for every board, and build the
device trees via the kernel's standard interface for building external
modules. This has two main advantages:
1. We can easily interpose a patched version of an upstream device
tree. E.g., we can easily set the factory password.
2. It is much easier to get an overview of which boards are enabled,
rather than scrolling back and forth in the `_INTREE_DTS_NAME` list.
Enable all fully supported boards in the defconfigs.
In the OSPF(v2) context, there is an "address-family" setting available.
However, since only IPv4 routes are supported in OSPF, currently
this setting is not relevant.
Fixes#813
When disabling a container in the configuration we should not remove it,
because then any volumes used by it may be lost. Instead, we now simply
disable the service to prevent it from starting, or stopping it, if it
was running.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This is a complete redesign of how the system creates/deletes containers
from the running-config. Containers are now removed synchronously from
confd before any interfaces they may be using are removed, and created
in parallel, using a Finit task, well after confd has finished setting
up interfaces.
The logic previously provided by execd to retry container create on any
route/address changes, or periodically every 60 seconds, is now handled
by a new 'setup' command in the container wrapper script.
Additionally, container create is now split in wget/curl/podman pull of
the image and 'podman create'. This to both consolidate image fetching
and improve user feedback since most of the retry logic (above) revolves
around the image download.
Fixes#835
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When deleting interfaces we skip a lot of steps in netdag_gen_iface(),
this patch ensures we at least call netdag_gen_ipv4_autoconf() to drop
any running zeroconf agent in the same generation.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Instead of having to scroll to the end (Shift G, or page down til end),
let's instead start at the end now that we have control over paging in
the CLI ourselves.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Mattias Walström