Shell script:
- Factor out big portions of code into more logical helper functions
- Simplify calling setup script by checking for remote image first
- Simplify meta/sha up-to-date handling and clarify terminology
- Consistent use of -f instead of -e in file-exists checks
- Fix unsafe use of 'mktemp -u'
C code:
- Clarify meta/sha terminology: rename meta-sha256 -> meta-image-sha256
- Refactor weird archive_offset() function to local_path() helper
- Factor out helper function calc_sha()
- Check len of sha256 >= 64
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
One unique feature of Infix OS is that you can embed your OCI archive in
the rootfs.squashfs. This way your container instance does not need to
download anything from the network. To upgrade you drop in a new image
and rebuild Infix, when the new Infix boots the container script 'setup'
command will recognize that the OCI archive has changed and will reload
it into the container store.
This patch is an improvement of the way too generic 'podman image prune'
command used previously. Instead of looking for any dangling image, we
now surgically remove the old image.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Container instances that run with mutable images, e.g., tagged with `:latest`
or similar non-versioned tags, can be upgraded without changing the config.
This commit fixes two issues found with this support:
- force container image re-fetch on upgrade, even if the file exists locally
- surgically remove old image from container store after upgrade
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Usually, when your system is up and running properly, you want to clean
up anything unused from your previous experiments. This change alllows
that by calling the interactive 'podman image prune -a -f' command from
the CLI command 'container remove all'
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When loading big OCI images at boot the `podman load` process completely
monopolizes all cores of an Arm Cortex-A72. It blocks on I/O, sure, but
with 'nice' we can get some attention at least to more critical services
at boot.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This commit reverts 477f7ae and bb19d06, which intended to fix an issue
with lingering old images, see #1098. However, as detailed in #1147,
this caused severe side effects while working with multiple larger
containers. Basically, the prune operation of one container removed
images of other containers that are just being created in parallel.
Instead of using the podman prune command we can use the meta datain the
start script to pinpoint exactly which image(s) to remove, including any
downloaded OCI archives when the container instance is removed.
Fixes#1147
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This commit adds metadata to track loaded OCI archives to allow skipping
'delete + load' of OCI images when restarting either the container or the
system as a whole. The sha256 of all loaded OCI archives is stored in a
sidecar file in our downloads directory. Then we verify the checksum of
the OCI archives against their same-named sidecar to determine if the OCI
archive is already loaded or not.
Additionally, the instance using the image is labled with metadata to detect
changes in the container configuration. This in turn allow skipping the
delete + create phase also of the instance.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The default timeout for 'podman stop foo' is 10 seconds, which for
heavily loaded systems with intricate shutdown process is *waaaay*
too short. Increase it to the container script default 30s, which
coincidentally is also the container@.conf template's kill delay.
Fixes#1149
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Add supoprt for infix-firewall.yang, modeled on the zone-based firewalld
The terminology is a mix of firewalld, classic netfilter and inspired by
Ubiquity. E.g., zone 'policy' -> 'action', and the zone matrix overview.
- Port forwarding allows forwarding a range of ports
- Operational data comes from firewalld active rules
- Firewall logging goes to /var/log/firewall.log
- Show implicit/built-in rules and zones (HOST) in firewall matrix,
includes "locked" policy for the default-drop behavior
- The zone services field in admin-exec 'show firewall' shows ANY when
the zone default action is set to 'accept'
- Zone 'forwarding' and 'masquerade' settings live in Infix in the
policys instead, meaning users need to explicitly add a policy
to allow both intra-zone and inter-zone forwarding
- Support for emergency lockdown (kill switch)
- Pre-defined services (xml+enums) are filtered and included as a
separate YANG model, extensions added for netconf and restconf
- Includes initial support for firewalld rich rules
firewalld policy rules, including rich rules, have an obnoxious priority
field which is extremely hard to get right, so in Infix we use the far
superior YANG construct 'ordered-by user;'. This ensure all rules are
generated in that order by setting the priority field, on read-back from
firewalld (operational) the priority field is used to sort the output
of rules in the CLI.
Fixes#448
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
In setups like this...
CPU
eth0
|
.----0----.
| dst0sw0 |
'-1-2-3-4-'
|
.----0----.
| dst1sw0 |
'-1-2-3-4-'
...both eth0 and dst0sw0p4 are DSA ports. But the latter is _also_ a
physical port as far as devlink is concerned. As a result, it would
first be marked as "internal" and is then instantly reclassified as a
"port".
Catch this condition and stick with the initial "internal"
classification.
In addition to matching on interface names, add support for matching
on ethtool information.
Example:
{
"@ethtool:driver=st_gmac": {
"broken-mqprio": true
}
}
This would mark any interface using the "st_gmac" driver as having a
broken mqprio implementation. Whereas this:
{
"@ethtool:driver=st_gmac;bus-info:30bf0000.ethernet": {
"broken-mqprio": true
}
}
Only matches an st_gmac-backed interface at the specified location.
As matching becomes more complicated, use the shell implementation
from confd as well, to make sure that they are always in agreement.
- cli: add '-r' to log/pager/follow for "raw" control chars, including unicode
- sysklogd: backport unicode fix for em-dash character (—) in slogan
- sysklogd: drop old patches
- sysklogd: run in 8-bit safe mode
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When a container's image is on an inaccessible remote server, the
container wrapper script waits in the background for any netowrk
changes to retry download of the image.
This change avoids the dangerous previous construct, and is also
easier to read: timeuot after 60 seconds unless ip monitor reads
at least one event before that.
Fixes#1124
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
From the documentation:
> 'podman image prune' removes all dangling images from local storage.
> With the all option, all unused images are deleted (i.e., images not
> in use by any container).
>
> The image prune command does not prune cache images that only use
> layers that are necessary for other images.
So, when the container script is called in the cleanup phase of the
lifetime of a container, we can use the '--all' option to ensure we also
remove this container's loaded image. In the case this happens before
a reboot of the system, there will be no old version of the image loaded
to /var/lib/containers after boot.
Issue #1098
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Previously, upgrading Podman containers with the same tag left
behind dangling images, causing overlay storage to grow and fill
disk space.
This change ensures dangling images are cleaned up using
podman image prune. The command is run without -a, so only
unreferenced images are removed. This provides safe cleanup while
preventing unnecessary overlay growth.
Fixes#1098
Signed-off-by: Richard Alpe <richard@bit42.se>
The reason for this is:
1) We want to use the same fetch tool for all operational data, to
avoid having duplicated xpaths scattered across the repo.
2) We want to call this directly from the CLI, which means it need to
be escape safe. A unprivileged user shall be able to use it but not
escape into a shell or do other malicious stuff.
Signed-off-by: Richard Alpe <richard@bit42.se>
There's several reasons for this. The main reason is that it was
developed as an intermediate way of getting system information when
the operational datastore was almost empty. There's not a lot of data
available in the operational datastore and we shall rely on that data
primarily. This command isn't safe to run from a jailed CLI as i
doesn't validate input and run commands directly in the shell, making
it hard to protect against a CLI jail break.
In upcoming patches we will introduce a new show tool that relies
solely on operational data and which sanitize the user input.
Signed-off-by: Richard Alpe <richard@bit42.se>
This silences a bogus warning in the log when runparts is called with no
argument. I.e., when there is no product specific init files available.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When stopping a container that is (stuck) in setup state, e.g., fetching
its container image, we need to send SIGTERM to the container wrapper
script rather than forwarding the 'stop' command to podman.
With this fix, and the Finit upgrade, there is no longer a 10 second
timeout before Finit sends SIGKILL to the PID. Instead, the script now
immediately react to the initial SIGTERM.
Related to issue #980
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
For tasks like upgrade we need, e.g., the image URI. Instead of
guessing where on the container command line the image is, use a
meta-data field in the container shell script.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
- More logging when deleting a container
- Update 'exec' and 'shell' commands to support running commands
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Containers in Infix are read-only, any persistent data is either stored
in volumes or content mounts. To prevent lingering containers, as have
been seen on some test systems, we move cleanup stage from confd to use
the Finit cleanup:script mechanism.
The resulting code is simpler, shorter, and easier to debug and use.
As an added benefit, confd no longer prunes "unused" volumes by itself
on (any!) reconfiguration event, which with the new container life cycle
in Infix was quite fragile. Instead the CLI command 'container prune'
is extended with 'podman volume prune', which coincidentally the help
text already alluded to.
This patch requires Finit v4.10-rc1, or later.
Fixes#902
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
When updating a container configuration we need to rerun the setup stage
of the container. However, a run/task in Finit only runs once for each
runlevel, meaning changes at runtime of a container was broken.
This is a refactor of the container script to allow it to also run as a
pre:script in the Finit setup stage of a sysv/service. Also included is
a fix to the timeout/retry handling in case the container setup fails.
We must call `read -t 60` *before* calling `ip monitor`, otherwise it
will not be called until `ip monitor` returns.
This patch requires Finit v4.9, or later.
Fixes#930
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
- Rename local variables
- Extract image name, using basename, for podman image exists
Fixes the following syslog error message:
root@foo:~$ podman image exists docker://ghcr.io/kernelkit/curios:edge
Error: unsupported transport "docker" for looking up local images
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The speed/duplex of a virtio interface is not eenforced in any way. The
driver initializes them to unknown/unkown:
https://github.com/torvalds/linux/blob/dccbe2047a5b0859de24bf463dae9eeea8e01c1e/drivers/net/virtio_net.c#L5375-L5381
It is however possible to set them since there are kernel subsystems
that rely on them, e.g., miimon in the bond driver, which in turn is
required for use with the 802.3ad (lacp) mode.
If we do not initialize speed/duplex the miimon will complain in the log
on virtual test systems every second:
Dec 13 22:34:08 laggy kernel: lag0: (slave e1): failed to get link speed/duplex
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Since `conmon` only logs the output from the monitored container's
`stdout`/`stderr`, we might as well get rid of `k8s-logger`, and let
the output pass through to finit, which will then pipe the messages to
syslog.
Fix#836
This is a complete redesign of how the system creates/deletes containers
from the running-config. Containers are now removed synchronously from
confd before any interfaces they may be using are removed, and created
in parallel, using a Finit task, well after confd has finished setting
up interfaces.
The logic previously provided by execd to retry container create on any
route/address changes, or periodically every 60 seconds, is now handled
by a new 'setup' command in the container wrapper script.
Additionally, container create is now split in wget/curl/podman pull of
the image and 'podman create'. This to both consolidate image fetching
and improve user feedback since most of the retry logic (above) revolves
around the image download.
Fixes#835
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The -S switch to less seemed at the time a good idea -- chop off too
long lines and allow horizontal scrolling -- however, the -F option, to
skip pager for content that fit the first screen is overridden by the -S
switch. This makes commands like "show containers", and similar that
normally have waaaaay too few lines for a pager to always page.
So let's drop -S to regain the original behavior. This will cause lines
to be wrapped, but with the new raw mode in the CLI this seems to work
without any weird screen artifacts.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This fixes a regression in v24.11.0, introduced in 46dd0c7, where line
drawing characters are not properly displayed in the CLI anymore.
admin@infix:/> show interfaces
INTERFACE PROTOCOL STATE DATA
lo ethernet UP 00:00:00:00:00:00
ipv4 127.0.0.1/8 (static)
ipv6 ::1/128 (static)
br0 bridge
<E2><94><82> ethernet UP 02:00:00:00:00:00
<E2><94><82> ipv4 169.254.1.3/16 (random)
<E2><94><82> ipv6 fe80::ff:fe00:0/64 (link-layer)
<E2><94><9C> swp1 bridge FORWARDING
<E2><94><9C> swp2 bridge FORWARDING
<E2><94><9C> swp3 bridge FORWARDING
<E2><94><9C> swp4 bridge FORWARDING
<E2><94><9C> swp5 bridge FORWARDING
<E2><94><9C> swp6 bridge FORWARDING
<E2><94><9C> swp7 bridge FORWARDING
<E2><94><9C> swp8 bridge FORWARDING
<E2><94><9C> swp9 bridge FORWARDING
<E2><94><94> swp10 bridge FORWARDING
This is because the raw tty from klish disables IUTF8 and the fact that
it does not source any environment, e.g., /etc/profile. (We do not have
any locale settings in the enviornment either, but that's a discussion
for another day.)
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Joachim Wiberg