Merge pull request #1038 from kernelkit/doc-upgrade-downgrade-policy

Doc upgrade/downgrade policy
This commit is contained in:
Jon-Olov Vatn
2025-05-05 09:45:49 +02:00
committed by GitHub
5 changed files with 378 additions and 2 deletions
+1
View File
@@ -10,6 +10,7 @@ All notable changes to the project are documented in this file.
- Upgrade libyang to 3.12.2
- Upgrade sysrepo to 3.6.11
- Upgrade netopeer2 (NETCONF) to 2.4.1
- Add documentation on Infix upgrading and downgrading, issue #1009
### Fixes
+3 -2
View File
@@ -349,5 +349,6 @@ can funtion reasonably well without a persistent `/var`, loosing
If `var` is not available, Infix will still persist `/var/lib` using
`cfg` as the backing storage.
[^1]: See [CLI Upgrade](cli/upgrade.md) for information on upgrading
via CLI.
[^1]: See [Upgrading procedures and boot
order](system.md#upgrade-procedures-and-boot-order) for
information on upgrading via CLI.
+5
View File
@@ -39,7 +39,12 @@ The secondary partition (`rootfs.1`) has now been upgraded and will be used as
the *active* partition on the next boot. Leaving the primary partition, with
the version we are currently running, intact in case of trouble.
See [upgrading procedures and boot order][2] for more information on
upgrading.
[^1]: It is not possible to upgrade the partition we booted from. Thankfully
the underlying "rauc" subsystem keeps track of this. Hence, to upgrade
both partitions you must reboot to the new version (to verify it works)
and then repeat the same command.
[2]: ../system.md#upgrade-procedures-and-boot-order
+8
View File
@@ -193,3 +193,11 @@ admin@example:/config/web/> edit restconf
admin@example:/config/web/restconf/> no enabled
admin@example:/config/web/restconf/>
```
# System Upgrade
See [upgrading procedures and boot order][1] for information on
upgrading.
[1]: system.md#upgrade-procedures-and-boot-order
+361
View File
@@ -323,7 +323,368 @@ reference ID, stratum, time offsets, frequency, and root delay.
> The system uses `chronyd` Network Time Protocol (NTP) daemon. The
> output shown here is best explained in the [Chrony documentation][4].
## Upgrade procedures and boot order
For resilience purposes, Infix maintains two software
images referred to as the _primary_ and _secondary_ partition image.
In addition, some bootloaders support [netbooting][6].
The _boot order_ defines which image is tried first, and is listed
with the CLI `show software` command. It also shows Infix version
installed per partition, and which image was used when booting (`STATE
booted`).
```
admin@example:/> show software
BOOT ORDER
primary secondary net
NAME STATE VERSION DATE
primary booted v25.01.0 2025-04-25T10:15:00+00:00
secondary inactive v25.01.0 2025-04-25T10:07:20+00:00
admin@example:/>
```
YANG support for upgrading Infix, inspecting and _modifying_ the
boot-order, is defined in [infix-system-software][5].
### Upgrading Infix
Upgrading Infix is done one partition at a time. If the system has
booted from one partition, an `upgrade` will apply to the other
(inactive) partition.
1. Download and unpack the release to install. Make the image *pkg*
bundle available at some URL[^10]
2. Assume the unit has booted the `primary` image. Then running the
`upgrade` command installs a new image on the `secondary`
partition
3. As part of a successful upgrade, the boot-order is implictly
changed to boot the newly installed image
4. Reboot the unit
5. The unit now runs the new image. To upgrade the remaining partition
(`primary`), run the same upgrade command again, and (optionally)
reboot to verify the upgrade
> [!CAUTION]
> During boot, the unit may [migrate](#configuration-migration) the
> startup configuration for any syntax changes. It is therefore
> important that you make sure to upgrade the other partition as well
> after reboot, of course after having verified your setup.
The CLI example below shows steps 2-4.
Upgrade: here the image *pkg bundle* was made available via TFTP.
```
admin@example:/> upgrade tftp://198.18.117.1/infix-aarch64-25.03.1.pkg
installing
0% Installing
0% Determining slot states
10% Determining slot states done.
...
98% Copying image to rootfs.1
99% Copying image to rootfs.1
99% Copying image to rootfs.1 done.
99% Updating slots done.
100% Installing done.
Installing `tftp://198.18.117.1/infix-aarch64-25.03.1.pkg` succeeded
admin@example:/>
```
Reboot: The unit will boot on the other partition, with the newly
installed image. The `Loading startup-config` step conducts migration
of startup configuration if applicable.
```
admin@example:/> reboot
[ OK ] Stopping Static routing daemon
[ OK ] Stopping Zebra routing daemon
...
[ OK ] Loading startup-config
[ OK ] Verifying self-signed https certificate
[ OK ] Update DNS configuration
[ OK ] Starting Status daemon
Infix -- a Network Operating System v25.03.1 (ttyS0)
example login: admin
Password:
.-------.
| . . | Infix -- a Network Operating System
|-. v .-| https://kernelkit.org
'-'---'-'
Run the command 'cli' for interactive OAM
admin@example:~$ cli
See the 'help' command for an introduction to the system
admin@example:/> show software
BOOT ORDER
secondary primary net
NAME STATE VERSION DATE
primary inactive v25.01.0 2025-04-25T10:15:00+00:00
secondary booted v25.03.1 2025-04-25T10:24:31+00:00
admin@example:/>
```
As shown, the *boot order* has been updated, so that *secondary* is
now the preferred boot source.
To upgrade the remaining partition (`primary`), run the `upgrade URL`
command again, and reboot.
### Configuration Migration
The example above illustrated an upgrade from Infix v25.01.0 to
v25.03.1. Inbetween these versions, YANG configuration definitions
changed slightly (more details given below).
During boot, Infix inspects the `version` meta information within the
startup configuration file to determine if configuration migration is
needed. In this specific case, the configuration file has version
`1.4` while the booted software expects version `1.5` (the
configuration version numbering differs from the Infix image version
numbering). The startup configuration is migrated to `1.5`
definitions and stored, while a backup previous startup configuration
is stored in directory `/cfg/backup/`.
```
admin@example:/> dir /cfg/backup/
/cfg/backup/ directory
startup-config-1.4.cfg
admin@example:/>
```
The modifications made to the startup configuration can be viewed by
comparing the files from the *shell*. An example is shown below.
```
admin@example:/> exit
admin@example:~$ diff /cfg/backup/startup-config-1.4.cfg /cfg/startup-config.cfg
--- /cfg/backup/startup-config-1.4.cfg
+++ /cfg/startup-config.cfg
...
- "public-key-format": "ietf-crypto-types:ssh-public-key-format",
+ "public-key-format": "infix-crypto-types:ssh-public-key-format",
...
- "private-key-format": "ietf-crypto-types:rsa-private-key-format",
+ "private-key-format": "infix-crypto-types:rsa-private-key-format",
...
- "version": "1.4"
+ "version": "1.5"
...
admin@example:~$
```
### Downgrading Infix
Downgrading to an earlier Infix version is possible, however,
downgrading is **not** guaranteed to work smoothly. In particular,
when the unit boots up with the downgraded version, it may fail to
apply the *startup config*, and instead apply its [failure config][7].
We consider two cases: downgrading with or without applying a backup
startup configuration before rebooting.
In both cases we start out with a unit running Infix v25.03.1, and
wish to downgrade to v25.01.0.
```
admin@example:/> show software
BOOT ORDER
primary secondary net
NAME STATE VERSION DATE
primary booted v25.03.1 2025-04-25T11:36:26+00:00
secondary inactive v25.03.1 2025-04-25T10:24:31+00:00
admin@example:/>
```
#### Downgrading when applying a backup startup configuration
This is the recommended approach to downgrade, given that you have a
backup configuration available. The objective is to avoid ending up
with the unit in *failure config*.
1. Find the backup configuration file.
2. Run `upgrade URL` to install Infix image to downgrade to.
3. Copy backup startup configuration to current startup configuration
(from shell).
4. Reboot.
*Find the backup configuration file:*
Assume you have a backup startup config for the version to downgrade
to (here Infix v25.01.0, config `version 1.4`).
```
admin@example:/> dir /cfg/backup/
/cfg/backup/ directory
startup-config-1.4.cfg
admin@example:/>
```
*Use `upgrade` command to downgrade:*
```
admin@example:/> upgrade tftp://198.18.117.1/infix-aarch64-25.01.0.pkg
installing
0% Installing
0% Determining slot states
10% Determining slot states done.
...
99% Copying image to rootfs.1 done.
99% Updating slots done.
100% Installing done.
Installing `tftp://198.18.117.1/infix-aarch64-25.01.0.pkg` succeeded
admin@example:/>
```
*Apply the backup configuration file:*
```
admin@example:/> copy /cfg/backup/startup-config-1.4.cfg /cfg/startup-config.cfg
Overwrite existing file /cfg/startup-config.cfg (y/N)? y
admin@example:/>
```
*Reboot:*
The unit will come up with the applied backup configuration instead of
failure config.
```
admin@example:/> reboot
[ OK ] Saving system clock to file
[ OK ] Stopping Software update service
[ OK ] Stopping Status daemon
...
[ OK ] Bootstrapping YANG datastore
[ OK ] Starting Configuration daemon
[ OK ] Loading startup-config
[ OK ] Update DNS configuration
[ OK ] Verifying self-signed https certificate
[ OK ] Starting Status daemon
Infix -- a Network Operating System v25.01.0 (ttyS0)
example login:
```
#### Downgrading without applying a backup startup configuration
This procedure assumes you have access to the unit's console port and
its default login credentials[^9].
1. Downgrade
2. Reboot
3. Login with unit's default credentials
4. Conduct factory reset
5. (Then go on configure the unit as you wish)
*Use `upgrade` command to downgrade:*
```
admin@example:/> upgrade tftp://198.18.117.1/infix-aarch64-25.01.0.pkg
installing
0% Installing
0% Determining slot states
10% Determining slot states done.
...
99% Copying image to rootfs.1 done.
99% Updating slots done.
100% Installing done.
Installing `tftp://198.18.117.1/infix-aarch64-25.01.0.pkg` succeeded
admin@example:/>
```
*Reboot:*
Conduct a reboot. During boot, the unit fails to apply the existing
startup configuration (config version `1.5` while software expects
version `1.4` or earlier), and instead applies its [failure
config][7]. This is what is seen on the console when this situation
occurs. Note that the login prompt displays `failed` as part of the
*hostname*.
```
admin@example:/> reboot
[ OK ] Saving system clock to file
[ OK ] Stopping Software update service
[ OK ] Stopping Status daemon
...
[ OK ] Verifying SSH host keys
[ OK ] Bootstrapping YANG datastore
[ OK ] Starting Configuration daemon
[FAIL] Loading startup-config
[ OK ] Loading failure-config
[ OK ] Verifying self-signed https certificate
[ OK ] Starting Status daemon
Infix -- a Network Operating System v25.01.0 (ttyS0)
ERROR: Corrupt startup-config, system has reverted to default login credentials
failed-00-00-00 login:
```
To remedy a situation like this, you can login with the unit's *default
login credentials*, preferrably via a [console port][8].
The unit's default credentials are typically printed on a sticker on
the unit.
```
failed-00-00-00 login: admin
Password:
Run the command 'cli' for interactive OAM
admin@failed-00-00-00:~$
```
When it is *safe* from a network operations perspective, you can
conduct a factory reset and reboot. It is recommended to remove the
unit from any production network before doing this, as a factory reset
may enable undesired connectivity between the unit's ports.
```
admin@failed-00-00-00:~$ factory
Factory reset device (y/N)? y
factory: scheduled factory reset on next boot.
Reboot now to perform reset, (y/N)? y
[ OK ] Saving system time (UTC) to RTC
[ OK ] Stopping mDNS alias advertiser
...
[ OK ] Starting Configuration daemon
[ OK ] Loading startup-config
[ OK ] Update DNS configuration
[ OK ] Verifying self-signed https certificate
[ OK ] Starting Status daemon
[ OK ] Starting Status daemon
Please press Enter to activate this console.
Infix -- a Network Operating System v25.01.0 (ttyS0)
example login:
```
Continued configuration is done as with any unit after factory reset.
[1]: https://www.rfc-editor.org/rfc/rfc7317
[2]: https://github.com/kernelkit/infix/blob/main/src/confd/yang/infix-system%402024-02-29.yang
[3]: https://www.rfc-editor.org/rfc/rfc8341
[4]: https://chrony-project.org/doc/4.6.1/chronyc.html
[5]: https://github.com/kernelkit/infix/blob/main/src/confd/yang/confd/infix-system-software.yang
[6]: boot.md#system-configuration
[7]: introduction.md#system-boot
[8]: management.md#console-port
[^9]: In failure config, Infix puts all Ethernet ports as individual
interfaces. With direct access, one can connect with e.g., SSH,
using link local IPv6 addresses. This as an alternative to
connecting via a console port.
[^10]: Set up an FTP/TFTP/SFTP or HTTP/HTTPS server on the same LAN.