board/common: Wait for slow devices before continuing boot

Before this change, we relied on `udevadm settle` to act as a barrier
which we could only cross after all devices had been probed. As it
turns out, when drivers defer probing due to missing dependencies,
`udevadm settle` may exit (as no events are generated for a period of
time) before all devices are set up. This was the root cause behind
issue #685, for which #752 was the initial attempt at a fix.

As we know that systems using mv88e6xxx will always have a device link
referencing the conduit interface, add a script to finit's system
runparts directory, which will block while any device link consumers
are still probing.

As a consequence of this, we can no longer rely on the sysklogd.conf
shipped with finit. Therefore, install an Infix specific one, which
will make sure that the aforementioned script has run before launching
the syslog daemon, which most other things depend on.

Fixes #685

board/common/rootfs/etc/finit.d/available/sysklogd.conf

@@ -0,0 +1,4 @@
+# Use <pid/syslogd> as barrier for other system tasks and service that
+# rely on modules, firmware, and device nodes to be ready.
+service if:udevd nowarn env:-/etc/default/sysklogd <run/udevadm:post/success> \
+	[S0123456789] syslogd -F $SYSLOGD_ARGS -- System log daemon

board/common/rootfs/usr/lib/finit/system/20-hw-wait.conf

@@ -0,0 +1,9 @@
+# Extend finit's default udevadm settle synchronization for situations
+# where device are very slow to probe (see #685)
+run nowarn if:udevd cgroup.init [S] <service/udevd/ready> log \
+	/usr/libexec/infix/hw-wait -- Probing hardware
+
+# Now that everything should be probed, do a final pass over the
+# uevent queue before starting syslogd and everything else
+run nowarn if:udevd cgroup.init :post [S] <service/udevd/ready> log \
+	udevadm settle -t 30 --

board/common/rootfs/usr/libexec/infix/hw-wait

@@ -0,0 +1,41 @@
+#!/bin/sh
+# (Ab)use the kernel's device link subsystem to detect consumer side
+# devices that may be very slow to probe (looking at you, mv88e6xxx!).
+
+ident=$(basename "$0")
+
+report()
+{
+    if [ -r /tmp/$ident ]; then
+	logger -k -p user.$1 -t "$ident" "Waited for slow devices:"
+	sort /tmp/$ident | uniq -c | logger -k -p user.$1 -t "$ident"
+    fi
+
+    rm -f /tmp/$ident
+}
+
+for i in $(seq 50); do
+    again=
+
+    for dl in /sys/class/devlink/*; do
+    	[ -r "$dl/status" ] || continue
+
+    	status=$(cat "$dl/status")
+	if [ "$status" = "consumer probing" ]; then
+	    basename $(readlink "$dl/consumer") >>/tmp/$ident
+	    again=yes
+	fi
+
+    done
+
+    if [ -z "$again" ]; then
+	report notice
+	exit 0
+    fi
+
+    sleep .2
+done
+
+logger -k -p user.error -t "$ident" "Timeout waiting for devices to come online"
+report error
+exit 1