updatehub-rollback-guard: Add guard for updates which never validate - #124
Open
otavio wants to merge 1 commit into
Open
updatehub-rollback-guard: Add guard for updates which never validate#124otavio wants to merge 1 commit into
otavio wants to merge 1 commit into
Conversation
Rolling a bad update back relies on the bootloader counting boot attempts, and that counter only advances when the board actually reboots: in bootcount_env.c both bootcount_load and bootcount_store are no-ops while upgrade_available is 0, and a healthy image clears that flag through updatehub-active-validated. A boot which fails services but leaves the init system running is not a hang, so nothing reboots it. The init system stays up and keeps petting the hardware watchdog, the counter never reaches its limit, and the broken update sits in the active slot indefinitely -- unreachable, when the same failure also takes the network down. Add a package closing that gap: a timer started at boot which, once UPDATEHUB_VALIDATION_TIMEOUT has elapsed without the image having validated itself, reboots so the bootloader counts the attempt and the boot script can switch back to the previous slot. It deliberately does not write bootcount or updatehub_active, as incrementing is U-Boot's job and the slot decision belongs to the boot script; causing the reboot is enough. It honours BOOTCOUNT_ENV from /etc/default/updatehub-active, so a machine keeping the counter outside the U-Boot environment is left alone rather than policed against state the backend never wrote. The timer sets DefaultDependencies=no and is wanted by emergency.target and rescue.target as well as timers.target. Both are needed: the default dependencies would order it after sysinit.target, which a boot broken early enough never reaches, and those two targets isolate without pulling timers.target in. Missing either leaves the timer inert in precisely the boot it exists to rescue. Enable it from updatehub-runtime.bbclass alongside the active/inactive backend rather than requiring every machine to opt in. Two conditions gate it, both of which would otherwise leave the package installed but silently doing nothing: the u-boot backend, since the boot counter state is read from the U-Boot environment, and systemd in DISTRO_FEATURES, since the grace period is a timer. Expressing a grace period under sysvinit needs a different mechanism, so no attempt is made to enable it there and those images keep their previous behaviour. Tested on an Allwinner A40i board (kirkstone-based downstream tree). An update whose boot failed to mount the data partition dropped to the emergency shell and left the board unreachable for thirty minutes, recoverable only from the serial console. With the guard the same failure rebooted itself once the grace period expired, U-Boot counted the attempt and reverted to the previous slot, and the board was back on the network about five minutes after the failed boot with no console intervention. A healthy update validates within seconds and is left untouched. Also verified that a sysvinit DISTRO_FEATURES leaves the package out of UPDATEHUB_RUNTIME_PACKAGES and parses without errors. Backport of master commit 68daa34 (PR #122). Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #122 to
langdale.Guards against updates that install successfully but never call
updatehub-active-validated(e.g. a failed data-partition mount that drops to an emergency shell but leaves systemd/the watchdog running): a systemd timer reboots the board afterUPDATEHUB_VALIDATION_TIMEOUT(default 5min) so U-Boot counts the failed attempt and rolls back. Enabled automatically with theu-bootactive/inactive backend whensystemdis inDISTRO_FEATURES; sysvinit images are untouched.See #122 for the full description and hardware test notes. No functional changes here beyond adapting to this branch's recipe conventions.