apertis-update-manager-ota-rollback manual

low

Image Types:
minimal-armhf / minimal-arm64
Image Deployment:
OSTree
Type:
functional

Description

Test the automatic rollback and blacklist mechanism of apertis-update-manager with network updates.


Resources

  • The DUT u-boot environment must be clean: in u-boot, run: `env default -a` followed by `saveenv`
  • A PC must be connected to DUT serial port
  • The DUT must be connected to network

Pre Conditions

  1. This test requires a properly configured time source: when testing devices not carrying a battery-backed real time clock in a network which prevents connections to NTP servers (and only in that case) manually ensure that the time on the device is set appropriately and that it is propagated to the hardware clock so it is stay set on the next reboot if the power is not plugged off.
  2. $ sudo timedatectl --adjust-system-clock set-time "2019-08-21 18:49:03"
  3. If direct access to repository with updates for DUT is restricted and proxy server should be used, then need to add the address of this proxy for OSTree on DUT by command:
  4. $ sudo ostree config set 'remote "origin"'.proxy "http://10.168.128.45:3128"

Execution Steps

  1. Check the initial deployment
  2. $ sudo ostree admin status
  3. Prepare the copy of commit and deploy to allow the upgrade to the same version
  4. Command below shows you an initial commit ID, for instance
  5. $ export BOOTID=$(sudo ostree admin status | sed -n -e 's/^\* apertis \([0-9a-f]*\)\.[0-9]$/\1/p'); echo $BOOTID
  6. Get the Collection ID and ref
  7. $ export CID=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 1 -d ' '); echo COLLECTION_ID=$CID
    $ export REF=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 2 -d ' '); echo REF=$REF
  8. Create the commit with changed timestamp to allow upgrade with recent update file
  9. $ export NEWID=$(sudo ostree commit --orphan --tree=ref=$BOOTID --add-metadata-string=ostree.collection-binding=$CID --bind-ref=$REF --timestamp="1 year ago"); echo "New commit: $NEWID"
  10. Deploy the prepared commit
  11. $ sudo ostree admin upgrade --allow-downgrade --deploy-only --override-commit=$NEWID --reboot
  12. Wait until the system is booted again and check the deployment
  13. $ sudo ostree admin status
  14. The booted commit (started with '*') must have ID which we prepare and the initial commit ID should be marked as '(rollback)'
  15. Remove the initial deployment
  16. $ sudo ostree admin undeploy 1
  17. Reboot the system
  18. Check the current deployment
  19. $ sudo ostree admin status
  20. Remove blacklist file if it exists
  21. $ sudo rm -f /var/aum_blacklist.conf
  22. Start the user interface agent with mode preventing automatic system reboot after update
  23. $ sudo updatectl --register-upgrade-handler &
  24. Enable network updates with CLI tool
  25. $ sudo updatectl --check-network-updates
  26. Check that the user interface agent reports the pending update
  27. ** Message: 21:31:26.117: Network connected: Yes

    ** Message: 21:31:26.123: Upgrade status: Checking

    ** Message: 21:31:28.057: Upgrade status: Downloading

    ** Message: 21:31:51.170: Upgrade status: Deploying

    ** Message: 21:32:11.879: An upgrade is pending

  28. After the update, the device does *not* reboot automatically
  29. Check if there is pending deployment and reboot the DUT
  30. $ sudo ostree admin status
    $ sudo reboot
  31. In `U-Boot` console check the status of upgrade
  32. $ printenv bootcount bootlimit

    bootcount=1

    bootlimit=3

  33. Restart the device by pressing the restart button before the boot finishes.
  34. Restart the device a second time by pressing the restart button before the boot finishes.
  35. Restart the device a third time by pressing the restart button before the boot finishes.
  36. U-Boot should be able to detect the rollback mode and boot the system in rollback mode
  37. Warning: Bootlimit (3) exceeded. Using altbootcmd.

    Hit any key to stop autoboot:  0 

    switch to partitions #0, OK

    mmc0 is current device

    Scanning mmc 0:1...

    Found /extlinux/extlinux-rollback.conf

    Retrieving file: /extlinux/extlinux-rollback.conf

  38. Wait for system boot
  39. Wait a few seconds after the boot to allow ostree to undeploy the deployment. Check the update has been rolled back and that only single deployment exists.
  40. $ sudo ostree admin status
  41. Check if the file with blacklisted commit exists
  42. $ cat /var/aum_blacklist.conf

    [blacklist]

    1239e167bddff9464efd58695ea15edff6679964aee20e1b1b9d6a126bd66941=true

  43. Start the user interface agent
  44. $ sudo updatectl &
  45. Enable network updates with CLI tool
  46. $ sudo updatectl --check-network-updates
  47. Check that the user interface agent reports the system is up to update
  48. ** Message: 21:47:11.501: Network connected: Yes

    ** Message: 21:47:11.508: Upgrade status: Checking

    ** Message: 21:47:13.079: Upgrade status: Downloading

    ** Message: 21:47:13.080: System is up to date

  49. Check the journal log should mention that the update ID has been blacklisted
  50. $ sudo journalctl -ef --unit apertis-update-manager

    Jun 01 22:01:46 apertis apertis-update-[365]: Auto update status: active

    Jun 01 22:01:46 apertis apertis-update-[365]: Ostree upgrade poll starting

    Jun 01 22:01:47 apertis apertis-update-[365]: Ostree upgrade progress: 1 metadata, 0 content objects fetched

    Jun 01 22:01:47 apertis apertis-update-[365]: Revision '1239e167bddff9464efd58695ea15edff6679964aee20e1b1b9d6a126bd66941' is marked as blacklisted; skipping

    Jun 01 22:01:47 apertis apertis-update-[365]: Ostree already up to date


Expected

U-Boot is able to detect rollback situation

U-Boot is able to use rollback configuration for bootloader

The "failed" update is rolled back

"Failed" update is marked as blacklisted

Apertis-update-manager is able to detect blacklisted update and refuse to update the system with it