apertis-update-manager-rollback manual

low

Image Types:
fixedfunction-amd64 / fixedfunction-armhf / fixedfunction-arm64
Image Deployment:
OSTree
Type:
functional

Description

Test the automatic rollback and blacklist mechanism of apertis-update-manager. The automated version of this test: https://qa.apertis.org/aum-rollback-blacklist.html


Resources

  • A static encrypted update bundle file of the same architecture, variant and version as the testing image
  • A Fat32 USB flash drive, preloaded with the encrypted update bundle at the root of the disk
  • The latest static encrypted update file can be downloaded at the same location than the Apertis image. It has the same basename, and a '.delta.enc' extension
  • The static encrypted update file should be copied to the flash drive using the name 'static-update.bundle.enc'.
  • A PC must be connected to DUT serial port

Pre Conditions

  1. Clone the tests repository from another computer (Note that the branch being tested may change depending on the release, please make sure to clone the correct branch for the release in question):
  2. $ git clone --branch apertis/v2023dev1 https://gitlab.apertis.org/tests/aum-offline-upgrade.git
  3. Copy the test directory aum-offline-upgrade to the target device:
  4. $ DUT_IP=<device-ip>
    $ scp -r aum-offline-upgrade user@$DUT_IP:
  5. Log into the target device:
  6. $ ssh user@$DUT_IP
  7. Lilliput/LVDS touch screens must be connected to the respective targets

Execution Steps

  1. Install boot-delay service
  2. $ sudo ./aum-offline-upgrade/boot-delay-install.sh
  3. Check the initial deployment
  4. $ sudo ostree admin status
  5. Prepare the copy of commit and deploy to allow the upgrade to the same version
  6. Command below shows you an initial commit ID, for instance
  7. $ export BOOTID=$(sudo ostree admin status | sed -n -e 's/^\* apertis \([0-9a-f]*\)\.[0-9]$/\1/p'); echo $BOOTID
  8. Get the Collection ID and ref
  9. $ export CID=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 1 -d ' '); echo COLLECTION_ID=$CID
    $ export REF=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 2 -d ' '); echo REF=$REF
  10. Create the list of files to skip and ensure there are some files in these directories
  11. $ ls -1d /usr/share/locale /usr/share/man /usr/share/zoneinfo > /tmp/skip
    $ du -sh /usr/share/locale /usr/share/man /usr/share/zoneinfo
  12. Create the commit with changed timestamp and skipped list from above to allow upgrade with recent update file
  13. $ export NEWID=$(sudo ostree commit --orphan --tree=ref=$BOOTID --add-metadata-string=ostree.collection-binding=$CID --bind-ref=$REF --timestamp="1 year ago" --skip-list=/tmp/skip); echo "New commit: $NEWID"
  14. Deploy the prepared commit
  15. $ sudo ostree admin upgrade --allow-downgrade --deploy-only --override-commit=$NEWID --reboot
  16. Wait until the system is booted again and check the deployment
  17. $ sudo ostree admin status
  18. The booted commit (started with '*') must have ID which we prepare and the initial commit ID should be marked as '(rollback)'
  19. Check booted deployment have no file objects which we skip
  20. $ du -sh /usr/share/locale /usr/share/man /usr/share/zoneinfo
  21. Remove the initial deployment
  22. $ sudo ostree admin undeploy 1
  23. Reboot the system
  24. Check the current deployment
  25. $ sudo ostree admin status
  26. Start the user interface agent with mode preventing automatic system reboot after update
  27. $ sudo updatectl --register-upgrade-handler &
  28. Plug the USB flash drive into the device
  29. The update starts automatically
  30. After the update, the device does *not* reboot automatically
  31. Check that the user interface agent reports the pending update
  32. ** Message: Upgrade status: Checking

     ** Message: An upgrade is pending

  33. Remove the USB flash drive
  34. Check if there is pending deployment and reboot the DUT
  35. $ sudo ostree admin status
    $ sudo reboot
  36. During boot a message will be displayed with a countdown
  37. Restart the device a first time by pressing the restart button before the countdown is over.
  38. Restart the device a second time by pressing the restart button before the countdown is over.
  39. Restart the device a third time by pressing the restart button before the countdown is over.
  40. The system should be able to detect the rollback mode and boot the system in rollback mode
  41. On platforms using U-Boot (iMX.6 SABRELite, Raspberry Pi, R-Car, etc.), Boot should warn about initiating the rollback boot (altbootcmd)
  42. Warning: Bootlimit (3) exceeded. Using altbootcmd.

    Hit any key to stop autoboot:  0 

    switch to partitions #0, OK

    mmc0 is current device

    Scanning mmc 0:1...

    Found /extlinux/extlinux-rollback.conf

    Retrieving file: /extlinux/extlinux-rollback.conf

  43. On platforms using UEFI (amd64), by default there's no information that rollback mode has been detected
  44. Wait for system boot
  45. Wait a few seconds after the boot to allow ostree to undeploy the deployment. Check the update has been rolled back and that only single deployment exists.
  46. $ sudo ostree admin status
  47. Start the user interface agent
  48. $ sudo updatectl &
  49. Plug the USB flash drive with the same update file into the device
  50. Check that the user interface agent reports the system is up to update
  51. ** Message: 09:21:45.026: Upgrade status: Checking

    ** Message: 09:21:45.300: System is up to date

  52. Check the journal log should mention that the update ID has been blacklisted
  53. $ sudo journalctl -ef --unit apertis-update-manager

    May 01 09:21:45 apertis apertis-update-[363]: mount added : /media/APERTIS

    May 01 09:21:45 apertis apertis-update-[363]: mount added : /media/update

    May 01 09:21:45 apertis apertis-update-[363]: Ostree static delta starting

    May 01 09:21:45 apertis apertis-update-[363]: Metadata read from commit '710dbec2943510e4deb279cd6738a4a1a5b589eb6c4976e486d056e0308a02e1': {'ostree.ref-binding': <['apertis/v2023dev3/armhf-uboot/fixedfunction']>, 'ostree.collection-binding': <'org.apertis.os'>}

    May 01 09:21:45 apertis apertis-update-[363]: Revision '710dbec2943510e4deb279cd6738a4a1a5b589eb6c4976e486d056e0308a02e1' is marked as blacklisted; skipping

    May 01 09:21:45 apertis apertis-update-[363]: Ostree already up to date


Expected

The system is able to detect rollback situation

The system is able to use rollback configuration for bootloader

The "failed" update is rolled back

"Failed" update is marked as blacklisted

Apertis-update-manager is able to detect blacklisted update and refuse to update the system with it