apertis-update-manager-rollback manual

low

Image Types:
fixedfunction-amd64 / fixedfunction-armhf / fixedfunction-arm64
Image Deployment:
OSTree
Type:
functional

Description

Test the automatic rollback and blacklist mechanism of apertis-update-manager. The automated version of this test: https://qa.apertis.org/aum-rollback-blacklist.html

Resources

  • A static encrypted update bundle file of the same architecture, variant and version as the testing image
  • A Fat32 USB flash drive, preloaded with the encrypted update bundle at the root of the disk
  • The latest static encrypted update file can be downloaded at the same location than the Apertis image. It has the same basename, and a '.delta.enc' extension
  • The static encrypted update file should be copied to the flash drive using the name 'static-update.bundle.enc'.
  • A PC must be connected to DUT serial port

Pre Conditions

  1. Clone the tests repository from another computer (Note that the branch being tested may change depending on the release, please make sure to clone the correct branch for the release in question):
    2. $ git clone --branch apertis/v2023dev1 https://gitlab.apertis.org/tests/aum-offline-upgrade.git
  2. Copy the test directory aum-offline-upgrade to the target device:
    3. $ DUT_IP=<device-ip>
    $ scp -r aum-offline-upgrade user@$DUT_IP:
  3. Log into the target device:
    4. $ ssh user@$DUT_IP
  4. Lilliput/LVDS touch screens must be connected to the respective targets

Execution Steps

  1. Install boot-delay service
    2. $ sudo ./aum-offline-upgrade/boot-delay-install.sh
  2. Check the initial deployment
    3. $ sudo ostree admin status
  3. Prepare the copy of commit and deploy to allow the upgrade to the same version
  4. Command below shows you an initial commit ID, for instance
    5. $ export BOOTID=$(sudo ostree admin status | sed -n -e 's/^\* apertis \([0-9a-f]*\)\.[0-9]$/\1/p'); echo $BOOTID
  5. Get the Collection ID and ref
    6. $ export CID=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 1 -d ' '); echo COLLECTION_ID=$CID
    $ export REF=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 2 -d ' '); echo REF=$REF
  6. Create the list of files to skip and ensure there are some files in these directories
    7. $ ls -1d /usr/share/locale /usr/share/man /usr/share/zoneinfo > /tmp/skip
    $ du -sh /usr/share/locale /usr/share/man /usr/share/zoneinfo
  7. Create the commit with changed timestamp and skipped list from above to allow upgrade with recent update file
    8. $ export NEWID=$(sudo ostree commit --orphan --tree=ref=$BOOTID --add-metadata-string=ostree.collection-binding=$CID --bind-ref=$REF --timestamp="1 year ago" --skip-list=/tmp/skip); echo "New commit: $NEWID"
  8. Deploy the prepared commit
    9. $ sudo ostree admin upgrade --allow-downgrade --deploy-only --override-commit=$NEWID --reboot
  9. Wait until the system is booted again and check the deployment
    10. $ sudo ostree admin status
  10. The booted commit (started with '*') must have ID which we prepare and the initial commit ID should be marked as '(rollback)'
  11. Check booted deployment have no file objects which we skip
    12. $ du -sh /usr/share/locale /usr/share/man /usr/share/zoneinfo
  12. Remove the initial deployment
    13. $ sudo ostree admin undeploy 1
  13. Reboot the system
  14. Check the current deployment
    15. $ sudo ostree admin status
  15. Start the user interface agent with mode preventing automatic system reboot after update
    16. $ sudo updatectl --register-upgrade-handler &
  16. Plug the USB flash drive into the device
  17. The update starts automatically
  18. After the update, the device does *not* reboot automatically
  19. Check that the user interface agent reports the pending update

    20. ** Message: Upgrade status: Checking

     ** Message: An upgrade is pending

  20. Remove the USB flash drive
  21. Check if there is pending deployment and reboot the DUT
    22. $ sudo ostree admin status
    $ sudo reboot
  22. During boot a message will be displayed with a countdown
  23. Restart the device a first time by pressing the restart button before the countdown is over.
  24. Restart the device a second time by pressing the restart button before the countdown is over.
  25. Restart the device a third time by pressing the restart button before the countdown is over.
  26. The system should be able to detect the rollback mode and boot the system in rollback mode
  27. On platforms using U-Boot (iMX.6 SABRELite, Raspberry Pi, R-Car, etc.), Boot should warn about initiating the rollback boot (altbootcmd)

    28. Warning: Bootlimit (3) exceeded. Using altbootcmd.

    Hit any key to stop autoboot:  0 

    switch to partitions #0, OK

    mmc0 is current device

    Scanning mmc 0:1...

    Found /extlinux/extlinux-rollback.conf

    Retrieving file: /extlinux/extlinux-rollback.conf

  28. On platforms using UEFI (amd64), by default there's no information that rollback mode has been detected
  29. Wait for system boot
  30. Wait a few seconds after the boot to allow ostree to undeploy the deployment. Check the update has been rolled back and that only single deployment exists.
    31. $ sudo ostree admin status
  31. Start the user interface agent
    32. $ sudo updatectl &
  32. Plug the USB flash drive with the same update file into the device
  33. Check that the user interface agent reports the system is up to update

    34. ** Message: 09:21:45.026: Upgrade status: Checking

    ** Message: 09:21:45.300: System is up to date

  34. Check the journal log should mention that the update ID has been blacklisted
    35. $ sudo journalctl -ef --unit apertis-update-manager

    May 01 09:21:45 apertis apertis-update-[363]: mount added : /media/APERTIS

    May 01 09:21:45 apertis apertis-update-[363]: mount added : /media/update

    May 01 09:21:45 apertis apertis-update-[363]: Ostree static delta starting

    May 01 09:21:45 apertis apertis-update-[363]: Metadata read from commit '710dbec2943510e4deb279cd6738a4a1a5b589eb6c4976e486d056e0308a02e1': {'ostree.ref-binding': <['apertis/v2023dev2/armhf-uboot/fixedfunction']>, 'ostree.collection-binding': <'org.apertis.os'>}

    May 01 09:21:45 apertis apertis-update-[363]: Revision '710dbec2943510e4deb279cd6738a4a1a5b589eb6c4976e486d056e0308a02e1' is marked as blacklisted; skipping

    May 01 09:21:45 apertis apertis-update-[363]: Ostree already up to date

Expected

The system is able to detect rollback situation

The system is able to use rollback configuration for bootloader

The "failed" update is rolled back

"Failed" update is marked as blacklisted

Apertis-update-manager is able to detect blacklisted update and refuse to update the system with it