apertis-update-manager-ota-rollback manual

medium

Image Types:
minimal-armhf
Image Deployment:
OSTree
Type:
functional

Description

Test the automatic rollback and blacklist mechanism of apertis-update-manager with network updates.

Resources

  • The DUT u-boot environment must be clean: in u-boot, run: `env default -a` followed by `saveenv`
  • A PC must be connected to DUT serial port
  • The DUT must be connected to network

Pre Conditions

  1. This test requires a properly configured time source: when testing devices not carrying a battery-backed real time clock in a network which prevents connections to NTP servers (and only in that case) manually ensure that the time on the device is set appropriately and that it is propagated to the hardware clock so it is stay set on the next reboot if the power is not plugged off.
    2. $ sudo timedatectl --adjust-system-clock set-time "2019-08-21 18:49:03"
  2. If direct access to repository with updates for DUT is restricted and proxy server should be used, then need to add the address of this proxy for OSTree on DUT by command:
    3. $ sudo ostree config set 'remote "origin"'.proxy "http://10.168.128.45:3128"

Execution Steps

  1. Check the initial deployment
    2. $ sudo ostree admin status
  2. Prepare the copy of commit and deploy to allow the upgrade to the same version
  3. Command below shows you an initial commit ID, for instance
    4. $ export BOOTID=$(sudo ostree admin status | sed -n -e 's/^\* apertis \([0-9a-f]*\)\.[0-9]$/\1/p'); echo $BOOTID
  4. Get the Collection ID and ref
    5. $ export CID=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 1 -d ' '); echo COLLECTION_ID=$CID
    $ export REF=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 2 -d ' '); echo REF=$REF
  5. Create the commit with changed timestamp to allow upgrade with recent update file
    6. $ export NEWID=$(sudo ostree commit --orphan --tree=ref=$BOOTID --add-metadata-string=ostree.collection-binding=$CID --bind-ref=$REF --timestamp="1 year ago"); echo "New commit: $NEWID"
  6. Deploy the prepared commit
    7. $ sudo ostree admin upgrade --allow-downgrade --deploy-only --override-commit=$NEWID --reboot
  7. Wait until the system is booted again and check the deployment
    8. $ sudo ostree admin status
  8. The booted commit (started with '*') must have ID which we prepare and the initial commit ID should be marked as '(rollback)'
  9. Remove the initial deployment
    10. $ sudo ostree admin undeploy 1
  10. Reboot the system
  11. Check the current deployment
    12. $ sudo ostree admin status
  12. Remove blacklist file if it exists
    13. $ sudo rm -f /var/aum_blacklist.conf
  13. Start the user interface agent with mode preventing automatic system reboot after update
    14. $ sudo updatectl --register-upgrade-handler &
  14. Enable network updates with CLI tool
    15. $ sudo updatectl --check-network-updates
  15. Check that the user interface agent reports the pending update

    16. ** Message: 21:31:26.117: Network connected: Yes

    ** Message: 21:31:26.123: Upgrade status: Checking

    ** Message: 21:31:28.057: Upgrade status: Downloading

    ** Message: 21:31:51.170: Upgrade status: Deploying

    ** Message: 21:32:11.879: An upgrade is pending

  16. After the update, the device does *not* reboot automatically
  17. Check if there is pending deployment and reboot the DUT
    18. $ sudo ostree admin status
    $ sudo reboot
  18. In `U-Boot` console check the status of upgrade
    19. $ printenv bootcount bootlimit

    bootcount=1

    bootlimit=3

  19. Restart the device by pressing the restart button before the boot finishes.
  20. Restart the device a second time by pressing the restart button before the boot finishes.
  21. Restart the device a third time by pressing the restart button before the boot finishes.
  22. U-Boot should be able to detect the rollback mode and boot the system in rollback mode

    23. Warning: Bootlimit (3) exceeded. Using altbootcmd.

    Hit any key to stop autoboot:  0 

    switch to partitions #0, OK

    mmc0 is current device

    Scanning mmc 0:1...

    Found /extlinux/extlinux-rollback.conf

    Retrieving file: /extlinux/extlinux-rollback.conf

  23. Wait for system boot
  24. Check the update has been rolled back and that the initial deployment is still the active deployment
    25. $ sudo ostree admin status
  25. Check if the file with blacklisted commit exists
    26. $ cat /var/aum_blacklist.conf

    [blacklist]

    1239e167bddff9464efd58695ea15edff6679964aee20e1b1b9d6a126bd66941=true

  26. Start the user interface agent
    27. $ sudo updatectl &
  27. Enable network updates with CLI tool
    28. $ sudo updatectl --check-network-updates
  28. Check that the user interface agent reports the system is up to update

    29. ** Message: 21:47:11.501: Network connected: Yes

    ** Message: 21:47:11.508: Upgrade status: Checking

    ** Message: 21:47:13.079: Upgrade status: Downloading

    ** Message: 21:47:13.080: System is up to date

  29. Check the journal log should mention that the update ID has been blacklisted
    30. $ sudo journalctl -ef --unit apertis-update-manager

    Jun 01 22:01:46 apertis apertis-update-[365]: Auto update status: active

    Jun 01 22:01:46 apertis apertis-update-[365]: Ostree upgrade poll starting

    Jun 01 22:01:47 apertis apertis-update-[365]: Ostree upgrade progress: 1 metadata, 0 content objects fetched

    Jun 01 22:01:47 apertis apertis-update-[365]: Revision '1239e167bddff9464efd58695ea15edff6679964aee20e1b1b9d6a126bd66941' is marked as blacklisted; skipping

    Jun 01 22:01:47 apertis apertis-update-[365]: Ostree already up to date

Expected

U-Boot is able to detect rollback situation

U-Boot is able to use rollback configuration for bootloader

The "failed" update is rolled back

"Failed" update is marked as blacklisted

Apertis-update-manager is able to detect blacklisted update and refuse to update the system with it