Upgrades with Multiple Hops Fails

[GrantGabbert]

Describe the bug

When performing an upgrade that requires multiple steps, the upgrade fails. The failure message was from panorama that the middle software version couldn't be installed because it wasn't downloaded.

Expected behavior

The upgrade should complete successfully, even with multiple steps.

Current behavior

When upgrading for multiple versions (our example was from 10.1.? to 11.1.3-h2 and the upgrade path was identified as 10.2.10-h3, 11.0.5-h1, 11.1.3-h3), the following is what we observed...

• Upgrade to 10.2.10-h3 completed successfully with no issues
• When upgrading to 11.0.5-h1, the software download is started
• Before the software download is completed, the process continues and the software install is started
• Software install fails because the download hasn't been completed

When looking at the war room...

• The command to get the software download status showed the download was in progress
• The conditional task to check if the software download was completed took the yes branch, despite the previous result showing it hadn't completed

And checking the context...

• The previous download and install results were both still in context

Before digging in, we reran the failed task and it started the software install and then rebooted before waiting for the install to complete.

Possible solution

Two solutions I can think of, but haven't had a chance to walk through could be...

1. Clean up context between each upgrade. But this would lose some history
2. Change the 'Yes' conditions to be able to handle multiple iterations of the polling tasks (download, install, reboot).

The customer has also reported that this upgrade path didn't require full upgrades. So potentially some tweaks to the logic that determines the upgrade path could also be beneficial.

Steps to reproduce

1. Start an upgrade with multiple steps required, our use case was from a 10.1 software to 11.1.3-h2
2. Wait for the upgrade to fail

Screenshots

War room showing job check for sw download not completed at 10:29:11, but upgrade process continuing at 10:29:12 (customer info removed)

Context showing both downloads in context. Second, incomplete one as second item (customer info removed)

Context

Customer was planning on upgrading their entire fleet this weekend with XSOAR. This bug was found and stops the customer from using XSOAR FW Upgrade Assurance and now they will need to manually upgrade.

Your Environment