Categories
Technology VMware

ESXi Host Upgrades fail with space warnings.

Have you been trying to update a stand alone ESXi host and running into the dreaded “no space left on device” error?

Failed to update bootloader: [Errno 28] No space left on device

Yeah, me too. Of course I check to see if there’s enough space – and there is. So what to do?

Well, I skipped the CLI update from VMware.com and downloaded the patch directly from VMware and SCP’ed it up to my host.

Then ran the command to run the patch locally:esxcli software vib install -d "/vmfs/volumes/5fc129bb-3b799b8c-57ab-001f296187c0/patches/VMware-ESXi-7.0U1b-17168206-depot.zip"

Behold, a new fresh and much more helpful error message:
[DependencyError]
VIB QLC_bootbank_qedf_2.2.4.0-1OEM.700.1.0.15525992 requires qedentv_ver = X.40.9.0, but the requirement cannot be satisfied within the ImageProfile.
VIB QLC_bootbank_qedi_2.19.5.0-1OEM.700.1.0.15525992 requires qedentv_ver = X.40.9.0, but the requirement cannot be satisfied within the ImageProfile.
Please refer to the log file for more details.

(bold is my doing to show you where I got the driver names from)


Ha, ok. So we’re not dealing with space issue – we’re dealing with one confusing error message and two dependency errors.

The drivers my HPE server has installed for QEDF and QEDI (both are storage drivers) are too old for 7.0 Update 1. So either I need to update them, wait for HPE to get their image through validation and release an OEM version of 7.0 Update 1, or remove the offending drivers.

I’m opting for the third option – if I’m not using them.

To find out if I’m actually using them, I can run the following commands:
esxcli storage core adapter list | grep -i qedf
and
esxcli storage core adapter list | grep -i qedi

If it returns nothing, then they’re not in use. To see what a hit looks like, try the same commands without the pipe to grep.

Mine spits out this list:
[root@esxi01:~] esxcli storage core adapter list
HBA Name Driver Link State UID Capabilities Description
-------- -------- ---------- ----------- ------------ -----------
vmhba0 vmw_ahci link-n/a sata.vmhba0 (0000:00:11.0) Advanced Micro Devices Inc AMD FCH SATA Controller [AHCI Mode]
vmhba1 vmw_ahci link-n/a sata.vmhba1 (0000:01:00.0) Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller
vmhba32 vmkusb link-n/a usb.vmhba32 () USB

This confirms I can safely remove both drivers (vibs) without much ado.

First, list the drivers out. Here’s an example after I removed qedi but before I removed qedf.

[root@esxi01:~] esxcli software vib list | grep qedi
[root@esxi01:~] esxcli software vib list | grep qedf
qedf 2.2.4.0-1OEM.700.1.0.15525992 QLC VMwareCertified 2020-08-10


To remove qedf, I simply run:
[root@esxi01:~] esxcli software vib remove -n qedf

This can take a while (like 1-3 minutes). If everything is successful, you’ll get this result:
Removal Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed:
VIBs Removed: QLC_bootbank_qedf_2.2.4.0-1OEM.700.1.0.15525992
VIBs Skipped:

Follow the recommendation and reboot your host. Upon your next attempt to patch, you should have smooth sailing from here on out. Also remember, if you end up upgrading in the future with an OEM version, these drivers may be reinstalled – hopefully the newer ones.

After reboot, I ran the update again:
[root@esxi01:~] esxcli software vib install -d "/vmfs/volumes/5fc129bb-3b799b8c-57ab-001f296187c0/patches/VMware-ESXi-7.0U1b-17168206-depot.zip"
Installation Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

(Removed the exhaustive list of VIBs that were touched, not touched, and removed)

Source for my wisdom was found on VMware’s KB site: https://kb.vmware.com/s/article/78389

Leave a Reply