Dec 072016
 

After successfully completing the migration from vCenter 6.0 (on Windows) to the vCenter 6.5 Appliance, all I had remaining was to upgrade my ESXi hosts to ESXi 6.5.

In my test environment, I run 2 x HPe Proliant DL360p Gen8 servers. I also have always used the HPe customized ESXi image for installs and upgrades.

It was easy enough to download the customized HPe installation image from VMware’s website, I then loaded it in to VMware Update Manager on the vCenter appliance, created a baseline, and was prepared to upgrade the hosts.

I successfully upgraded one of my hosts without any issues, however after scanning on my second host, it reported the upgrade as incompatible and stated: “The upgrade contains the following set of conflicting VIBs: Mellanox_bootbank_net.XXXXversionnumbersXXXX. Remove the conflicting VIBs or use Image Builder to create a custom ISO.”

I checked the host to see if I was even using the Mellanox drivers, and thankfully I wasn’t and could safely remove them. If you are using the drivers that are causing the conflict, DO NOT REMOVE them as it could disconnect all network interfaces from your host. In my case, since they were not being used, uninstalling them would not effect the system.

I SSH’ed in to the host and ran the following commands:

esxcli software vib list | grep Mell (This shows the VIB package that the Mellanox driver is inside of. In my case, it returned “net-mst”)

esxcli network nic list (this command verifies which drivers you are using on your network interfaces on the host)

esxcli software vib remove -n net-mst (this command removes the VIB that contains the problematic driver)

After doing this, I restarted the host, scanned for upgrades, and successfully applied the new vCenter 6.5 ESXi Customized HPe image.

Leave a comment!

  30 Responses to “VMware vSphere 6.5 – Conflicting VIBs when upgrading ESXi 6.0 to 6.5”

  1. […] A note on a problem I dealt with during ESXi 6.5 upgrade (conflicting VIBs marking image as incompatible when deploying HPe customized image) can be found here: http://www.stephenwagner.com/?p=1120 […]

  2. Just went through this myself, thanks for the post! A few notes:

    The VIBs my esxi hosts complained about were different, but still not used, so I had to change the grep/remove commands to match

    The VIB remove commands take a loooong time on my boxes to execute, like 5+ min, and I thought they were stalled/doing something horrible. Nope, they just take a long ass time.

  3. Thank you! Same problem. Worked like a charm.

  4. Hi All,

    Need your help. I tried to upgrade our host Esxi 6.0.0, 3380124 to ESXi-6.5.0-20170104001-standard but got this error “The upgrade contains the following set of conflicting VIBs:” I am fully aware that I should delete if conflicting VIB but on this one it doesn’t display the conflicting Vib.

    Thanks,
    JC

  5. Hi JC,

    Is this happening on one, or multiple hosts?

    If it’s one, maybe burn an ISO of the install image and try upgrading that way.

    Have you checked any other logs to see what package is causing the conflict?

    Out of curiosity, are you using a custom image, have you loaded any special VIBs or drivers on your install?

    Cheers

  6. Hi JC,

    My problem is a bit different, I wasn’t upgrading to a higher version of VM Ware vSphere, I was simply installing ESXi 6.5 on my first lab ESXi host as part of my VMware training, and got the following error:

    <CONFLICTING_VIBS_ERROR: Vibs on the host are conflicting with vibs in metadata. Remove conflicting vibs or use Image Builder to create a custom ISO providing newer versions of the conflicting vibs. ['Hewlett-Packard_bootbank_char-hpcrv_5.5.6.6-10EM-550.0.0.1198610'
    'Mellanox_bootbank_net-mst_2.0.0.0-10EM.550.0.0.600000', etc, etc

    Thanks for your help. Jorge

  7. Hi Jorge,

    It sounds like your install image is corrupt, or you’re maybe installing on to a previous install?

    I would recommend zero’ing the drive you are installing to, as well as re-downloading the ESXi installer images.

    Cheers

  8. In my case my error was:
    HUAWEI_bootbank_hio

    So i ran this command:
    esxcli software vib remove -n hio

    After confirming it wasnt being used. Thanks for the article.

  9. Hello I found your article very helpful. I had bootbank vib conflicts on HP DL360p G8s.

  10. Ever seen it went there were about 15 conflicting vibs? I am upgrading a different ESXI environment and one of my host has about 15 vibs conflicting.

  11. Hi Joe,

    That’s crazy. I’m curious what the conflicts are?

    Have you tried doing an upgrade using the install media (instead of vSphere Update Manager)?

    If that was me, I may just blow away the host and do a fresh install after trying to two methods above. For that to happen, it sounds like the host is either old, or has had quite a bit of work done to it while it’s been in production!

    Cheers,
    Stephen

  12. I had this exact same problem and your solution fixed it. Thanks!

  13. Same issue – but 3 identical hosts 1st one upgraded fine the second two come up with the VIB issues. All kit 1 year old and super fast. going to try the ISO system

  14. This helped me aswell – thank you so much!

    I had 3 VIBS that I removed on 6 hosts and the upgrade to ESXi 6.5 went smooth.

  15. How would one determine if the conflicting VIB is in use or not by the host? I’d like to remove the conflicts and just do an upgrade via Update Manager rather than rebuild the OS.

    Our environment happens to have 11 hosts complaining about;
    LSI_bootbank_scsi-mpt3sas_04.00.00.00.1vmw-1OEM.500.0.0.472560
    LSI_bootbank_scsi-mpt3sas_04.00.00.00.1vmw-1OEM.500.0.0.472560

  16. Hi Robert,

    In your case I think those VIBs are needed as they have to do with your SAS/SCSI/HBA Host Adapter (you may need to verify this).

    In some cases, the issue is recoverable… In other’s it’s not and a full re-install may be needed…

    Cheers,
    Stephen

  17. Thank you for the reply. I am going to try 1 redundant host to see what happens. All of our hosts have redundant internal SD cards with the ESXi installation on them so hopefully we wouldn’t need the host adapter/controller as we’re not using any 3.5 internal storage.

  18. HI Robert

    In that case you should be fine. I’m pretty sure that driver is for internal storage only.

    Let me know how you make out…

    Cheers

  19. Thanks so much. Worked for me

  20. Hi Robert,
    Thanks for wonderful article

    I get the below conflicts for one of the host
    ________________________________________
    DEBUG:root:Potentially conflicting vibs : set(['VMware_bootbank_scsi-mpt2sas_19.00.00.00-1vmw.600.0.0.2494585', 'VMware_bootbank_sata-ahci_3.0-26vmw.600.3.57.5050593', 'VMware_bootbank_ehci-ehci-hcd_1.0-4vmw.600.3.69.5572656', 'MEL_bootbank_nmlx4-en_3.15.5.5-1OEM.600.0.0.2768847', 'VMware_bootbank_net-e1000e_3.2.2.1-2vmw.600.3.57.5050593', 'VMware_bootbank_vsan_6.0.0-3.69.5568629', 'VMware_bootbank_esx-ui_1.19.0-5387100', 'VMware_bootbank_vsanhealth_6.0.0-3000000.3.0.3.69.5572665', 'VMware_bootbank_lsu-hp-hpsa-plugin_2.0.0-4vmw.600.3.57.5050593', 'MEL_bootbank_nmlx4-core_3.15.5.5-1OEM.600.0.0.2768847', 'MEL_bootbank_nmlx4-rdma_3.15.5.5-1OEM.600.0.0.2768847', 'VMware_bootbank_xhci-xhci_1.0-3vmw.600.3.69.5572656', 'QLogic_bootbank_ima-qla4xxx_500.2.01.31-1vmw.0.3.100400', 'QLogic_bootbank_scsi-qla4xxx_644.55.36.0-1OEM.550.0.0.1331820', 'NetApp_bootbank_NetAppNasPlugin_1.1.0-0', 'VMware_bootbank_net-vmxnet3_1.1.3.0-3vmw.600.2.43.4192238', 'VMware_bootbank_vmware-fdm_6.0.0-3634794', 'VMware_bootbank_esx-xserver_6.0.0-3.66.5485776', 'VMware_bootbank_qlnativefc_2.1.50.0-1vmw.600.3.57.5050593', 'VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-3vmw.600.3.57.5050593', 'VMware_bootbank_esx-tboot_6.0.0-3.57.5050593', 'VMware_bootbank_misc-drivers_6.0.0-3.69.5572656', 'VMware_bootbank_lsu-lsi-megaraid-sas-plugin_1.0.0-3vmw.600.3.57.5050593','VMware_bootbank_esx-base_6.0.0-3.69.5572656'])
    ____________________________________________________________

    Do you think I will be a good idea to remove all the vibs and reboot the host.

  21. So this was the same problem at my site – but how to see this coming before you run the ESXi update? Because after i have run the update and i get this error I have to reboot the host and then i need to start the installation process from scratch (no update option)! Any ideas for future upgrades?

  22. This message is from the logs after upgrade was done.
    So now there is an error for this host on in update manager. need to fix that

  23. Thanks for your help very useful 🙂

    Used commands to resolve conflicts with Dell PowerEdge servers 720/410 upgrading to 6.5.0 update 1

  24. Just a quick note for the googlers – I ran into this issue when I tried to go from 6.0 U1 to 6.5 on the Dell Custom ISO. Updating the host to 6.0U3 then 6.5 worked for me without having to remove the VIB. YMMV.

  25. Thank you!

    Worked for me when updating 5.5 to 6.5 with HPE custom iso

  26. I found the “VMware-ESXi-5.5.0-Update3-3568722-HPE-550.9.6.5.9-Dec2016-depot.zip” update for my HP ESXi 5.5 server that appeared to have the required files/vibs that was needed to do the upgrade to 6.5

  27. I came across this on one of my Dell R710 servers that is currently on 6.0. Trying to update to 6.5 (from Dell Bootable Media), I get the error message about conflicting VIBs. Now, I downloaded this Dell ISO image for an R720 server, and in the past, have been able to use the same media to upgrade both the R720s and R710s with it. When I went to the Dell compatability matrix for 6.5 on Dell servers, the R710 is not even listed anywhere, so I assume the hardware isn’t supported any longer. Not a huge deal as these servers were repurposed from being CASHUB Exchange servers a few years ago and only have a couple VM’s that users connect to for a few specific applications, so if I can’t upgrade them, it’s not the end of the world. Hopefully this helps someone looking for the info on this same server. It’s unsupported by dell for ESXi 6.5. Dell matrix: http://topics-cdn.dell.com/pdf/vmware-esxi-6.5.x_reference%20guide_en-us.pdf

  28. Thanks dude. Managed to upgrade my HPE microserver gen7 N54L from esxi 6.0 update-2 to 6.5 update-1, thanks to your info.
    Cheers mate!!!

  29. Hello Robert,

    First, thanks for this post as it sent me in the right direction. For me, the difficult part was to understand the various command output, because I have never dealt with them before.

    So if this can help someone:
    1- The command : esxcli software vib list | grep “xxxx”
    “xxxx” is the name of the package. For example, in the error: “Brocade_bootbank_net-bna_3.0.3.0-10EM.5000.0.47250” the required portion of the VIB name is “Brocade”.

    2- In the previous error, we can already see the driver name, while this is not evident. In this case the driver name is “net-bna”, followed by the version number.

    3- The command “esxcli network nic list” is fine to list the driver loaded for the NIC cards, but it does not cover the other errors I had related to SCSI adapter.

    4- The output of the command, “esxcli software vib list | grep Brocade” produces:

    net-bna 3.0.3.0-1OEM.500.0.0.472560 Brocade VMwareCertified 2016-10-17
    scsi-bfa 3.0.3.0-1OEM.500.0.0.472560 Brocade VMwareCertified 2016-10-17

    The first field of the line is the driver name. In this example, I needed to remove the net-bna driver (the error in step 1). It took me a bit to understand what was the driver name because of the format of the output, the limited screen real estate I had at the time, and the fact I was wondering if there were missing lines in the output. So, if you are like me and want to get the smallest details ironed out before proceeding, then know that the first field IS the driver name.

    5- I had another error pertaining to a SCSI driver. I found that I could check if it was loaded using the command “esxcfg-info”. This produces a very long output, so I copied the results to my workstation, opened and search through it using notepad++. Once I was satisfied that the SCSI driver was not found, in the file, making multiple searches (LSI, mpt3sas, any other string I thought was relevant), I removed it.

    6- Needless to say, I moved all the VM off the host before starting and set it in maintenance mode. I took screenshots of the config in case I blew something up and be forced to start from scratch, but in the end, everything went well.

    7- Everything I`ve read regarding this issue, the drivers are always unused. What happens if they are in use? I couldn’t find any solutions, so I guess reformatting the installation partition and starting from scratch, is the only solution I can think of. If someone went through this and has something to add, please do. Fortunately, that was not the case for me.

    8- One final note, VMWare complains that the host must be rebooted for the changes to take effect, I did not reboot between the removal, but I rebooted and let the server go until it was completely booted before proceeding with the update. I felt it necessary regardless if the updates are done from VUM or directly from the custom CD.

  30. Well after waiting forever to get a maintenance window I was able to removed the following VIBs from our hosts;

    LSI_bootbank_scsi-mpt3sas_04.00.00.00.1vmw-1OEM.500.0.0.472560
    LSI_bootbank_scsi-mpt3sas_04.00.00.00.1vmw-1OEM.500.0.0.472560

    After rebooting the hosts and scanning for updates/upgrades it was listing the 6.5 upgrade as Not Compliant rather than Not Compatible now. I was able to successfully upgrade to 6.5 without having to do a full fresh install of esxi on our 11 hosts.

    I don’t remember ever using a customized esxi installer, however, I cannot explain otherwise how those drivers were supplied on our hosts even though they are not used. Either way I was able to remove the hurdle and get things upgraded without any issues.

    Thanks for the great article Stephen!

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)