Oct 232020
 
vCSA Update Installation

When updating VMware vCenter vCSA 7.0 U1 (Build 16858589) to vCSA 7.0 U1 (Build 17004997/17005016, Version 7.0.1.00100), you may notice that the update fails and reports issues with pre-update checks.

Pre-update checks done prior to the update will pass and allow you to proceed, however it’s the installation that will fail and crash reporting this error.

After the installation fails, you will no longer be able to log in to the vCSA VAMI reporting the error “Unable to Login” using the root account.

You are able to login via SSH. Resetting the root password via SSH will not resolve this issue.

The Problem

In the past, issues with the root password expiring have caused similar behavior on the vCSA VAMI. Changing the root password does not resolve this specific issue.

Further troubleshooting, it appears that special characters in the root password such as “!”, “.”, and “@” caused this issue to occur in my environment.

I was not able to fix the broken vCSA after the failed update. Access to the vCSA was not possible, however vCenter functions were still operating.

The Solution

To resolve this situation in my environment, I restored a snapshot of the vCSA taken prior to updating.

After restoring the snapshot, I changed the root password for VAMI and restarted the vCSA.

Another snapshot was taken prior to attempting the upgrade, which was now succesfull after removing special characters out of the root password.

Oct 182020
 
Screenshot of The Tech Informative Side Chat - HPE Integrated Lights-Out

A new Side Chat Episode of the Tech Informative is now live on YouTube. In this episode we are covering HPE Integrated Lights-Out, also known as HPE iLO.

The Tech Informative Side Chat on HPE Integrated Lights-Out (HPE iLO)

The Tech Informative is a video podcast by Stephen Wagner and Rob Dalton that hopes to explore everyday technologies from the perspective of Information Technology professionals.

Rob Dalton is a lover of IT and a Director by profession. Rob considers himself a jack of all trades, an IT veteran, and is also the author of “Secured Packets”, a technology blog with a focus on security. Rob’s blog can be found at: https://www.securedpackets.com

Stephen Wagner is the President of Digitally Accurate Inc., an IT Solutions and Managed Services company. Stephen is also the author of “The Tech Journal”, an online Technology Blog. Stephen’s blog can be found at: https://www.stephenwagner.com

In this Side Chat, we cover the following:

What is HPE iLO

  • iLO and OneView
  • iLO Amplifier Pack
  • iLO, InfoSight, and HPE Servers

What does iLO do

  • Remote Access
  • Firmware Management
  • Server provisioning (Install OS, Recover, DR, etc.)
  • Features we find most valuable

How is HPE Integrated Lights-Out licensed

Competing Products

  • iDRAC
  • IPMI

We also cover what’s on the next episode of The Tech Informative

Don’t forget to like and subscribe! Leave a comment, feedback, or suggestions in the video comments section.

Oct 152020
 
VMware vCLS VM in VM List

Did a new VM appear on your VMware vSphere cluster called “vCLS”? Maybe multiple appeared named “vCLS (1)”, “vCLS (2)”, “vCLS (3)” appeared.

VMware vCLS VM in vSphere Cluster Objects
VMware vCLS VM in vSphere Cluster Objects

This could be frightening but fear not, this is part of VMware vSphere 7.0 Update 1.

What is the vCLS VM?

The vCLS virtural machine is essentially an “appliance” or “service” VM that allows a vSphere cluster to remain functioning in the event that the vCenter Server becomes unavailable. It will maintain the health and services of that cluster.

Where did the vCLS VM come from?

The vCLS VM will appear after upgrading to vSphere 7.0 Update 1. I’m assuming it was deployed during the upgrade process.

It does not appear in the standard Cluster, Hosts, and VMs view, but does appear when looking at the vSphere objects VM lists, Storage VM lists, etc…

Is it normal to have more than one vCLS VM?

The vCLS VMs are created when hosts are added to a vSphere Cluster. Up to 3 vCLS VMs are required to run in each vSphere Cluster.

The vCLS VMs will also appear on clusters which contain only one or two hosts. These configurations will result in either 1 or 2 vCLS VMs named “vCLS (1)” and “vCLS (2)”.

A note on licensing in regards to the vCLS VM

For VMware environments that use VM based licensing like vSphere for ROBO (Remote Office Branch Office), vCLS VMs are shown in the licensing interface as counting towards licensed VMs. Please Note that these VMs do not official count towards your purchased licenses as these are VMware System VMs. Please read VMware KB 80472 for more information on this.

More Information on vCLS VMs

For more information and technical specifics, you can visit the link below:

https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vcenterhost.doc/GUID-96BD6016-4BE7-4B1C-8269-568D1555B08C.html

Hope this posts helps, and puts some minds at ease. Your VMware environment has NOT been compromised.

Oct 102020
 

If you’re like me and use an older Nvidia GRID K1 or K2 vGPU video card for your VDI homelab, you may notice that when using VMware Horizon that VMware Blast h264 encoding is no longer being offloaded to the GPU and is instead being encoded via the CPU.

The Problem

Originally when an environment was configured with an Nvidia GRID K1 or K2 card, not only does the card provide 3D acceleration and rendering, but it also offloads the VMware BLAST h264 stream (the visual session) so that the CPU doesn’t have to. This results in less CPU usage and provides a streamlined experience for the user.

This functionality was handled via NVFBC (Nvidia Frame Buffer Capture) which was part of the Nvidia Capture SDK (formerly known as GRID SDK). This function allowed the video card to capture the video frame buffer and encode it using NVENC (Nvidia Encoder).

Ultimately after spending hours troubleshooting, I learned that NVFBC has been deprecated and is no longer support, hence why it’s no longer functioning. I also checked and noticed that tools (such as nvfbcenable) were no longer bundled with the VMware Horizon agent. One can assume that the agent doesn’t even attempt to check or use this function.

Symptoms

Before I was aware of this, I noticed that while 3D Acceleration and graphics were functioning, I was experiencing high CPU usage. Upon further investigation I noticed that my VMware BLAST sessions were not offloading h264 encoding to the video card.

VMware Horizon Performance Tracker
VMware Horizon Performance Tracker with NVidia GRID K1

You’ll notice above that under the “Encoder” section, the “Encoder Name” was listed as “h264 4:2:0”. Normally this would say “NVIDIA NvEnc H264” (or “NVIDIA NvEnc HEVC” on newer cards) if it was being offloaded to the GPU.

Looking at a VMware Blast session (Blast-Worker-SessionId1.log), the following lines can be seen.

[INFO ] 0x1f34 bora::Log: NvEnc: VNCEncodeRegionNvEncLoadLibrary: Loaded NVIDIA SDK shared library "nvEncodeAPI64.dll"
[INFO ] 0x1f34 bora::Log: NvEnc: VNCEncodeRegionNvEncLoadLibrary: Loaded NVIDIA SDK shared library "nvml.dll"
[WARN ] 0x1f34 bora::Warning: GetProcAddress: Failed to resolve nvmlDeviceGetEncoderCapacity: 127
[WARN ] 0x1f34 bora::Warning: GetProcAddress: Failed to resolve nvmlDeviceGetProcessUtilization: 127
[WARN ] 0x1f34 bora::Warning: GetProcAddress: Failed to resolve nvmlDeviceGetGridLicensableFeatures: 127
[INFO ] 0x1f34 bora::Log: NvEnc: VNCEncodeRegionNvEncLoadLibrary: Some NVIDIA nvml functions unavailable, unloading
[INFO ] 0x1f34 bora::Log: NvEnc: VNCEncodeRegionNvEncUnloadLibrary: Unloading NVIDIA SDK shared library "nvEncodeAPI64.dll"
[INFO ] 0x1f34 bora::Log: NvEnc: VNCEncodeRegionNvEncUnloadLibrary: Unloading NVIDIA SDK shared library "nvml.dll"
[WARN ] 0x1f34 bora::Warning: GetProcAddress: Failed to resolve nvmlDeviceGetEncoderCapacity: 127
[WARN ] 0x1f34 bora::Warning: GetProcAddress: Failed to resolve nvmlDeviceGetProcessUtilization: 127
[WARN ] 0x1f34 bora::Warning: GetProcAddress: Failed to resolve nvmlDeviceGetGridLicensableFeatures: 127

You’ll notice it tries to load the proper functions, however it fails.

The Solution

Unfortunately the only solution is to upgrade to newer or different hardware.

The GRID K1 and GRID K2 cards have reached their EOL (End of Life) and are no longer support. The drivers are not being maintained or updated so I doubt they will take advantage of the newer frame buffer capture functions of Windows 10.

Newer hardware and solutions have incorporated this change and use a different means of frame buffer capture.

To resolve this in my own homelab, I plan to migrate to an AMD FirePro S7150x2.

Oct 062020
 
Video Capture of The Tech Informative

A joint project of mine with Rob Dalton, The Tech Informative is a video podcast on YouTube dedicated to everything tech with a focus on corporate IT.

We intend to keep it informal, fun, and packed full of handy and useful information.

Check it out, and don’t forget to like and subscribe. We also appreciate suggestions and feedback!

This introduction episode is the first of many to come!