We’ve all been in the situation where we need to install a driver, vib file, or check “esxtop”. Many advanced administration tasks on ESXi need to be performed via shell access, and to do this you either need a console on the physical ESXi host, an SSH session, or use the Remote vCLI.
In this blog post, I’m going to be providing a quick “How to” enable SSH on an ESXi host in your VMware Infrastructure using the vCenter flash-based web administration interface. This will allow you to perform the tasks above, as well as use the “esxcli” command which is frequently needed.
This method should work on all vCenter versions up to 6.7, and ESXi versions up to 6.7.
How to Enable SSH on an ESXi Host Server
Log on to your vCenter server.
On the left hand “Navigator” pane, select the ESXi host.
On the right hand pane, select the “Configure” tab, then “Security Profile” under “System.
Scroll down and look for “Services” further to the right and select “Edit”.
In the “Edit Security Profile” window, select and highlight “SSH” and then click “Start”.
Click “Ok”.
This method can also be used to stop, restart, and change the startup policy to enable or disable SSH starting on boot.
Congratulations, you can now SSH in to your ESXi host!
VMware Horizon is great at providing an end user computing solution for your business, a byproduct of which is an amazing remote access system. With any type of access, especially remote, comes numerous security challenges. DUO Security’s MFA solution is great at provided multi-factor authentication for your environment, and fully supports VMware Horizon View.
In this guide, I’ll be providing a quick how to guide on how to get setup and configured with DUO MFA on your Horizon Server to authenticate View clients.
If you are looking to only implement DUO 2FA on the VMware Unified Access Gateway (and not the connection server), head over to my colleague’s post here: https://securedpackets.com/?p=424
Here’s a video of DUO on VMware Horizon View in action! Scroll down for instructions on how to set it up!
Video Demonstration of DUO MFA 2FA on VMware Horizon View
Enabling DUO MFA on VMWare View will require further authentication from your users via one of the following means:
DUO Push (Push auth request to mobile app)
Phone call (On user’s pre-configured phone number)
SMS Passcode (Texted to users pre-configured phone number)
VMware Horizon View Connection Server (Configured and working)
VMware View Client (for testing)
DUO Authentication Proxy installed, configured, and running (integrated with Active Directory)
Completed DUO Auth Proxy config along with “[ad_client]” as primary authentication.
Please Note: For this guide, we’re going to assume that you already have a Duo Authentication Proxy installed and fully configured on your network. The authentication proxy server acts as a RADIUS server that your VMware Horizon View Connection Server will use to authenticate users against.
Instructions
The instructions will be performed in multiple steps. This includes adding the application to your DUO account, configuring the DUO Authentication Proxy, and finally configuring the VMware View Connection Server.
Add the application to your DUO account
Log on to your DUO account, on the left pane, select “Applications”.
Click on the Blue button “Protect an Application”.
Using the search, look for “VMware View”, and then select “Protect this Application”.
Record the 3 fields labelled “Integration key”, “Security key”, and “API hostname”. You’ll need these later on your authentication proxy.
Feel free to modify the Global Policy to the settings you require. You can always change and modify these later.
Under Settings, we’ll give it a friendly name, choose “Simple” for “Username normalization”, and optionally configure the “Permitted Groups”. Select “Save”.
Configure the DUO Authentication Proxy
Log on to the server that is running your DUO Authentication Proxy.
Open the file explorer and navigate to the following directory.
Before any changes I always make a backup of the existing config file. Copy and paste the “authproxy.cfg” file and rename the copy to “authproxy.cfg.bak”.
Using the values from the “Protect an Application”, replace the “ikey” with your “integration key”, “skey” with your “secret key”, and “api_host” with the API hostname that was provided. Additionally “radius_ip_1” should be set to your View Connection Server IP, and “radius_secret_1” is a secret passphrase shared only by DUO and the View connection server.
Save the file.
Restart the DUO Authentication Proxy either using Services (services.msc), or run the following from a command prompt:
net stop DuoAuthProxy & net start DuoAuthProxy
Configure the VMware View Connection Server
Log on to your server that runs your VMware View Connection Server.
Open the VMware Horizon 7 Administrator web interface and log on.
On the left hand side, under “Inventory”, expand “View Configuration” and select “Servers”.
On the right hand side in the “Servers” pane, click on the “Connection Servers” tab, then select your server, and click “Edit”.
On the “Edit Connection Server Settings” window, click on the “Authentication” tab.
Scroll down to the “Advanced Authentication” section, and change the “2-factor authentication” drop down, to “RADIUS”. Check both check boxes for “Enforce 2-factor and Windows user name matching”, and “Use the same user name and password for RADIUS and Windows Authentication”.
Below the check boxes you will see “Authenticator”. Open the drop down, and select “Create New Authenticator”.
In the “Add RADIUS Authenticator” window, give it a friendly name, friendly description, and populate the fields as specified in the screenshot below. You’ll be using the shared RADIUS/DUO secret we created above in the config file for the proxy auth. Please Note that I changed the default RADIUS port in my config to 1813.
Click “Ok”, then make sure the newly created authenticator is select in the drop down. Proceed to click “Ok” on the remaining windows, and close out of the web interface.
That’s it!
You have now completely implemented DUO MFA on your Horizon deployment. Now when users attempt to log on to your VMware View Connection server, after entering their credentials they will be prompted for a second factor of authentication as pictured below.
In the many years I’ve been providing IT Services, I’ve noticed that whenever taking over a customer from a competitor, or providing consulting services for a company that has IT staff, that I don’t see DHCP reservations being used all that frequently.
I wanted to write a post and create a video to discuss the comparison, when each should be used and the various case scenarios. I’m hoping my readers may provide their own input in the comments.
See below for the video, or read on for the blog post!
As an example: When a customer was purchasing a VoIP PBX, the PBX vendor get angry when I requested that it be configured for DHCP so that a DHCP reservation could be used, I advised I’d prefer this method so I could change the IP when needed for maintenance or network restructuring. They tried to convince me the IP will change on a DHCP Server and the port forwarding will stop working, because they simply had no idea of what a DHCP reservation was. Ultimately when the day came where I had to change the IP and firewall rules for the PBX, I had to log a support call with the vendor since I couldn’t change the IP myself (which resulted in delays, and costs). If we were using DHCP reservations, I could have simply modified the firewall rules, modified the IP address on the reservation, and restarted the device using the buttons on the front panel (I didn’t have any other access to the device).
Just to state the obvious:
A static IP address is an IP address that’s manually set on a NIC (Network Interface Card).
A DHCP Reservation is a pre-set IP that’s provided by a DHCP Server, and given to a NIC when a NIC calls out to a DHCP server for an IP address.
Static IP Addresses
It’s in my opinion that for server, network, core, and all top level infrastructure, all of these devices and services should be configured with Static IP addresses.
These devices which are almost always running, and have other services that rely on them, require a set static IP that should and will not change. Typically, these IP addresses will never change, even when major changes are being made to the core infrastructure.
These addresses should always be logged, documented, and added to network topology maps.
An example of devices commonly seen with Static IPS:
Computers/Workstations using special services (or requiring firewall exceptions)
DHCP Reservations
DHCP stands for Dynamic Host Configuration Protocol, and was created to dynamically configure hosts networking configuration on the fly for easy deployment.
In it’s most simplest explanation, when a computer (or device) that is configured to use DHCP reaches out to the network, the DHCP server will assign and provide an IP address for the computer to use.
In home networks, pretty much every computer and device will get it’s IP address from the DHCP server running on the router.
In business networks, pretty much every computer and device that isn’t hosting services will get it’s IP address from the DHCP server running on one of their servers or routers.
DHCP Servers support something called a “DHCP Reservation”, which essentially allows you to provide a pre-set IP address to a specific client based on it’s physical MAC address. This means that the device will always get the same IP address and it will never change (whereas they typically do on occasion).
I’m surprised I don’t see these used more often, as they can become quite the powerful tool on the IT tool belt when used properly. I’ve listed some pros and cons below.
The Pros:
Manage IP addresses (IP reservations) from a single console
Ability to change IP addresses on the fly easily from a single console without having to log in to the device.
Manage network topology for ROBO (Remote Office, Branch Office) remotely, easily, and efficiently.
Manage IP addresses for 3rd party devices that you don’t normally have access to modify (tell the vendor to set to DHCP), reducing support calls for external services.
Ability to create different PXE boot environments as each reservation can have it’s own PXE boot options assigned.
The Cons:
Device must support DHCP Configuration.
The device MUST RELY on a DHCP Server once set to use DHCP. If the DHCP Server is down, so is the device.
If rogue DHCP servers appear on your network, it may disrupt communication (this can also happen with static IPs and conflicts).
So with the list above, DHCP reservations look pretty powerful. The next question, is where do we use DHCP reservations. Let’s finish off with the devices we’d use them on, and what use case scenarios apply.
Devices:
Wireless Access Points
Printers
2nd Level (non core) Routers and Gateways
IoT Devices
IP Phones
IP PBX Systems (VoIP, Traditional with IP Management, etc).
Thin Clients and Zero Clients
Use Cases:
Remote Offices (remote sites with limited access)
Remote Support environments
Branch Offices
IP Phone Networks
Wireless LAN Access Point VLANs
DHCP Reservation Use Cases
I use DHCP reservations frequently with customers that have remote or branch offices in remote geographical areas. When supporting these users and troubleshooting issues, it’s awesome to be able to just log in to the DHCP server to change IP addresses of printers, phones, and wireless access points.
Also, when configuring, shipping, and deploying new devices to these offices, I can simply log and write down the MAC address, configure the DHCP reservation, and the device will get the IP address I’ve chosen once it’s connected to the network and powered on.
Using DHCP reservations, you can easily make big changes to these remote networks without having to be present. If you were to use Static IPs and something was misconfigured, this might cause a physical visit to the site to resolve.
If by change a vendor directly dropships equipment to the remote site, I can simply call someone at that office to get the MAC address. Most devices with a NIC (printers, MFPs, wireless access points), all usually have their MAC addresses printed on the outside of the box. With this information provided, I can login to the remote server, create a DHCP reservation, configure drivers, and push the device config out to the network.
DHCP reservations add to the whole concept of a centrally managed environment, which further helps ease of maintaining, and supporting it.
You have VMware Horizon View deployed along with Duo Multi-Factor Authentication (2FA, MFA), and you’re you having user experience issues with 10ZiG Zero Clients and multiple login dialog boxes and planning on how to deal with the MFA logins.
I spent some time experimenting with numerous different settings trying to find the cleanest workaround that wouldn’t bother the user or mess up the user experience. I’m going to share with you what I came up with below.
If you’re interesting in 10ZiG products and looking to buy, don’t hesitate to reach out to me for information and/or a quote! We can configure and sell 10ZiG Zero Clients (and thin clients), help with solution design and deployment, and provide consulting services! We sell and ship to Canada and the USA!
The Issue
When you have DUO MFA deployed on VMware Horizon, you may experience login issues when using a 10ZiG Zero Client to access the View Connection Server. This is because the authentication string (username, password, and domain) aren’t passed along correctly from the 10ZiG Login Dialog Box to the VMware Horizon View Client application.
Additionally, when DUO is enabled on VMware View (as a RADIUS authentication), there is no domain passed along inside of the DUO login prompt on the view client.
This issue is due to limitations in the VMware Horizon View Linux Client. This issue will and can occur on any system, thin-client, or Zero Client that uses a command string to initialize a VMware View session where DUO is configured on the View Connection Server.
Kevin Greenway, the CTO at 10ZiG, reached out to say that they have previously brought this up with VMware as a feature request (to support the required functionality), and are hopeful it gets committed.
At this point in time, we’d like to recommend everyone to reach out to VMware and ask for this functionality as a feature request. Numerous simultaneous requests will help gain attention and hopefully escalate it on VMware’s priority list.
The Workaround
After troubleshooting this, and realizing that the 10ZiG VMware login details are completely ignored and not passed along to the VMware View client, I started playing with different settings to test the best way to provide the best user experience for logging in.
At first I attempted to use the Kiosk mode, but had issues with some settings not being passed from the 10ZiG Client to the View Client.
Ultimately I found the perfect tweaking of settings that created a seamless login experience for users.
The Settings
On the 10ZiG Zero Client, we view the “Login” details of the “VMware Horizon Settings” dialog box.
10ZiG Zero Client VMware Horizon Settings Login Settings
Login Mode: Default
Username: PRESS LOGIN
Password: 1234
Domain: YourDomain
Please Note: In the above, because DUO MFA is enabled, the “Username”, “Password” and “Domain” values aren’t actually passed along to the VMware View application on the Zero Client.
We then navigate to the “Advanced” tab, and enable the “Connect once” option. This will force a server disconnection (and require re-authentication) on a desktop pool logoff or disconnection.
10ZiG Zero Client VMware Horizon Settings Advanced Settings
Please Note: This option is required so that when a user logs off, disconnects, or get’s cut off by the server, the Zero Client fully disconnects from the View Connection Server which causes re-authentication (a new password prompt) to occur.
The Login User Experience
So now that we’ve made the modifications to the Zero Client, I want to outline what the user experience will look like from Boot, to connection, to disconnection, to re-authentication.
Turning on the 10ZiG Zero Client, you are presented with the DUO Login Prompt on the View Connection Server.
You then must pass 2FA/MFA authentication.
You are then presented with the desktop pools available to the user.
Upon logging off, disconnecting, or getting kicked off the server, the session is closed and you are presented to the 10ZiG VDI Login Window.
To re-establish a connection, click “Login” as instruction by the “Username” field.
You are presented with the DUO Login Window.
And the process repeats.
As you can see it’s a simple loop that requires almost no training on the end user side. You must only inform the users to click “Login” where the prompt advises to do so.
Once you configure this, you can add it to a configuration template (or generate a configuration template), and then deploy it to a large number of 10ZiG Zero Clients using 10ZiG Manager.
Let me know if this helps, and/or if you find a better way to handle the DUO integration!
In this post, we’ll be going over how to deploy an existing configuration template that is stored inside of your 10ZiG Manager management software.
This allows you to push out configs on the fly to either a single device, or 10,000 devices at once. This is a MUST for managing small, medium, and large sized 10ZiG deployments.
If you’re interesting in 10ZiG products and looking to buy, don’t hesitate to reach out to me for information and/or a quote! We can configure and sell 10ZiG Zero Clients (and thin clients), help with solution design and deployment, and provide consulting services! We sell and ship to Canada and the USA!
This post is part three of a three part 10ZiG Manager Tutorial series:
Choose the 10ZiG Zero Client(s) that you’d like to deploy the configuration to. You can “CTRL + Click” or “SHIFT + Click” to select more than one 10ZiG Zero Client.
In the menu, expand “Configuration” -> and select “Apply Template”.
A “Configuration Template Note” is displayed. Please read and understand this, then click “Ok”.
In the “Configuration Templates” window, select and highlight the configuration template you’d like to deploy, and then click “Ok”. In my example, I’m choosing “DA-MainTemplate”.
The “Configuration Cloning Target” dialog box is displayed. Here you can change the target hostname, and choose to immediately push the configuration. Select “Ok”.
And now the “Reboot” dialog box is displayed. Here you can choose how you’d like the reboot to be handled once the configuration is pushed to the device(s). Select your preference, or leave as default and select “Ok”.
You’ll be brought back to the 10ZiG Manager interface. Here you’ll see a new task in the tasks list at the bottom of the window.
Once completed, you have successfully deployed the configuration.
You’re done! You have successfully pushed the configuration template to your 10ZiG Zero Client(s).
You can maintain, edit, and use multiple templates for different users, organizational units, or geographical units.
Let’s say you manage numerous 10ZiG Zero clients and your users all have similar USB hardware that needs to be redirected to the VDI session. In most cases the hardware will be redirected without any configuration necesary, but what about when that doesn’t happen. You need to push a configuration template with the device information to your 10ZiG Zero Clients.
In my case, I use a YubiKey Security Key. I regularly use this for logins in Chrome and noticed that it wasn’t being directed via USB redirection.
If you’re interesting in 10ZiG products and looking to buy, don’t hesitate to reach out to me for information and/or a quote! We can configure and sell 10ZiG Zero Clients (and thin clients), help with solution design and deployment, and provide consulting services! We sell and ship to Canada and the USA!
This post is part two of a three part 10ZiG Manager Tutorial series:
On the 10ZiG Zero Client, go to settings, USB redirection, and change the preference from “Default” to “Include”. This must manually be done on every Zero Client inside of your infrastructure (time consuming).
Add the USB hardware ID to your configuration template inside of the 10ZiG Manager and then push this to all your 10ZiG Zero Clients that you manage (super fast, can be deployed to thousands of devices in seconds).
In this post we’re going to cover the later, and show you how to add this to a config template. In my example, we’ll be adding the YubiKey security key with a hardware identifier (USB Product ID/PID) of 1050/0120 (Vendor ID: 1050, Product ID: 0120). We’ll be manually adding the hardware ID/PID to the config template in this tutorial.
Please Note: You can also add the settings on a 10ZiG Zero Client, and generate a template by pulling the config from that client. You can then push this to others as well.
To find out the Hardware ID/PID, you can either use the “Device Manager” on Windows, or plug in the device in to a 10ZiG Zero Client, go to settings, USB Redirection, and you should see the device name, along with the HID/PID info.
Instructions
Open the 10ZiG Manager.
Randomly choose a 10ZiG Zero Client from the list, right-click on it to open the menu. Expand “Configuration” -> Select “Manage templates”.
In the “Configuration Templates” window, right-click on your existing template (or create a new one), and select “Edit”.
In the “Template Configuration – Template Name” window, double-click on “USB Device Redirection”.
In the “USB Device Redirection” window, click on “Add”.
Enter in a friendly name, and enter your Vendor ID and Product ID in to the fields. For a YubiKey Security key, I did the following.
Click OK on all the fields, save the template. The configuration has been saved to the configuration template.
You’re done! You can now deploy this template to a single 10ZiG Zero Client, or deploy it as a batch to many 10ZiG Zero Clients.
So you’ve purchase some 10ZiG Zero Clients, configured the 10ZiG Manager, and want to create a configuration template to deploy to all your devices.
In this post, we’ll be going over how to create a configuration template from a manually configured 10ZiG Zero Client, so that you can edit it, and then deploy it to other 10ZiG Zero Clients (whether it’s a single unit, or 10,000).
Once you have a configuration template, you can add certificates, modify the VDI configuration, configure keyboard/mouse input, USB Redirection, and more! Doing all this with a configuration template allows you to manage and maintain a large amount of 10ZiG Zero Clients with ease.
If you’re interesting in 10ZiG products and looking to buy, don’t hesitate to reach out to me for information and/or a quote! We can configure and sell 10ZiG Zero Clients (and thin clients), help with solution design and deployment, and provide consulting services! We sell and ship to Canada and the USA!
This post is part one of a three part 10ZiG Manager Tutorial series:
Please Note: We are going to assume that you have manually configured at least one of your 10ZiG Zero Clients as a base configuration that you want to generate a template from. If not, make sure you do this before generating a template. We are also assuming that you have configured the 10ZiG Management software so that the Zero Clients can connect to it.
Instructions
Open the 10ZiG Manager.
Choose the 10ZiG Zero Client that you have already configured in the list and right-click on the unit.
In the menu, expand “Configuration” -> and Select “Generate Template”.
A warning explaining how the configuration is merged is presented, please read and understand this.
In the “Configuration Templates” window, type in a template name in to the “Template Name” field, and then select “Ok”. I’m calling mine “DA-MainTemplate”.
A warning explaining changes is presented, please read and understand this.
You will be brought back to the 10ZiG Manager, and will see the “Generate configuration template” task in the tasks list at the bottom of the window. It should eventually complete and be marked as successful.
The configuration template has been created.
You have now created a configuration template inside of 10ZiG Manager! You can edit this, and eventually deploy it to other 10ZiG Zero Clients on your network.
After upgrading a computer from Ubuntu 16.04 LTS to Ubuntu 18.04 LTS or Ubuntu 18.04 LTS to Ubuntu 20.04 LTS, during boot the screen goes blank (turns black), all HD disk activity halts, and the system becomes frozen. This event can also occur on a fresh installation or when updates are installed.
This is due to a video mode issue that causes the system to halt or freeze. It’s much like the issue I described here on a Fedora Linux system.
Temporary Fix
To get the system to boot:
After turning on your PC, hold the right SHIFT key to get to the GRUB bootloader if your computer uses a BIOS. If your computer uses EFI or UEFI, continuously tap the “ESC” (escape) key after turning on your PC.
Once GRUB is open, press the “e” key to edit the first highlighted entry “Ubuntu”.
Move your cursor down to the line that starts with “linux”, and use the right arrow key to find the section with the words “ro quiet splash”.
Add “nomodeset” after these words.
nomodeset
Feel free to remove “quiet” and “splash” for more verbosity to troubleshoot the boot process.
Press “CTRL + X” or “F10” to boot.
The system should now boot.
Permanent Fix
To permanently resolve the issue:
Once the system has booted using the temporary fix, log in.
Open a terminal window (Applications -> Terminal, or press the “Start” button and type terminal).
Either “su” in to root, or use “sudo” to open your favorite text editor and edit the file “/etc/default/grub” (I use nano which can be install by running “apt install nano”):
nano /etc/default/grub
Locate the line with the variable “GRUB_CMDLINE_LINUX_DEFAULT”, and add “nomodeset” to the variables. Feel free to remove “splash” and “quiet” if you’d like text boot. Here’s an example of my line after editing (yours will look different):
You’re trying to install Ubuntu on your computer, but it freezes due to lack of resources, specifically memory. This can happen when you’re trying to re-purpose old laptops, netbooks, etc.
This recently happened to met as I tried to install Ubuntu on an old HP Netbook. Originally I used Fedora, but had to switch to Ubuntu due to library issues (I wanted to use the VMware Horizon Client on it).
Unfortunately, when I’d kick off the USB installer, the OS would completely freeze (mouse either unresponsive, or extremely glitchy).
The Fix – External SWAP File
In the ~5 minutes where the system is operable, I used the key sequence “CTRL + ALT + F2” to get to a text tty console session. From here I noticed the system eventually uses all the RAM and maxes out the memory. When this occurs, this is when the system becomes unresponsive.
Since this is a Live CD installer, there is no swap file for the system to use once the RAM has filled up.
To fix this and workaround the problem, I grabbed a second blank USB stick and used it as an external swap file. Using this allowed me to run the installer, complete the installer, and successfully install Ubuntu.
Please make sure you are choosing the right device names in the instructions below. Choosing the wrong device name can cause your to write to the wrong USB stick, or worse the hard drive of your system.
Instructions:
Attached USB Installer, boot system.
Once system has booted, press “CTRL + ALT + F2” to open a tty console session.
Login using user: “Ubuntu” with a blank password.
Type “sudo su” to get a root shell.
Type in “tail -f /var/log/kern.log” and connect your spare blank USB stick that you want to use for SWAP space. Note the device name, in my case it was “/dev/sdd”.
Press “CTRL + C” to stop tailing the log file, then run “fdisk /dev/sdd” and replace “/dev/sdd” with whatever your device was. PLEASE MAKE SURE YOU ARE CHOOSING THE RIGHT USB DEVICE NAME.
Use “n” to create a new partition, follow the prompts, when it asks for size I randomly chose “+2G” for a 2GB swap file. Use “w” to write the partition table and then quit the fdisk application.
Run “mkswap /dev/sdd1” and replace “sdd1” with the device and partition number of your USB Swap stick. This will format the partition and mark it as a SWAP filesystem.
Run “swapon /dev/sdd1” and replace “sdd1” with your swap partition you created. This will activate the external swap file on the USB stick.
Press “CTRL + ALT + F1” to return to the Ubuntu installation guide. Continue the install as normal.
This should also work for other Linux distributions, as I have also used this in the past with Fedora (on a Single Board Computer with almost no RAM).
During the install process where the Ubuntu installer formats your hard drive, the install will actually mount the hard drive swap file as well (it’ll use both). Once the installer is complete, shut down the system and remove the USB SWAP stick.
I can’t tell you how excited I am that after many years, I’ve finally gotten my hands on and purchased an Nvidia Quadro K1 GPU. This card will be used in my homelab to learn, and demo Nvidia GRID accelerated graphics on VMware Horizon View. In this post I’ll outline the details, installation, configuration, and thoughts. And of course I’ll have plenty of pictures below!
The focus will be to use this card both with vGPU, as well as 3D accelerated vSGA inside in an HPE server running ESXi 6.5 and VMware Horizon View 7.8.
Please Note: Some, most, or all of what I’m doing is not officially supported by Nvidia, HPE, and/or VMware. I am simply doing this to learn and demo, and there was a real possibility that it may not have worked since I’m not following the vendor HCL (Hardware Compatibility lists). If you attempt to do this, or something similar, you do so at your own risk.
For some time I’ve been trying to source either an Nvidia GRID K1/K2 or an AMD FirePro S7150 to get started with a simple homelab/demo environment. One of the reasons for the time it took was I didn’t want to spend too much on it, especially with the chances it may not even work.
Essentially, I have 3 Servers:
HPE DL360p Gen8 (Dual Proc, 128GB RAM)
HPE DL360p Gen8 (Dual Proc, 128GB RAM)
HPE ML310e Gen8 v2 (Single Proc, 32GB RAM)
For the DL360p servers, while the servers are beefy enough, have enough power (dual redundant power supplies), and resources, unfortunately the PCIe slots are half-height. In order for me to use a dual-height card, I’d need to rig something up to have an eGPU (external GPU) outside of the server.
As for the ML310e, it’s an entry level tower server. While it does support dual-height (dual slot) PCIe cards, it only has a single 350W power supply, misses some fancy server technologies (I’ve had issues with VT-d, etc), and only a single processor. I should be able to install the card, however I’m worried about powering it (it has no 6pin PCIe power connector), and having ESXi be able to use it.
Finally, I was worried about cooling. The GRID K1 and GRID K2 are typically passively cooled and meant to be installed in to rack servers with fans running at jet engine speeds. If I used the DL360p with an external setup, this would cause issues. If I used the ML310e internally, I had significant doubts that cooling would be enough. The ML310e did have the plastic air baffles, but only had one fan for the expansion cards area, and of course not all the air would pass through the GRID K1 card.
The Purchase
Because of a limited budget, and the possibility I may not even be able to get it working, I didn’t want to spend too much. I found an eBay user local in my city who had a couple Grid K1 and Grid K2 cards, as well as a bunch of other cool stuff.
We spoke and he decided to give me a wicked deal on the Grid K1 card. I thought this was a fantastic idea as the power requirements were significantly less (more likely to work on the ML310e) on the K1 card at 130 W max power, versus the K2 card at 225 W max power.
We set a time and a place to meet. Preemptively I ran out to a local supply store to purchase an LP4 power adapter splitter, as well as a LP4 to 6pin PCIe power adapter. There were no available power connectors inside of the ML310e server so this was needed. I still thought the chances of this working were slim…
I also decided to go ahead and download the Nvidia GRID Software Package. This includes the release notes, user guide, ESXi vib driver (includes vSGA, vGPU), as well as guest drivers for vGPU and pass through. The package also includes the GRID vGPU Manager. The driver I used was from: https://www.nvidia.com/Download/driverResults.aspx/144909/en-us
To install, I copied over the vib file “NVIDIA-vGPU-kepler-VMware_ESXi_6.5_Host_Driver_367.130-1OEM.650.0.0.4598673.vib” to a datastore, enabled SSH, and then ran the following command to install:
The command completed successfully and I shut down the host. Now I waited to meet.
We finally met and the transaction went smooth in a parking lot (people were staring at us as I handed him cash, and he handed me a big brick of something folded inside of grey static wrap). The card looked like it was in beautiful shape, and we had a good but brief chat. I’ll definitely be purchasing some more hardware from him.
Hardware Installation
Installing the card in the ML310e was difficult and took some time with care. First I had to remove the plastic air baffle. Then I had issues getting it inside of the case as the back bracket was 1cm too long to be able to put the card in. I had to finesse and slide in on and angle but finally got it installed. The back bracket (front side of case) on the other side slid in to the blue plastic case bracket. This was nice as the ML310e was designed for extremely long PCIe expansion cards and has a bracket on the front side of the case to help support and hold the card up as well.
For power I disconnected the DVD-ROM (who uses those anyways, right?), and connected the LP5 splitter and the LP5 to 6pin power adapter. I finally hooked it up to the card.
I laid the cables out nicely and then re-installed the air baffle. Everything was snug and tight.
Please see below for pictures of the Nvidia GRID K1 installed in the ML310e Gen8 V2.
ML310e with GRID K1 Side Shot
ML310e with GRID K1 Side Shot (with Flash)
ML310e w/ Air Baffle and cabling
ML310e LP4 Splitter for GRID K1
Nvidia GRID K1 in ML310e w/ Air Baffle
Nvidia GRID K1 Installed and Running
Nvidia GRID K1 in ML310e w/o Air Baffle
Host Configuration
Powering on the server was a tense moment for me. A few things could have happened:
Server won’t power on
Server would power on but hang & report health alert
Nvidia GRID card could overheat
Nvidia GRID card could overheat and become damaged
Nvidia GRID card could overheat and catch fire
Server would boot but not recognize the card
Server would boot, recognize the card, but not work
Server would boot, recognize the card, and work
With great suspense, the server powered on as per normal. No errors or health alerts were presented.
I logged in to iLo on the server, and watched the server perform a BIOS POST, and start it’s boot to ESXi. Everything was looking well and normal.
After ESXi booted, and the server came online in vCenter. I went to the server and confirmed the GRID K1 was detected. I went ahead and configured 2 GPUs for vGPU, and 2 GPUs for 3D vSGA.
ESXi Host Graphics Devices Settings
VM Configuration
I restarted the X.org service (required when changing the options above), and proceeded to add a vGPU to a virtual machine I already had configured and was using for VDI. You do this by adding a “Shared PCI Device”, selecting “NVIDIA GRID vGPU”, and I chose to use the highest profile available on the K1 card called “grid_k180q”.
VM Settings to add NVIDIA GRID vGPU
After adding and selecting ok, you should see a warning telling you that must allocate and reserve all resources for the virtual machine, click “ok” and continue.
Power On and Testing
I went ahead and powered on the VM. I used the vSphere VM console to install the Nvidia GRID driver package (included in the driver ZIP file downloaded earlier) on the guest. I then restarted the guest.
After restarting, I logged in via Horizon, and could instantly tell it was working. Next step was to disable the VMware vSGA Display Adapter in the “Device Manager” and restart the host again.
Upon restarting again, to see if I had full 3D acceleration, I opened DirectX diagnostics by clicking on “Start” -> “Run” -> “dxdiag”.
dxdiag on GRID K1 using k180q profile
It worked! Now it was time to check the temperature of the card to make sure nothing was overheating. I enabled SSH on the ESXi host, logged in, and ran the “nvidia-smi” command.
“nvidia-smi” command on ESXi Host
According to this, the different GPUs ranged from 33C to 50C which was PERFECT! Further testing under stress, and I haven’t gotten a core to go above 56. The ML310e still has an option in the BIOS to increase fan speed, which I may test in the future if the temps get higher.
With “nvidia-smi” you can see the 4 GPUs, power usage, temperatures, memory usage, GPU utilization, and processes. This is the main GPU manager for the card. There are some other flags you can use for relevant information.
“nvidia-smi vgpu” for vGPU Information“nvidia-smi vgpu -q” to Query more vGPU Information
Final Thoughts
Overall I’m very impressed, and it’s working great. While I haven’t tested any games, it’s working perfect for videos, music, YouTube, and multi-monitor support on my 10ZiG 5948qv. I’m using 2 displays with both running at 1920×1080 for resolution.
I’m looking forward to doing some tests with this VM while continuing to use vGPU. I will also be doing some testing utilizing 3D Accelerated vSGA.
The two coolest parts of this project are:
3D Acceleration and Hardware h.264 Encoding on VMware Horizon
Getting a GRID K1 working on an HPE ML310e Gen8 v2
Highly recommend getting a setup like this for your own homelab!
Uses and Projects
Well, I’m writing this “Uses and Projects” section after I wrote the original article (it’s now March 8th, 2020). I have to say I couldn’t be impressed more with this setup, using it as my daily driver.
Since I’ve set this up, I’ve used it remotely while on airplanes, working while travelling, even for video editing.
Some of the projects (and posts) I’ve done, can be found here:
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.
Do you accept the use of cookies and accept our privacy policy? AcceptRejectCookie and Privacy Policy
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
