In this example, I will be using SolarWinds TFTP, but really, it can be any TFTP server you are feeling comfortable working with..
File > Configure, Start The Service, point the root directory to the folder where you put the ESXi bundle.
on the TFTP Server, open a vSphere client to the vCenter Server, click the “Auto Deploy” icon > download TFTP Boot zip and save it to your drive.
extract the zipped file to the TFTP server root folder (C:\ In this example)
Configuring the DHCP Server
on your DC > Manage > Manage this DHCP Server, you will have to configure the scope options that corresponds to your subnet that you will boot the ESX from, the first Option is 066 – here you will have to put the IP of your TFTP Server, the second option type that you will have to configure is 067 (undionly.kpxe.vmw-hardwired), that’s it, click ok.
if you didn’t install PowerCLI by now, then please do.
once installed, please run the following commands: (marked in white)
Connect-VIServer <vCenter FQDN)
let’s add the software repository:
Add-ESXSoftwareDepot <file of the ESXi offline image)
new auto deploy rule:
$channel = Get-ESXSoftwareChannel
$ip = Get-ESXImageProfile –SoftwareChannel $channel
New-DeployRule –Name “ForAllHosts” –Item $ip –AllHosts
(this could take a while..)
add the new rule
the next step is to reboot the Physical Server that we want to deploy ESXi to, the Server will have to boot using it’s network cards, so make sure that in the Server’s BIOS, the following options will be enable:
2. Boot order is set to network as the first boot priority.
- Download vCenter Server 5
- Download the ESXi 5.0 Offline Bundle
- Install vCenter Server 5
- Install Powershell 2.0 (installed it on the vCenter Server)
- Install PowerCLI (installed it on the vCenter Server)
- Install Auto-deploy (also located on vCenter iso) (installed it on the vCenter Server)
- Install TFTP Server (I used Solarwinds) (installed it on the vCenter Server)
- Click “File” and “Configure” and start the TFTP Server
- A folder C:\TFTP-Root will be created
- Make sure if you are running this on Windows that the Firewall is open for TFTP
- Open up your vCenter Client
- Go to the Home screen and click “Auto Deploy”
- Click “Download TFTP Boot Zip”
- I temporarily disabled IE ESC to be able to quickly download the zip file (Control Panel –> Add / Remove Programs –> Turn Windows Features On/Off –> Look for “Configure IE ESC in the Security Information section)
- Unzip the TFTP Boot Zip file in your TFTP-Root folder
- Copy the location of “BIOS DHCP File Name as mentioned in your Auto Deploy screen. This will be “undionly.kpxe.vmw-hardwired”.
- Go to your DHCP Server and add the filename and the ip address of the
- In my case this is what I added to the advanced settings in my Tomato Router/DHCP server:
- In the case of a MS DHCP Server you will need to follow this MS KB article. (#66 and #67 should be sufficient)
- In my case this is what I added to the advanced settings in my Tomato Router/DHCP server:
- Now it should be possible to boot your host, however you should see a screen that tells you that there no ESXi image associated yet. Now we need to do some PowerCLI magic…
- First of all we will need to set the execution policy to “remotesigned” if you haven’t done so already, in my case I haven’t as this is a brand new installed Windows VM:
- Now you can open up PowerCLI and connect to your vCenter Server:
Connect-VIServer <vcenter name>
- Next we will need to add a software depot, this is the Offline Bundle file you downloaded at the beginning:
- Now that you’ve added the depot you will need to identify the ImageProfile you wan to use. Run the following command to list the image profiles:
- In my case it returns the following:
- We will use the “Standard” image profile as we want to be able to install VMware Tools as well through vCenter. In order to use it we will need to define a rule. This basically tells the client which image it should pick up during the PXE Boot. No key thing with this command is the “-Pattern” as that basically defines which machines will get boot the image. In my case everything is running within VMware Workstation and the vendor will always be “VMware Virtual Platform” so I decided to use that is the identifier for this specific rule.
New-DeployRule –Name “FirstTimeBoot” –Item “ESXi-5.0.0-469512-standard” –Pattern “model=VMware Virtual Platform”
- This will take a while as it is unzipping and uploading the required packages. When it has successfully completed we will need to make this deploy rule active:
Add-DeployRule -DeployRule FirstTimeBoot
- That’s it, now the host should be able to boot up using the assigned image profile. See the screenshot below of my PXE booted ESXi host within VMware Workstation
Now if you would go to vCenter and check your Datacenter you will see a host appearing. This host however will still need to be configured and that is the second portion of this exercise…
What we creating here is a stateless configuration. That means we will need to use host profiles in order to configure this host correctly after reach reboot. These are the steps I took to create a host profile for my Lab hosts:
- Configure the host as needed. Don’t forget things like NTP, Storage, Networking, Scratch etc.
- Go to Host Profiles
- Create a New Profile from an existing host
- Select the host you just configured
- Give it a name that you will remember, I used “ESXiLabHostProfile”
- After creating the host profile you will need to attach the host to the profile and check for compliancy. This test should fail as you will need to create an answer file first.
- Right click the host and select “Update Answer File”
- Check all settings (ip-address etc) and when finished click “update”
- Right click the host and select “Check Answer File”, result should be “x”
- Select the host and click “Check Compliance”, result should be “compliant”
- Before we will create a new deploy rule we will want to add the HA depot and create a new image profile which includes the HA agent as we will want our hosts to be part of that. If you don’t add it the “new-deployrule” statement will throw a message that you will need to add it. First we will add the vSphere HA depot. The ip-address should of course be replace with the ip-address of your own vCenter server:
- Now we will need to clone the ESXi Standard depot and add the HA agent to it:
New-EsxImageProfile -CloneProfile ESXi-5.0.0-469512-standard -name “ESXiHA”
Add-EsxSoftwarePackage -ImageProfile “ESXiHA” -SoftwarePackage vmware-fdm
- Now we will create a new deploy rule and associate it with the just created host profile called “ESXiLabHostProfile”. We will also directly enable this new rule. Note that “HA-DRS” is the name of my cluster in this case. First we will remove the old rule to make sure we start with a clean slate:
Remove-DeployRule -DeployRule FirstTimeBoot -delete
New-DeployRule –Name “ProductionBootRule” –Item “ESXiHA”, ESXiLabHostProfile, HA-DRS –Pattern “model=VMware Virtual Platform”
Add-DeployRule -DeployRule “ProductionBootRule”
- Now you can restart the ESXi host and it should end up in the cluster (HA-DRS) with the correct host profile (ESXiLabHostProfile) and completely configured ready for action! (Note that for the next host you will need to create the answer file as well initially for it to work correctly.)
A great new feature of vSphere 5 is the possibility to run ESXi stateless. Long, long time ago when ESX 3.0 was hip, we would all install ESX on the local harddisk (or SAN disk). With ESX 3.5, the first ESXi version was released but only few were using it. With 4.x ESXi really got a large install base and more and more people were moving to installing ESXi on USB or SD card. Now with vSphere 5 and ESXi as the only hypervisor (no more ESX), we don’t need to install ESXi at all.
When running ESXi stateless, the host will PXE boot and load an ESXi image into memory. There is no more need to have local disks, SD card or USB on your host. Another advantage is that you can now switch the ESXi version the host is running by just rebooting or refresh the host configuration with a reboot. Adding new ESXi hosts to your cluster has become a task of just a few minutes. Let’s have a look at how to configure auto deploy. This blog post will take you through the steps of setting up the required infrastructure and prepare an image and deploy it to a host.My series on VMware vSphere 5 Auto Deploy:vSphere 5 – How to run ESXi stateless with vSphere Auto Deploy
vSphere 5 Auto Deploy PXE booting through Cisco ASA firewall
Updating your ESXi host using VMware vSphere 5 Auto deploy
My first Auto Deploy design for real production environment
Important: When running stateless, your ESXi host cannot boot without all vCenter Auto Deploy components being active. Changes are big that your vCenter Auto Deploy is running in a virtual machine. Should your whole environment power down, you will have a problem getting everything running again. Personally I’m thinking of having an extra USB install of ESXi 5 at hand at all times. Just in case. The bigger the environment the smaller the chance this will happen. But especially for some home-lab environments this might be something to think about.
When using Auto Deploy a host will use DHCP to get an IP address and learn if there is a PXE boot server. The PXE boot image is downloaded and the pre-boot environment is ran on the host. It will now contact the Auto Deploy server which will now check the rule engine. Based on the rules in the rule engine an image is now offered to the host and a host profile is applied to the host.
An image is based on a VMware ESXi deployment image combined with optional extra VIBs (VMware Image Bundles). You can download a “ready-to-deploy” image of ESXi from the VMware download site, just like you would download your normal ESXi install files. For example the latest ESXi version I could download in the beta, was build 441354. The normal ESXi install to download would be: VMware-VMvisor-Installer-5.0.0-441354.x86_64.iso, but on the download page there also is a: VMware-ESXi-5.0.0-441354-depot.zip. This is an ESXi image suited for auto deploy, containing the ESXi VIB. It is also possible to combine VIBs into a new image if needed, for example when you want to add extra drivers or special VIB supplied by the hardware manufacturer.
Another component is the Image Builder PowerCLI. Using PowerCLI commands a rule set is created and together with a host profile connected to a host or number of hosts.
To make all this possible you need to have the following in your environment:
- Hosts that are capable of running ESXi 5
- TFPT boot server
- DHCP / DNS server
- Windows server to run Auto Deploy or vSphere vCenter Appliance
- vSphere PowerCLI
The vCenter Auto Deploy manual will show you how to enable the vSphere vCenter Appliance for Auto Deploy. In this post I will use a Windows server to act as TFTP and Auto Deploy server.
Installing TFTP server
To keep things simple I downloaded the WinAgents TFTP server. It is a 30 day trial. Installation is a simple next, next, finish. Create a directory on your Windows server, for example: E:\tftproot and make this your TFTP root directory.
Set up DHCP
To be able to PXE boot, you need to set some options in your DHCP scope. Open the scope on your DHCP server and add the following options:
066 – Boot server host name: <ip of your TFTP / PXE boot server> 067 – Boot file name: undionly.kpxe.vmw-hardwired
Set up vSphere Auto Deploy
On the vCenter installation media you’ll find the Auto Deploy software. It can be installed on your vCenter server but you can also run it on a separate Windows server and have it point to your vCenter server. I kept everything, including the TFTP server, on my vCenter host. Installation again is a simple next, next, finish in which you only need to supply the IP of the vCenter server and the correct credentials. After the installation has finished you can install a plug-in in your vSphere Infrastructure Client. You can see the Auto Deploy icon on the home tab.
Install TFTP boot image
The last step is now to add a boot image to your TFTP server root directory. In the vSphere client click the Auto Deploy plug-in and see the following page:
Now choose to download the TFTP Boot Zip and extract this ZIP file into your TFTP root directory. With this, the first part is ready. To test if things are working, you can boot an ESXi host or just for test boot a VM. Make sure that the host or VM is using PXE boot. You should see that it is assigned an IP address and it starts loading a TFTP image.
Next you’ll see that although a TFTP image was loaded, there was no ESXi image associated with this host. This screen also displays a number of Machine attributes we could use later on when tying a rule set to a host.
Installing vSphere PowerCLI
Well this should be a piece of cake since I expect many of you to have done this many times before. Download the vSphere 5 PowerCLI and install it on the server on which you will be working with your images. To test if your VMware PowerCLI is working, start the VMware vSphere PowerCLI command prompt and run: Get-DeployCommand. This should give you a list of all the commands you’ll need to work with Auto Deploy. After this you’re done and all requirements for vSphere Auto Deploy have been installed.
Preparing your first image
When installing vSphere Auto Deploy, you’re asked for a directory for your SoftwareDepot. This might be a bit confusing since it actually is never used by YOU, only by the Auto Deploy software for internal ‘housekeeping’. My advice is to also create a directory called ‘VIB-downloads’ in which you store the VIBs and images you want to deploy.
First we’ll deploy the basic VMware ESXi 5.0 image to a new host without any further configuration. We’ll attach the image to the host based on mac-address and want the host to show up in your vCenter in a folder named ‘Staging’. Since it has no configuration, I don’t want it to show up in a cluster yet.
All the info we need:
- IP address of the host and the DNS hostname
- Folder name: ‘Staging’
- Mac address: 001a92b8da77
- Image name: VMware-ESXi-5.0.0-441354-depot.zip (as downloaded from VMware site)
- Image name: ESXi-5.0.0-441354-standard (after being added to the depot)
Important: Since we’ll be using DHCP to assign a ‘fixed’ IP and hostname to the host, you need to create a DHCP reservation for the mac-address. In your DHCP scope create the reservation and use the proper hostname. In DNS create an A-record for this hostname and IP address and make sure you give it a PTR / reverse lookup record too.
Adding a new image to your depot
This is what confused me during my first steps with vSphere Auto Deploy. I was doing all sorts of things trying to add the VMware-ESXi-5.0.0-441354-depot.zip to the software depot somehow. I copied the whole zip into it, but that didn’t work, unzipping and copying didn’t work either. Until I learned that you don’t do anything with the software depot directory. Place the downloaded image ‘VIB-downloads’ and then run the command:
You should now see how the image is inserted into the software depot. After this is finished, you can run the command Get-EsxImageProfile, to see what images are present in your depot. In my example you can see I now added two versions of ESXi beta. Each image contains two flavors, a no-tools and a standard version. I usually use the standard version to deploy.
Deploying your first host
We now have an image ready to deploy, when a host reboots he’ll pick up the TFTP image and will ask the vSphere Auto Deploy server for an image. Next step is to create the rule to match a host to an image. When creating rules, you should know there are two rule sets, a ‘working-set’ and an ‘active-set’. The ‘working-set’ is like a depot of rules, the ‘active-set’ are the rules that are really available to hosts. It is a bit confusing sometimes, especially since the commands to attach / create rules are the same. I would have preferred a different naming or just use an extra attribute like “active” and “in-active”. Think that would have made it easier to understand. Also, I was having or actually still have problems editing rules. I just can’t seem to get that work so instead of editing them I delete and re-create them. When I get it to work I’ll update this blog post.
As I was saying, we’re now going to create a rule to connect the image to the host. This is where the ‘New-DeployRule’ command comes in.
New-DeployRule –Name “PreStaging” –Item “ESXi-5.0.0-441354-standard”, “Staging” –Pattern “mac=00:1a:92:b8:da:77”
The new rule that has just been created in called “PreStaging”, it will make sure that the image called “ESXi-5.0.0-441354-standard” ( Get-EsxImageProfile ) will be deployed to a host with the mac address of 00:1a:92:b8:da:77 and will be placed in the “Staging” folder in vCenter.
With the command Get-DeployRule you can see the rule you just created. This is a rule in the ‘working set’. To make the rule part of the ‘active set’ you need to add it like this:
Add-DeployRule -DeployRule “PreStaging”
To check the rules in the ‘active set’ you run the Get-DeployRuleSet command. You’re all set now to boot your host and see how it installs and then shows up in your vCenter.
Using a host profile
After your host has shown in vCenter you’ll see that it is not configured. If you would configure it now and reboot it, all changes will be lost. The way to save your configuration settings is to create a host profile. A great feature which I found particularly useful when trouble shooting host profiles is “Enable / disable Profile Configuration”. This gives you the option to only partly apply a host profile.
You’ll notice that when working with the host profiles and get more advanced configurations, you’re going to run into problems where settings can’t be applied the way you want it and errors without great detail will be thrown at you. For now I’ll show you how to work with a simple host profile.
Configure the new host like normally. Create the vSwitches, attach the datastores, make sure the NTP settings are correct, etc, etc. Since you’re running diskless hosts, it is important to correctly setup syslogging and the core dump location.
For setting up syslogging, I would like to refer you to this post from Jason Boch: “Configure a vCenter 5.0 integrated Syslog server” and use his post to configure syslogging for your ESXi host.
For redirecting the core dump, it is advised to install the vCenter Coredump tool. Just like the syslog tool, the Coredump utility can also be found in the vCenter tools directory. Again install it with a simple next, next, finish. Your server can now also receive coredumps in case an ESXi host would encounter a purple screen of death (PSOD). After configuring the vCenter Coredump utility, configure your ESXi host to use the coredump server. To do this, go into the configuration screen of your host, go to the security profile and enable SSH. Now logon to the ESXi console using your favorite SSH client and run the following commands:
esxcli system coredump network set --interface-name vmk0 --server-ipv4 192.168.0.40 --server-port 6500 esxcli system coredump network set --enable true esxcli system coredump network get
The last line will show you if the new setting has been enabled. Now logout of your ESXi host and switch back to your vSphere Client.
Important: Don’t forget to disable SSH again, because we’ll be generating a profile and wouldn’t want SSH to be enabled on all new hosts running from this profile.
Now go to the “Host and Clusters” view in your vSphere client and select the host you have just prepared. Right click the host and select “Create Profile from host”. Give the profile a name, for example ‘Profile-Cluster01”. Now attach the profile to this host using the Host Profiles section in the vSphere client and have the profile checked to be compliant.
Auto deploy the host with the host profile
The first rule we created made sure that the host with certain mac address would be connected to the standard image and would be put in the “Staging” folder.
New-DeployRule –Name “PreStaging” –Item “ESXi-5.0.0-441354-standard”, “Staging” –Pattern “mac=00:1a:92:b8:da:77”
Now we are going to create a new rule. The new role will move the host into the production cluster called “CL01” and the host profile “Profile-Cluster01” will be used. Because we’re pretty sure this works without testing (yeah right), I’m not going to do this based on mac address, but on IP address range. You should use the IP range that you use in the DHCP scope for the ESXi hosts and create a reservation in the DHCP scope for each host and also create a DNS record. The new rule would be created like this:
New-DeployRule –Name “Prod-CL01” –Item “ESXi-5.0.0-441354-standard”, “CL01”, “Profile-Cluster01” –Pattern “ipv4=192.168.0.100-192.168.0.110”
In the above rule, ‘Prod-CL01’ is the name of the rule, ‘CL01’ is the name of the cluster, ‘Profile-Cluster01’ is the name of the host profile and ipv4 is the DHCP range.
In the ‘working-set’ we now have two rules (“PreStaging” and “Prod-CL01”) and in the ‘active-set’ the “PreStaging” rule is active. Using the remove command, we can remove the “PreStaging” rule from the ‘active-set’ and next we add the “Prod-CL01” to the ‘active-set’ and double check what we have done:
Remove-DeployRule -DeployRule “PreStaging” Add-DeployRule -DeployRule “Prod-CL01” Get-DeployRuleSet
And we’re done !!! You’ll see that if you now reboot your hosts, they will come back and will be added to the CL01 cluster fully participating as a normal host.