how to setup a nested esx lab at home

Introduction

This is meant as a walk through for setting up a small lab at home with access to the internet via your home’s residential  ISP provider. The network has only one lan and either a DSL or Cable modem link

A few hints:

• The browser of choice by VMware is Google Chrome. Be sure to allow popups.
• The password on the vSphere Appliance is “vmware”; you can change it by logging into the vcenter via ssh and type passwd
• Verify your CPU has support for Virtualization and SLAT. For intel it is at http://ark.intel.com
• I would also suggest reading the references below before continuing with your build out.
• While cloning the ESX vm is faster, it might be faster to install from ISO unless you have read the documents properly and configured your original server to avoid duplicate MAC, IP, SID etc
• If you can afford it, I SERIOUSLY recommend using 16GB of ram. An SSD disk is optional
• You dont need to enter the license to run the 60 day trial. you get the 60 day trial by default

Step 1 : The bare metal server

Use a machine, desktop or otherwise, with Virtualisation features ENABLED. If it is disabled, enable it via BIOS

Step 2: Install ESXi 5.5 on the bare metal server

I named this server esxhost

Step 3: Install the vSphere client on a remote box

Install the vSphere client elsewhere as it will be used sparingly. You can use your laptop or desktop as you prefer.

Step 4: Download the vCenter 5.5 appliance in ovf format and its components

Download the vCenter 5.5 appliance in ovf format and its components from http://www.vmware.com

Step 5: Deploy vCenter from OVF template

File > Deploy OVF Template and point to your downloaded OVF file. You can perform the next step in parallel to save time.

Step 6: Deploy New Virtual Machine for Nested ESXi server

Click on your physical ESX host, right click and select “New Virtual Machine” and configure your VM guest to have the following minimum specs.

•  4gb Memory
•  5gb Disk
• Add 3 NICs for a total of 4 vNICs in the VM

·          Edit settings before installing and make sure to enable the 4th option, use Intel VT-x virtualisation is selected

Step 7: Deploy Storage Server

I used FreeNAS but you can use anything you want. The important features are iSCSI and NFS

 The document source that I used is from this link. A couple of items of note:

• add a hard disk of minimum size 20gb to the vm - this will be the disk called da0 later
• set your hostname

Step 8: Install VMware Tools for Nested ESXi

Log into your nested ESXi server from the command line and run the following command:

esxcli software vib install -v http://download3.vmware.com/software/vmw-tools/esxi_tools_for_guests/esx-tools-for-esxi-9.7.0-0.0.00000.i386.vib -f

You may want to check for the latest version over here: https://labs.vmware.com/flings/vmware-tools-for-nested-esxi

Step 9: Modify your guest esx hosts file

If you do not have a DNS on your local network, you will need to modify the /etc/hosts file on your esx01 box. The alternative is to only use IPs for everything. So here is my sample hosts file additions:

192.168.0.249   esxhost.marafa.vm esxhost
192.168.0.248   esx01.marafa.vm esx01
192.168.0.247   esx02.marafa.vm esx02
192.168.0.246   vc.marafa.vm vc
192.168.0.245   freenas.marafa.vm freenas

Step 10: Configure the vCenter Server

Point your browser to the admin view of your vcenter. Get the ip from the console on your vcenter and the port is 5480. Login with root/vmware. Accept the EULA, Configure with default settings and finally click “Start” and hurry up and wait

The next step is to configure NTP. I use 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org

The penultimate step is to configure the network. Go to Network > Address and change “From DHCP” to Static and configure your IPv4 settings. Note you will need to put the new URL/ip into your browser to continue

The final step is to regenerate the SSL certificate. That one is at Admin / “Certificate regeneration enabled”. Reboot for the SSL certs to take effect

Step 11: Login to the new vCenter

Ssh into your VC and configure /etc/hosts for your machines

Now that your vCenter is up and running and configured with a static ip. Browse over to the vsphere web client on (port:9443) and you will find the login screen. Just below the login prompt is a link to the browser plugin to open the console in your browser. Install the plugin first, restart your browser then use the credentials to login.

And finally, continue with the below steps.

Step 12: Add physical host to vCenter

Add the physical bare metal ESX host to the vCenter. You should see your physical host and the vCenter, ESX01 and the FreeNAS guests.

Step 13: Clone the Nested ESXi server

Cloning your ESX server may be faster but it might also be faster to install your second (esx02) server from scratch to avoid duplicating datastore id, uuid, sid, mac address and ip.

However, if you wish to continue with the cloning ...

Now that you have vCenter up and running, use it to clone your Nested ESXi server and configure it but before you do that, read this http://www.virtuallyghetto.com/2013/12/how-to-properly-clone-nested-esxi-vm.html Pay particular attention to the MAC address section of that document

Then power off your ESX guest and make sure the Guest OS Version says “VMware ESXi 5.x” http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1011387

After the cloning is complete ensure the following is reconfigured:

•  Hostname
•  IP

Step 14: Set VM Startup/Shutdown order

Right click on the host and select “Settings” / VM Startup/Shutdown and make sure the VC is always first to come up as it takes quite a bit of time. Make your NAS device start up second and finally your ESXi servers

Step 15: Create a cluster in vCenter

In vCenter, create a cluster and add  your 2 VM guests: ESX01 and ESX02

Step 16: Create the local datastore for each nested esx host in the cluster

It took me a long time to discover where in the new Virtual Center 5.5 to create a datastore so I am recording it here for posterity.

Go to Home/Datastores/Related Objects/Datastores

click the "Create a New Datastore" button and you will get the following picture

Step 17: Add a 2nd NIC to vSwitch0 for redundancy

Add a 2nd NIC to vSwitch0 this will carry the management traffic for the cluster. Go to the virtual ESX host in the virtual cluster and click on Manage/Networking/Virtual Switches then click on the 3rd icon "Manage Physical Network Adapaters for vSwitch0" then click on the + sign

Step 18: Adding shared storage

So now we will utilise the iSCSI that we set up early on. We go back to the nested ESX then Manage/Storage/Storage Adapters and click on the iSCSI Software Adapater. Go to Targets and then click the "Add" button. Enter the FQDN of the iSCSI server. Make sure that the hostname is resolvable. The rescan all. Your end result should look similar to the picture below.

Do this for each host in the Virtual Cluster.

Step 19: Disable HA heartbeat datastore warnings

So now that we have one shared datastore, we are still getting warning messages. This VMware KB gives directions on disabling the error message "The number of heartbeat datastores for host is 1, which is less than required: 2"

Step 20: Turn on DRS

Right click on the Virtual Cluster and select Settings. Then turn on DRS

Step 21: Finally, test a VM on the nested ESX

Now that we have configued the nested ESX HA Cluster, we test it out with a VM. I will be using this one called CirrOS a cloud test VM

References

·          http://www.yellow-bricks.com/2012/06/12/creating-a-nested-lab/

·          http://doc.freenas.org/index.php/ISCSI

·           http://pawellakomski.pl/blog/definite-guide-to-iscsi-with-freenas-on-esxi-part-2/

·           http://www.virtuallyghetto.com/2013/12/how-to-properly-clone-nested-esxi-vm.html