VM NIC Won't Stay Connected

If you are having problems with a NIC staying connected, it could be due to the number of ports configured on the vSwitch. By default there are 24 ports configured, so if the number of virtual machines running exceeds that number they may start to become disconnected. Check /var/log/vmkernel for these messages:

Net: 1317: can't connect device: VM Network: Out of resources

If this is the case, then either move some VMs to a different vSwitch or increase the number of ports on the vSwitch. Increasing the ports will require a reboot.

Instance GUID is improperly specified

Recently I was trying to install vCenter when I received the error "Instance GUID is improperly specified". This was a brand new Windows 2008 Server and a fresh install of vCenter, but the problem was that the install had failed a few times already due to other issues and rolled back. It seems that the rollback process does not completely clean up the files it creates, because one such file called 'instance.cfg' is created in c:\Program Data\VMware\VMware VirtualCenter and in it there is a line with the GUID. So to resolve this, simply delete that file and it will be recreated upon installation.

ESX Host Loses Connectivity Briefly

I have seen this issue with ESX 3.5 Update 2 and Update 4, the host disconnects briefly from the console and there are no real error messages in VirtualCenter other than indicating the disconnection.

Some research revealed that this is due to a memory leak in hostd. To mitigate the issue, I increased the Service Console memory 800 MB and the modified the soft and hard memory limits in the config.xml file.

You can identify this issue from the following errors in /var/log/vmware/hostd.log:

[2009-09-29 01:25:54.857 'Memory checker' 19467184 warning] Current value 204800 exceeds soft limit 122880.
[2009-09-29 01:26:24.868 'Memory checker' 18668464 warning] Current value 204800 exceeds soft limit 122880.
[2009-09-29 01:26:54.879 'Memory checker' 3076465568 error] Current value 204852 exceeds hard limit 204800. Shutting down process.

To change the console memory configuration from the default of 272 MB with ESX 3.5 to the max of 800 MB, edit the memory settings from VirtualCenter under Configuration -> Memory -> Properties. This change requires a server reboot.

To change the hard/soft limits, add the following into the <config> section of /etc/vmware/hostd/config.xml
<hostdWarnMemInMB>300</hostdWarnMemInMB>
<hostdStopMemInMB>380</hostdStopMemInMB>

Do a restart on hostd process by:
service mgmt-vmware restart

Unable to Apply DRS Resource Settings on Host

"Unable to apply DRS resource settings on host (Reason: A general system error occurred: Invalid fault). This can significantly reduce the effectiveness of DRS"

Check each VM running on the host to make sure VMTools is unmounted. I had one machine with the tools still mounted, as soon as I ended the tools install, the error resolved on its own and the host was fully integrated into the cluster.

Error 25003. Setup failed to create the vCenter Server repository

http://www.vmware.com/support/vsphere4/doc/vsp_esxi40_vc40_rel_notes.html

vCenter Server installation on Windows Server 2008 with a remote SQL Server database fails in some circumstances
If you install vCenter Server on Windows 2008, using a remote SQL Server database with Windows authentication for SQL Server, and a domain user for the DSN that is different from the vCenter Server system login, the installation does not proceed and the installer displays the following error message:
25003.Setup failed to create the vCenter repository.
Workaround: In these circumstances, use the same login credentials for vCenter Server and for the SQL Server DSN.

Storage vMotion Enhancements

Improved scalability

Improved performance

Less resource utilization

UI support

Migrate between thick and thin disks

Convert RDMs to VMDKs

No memory doubling requirement

Move a VM's storage from VMFS to NFS

Change storage while VM is running

Quick Hit - New vSphere 4 Storage Features

Paravirtualized iSCSI

What: This is a driver for high performance storage adapters

Benefits: It offers greater throughput and lower CPU utilization

When to use it: For very I/O intensive applications

VMDirect I/O for Storage

What: Enables VMs to directly access the underlying hardware

Benefits: Enhances CPU efficiency in applications with constant and frequent I/O

Drawbacks:
     1) VMotion is not available
     2) HBA is not shareable by other VMs

Virtual Disk Thin Provisioning

What: Space is allocated on an as-needed basis in 1 MB chunks

Benefits:
     1) Improves storage efficiency
     2) Increases utilization rates
     3) Reduces storage costs

Drawbacks: You could over provision the disks and run out of space, so monitoring of resources is essential.

How to Enable Hot Add CPU and Memory in vSphere

In vSphere, there is a new feature, hot add memory and CPU. This is very cool as downtime can be avoided which is, at least in my company, very helpful during daily operations. This feature is not enabled by default. To enable these features, do the following:

1. Make sure the virtual machine hardware is upgraded to version 7, if it is not, then right click the VM and choose 'Upgrade Virtual hardware'. You can find the hardware version by clicking on the 'Summary' tab.



2. Go to edit settings for the VM, then the options tab, under 'Advanced', select 'Memory/CPU hotplug' and in the pane to the right, enable the memory hot add and CPU hot plug features. This change must be made when the VM is powered down. If the VM is running, the options will be greyed out as in the image below.



Hot add memory and CPU is not supported by all OS's, Windows 2008 Enterprise 64 bit edition is one of the suppoted OS's, for a complete list, take a look at the Guest OS Guide.

Windows 2008 Activation - 0x8007232B – DNS name does not exist

When trying to activate a Windows 2008 standard server recently, I received error '0x8007232B - DNS name does not exist'.  The error description 'DNS name does not exist' can be misleading, the error occured because an invalid product key was entered.  If you didn't enter a product key during setup and then try to activate you will also receive this error.  Just correct the product key and activation will work.

Restart VMware Management Service

When all else fails..., run:

"service mgmt-vmware restart"

Recycling this service has cleared up many errors, I keep it handy. Case in point, the following error was thrown and then I lost control of the CD-ROM in one of my VMs. I recycled the VM and I still couldn't access the CD.

Error message:




Before recycling service:



After recycling service:

The Session is not Authenticated

When using the VMware vCenter Converter Standalone tool, you may find that when you get to step 2 - 'Specify Destination', you sometimes get the error 'The session is not authenticated'. There are two possible fixes that I know of. First, try killing the vmware-converter-a.exe process from task manager. If that does not work, you are probably converting a live machine, so go to that machine and recycle the 'VMware vCenter Converter Agent' service.

Mount and Unmount a CD in Linux

Mount:

mount -t auto /dev/cdrom /mnt/cdrom

Unmount:
umount /mnt/cdrom

In case of 'device is busy' when unmounting, try the 'lazy' unmount:
umount -l /mnt/cdrom

VMware vCenter - Insufficient Permissions

I was trying to P2V a Windows XP Tablet PC to the new ESX host that I just built and I kept getting the following error: "Insufficient permissions to connect to .." when vCenter Converter was trying to install the agent. Pretty easy right? I was using the target machines local administrator account to connect and that should have worked. I also tried other admin accounts, nothing worked. I then tried connecting to the admin$ share using the local admin account and still that failed. I finally figured it out, it may be obvious to those that work on desktop systems regularly, but it was not obvious to me. In Windows Explorer, under Options at the very bottom there is a check box to enable simple file sharing. After I unchecked this box, I was able to continue.

How to Change the ESX Host IP Address

Use PUTTY to telnet to server
login as root

To change the IP address, do the following
Delete the current IP setting: esxcfg-vswif –d vswif0
Add the new IP address for interface vswif0:
esxcfg-vswif –a vswif0 -p Service\ Console -i 192.x.x.x -n 255.255.252.0 –b 192.x.x.255
The ‘\’ between service and console is required.

After the change, type in esxcfg-vswif –l to verify that the change was applied

If the default gateway needs to change, edit /etc/sysconfig/network. Use VI or your favorite editor to accomplish this.
Example using VI:
Type vi /etc/sysconfig/network
Press the letter i to put vi into ‘insert’ mode
Use the arrows on the keyboard to navigate to the GATEWAY and make the change.
To save the change, press ESC then colon :, then w then q
ESC :wq
To exist without saving, press ESC, then colon :, then q
ESC :q

Restart networking
Execute /etc/init.d/networking restart

Other tasks
Update the IP address in DNS
Remove and re-add the host to Virtual Center

How to Start all the VMs on a Host from the Command Line

vmware-cmd -l | while read vmxpath; do vmware-cmd $vmxpath start;done

How to remove snapshots for all VMs on a Host

From the ESX console, run the following command

vmware-cmd -l | while read vmxpath; do vmware-cmd $vmxpath removesnapshots;done

This gets a list of the VMS, for each item returned, pass in the path through the variable $vmxpath and run the removesnapshots command.