ESX 2.5.2 and VirtualCenter 1.3

I upgraded our production ESX servers to 2.5.2 without incident. The only issue I had was when I tried to VMotion VMs off of each server to do the patch. We have in total over 32 VMs between the two ESX servers with a single virtual switch on each server. We used to be able to "cheat" and VMotion and run all 32+ VMs on a single server. Apparently VirtualCenter 1.3 is getting a little more cautious and now checks to see if you have reached the 32 VM limit on a virtual switch before it does a VMotion. I had to shutdown a few VMs to move over VMs from the other server while I did the upgrades.

ESX 2.5.2 and VirtualCenter 1.3 are out

The ESX 2.5.2 update is available and the VirtualCenter 1.3 update are available for download now. As HP has recommended installing this as a "possible" solution for our XP SAN problems, I will be trying this shortly. According to HP, after their testing with VMWare, it appears our SAN problem is a ESX problem that won't be fixed until ESX 3.0. They still recommended that I give 2.5.2 a try.

It still bugs me that we didn't have any of our SAN problems with our EMC Symmetrix. They didn't occur until we switched to the HP XP. From the last I heard, it is due to the way ESX handles SCSI reservations for share LUNs.

ESX/VMotion duplex gotcha

I was experiencing some problems when doing a VMotion from one ESX server to another. Occasionally when the VMotion completed, the VM would lose network connection and could not be reestablished without removing and readding the VM's virtual NIC. I thought this was due to a problem I was also experiencing with our new SAN causing some corruption to our virtual disks. When I opened a case with VMWare on the disk corruption issue, I also mentioned the NIC problem. Turns out the NIC problem is a known issue with ESX 2.5.1 and the duplex settings on the physical NICs in the ESX server. We have always forced the speed and duplex settings on our NICs due to issues we have experienced with using auto setting with our Cisco switches. It has never worked reliabily when we we use auto. Unfortunately, the NIC driver for ESX 2.5.1 has some problems and one problem is our loss of network connections within a VM after a VMotion. Setting the switch ports and ESX server NICs to auto for the duplex setting resolves this problem.

Since configuring our ESX NICs to auto-duplex, I have not had any VMs lose network connections after a VMotion.

West Michigan VMWare Users Group

I will be hosting the inaugural meeting for the West Michigan VMWare Users Group 4/19/2005 at 7:30pm EST. The meeting will be held at Herman Miller in Zeeland, MI.

VMWare VirtualCenter performance problems

Since I've upgraded to ESX 2.5 and the latest VirtualCenter release, I have been having some performance issues particularily when deploying VMs from templates. I have had it take up to 8 hours to deploy from a template and frequently the deployment times out before it completes.

One source of the problem I discovered, was some SCSI drive failures I had on our test ESX server. SCSI operations were timing out causing problems with deployments from templates and other VirtualCenter operations. Once the drives were replaced, the problems improved somewhat but I still had periodic problems. Performance monitor on the VirtualCenter server did not show any high CPU utilization, paging, network or disk activity when I had the performance issues. Sometimes the performance would be so bad, I couldn't even use VNC to access the VirtualCenter server remotely. It would also respond sluggishly when I logged in locally.

I went through my change log and I had installed the latest IBM Director agents shortly after I did the ESX and VirtualCenter upgrades. After trying everthing else I could think of, I removed the Director Agents. I did this a week ago and I have not had anymore VirtualCenter performance issues. Deployments now take under 20 minutes and VirtualCenter seems to be behaving properly.

My VirtualCenter server is a HP Proliant DL380 running the HP Insight Agents. My guess is there may have been some conflict between the HP and IBM management agents. The Director Agents my just be buggy too. I've seen some posts about the latest Director Agents just spontaneously stopping on ESX servers and having to be restarted. I primarily use the Director agents for monitoring the hardware but the latest version included integration with VMWare products. I don't really need that integration so I am leaving them off the VirtualCenter server for now.

First Vonage Aggravation

Our phone number hasn't been switched over to Vonage yet, but so far I've been satisfied with the performance and quality of the outgoing calls. I've been less than impressed with their customer support though. I decided to clean up the wiring in my computer room on Sunday. When I plugged the Vonage Linksys adapter back in, the power light was red. I still had a dial tone and unplugging and plugging back in the power adapter didn't change anything. Monday morning, there were no lights on the Linksys adapter. It was dead. I returned the adapter to Best Buy and they exchanged it for a new adapter. I hooked it back up and then tried to contact Vonage to register the new MAC address. The web site didn't have any information on what to do so I just called the main customer support number.

The first problem was trying to wade through the menu options to get to the correct support area. There wasn't any clear option for registering a new MAC address for an existing account, so I just chose the option for setting up a new adapter. After a 15 minute wait, I finally was connected with someone who promptly told me that I would have to be forwarded to someone else. Somehow I was disconnected when they attempted to forward my call.

I called back and waited another 10 minutes. This time they did manage to forward my call without dropping me. I sat on hold for about 5 minutes. The technician that answered knew exactly what to do and I was back in service in a few minutes.

It would be nice if Vonage had clearer instructions on their voice menus on how to handle this situation. There is an insert in the Linksys box that explains if you are upgrading or replacing your adapter to call the main support number. It doesn't tell what options to choose though when you do call the number though.

I'm still satisfied overall with Vonage. I can't say their customer support is really any worse than most other support organizations I've had to deal with. Hopefully I won't have to call too often. I am getting a little concerned about Comcast's broadband Internet connections though. It seems every weekend since I've had it, there has been some service interruption ranging anywhere from 30 minutes to 3 days. I hope that's not a trend!

Almost rid of SBC!

I went out and bought a Linksys VoIP adapter for Vonage Friday night. The installation was painless and I had a telephone hooked up and working within a short period of time. I sent in the forms to transfer my phone number from SBC over to Vonage. I've also put in a request to have SBC DSL service disconnected. Hopefully within a few days, I will be SBC free!

I fiddled around with the Vonage service a little. The quality seemed to be good and I didn't have any connection problems. I like all of the features that are included with their standard packages like voicemail, call waiting and caller ID. I was paying extra for all of this with SBC. To begin with, I'm starting with the 500 minute basic plan. Looking at our old phone bills, we were barely using a 100 minutes per month so 500 minutes should be more than sufficient.

I HATE SBC!!

I have been having an ongoing argument with SBC, my telephone and DSL provider, over some charges I have no idea where they came from. After several months of contending these charges, SBC just deactivated my email accounts and DSL access. This comes on top of SBC recently raising their DSL rates almost $20. Since I was now paying close to the same amount Comcast is charging for cable broadband Internet, I decided to switch to Comcast. I had the connection setup in the house on Saturday. As soon as the cable guy left, I tried running Comcast's registration program. It took me several attempts but I was eventually able to create a userid but then the program failed again. I contacted Comcast and was told they were doing a major upgrade on their registration servers and to try back in 24 hours. I tried again Sunday but to no avail. I contacted Comcast again and was told they were STILL doing server maintenance.

When I got into work Monday, I logged into Comcast's site and noticed their were several service notices stating that their ENTIRE Internet service was down. Not just for my location, not just registration but EVERYTHING! I kept trying each day and checking the service notices. There was finally a notice posted Wednesday that their maintenance had been completed. I still could not complete my registration Wednesday night. I called Comcast Thursday morning and they were finally able to complete the registration and my Internet connection was working.

I originally was intending to get rid of SBC telephone service to and switch to VoIP, but now I'm questioning the reliability of Comcast's Internet service. I might do it anyways. I can always fall back on cell phones. I really, really, REALLY want to be rid of SBC!!!!