A tip when using the vCenter Server Converge Tool

As you might know, VMware is dropping support for the external Platform Services Controller (PSC) deployment model with the next major release of vSphere.

To make a smooth transition, you have to use a command-line interface of the vCenter Server Converge Tool to do the migration to an embedded PSC. This functionality is available starting from vCenter Server 6.7 Update 1 and also in vCenter Server 6.5 Update 2d and onward.

With the upcoming release of vSphere 6.7 Update 2, there will be an option to complete the whole migration using the vSphere Client – super easy!

Meanwhile, the process of moving from an external PSC deployment to the embedded one using CLI consists of two manual steps – converge and decommission. A detailed instruction of how to prepare for and execute each of those steps is documented in the David Stamen’s post ‘Understanding the vCenter Server Converge Tool‘.

What I found tricky was running the converge step when the external PSC had been previously joined to the child domain in Active Directory. In this case, the vCenter Server Converge Tool precheck ran with the default parameters generates the following error message in vcsa-converge-cli.log:

2019-04-06 03:08:15,979 – vCSACliConvergeLogger – ERROR – AD Identity store present on the PSC:root.domain.com
2019-04-06 03:08:15,979 – vCSACliConvergeLogger – INFO – ================ [FAILED] Task: PrecheckSameDomainTask: Running PrecheckSameDomainTask execution failed at 03:08:15 ================
2019-04-06 03:08:15,980 – vCSACliConvergeLogger – DEBUG – Task ‘PrecheckSameDomainTask: Running PrecheckSameDomainTask’ execution failed because [ERROR: Template AD info not providded.], possible resolution is [Refer to the log for details]
2019-04-06 03:08:15,980 – vCSACliConvergeLogger – INFO – =============================================================
2019-04-06 03:08:16,104 – vCSACliConvergeLogger – ERROR – Error occurred. See logs for details.
2019-04-06 03:08:16,105 – vCSACliConvergeLogger – DEBUG – Error message: com.vmware.vcsa.installer.converge.prechecksamedomain: ERROR: Template AD info not providded.

In this example, the root.domain.com refers to the root domain; whereas, the computer object for PSC is in the child domain.

To workaround this issue, I had to use the –skip-domain-handling flag to skip the AD Domain related handling in both precheck and actual converge.

By doing this, the vCenter Server Appliance should be joined to the correct AD domain manually after the converge succeed and before the external PSC will be decommissioned.

‘Response with status: 401 Unauthorized for URL’ while performing Update Manager tasks on Firefox in vSphere 6.5

Updating to vCenter Server 6.5 Update 2d has brought long-awaited support for the vSphere Update Manager (VUM) functionality to vSphere Client.

However, you should be aware of some incompatibility between a VUM plug-in in vSphere Client and Mozilla Firefox.

Update Manager gives a ‘Response with status: 401 Unauthorized for URL‘ error message at random occasions when used with the recent versions of this web-browser (tested with Firefox 63.x and 64.x):

Searching on the VMware’s web-site led me to the KB 59696 that describes the similar issue in vSphere 6.7 and the workaround provided worked for me.

First you need to clear the browser cache:

Then stop Firefox from caching pages by modifying the browser’s default configuration:

According to VMware, they are currently working towards resolving this issue in future releases.

IMPORTANT: Removing a virtual machine folder from inventory deletes the VMs within it from disk when using the vSphere Client

VMware has recently released a new KB 65207 warning about this issue.

Normally, all powered off VMs would be unregistered from vCenter when you remove a parent folder that includes those objects using the vSphere Web Client. The same is expected in vSphere Client.

vCenter logs similar events when the folder object was removed in vSphere Web Client or vSphere Client.

In my case, vCenter generated two events – one for deleting the folder, and another one for removing TEST-VM-02 from the inventory.

The only difference is that vCenter deleted TEST-VM-02 from the corresponding datastore when removing TEST-Folder from the inventory. Moreover, no events are generated during the delete operation!!!

This issue affects both VMware vCenter Server 6.5.x and VMware vCenter Server 6.7.x.

Workaround (provided by the vendor): Use the vSphere Web Client (Flex) for unregistering all virtual machines in a VM folder or move the VMs from that folder before removing it.

22/01/2019 – Update 1: This issue has been resolved in VMware vCenter Server 6.7 Update 1b. Please see the release notes for more details.

PowerCLI: Housekeeping after upgrading to the latest version of PowerCLI

With a new version of VMware PowerCLI 11.1.0 released, it is time to plan for an upgrade.

Even if it is simply a matter of one command in PowerShell, you need to remember that the Update-Module cmdlet does not remove the old modules from the operating system (OS). So over the time, it might look a bit messy:

Following a few threads in regards to this issue (here and here), I wrote a simple script that removes the old versions of VMware.PowerCLI and its dependencies from Windows OS.

A few minutes later, it looks much cleaner with only the latest version of VMware.PowerCLI remaining:

Hope it will be useful for you as well.

vCSA 6.x: WinSCP fails with the error ‘Received too large SFTP packet’

Back to basics… When you try connecting to vCenter Server Virtual Appliance 6.x (vCSA) using WinSCP, the error message ‘Received too large (1433299822 B) SFTP packet‘ might appear.

vCSA6x-WinSCP-01

This is due to the configuration of vCSA when the default shell used for the root account set to the Appliance Shell.

To fix this issue, VMware recommends switching the vCSA 6.x to the Bash Shell. This can be done in the SSH session with the following command:

chsh -s /bin/bash root

Note: You need to log out from the Appliance Shell and log in back again for the changes to take effect.

[URGENT] vSAN 6.6.1: Potential data loss due to resynchronisation mixed with object expansion

Last week VMware released an urgent hotfix to remediate potential data loss in vSAN 6.6.1 due to resynchronisation mixed with object expansion.

This is a known issue affecting earlier versions of ESXi 6.5 Express Patch 9. The vendor states that a sequence of the following operations might cause it:

  1. vSAN initiates resynchronisation to maintain data availability.
  2. You expand a virtual machine disk (VMDK).
  3. vSAN initiates another resync after the VMDK expansion.

Detailed information about this problem is available in KB 60299.

If you are a vSAN customer, additional considerations are required before applying this hotfix:

  • If hosts have already been upgraded to ESXi650-201810001, you can proceed with this upgrade,
  • If hosts have not been upgraded to ESXi650-201810001, and if an expansion of a VMDK is likely, the in-place expansion should be disabled on all of them by setting the VSAN.ClomEnableInplaceExpansion advanced configuration option to ‘0‘.

The VSAN.ClomEnableInplaceExpansion advanced configuration option is not available in vSphere Client. I use the following one-liner scrips to determine and change its value via PowerCLI:

# To check the current status
Get-VMHost | Get-AdvancedSetting -Name “VSAN.ClomEnableInplaceExpansion” | select Entity, Name, Value | Format-Table -AutoSize

# To disable the in-place expansion
Get-VMHost | Get-AdvancedSetting -Name “VSAN.ClomEnableInplaceExpansion” | ? {$_.Value -eq “1”} | Set-AdvancedSetting -Value “0”

Note: No reboot is required after the change.

After hosts were upgraded to ESXi650-201810001 or ESXi650-201811002, you can set VSAN.ClomEnableInplaceExpansion back to ‘1‘ to enable the in-place expansion.