Problem description
VMware backups fail with the VMWARE_VMOMI1 error. The guest virtual machine's event viewer displays the VSS error 12289. Manual snapshots get stuck at 100%, time out, and eventually fail.
Cause
The error can occur because of a conflict between a third party snapshot provider and the Microsoft VSS provider. If a third-party snapshot provider, including the VMware VSS provider, takes precedence over the Microsoft VSS provider and interferes with the snapshot process, the quiescing operation can fail. You can also run into this issue when a third-party storage provider or a third-party backup tool provider like Veeam or Networker are installed on the system and interfere with the Microsoft VSS provider.
Traceback
Phoenix logs:
[2020-06-08 14:09:37,885] [ERROR] Error <class 'pyVmomi.VmomiSupport.vim.fault.ApplicationQuiesceFault'>:(vim.fault.ApplicationQuiesceFault)
value = 'msg.snapshot.error-QUIESCINGERROR'
}
],
message = 'An error occurred while saving the snapshot: Failed to quiesce the virtual machine.'
Event Viewer on the guest virtual machine
Resolution
We recommend you disable third-party backup provider services if they aren't needed.
Although VMware tools are mandatory for a quiesced snapshot, the VMware snapshot provider is an optional feature that is installed by default. It can conflict with the Microsoft VSS provider. If you notice VSS Volsnap errors and VMware snapshot provider error 12289, uninstall the VMware snapshot provider from VMtools. After the VMware VSS provider is uninstalled, the system uses Microsoft VSS provider for snapshots.
Uninstall the VMware VSS provider
Log in to the guest virtual machine.
Navigate to Control Panel > Programs > Programs and Features.
Select VMware Tools, and click Change.
In the VMware Tools Program Maintenance screen, select the Modify option, and then click Next.
In the Custom Setup screen, expand aVolume Shadow Copy Services Supportnd select the red X (This feature will not be available).
Ensure that the Volume Shadow Copy Services Support option now has a red X next to it, and then click Next.
Complete the process, and ensure that the VMware VSS service is disabled.
Reboot the system if required.
Uninstalling the VMware VSS provider from registry editor
Instead of running VMware tools to uninstall the VMware VSS provider, you can also uninstall the provider by deleting the associated registry key.
⚠️ Warning
Incorrect use of the Windows registry editor may prevent the operating system from functioning properly. Great care should be taken when making changes to a Windows registry. Registry modifications should only be carried-out by persons experienced in the use of the registry editor application. It is recommended that a complete backup of the registry and workstation be made prior to making any registry changes.
Delete the VSS folder from
<volume>:\Program Files\VMware\VMware Tools\DriversDelete the VMware Snapshot Provider service from
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\vmvssDelete the VMware Snapshot Provider {564d7761-7265-2056-5353-2050726f7669} from
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\ProvidersReboot the guest machine if required.
Re-run the manual snapshot operation.
Tags
vmware_vmomi1_error_and_vss_error_12289, vmware-vmomi1-error-and-vss-error-12289, vmware_backups_fail_with_vmware_vmomi1_error_and_vss_error_12289