Problem description

App-aware T-Log backups fail with exit code VMware21. T-Logs are copied to the system drive ProgramData%\Phoenix\VMWare\SQL\TlLogs during backups. If the system drive has less than 1 GB free space, the backups fail.

Cause

Free space of less than 1 GB on the system drive can cause the backups to fail.

Traceback

Follow the instructions in the Logs article to obtain the job logs.
Extract the logs and access the Phoenix<YYYYMMDD>_<timestamp>.log to determine if it has the following traceback.

[2020-03-18 10:41:18,319] [ERROR] Error <class 'inSyncLib.inSyncError.SyncError'>:Failed to get TL backup status. error code: 4026 error msg: App-Aware processing failed since there is not enough space on the system volume <1 GB. Please ensure there is enough free space for app-aware processing to succeed (#100010015) (Error Code : VMWARE21). Traceback -Traceback (most recent call last):

File "agents/vmware/appaware_sql_tl_backup_restore.py", line 399, in upload_trans_log_file

SyncError: Failed to get TL backup status. error code: 4026 error msg: App-Aware processing failed since there is not enough space on the system volume <1 GB. Please ensure there is enough free space for app-aware processing to succeed (#100010015) (Error Code : VMWARE21)
Access the Phoenix-SQL-<YYYYMMDD>_<timestamp>_sqlguestplugin.log to check the complete pattern of disk usage. The log shows the amount of free disk space at the start of a backup, how it gradually decreases as the backup progresses and as the free space on the system drive reduces to less than 1 GB, the backup fails.

[2020-03-18 10:22:52,903] [INFO] Creating and opening virtual device for db[<DBname>] backup

[2020-03-18 10:22:52,997] [DEBUG] free system drive size: 40419921920

[2020-03-18 10:41:27,279] [DEBUG] free system drive size: 1074597888

[2020-03-18 10:41:27,303] [DEBUG] free system drive size: 1073549312

[2020-03-18 10:41:27,305] [ERROR] App-Aware processing failed since there is not enough space on the system volume <1 GB. Please ensure there is enough free space for app-aware processing to succeed

[2020-03-18 10:41:27,305] [ERROR] Low System Drive space

Note: The free system drive size above is in bytes

Resolution

Determine the Total Size of .LDF Files:
1. Use SQL Server Management Studio (SSMS) or a similar tool to connect to your SQL Server instance.
2. Execute the following query to get the total size of .LDF files for each user database:
```
SELECT name AS DatabaseName,     size * 8 / 1024 AS SizeMB FROM    sys.master_files WHERE     type_desc = 'LOG';
```
3. Sample output for the query might look like this;
4. Sum the SizeMB values to determine the total size of all .LDF files.
```
SELECT SUM(size * 8 / 1024) AS TotalSizeMB FROM sys.master_files WHERE type_desc = 'LOG';
```
5. Sample output for the total size might look like this:
Free Up Space on the System Drive:
- Identify unnecessary files or move non-essential data to another drive to free up space.
- Verify the freed space by checking the available space on the system drive using Windows Explorer or the dir command in Command Prompt.
Expand the System Drive:
- Use Disk Management or a third-party tool to expand the system drive.
- Verify the expansion by checking the new size of the system drive in Disk Management.
Shrink Log Files:
1. Ask your database administrator to shrink the log files using the following steps in SSMS:
  - Right-click on the database, go to Tasks > Shrink > Files.
  - Select the File type as 'Log' and choose the appropriate file.
  - Click OK to shrink the file.
2. Alternatively, use the following SQL command to shrink log files:
```
DBCC SHRINKFILE (LogFileName, TargetSizeInMB);
```
  NOTE: Replace LogFileName with the name of your log file and TargetSizeInMB with the desired size. Also, please consult Database Administrator before performing shrinking operation.
  
  For example, to shrink a log file named Database1_log to 100 MB, you would use:
```
DBCC SHRINKFILE (Database1_log, 100);
```
3. Verify the reduction in log file size by re-running the query from step 1 and comparing the new sizes.
  - Sample output after shrinking might look like this:
  - Summing the new SizeMB values:
    
    SELECT SUM(size * 8 / 1024) AS TotalSizeMB FROM sys.master_files WHERE type_desc = 'LOG';
  - Sample output for the new total size might look like this:

Phoenix backup fails with PHOENIX54 error

File server backup fails with Backup Window Expired

Backup job stuck at 0 percent due to Delayed write failed or Disk is full

App aware T-Log backup job fails with exit code VMware24

App aware T-Log backup fails with the VMware27 error