Skip to main content

The backup snapshot for this VM is not application consistent. The backup will proceed in crash-consistent mode. (Error code 'RCTCONTROLLER_011')

  • Updated

APPLIES TO

 

Windows Server 2016 Hosts or newer

 

 

PROBLEM

 

Backup completes but gives a warning that it "The backup snapshot for this VM is not application consistent. The backup will proceed in crash-consistent mode. (Error code 'RCTCONTROLLER_011')"

 

mceclip0.png

 

 

SOLUTION

 

The Microsoft Volume Shadow Service (VSS) is Microsoft technology that forms part of Windows Server 2008 R2. This component allows applications to access a "point in time" snapshot of a logical drive on the host machine, including any VHDX and related virtual machine files on that drive. This enables these files to be accessed even if they are in use or locked.  It also ensures that the VHDX and related files are in a consistent state and all data has been flushed to disk before they are accessed for backup purposes.

Going forward, Microsoft have made improvements to this technology and in Server 2016 and newer they have changed the way it works once again. Now they're using "Production Checkpoints" as a gateway between backup applications and the operating system. VSS exists to address the fact that data can and does change while backups are being taken.

In fact if you're getting this warning through VM Backup, you'll see that you will also get this when running a Production Checkpoint. In order to run a production checkpoint as opposed to a standard one, you can follow the steps below:

 

  • Go to Hyper-V Manager
  • Right-click on the VM Settings
  • Go to Checkpoints
  • Un-tick the option "Create standard checkpoints if it's not possible to create a production checkpoint"
  • Apply and OK
  • Right-click on the VM "Checkpoint"

 

The checkpoint operation should now fail, the reason is that it didn't manage to run VSS inside the VM and tell applications to cease all I/O and flush outstanding data and operations from memory to disk so that the backup doesn’t miss anything. VSS in general is broad and can be affected by any application running inside the VM.

 

With that said, this means that it's not always easy or straight-forward to resolve a 'crash-consistent' backup, however you should proceed to troubleshoot as follows:

 

  1. Firstly, check whether your VM actually requires an Application-Consistent backup. If it does not, you can disable it from the "VSS Settings" screen. Simply uncheck "Application Consistent" and Save changes. More information here.

  2. If your VM is running a non-VSS aware guest, such as a Linux OS, you can simply go to "VSS Settings" and disable "Application Consistent" for these VMs.

  3. The Guest must be running one of the supported OS's of Server 2016 as the following link: https://technet.microsoft.com/en-gb/windows-server-docs/compute/hyper-v/supported-windows-guest-operating-systems-for-hyper-v-on-windows

  4. Ensure you have the latest Windows Updates installed as "Integration Services" are now being deployed through the updates.

  5. If the VM giving trouble is running Microsoft SBS 2011, then please go through this article.

  6. In the Properties dialog of the VM, from either Hyper-V Manager or SCVMM, look on the Integration Services tab and ensure that “Backup (volume checkpoint)” is checked.

  7. The guest VM is in a running state.

  8. All the guest VM's disks have ample free space available for the internal shadow copy to complete. You must have 10% free disk space on each disk.

  9. In the guest VM, the service with name "Hyper-V Volume Shadow Copy Requestor" is running.

  10. The Checkpoint File Location for the VM must be set to be the same volume in the host operating system as the VHDX files for the VM.

  11. The guest VM must have a SCSI controller attached in the VM settings (in Hyper-V Manager). There is no need to have any disks on the controller, but it must be present.

  12. The guest VM must not have any Shadow Storage assignment of a volume explicitly set to a different volume other than itself. This can be checked by running "vssadmin list shadowstorage" through command line.

  13. If the guest OS has a system reserved partition, verify that it has at least 45MB of free space. If less than that is free, ensure that a windows shadow copy can be created as per the screenshot below:




  14. All of the virtual machine’s volumes must be formatted with NTFS/ReFS. The volume that contains the .VHD(s) for the VM must also be formatted with NTFS/ReFS. The guest operating system’s disks must be “Basic”, not “Dynamic” (this is not the same as dynamic vs. fixed VHDs, see screenshot below):




  15. Run the command below in command prompt inside the VM that is crash-consistent:

    vssadmin list writers

    In the results check that all writers inside the VM are in a "Stable" state and showing "No error".


  16. Check that the shadowstorage on each drive is not full, ideally set to unbounded. You can set it as unbounded by running the following command in command prompt:

    vssadmin add shadowstorage /For=C: /On=C: /MaxSize=UNBOUNDED
    vssadmin resize shadowstorage /For=C: /On=C: /MaxSize=UNBOUNDED

    Note: Run the two above commands for each drive, each time replacing the drive letter from C: to the drive in question.


  17. The “COM+ Event System”, “Distributed Transaction Coordinator”, “Remote Procedure Call (RPC)”, and “System Event Notification” services must be running within the VM. By default, these are set to “Automatic” and/or “Automatic (Delayed Start)”. The “COM+ System Application” and “Microsoft Software Shadow Copy Provider” and “Volume Shadow Copy” services must at least be set to Manual, which is the default for these. It is acceptable, but not required, to set them to “Automatic” or “Automatic (Delayed Start)”.

  18. Check if you are getting a "vmicvss" with event ID 13 inside the Application event log of the VM that is specifying that Windows cannot perform an online backup of this system. Event below:

    Event ID: 13
    Source: vmicvss    
    Description: Windows cannot perform an online backup of this system because scoped snapshots are enabled. To resolve this, disable scoped snapshots by creating the following registry value on this computer:
         PATH: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SystemRestore\
         DWORD: ScopeSnapshots
         Value: 0

  19. If the issue persists check for warnings or errors in the Application and System event logs on the VM that is giving trouble.

  20. Ensure that the latest Windows Updates for the Host and VM are also applied. Please note that "Optional Updates" are usually also important for VSS operations and are suggested as well.

  21. NTFS compression on the disk hosting the virtual drives should be disabled, otherwise this will cause the checkpoint operation from Hyper-V to fail

  22. If none of the above helps, please contact our support team.

Related articles

Articles in this section
See more