Using Shadow Copy with Robocopy

In my previous post I presented a batch file that uses robocopy to make mirror backups of local files to a NAS.

In this post, I will build upon that to add Volume Shadow Copy functionality in order to copy locked files.

Since this is a radical overhaul of my previous batch script, and also imposes upon the user a need to run with elevated privileges, and also to install the Microsoft Windows SDK, I thought it best to create a new post so that the original version is still available if you want to use that.

Overview

Whilst the batch file I wrote in my previous post is sufficient for many Use Cases, it can’t cope with files that are in use. For that we need to use Volume Shadow Copy, which is not something that robocopy supports directly. However, we can temporarily create a shadow copy of the source drive and then use robocopy to copy from that.

Caveats

  • Unlike the previous version, this version will need to be run with elevated privileges.
  • It will also require that the Windows Software Development Kit is installed, since it requires the Windows vshadow tool.
  • It requires that all source drives are formatted to NTFS. It cannot work with FAT or FAT32 volumes.
  • It requires that drive letter B: is available for temporary mapping of the shadow drive

The version of the Windows Software Development Kit that you require will be dependent on your version of Windows.

Requirements

My requirements for this extension are:

  • Use shadow copy
  • Creation of shadow copies should be as efficient as possible, which means only deleting a shadow drive and creating a new one when the drive letter changes.
  • (optional) Provide a way of specifying which file names and folder names to exclude, like the xcopy parameter /EXCLUDE:file does for file extensions.
  • (optional) It would be desirable to gracefully fail if vshadow is not available, or if the source volume is not formatted to NTFS, or if the batch file is not executing with elevated privileges. It would be even more desirable to fall back to previous behaviour if any of these requirements are not met.

Solution

Volume Shadow Copy

I looked at ShadowSpawn as a possible solution, but its Use Case is to be called once per source, which does not fulfil my requirements.

The other option is to use vshadow to create Volume Shadow Copies as required. This is the option I decided to take for now.

File list

The file list only needs to change if the optional requirements of the ability to specify file names and folder names is implemented. Currently it is not.

 

The batch file

robobackup.bat

@echo off
setlocal DisableDelayedExpansion

REM Settings. TODO - read from a settings file
set "destinationPath=\\GEORGE\backup\%COMPUTERNAME%\"
set "logfile=%destinationPath%\backup.log"
set vssvarFile=robobackup-vssvar.cmd
set vssDriveLetter=B:

REM Make sure the destination path exists. Then clear the logfile from the destination path, if it exists from a previous run. 
if not exist %destinationPath% mkdir %destinationPath% > nul
if exist %logfile% copy nul %logfile% > nul

REM This is the volume that is currently mapped to vssDriveLetter, and it starts off as unassigned.
set shadowDrive=

REM Check if there is a shadow drive hanging around from a previous run. TODO - See if we can recover from this.
if exist %vssDriveLetter% ( 
	echo ERROR - Shadow drive "%vssDriveLetter%" is still mounted. Cannot continue. Please use vsshadow manually to resolve.
	exit /b 1
)

REM Now iterate over all the lines in robobackup_dirs.txt
REM %%~i is the complete source path, but stripped of quotes
REM %%~di is the drive letter of the source, stripped of quotes, eg. C:
REM %%~pni is the rest of the source path, stripped of the drive. The ~p is the path and ~n is the filename. 
REM Since paths that don't end in a slash are treated as a filename, ~pn captures both.

for /F "delims=" %%i in (robobackup_dirs.txt) do (
	if not "%%~di"=="\\" call :processLine "%%~i" "%%~di" "%%~pni"
)

REM Finally, clear up the shadow drive.
call :removeshadow %shadowDrive%

echo.
echo Done!
exit /b
goto :EOF

REM Processes each path, creating a shadow drive if necessary, and then performing the backup. This is the meat of the script.
:processLine
	setlocal EnableDelayedExpansion
	
	set currentSource=%~1
	set currentDrive=%~2
	set currentPath=%~3

	REM Strip the colon off the current drive to give just the drive letter, and make it upper case. 
	REM We need this to construct the destination path. 
	set driveLetter=!currentDrive:~0,1!
	call :upper driveLetter
	
	REM Strip the trailing slash off the path if there is one. Not strictly necessary, but I like consistency.
	if "!currentPath:~-1!"=="\" set currentPath=!currentPath:~0,-1!

	REM Create the shadow drive, if needed, or else use the current.
	call :createshadow !currentDrive! !shadowDrive!
	
	REM Adjust the source path to be the shadow drive copy and construct the destination path.
	set source=%vssDriveLetter%!currentPath!
	set dest=!destinationPath!!driveLetter!!currentPath!

	REM Finally, call robocopy to perform the backup.
	call robocopy "!source!" "!dest!" /MIR /W:10 /R:3 /XA:SH /MT:32 /XJD /XJF /FFT /DST /Z /dcopy:T /a+:a /NP /TEE /log+:%logfile% /xf *.pch *.obj *.ilk *.plg *.idb *.sbr *.exp *.pdb *.ncb *.aps /xd ".hg" ".git" ".svn"
	
	endlocal & set shadowDrive=%shadowDrive%
goto :EOF

REM Creates a shadow drive
:createshadow
	setlocal EnableDelayedExpansion
	
	set currentDrive=%~1
	set shadowDrive=%~2
	
	REM Do we need to (re)create the shadow drive or is the current one still good?
	if /I "%currentDrive%" NEQ "%shadowDrive%" (
	
		REM If there is already a shadow drive then we need to delete it now. 
		REM However, on the first pass this variable will be undefined so we want to skip the deletion in that scenario.
		if defined shadowDrive call :removeshadow %shadowDrive%
	
		echo Creating shadow copy of "%currentDrive%" - please wait...
	
		call vshadow.exe -script=%vssvarFile% -p %currentDrive% > nul
		call %vssvarFile% > nul
		call vshadow.exe -el=!SHADOW_ID_1!,%vssDriveLetter%

		if %ERRORLEVEL% NEQ 0 ( 
			echo Oh noes! Something went wrong.
			REM TODO - do something better here. 
			exit /b
		)

		echo Created shadow of "!currentDrive!" using "!SHADOW_DEVICE_1!" "!SHADOW_ID_1!"
	) else (
		echo Re-using current shadow of "!currentDrive!" 
	)
	
	endlocal & set shadowDrive=%currentDrive%
goto :EOF

REM Deletes a shadow drive
:removeshadow
	setlocal EnableDelayedExpansion
	set shadowDrive=%~1
	echo removeshadow "!shadowDrive!"

	if "%shadowDrive%"=="" ( 
		echo Huh? No shadowDrive set. 
		REM TODO - do something better here. 
		exit /b 1
	)

	REM This check is left over from when I was developing the script, but might as well stay.
	if not exist %vssvarFile% (
		echo The file %vssvarFile% doesn't exist. Something has gone seriously wrong here.
		REM TODO - do something better here. 
		exit /b 2
	)

	call %vssvarFile% > nul
	echo deleting "!SHADOW_ID_1!" ("!shadowDrive!")
	
	call vshadow.exe -ds=!SHADOW_ID_1!
	
	if %ERRORLEVEL% NEQ 0 ( 
		echo Oh noes! Something went wrong.
		REM TODO - do something better here. 
		exit /b 3
	)
	
	del /Q %vssvarFile%
	echo deleted shadow of "!shadowDrive!"

	REM This will happen if something went wrong, or if there are multiple shadow copies of the same volume. 
	REM I'm not really sure what we will do in this scenario.
	if exist %vssDriveLetter% ( 
		echo Shadow drive !vssDriveLetter! is still mounted. This is a problem!
		exit /b 4
	)
	
	endlocal
goto :EOF

REM Makes the parameter upper case
:upper
	for %%a in (A B C D E F G H I J K L M N O P Q R S T U V W X Y Z) do call set "%1=%%%1:%%a=%%a%%%"
goto :EOF

Rather than do a detailed breakdown of this, I have commented the code. Whilst REM statements do slow down the execution by a tiny amount, on a modern computer the effect is negligible.

 

Task schedule

This is unchanged from my previous post.

 

Sources

https://rakhesh.com/windows/how-to-backup-open-pst-via-robocopy

http://blog.johnwray.com/post/2016/05/06/robocopy-from-shadowcopies-previous-versions

https://msdn.microsoft.com/en-us/library/windows/desktop/bb530725(v=vs.85).aspx

https://msdn.microsoft.com/en-us/library/windows/desktop/bb530726(v=vs.85).aspx

Arrays, structures and linked lists in Batch files (www.dostips.com)

https://helloacm.com/how-to-use-array-in-windows-batch-programming/

 


Update

Since writing this, I have discovered RoboMirror, which is a Windows application that pretty much fulfils all my requirements, so I have started using that instead.

I will leave this post here as the batch file stuff was quite interesting.

Tagged , , , , , , , . Bookmark the permalink.

About DataHamster

The Data Hamster stores facts and information in its capacious cheek pouches and regurgitates them from time to time.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.