Fast and reliable data backup to the cloud 2
In my previous article, I described an example of a script that can archive and upload a certain amount of user files to the cloud without its participation, thereby ensuring the simplest backup of user files. The script closes the following questions:
Naturally, the simplicity of the script turned its flaws, namely:
In this regard, the script code of the full and incremental archive has been added with the necessary functionality. For a general description, see the previous article , here I want to dwell only on the changes.
call pass.bat xx x - calls an external program that returns the variable xx back with the value specified for the identifier x.
Accordingly, the user variable is assigned the value associated with the identifier 1 (username of the account yandex), pass - 2 (password of the account yandex), pwd - 3 (password to the archive), recipient - 4 (mail, where to send the operation log).
The pass.bat script is given conditionally, the password protection method can be chosen differently, but for those and this option is appropriate, I quote the simplest code of this script:
When calling this code, it finds next to itself the file pass.txt, and assigns the variable whose name is specified in the first argument of the script call to the value of that line of the pass.txt file whose number is specified in the second argument of the script call. The location of the pass.bat and pass.txt files is not critical, as long as they are in the same directory as specified in the system path variable.
set mailsend = mailsend1.17b15.exe - an indication of the program for sending a report to the user's mail. The letter is sent from the account yandex.ru, from which the files are sent to the cloud.
set i = 1 - set the retry count to 1
call: loging "-----------------------------------" - a call to the logging procedure. A string is sent to the procedure, which is then written to 2 files - the % filebkp% -log.txt log file and the % currentbkp% -log.tmp single file of current operations, which is then sent to the user as a report on the archive result.
Accordingly, logging lines in the body of scripts simultaneously show the user the operations performed during operation, write them to the logs and for me are comments in the body of the script, explaining what is being done on the following lines.
Accordingly, a full copy script and an incremental copy script perform the following operations:
These scripts are given only as examples, there is no desire to rewrite under PowerShell, since It is important for me that it works on any system, including Windows XP. There is no protection against password theft; I’d be very interested in what security experts would suggest, but separating the code and passwords with the possibility of placing the passwords in another directory at least removes the possibility of a stupid mistake that when sending a script to a friend, he will also get the passwords.
- fast (perfect in the background);
- would save both the current version of the files and the previous ones;
- file storage would not be available to viruses or natural disasters such as killing a carrier or stealing it;
- the contents of the backup would be available only to the user, i.e. has been polished.
Naturally, the simplicity of the script turned its flaws, namely:
- Storing passwords in the body of the script. Yes, many programs store passwords unencrypted in text files, and protection of this data occurs through file access control, but still removing the password from the code is a sign of good tone and minimal protection in case the working script somehow falls into the wrong hands .
- Lack of verification of successful completion of the operation, and rollback to the previous state in case the operation was not successfully completed.
- The lack of logging and notifying the user about the success of the operation, if the script is launched through the Task Scheduler.
In this regard, the script code of the full and incremental archive has been added with the necessary functionality. For a general description, see the previous article , here I want to dwell only on the changes.
The script to create a complete archive
@echo off chcp 1251 >nul set filebkp=server call pass.bat user 1 call pass.bat pass 2 call pass.bat pwd 3 call pass.bat recipient 4 set mailsend=mailsend1.17b15.exe set i=1 set pathbkp=..\backup set pathtemp=..\temp set srvbkp=https://%user%:%pass%@webdav.yandex.ru/backup/%filebkp% set full=%filebkp%-full set inc=%filebkp%-inc set h=%TIME:~0,2% set m=%TIME:~3,2% set s=%TIME:~6,2% set ms=%TIME:~9,2% set curtime=%h%-%m%-%s% set dd=%DATE:~0,2% set mm=%DATE:~3,2% set yyyy=%DATE:~6,4% set curdate=%yyyy%-%mm%-%dd% set curdatetime=%curdate: =0%-%curtime: =0% call :loging "-----------------------------------" call :loging "Текущее время - %curdatetime%" call :loging "Создаем резервную копию текущих архивов" ren "%pathtemp%" temp-bkp md "%pathtemp%" call :loging "Создаем полный архив %full%.7z" 7z.exe a -xr0!*.log -xr0!*.bak -xr0!*.tmp "%pathtemp%\%full%".7z -r -mx1 "%pathbkp%\*" -%pwd% call :loging "Создаем резервную копию текущих архивов на сервере" curl.exe -k -X MOVE -H "Destination:/backup/%filebkp%"-bkp "%srvbkp%" -o .\stdout curl.exe -k -X MKCOL "%srvbkp%" -o .\stdout :upload call :loging "Попытка отправки номер %i%" call :loging "Отправляем на сервер файл %full%.7z" curl.exe -k -T "%pathtemp%\%full%".7z "%srvbkp%"/ --progress-bar -o .\stdout call :loging "Скачиваем файл %full%.7z с сервера для сравнения с исходным" curl.exe -k "%srvbkp%/%full%".7z -o "%pathtemp%"\test.7z --progress-bar -o .\stdout call :loging "Сравниваем скачанный файл с исходным" fc /LB1 /B "%pathtemp%\%full%".7z "%pathtemp%"\test.7z > nul if ERRORLEVEL 1 goto Different call :loging "Файлы одинаковы:" set tolog="%full%.7z успешно отправлен на сервер" call :loging %tolog% set mailsubject=%tolog:"=% call :loging "Удаляем устаревшие версии архива с компьютера и сервера" RD /s/q "%pathtemp%"-bkp curl.exe -k -X DELETE "%srvbkp%"-bkp -o .\stdout del /q "%pathtemp%"\test.7z goto exit :Different call :loging "Файлы различаются: %full%.7z на сервере НЕ совпадает с исходным" call :loging "Повторяем отправку" set /a i+=1 if %i% = 5 goto exit-f goto upload :exit-f set tolog="Отправка файла %full%.7z на сервер не удалась" call :loging %tolog% set mailsubject=%tolog:"=% call :loging "Восстанавливаем файлы предыдущего архива" RD /s/q "%pathtemp%" ren "%pathtemp%"-bkp temp curl.exe -k -X DELETE "%srvbkp%" -o .\stdout curl.exe -k -X MOVE -H "Destination:/backup/%filebkp%" "%srvbkp%"-bkp -o .\stdout :exit echo Отправляем лог %filebkp%-log.txt на сервер curl.exe -k -T %filebkp%-log.txt "%srvbkp%"/../ --progress-bar -o .\stdout echo Отправляем отчет в почту %mailsend% -to %recipient% -from %user%@yandex.ru -ssl -port 465 -auth -smtp smtp.yandex.ru -user %user% -pass %pass% -sub "%filebkp%: %curdatetime% - %mailsubject%" -cs "windows-1251" -mime-type "text/plain" -msg-body %filebkp%-log.tmp del /q %filebkp%-log.tmp exit :loging echo %~1 echo %~1 >> %filebkp%-log.txt echo %~1 >> %filebkp%-log.tmp exit /b Script of creation of incremental archive
@echo off chcp 1251 >nul set filebkp=server call pass.bat user 1 call pass.bat pass 2 call pass.bat pwd 3 call pass.bat recipient 4 set mailsend=mailsend1.17b15.exe set tolog=zero set mailsubject=zero set i=1 set pathbkp=..\backup set pathtemp=..\temp set srvbkp=https://%user%:%pass%@webdav.yandex.ru/backup/%filebkp% set full=%filebkp%-full set inc=%filebkp%-inc set h=%TIME:~0,2% set m=%TIME:~3,2% set s=%TIME:~6,2% set ms=%TIME:~9,2% set curtime=%h%-%m%-%s% set dd=%DATE:~0,2% set mm=%DATE:~3,2% set yyyy=%DATE:~6,4% set curdate=%yyyy%-%mm%-%dd% set curdatetime=%curdate: =0%-%curtime: =0% call :loging "-----------------------------------" call :loging "Текущее время - %curdatetime%" call :loging "Создаем инкрементный архив %inc%-%curdatetime%.7z" 7z.exe u -xr0!*.log -xr0!*.bak -xr0!*.tmp "%pathtemp%\%full%".7z -u- -up3q3r2x2y2z0w2!"%pathtemp%\%inc%-%curdatetime%".7z "%pathbkp%\*" -%pwd% :upload call :loging "Попытка отправки номер %i%" call :loging "Отправляем архив на сервер" curl.exe -k -T "%pathtemp%\%inc%-%curdatetime%".7z "%srvbkp%"/ --progress-bar -o .\stdout call :loging "Скачиваем архив с сервера для сравнения с исходным" curl.exe -k "%srvbkp%/%inc%-%curdatetime%".7z -o "%pathtemp%"\test.7z --progress-bar -o .\stdout call :loging "Сравниваем скачанный архив с исходным" fc /LB1 /B "%pathtemp%\%inc%-%curdatetime%".7z "%pathtemp%"\test.7z > nul if ERRORLEVEL 1 goto Different call :loging "Файлы одинаковы" set tolog="%inc%-%curdatetime%.7z успешно отправлен на сервер" call :loging %tolog% set mailsubject=%tolog:"=% goto exit :Different call :loging "Файлы различаются: %inc%-%curdatetime%.7z на сервере НЕ совпадает с исходным" call :loging "Повторяем отправку" set /a i+=1 if %i% = 5 goto exit-f goto upload :exit-f set tolog="Отправка файла %inc%-%curdatetime%.7z на сервер не удалась" call :loging %tolog% set mailsubject=%tolog:"=% :exit del /q "%pathtemp%"\test.7z echo Отправляем лог %filebkp%-log.txt на сервер curl.exe -k -T %filebkp%-log.txt "%srvbkp%"/../ --progress-bar -o .\stdout echo Отправляем отчет в почту %mailsend% -to %recipient% -from %user%@yandex.ru -ssl -port 465 -auth -smtp smtp.yandex.ru -user %user% -pass %pass% -sub "%filebkp%: %mailsubject%" -cs "windows-1251" -mime-type "text/plain" -msg-body %filebkp%-log.tmp del /q %filebkp%-log.tmp exit :loging echo %~1 echo %~1 >> %filebkp%-log.txt echo %~1 >> %filebkp%-log.tmp exit /b New teams:
call pass.bat xx x - calls an external program that returns the variable xx back with the value specified for the identifier x.
Accordingly, the user variable is assigned the value associated with the identifier 1 (username of the account yandex), pass - 2 (password of the account yandex), pwd - 3 (password to the archive), recipient - 4 (mail, where to send the operation log).
The pass.bat script is given conditionally, the password protection method can be chosen differently, but for those and this option is appropriate, I quote the simplest code of this script:
for /f "usebackq delims=" %%i in (`find /n /v "" "%~dp0"pass.txt ^| find "[%2]"`) do ( set newvar=%%i ) set %1=%newvar:~3% When calling this code, it finds next to itself the file pass.txt, and assigns the variable whose name is specified in the first argument of the script call to the value of that line of the pass.txt file whose number is specified in the second argument of the script call. The location of the pass.bat and pass.txt files is not critical, as long as they are in the same directory as specified in the system path variable.
set mailsend = mailsend1.17b15.exe - an indication of the program for sending a report to the user's mail. The letter is sent from the account yandex.ru, from which the files are sent to the cloud.
set i = 1 - set the retry count to 1
call: loging "-----------------------------------" - a call to the logging procedure. A string is sent to the procedure, which is then written to 2 files - the % filebkp% -log.txt log file and the % currentbkp% -log.tmp single file of current operations, which is then sent to the user as a report on the archive result.
Accordingly, logging lines in the body of scripts simultaneously show the user the operations performed during operation, write them to the logs and for me are comments in the body of the script, explaining what is being done on the following lines.
Accordingly, a full copy script and an incremental copy script perform the following operations:
- for full archive - creates backup copies of current archives on the server and on the local computer
- creates a new backup
- trying to send it to the server
- after sending downloads a backup and compares with the original
- in case of success of the comparison — sends a log of operations to the server and a user in the mail a report on a successful operation through the program mailsend1.17b15.exe + for a full archive — deletes outdated versions of the archive
- if there is a difference between the files - it tries to resend the file to the server, and if it fails to send it within 5 attempts - sends a log of operations to the server and the user in the mail a failure report through the mailsend1.17b15.exe program:
- cleans up a temporary file of current operations.
PS
These scripts are given only as examples, there is no desire to rewrite under PowerShell, since It is important for me that it works on any system, including Windows XP. There is no protection against password theft; I’d be very interested in what security experts would suggest, but separating the code and passwords with the possibility of placing the passwords in another directory at least removes the possibility of a stupid mistake that when sending a script to a friend, he will also get the passwords.
Source: https://habr.com/ru/post/437794/