All Questions
Tagged with rsync shell-script
95
questions
3
votes
1
answer
466
views
How to exclude swapfile from rsync backup?
I'm trying to write a simple script that would back up my system files. Here's what I wrote so far:
#!/usr/bin/sh
if [ $UID != 0 ]; then
echo "Please run me as root"
exit 1;
...
- 33
0
votes
1
answer
496
views
rclone - warning: shell level (1000) too high, resetting to 1
I am running a bash script that performs rclone on a directory with multiple files. However, the bash command does not finish and I get a repeating warning instead. Below you will find my bash file
#!...
1
vote
0
answers
167
views
Is there a way to use rsync to copy only what has changed
Im having a hard time with rsync to make an incremntal backup of a directory and upload it to a remote server, I have easily done this using a full backup the hard part is with incremental is that ...
- 11
1
vote
1
answer
270
views
rsync / mv how to skip empty files
I have a requirement to copy files from source1 directory to destination1 directory and move from source1 directory to destination2 directory. It is working fine. But script is copying even empty ...
- 113
0
votes
1
answer
192
views
How to initiate rsync transfer from one server to another through ssh?
I'm trying to easily move some data with rsync from one server to another without actually connecting manually and doing all that, but only giving the IPs as arguments.
# -- Variables
my_key="...
- 591
1
vote
1
answer
496
views
zsh script to rsync multiple files from same remote directory
I'm running Debian Bullseye (if that matters), and I'm writing a zsh script to use rsync to copy multiple files/directories (passed as arguments to the script) from a given remote directory. They're ...
- 53
0
votes
1
answer
172
views
Synchronizing access to shared, remote resource
I have a shared cache on a remote server that multiple clients are reading and writing to, so I need to synchronize access to this cache. I imagine I could:
SSH into the remote and acquire a flock on ...
- 101
0
votes
1
answer
1k
views
Why is my bash variable not expanded when used in this variable assignment?
I'm working to make a small script to transfer some files with rsync. I'm stuck with a variable issue.
I have the folder: /home/folder_1/, which have the folders 20210725, 20210726, 20210727 and ...
- 13
-1
votes
1
answer
669
views
Bash function with dry-run and exec options [closed]
Am writing a bash function to run rsync, with possibility of using --dry-run option and --exec option. Need some criticism improvements and additional checks.
Might not make much sense for the user ...
- 391
0
votes
0
answers
449
views
RSync creates empty folders
I have following directories. My goal is to compare OLD vs NEW directory and save the different files in the DEST directory:
OLD/
...src/
...src/config.js
...test/
...test/main.js
NEW/
...src/
......
- 113
1
vote
1
answer
187
views
Bash script to copy files and then move to trash (non-destructive)
I am trying to write a bash script to copy files with cp or, preferably, rsync and then move the source files to the trash. I do not want to use mv because in case of an error, I want to be able to ...
- 11
0
votes
1
answer
372
views
rsync delete older `--backup-dir` directories if sufficient space is not available to make the current backup
I create backup using rsync with the following two scripts:
Script 1:
#!/bin/bash
rsync --dry-run --stats -ab --human-readable --inplace --delete-delay --debug=NONE --log-file=/media/blueray/...
- 2,648
7
votes
3
answers
668
views
How would you gracefully handle this snippet to allow for spaces in directories?
I have a series of commands, e.g.:
ssh -i key 10.10.10.10 mkdir -p "${DEST_PATH}/subdir1" "${DEST_PATH}/subdir2"
rsync "${SOURCE_PATH}" "$DEST_HOST:${DEST_PATH}/...
- 128
2
votes
1
answer
632
views
Detecting Directory Moves with Rsync
The command I am currently using to backup one HDD to another (locally, not remotely) is
rsync \
--info=PROGRESS2,BACKUP,DEL \
-ab \
--human-readable \
--inplace \
--delete-after \
...
- 2,648
0
votes
1
answer
51
views
warn and provide the number of files that will be affected by rsync command
My rsync's --dry-run output is:
% rsync --dry-run -avi --delete-after /home/blueray/Documents/rsync-test/src /home/blueray/Documents/rsync-test/dest
building file list ... done
.d..t...... src/
>f....
- 2,648
0
votes
1
answer
1k
views
Rsync script error "Is a directory"
I use rsync to backup my data to an external drive by running a bash script. In it, the instructions are as follows:
#!/usr/bin/env bash
#Backup
#-r : recursive
#--size-only : ignore timestamp, use ...
- 337
0
votes
1
answer
458
views
Source file corrupted when the rsync was interrupted
I am running a script to rsync a folder from client to server.
Syntax applied on the script is
rsync -avzP --append-verify --delete -e "SSH parameters" source_dir user@host:destination
But ...
0
votes
1
answer
131
views
Which protocol needs rsync to enable checksum calculation through source and destination server?
I execute rsync with the --checksum option on the destination server as follows:
rsync -av --bwlimit=800000 --delete --checksum /mnt/disks/SMB_SRC/ /mnt/user/dst
But as I mounted the source server as ...
- 467
0
votes
0
answers
141
views
rsync links to point on a new destination
I need to copy directory structure from one place to another. I need to copy only folders and symlinks.
I use command
find $SOURCE_PATH/ \( -type l -o -type d \) -printf "%P\0" | rsync --...
0
votes
2
answers
618
views
rsync copies the current directory when I don't pass an option to my script
Below is a script on Ubuntu 18.04 used to copy files from one directory to another. All works fine except the files from the current directory (/media/rmerriam/Backup/rsync_backup) are copied to the ...
- 101
1
vote
0
answers
190
views
Rsync, remove source files, symlink
Need help with this one. I am trying to move some folders from source to destination with error handling (because mv failed to close files looking at rsync now) so far this is what I have. But it’s ...
- 11
1
vote
2
answers
2k
views
How to sync/backup files whenever change is detected?
I have two partition, one for use and one for backup. So, is there any tool for syncing the folder from first partition to second. Also, the tool should sync immediately whenever a change is made to ...
- 189
1
vote
1
answer
355
views
Rsyncing directory to a backup gzip file
Objective: The efficient backup and transfer of ~/dircontaininginnumeraltinyfiles/ to /mnt/server/backups/dir.tgz. My first thought was of course rsync and gzip. But I'm open to other routes as well.
...
- 11
0
votes
2
answers
80
views
copying specific files to different directory
I am trying to find files with specific extensions and copy or rsync it to another hard drive / directory
sudo find . \( -type f \( -iname '*.jpg' -o -iname "*png" -o -iname "*mov" -o -iname "*mp3" -...
0
votes
1
answer
91
views
Shell Script that create log if file is not send to remote server
I have this small script which basically do rsync of files between to remote server. ssh key has been generated for authentication.
only this is present in the script
rsync /root/Desktop/cpfies/...
user404480
0
votes
2
answers
126
views
Shell script that transfer file which takes date from till(Nov1) to to(Dec1) from name
Okay so this shell script is might sound complicated but I don't know how to achieve this.
so here is the requirement.
note the above files kindly.
filname are as follows.
Meter_20200120100846.xls
...
user404480
4
votes
3
answers
3k
views
Use variable to exclude multiple directories with rsync
The following snippet of code is part of a larger Bash script I'm working on. My intention is to pass input variables as arguments to Rsync.
I managed to make most of it work, except for the "--...
- 63
0
votes
1
answer
437
views
Failure of rsync to detect new files and directory
I have been performing backups to a remote machine via rsync for several years. At least over the past two months, rsync has stopped detecting new files and directories on the machine to be backed up. ...
- 111
0
votes
1
answer
133
views
How to rsync faster while checking the folder size first
I have just written a simple shell script with Automator on my Mac, which looks like this:
rsync -a --delete "/Volumes/Backup Plus/photo" "/Volumes/My Passport/photo"
rsync -a --delete "/Volumes/...
- 101
2
votes
1
answer
369
views
Connection to xx.xx.xx.xx closed by remote host during rsync backup
I have 2 web servers. Let's call them local and remote just for simplicity.
Local: I've only ftp and cron access, Remote: ssh/ftp
I wrote a simple backup script, which tars the files and then sends ...
- 131
1
vote
1
answer
275
views
rsync fails only when initiated via cron
I'm attempting to rsync files to my local machine from a remote server on boot via a crontab entry. My rsync script waits for resolv.conf to be configured by DHCP processes, then begins the rsync. ...
- 61
0
votes
1
answer
165
views
Sync two directories: same filename, different fileending, different filecontent
I have 2 folders with subdirectories. One with training images and one with training labels.
One image belongs to exactly one label
The image and the label have different content
The image and the ...
- 15
0
votes
0
answers
94
views
scp or rsync multiple files from source server to another servers
Since my nfs is a win type of server, it can't contain :. I want to copy multiple files from source server to the nfs directory on another server and rename them at the same time.
The script looks ...
- 11
1
vote
0
answers
40
views
Bash shell script error after rsync to server [duplicate]
I get this after rsync the .sh file to the server:
sudo: unable to execute ./run_vps_post_config.sh: No such file or directory
To fix, I open the file with winscp and save it. Then it runs again.
(...
- 457
2
votes
2
answers
669
views
How to write shell script to execute rsync each time I plug in my external ssd?
I want to execute rsync command which syncs my local directory with the external ssd. I followed the instructions given in this link. to execute some command based on udev rules. This shows how to run ...
- 121
1
vote
1
answer
731
views
How do I confirm a file sync has completed before executing a command?
I have a computer periodically syncing folders of content with another computer using Resilio Sync. The receiving computer has a sorting process on an hourly cron, which analyses the folders and their ...
-1
votes
3
answers
4k
views
Use scp to copy files of specific extension from directory
I'm writing a bash script that needs to fetch all *_out.csv from a directory, on a remote server. All these files are several directories deep inside of another directory. So for instance, say the ...
- 201
1
vote
1
answer
1k
views
Run Rsync for multiple files in parallel from 2 different directories and find elapsed time for each file and all files
I have lots of dump files in 2 different directories that need to be uploaded using rsync, I want to run rsync for each dump file simultaneously to save time.
Dump Files:
Dir1:
- /u05/expdpdump/...
- 73
2
votes
1
answer
308
views
Bash scripting and rsync: how to include just some folders and subfolders and what happends if I use ../ as path with --delete
I have a bash script where I'm trying to copy some folders and archives in local with rsync. I have something like this:
RootFolder
A
B
C
D
D1
D2
D3
.
.
.
...
- 131
2
votes
2
answers
3k
views
How to recursively copy only the files from folders and subfolders?
Under /home folder we have many subfolders, as the following:
/home/user1
/home/user2/user_sub_2
/home/user3/user_sub_3/info_sub
/home/user4/INFO_FOLDER
We want to copy all the files under /home, ...
- 13k
0
votes
1
answer
910
views
How to know if rsync did not change any files?
In a bash script, I am using rsync to make an exact copy of a directory:
rsync --verbose --archive --update --delete \
~/cat-pics/ \
~/top-secret/
But how can I get feedback on whether or ...
- 2,858
0
votes
1
answer
624
views
rsync not quitting and running next rsync
I have a shell script that has 6 rsync commands. What it should do is run the first and then move on to the next. However, what it does right now is do the first once it is finished it'll hang and ...
- 101
-1
votes
1
answer
58
views
Why does adding -prune to my sync script cause rsync to do a DRY-RUN?
I'm testing a script to do a by-directional sync of two directories intelligently using rsync.
Since I'm testing many of the rsync options are not applicable to the testing environment including the ...
0
votes
3
answers
706
views
How to properly substitute a string containing spaces in my script
In my script I've got:
sshkey="/Users/me/some path/with spaces/id_rsa"
dstport=...
dstparent=...
dstuserhost=...
rsync -az --delete -e "ssh -i $sshkey -p $dstport" $src $dstuserhost:$dstparent
And ...
- 2,788
2
votes
2
answers
536
views
Using rsync to delete selected patterns, matches all files
I mean to use rsync to remove certain files (for Efficiently delete large directory containing thousands of files), in this case given to a shell script as patterns in the command line.
So far, this ...
0
votes
1
answer
292
views
How to read only the last line of a live input stream in a while read loop
I have a very simple bash script that synchronizes data to a USB key as soon as new files appear. It is poorly written but it works, however I want to minimize CPU time so I used sleep but since I use ...
- 1
5
votes
1
answer
9k
views
Scheduled folder backup
I'm looking for how to automatically backup a user's home directory in CentOs 7 to a remote host or NAS or just to ~/.snapshot. In some Linux setups, I have seen a .snapshot folder in the user's home ...
- 155
0
votes
1
answer
744
views
rsync the info to a file
I use rsync to backup my data from one server to the other. This works great.
rsync -aktvu -e 'ssh -p 22' --modify-window=1 --progress $SOURCE $USER@$SERVER:$DESTINATION
The output display what has ...
- 3
2
votes
1
answer
2k
views
Help with script/rsync command to move file with md5 sum comparison before deleting the source file/ [closed]
Referencing this post to find and delete duplicate files based on checksum, I would like to modify the approach to perform a copy operation followed by a file integrity check on the destination file.
...
- 23
0
votes
1
answer
1k
views
How to rsync and grep only path of modified files in source directory
Is there a way to just grep only changes ie; (add, delete, modified) and print path and file name using rsync.
I tried with below command but I am getting the path and file modified in separate line.
...
- 15