Scenario: Ftp site has 100s of files. Once a day, all the .gz files are copied to an AWS site then renamed. Only the date is kept in the file name.
Some of the files corrupted during copy. A txt file was delivered with a snapshot of ALL the files and attributes from the FTP site.
I have to write a script that will
- read the text file only for .gz files
- convert the name of the file to the filename on the S3 site
- Compare the file size from the txt file to the file size on the S3 server
- If the file sizes doen't match, write the filename and percent diff out to a txt file.
This is what I have so far, which is not even close to working
Suggestions?:
# Create array from files in FTP site text file
f=cat ftpfiles.txt | grep .gz | awk '{print $9,$5}' #this doesn't work
# Start ForEach loop for files in created array
for f in *.gz
do
# Create variable for file size of source file
file1size=$( cat $f | wc -c )
# Create variable for file size of destination file
fiesize2=aws s3 ls s3://folder1/folder2/$f | awk '{print $3}'
#Compare sizes and print result to .txt file by percent diff
echo "$f"
if [ $file1size -lt $file2size ]; then
size=$file1size
else
size=$file2size
fi
dc -e "
3k
$( cmp -n $size -l $file1 $file2 | wc -l )
$size
/
100*
p"
#close loop
done