Shell script to archive files to AWS S3

Question

This is a simple script that moves files to S3 if they are > 14 days old. It ignores files that are < 14 days old. If the file is successfully synced to AWS S3 then we can remove it from the server. If not, we leave the file where it is. The echo commands in the script are for debugging. I am aware that this could be a one liner, so interested to see what people suggest.

#!/bin/bash

fortnight=$(python -c "from datetime import datetime, timedelta; print (datetime.now()-timedelta(days=14)).strftime('%Y%m%d')")

for f in /var/mail/catchall-????????.gz
do
    if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then
        mv "$f" ./archive
        echo "$f"
        echo "$(basename $f)"
        cd ./archive
            if s3cmd sync "$(basename $f)" s3://vpcgwmail ; then
                echo "S3 sync successful, removing file"
                rm "$(basename $f)"
            else
                echo "s3 sync not succesful"                
            fi
        fi
done

Answer 1

It's a fine script, but you have a bug: executing cd ./archive in a loop. Perhaps you did not notice it because you tested with only a single matching file. If there are multiple matching files, then cd ./archive will not work, unless you have multiple nested archive/archive/archive/... directories, which would be silly.

One solution is to wrap the cd command and the rest of the loop body in a (...) subshell:

if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then
    mv "$f" ./archive
    echo "$f"
    echo "$(basename $f)"
    (
        cd ./archive
        if s3cmd sync "$(basename $f)" s3://vpcgwmail ; then
            echo "S3 sync successful, removing file"
            rm "$(basename $f)"
        else
            echo "s3 sync not succesful"                
        fi
    )
fi

Another solution is to not cd ./archive at all:

if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then
    mv "$f" ./archive
    echo "$f"
    echo "$(basename $f)"
    if s3cmd sync ./archive/"$(basename $f)" s3://vpcgwmail ; then
        echo "S3 sync successful, removing file"
        rm ./archive/"$(basename $f)"
    else
        echo "s3 sync not succesful"                
    fi
fi

Whichever alternative is fine.

The indentation was also a bit off, from about the middle of the for-loop body.