Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm trying to put up a temporary band-aid while I work out a solution for a app's memory leak. What I wrote was a small bash script that I put in the root on the server. This is what the script is suppose to do:

  1. Get the location it's run in
  2. Check to to see if the script is the crontab
  3. If not in the crontab add to the crontab to run every 5mins
  4. Test the memory and check to see if the % is above percent_allowed
  5. If above test then restart nginx & php-fmp services

memory_protect.sh:

#!/bin/sh
cronfile=memory_protect.sh    #NOTE THIS SHOULD DETECT IT'S SELF
path=pwd                      #this is the path to the file 
percent_allowed=80            #this should be max memory before action

has_cron(){
    #is the file in the cron?
    return [[ crontab -l | egrep -v '^$|^#' | grep -q $cronfile ]] && return 1 || return 0
}
test_memory(){
    memusage=`top -n 1 -b | grep "Mem"`
    MAXMEM=`echo $memusage | cut -d" " -f2 | awk '{print substr($0,1,length($0)-1)}'`
    USEDMEM=`echo $memusage | cut -d" " -f4 | awk '{print substr($0,1,length($0)-1)}'`

    USEDMEM1=`expr $USEDMEM \* 100`
    PERCENTAGE=`expr $USEDMEM1 / $MAXMEM`
    #if it's above 80% alert
    return [[ $PERCENTAG>$percent_allowed ]] && return 1 || return 0
}

if [[ has_cron -eq 0 ]]
then
    #was not here so add
    #run this script every 5 mins
    */5 * * * $path/$cronfile
fi

if [[ test_memory ]]
then
    #clear some memory
    /etc/init.d/nginx restart
    /etc/init.d/php-fpm restart
fi

The memory test seems to work when I run that by it's self, but this in whole doesn't seem to be working.


Update I needed to run dos2unix on the file, but I also realized I have a return on the conditions of each function in the end.. so that was not going to work. Right now it seems to say that [[ on the if statement is not found.


Update 2 Seems close, it's running the restarting of the services, but it's not putting the cron job in.. so I don't see it running

#!/bin/bash
cronfile=memory_protect.sh    #NOTE THIS SHOULD DETECT IT'S SELF
path=pwd                      #this is the path to the file 
percent_allowed=80            #this should be max memory before action

has_cron(){
    #is the file in the cron?
    #return 0 #returning this just to test should
    #be the next line but it's not working
    return 0
    [[ crontab -l | egrep -v '^$|^#' | grep -q $cronfile ]] && return 1 || return 0
}
test_memory(){
    memusage=`top -n 1 -b | grep "Mem"`
    MAXMEM=`echo $memusage | cut -d" " -f2 | awk '{print substr($0,1,length($0)-1)}'`
    USEDMEM=`echo $memusage | cut -d" " -f4 | awk '{print substr($0,1,length($0)-1)}'`

    USEDMEM1=`expr $USEDMEM \* 100`
    PERCENTAGE=`expr $USEDMEM1 / $MAXMEM`
    #if it's above 80% alert
    [[ $PERCENTAG -gt $percent_allowed ]] && return 1 || return 0
}

if [[ has_cron -eq 0 ]]
then
    #was not here so add
    #run this script every 5 mins
    #crontab -e */5 * * * $path/$cronfile
    cat <(crontab -l) <(echo "*/5 * * * $path/$cronfile") | crontab -
else
    echo "cron present"
fi

if [ test_memory ]
then
    #clear some memory
    sudo /etc/init.d/nginx restart
    sudo /etc/init.d/php-fpm restart
fi

It's close now I think to being corrected.

share|improve this question
    
@sim well the has_cron seems to be the current place of issue. But when I ran it after once with has_cron returning a 0 it didn't set the task either. –  jeremy.bass Sep 21 '13 at 17:37
    
@sim to set the cron job if it doesn't exist –  jeremy.bass Sep 21 '13 at 17:47
    
You need to do a crontab -e to add to a user's crontab entry. Not sure how to do that from a script though, let me look it up. HERE: stackoverflow.com/questions/878600/… –  slm Sep 21 '13 at 17:48
    
I would never use kind of script, there are application to keep your application/server running and monitor , Example monit –  Rahul Patil Sep 21 '13 at 20:31
add comment

2 Answers 2

up vote 2 down vote accepted

To create a crontab entry via a Bash script you'll need to change this line:

*/5 * * * * $path/$cronfile

To something like this:

# Write out current crontab
crontab -l > mycron

# Echo new cron into cron file
echo "*/5 * * * * $path/$cronfile" >> mycron

# Install new cron file
crontab mycron
rm mycron

You could also get fancy and do it all with this one liner:

cat <(crontab -l) <(echo "*/5 * * * $path/$cronfile") | crontab -

Your script

Here's a modified version of your script that works for me.

#!/bin/sh

cronfile=memory_protect.sh    #NOTE THIS SHOULD DETECT IT'S SELF
path=$(pwd)                   #this is the path to the file 
percent_allowed=80            #this should be max memory before action

has_cron(){
    #is the file in the cron?
    #return 0 #returning this just to test should
    #be the next line but it's not working
    if crontab -l | egrep -v '^$|^#' | grep -q $cronfile; then
      return 1
    else
      return 0
    fi
}
test_memory(){
    memusage=$(top -n 1 -b | grep "Mem")
    MAXMEM=$(echo $memusage | cut -d" " -f2 | awk '{print substr($0,1,length($0)-1)}')
    USEDMEM=$(echo $memusage | cut -d" " -f4 | awk '{print substr($0,1,length($0)-1)}')

    USEDMEM1=$(expr $USEDMEM \* 100)
    PERCENTAGE=$(expr $USEDMEM1 / $MAXMEM)
    #if it's above 80% alert
    [[ $PERCENTAG>$percent_allowed ]] && return 1 || return 0
}

if has_cron;
then
    #was not here so add
    #run this script every 5 mins
    #crontab -e */5 * * * $path/$cronfile
    #cat <(crontab -l) <(echo "*/5 * * * $path/$cronfile") | crontab -
    crontab -l > mycron

    # Echo new cron into cron file
    echo "*/5 * * * * $path/$cronfile" >> mycron

    # Install new cron file
    crontab mycron
    rm mycron
else
    echo "cron present"
fi

if test_memory;
then
    #clear some memory
    echo "/etc/init.d/nginx restart"
    echo "/etc/init.d/php-fpm restart"
fi

Example

$ ./memory_protect.sh 
/etc/init.d/nginx restart
/etc/init.d/php-fpm restart

$ crontab -l
*/5 * * * * /home/saml/tst/91789/memory_protect.sh

The script needs to have these two lines modified so that it will actually restart the nginx and php-fpm services.

Change these lines: echo "/etc/init.d/nginx restart" echo "/etc/init.d/php-fpm restart"

To these:

    /etc/init.d/nginx restart
    /etc/init.d/php-fpm restart

I did this just so I could see that the script was running correctly. NOTE: that these restart lines should be prefixed with sudo if this script is running as anyone other than root!

        sudo /etc/init.d/nginx restart         sudo /etc/init.d/php-fpm restart

This use will likely need to have NOPASSWD privileges at least on these 2 scripts otherwise it will be waiting for the user that owns the cron to supply a password.

Crontab entry doesn't exist?

You'll encounter this problem when your crontab hasn't been created yet in the directory, /var/sppol/cron. You'll encounter it when you run crontab -l like this:

$ crontab -l
no crontab for saml

Double check:

$ sudo ls -l /var/spool/cron/
total 0
-rw------- 1 root root 0 Sep 16 23:47 root

So just simply make like you're going to edit it, and save the empty file to create it:

$ crontab -e

# now you're in the vim editor, add a empty line
# type "i", hit return, hit Escape,
# and do a Shift + Z + Z to save!

Now you should see this:

$ crontab -l
$ 

And this:

$ sudo ls -l /var/spool/cron/
total 0
-rw------- 1 root root 0 Sep 16 23:47 root
-rw------- 1 saml root 0 Sep 21 16:20 saml

Restarting the services

Another issue you'll run into is if this crontab entry is running as user, user1, then user1 will require sudo rights to restart them.

share|improve this answer
    
so I just get a : not foundect.sh: 4: return. I changed the line return [[ $PERCENTAG>"80" ]] && return 1 || return 0 to add quotes on the 80 as it was complaining about that –  jeremy.bass Sep 21 '13 at 18:22
1  
@jeremyBass_DC - can you update your Q with the new problem? I'm not following what you're saying. –  slm Sep 21 '13 at 18:38
    
so when I check for the cron job with crontab -l I just get no crontab for vagrant. Vagrant is the user. But not seeing anything .. and when I check for root it has none either –  jeremy.bass Sep 21 '13 at 20:09
    
@jeremyBass_DC - look at my version of your script, I fixed that too. Need to tell Bash to exec $(pwd) not the string "pwd". –  slm Sep 21 '13 at 20:21
    
@jeremyBass_DC - OK it only took like 2 hours, I understand what you're doing now, sorry it's a Saturday, see my updates. The [[ ... ]] blocks weren't doing what you thought. You can use the if ..cmd.. bare without them as long as you do the returns as you are. –  slm Sep 21 '13 at 20:33
show 10 more comments

I would use monit for this

Here is monit script example, for your requirement

check process nginx with pidfile /var/run/nginx.pid
  start program = "/etc/init.d/nginx start"
  stop program  = "/etc/init.d/nginx stop"
  if memory usage > 95% then restart
  group www-data 
  if 5 restarts within 5 cycles then timeout 

Try to restart 5 times; if monit cannot restart webserver 5 times; just time out to avoid race condition.

Reference Link

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.