Implementing a full Python Unix-style daemon process

Question

I'm currently trying to manually create a simple daemon process, I don't want to use the existing externals libraries to avoid overhead.

I'm currently checking when my process runs that it doesn't have a PID file already created (meaning it's running), like described in this post.

I also have a daemonizing module to detach the PID from current process and redirect stdout and stderr (so my daemon will keep running even if I end my session):

import os
import sys

def daemonize(stdin="/dev/null", stdout="/dev/null", stderr="/dev/null"):
    try: 
        pid = os.fork() 
        if pid > 0:
            sys.exit(0)
    except OSError, e: 
        sys.stderr.write ("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror) )
        sys.exit(1)

    os.chdir("/") 
    os.umask(0) 
    os.setsid() 

    try: 
        pid = os.fork() 
        if pid > 0:
            sys.exit(0)
    except OSError, e: 
        sys.stderr.write ("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror) )
        sys.exit(1)

    stdin_par = os.path.dirname(stdin)
    stdout_par = os.path.dirname(stdout)
    stderr_par = os.path.dirname(stderr)
    if not stdin_par:
        os.path.makedirs(stdin_par)
    if not stdout_par:
        os.path.makedirs(stdout_par)
    if not stderr_par:
        os.path.makedirs(stderr_par)

    si = open(stdin, 'r')
    so = open(stdout, 'a+')
    se = open(stderr, 'a+', 0)
    os.dup2(si.fileno(), sys.stdin.fileno())
    os.dup2(so.fileno(), sys.stdout.fileno())
    os.dup2(se.fileno(), sys.stderr.fileno())

So currently I can run my process like sample line below and it will run my daemon correctly:

>$ python myapp.py

But to stop it, I have to grep the PID (or take it from the lock file) and manually erase the PID after:

>$ ps -ef | grep myapp
xxxxx  11901     1  0 19:48 ?        00:00:00 python src/myapp.py
xxxxx  12282  7600  0 19:54 pts/7    00:00:00 grep myapp
>$ kill -9 11901
>$ rm -rf /path/to/lock.pid

I'd like to have a more Unix-like daemon where I can manage the daemon lifecycle with the following commands:

>$ python myapp.py start
>$ python myapp.py stop
>$ python myapp.py restart

I can certainly do it with the argparse module, but that seems a bit tedious and ugly.

Do you know a simple and elegant solution to have a Unix-style daemon process in Python?

I think you are confusing daemon processes with startup scripts. They are usually different things. I mean, a Unix-like daemon behaviour does not have to include stopping and restarting facilities, it is just the TTY detaching mechanism.
You can try using the daemon package: pypi.python.org/pypi/python-daemon
@C2H5OH Yes you're right, i actually have the daemonization process working fine, but i'd like in the end to have it managed by the python application itself, without having to kill or rm manually.
@linker: Then you would have to implement whatever PID finding mechanism you like and then send that process a signal. There is no mystery.
That's actually what I'm looking for, if you know how to do this feel free to add an answer and I'll accept it.

Charles Menguy · Accepted Answer · 2012-04-18 20:24:53Z

After looking a bit more I found a great example that does just this here.

It uses a generic Daemon class, which can be subclassed afterwards:

#!/usr/bin/env python

import sys, os, time, atexit
from signal import SIGTERM

class Daemon:
        """
        A generic daemon class.

        Usage: subclass the Daemon class and override the run() method
        """
        def __init__(self, pidfile, stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'):
                self.stdin = stdin
                self.stdout = stdout
                self.stderr = stderr
                self.pidfile = pidfile

        def daemonize(self):
                """
                do the UNIX double-fork magic, see Stevens' "Advanced
                Programming in the UNIX Environment" for details (ISBN 0201563177)
                http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16
                """
                try:
                        pid = os.fork()
                        if pid > 0:
                                # exit first parent
                                sys.exit(0)
                except OSError, e:
                        sys.stderr.write("fork #1 failed: %d (%s)\n" % (e.errno, e.strerror))
                        sys.exit(1)

                # decouple from parent environment
                os.chdir("/")
                os.setsid()
                os.umask(0)

                # do second fork
                try:
                        pid = os.fork()
                        if pid > 0:
                                # exit from second parent
                                sys.exit(0)
                except OSError, e:
                        sys.stderr.write("fork #2 failed: %d (%s)\n" % (e.errno, e.strerror))
                        sys.exit(1)

                # redirect standard file descriptors
                sys.stdout.flush()
                sys.stderr.flush()
                si = file(self.stdin, 'r')
                so = file(self.stdout, 'a+')
                se = file(self.stderr, 'a+', 0)
                os.dup2(si.fileno(), sys.stdin.fileno())
                os.dup2(so.fileno(), sys.stdout.fileno())
                os.dup2(se.fileno(), sys.stderr.fileno())

                # write pidfile
                atexit.register(self.delpid)
                pid = str(os.getpid())
                file(self.pidfile,'w+').write("%s\n" % pid)

        def delpid(self):
                os.remove(self.pidfile)

        def start(self):
                """
                Start the daemon
                """
                # Check for a pidfile to see if the daemon already runs
                try:
                        pf = file(self.pidfile,'r')
                        pid = int(pf.read().strip())
                        pf.close()
                except IOError:
                        pid = None

                if pid:
                        message = "pidfile %s already exist. Daemon already running?\n"
                        sys.stderr.write(message % self.pidfile)
                        sys.exit(1)

                # Start the daemon
                self.daemonize()
                self.run()

        def stop(self):
                """
                Stop the daemon
                """
                # Get the pid from the pidfile
                try:
                        pf = file(self.pidfile,'r')
                        pid = int(pf.read().strip())
                        pf.close()
                except IOError:
                        pid = None

                if not pid:
                        message = "pidfile %s does not exist. Daemon not running?\n"
                        sys.stderr.write(message % self.pidfile)
                        return # not an error in a restart

                # Try killing the daemon process       
                try:
                        while 1:
                                os.kill(pid, SIGTERM)
                                time.sleep(0.1)
                except OSError, err:
                        err = str(err)
                        if err.find("No such process") > 0:
                                if os.path.exists(self.pidfile):
                                        os.remove(self.pidfile)
                        else:
                                print str(err)
                                sys.exit(1)

        def restart(self):
                """
                Restart the daemon
                """
                self.stop()
                self.start()

        def run(self):
                """
                You should override this method when you subclass Daemon. It will be called after the process has been
            daemonized by start() or restart().
                """

Once you have this module, you can do the following to have all different modes of your daemon:

#!/usr/bin/env python

import sys, time
from daemon import Daemon

class MyDaemon(Daemon):
        def run(self):
                while True:
                        time.sleep(1)

if __name__ == "__main__":
        daemon = MyDaemon('/tmp/daemon-example.pid')
        if len(sys.argv) == 2:
                if 'start' == sys.argv[1]:
                        daemon.start()
                elif 'stop' == sys.argv[1]:
                        daemon.stop()
                elif 'restart' == sys.argv[1]:
                        daemon.restart()
                else:
                        print "Unknown command"
                        sys.exit(2)
                sys.exit(0)
        else:
                print "usage: %s start|stop|restart" % sys.argv[0]
                sys.exit(2)

Hope that helps other people who get stuck on the same problem !

Note that the second fork is not necessary in modern Unices.

C2H5OH · Answer 2 · 2012-04-18 20:14:28Z

In order to ease the process of signaling the daemon process, we use the following function at work:

def set_procname(name):
    import ctypes
    lc = ctypes.cdll.LoadLibrary("libc.so.6")
    lc.prctl(15, name[:15])

Then, the daemon process can use it:

set_procname("foo")

And then we can do pkill foo to send a TERM signal. This is not a proper answer but it could be useful. However, the set_procname() function has some limitations:

It only works on Linux kernels.
You can only use up to 15 characters for the process name.
If you launch more instances of the same daemon process, pkill will signal them all.

asked	1 year ago
viewed	644 times
active	1 year ago

Implementing a full Python Unix-style daemon process

2 Answers

Your Answer

Not the answer you're looking for? Browse other questions tagged python unix daemon or ask your own question.

Linked

Implementing a full Python Unix-style daemon process

2 Answers

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged python unix daemon or ask your own question.

Linked

Related