Java background server class that might be called from ui thread

Disclaimer: This answer uses the lambda-feature for java 8. Assuming non-availability of java 8, the expressions can be easily converted to Runnables.

You're completely overexerting yourself. Use inbuilt features of Java with a little composition and your server becomes by far easier:

public class Server {
     private final ExecutorService threadPool = Executors.newFixedThreadPool(1);

     public Server() {}

     public void start() {
         threadPool.execute(() -> handleStart()); //Runnable
     }

     public void stop() {
         threadPool.execute(() -> handleStop());
         threadPool.shutdown(); //depends a little on your functionality
     }
     //rest can remain unchanged
}

The Thread-Pool can be expanded manually. Additionally I'd recommend to override finalize() to shut down your executor.

What you have done is not the ideal way. For UI applications that generate events processed by a "Server", there are established design patterns. One such pattern is the "Event loop" or the "Scheduler queue". Goes something like this (In a pseudo-Java like language that is not quite verbose):

UI:

void onSomeUIEvent(UIEvent ev) {
    ServerContext.pushEvent(ev);
}

ServerContext:

static void pushEvent(UIEvent ev) {
    _blockingQueue.push(ev);
}

Server:

class Server {

    private BlockingQueue<UIEvent> _blockingQueue; //Shared with ServerContext
    Thread worker {
        void run() {
            UIEvent e = _blockingQueue.pop();
            //Now do something with the event, like start/stop something

        }
    }
}