Our application is making a 3rd party webservice call which is pretty slow. A 1 user response time is about 6 seconds.
When we load these up with Siege for testing, approximately 50 concurrent users (roughly the tipping point), the response time goes up towards 60 seconds. The CPU and I/O and memory are all NORMAL during this time (no swapping). At the same time this is happening, if I try to load a local DB driven page that is normally very fast (1 second), it also takes forever to load (40+ seconds).
I'm looking for suggestions as to what might be the problem here and how to fix. Are the webservice calls somehow blocking the local DB driven calls?
Current lighttpd.conf settings are as follows:
server.max-fds = 2048
"min-procs" => 1,
"max-procs" => 10,
"idle-timeout" => 20,
"PHP_FCGI_CHILDREN" => "10",
"PHP_FCGI_MAX_REQUESTS" => "500"
Decreasing max-procs and increasing children yields worse results. CPU stays at 0 instead of going up to 1.0. The server is RHEL5 with 2 2.53GHz Xeons and 3GB ram.
