Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 2 down vote favorite

Here's the thing: I have an application server, called A, and a DB server, called B.

On B(DB), when I run netstat -ntp | grep 'A's IP' | wc -l, I see over 400 TCP connections. And when I connected to MySQL using MySQL client, and run select count(*) from processlist where host like 'A's IP%', I see over 400 results too, so it seems that both MySQL & OS agree there are over 400 open connections to A.

However, when I logon to A, run netstat -ntp | grep 'B's IP' | wc -l, I see only over 100 TCP connections.

So how can this happen? Am I using netstat wrong, or what? I just don't see why two hosts disagree on how many TCP connections that are between them.

Quick update: We actually have 6 different projects on A(Some are Rails 3.2, Some are Rails 2.3), including redmine. I grab the show processlist result on B, comparing it to netstat -ntp on A, found out that the unmatched connections on B is from different projects, including redmine. So I'm basically ruling out the application code, any wild guesses?

PS: Both OS is CentOS 5.4, MySQL is Percona 5.1.57-rel12.8-log

share|improve this question

1 Answer

up vote 1 down vote

Applications on A probably do not close their connections to MySQL properly on completion.

When a process terminates on A, the connections they own must be discarded automatically by the OS, but B is still awaiting for a signal from A.

B will eventually release these connections when the (MySQL) time-out delay is reached.

It is hard to be more specific without knowing more about the application on A, but there is nothing to be surprised of here (unless you didn't expect applications on A to be broken :)

share|improve this answer
Actually it is a Rails application using ActiveRecord, and DbCharmer to connect multiple DBs, nothing special here. I found this problem when I see a 'mysql too many connections' error the other day, it just came out of the blue, so anyway you guess it's the application's fault? – Dean Winchester 1 hour ago
Yes it is. You can mitigate the problem by lowering the value of wait_timeout or interactive_timeout on B, but this is only a workaround, there is something wrong on A. – YaK 1 hour ago
1  
I just did a quick test. On machine a, mysql -h 'b's ip' blah blah to connect to the MySQL on machine b. Meanwhile, when I run netstat or mysql's show processlist on machine b, I can see the connection's there. Then on machine a, I kill the process by kill -9 process_id, then run netstat again on machine b, the connection is gone, and it doesn't exist in show processlist either. – Dean Winchester 1 hour ago
I mean first, I'm wondering, theoritically, how can things like 'two hosts disagree on how many TCP connections that are between them' happen? – Dean Winchester 55 mins ago
Then I must be wrong. What is the status of the 400 connections you referred to in your question? – YaK 55 mins ago
show 4 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.