Tell me more ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
up vote 8 down vote favorite

I'm working on a software which connects to a Real Time data server (using TCP) and I have some connections dropping. My guess is that the clients do not read the data coming from the server fast enough. Therefore I would like to monitor my TCP sockets. For this I found the "ss" tool.

This tool allows to see the state of every socket - here's an example line of the output of the command ss -inm 'src *:50000'

ESTAB      0      0             184.7.60.2:50000       184.92.35.104:1105
  mem:(r0,w0,f0,t0) sack rto:204 rtt:1.875/0.75 ato:40

My question is: what does the memory part mean? Looking at the source code of the tool I found that the data is coming from a kernel structure (sock in sock.h). More precisely, it comes from the fields :

r = sk->sk_rmem_alloc
w = sk->sk_wmem_queued;
f = sk->sk_forward_alloc;
t = sk->sk_wmem_alloc;

Does somebody know what they mean? My guesses are:

  • rmem_alloc : size of the inbound buffer
  • wmem_alloc : size of the outbound buffer
  • sk_forward_alloc : ???
  • sk->sk_wmem_queued : ???

Here are my buffers sizes :

net.ipv4.tcp_rmem = 4096        87380   174760
net.ipv4.tcp_wmem = 4096        16384   131072
net.ipv4.tcp_mem = 786432       1048576 1572864
net.core.rmem_default = 110592
net.core.wmem_default = 110592
net.core.rmem_max = 1048576
net.core.wmem_max = 131071
share|improve this question
What's your buffer size configuration? Do you see receive buffers getting saturated on the socket connections? Does your party drop connection on EWOULDBLOCK? – Karlson Mar 10 '12 at 6:28
My sockets sizes are quite small I think, i updated the post with them. For the EWOULDBLOCK I can't tell. My client is in JAVA and just say it has been disconnected by the server. The server is in C++ and it just say that he dropped the connection without any information. I do not have the source code of the server so I cannot change its behavior. It seems that clients get disconnected when they are a bit overloaded, even if it only last a few seconds. – Twister Mar 12 '12 at 10:56
Is the configuration of buffer sizes adjustable on the server? Can you watch buffer sizes on the client? Do you have access to the client's source? Have you run netstat -apnc to watch the buffer sizes? Did you try to increase the buffer sizes in the kernel to see what happens? – Karlson Mar 12 '12 at 12:42
Yes they are, and are already set to the max value of the server (i believe they can't be bigger than the net.ipv4.tcp_* properties, right ?) For netstat -apnc it does not give me the buffers sizes, that why I looked at ss. For the kernel I am not root on the server, and the IT teams here are pretty stubborn. I need to be sure of what happens before I ask them to change the values... And yes I have access to the client source, and my investigation on the client confirm that the disconnection comes from the server. – Twister Mar 12 '12 at 13:05
netstat -apnc gives you total size of send and receive queues on Linux. If the server sets the buffer to maximum available and you are still saturating maybe you need higher buffer settings at the OS level – Karlson Mar 12 '12 at 13:45
show 3 more comments

1 Answer

up vote 0 down vote

sk_forward_alloc is the forward allocated memory which is the total memory currently available in the socket's quota.

sk_wmem_queued is the amount of memory used by the socket send buffer queued in the transmit queue and are either not yet sent out or not yet acknowledged.

You can learn more about TCP Memory Management in chapter 9 of TCP/IP Architecture, Design and Implementation in Linux By Sameer Seth, M. Ajaykumar Venkatesulu

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.