Ubuntu: What exactly is a connection through socket or what does the output of lsof tell us?



Question:

I'm streaming some files from S3 to an EC2 instance to get summary data from the headers. I use key.close(fast=True) for all the keys I get from boto.s3.bucket.Bucket.list() (python library to access S3), but pass around a reference to the keys so I can read parts of them, and this works. However, I sometimes get [Errno 24]: Too many open files.

When I run lsof | grep python I get a variety of these:

python    10573 ec2-user  399u     IPv4  59221       0t0    TCP ip-10-0-0-113.ec2.internal:59293->s3-1-w.amazonaws.com:https (CLOSE_WAIT)  python    10573 ec2-user   72u     IPv4  60910       0t0    TCP ip-10-0-0-113.ec2.internal:37952->s3-1-w.amazonaws.com:https (ESTABLISHED)  

and one of this:

python    10573 ec2-user   91u     sock    0,6       0t0  61105 can't identify protocol  

Are these webservers? What do the columns mean? I will need to research more to figure out how to close these connections in python, but do I want to close these connections? Is it smarter to make and reuse one connection, or is that not feasible? I'll figure out the python side when I understand more of what is happening, this is mostly a question for what is going on.

ulimit -a says open files maximum is 1024, and an option is simply to raise it. However, I want to make sure I'm closing the files I ought to be.


Solution:1

Are these webservers? [...] do I want to close these connections?

These are indeed connections you have made, and you should properly close at least the ones marked CLOSE_WAIT (this needs one extra step, because of the HTTPS connection). If you're doing a lot of things involving HTTPS sites, cleaning up properly should be enough to solve this problem. The can't identify protocol is probably a connection that hasn't been fully set up yet (see SF question linked above or this SO question).

Is it smarter to make and reuse one connection, or is that not feasible?

Yes, and it should be feasible. That is a programming question, though.

What exactly is a connection through socket[?]

A socket is typically defined as an endpoint for communication (in man 2 socket and man 3 socket. It depends on which particular protocol you are using.


What do the columns mean?

According to man lsof:

 An  open file may be a regular file, a directory, a block special file,   a character special file, an executing text  reference,  a  library,  a   stream  or  a  network  file  (Internet socket, NFS file or UNIX domain   socket.)  A specific file or all the files in  a  file  system  may  be   selected by path.  

The default list of columns for lsof is:

$ lsof | head -1  COMMAND     PID   TID    USER   FD      TYPE             DEVICE  SIZE/OFF       NODE NAME  

You seem to be missing the TID (or thread ID) column. COMMAND, PID and USER are self explanatory. FD is the number of the file descriptor. 399u indicates that the fd is numbered 339 (it can be found at /proc/10573/fd/339) and has both read and write access (u).

The TYPE IPv4 indicates that it is an IPv4 socket (sock for a socket of unknown domain).

The DEVICE output for normal files:

The DEVICE column tells us what device we're working on. The two numbers are called major and minor numbers. The list is well known and documented. For instance, major number 8 stands for SCSI block device. For comparison, IDE disks have a major number 3. The minor number indicates one of the 15 available partitions. Thus (8,1) tell us we're working on sda1.

(0,16), the other interesting device listed refers to unnamed, non-device mounts.

For detailed list, please see:

http://www.kernel.org/pub/linux/docs/device-list/devices.txt

And the list (actually at https://www.kernel.org/pub/linux/docs/lanana/device-list/devices-2.6.txt) is of not much use for devices with major number 0.

The NAME field, for such sockets, is:

[...]  the  local  and  remote  Internet addresses of a network  file; the local host name or IP  number  is  followed  by  a  colon  (':'),  the  port,  ``->'',  and  the two-part remote  address; IP addresses may be reported as numbers  or  names, [...]  

Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »