andrew@gadsby.me.uk [H390-MVS]
2018-07-15 19:35:49 UTC
I've just spent some time diagnosing a problem with the use of sockdev when running Hercules MVX tk4- on Ubuntu (kernel 4.4.0) in a cloud image. I suspect the problem is likely to impact Hercules running on other versions of LINUX over time. I'm sharing my workaround in the hope that it helps anyone else having problems getting the sockdev device working e.g. for 3505 and print devices.
The problem presented as external processes such as Hercrdr and Hercprt could not connect to any sockdev(s) defined in the conf/tk4-.cnf . After tracing this through the problem appeared to be that the LINUX kernel did not pass the socket connection to Hercules: a netstat -a showed the connection to the socket e.g. 3505, as hanging in SYN sent and dmesg showed messages relating to "
kernel: possible SYN flooding on port 3505. It appears that there is a relationship when "ufw" is enabled, default tcp settings in the later versions of LINUX kernels and evolution in the listen() system call [more on that later]. The workaround that fixed this problem for me was to enable SYN Cookies for the running image:
echo "1" > /proc/sys/net/ipv4/tcp_syncookies
However, this is only a workaround. I believe that the correct solution requires a change to the two uses of listen() in sockdev.c
if (0 || bind (sd, (struct sockaddr*) &sin, sizeof(sin)) == -1 || listen (sd, 0) == -1
The second argument to listen being the backlog, i.e. number of connections that can be pending at any point in time. Reading the manual page, and history, for listen implies that the use of 0 may be problematic and is likely to be the cause of the socket hanging during the connection.
I'm going to do some work with rebuilding Hercules changing the backlog from 0 to, at least, 1. I cannot see any real downside to allowing multiple outstanding connections as I believe that Hercules will only service one connection at a time. Obviously, I need to check this change doesn't break anything else.
I think the reason I've encountered these problems is that I'm using a standard LAMP image hosted by Digital Ocean and by default "ufw" is enabled whereas my normal private LINUX image isn't locked down.
Andy
***@gadsby.me.uk
The problem presented as external processes such as Hercrdr and Hercprt could not connect to any sockdev(s) defined in the conf/tk4-.cnf . After tracing this through the problem appeared to be that the LINUX kernel did not pass the socket connection to Hercules: a netstat -a showed the connection to the socket e.g. 3505, as hanging in SYN sent and dmesg showed messages relating to "
kernel: possible SYN flooding on port 3505. It appears that there is a relationship when "ufw" is enabled, default tcp settings in the later versions of LINUX kernels and evolution in the listen() system call [more on that later]. The workaround that fixed this problem for me was to enable SYN Cookies for the running image:
echo "1" > /proc/sys/net/ipv4/tcp_syncookies
However, this is only a workaround. I believe that the correct solution requires a change to the two uses of listen() in sockdev.c
if (0 || bind (sd, (struct sockaddr*) &sin, sizeof(sin)) == -1 || listen (sd, 0) == -1
The second argument to listen being the backlog, i.e. number of connections that can be pending at any point in time. Reading the manual page, and history, for listen implies that the use of 0 may be problematic and is likely to be the cause of the socket hanging during the connection.
I'm going to do some work with rebuilding Hercules changing the backlog from 0 to, at least, 1. I cannot see any real downside to allowing multiple outstanding connections as I believe that Hercules will only service one connection at a time. Obviously, I need to check this change doesn't break anything else.
I think the reason I've encountered these problems is that I'm using a standard LAMP image hosted by Digital Ocean and by default "ufw" is enabled whereas my normal private LINUX image isn't locked down.
Andy
***@gadsby.me.uk