SunRPC reconnection

Introduction

We tested SunRPC reconnections when the transport was UDP, TCP and Unix domain sockets. The basic methodology was simply to start matching server and client processes, and then examine what happens to the client when we kill, wait and restart the server. We also looked at the client under strace.

Downloads

UDP

As expected, when the server is killed we see errors on the client calls:

int *ret = reconn_echo_1 (&serial, cl);

if (!ret)
  clnt_perror (cl, "warning");       // This warning triggers
warning: RPC: Unable to receive; errno = Connection refused
warning: RPC: Unable to receive; errno = Connection refused
warning: RPC: Unable to receive; errno = Connection refused
warning: RPC: Unable to receive; errno = Connection refused

As soon as the server starts up again, calls succeed. Calls which were "missed" while the server was down are not retried.

$ ./reconn_client 
echo: sent 1, recv 1
echo: sent 2, recv 2
echo: sent 3, recv 3
warning: RPC: Unable to receive; errno = Connection refused
warning: RPC: Unable to receive; errno = Connection refused
echo: sent 6, recv 6
echo: sent 7, recv 7
echo: sent 8, recv 8

SunRPC over UDP apparently offers a way to retry failed RPC calls, but we could not work out how to enable this feature.

TCP

SunRPC over TCP does not reconnect automatically when the server is killed and then comes back up (we rather expected that the client would do this, but in fact it does not).

It would be possible to enable this by catching the first failed call and reconnecting. Something like this:

int *ret = reconn_echo_1 (&serial, cl);

if (!ret) {
  clnt_perror (cl, "warning");
  cl = do_reconnection ();
}

If the first failed call is ignored and a second call attempted, then the second call causes the process to die with SIGPIPE. To avoid this, ignore SIGPIPE (ie. signal (SIGPIPE, SIG_IGN) or the equivalent sigaction(3)), and then you will get safe errors like:

warning: RPC: Unable to receive; errno = Connection reset by peer
warning: RPC: Unable to send; errno = Broken pipe
warning: RPC: Unable to send; errno = Broken pipe

However this doesn't change the fact that the client will not automatically reconnect - you must do it manually.

Unix domain socket

As far as we can tell Unix domain sockets act exactly like TCP. You must manually reconnect.


rjones AT redhat DOT com

$Id: index.html,v 1.1 2007/02/21 17:54:30 rjones Exp $