1

Imagine such a situation - TCP connection(NFS4 or database) between client(Linux) and virtual IP hosted on a server(Linux). Server fails and VIP is taken over by another server(using ARP).

The client is not aware of that and thinks that TCP connection is still open. The client keeps sending re-transmits(5-8 of them) to to a secondary server, and it ignores them. It takes about 3 minutes, will the client gives up, closes failed TCP connection and tries to open a new one.

What is the best mechanism to let client fail over onto the new server as fast as possible?

1
  • If possible, you can minimize the failover by having a mechanism that syncs socket state between the primary and secondary server. But questions about servers and applications are off topic here. You could try asking this question on Server Fault.
    – Ron Trunk
    Apr 26 at 16:47
2

The client keeps sending re-transmits(5-8 of them) to to a secondary server, and it ignores them.

That would be incorrect. If the failover server does not have a TCP connection with the sending host, TCP will send a RST upon receiving a segment in a non-existent connection, killing the connection on the host, which will need to establish a new connection with the failover server.

RFC 793, Transmission Control Protocol is the definition of TCP, and you should really be familiar with everything in it if you are trying to do what you describe. It has a general explanation, but there are many other things that you should understand:

Reset Generation

As a general rule, reset (RST) must be sent whenever a segment arrives which apparently is not intended for the current connection. A reset must not be sent if it is not clear that this is the case.

There are three groups of states:

  1. If the connection does not exist (CLOSED) then a reset is sent in response to any incoming segment except another reset. In particular, SYNs addressed to a non-existent connection are rejected by this means.

    If the incoming segment has an ACK field, the reset takes its sequence number from the ACK field of the segment, otherwise the reset has sequence number zero and the ACK field is set to the sum of the sequence number and segment length of the incoming segment. The connection remains in the CLOSED state.

  2. If the connection is in any non-synchronized state (LISTEN, SYN-SENT, SYN-RECEIVED), and the incoming segment acknowledges something not yet sent (the segment carries an unacceptable ACK), or if an incoming segment has a security level or compartment which does not exactly match the level and compartment requested for the connection, a reset is sent.

    If our SYN has not been acknowledged and the precedence level of the incoming segment is higher than the precedence level requested then either raise the local precedence level (if allowed by the user and the system) or send a reset; or if the precedence level of the incoming segment is lower than the precedence level requested then continue as if the precedence matched exactly (if the remote TCP cannot raise the precedence level to match ours this will be detected in the next segment it sends, and the connection will be terminated then). If our SYN has been acknowledged (perhaps in this incoming segment) the precedence level of the incoming segment must match the local precedence level exactly, if it does not a reset must be sent.

    If the incoming segment has an ACK field, the reset takes its sequence number from the ACK field of the segment, otherwise the reset has sequence number zero and the ACK field is set to the sum of the sequence number and segment length of the incoming segment. The connection remains in the same state.

  3. If the connection is in a synchronized state (ESTABLISHED, FIN-WAIT-1, FIN-WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT), any unacceptable segment (out of window sequence number or unacceptible acknowledgment number) must elicit only an empty acknowledgment segment containing the current send-sequence number and an acknowledgment indicating the next sequence number expected to be received, and the connection remains in the same state.

    If an incoming segment has a security level, or compartment, or precedence which does not exactly match the level, and compartment, and precedence requested for the connection,a reset is sent and connection goes to the CLOSED state. The reset takes its sequence number from the ACK field of the incoming segment.

Reset Processing

In all states except SYN-SENT, all reset (RST) segments are validated by checking their SEQ-fields. A reset is valid if its sequence number is in the window. In the SYN-SENT state (a RST received in response to an initial SYN), the RST is acceptable if the ACK field acknowledges the SYN.

The receiver of a RST first validates it, then changes state. If the receiver was in the LISTEN state, it ignores it. If the receiver was in SYN-RECEIVED state and had previously been in the LISTEN state, then the receiver returns to the LISTEN state, otherwise the receiver aborts the connection and goes to the CLOSED state. If the receiver was in any other state, it aborts the connection and advises the user and goes to the CLOSED state.

Not the answer you're looking for? Browse other questions tagged or ask your own question.