TCP Handshaking Meaning of RST

Question

I am trying to implement TCP handshaking but receive RST instead of ACK:

No.     Time           Source                Destination           Protocol Length Info
     62 24.622012890   192.168.0.147         192.168.0.41          TCP      76     51486 → 8888 [SYN] Seq=1240744644 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2095798637 TSecr=0 WS=128

Frame 62: 76 bytes on wire (608 bits), 76 bytes captured (608 bits) on interface 0
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.0.147, Dst: 192.168.0.41
Transmission Control Protocol, Src Port: 51486, Dst Port: 8888, Seq: 1240744644, Len: 0

No.     Time           Source                Destination           Protocol Length Info
     63 24.627320274   192.168.0.41          192.168.0.147         TCP      62     8888 → 51486 [SYN, ACK] Seq=4039242319 Ack=0 Win=64240 Len=0

Frame 63: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) on interface 0
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.0.41, Dst: 192.168.0.147
Transmission Control Protocol, Src Port: 8888, Dst Port: 51486, Seq: 4039242319, Ack: 0, Len: 0
VSS-Monitoring ethernet trailer, Source Port: 0

No.     Time           Source                Destination           Protocol Length Info
     64 24.627364199   192.168.0.147         192.168.0.41          TCP      56     51486 → 8888 [RST] Seq=0 Win=0 Len=0

Frame 64: 56 bytes on wire (448 bits), 56 bytes captured (448 bits) on interface 0
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.0.147, Dst: 192.168.0.41
Transmission Control Protocol, Src Port: 51486, Dst Port: 8888, Seq: 0, Len: 0

I am setting up the enc28j60 ethernet controller as a server, and have script to serve as the client. I try to send a SYN,ACK packet from the enc28j60 (an ethernet controller) with my own TCP/IP stack in response to the SYN from the client, but I always receive RST instead of ACK.

The checksum in my SYN,ACK is correct according to Wireshark. I have set up ARP.

What are some reasons why a client would send a RST?

Answer 1

There are various reasons a RST is sent. We have no idea for the particular reason you are getting a RST, and what your host implementation of TCP is doing is off-topic here.

RFC 793, Transmission Control Protocol is the definition of TCP, and you should really be familiar with everything in it if you are trying to do what you describe. It has a general explanation, but there are many other things that you should understand:

Reset Generation

As a general rule, reset (RST) must be sent whenever a segment arrives which apparently is not intended for the current connection. A reset must not be sent if it is not clear that this is the case.

There are three groups of states:

1. If the connection does not exist (CLOSED) then a reset is sent in response to any incoming segment except another reset. In particular, SYNs addressed to a non-existent connection are rejected by this means.

   If the incoming segment has an ACK field, the reset takes its sequence number from the ACK field of the segment, otherwise the reset has sequence number zero and the ACK field is set to the sum of the sequence number and segment length of the incoming segment. The connection remains in the CLOSED state.

2. If the connection is in any non-synchronized state (LISTEN, SYN-SENT, SYN-RECEIVED), and the incoming segment acknowledges something not yet sent (the segment carries an unacceptable ACK), or if an incoming segment has a security level or compartment which does not exactly match the level and compartment requested for the connection, a reset is sent.

   If our SYN has not been acknowledged and the precedence level of the incoming segment is higher than the precedence level requested then either raise the local precedence level (if allowed by the user and the system) or send a reset; or if the precedence level of the incoming segment is lower than the precedence level requested then continue as if the precedence matched exactly (if the remote TCP cannot raise the precedence level to match ours this will be detected in the next segment it sends, and the connection will be terminated then). If our SYN has been acknowledged (perhaps in this incoming segment) the precedence level of the incoming segment must match the local precedence level exactly, if it does not a reset must be sent.

   If the incoming segment has an ACK field, the reset takes its sequence number from the ACK field of the segment, otherwise the reset has sequence number zero and the ACK field is set to the sum of the sequence number and segment length of the incoming segment. The connection remains in the same state.

3. If the connection is in a synchronized state (ESTABLISHED, FIN-WAIT-1, FIN-WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT), any unacceptable segment (out of window sequence number or unacceptible acknowledgment number) must elicit only an empty acknowledgment segment containing the current send-sequence number and an acknowledgment indicating the next sequence number expected to be received, and the connection remains in the same state.

   If an incoming segment has a security level, or compartment, or precedence which does not exactly match the level, and compartment, and precedence requested for the connection,a reset is sent and connection goes to the CLOSED state. The reset takes its sequence number from the ACK field of the incoming segment.

Reset Processing

In all states except SYN-SENT, all reset (RST) segments are validated by checking their SEQ-fields. A reset is valid if its sequence number is in the window. In the SYN-SENT state (a RST received in response to an initial SYN), the RST is acceptable if the ACK field acknowledges the SYN.

The receiver of a RST first validates it, then changes state. If the receiver was in the LISTEN state, it ignores it. If the receiver was in SYN-RECEIVED state and had previously been in the LISTEN state, then the receiver returns to the LISTEN state, otherwise the receiver aborts the connection and goes to the CLOSED state. If the receiver was in any other state, it aborts the connection and advises the user and goes to the CLOSED state.