Client A sends a packet to B with a new port because NAT uses a new port if the destination is different.
Client A will use new port for different destination only if client A is using SYMMETRIC NAT which is the biggest problem in VOIP communication. Symmetric NAT not only translates your internal IP but also translates your port each time you connect to new destination. Rendezvous server S doesn't know that Client A has changed its port for contacting Client B.
Then B sends a packet to A's port that A used to communicate with server.
When client B sends a packet to A, that will be useless because B doesn't know that A has changed its port and the port which client B has taken from the Rendezvous server is obviously closed.
Won't A's NAT device drop the connection because A used this port for a different destination?
Exactly client A will block that traffic from B because client B has sent packet to that port of A which was meant to be opened for rendezvous server and that port is now closed. Even if that port is open it will still not accept the packets from B because that port is only open for rendezvous server. Anything from outside will be discarded.
That's why symmetric NAT is always a problem.
If either of the client was not using symmetric NAT then simply client A sends a "UDP packet" to that IP:port of B which was taken by A from the Rendezvous server earlier. Ultimately this packet will be blocked too by B because B didn't request anything from A but A still thinks that its packet has reached to B because packet was sent over UDP.
Then B will send a packet to A on that IP:Port on which A was listening to rendezvous server. But this time A has to accept the packet from B because A thinks that this a response of that request which A has sent to B earlier. Then A will send a packet to B again and B will accept it as a response. Connection is established. This is called UDP hole punching.
But how the communication will be established in the case of symmetric NAT?
The standard solution is using the TURN server which stands for traversal utility over relay NAT. Turn server will act as a middle-men between both the clients and it won't be affected even If either or both the clients are behind symmetric NAT.
Instead of TURN server clients can also use DMZ and Port forwarding. Using either of them will redirect every request and response directly to the host behind NAT.
Why can't you use STUN server?
Unfortunately, even STUN server fails to initialize session if one or both clients are behind Symmetric NAT.
