I am new to networking and was studying about TCP and UDP. One question came to my mind that what will happen if a sender uses TCP and receiver uses UDP for the communication.
Exactly nothing happens. When host A sends a TCP SYN (initiate connect) to host B where the indicated TCP port isn't open/listening, the connection attempt fails. The destination host might actively reject the connection due to the TCP port being closed but it might just as well just drop the TCP SYN attempt (or it might get dropped by a firewall).
TCP and UDP are completely distinct protocols. They have no relation with each other.
Both use the same range of ports for subaddressing - 0 through 65535, but UDP port x and TCP port x are not related to each other (although they might be used by the same service/application).