Assuming you mean to protect confidentiality of the communication at IP layer with IPsec:
How would the underlying network be able to differentiate between UDP
and TCP since they're at the transport layer.
The next header field of the ESP header tells you the type of payload.
If you use tunnel mode (which is custom for VPNs), then without the necessary keys you cannot decide what's at transport layer because the next header field will tell you just that there's a whole IP packet encapsulated.
If you use tunnel mode, then the next header field tells you the type of payload at transport layer.
Will we still have TCP and UDP when we move the IPv6(Although I see
that IPsec has been made optional for IPv6)?
TCP and UDP are agnostic to the layer-3 protocol. In fact, TCP and UDP (and SCTP and DCCP) exist also for IPv6.
What seems to puzzle you is that in IPsec tunnel (VPN) mode there is no way to inspect the content. This is supposed to happen at the tunnel end points. An organization that is worried by this loss of control should not allow IPsec that is not under it's own control.
Further reading: An Illustrated Guide to IPsec