What appropriate MTU size should be used in datacenter switches and routers? is there any standard or pros and cons?
Default is 1500 but that isn't enough to push data.
should we used 9000 MTU on all datacenter switches?
Jumbo frames are non-standard, so you are at the mercy of what your devices can do. For instance, some Cisco switches can do 9000 on some interfaces, but only 3000 on others. You need to make sure that you can use the same MTU all the way through, and that any hosts can use jumbo frames. Any traffic headed toward the Internet probably shouldn't use jumbo frames.