I've read several artical about it, but can't seem to understand this. What sort of routing algorithm is typically used in the fat-tree topology?
It depends if your tree is layer 2 or layer 3. Either way, you need a protocol that provides equal-cost multi-path (ECMP), so you can use parallel paths from source to destination.
Without it, the largest cluster = 1,280 nodes. It helps to perform static load splitting among data flows. It Leads to over subscription for simple communication patterns. Routing table entries grows multiplicatively with number of paths, cost ++, lookup latency ++.