Ultimately... DOCUMENTATION. You need to know where every patch cable goes to be 100% certain you are disconnecting something someone may expect to work at some point. Just because a port is currently "down" doesn't mean someone has been using it. Also just because the counters are currently zero doesn't mean it's never been used or not going to be used again in the (near) future. (counters can be cleared, and counters reset to zero at boot)
I've worked at ISPs and large enterprises, and documentation is the key to knowing what's what. Without it, random guessing will lead to numerous bad days... customers get rather pissed when you run their DSL line to someone else, multiply assign an address range, shutdown the wrong T1 interface, etc. Enterprise wiring closets can be a real mess; without documentation, tracing dozens of cables among thousands can be a pain. (and something as simple as port descriptions count)