I'm not completely sure what you mean.
A link will have a bandwidth. Let's assume 10 Mbps ethernet. Ethernet has some overhead: a seven octet synchronization, a one octet start-of-frame delimiter, a 14 octet frame header, and a four octet frame-check-sequence. There is also a 12 octet inter-packet-gap between frames.
When your 100 octet packet is encapsulated in an ethernet frame, then it ends up as a 126 octet frame. At 10 Mbps, it takes
126/10,000,000=0.0000126 seconds to fully transmit. Your 5000 octet packet is a 5026 octet frame that takes
5026/10,000,000=0.0005026 seconds to fully transmit.
Obviously, it takes nearly 40 times longer to transmit the 5000 octet packet than it does to transmit the 100 octet packet on ethernet. The bits travel at the same rate, but you have 50 times the number of bits in the larger packet.
All this assumes that everything is on the same LAN.
Notice that the percentage of overhead is much larger on the smaller packet, so if you are transmitting a lot of data, it will take longer with smaller packets than larger packets due to the amount of overhead on the frames. That doesn't take into account the overhead on the packet headers themselves, nor the overhead on the transport layer datagrams.
Individually, larger packets take more time to transmit, but in aggregate, data may be sent faster with larger packets because the percentage of overhead is smaller with larger packets.