It’s not so much a matter of spending time processing unwanted signal, but that unwanted traffic is “noise” that reduces your signal to noise ratio. Signal to noise ratio (SNR) is more important than absolute received power, and really determines the maximum transmission rate that you can reliably decode signal.
The unwanted traffic (i.e. noise) causes errors in the decoding of your desired signal, causing retransmissions. So your throughput drops off, not because your receiver is “too busy” decoding the unwanted noise (you can only decode one signal at a time), but because the noise causes decoding errors.
In theory, a lower sensitivity will reduce noise, but at the cost of not being able to decode weaker signals. But even if your receiver isn’t activated by the noise, the noise can still cause distortion and errors in the received signal, especially when the signal is marginal.
Since 802.11 uses collision avoidance, at a certain noise threshold, the AP or station won’t transmit because it considers the channel “in use.” It will wait until the channel is idle before transmitting. So this also reduces throughput, much like not being able to say something with a group of friends because one person won’t stop talking.
Also remember that the receiver’s sensitivity is measured at the center of the channel, and drops off as you move higher or lower. So the receiver is less sensitive to interference as you move away from the center of the channel.