The shift knob on your manual transmission car acts as an outlet for vibration from the engine and transmission. Holding the shift knob continuously causes the vibration to affect the transmission internals.
TIP: Only have your hand on the shift knob when you are shifting so it can vibrate freely.