If you look at a diagram of the anatomy of a speaker, you'll see that the dust cap covers up the "engine" at the center of the speaker, where most of the action happens. The voice coil is the piston of the speaker. If any dirt or debris get into the piston, it could cause unwanted friction, which would lead to wear and eventually breakdown.