Perhaps you could fill me in a bit. The speaker is rated from 10-400.
A 400W speaker doesn't suck 400W out of your amp unless you want it to. 400W is how much power you can sink into your speaker before it starts to misbehave.
400W likely represents the start of the excursion limit of the driver. You could probably inject up to 700W (of undistorted music content) into it before you reach complete dynamic compression. Any higher than that and you are probably going to severely encroach on the thermal limits of the speaker.
Before you reach a couple of hundred watts, you'll start damaging your hearing. And if your amp can't provide undistorted power, you might even end up damaging the speaker because clipping will thermally load the speaker.
When you are listening to music, I doubt your amp is peaking out at more than 20 Watts.