The M5 is the best bookshelf choice you have. It's not as sensitive as the M3v4 and M22v4 but is higher fidelity than the M3 and has better bass response than the M22 (even when crossed to a sub). I've never heard the M22v4 but I'd expect the M5 to have better highs too because the mid is crossed lower on the M5.
The relatively low sensitivity of the M5 demands high power particularly for music. The extra 30W of power from the 4500 is a drop in the bucket for your application. To get 102dB peaks when you're in the dining room, you will need to pair the M5s with an ADA-1500.
I love my 160v4 but I'm also only 13 feet away compared to your 20+ feet. At the peak levels you need, a 180 is more appropriate. If you cross it at 80Hz, I expect you won't need the HP drivers. I do expect you to need a third ADA-1500 channel to drive whatever center you decide on.
I strongly urge you to trade in your QS8 for QS10. You can drive them off the Denon.
