I think it is best to set the crossover on the sub to the highest setting (it may say 150 or bypass)
I have my denon 3808 set at 60 hz for m80's, 80 hz for vp150, 90 hz for QS8, and 90 hz for LFE... these were entered by hand as I am not running audyssey right now.
i have the crossover setting in 2 channel mode at 40 hz, but I often have it at 60.
Many people use 80 hz for the m80's too.
Have you tried different locations for the sub?
One thing you might try is to set the m80's to "large" and change the LFE option to "LFE+main" try this with a 40 or 60 hz crossover on the m80's. This will essentially duplicate the bass below the crossover frequency in both the sub and the mains. you will probably find it too "boomy" but it may be worth a shot. have you tried setting the trim on the ep 500 to half or full instead of flat?
Also, don't be afraid to go into the channel level adjustment and bump the sub up 2 or 3 db in the receiver.
