If you're running the sub from the low level outputs from your receiver, you should put the crossover on the sub either as high as possible, or preferably disable it completely (if it allows you to do this). The reason is that your receiver is already controlling the crossover before it sends the singal out to the sub. If you tell the sub to control it as well, it may cut even more out the the signal then the receiver intended, which could be creating the hole that you describe.
I also agree with Bridgman's idea that your sub may be out-of-phase.
