Could the problem arise from the difference in distance from the sweet spot to the top speaker and the distance from the sweet spot to the bottom speaker?

Play it loud...