Per JohnK, that type of prismatic disharmony is well-researched and known to cause audio signals to develop tiny but audible distortions, adversely affecting audio quality.
I think he recommended 1950s era tube amps to address this properly.
But behind the walls, I've coated the speaker wires with vasoline, then wrapped them in tin-foil. This should counter-balance any harmonic abnormalities caused by color imbalances.
And if that fails, I've purchased a large quantity of ear plugs for those people that can't live with it.