Keep in mind frequency sweeps are not (often) found in music, so the chances of regular program material exciting those same resonances are slim. Still, now that you know it's there, it'll probably bug you until you know it's gone.

You might try pulling the drivers out one by one and repositioning the wires a bit. One wire pair could be touching and buzz against each other at specific frequencies.