I was thinking that the sound waves would easily pass through the fabric and allow the traps to work as intended.

Speaker grills are made of "acoustically transparent" fabric. I'm not sure what snake oil it is that transforms regular fabric into acoustically transparent fabric, but I suspect that most light fabrics would be close enough.

M- M60s/VP150/QS8s/SVS PC-Ultra/HK630 Sit down. Shut up. Listen.