I keep my grilles on 100% of the time. Not to protect the speaker, but to protect the grilles. The safest place for the grilles is actually on the speaker.
In 1989 I was showing a buddy of mine my AX2's and he accidently sat on one of my grilles which I had placed on my bed (still living with my parents at the time). It broke, and I had to get a new one.
Ever since then I keep the grilles on the speakers, even when showing other people. Only take them off if they ask to see the driver compliment. And the I put them right back on.
Once bitten.....
