I don't think there is any evidence they sound better without the grills. I've done tons of listening with and without and couldn't tell a difference. Having said all that, I still left the grills off in the off chance that there a subtle change. Your mind can play havoc with your happiness.

I'd say if you don't have kids, leave the grills off and enjoy what you hear.

***********
"Nothin' up my sleeve. . ." --Bullwinkle J. Moose