I have an 11-year old son and a 7-year old daughter and have been pretty lucky(so far!). They don't bother the speakers. I never remove the grills when they are around so they don't really know there are drivers to poke or prod. When I used to have Vandersteen 1b's one of them got bumped, and being mounted to the Vanderstands, which have spikes in a tripod configuration to give the speaker a tilted back stance, caused it to fall diagonally into the corner of the room, marring the top oak cap. Ever since going with stand mounted speakers I've never had a problem.
