Thanks for the advice! I found a piece of poplar in my wood pile that I'll paint the wall color and attach that way to spread the load. I zip tied the speaker with the bracket to it and held it up and it does significantly reduce the torsional load on the screw at the top of the plate, so it should be much easier on the drywall.

Any other advice / horror stories welcome too!