This is truly weird. The optical cable is only carrying the digital image of what to play. Sounds like either a defective AVR or that the MAC's connectors are optical, but they are configured for a different protocol and cause the AVR to misinterpret what it is receiving.

See Mojo's signature