From the Knowledge Base:

Question: Can I use this switch to switch 5.1 channel analog audio instead?
Answer: No, it won't work. Levels are not matched between the inputs.

Bad luck. Seems there are re-transmission amps in there, and the video ones have a different gain than the audio.

Pioneer PDP-5020FD, Marantz SR6011
Axiom M5HP, VP160HP, QS8
Sony PS4, surround backs
-Chris