Toslink or Coaxial SPDIF - SPDIF does not have enough bandwidth to carry a full 5.1 PCM signal, so the audio track will be downgraded to 2 channels only. This is generally an undesirable result.
HDMI - A PCM track can be transmitted in full quality over any version of HDMI and delivered to the receiver for D-to-A conversion and amplification.
Multi-channel analog - In this case, the player converts the PCM to analog and sends it to the receiver for amplification. The quality of the DACs in the player will determine the final audio quality. If the disc player has inferior DACs to the receiver, an HDMI connection is preferred.
From High-Def Digest.
Blu-ray and HD DVD Audio Explained A good article for the HD sound formats.
