Update:

Regarding the more crisp sound when in "Stereo Direct", I switched the equalization from Parametric or PEQ to Graphic or GEQ. Now the mains sound Identical when I switch from 2 channel stereo to stereo direct. It makes perfect sense though. What the above step does is remove the equalization YPAO performed. That's the reason why the "optimizer" light stayed on even after manually configuring the speakers. IMO it sounds better in GEQ mode than PEQ. I had to turn the sub down a bit but I'm happier with the overall sound.

Thanks again for the setup tips.