Even with the sub that high, there is enough trim in the 3808 to compensate with the sub at 50%. I believe the 3808 will go to a -30 db with the LFE channel.
After digging around a bit, I'm fairly sure that on my 3808, I only have the flexibility to go down to -12 dB. Just in case anyone else is in this situation and fruitlessly searching for how to get down to -30 dB, you're not alone.
Note that I haven't updated firmware in awhile, so that could be my problem.