If reference level is too loud for you, then there's no real reason to calibrate to that point. If you always listen at softer levels, then your speakers won't be calibrated to YOUR preferences. What I usually do is calibrate my speakers at the loudest level I'm comfortable listening.
The disadvantage of calibrating loud and listening soft is that the relative perceived volumes change, especially between subwoofers and speakers as the volume decreases. So if you calibrate at 85db, but normally listen at 70db, your sub will sound too soft.
