You're rapidly approaching the point at which 1) I can't help you and 2) you need to just give it a go.
So as I understand it, it would sit in between the AVR and the sub. LFE preout from the AVR to the phono in of the BFD, the phono out from the BFD to RCA line IN of the sub.
What about XLR from the BFD to XLR of the sub, any advantage or disadvantage to that?
Well, I think the BFD accommodates that. My sub doesn't have XLR input, so the point was moot for me. You need an EE who cares to answer that.
Then I guess I need run Audyssey just so the distance and ch level are set, then disable Audyssey, equ the sub with REW and then run Audyssey again. Is that correct? When running Audyssey the second time, wouldn't that affect the settings that I come up with REW?
I think you're supposed to do it in the other order. Disable Audyssey, then run the REW/BFD stuff. THEN run Audyssey. Yeah, Audyssey is not going to "leave it alone", but I don't know that the granularity of the Audyssey filters/adjustments is finer than the BFD at those frequencies. You know, the BFD might be redundant to Audyssey. I don't care enough to test it both ways. Sorry.
do you equ the bass from the sub only or sub + mains?
I ran REW including the mains (M60's in my case) because I like to cross them at 60Hz. YMMV. Gotta try it a few ways.