If this is the case - and it jibes with my first thoughts.

If the goal is to produce a smooth response curve over the audible frequency range, and a cabinet can be designed to produce that smooth curve, even with a cheaper driver, then why ever use the more expensive drivers that I see on speakers liek B&W or Swan? Are those drivers easier to design a cabinet around?

It just seems that with a little bit of engineering, a sow's ear can really be made into a silk purse. Am I wrong in believing this, or do the materials/construction, etc. of the drivers influence/interact/create the "audio waves" that differently?