Your viewpoint is well beleived in audio circles. However, there are more factors involved in differing sound than simply the crossover. I am not going to debate which sounds better, but will point out that cabinet volume, driver choice, crossover points, etc will make different speaker sizes sound different.
In actuality, most speakers of 3+ way desing have a single crossover. But there are multiple output points designed and used from the crossover. Think of a speaker as a very inefficient motor, which wastes most of your amplifier power! By allowing multiple drivers to work in certain areas (as decided by the crossover), each driver can work a little better in it's particular area. Crossovers can and do introduce other issues and problems as well. Designers can offset certain problems in other areas of the design - multiple drivers, etc.
