I disagree. Speaker measurements are typically taken by a microphone placed one meter from the baffle. A microphone can only measure the sound that hits its diaphragm. At one meter away, pretty much all that sound is direct. At a typical listening distance, the sound of a speaker, to a human ear, is the sum of all the auditory information reaching your ear. This includes all direct and reflected (from the floor, ceiling, walls, furniture, etc) sound information.

Unless you're in an anechoic chamber, I'd say two speakers that have virtually identical response curves -- even on-axis and off-axis measurements -- might easily have distinguishable differences.