Excellent argument. I had a similar experience when I went shopping for a new receiver two years ago. I compared a Denon 2802 to an Arcam AVR100. Granted, the audition wasn't done with an SPL meter, but volumes were adjusted up and down for comparisons. The Arcam won the audition, and my business, in a no-contest. I had read a single review in Home Theater magazine on the Arcam, and multiple reviews on the Denon. I don't think my expectation was great enough going into the audition to explain the differences I heard.
I believe there are significant differences between electronic components that are compatible with a scientific viewpoint. Noise floor/background hiss is a perfect example, as is attack and decay. Neither would show up on a frequency response graph (which is done with test tones), but both have easily appreciated audible differences.
