Chess,
I kept notes of the master gain level and had to adjust the gain between speakers.
This is kind of what i figured.
It's not ideal, scientific or a proper A/B test. All of these speakers however are so different that your audio memory retains them quite well.
I don't agree with drawing this conclusion at all. Results can greatly altered especially in regards to the details that are being reported, as opposed to gross results, and the dubious conclusions drawn further from them.
With an instantaneous and adjusted switcher, with a blacked transparent screen (or blindfold) over the eyes (part of the 'blind' aspects of the a/b switching tests), i've found the
specific descriptions of speakers can change dramatically from the subjective ones.
For example, speaker #1 described as having less tight bass than #2 were both described as having the same bass in a blind test, though overall the person still preferred the same speaker #1 over #2 that was chosen during the subjective run. That same person described the first speaker as having less tight bass, but when listening to #1 again after a blind switch (they incorrectly guessed it was now speaker #2) they stated that the bass sounded tighter.
How is this possible?
Unintentional, unavoidable bias.
I ran this 'experiment' twice now with a few different people including myself, obtaining subjective opinions first and moving to the blind version of the a/b switching.
A true blind test is much harder especially if you never know if you are listening to the same speaker or not after each switch!! (and yes this is still dependent on the units being compared since a 1" tweeter, 2" box computer speaker will NEVER sound anything like a M80 under any test conditions, but pairing M60 vs. M80 WOULD be a much more difficult test).
As honest and objective as one may try to be in A/B tests, i find alot of the reported details mixed which may prescribe to the different tastes of the listener. No one ever chalks it up to bias of any kind.
In an older A/B (and then blind) test i ran with Tannoy Definition speakers and the Axiom M60s, both listeners found the M60s to be bright during the subjective tests and then guessed the M60s were the more articulate speaker during the blind test and both guessed wrong! I figured for sure the M60s would be the brighter speaker, more forward sounding and in comparison, during blind tests, they actually sounded slightly muffled!
Believing that one's audio memory is good enough to retain the required information between even the few seconds it takes to switch a speaker, adjust the gain and start listening again is a big assumption. Throw in the fact that a person KNOWS a different speaker is playing and the expectation to have a different reported description is obvious. Given the circumstances, having a gain adjustable switcher is the limiter.
I've considered buying a used Parasound Halo A23, or the larger A21, for the very purpose of having an amp with independent, gain adjustable channels for A/B tests. The A23 is what i used in the past for the blind tests. It was very useful and can be had used for under $700 now.
I use a home built switcher for the instantaneous part. You can flip it back and forth as many times as quickly as you want and the listener will never know which item is playing at any time (if blindfolded). It is simple, but effective.
