Axiom Home Page Forums General Discussion Technical Questions Critical Listening

I'm still not 100% sure where I stand regarding listening tests. Audiojunkies.com had a link to an editorial by Robert Harley of The Absolute Sound:

Blind Listening Tests Are Flawed: An Editorial

And a more in-depth essay by him on the subject:

The Role of Critical Listening In Evaluating Audio Equipment Quality

I'm only part-way through the essay, but it seemed worth posting. If nothing else, we can collect more points and counterpoints on the topic in one thread.

PLEASE, someone correct me if I'm wrong, but his general theory is "how can you expect the results to match your hypothesis unless you give the listener the hypothesis in the first place?"

If that is indeed what he's saying and I didn't just misread or misunderstand his comments - that's so stupid it makes my brain ache.

"But Chief Clancy... how are we supposed to get the witness to identify our accused unless we tell him #4 in the lineup is our suspect?"

That's it... I quit... last one out of civilization, turn out the lights.

Bren R.

Sure Charles, more of the same sort of drivel that Bob Harley has been spouting for about 20 years in his "Complete Guide to High-End Audio" and various illusion-oriented audio mags. If the tests don't support what's "obviously" clear to them, then it's the tests that have to be at fault, not them. It doesn't work; facts don't cease to exist when they're ignored.

I guess my hangup is that knowing what the products are that you're listening to doesn't always lead to you liking the sound of the higher-end component. I'm wondering how the mind decides which one it prefers in those cases. Is it simply the more inconsistent methodology making the results more erratic?

The problem is that when you know the product(s) being used and if you already have a preconcieved notion of one of the said products, then you will be relating/comparing all tests back to that one (or more) item(s) you already know, creating a biased test either in favor or not in favor of the known product.

So let's say someone is told that the components have been switched but nothing was in fact switched, and they decide they prefer one over the other, even though it's the same thing. Is this trick okay, or is it accepted that the listeners have to know when different components are in use? I've never read too far into how listening tests are done.

In a double-blind test, ideally the listener will never know whether the same or a different piece of equipment is being used to reproduce the sound. If you're doing a psychological experiment, you can, of course, tell them you're switching components and not switch them. In this way, you could gather data about how much that affects their actual perception.

Echoing Bren's comments... it seems that Harley's conclusion is that, 'if the test is to determine whether an error is audible, and given that the error exists, if no error is detected, then the test mut be faulty.'

It seems that this totally misses the point. If the purpose is to detect the anomaly, then perhaps double blind testing isn't the way to go. But if the purpose is to determine if the anomoly is detectable, thendouble blind testing may be an apporpriate means of doing this.

The purpose, it seemed to me, of the test was to determine if the compressed codec proposed for use would be detectably different (by the audiophile, let alone the commoon man) from the original, uncompressed sound. The test determined that the error was indetectable - i.e., good enough for broadcast standards.

If the test was aimed at challenging audiophiles to find and locate an error, then perhaps it might make sense to tell them which sound was compromised and permit them an opportunity to first listen to the unadulterated signal, then listen to the error laden one and try to tell where the differences lie.

The first test is a practical one, for use in the real world, whereas the second test seems academic, at best.

I think that Harley has really drank the coolaid on this one. His example to "prove" the case - that monoblocks, tube amps and a $99 receiver all sound the same in a double blind test - does nothing more than to reveal HIS obvious bias.

I like how he concedes that the double-blind test has been proven useful in medical testing but it must be suspect because it has amassed a pile of data that contradicts what he believes to be true.

It's a classic battle empiricism (the idea that we can observe the world and evaluate those observations in relation to facts) and post-modernism (which holds that facts are fluid and elusive, so that we should focus only on our observational claims).

Here's the thing.

You can criticize anything you want about any particular double-blind test: The amount of time a person is listening, whether the audio material is familiar, whatever.

Fine. Make a double-blind test that addresses those problems.

Harley can write for pages and pages (and pages and pages), and not a single thing he writes (that I've come across) addresses the primary point of double-blind tests: you know, the double-blind part.

At the extreme, his arguments against "objectivism" could be taken to mean that no one should listen to his opinions, either. If there is no objective viewpoint, then there's simply no point accepting his viewpoint, either. It's a common problem of post-modern flavored thinking.