Axiom Home Page Forums General Discussion Home Theater Want New CD Player

yeah but in the real world if one can get one's hands on a $2,000 player and a $150 player and if one thinks the $2,000 player sounds better, what should one do? if the person wants to enjoy his/her sound, it would seem to be unwise to tell this person to buy the cheaper player simply because scientists said there is no difference. those experiments have nothing to do with said person's bias. those biases are indeed real. and the normal person isn't likely to put each player into a perfectly controlled environment that will prove to them they are the same. so, in the real world, the average person is always left to their biases and what/which devices make them feel better. always.

"which devices make them feel better": Yes. Therefore buy the more expensive cd player? Not always. If I heard those two cd players, and felt that the $2000 sounded better, I would have to seriously ask myself if it sounded $1850 better. For that sum, I could buy a projector, or significantly increase my CD/DVD collection, or build myself a new top of the line PC. Now if the person in question had 2k to drop on a cd player, and was fine with that, then sure, buy it. There's no thought required. But in terms of "feeling better" about a purchase, as you said, it can often be the right decision to buy the moderately priced option and free up money for other things.

Well said, ringmir. To use terms from economics, always remember to factor opportunity cost into your purchases, and always be mindful of diminishing returns. The sweet spot of the cost/performance ratio will be different for every person.

To some, a $2000 CD may indeed be worth their money, even though scientific results indicate (non-psychological) audio benefits to be negligible. To others, verifiable audio benefits are a very large factor in justifying increased expenditure. If science says a $200 CD player plays just as well as a $2000 CD player in a real-world test, that $2000 player suddenly doesn't seem worth the money.

Science speaks more to some people than it does to others. And that's the crust of this stale loaf of bread.

Plinko:

I don't agree with your comment, "In double blind testing it always comes down to a human to say whether it "sounds better". This is the double blind intrinsic flaw".

How is that an intrinsic flaw? Your premise isn't clearly stated, so someone could assert whether or not it is indeed flawed.

Personally I love A/B tests (blind or not). Though when it gets soooooo nit picky then it is just plain frustrating. But sometimes it's a noticable difference. This is when it comes in handy. It's all the easier when you have someone to help you. And a second pair of ears.

Plinko,

There is no intrinsic flaw of double-blind or A/B/X testing. The point of the latter is to simply consistently hear a difference, any difference. If you correctly pick A over B seventeen times out of 20, rather than 10 out of 20, which chance or flipping a coin would predict, then you can conclude that A sounds different than B.

Of course, anyone can purchase whatever they want, and if they know they are listening to a $1500 CD player, they have a psychological expectation for it to sound better. But the point of double-blind tests is to remove the effects of variables like that, so you conceal the two CD players under comparison, equalize the output levels, and call one A, the other B. If you can detect differences reliably, beyond chance, between the two, then a difference exists and you can explore that further with more tests. In my tests, once I equalized the output levels and ran two identical discs in sync, switching instantaneously between each, I could not reliably distinguish one over the other, or identify A over B beyond what flipping a coin would yield. Nor could any of my colleagues.

This type of testing forms the basis of all medical studies of drugs, to isolate the real effects of a drug from the patient's and the experimenter's expectations, which is why they give a sugar pill to a control group and the real drug to the experimental group. No-one knows who gets the real drug and who gets the placebo (the sugar pill).

By the way, my background is in journalism and I've been writing about and participating in double-blind and A/B/X testing for 20 years, long before I joined Axiom, when I was an editor of audio-video magazines. Axiom gives me enormous freedom to write about these subjects, far more in fact than I had when I was a magazine editor, where the publishers and advertisers often impose considerable pressure.

Regards,

Alan: I would assume your new found freedom has caused you to spend far more time writing then you even imagined. Isn’t it interesting just how much others have benefited from your liberation? Editors and publishers most definitely have their place; however, it is often at the cost of getting the word (truth) out.

As you’ve already read numerous times, many of us have enjoyed the years of wisdom and experience you bring to this industry. I personally and thrilled to see someone who brings an objective and scientific view to an industry that is clearly inundated with snake oil scams, and bias reviewers who mimic independence—yet receive their salary from publishers who accept advertising revenue from the vendors they just reviewed.

ringmir, pmbuko: well said. I agree. We all decide if certain expenditures are worth it.

Alan/TK: I agree with pretty much all of your findings. I think didn't communicate well enough. What I mean about the flaws are twofold: one human can hear better or worse than another human so test results on one human won't yield the same results for another and secondly, how many people are going to go to the trouble of setting up a perfect double blind test in order to remove their biases before making a cdp purchase? Answer: None. The bias remains. Therefore, to tell someone to get the cheaper cdp because of science, is not always the wise advice. This is what I think.

In reply to:

---

one human can hear better or worse than another human so test results on one human won't yield the same results for another

---

Plinko, if you actually look at a bunch of data, then statistically speaking, the results are the same. You just need a statistically significant sample size.