Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Dec 2003
Posts: 2,021 Likes: 1
connoisseur
|
OP
connoisseur
Joined: Dec 2003
Posts: 2,021 Likes: 1 |
Here is an example of why I personally have a problem with the reviewing industry - it's dismissal of blind listening tests. Their reasoning is the A/B test does not allow enough time to notice subtle differences.
Fine - then set it up where you have a 1 month, 3 month or 6 month blind test. All one needs to do is have the will to do it. But, again, it doesn't fit the narrative.
Rich - I have some funny stories about financial industry guys. Maybe sometime over drinks.
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Jan 2007
Posts: 10,490 Likes: 116
shareholder in the making
|
shareholder in the making
Joined: Jan 2007
Posts: 10,490 Likes: 116 |
Like Rich, I am always skeptical - about everything! I always read the written word and compare with vendor and reviewer measurements. Many times that puts the reviewer's description into context. Other times I get confused because the two don't mesh. Bottom line is I always need data I can trust or I just procrastinate and it doesn't bother me. I learned how to do that in a Decision Science course. I also learned you can't judge a decision by the outcome but rather on the data you had when you made the decision and the process, tools and people you leveraged. So even though I am a teen living in an adult body, I am wise beyond my teen years. And humble also.
House of the Rising Sone Out in the mid or far field Dedicated mid-woofers are over-rated
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Sep 2009
Posts: 1,902 Likes: 101
connoisseur
|
connoisseur
Joined: Sep 2009
Posts: 1,902 Likes: 101 |
procrastination ... I've perfected that one ...
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Jan 2007
Posts: 10,490 Likes: 116
shareholder in the making
|
shareholder in the making
Joined: Jan 2007
Posts: 10,490 Likes: 116 |
Congrats. You are like Merkel, Eisenhower and Franklin.
House of the Rising Sone Out in the mid or far field Dedicated mid-woofers are over-rated
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Mar 2001
Posts: 1,387 Likes: 8
President connoisseur
|
President connoisseur
Joined: Mar 2001
Posts: 1,387 Likes: 8 |
This is an interesting topic, thanks to Craig for bringing it up. Back in the beginning days of Axiom, Sound Canada (which later became Sound&Vision Canada) did their speaker testing at the NRC and it included a full set of anechoic measurements and double-blind listen testing. As it turned out this was the peak for reviews based in science. The reality is that speaker testing involves a lot of expensive gear and a lot of time. To do a full set of amplitude response curves requires an anechoic chamber and enough sweeps to get a total radiated sound power curve, that is expensive and time consuming and you are really just getting started. To do proper double-blind listen tests you will either need a shuffler, like Harmon has, or you need to repeat all the tests with each product in each position to weed-out the position error from the results. Then there is the testing to find out where the upper limits of the output exist and what happens when you reach them. Since this sort of testing involves destroying a few speakers along the way it is quite impractical to do as part of a review. Given all this it is not surprising that most “professional reviews” are really just “professional opinions”, and in many cases just “opinions” .
Last edited by Ian; 10/28/19 08:35 AM.
Ian Colquhoun President & Chief Engineer
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Dec 2003
Posts: 2,021 Likes: 1
connoisseur
|
OP
connoisseur
Joined: Dec 2003
Posts: 2,021 Likes: 1 |
Of course the double blind, ABX test is the purest form of a test. But - let's explore options that would make current testing much better. We should have a standard as follows:
1. The reviewer has a reference system. This reference system doesn't need to include $500,000 worth of gear, but it should be high quality.
2. All components should be hidden behind a speaker cloth style barrier to prevent being able to see the component.
3. When a new component is introduced into the system, someone else sets it up. The reviewer has NO idea which component was swapped out, and what was swapped in. All he/she knows is a change was made.
Listening is done, and results posted, before the person is allowed to know the component. This method is very easy with speakers and amps.
Pre-amps and sources are more challenging, but still possible.
The issue is - none of the magazines is even willing to TRY. They tell us that one HAS to have the visual cues in order to do a proper test.
As Ian said - now it's just an opinion, and it's based on price, company name, personal bias and MAYBE some sound quality.
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Jun 2006
Posts: 533
aficionado
|
aficionado
Joined: Jun 2006
Posts: 533 |
. .. "As Ian said - now it's just an opinion, and it's based on price, company name, personal bias and MAYBE some sound quality." . . .
AMEN !!!
jc
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Jan 2007
Posts: 10,490 Likes: 116
shareholder in the making
|
shareholder in the making
Joined: Jan 2007
Posts: 10,490 Likes: 116 |
How do you judge soundstage amongst multiple sets of speakers?
I've dropped non-Axiom speakers right where my M100s are located. Imaging/soundstage was terrible. I moved them further back and closer together and although they were no M100s, the imaging vastly improved. So did bass.
How do you reconcile that in double-blind testing? You'd need to know where to place the equipment under test before-hand I suppose.
Last edited by Mojo; 10/28/19 01:42 PM.
House of the Rising Sone Out in the mid or far field Dedicated mid-woofers are over-rated
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Dec 2003
Posts: 2,021 Likes: 1
connoisseur
|
OP
connoisseur
Joined: Dec 2003
Posts: 2,021 Likes: 1 |
Guys ... let's forget the ABX model, as it is not a realistic way to test a speaker or other component in one's home, which is where reviewers do the test.
Let's say we want a blind test on speakers. All that needs to be done is to have someone spend some time to get the best location for speakers in the listening room.
This isn't that hard to do. We audiophiles do it every time we buy speakers.
Then, without the reviewer having any contact with the person who set it up, he spends as much time as he wants listening to the speakers.
Let's call them ACME speakers during the review.
He does a complete write up, having NO idea what the speakers are. The review is published.
The next month issue will then have the reveal.
This would allow the reviewer all the time he wants, and an honest, unbiased review posted.
Everyone wants to complicate this, and it's really simple.
|
|
|
Re: Who reviews the reviewers in the Audio Industry?
|
Joined: Sep 2009
Posts: 1,902 Likes: 101
connoisseur
|
connoisseur
Joined: Sep 2009
Posts: 1,902 Likes: 101 |
The audio magazine industry has been weeded out by natural selection ... the survivors are the ones who support their advertisers ...
This will slowly change as the cost of getting information out to the public has come down significantly. There are a lot of audio people who value honesty and reputation over money.
|
|
|
Forums16
Topics24,948
Posts442,497
Members15,618
|
Most Online2,082 Jan 22nd, 2020
|
|
0 members (),
184
guests, and
4
robots. |
Key:
Admin,
Global Mod,
Mod
|
|
|
|