Would you buy a product which has only five star reviews?
I recently wanted to buy a new ring light. I’m no expert in ring lights, so I was looking at ratings and reviews. Then it dawned on me: Perhaps a perfect string of five star reviews is a sign of a systematic effort to massage the numbers and game the system. Could I trust this information?
If you want to judge the quality of information, the obvious approach would be to ask an expert.
Yet, experts have knowledge which is domain specific: If you ask a marketing expert to provide the best strategies to grow your business, she will provide a marketing solution.
Furthermore, experts have spent an enormous of time and effort to actually become experts in their field. They have fallen in love with the subject. This often makes them unable to contemplate that much of what they believe in, simply might not be true. Would any AI expert tell you that AI is an overrated hype?
Therefore, the best way to judge the quality of expert advice is to ask the expert a better question: What needs to happen, so that you no longer believe in your own advice?
If she doesn’t provide a solid answer, you’re dealing with a severe case of mental myopia. Look elsewhere to break the spell of five star reviews to obtain objective information.
Which are the critical areas where you need to challenge your experts with a better question?