The problem is, if you're not a hacker, you can't tell who the good hackers are. A similar problem explains why American cars are so ugly. I call it the design paradox. You might think that you could make your products beautiful just by hiring a great designer to design them. But if you yourself don't have good taste, how are you going to recognize a good designer? By definition you can't tell from his portfolio. And you can't go by the awards he's won or the jobs he's had, because in design, as in most fields, those tend to be driven by fashion and schmoozing, with actual ability a distant third. There's no way around it: you can't manage a process intended to produce beautiful things without knowing what beautiful is. American cars are ugly because American car companies are run by people with bad taste.
I don’t know how much I believe this claim about cars, but I certainly believe it about software. A startup without a technical cofounder will usually produce bad software, because someone without software engineering skills does not know how to recognize such skills in someone else. The world is full of bad-to-mediocre “software engineers” who do not produce good software. If you don’t already know a fair bit about software engineering, you will not be able to distinguish them from the people who really know what they’re doing.
Same with user interface design. I’ve worked with a CEO who was good at UI; both the process and the results were visibly superior to others I’ve worked with. But if you don’t already know what good UI design looks like, you’d have no idea - good design is largely invisible.
Yudkowsky makes the case that the same applies to security: you can’t build a secure product with novel requirements without having a security expert as a founder. The world is full of “security experts” who do not, in fact, produce secure systems - I’ve met such people. (I believe they mostly make money by helping companies visibly pretend to have made a real effort at security, which is useful in the event of a lawsuit.) If you don’t already know a fair bit about security, you will not be able to distinguish such people from the people who really know what they’re doing.
But to really drive home the point, we need to go back to 1774.
As the American Revolution was heating up, a wave of smallpox was raging on the other side of the Atlantic. An English dairy farmer named Benjamin Jesty was concerned for his wife and children. He was not concerned for himself, though - he had previously contracted cowpox. Cowpox was contracted by milking infected cows, and was well known among dairy farmers to convey immunity against smallpox.
Unfortunately, neither Jesty’s wife nor his two children had any such advantage. When smallpox began to pop up in Dorset, Jesty decided to take drastic action. He took his family to a nearby farm with a cowpox-infected cow, scratched their arms, and wiped pus from the infected cow on the scratches. Over the next few days, their arms grew somewhat inflamed and they suffered the mild symptoms of cowpox - but it quickly passed. As the wave of smallpox passed through the town, none of the three were infected. Throughout the rest of their lives, through multiple waves of smallpox, they were immune.
The same technique would be popularized twenty years later by Edward Jenner, marking the first vaccine and the beginning of modern medicine.
The same wave of smallpox which ran across England in 1774 also made its way across Europe. In May, it reached Louis XV, King of France. Despite the wealth of a major government and the talents of Europe’s most respected doctors, Louis XV died of smallpox on May 10, 1774.
The point: there is knowledge for which money cannot substitute. Even if Louis XV had offered a large monetary bounty for ways to immunize himself against the pox, he would have had no way to distinguish Benjamin Jesty from the endless crowd of snake-oil sellers and faith healers and humoral balancers. Indeed, top medical “experts” of the time would likely have warned him away from Jesty.
The general pattern:
- Take a field in which it’s hard for non-experts to judge performance
- Add lots of people who claim to be experts (and may even believe that themselves)
- Result: someone who is not already an expert will not be able to buy good performance, even if they throw lots of money at the problem
Now, presumably we can get around this problem by investing the time and effort to become an expert, right? Nope! Where there are snake-oil salesmen, there will also be people offering to teach their secret snake-oil recipe, so that you too can become a master snake-oil maker.
So… what can we do?
The cheapest first step is to do some basic reading on a few different viewpoints and think things through for yourself. Simply reading the “correct horse battery staple” xkcd will be sufficient to recognize a surprising number of really bad “security experts”. It probably won’t get you to a level where you can distinguish the best from the middling - I don’t think I can currently distinguish the best from the middling security experts. But it’s a start.
More generally: it’s often easier to tell which of multiple supposed experts is correct, than to figure everything out from first principles yourself. Besides looking at the object-level product, this often involves looking at incentives in the broader system - see e.g. Inadequate Equilibria. Two specific incentive-based heuristics:
- Skin in the game is a good sign - Jesty wanted to save his own family, for instance.
- Decoupling from external monetary incentives is useful - in other words, look for hobbyists. People at a classic car meetup or a track day will probably have better taste in car design than the J.D. Powers award.
That said, remember the main message: there is no full substitute for being an expert yourself. Heuristics about incentives can help, but they’re leaky filters at best.
Which brings us to the ultimate solution: try it yourself. Spend time in the field, practicing the relevant skills first-hand; see both what works and what makes sense. Collect data; run trials. See what other people suggest and test those things yourself. Directly study which things actually produce good results.
I try to make an isolated example to show how difficult the problem of "Knowing" and "Persuasion" is in the personal domain.
Let us look at this question:
It is established scientifically that non-ionizing electromagnetic radiations are not carcinogenic (do not lead to cancer or other disease), for example, electromagnetic radiation from power lines and cell phones.
There was a research decades ago that linked living near power lines to blood cancer. This research however later was shown to be invalid.
I am an electrical engineer by education and my wife is a physics graduate so both of us should be able to follow the reasoning fairly well, as long as it is related to electromagnetic and not biology.
My wife, however, opposes to buy a house within a visible distance of high voltage power lines and whatever I did I could not persuade her that this is harmless. She is not alone as I heard this from many highly educated engineers around me. They just prefer to stay on the safe side which might be wrong. They always combine it with the argument of 'there are things that we don't know'. This is even reflected in the market price of houses near power lines.
Now how can you prove that living near power lines is safe and more importantly persuade someone else? Can you run your own tests? Can you follow the health of people living near power lines? If your full-time job is not that then this would be impossible.
When I google the question I get a boxed search result from a "snake oil seller" feeding on the fears of people:
So much praise should go to the power of page-rank and other algorithms in google search to bring this up. I am certain that the majority of people won't go further than the boxed results.
Now, this seems like a trivial and not so important example. But we are just following the same line of reasoning for many more decisions in everyday life.
Well, doesn't that lead to an opportunity for those that are confident in the fact that there is no risk. Buy the house for cheap, likely get a windfall profit in years as more people come to accept that the facts are no risk and they will have increased options for investing the price differential so able to retires earlier or with greater resources?
(I understand that doesn't solve the problem with your wife but....)