The Drunkard's Walk: How Randomness Rules Our Lives

We judge people and initiatives by their results, and we expect events to happen for good, understandable reason. But our clear visions of inevitability are often only illusions.

For while anyone can sit back and point to the bottom line as justification, assessing instead a person's actual knowledge and actual ability takes confidence, thought, good judgement, and, well, guts. You can't just stand up in a meeting with your colleagues and yell, "Don't fire her. She was just on the wrong end of a Bernoulli series." Nor is it likely to win you friends if you stand up and say of the gloating fellow who just sold more Toyota Camrys than anyone else in the history of the dealership, "It was just a random fluctuation.

Perception requires imagination because the data people encounter in their lives are never complete and always equivocal. For example, most people consider that the greatest evidence of an event one can obtain is to see it with their own eyes, and in a court of law little is held in more esteem than eyewitness testimony. Yet if you asked to display for a court a video of the same quality as the unprocessed data catptured on the retina of a human eye, the judge might wonder what you were tryig to put over. For one thing, the view will have a blind spot where the optic nerve attaches to the retina. Moreover, the only part of our field of vision with good resolution is a narrow area of about 1 degree of visual angle around the retina’s center, an area the width of our thumb as it looks when held at arm’s length. Outside that region, resolution drops off sharply. To compensate, we constantly move our eyes to bring the sharper region to bear on different portions of the scene we wish to observe. And so the pattern of raw data sent to the brain is a shaky, badly pixilated picture with a hole in it. Fortunately the brain processes the data, combining input from both eyes, filling in gaps on the assumption that the visual properties of neighboring locations are similar and interpolating. The result - at least until age, injury, disease, or an excess of mai tais takes its toll - is a happy human being suffering from the compelling illusion that his or her vision is sharp and clear.We also use our imagination and take shortcuts to fill gaps in patterns of nonvisual data. As with visual input, we draw conclusions and make judgments based on uncertain and incomplete information, and we conclude, when we are done analyzing the patterns, that out “picture” is clear and accurate. But is it?

The cord that tethers ability to success is both loose and elastic. It is easy to see fine qualities in successful books or to see unpublished manuscripts, inexpensive vodkas, or people struggling in any field as somehow lacking. It is easy to believe that ideas that worked were good ideas, that plans that succeeded were well designed, and that ideas and plans that did not were ill conceived. And it is easy to make heroes out of the most successful and to glance with disdain at the least. But ability does not guarantee achievement, nor is achievement proportional to ability. And so it is important to always keep in mind the other term in the equation—the role of chance…What I’ve learned, above all, is to keep marching forward because the best news is that since chance does play a role, one important factor in success is under our control: the number of at bats, the number of chances taken, the number of opportunities seized.

Another mistaken notion connected with the law of large numbers is the idea that an event is more or less likely to occur because it has or has not happened recently. The idea that the odds of an event with a fixed probability increase or decrease depending on recent occurrences of the event is called the gambler's fallacy. For example, if Kerrich landed, say, 44 heads in the first 100 tosses, the coin would not develop a bias towards the tails in order to catch up! That's what is at the root of such ideas as "her luck has run out" and "He is due." That does not happen. For what it's worth, a good streak doesn't jinx you, and a bad one, unfortunately , does not mean better luck is in store.

The first step in battling the illusion of control is to be aware of if. But even then it is difficult, once we think we see a pattern, we do not easily let go of our perception.

We all understand that genius doesn’t guarantee success, but it’s seductive to assume that success must come from genius.

We unfortunately seem to be unconsciously biased against those in the society who come out on the bottom.

Why is the human need to be in control relevant to a discussion of random patterns? Because if events are random, we are not in control, and if we are in control of events, they are not random, there is therefore a fundamental clash between our need to feel we are in control and our ability to recognize randomness. That clash is one of the principal reasons we misinterpret random events. In fact, inducing people to mistake luck for skills, or pointless actions for control, is one of the easiest enterprises a research psychologist can engage in ask people to control flashing lights by pressing a dummy button, and they will believe they are succeeding even though the lights are flashing at random. Show people a circle of lights that flash at random and tell them that by concentrating they can cause the flashing to move in clockwise direction, and they will astonish themselves with their ability to make it happen.

When we are in the grasp of illusion – or, for that matter, whenever we have a new idea – instead of searching for ways to prove our ideas wrong, we usually attempt to prove them correct. Psychologists call this the confirmation bias, and it presents a major impediment of our ability to break free from the misinterpretation of randomness.

probability is the very guide of life

Dershowitz may have felt justified in misleading the jury because, in his words, “the courtroom oath—‘to tell the truth, the whole truth and nothing but the truth’—is applicable only to witnesses. Defense attorneys, prosecutors, and judges don’t take this oath…indeed, it is fair to say the American justice system is built on a foundation of not telling the whole truth.

In his theory Perrow recognized that modern systems are made up of thousands of parts, including fallible human decision makers, which interrelate in ways that are, like Laplace´s atoms, impossible to track and anticipate individually. Yet one can bet on the fact that just as atoms executing a drunkard´s walk will eventually get somewhere, so too will accidents eventually occur. Called normal accident theory, Perrow´s doctrine describes how that happens – how accidents can occur without clear causes, without those glaring errors and incompetent villains sought by corporate or government commission.

Einstein had, for the first time connected new and measurable consequences to statistical physics. That might sound like a largely technical achievement, but on the contrary, it represented the triumph of a great principle: that much of the order we percieve in nature belies an invisible underlying disorder and hence can be understood only through the rules of randomness.

The appeal of many conspiracy theories depends on the misunderstanding of this logic. That is, it depends on confusing the probability that a series of events would happen if it were the product of a huge conspiracy with the probability that a huge conspiracy exists if a series of events occurs.

That’s why successful people in every field are almost universally members of a certain set—the set of people who don’t give up.

if events are random, we are not in control, and if we are in control of events, they are not random. There is therefore a fundamental clash between our need to feel we are in control and our ability to recognize randomness.

true randomness sometimes produces repetition,

We miss the effects of randomness in life because when we assess the world, we tend to see what we expect to see. We in effect define degree of talent by degree of success and then reinforce our feelings of causality by noting the correlation. That’s why although there is sometimes little difference in ability between a wildly successful person and one who is not as successful, there is usually a big difference in how they are viewed.

In fact, when some wedding guest inevitably complains about the seating arrangements, you might point out how long it would have taken you to consider every possibility: assuming you spent one second considering each one, it would come to more than half a million years. The unhappy guest will assume, of course, that you are being histrionic.

To understand my doctor’s error, let’s employ Bayes’s method. The first step is to define the sample space. We could include everyone who has ever taken an HIV test, but we’ll get a more accurate result if we employ a bit of additional relevant information about me and consider only heterosexual non-IV-drug-abusing white male Americans who have taken the test. (We’ll see later what kind of difference this makes.) Now that we know whom to include in the sample space, let’s classify the members of the space. Instead of boy and girl, here the relevant classes are those who tested positive and are HIV-positive (true positives), those who tested positive but are not positive (false positives), those who tested negative and are HIV-negative (true negatives), and those who tested negative but are HIV-positive (false negatives). Finally, we ask, how many people are there in each of these classes? Suppose we consider an initial population of 10,000. We can estimate, employing statistics from the Centers for Disease Control and Prevention, that in 1989 about 1 in those 10,000 heterosexual non-IV-drug-abusing white male Americans who got tested were infected with HIV.6 Assuming that the false-negative rate is near 0, that means that about 1 person out of every 10,000 will test positive due to the presence of the infection. In addition, since the rate of false positives is, as my doctor had quoted, 1 in 1,000, there will be about 10 others who are not infected with HIV but will test positive anyway. The other 9,989 of the 10,000 men in the sample space will test negative. Now let’s prune the sample space to include only those who tested positive. We end up with 10 people who are false positives and 1 true positive. In other words, only 1 in 11 people who test positive are really infected with HIV.

few people would engage in extended activity if they believed that there were a random connection between what they did and the rewards they received,”15 Lerner concluded that “for the sake of their own sanity,” people overestimate the degree to which ability can be inferred from success.

Unfortunately, in 1861, when he was forty, Buckle caught typhus while traveling in Damascus. Offered the services of a local physician, he refused because the man was French, and so he died.

The normal distribution describes the manner in which many phenomena vary around a central value that represents their most probable outcome;

But if she is allowed to move at constant speed without pausing at Zeno’s imaginary checkpoints—and why not?—then the time it takes to travel each of Zeno’s intervals is proportional to the distance covered in that interval, and so since the total distance is finite, as is the total time—and fortunately for all of us—motion is possible after all.

Annoyed that although outlawed in Rome, astrology was nevertheless alive and well, Cicero noted that at Cannae in 216 B.C., Hannibal, leading about 50,000 Carthaginian and allied troops, crushed the much larger Roman army, slaughtering more than 60,000 of its 80,000 soldiers. “Did all the Romans who fell at Cannae have the same horoscope?” Cicero asked. “Yet all had one and the same end.

The answer lies in a phenomenon called regression toward the mean. That is, in any series of random events an extraordinary event is most likely to be followed, due purely to chance, by a more ordinary one.

along with our responses to them, determine