Despite the ridiculous ‘70’s fashion, the scene is still the epitome of cool.

It resonates not just because of Dirty Harry’s verve under pressure, but because we recognize that luck plays a large role in the fate of our hopes and dreams. Sometimes it seems like we can do no wrong, other times we just can’t get a break.

And that’s what makes luck so hard to deal with. We can’t count on it, but we can’t ignore it either. No matter how long or hard we struggle and strive, our destiny will often be decided by a chance encounter, a missed call or a fruitful trend. There are, however, some sensible ways of thinking about and dealing with luck.

“Hot hands”

Very much related to luck is the idea of streaks. In basketball, the phenomenon is known as ‘hot hands.” If a player make three or four shots in a row, he is said to be ‘hot” and will therefore be more likely to sink more baskets. This is widely accepted and both teams will adapt their strategies to a player who is hot.

The underlying assumption, of course, is that making three or four baskets in a row is unusual, not random. In a 1985 study, Thomas Gilovich of Cornell decided to test that assumption and found that the streaks fell within the range of what you would expect to see randomly. In other words, “hot streaks” are nothing more than an urban myth.

Gilovich’s work has been replicated many times since and the phenomenon has generally become known as the clustering illusion. We tend to see random groupings as systemic and go looking for a cause. When there is no explanation at hand, we attribute the grouping to a “lucky streak” or a “hot night.”

What’s interesting about “hot hands” is how strongly people cling to it. There is even a website dedicated to dispelling the myth, all to no avail. Sportscasters, coaches and players all still cling to it, no matter what the evidence is to the contrary.


Searching for Patterns

The clustering illusion brings up an interesting point. Our brains are wired to find patterns, not calculate statistics. Each one of our neurons is connected to thousands of others creating chain reactions in response to what we perceive. A familiar smell, as in the case of Proust’s Madeleines, can unleash a torrent of memories.

This isn’t always bad. In fact, it’s very much related to the phenomenon of chunking, which is associated with expertise in a specific domain. Chess masters, for instance, are able to memorize entire chessboards not because they have superior abilities of recall, but because they become familiar with groupings.

Children are able to catch a ball not because they are able to perform instantaneous differential equations, but through pattern recognition. Robots, on the other hand, with can massive calculations, but can’t catch a ball or avoid ordinary household obstacles. Our brains are, in computer parlance, massively parrallel.

So, in a very real sense, the concept of luck is a glitch in our software. Pattern recognition helped us survive for millions of years, allowing us to find food, evade enemies and perceive danger. In our modern world, however, it can sometimes lead us astray.


Monkeys flipping coins

None of this, however means that luck doesn’t exist. Some people win the lottery, others get killed by a stray bullet. Luck, in a very real sense, is the effect randomness has on our everyday lives. In his book, The Matthew Effect, author Daniel Rigney shows that short term luck can have long term consequences.

Take the example of 100 monkeys flipping coins, doubling their money for each time they get “heads.” A few will be, on average, much more successful and become rich very quickly. The problem arises when luck comes to be seen as a property of a particular monkey rather than as a byproduct of randomness.

The distinction becomes important when we start considering luck as a performance attribute. When it is not monkeys, but Wall Street traders flipping coins, the random beneficiaries come to be considered special, are promoted, invited to join boards and then go on to bid up the price of vacation property and fancy art.


The “20 Mile March”

In his new book, Great by Choice, author Jim Collins addresses the problem of luck and finds a solution he calls the “20 Mile March.”

The name came from two polar explorers, one of whom, Robert F. Scott went as far as he could on good days and stayed in his tent on bad days. The other, Roald Amundsen, made a point of marching 20 miles everyday, no matter what the conditions. Scott perished on the journey while Amundsen reached the South Pole first and returned home safely.

Collins extends the same principle to the business world. He points out that high performing companies, which he calls 10Xers, follow Amundsen’s practice of struggling to keep pace in bad times and holding back in good times. These firms greatly outperformed comparison companies in similar industries.

Interestingly, Collins also researched the luck the 10Xers had on their journeys and found that while luck certainly played a role in both success and failure, the winners and losers had roughly equal amounts of it. The 10Xers, however, were able to weather the storms and capitalize when fortune smiled on them.


Dealing with luck

We all have luck, both good and bad. No amount of planning or skill can immunize us from the effects of an essentially random universe. We do, however, have the power to choose how we deal with luck.

Here are two principles I have found helpful over the years:

Bad Luck – Play Defense: As Bill Clinton liked to say, “When you’re in a hole, the first thing to do is stop digging.” Our natural response to bad luck is to swing for the fences. That’s usually a mistake. Taking risks when you’re most vulnerable only courts disaster. Eventually, the law of averages will kick in and your luck will turn.

So the best way to deal with bad luck is to make sure it doesn’t get any worse. If you can survive, you can thrive.

Good Luck – Build a Reserve: The more common mistake is to try to push good luck to far. We are naturally inclined to believe in lucky streaks, so when we find ourselves on the positive side of randomness, we want to make the most of it.

The much wiser course is to pull back a bit, build up your reserve and ensure that a stroke of bad luck won’t take you down. Collins’ 10Xers were able to thrive in bad times because their competitors overshot and were weakened.

So go ahead and believe in luck. From a certain perspective, it is very real. Just don’t depend on it.

Greg Satell is a U.S.-based independent media analyst. You can read his blog entries at http://www.digitaltonto.com