Results tagged “probability”

I've often written on my blog about how poorly the human brain intuitively understands probability. My very basic understanding from statistics courses (and a vague interest) is the reason I don't buy lottery tickets. Yes, your chances of winning a jackpot if you don't play are zero, but your chances of winning if you do play are so close to zero it makes no difference. I might do better wandering around town hoping to find a few million dollars lost in a bag on the street (which has happened, here in Vancouver).

People who know me are tired of my saying that if I ever do buy a lottery ticket, my numbers will be consecutive: 1, 2, 3, 4, 5, 6 for the Lotto 6/49, for example. Those are just as likely to be a winning combination as anything else. Here's proof: not just one, but four people in New South Wales, Australia just won a jackpot using the numbers 1 through 10 as their picks, getting more than $2 million Australian each.

I think it was my friend Karen who pointed out that choosing consecutive numbers (or any other set that might be easy to think of, or might have some meaning to people) isn't the smartest strategy. Why? Because, as for those Australians, it's more likely that several players will choose them, and that you'll have to split any winnings you do get. That's because, unlike the numbers that win, many numbers that players pick are non-random. Going with a random set of numbers (the same ones or different ones, whatever) each draw would bring the best likelihood—still trivially small—of keeping it all yourself, or splitting with fewer co-winners. Nevertheless, I'd take $2 million and the good story.

But if I have a few bucks to spend, I'll probably still get myself a burger.

Only once in the past three and a half years, since I found out I had metastatic colorectal cancer that had spread to my lungs, have any of my doctors said anything about how long I might live. At the beginning, my oncologist Dr. Kennecke noted that the median survival for patients with my condition is two years after diagnosis.

That was, I repeat, three and a half years ago. You might think that he predicted I had about two years to live, and was simply (and happily) wrong. But that's not even what he was saying. Because he used the word median, he meant that two years after diagnosis, half of patients with metastatic colon cancer are still alive. Therefore, in 2007, my chances of living more than two years were about 50% (assuming I was a typical patient—more on that below).

And he was right about that, since I'm still here. However, if I'd died within two years instead, he'd still have been right, since I would have been in the other 50%. You can see why doctors like using medians for survival prognoses!

Woe to the prognosticators

According to Slate, doctors are very, very reluctant to make any predictions about how long a specific patient will live, mostly because they're notoriously bad at it, unless the patient is pretty much at death's door—within days or hours of the end. In part that's also because they don't learn how:

[In] major medical textbooks that have been used by medical students (and practicing physicians) for decades [...] the relative percentage of space for each disease entity devoted to prognosis diminished with each subsequent edition, often to a paragraph or less. [Doctors] focus almost exclusively on the ever-expanding sciences of diagnosis and treatment, leaving prognosis almost entirely to the side.

But it's also difficult to predict correctly, especially for someone like me. A typical colon cancer patient is over 50 (maybe decades over), often with a family history of the disease, and perhaps other health problems that go with advancing age. I got the disease in my mid-30s, with no family history of it—and, later testing showed, no known genetic predisposition either—as well a relatively healthy body otherwise, despite having type 1 diabetes since 1991.

So my personal chance of survival two years past diagnosis was probably higher than the median, even if no one knew by how much. Plus my cancer team has felt it worthwhile to try all sorts of semi-experimental treatments, for which more typical patients might not have been eligible. And they've been willing to subject me to fairly high doses of chemotherapy—of which I'll get more on Monday—that I'm guessing might kill someone in more fragile health.

Time is not infinite

However, all that doesn't mean I'm likely to live an 80-year lifespan like your typical newborn Canadian in the 21st century. I've seen the CT scans, and I've watched my cancer progress slowly but relentlessly over the past few years. It's never been in remission, not once.

When my doctors talk to me about my treatments, they never use the word cure anymore. When they see a treatment as successful, that means it has slowed or stopped or maybe slightly reversed the growth of my tumours for a few months, or perhaps a year. Success means buying me time, extra months or perhaps years, but almost certainly not decades—unless, during those extra months, some remarkable new treatment becomes available, and it works for me.

I can hope for that, but I can't expect it. I'm 41 now. My own estimate, not made scientifically, but as an educated guess, is that I'll be pretty lucky if I reach 45. I'll be absolutely astonished if I celebrate my 50th birthday in 2019, and that's what I tell people now. (I was a little bit surprised to reach 40 last year.) The chances are pretty good that—in addition to my wife Air and my daughters and almost all of my friends—my parents, my aunts and uncles, and even our pup Lucy will outlive me. In other words, I'm living in dog years.

Like that initial median estimate, those are all probabilities, not certainties. There's no guarantee that my cancer will kill me within the decade, but being reasonable and realistic means I have to treat that as the most likely result, and live (and plan) accordingly. That's not easy to do, especially for a procrastinator like me, but there it is.

By the way, most of you will have to do the same eventually, but with any luck not until your 70s or 80s. My time is probably briefer than most, and I don't like that. I'm not okay with it. But I can live with it. Woof woof.

One of my repeated themes here over the years is how genuinely lousy the human brain is at intuitively understanding probability and statistics. Two articles this week had me thinking about it again.

The first was Clive Thompson's latest opinion piece in Wired, "Why We Should Learn the Language of Data," where he argues for significantly more education about stats and probability in school, and in general, because:

If you don't understand statistics, you don't know what's going on—and you can't tell when you're being lied to.

Climate change? The changing state of the economy? Vaccination? Political polls? Gambling? Disease? Making decisions about any of them requires some understanding of how likelihoods and big groups of numbers interact in the world. "Statistics," Thompson writes, "is the new grammar."

The second article explains a key example. At the NPR Planet Money blog (incidentally, the Planet Money podcast is endlessly fascinating, the only one clever enough to get me interested in listening to business stories several times a week), Jacob Goldstein describes why people place bad bets on horse races.

After exhaustive statistical analyses (alas, this stuff isn't easy), economists Erik Snowberg and Justin Wolfers have figured out that even regular bettors at the track simply misperceive how bad their bets are, especially when wagering on long shots—those outcomes that are particularly unlikely, but pay off big if you win, because:

...people overestimate the probability of very rare events. "We're dreadful at perceiving the difference between a tiny probability and a small probability."

In our heads, extremely unlikely things (being in a commercial jet crash, for instance) seem just as probable, or even more probable, than simply somewhat unlikely things (being in a car crash on the way to the airport). That has us make funny decisions. For instance, on occasion couples (parents of young children, perhaps) choose to fly on separate planes so that, in the rare event that a plane crashes, one of them survives. But they both take the same car to the airport—as well as during much of the rest of their lives—which is far, far more likely to kill them both. (Though still not all that likely.)

Unfortunately, so much of probability is counterintuitive that I'm not sure how well we can educate ourselves about it for regular day-to-day decision-making. Even bringing along our iPhones, I don't think we should be using them to make statistical calculations before every outing or every meal. Besides, we could be so distracted by the little screens that we step out into traffic without noticing.

Our minds are required be good at filtering out irrelevancies, so we're not overwhelmed by everything going on around us. But the modern world has changed what's relevant, both to our daily lives and to our long-term interests. The same big brains that helped us make it that way now oblige us to think more carefully about what we do, and why we do it.

Read the archived comments on this entry...