You are a nontechnical person and stumble upon what appear to be plans for a terrorist attack. You talk to the person about it and they say "Don't worry. is computer game".
A great exploration of this is the 30 Rock episode where Tina Fey reports her neighbor for what ends up being a plan to get on the show 'The Amazing Race'.
The problem is, not knowing any better, you feel obligated to report the activity just in case. Let someone much smarter than yourself decide what is really going on. If you say nothing and someone gets hurt, can you forgive yourself?
As a hacker, I would understand this is definitely a game. But can I really expect the same from non-technical people?
The problem is that everyone in the chain ends up buying into this. The police make a record of it 'just in case.' You end up on a no-fly list 'just in case.' Everyone in the chain is covering their ass 'just in case' this person really is a terrorist.
No one ends up willing to stick their neck out and say, "this person probably isn't a terrorist"... just in case.
This is exactly what happened in the Twitter Joke Trial:
"The airport management considered the message to be "not credible" as a threat, but contacted the police anyway."
They ended up finding something unrelated to actually protecting the public from a terrorist threat that they thought would stick after it got escalated to the point where they couldn't back down. This led to a conviction (eventually overturned on appeal).
The extreme example of "everyone in the chain" buying into an extreme form of this thinking came in the form of the "1% doctrine" articulated by then-Vice President Dick Cheney: "If there's a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response. It's not about our analysis ... It's about our response."
Much of the post-9/11 response to terrorism fits it.
"We captured a bunch of people. Some are harmless unfortunate bystanders. Some are dangerous people who are nevertheless not immune to handcuffs. Still, we'd better lock them up in harsh conditions without trial just in case."
"Iraq probably has no nuclear weapons and no means to deliver one, but we'd better invade the country just in case. We don't want to be wrong and end up with a mushroom cloud over Manhattan."
> Much of the post-9/11 response to terrorism fits it.
Naturally; the "1% Doctrine" was formulated very shortly after 9/11 and, while the quote articulating it addresses Pakistani scientists aiding al-Qaeda, the significance is that it was a critical framing mindset from the highest levels for the Bush Administration policy on terrorism.
>Shortly after September 11th 2001, I thought to myself, and now someone will turn up minor intelligence warnings of something-or-other, and then the hindsight will begin. Yes, I'm sure they had some minor warnings of an al Qaeda plot, but they probably also had minor warnings of mafia activity, nuclear material for sale, and an invasion from Mars.
>Because we don't see the cost of a general policy, we learn overly specific lessons. After September 11th, the FAA prohibited box-cutters on airplanes—as if the problem had been the failure to take this particular "obvious" precaution. We don't learn the general lesson: the cost of effective caution is very high because you must attend to problems that are not as obvious now as past problems seem in hindsight.
And it all becomes Kafkaesque and you can't get the bureaucracy to respond because '... it might reveal our methods and tactics in combating global terrorism.'
One big problem is that people take the idea of "better safe than sorry" as a way to avoid thought. Don't analyze the situation, just pick the option that appears safest after a glance!
What it should mean is that you should take both probabilities and costs into account when deciding what to do, as it's often worth doing something easy to reduce the risk of something unlikely but catastrophic.
But that doesn't mean you can just avoid thinking about it! "Often" doesn't mean "always". For example, it's worth checking the tire pressure in your car on a regular basis, because incorrect pressure could lead to a blowout at high speeds which can be most unpleasant, and checking it takes little time. But it's not worth checking the car for bombs every time you drive it, at least not for most of us, because the odds of being the victim of a car bomb for most of us are indistinguishable from zero.
A little thought reveals that this case is clearly in the second category, and thus calling the police is just a waste of public resources. But if you're incorrectly using "better safe than sorry" to mean "pick the option that appears to be safe without any analysis", you're not going to apply a little thought.
> One big problem is that people take the idea of "better safe than sorry" as a way to avoid thought.
Sure, and while things like responses to terrorism are rather dramatic examples of that, "zero tolerance policies" in general are also a place where this manifests.
no. It is a way to avoid liability, an insurance of a kind. Property owner would get his property confiscated if he knew or had reason to know about an illegal activity happening on the property. Trying to apply "thought" would expose her/him even to larger liability in case of a mistake. Thus blanket reporting of anything is the best liability minimizing strategy.
> One big problem is that people take the idea of "better safe than sorry" as a way to avoid thought. Don't analyze the situation, just pick the option that appears safest after a glance!
Which in the innocuous cases is the option that is least safe:
If you flag everything as suspicious then you waste law enforcement resources investigating nothing that could have been used to investigate something.
Increasing the false positive rate makes it easier for genuine threats to blend into the noise. If you cry wolf for every neighbor's puppy the townspeople soon stop responding.
And then you make enemies out of the communities you're unnecessarily harassing, which leads to "Fuck the Police" culture where people won't even report obvious and serious threats because they don't want to attract a bunch of unthinking jackboots who might just as soon arrest victims and bystanders as the perpetrators.
Fantastic point that cannot be understated. When thinking about this I assumed the authorities would run some quick sanity check and dismiss it. While that probably does happen, we often see cases where it has not worked that way.
It takes only the barest application of common sense to realize that an ICBM attack is not going to be planned on a whiteboard in plain view in some random apartment. It does not actually appear to be plans for a terrorist attack, because terrorists don't have ICBMs.
1. How do I know they don't have ICBMs? Bill O'Reilly hasn't told me that yet.
2. Or, we don't have ICBM's yet. Hoping that part of the plan will work itself out. Step 1: Collect underpants ... Step 3: Profit!
2. Maybe this terrorist is just working from home and trying to understand the larger plan. Or he could be brainstorming for a future powerpoint presentation on the subject.
This example illustrates the very slippery slope we find ourselves on today. The problem is that it's really no ones business unless and until I break a law. I shouldn't have to justify may legal actions to anyone, regardless of what they do and don't understand.
Sometimes it is because, even if the chance it's real is slim, the stakes would be incredibly high. One time I thought I witnessed a kidnapping, a woman shackled and gagged in the back of a car. I contacted the police and they very thoroughly and professionally pursued it. Afterwards they told me the two people involved were engaged in sexual roleplay. She wasn't being kidnapped, it seemed kind of unusual (you had to be there) but I would have been irresponsible in not reporting it.
On the other hand, they could have been SWATed, and it probably cost my city government five digits. I don't have an answer.
But the people higher up the chain don't get to see the whiteboards and so can't easily dismiss this as planning a computer game. Unless the agents took photos?
The priori of a dude planning an ICBM attack on whiteboard in an apt is so low that unless you have very very strong evidences that it exists (and words by random people without photos certainly don't count as one), for all practical purposes, it isn't true.
And I sincerely do hope that if such a plan truly exists somewhere, the power that be have better evidences/ discovery than being informed by letting agents.
Unfortunately, the probability that someone is able to understand this logic is lowered considerably once you realize that they're working in a job where they're required to act on such threats. Most people who are capable of such calculations will not find such jobs fulfilling, and those who do will find themselves constantly second-guessed by bosses who don't have that skill. It's a profession that selects against Bayesian reasoning.
Humans are -really- bad at intuitively calculating risk, and I have yet to encounter any sort of formal education in the US that teaches us how to properly calculate it.
You are a nontechnical person and stumble upon what appear to be plans for a terrorist attack. You talk to the person about it and they say "Don't worry. is computer game".
A great exploration of this is the 30 Rock episode where Tina Fey reports her neighbor for what ends up being a plan to get on the show 'The Amazing Race'.
The problem is, not knowing any better, you feel obligated to report the activity just in case. Let someone much smarter than yourself decide what is really going on. If you say nothing and someone gets hurt, can you forgive yourself?
As a hacker, I would understand this is definitely a game. But can I really expect the same from non-technical people?