Thursday, March 09, 2017

Never Tell a Conservative the Odds



I'm quoted extensively in a Taki's Magazine piece by David Cole regarding the allegations by Trump and others that some of the recently reported hate crimes done against Jews are in fact, the "reverse" (I termed these "false flag" allegations, though as we'll see the fairness of that contention is a matter of considerable dispute). Cole reached out to me because of the Ha'aretz column I wrote on this topic, a column which dropped a day before a former Intercept reporter named Juan Thompson was arrested for 8 of the JCC bomb threats (allegedly seeking to frame his ex-girlfriend for the crime).

Cole says that he "didn't much care" for me at first -- in part due to the Ha'aretz piece, in part due to my "defense" of trigger warnings -- but he respected that I actually responded to his questions and noted that we had a pleasant and civil conversation. For my part, I had never heard of Cole until this week, but I will also say that I found our conversation perfectly pleasant and civil.

Moreover, I think the way I was portrayed in the final product was generally fair. It's always fraught for academics to speak in to the media -- we tend to speak in paragraphs, but journalists tend to quote us in sentences (see, e.g., how I'm quoted in this Forward piece on the "false flag" issue) -- so it was nice to see myself get excerpted extensively rather than in tiny blurbs. I'm actually going to reprint the transcript of our email "interview" below, because I think full context is always a good thing and I have no space limitations on the blog, but to be clear I am not criticizing Cole for not doing this (no journalist ever reprints the whole back-and-forth).

All of that said, there was one portion of our conversation which quite notably did not make it into the final product, and seems rather importantly germane given the conclusions Cole wants to draw. Here's Cole's view:
Schraub wrote an op-ed for Haaretz titled “Trump’s anti-Semitic ‘False Flag’ Allegation Is Dangerous (Or, how the ‘blame the Jews for their own victimization’ conspiracist fringe is going mainstream conservative).” Everything in that title is wrong. Trump didn’t say “false flag,” he didn’t blame Jews, and suggesting that a hate crime is a hoax is not “conspiracist fringe,” but a good bet. ....  [R]eacting with suspicion to “hate crime” stories that seem too good to be true does not indicate that one is being (as Schraub claimed) “disrespectful to the victim.” It just indicates good common sense.
We go back and forth a bit on whether "false flag" is a fair characterization, but the real interesting thing is this idea that "suggesting a hate crime is a hoax" is "a good bet" or an example of "good common sense."

Cole first extensively quotes me on the point that all reported crimes have some instances of false reports -- arson that's really insurance fraud, for example. Hate crimes are no doubt no different. But whereas our first response to a reported arson is virtually never "hold on -- maybe it's a hoax!", we see that reaction all the time in the hate crime context. And so my core argument is that we pay vastly disproportionate attention to "hoax" hate crime cases as compared to their overall incidence rate.

Cole replies by citing several anecdotes where, he contends, the initial report of a hate crime received far more attention than the subsequent revelation that it was false. And here's where things get interesting, because this is what he says my "reply" was:
The relatively few hoax cases are more likely to become news because typically, the whole reason one perpetrates a hoax is to get attention and media coverage (to gin up sympathy or smear political opponents). It’d be kind of pointless to fake a hate crime against yourself and then tell nobody about it. Whereas in the genuine cases, it is far more likely that the victim may not have any interest in drawing further attention to themselves, and so those crimes are less likely to become major stories. My suspicion is that most hoax cases end up making the news because the faux-victim’s purpose is to create a news story.
And Cole crows that this justifies why "common sense" suggests significant skepticism towards hate crime incidents reported in the media.

But here's the problem: the above quote actually was not my reply to those anecdotes -- I said that elsewhere, in response to a wholly different and hypothetical statement (again, check the transcript). Here's my actual, direct response -- a part which was not quoted at all:
You say you can find "dozens" of examples of a hoax which gets less attention than the initial report. I've also seen dozens of cases that I only heard of once it became known that it was a hoax (so the hoax finding got far more attention than the initial report). Dueling anecdotes don't tell us that much. Again, the question is proportions, and here's where it's useful to keep figures in mind.

Breitbart -- which, credibility issues aside, I think we can agree has no incentive to understate the number of hate crimes hoaxes in the US -- published an article this past summer claiming that it had found "over 100 hate crime hoaxes in the past decade" (2006 - 2015), including 20 in 2015 alone. Is that a lot? Well, the FBI uniform crime reporting tables for hate crimes found that there were 5,850 hate crimes incidents reported in 2015. So it seems (if we take Breitbart's word) in 2015, .3% of reported hate crimes were found to be hoaxes. That's tiny! More to the point, no matter how undercovered you might think hate crime hoaxes are, surely you must agree that they get more than .3% of total coverage or public attention as hate crimes more broadly? (And, eyeballing the FBI figures over the last decade, even .3% is a highball estimate, as 20 is relatively high number of hoaxes -- Breitbart's figures average closer to 10 a year -- and 5,850 was a relatively low number of reported hate crimes compared to other years). If that's right, that suggests hoaxes are overcovered as compared to the problem of hate crimes generally.
In short, my justification for saying we pay disproportionately too much attention to hoaxes isn't that media-reported cases are unrepresentative of the broader phenomenon. My justification is that hoaxes are really, really rare. Even Breitbart could only get us to a .3% hoax rate! In that context, it's neither a "good bet" nor "common sense" to assume that any given reported hate crime is an instance of a hoax (if David Cole really is in the habit of viewing 3 in 1000 bets as "good" ones, I need to get him in on my poker nights).

After I made that argument, Cole replied by saying that:
As most bias incidents (real or fake) never become national news, the statistical study I’d like to see is, of the ones that do become news, how many of those turn out to be fake. Because one might argue that it’s the publicity (and the accompanying scrutiny, including pressure on police to solve the case) that leads to the exposure of such hoaxes.
This was the statement that I was responding to in the quote Cole provides. Here he forwards two claims -- first, speculation that hate crime incidents which become news have higher rates of being hoaxes than the baseline, and second (if the first is true) a causal explanation suggesting that the baseline rate of hoaxes is in fact higher than we believe. And after noting that we had no evidence supporting the first part, I observed that even if it was true, there's an alternative (and to my mind more plausible) causal story that basically says "most hoaxes receive attention (because the hoaxer seeks it out) but many true cases don't (because real victims are diverse in how they respond)." That would suggest that the overall rate of hate crime hoaxes remains as low as we thought even if it appeared that more media-covered cases turned out to be hoaxes.

And it's worth noting that even if the conditional turned out to be correct and my causal story turned out to be correct, it still wouldn't necessarily make it a "good bet" to assume a hate crime reported in the media is actually a hoax. Let's start with the Breitbart-approved average rate of hate crime hoaxes of .3%. In fact, let's be nice and goose it a bit -- push it all the way up to .5%. Now let's say that, because hate crime hoaxers want their stories covered, it is far more likely -- we'll say four times likelier -- that those incidents will get media attention versus genuine cases. In that circumstance, it would still be the case that just 2% of media-reported hate crimes would turn out to be hoaxes. Again, a really bad bet!

In short, the issue is not whether any hate crimes claims are hoaxes. Like all criminal reports, some no doubt are, and clearly some of those instances loom large in Cole's mind. But, as the old social science adage goes, "the plural of anecdote is not data." As it turns out, based on the evidence we have hoaxes appear to be exceptionally rare as compared to the total number of reported hate crimes. The odds that a given reported hate crime is in fact a hoax are teensy-tiny! And assuming we devote even 5% of our attention to hoaxes versus the real deal, we're vastly malapportioning our deliberative attention compared to what the evidence suggests is appropriate. To the extent we want even greater focus on potential hoaxes versus genuine cases, we are neither expressing "common sense" nor "good bets". And at that point, it's worth asking ourselves exactly why our intuitions on the matter are so far off base from what the data tells us.

TRANSCRIPT (Asterisks denote new message)

* * *


Dear Mr. Schraub,

Thank you! Out of respect for your time, I’ll keep my questions brief.

1) In your op-ed, you put Trump’s use of the term “false flags” in quotation marks. Did Trump actually say that term, or were you using (for lack of a better word) “scare quotes?”

2) According to sources present at the meeting to which you referred, Trump told the attorneys general that the reported threats might have been done “to make people – or to make others – look bad.” In the case of Juan Thompson, the man arrested in connection with at least eight of those threats, wasn’t that exactly the case? Wasn’t he trying to “frame” the ex-girlfriend he’d been harassing?

3) Isn’t it true that, in the past, there have been many verified cases of hate crimes hoaxes, perpetrated by a variety of people for a variety of reasons? When you link the claim that a hate crime might be a hoax to claims of Holocaust denial or 9/11 as a Mossad covert op, are those accurate comparisons? I mean, Holocaust denial and “the Jews did 9/11” have zero evidence to back them up. But hate crimes hoaxes, they really happen, right?

I thank you in advance for your time and assistance!

* * *

1) Scare quotes. "False flag" is a term of art for an operation that is secretly done for the benefit of the group that appears to be the victim.

2) President Trump has been clear that he believes a substantial portion of the attacks are being done to make him and his allies, specifically, look bad (it seems unlikely that he was referring to making random ex-girlfriends look bad, as appears to be the motive in Thompson's case). Gov. Huckabee was more explicit on this point -- blaming, with no evidence, Jewish students for vandalism near Northwestern and specifically saying it was done to make Trump look bad -- but Trump's repeated statement that some of these attacks were "reverse" indicates his belief that a significant number of cases are the "reverse" of far-right individuals seeking to make Jews look bad, that is, Jews (or allies) seeking to make far-right individuals look bad. It is worth stressing, then, that there is no evidence I am aware of that Thompson possessed that motivation -- that is, to make Trump or the right look bad in order to gin up sympathy for Jews. Rather, he had the idiosyncratic motivation of seeking to set up his ex-girlfriend -- clearly repulsive (and, I'd add, antisemitic in its own right -- putting your personal grudges ahead of the terror you place Jewish lives in is obviously incompatible with egalitarian treatment of Jews. It is entirely fair and proper to say that Thompson propagated an antisemitic attack on Jewish community centers even as he also sought to frame his ex-girlfriend for the crime). So while these incidents do not appear to be a standard instance of far-right individuals targeting Jewish institutions, neither are they cases of what Trump termed "the reverse" -- Jewish individuals targeting their own institutions to smear the right.

3) "Many" is vague language. I know of no evidence that hoaxes of the sense under discussion comprise a significant number of the cases of hate crimes, and the FBI has indicated it does not view Thompson as a suspect in any of the other 92 bomb threats that have so far been called in. The most reliable evidence we have suggests that false claims punch above their weight -- the salience we give them vastly outstrips the proportion of times they occur (this also is why the appeal to statistics doesn't establish neutrality -- we don't actually devote attention to the small number of hoaxes in proportion to their real social frequency).

Of course, in any sort of criminal activity there are cases of false reports -- "arsons" which turn out to be insurance fraud, "robberies" which cover up carelessly losing a precious item, and so on. Hate crimes no doubt also have their share, though again that share tends to be a small one.  But when faced with a wave of arsons or robberies or other crimes we generally don't -- and shouldn't -- begin by talking of the "sometimes" when the claims are hoaxes, even though statistically we would be right some of the time. Doing so would come off as disrespectful to the victim whom (as an initial matter at least) is overwhelmingly likely to be nothing more than a straightforward victim. So when we choose shift attention away from the majority of cases that are more or less exactly what they appear to be, to the minority of cases which soothe our political priors and give warrant for disclaiming further vigilance, that is a decision that typically is less about rigorous statistical accuracy and more about who and what we value as political agents, and is properly interpreted as such.

* * *

Dear Mr. Schraub,

Thank you for your very thoughtful reply. I truly appreciate the fact that you would take the time to answer my questions so thoroughly. I have only one follow-up: You state that hate-crime hoaxes receive a disproportionately greater amount of attention than genuine hate crimes, but how do you know that to be true? In my years of covering this topic, I’ve found exactly the opposite to be true. Just to pick the first two instances that come to mind:

2002: Muslim motel owner claims that Islamophobes burned down his motel as part of a hate crime. Breathless, unquestioning front-page coverage in the L.A. Times (http://articles.latimes.com/2002/jul/29/nation/na-heber29). Later that year? Oh dang, turns out he burned down his own motel. Tiny, tiny “correction” blurb in the Times, a fraction of the attention given to the story when it was thought to be a hate crime (http://articles.latimes.com/2002/sep/12/nation/na-briefs12.4).

2015: Gay man alleges that homophobes beat, tortured, and branded him. Lengthy, breathless article in the Daily Beast (http://www.thedailybeast.com/articles/2015/06/14/they-carved-die-fag-into-his-arms.html). A month or so later? Oh dang, he made it all up. Tiny, tiny “correction” blurb in the Beast, a fraction of the attention given to the story when it was thought to be a hate crime (http://www.thedailybeast.com/cheats/2015/07/01/utah-gay-hate-crime-was-a-hoax.html?via=desktop&source=copyurl).

Now, I could show you dozens of examples like that, in which the initial “hate crime” story received massive attention, and the follow-up “it was all a hoax” story received a tiny blurb. So I have to ask, what is your evidence that hate crime hoaxes receive disproportionately more attention than real hate crimes?

* * *

You say you can find "dozens" of examples of a hoax which gets less attention than the initial report. I've also seen dozens of cases that I only heard of once it became known that it was a hoax (so the hoax finding got far more attention than the initial report). Dueling anecdotes don't tell us that much. Again, the question is proportions, and here's where it's useful to keep figures in mind.

Breitbart -- which, credibility issues aside, I think we can agree has no incentive to understate the number of hate crimes hoaxes in the US -- published an article this past summer claiming that it had found "over 100 hate crime hoaxes in the past decade" (2006 - 2015), including 20 in 2015 alone. Is that a lot? Well, the FBI uniform crime reporting tables for hate crimes found that there were 5,850 hate crimes incidents reported in 2015. So it seems (if we take Breitbart's word) in 2015, .3% of reported hate crimes were found to be hoaxes. That's tiny! More to the point, no matter how undercovered you might think hate crime hoaxes are, surely you must agree that they get more than .3% of total coverage or public attention as hate crimes more broadly? (And, eyeballing the FBI figures over the last decade, even .3% is a highball estimate, as 20 is relatively high number of hoaxes -- Breitbart's figures average closer to 10 a year -- and 5,850 was a relatively low number of reported hate crimes compared to other years). If that's right, that suggests hoaxes are overcovered as compared to the problem of hate crimes generally.

(As a side note: Thompson case is not exactly a "hoax" in the way that your above examples are. In the above cases, persons faked a criminal report in order to gin up sympathy or smear political opponents -- a classic hoax. There was no arson, there was no battery. But Thompson really did call in bomb threats, he just tried to frame someone else for the crime. The crime itself was quite real. If I rob a bank, but try to set you up as the fall guy, the bank robbery wasn't a hoax -- I'm just a massive dick in addition to being a bank robber.)

* * *

Dear Mr. Schraub,

My thanks for this very stimulating back-and-forth. I have no further questions; I just wanted to say thank you. You raise some interesting points. As most bias incidents (real or fake) never become national news, the statistical study I’d like to see is, of the ones that do become news, how many of those turn out to be fake. Because one might argue that it’s the publicity (and the accompanying scrutiny, including pressure on police to solve the case) that leads to the exposure of such hoaxes.

Thanks again for your replies.

* * *

It's possible that a higher percentage of major news hate crimes turn out to be fake (though as you allude to, right now that's a speculative hypothesis without evidence backing it up). But even were that the case, I'd suggest that a few different causal stories would better explain the phenomenon than that the baseline rate of hoaxes is really in-line with the rate of attention given to them in national media stories:

First, hoax cases may drive more media coverage because of their "man bites dog" quality. ("If a Nazi attacks a Jew, that's no story. But if a Jew attacks a Nazi -- that's a story!").

Second, and to my mind more importantly, the relatively few hoax cases are more likely to become news because typically, the whole reason one perpetrates a hoax is to get attention and media coverage (to gin up sympathy or smear political opponents). It'd be kind of pointless to fake a hate crime against yourself and then tell nobody about it. Whereas in the genuine cases, it is far more likely that the victim may not have any interest in drawing further attention themselves, and so those crimes are less likely to become major stories. My suspicion is that most hoax cases end up making the news because the faux-victim's purpose is to create a news story. In the real cases, by contrast, frequently (though not always) the victim wishes to keep their head down and so there is less coverage.

Finally I'd observe that a genuine statistical inquiry can't only look at factors which might raise the numerator. Yes, there may be additional factors which mean there are more hoax cases than have been discovered (though again, that even sites like Breitbart -- with an incentive to vastly inflate the figures -- still only get us to less than half of a percent of all reported cases suggests that the numbers really are quite small).  But there are also factors which suggests that the denominator (total reported hate crimes) also understates the actual number of hate crimes cases (the classic example is that the 2014 hate crime figures did not end up including the Kansas City JCC shootings, as for whatever reason KC police didn't report that attack to the FBI as a hate crime). If there are in fact some hoaxes which have not yet been revealed as such, there are also plenty more hate crime incidents that are never reported as such. Unless we have reason to believe that the under-discovery rate of hoaxes is much, much greater than the under-reporting rate of hate crimes, the fact of the former wouldn't materially effect the overall proportions.

No comments: