Saturday, February 20, 2016

The Exaggeration of Underreporting (a digression)

It occurs to me that I should explain why SAFE13 didn't impress me in regards to the "underreporting" problem. This is the chart from the SAFE13 study that Michael Brown tweeted as support for the claim that physical harassment is underreported in science:

To many people, this chart paints a "troubling picture" (as Miriam Kramer put it about the similar study of the workplace climate in astronomy). It's not hard to see why. The second and third column (from the left) represent answers to these two questions:

“Have you ever personally experienced inappropriate or sexual remarks, comments about physical beauty, cognitive sex differences, or other jokes, at a field site? (If you have had more than one experience, the most notable to you).”

“Have you ever experienced physical sexual harassment, unwanted sexual contact, or sexual contact in which you could not or did not give consent or felt it would be unsafe to fight back or not give your consent at a field site? (If you have had more than one experience, the most notable to you).”

Here's how the authors of the study describe the results:

A majority (64%, N = 423/658) of all survey respondents, stated that they had personally experienced sexual harassment: i.e. inappropriate or sexual remarks, comments about physical beauty, cognitive sex differences, or other such jokes. Over 20% of respondents reported that they had personally experienced sexual assault: i.e. physical sexual harassment, unwanted sexual contact, or sexual contact in which they could not or did not give consent, or felt it would be unsafe to fight back or not give consent (N = 140/644, 21.7%).

As I read the chart, then, there are 658 dots in the very left colum, 423 in the next one, and 140 in the next. The fifth column represents the 37 respondents who reported the physical contact and the sixth represents the 7 respondents who felt they got a satisfactory outcome from their reporting.

It is neither surprising nor distressing that more people will experience unwanted sexual comments than will experience unwanted sexual contact. Nor is it surprising or distressing that more people will experience something than will report it. Presumably, if a man steals a kiss and the woman pushes him away, this would put a red dot in the third column. If she leaves it at that, and he gets the message and does nothing further, the incident will go unreported. (No red dot in the fifth column.) I think we would all expect this to be a very common way of dealing with unwanted physical contact and inappropriate remarks, i.e., adults working things out among themselves. A certain amount of "underreporting" in this sense is going to be normal.

It's also desirable, since it would put a completely unreasonable burden on administrators to formally adjudicate the appropriateness of all 140/658 instances of unwanted physical contact, let alone the 423 instances of sexual remarks. The more people can establish their boundaries among themselves the better. If this idea seems outrageous, it may be because of the language that the authors of the study use to summarize the survey responses.

Like the CSWA study, which I have criticized on this blog before, I'm uncomfortable with the rather strong wording the authors use to describe the survey data. "Over 20% of respondents reported that they had personally experienced sexual assault," they say, meaning: "physical sexual harassment, unwanted sexual contact, or sexual contact in which they could not or did not give consent, or felt it would be unsafe to fight back or not give consent." Similarly, they interpret "inappropriate or sexual remarks, comments about physical beauty, cognitive sex differences, or other such jokes" simply as "sexual harassment".

What this means (unless someone can convince me otherwise)* is that a misguided attempt to kiss a coworker at a field-site party can be counted as an "assault", even when retracted and apologized for in the moment, and that a dirty joke told in mixed company is counted as "harassment", perhaps even if no one took offense or if the point of the joke was simply misunderstood. In both cases, the non-reporting of the incident would also constitute a case of under-reporting. Needless to say, I think this exaggerates the problem.

Katie Hinde is very familiar, it seems, with this criticism of her "operational definition" of harassment.* Indeed, she would probably characterize the imagined situations I describe above as just more "contorted scenarios that quite likely [are] not harassment but could fall within SAFE’s questions about inappropriate remarks" presented as an "[attempt] to disprove [her] with a single counter-example." But my scenarios are of course merely examples that can be multiplied endlessly, and the point is only that we don't know how many of the respondents were referring to such situations when they answered "yes" to SAFE's questions.

My point is not that we therefore know that harassment doesn't take place. I'm saying this is a poorly designed study, at least if our interest is (as Michael's was) in the underreporting of sexual harassment in the sciences. (Katie has said on Twitter that I've misunderstood the purpose of the study, but I have to say I don't understand what the purpose of such a survey could be if not to gauge the size of a problem.)

"The worst way to measure sexual misconduct," she rightly says, "is to query 'Have you been sexually harassed?' or 'Have you been sexually assaulted?' People’s working definitions of these experiences are expected to fall far short of the legal definitions." But I'm not at all convinced that the questions I quoted above do anything other than err in the other direction. She dismisses this concern as overly pedantic (and not befitting the usual "squishy" imprecision of biologists, as far as I understand her argument) but the issue seems easy to clarify, precisely in terms of the legal definition.

Katie cites the US EEOC website but conveniently does not cite its parent page, which contains this sentence: "To be unlawful, the conduct must create a work environment that would be intimidating, hostile, or offensive to reasonable people." She does quote the part stating that "the law doesn’t prohibit simple teasing, offhand comments, or isolated incidents that are not very serious," and requiring that, to break the law, the behavior must be "so frequent or severe that it creates a hostile or offensive work environment or when it results in an adverse employment decision." I'm not convinced that SAFE13's questions are sensitive to this at all, nor that it was based on "multiple, behaviorally specific questions" as the NRDI report she cites suggests.

To meet SAFE13's standard, it seems, the conduct need only create a work environment that would be intimidating, hostile, or offensive to any person, even a single unreasonable person. Indeed, it's altogether likely that SAFE's sample is biased by the self-selection of precisely this kind of person and Katie's "principles of community" seem designed specifically to protect them. If such people not report their offense, we are even to count this an instance of "underreported" sexual harassment in SAFE13.

This is not a community that I would feel comfortable in. I will explain why in the next post.

________
*She refers to "conversation with colleagues ... scholarly publications, media reports, and the comment threads on face-melting MRA Reddits" but links only to the last one, which I'm not going to bother to click through to for the moment. If someone could point me in the direction of the discussion among scholars, I'd be grateful.

No comments: