(Mis)Uses of Technology

 

from the Detective-Mesmer-is-on-the-case dept

Ken Armstrong of ProPublica and Christian Sheckler of the South Bend Tribune have uncovered more pseudoscience cops rely on to put people behind bars. It's called SCAN, and it's the creation of Avinoam Sapir.

SCAN stands for Scientific Content Analysis. What's "scientific" about it is undetermined. Sapir believes it is. Sapir's background as a polygraph examiner for Israeli law enforcement apparently turned him into a "human lie detector." Sapir appears to believe his own press, as long as it's positive.

Here's how SCAN works. Or, rather, here's how SCAN operates.

1. Give the subject a pen and paper.

2. Ask the subject to write down his/her version of what happened.

3. Analyze the statement and solve the case.

Those are the steps according to Sapir's site. Law enforcement agencies around the world have spent money purchasing Sapir's SCAN products and training. The scientific breakthrough they're buying has no science backing it. According to Sapir, different words or tense changes in people's written statements can let officers know whether or not suspects are lying.

How does anyone know if SCAN works? Just ask Sapir, apparently.

SCAN works, the site says. “Analysis of statements has been found to be highly accurate and supported by a validation survey conducted in a U.S. governmental agency. In that survey, when SCAN was compared to other methods, the validity of SCAN reached above 95%,” the site says, without identifying the agency or citing or linking to any survey.

SCAN's junk science may be in wide use but it hasn't really made headlines like other forensic pseudoscience -- bite mark analysis, blue jeans analysis, etc. -- but it's no less questionable or potentially dangerous. No one runs SCAN on confessions. They only use the tool to "analyze" statements made by suspects and witnesses. These analyses are rarely introduced as evidence in court.

Good thing, too. It seems unlikely stuff like this would be considered admissible evidence.

After the person writes a statement, the SCAN investigator looks for signs of deception, analyzing, among other things, pronouns used, changes in vocabulary, what’s left out and how much of a statement is devoted to what happened before, during and after an event. Indications of truthfulness include use of the past tense, first-person singular (“I went to the store”); pronouns, such as “my,” which signal commitment; and direct denials, the best being: “I did not do it.” Signs of deception include lack of memory, spontaneous corrections and swapping one word in for another — for example, writing “kids” in one place and “children” in another.

It's not exactly handwriting analysis, which has its own problems. It's its own animal, one that's only available through Sapir. He's offered up some bizarre analysis of his own -- applying his proprietary SCAN to things like Anita Hill's statements about Clarence Thomas, Magic Johnson's disclosure that he has HIV, and James Comey's memoir. His conclusions? Anita Hill had sexual identity issues, Johnson "admitted" to being bisexual, and James Comey is a victim of child abuse.

Comey's statement on Sapir and the SCAN analysis of his memoirs?

“No comment. Never heard of the alleged tool. (And by using the word ‘never’ in conjunction with the word ‘heard,’ I mean only ‘never heard’ and not to suggest childhood trauma. Yikes.) I’m sorry about your five bucks.”

The "five bucks" refers to the amount ProPublica spent to obtain Sapir's analysis of Comey's memoirs. He also sells an analysis of the Mueller Report.

Law enforcement agencies rely on this dubious technique to solve crimes. Their reliance on Sapir's assertions SCAN actually has something to do with science is misplaced. SCAN is apparently used by the FBI (despite Comey's denial), CIA and US Army Intelligence. A government study commissioned to find which, if any, interrogation methods produced usable intelligence had this to say about SCAN:

Studies commonly cited in support of SCAN were scientifically flawed, the review said. “When all 12 SCAN criteria were used in a laboratory study, SCAN did not distinguish truth-tellers from liars above the level of chance,” the review said. The synopsis also specifically challenged two of those 12 criteria, noting: “Both gaps in memory and spontaneous corrections have been shown to be indicators of truth, contrary to what is claimed by SCAN.”

Four more studies -- conducted over a span of twenty years -- arrived at the same conclusion:

Each of those studies (in 1996, 2012, 2014 and 2015) concluded that SCAN failed to help discriminate between truthful and fabricated statements. The 2016 study found the same. Raters trained in SCAN evaluated 234 statements — 117 true, 117 false. Their results in trying to separate fact from fiction were about the same as chance.

This is what scientists think about SCAN, something Sapir says can turn normal cops into "walking polygraphs." If law enforcement experts were upfront about the dubiousness of SCAN conclusions, it wouldn't be an issue. But they're not. Much like every other pseudoscience deployed by investigators, experts testifying in court routinely overstate the certainty of their conclusions. SCAN is a coin toss, but no one presenting this analysis in court is ever going to admit this under oath.

Cops apparently believe it's a coin flip worth investing in. ProPublica's investigation found at least 40 agencies had purchased SCAN training in the last decade. Sapir's website claims agencies in 49 states utilize SCAN. Very few agencies want to talk about their SCAN expenditures and the CIA has refused to confirm or deny the existence of SCAN use records.

The agencies that do believe SCAN works really believe it works. But these are statements of faith. They have no evidence it works, only the word of its creator who says it works. And if you're spending money to train investigators to deploy SCAN analysis, you kind of have to believe it works because the alternative is knowingly throwing money down a hole. Fervently believing something is scientific just isn't good enough when people's freedom is on the line. But not good enough has been good enough for plenty of agencies and the junk science their investigations rely on for years now, and it's not going to change just because another emperor has been discovered wandering naked.