Facial recognition for fun, profit, protection - and control?


Where's the new edge of politics in our highly technologized, interconnected, and increasingly monitored age? We're always looking for it - the things we should be newly concerned about, if we want fully-empowered citizens. Surely one of them coming along will be our right not to be surveilled and measured as we move about the public realm, whether by state or corporations. 

Big data, and artificial intelligence, have taken our CCTV cameras - generally if grudgingly accepted as a means of detecting crime and protecting people - to a weird and near science-fictional level. You may have heard how a giant billboard in Piccadilly Circus, London, is going to use recognition technology to customise its displays to what it "sees" around it - which will include "the age, gender, and even the feelings of nearby pedestrians". 

The FT reports that British police are being warned about using facial recognition systems too much and too readily in their CCTV systems: 

Several UK-based companies are developing automatic facial recognition software. Viseum, a company in north London, identifies people in crowds through social media data. AnyVision, which has offices in Belfast, says it has developed facial recognition algorithms capable of handling large numbers of faces in real time at airports, in cities and at borders. Both companies declined to comment. 

Last year, Paul Wiles, the UK’s biometrics commissioner, raised concerns about the advancing technology and the police’s use of custody images — pictures recorded and stored by police forces whenever a suspect is arrested. He said the use of 20m facial images gathered by the police had “gone far beyond using them for custody purposes”.

How deeply can this go? Could we get to the level of "thought-crime", in the Orwellian sense, detectable by facial expression? The avant-garde magazine Cabinet has printed an extraordinary interview with the psychologist Paul Ekman, who runs the Human Interaction Lab at the University of California, San Francisco:

Thirty years ago, Ekman co-published the Facial Action Coding System (1978), a 500-page catalogue of 3,000 “meaningful” facial expressions that is used by organizations as varied as Pixar and the CIA. Ekman is the world authority on facial expression...He is currently working to help the Department of Homeland Security identify “expressions of immediate deadly intent” with CCTV systems, and, before he met with Cabinet in New York, he had just spent a week in Washington DC coaching government employees in the subtleties of the human face.

In an expansive conversation, Ekman shows himself and his research to be central to the persuasion and projection strategies of governments and corporations alike. His academic expertise is in identifying the tiny facial indications that show a person is consciously lying. An excerpt:

Do you think the face never lies?

No, it lies all the time. It lies more often than it tells the truth. But micro-expressions, the very fast signs of concealed emotion that occur in 1/25th of a second, never lie. Most people miss them. But we are, much to my surprise, learning that we can teach people to recognize them very quickly, even after an hour of training. People find out what I do at a party or something and they say, “Oh, you can read my mind.” I can’t read your thoughts, I can’t know what triggered your emotions, but I probably know what you’re feeling even if you don’t want me to know. Because the face reveals it in a number of ways, probably in a micro-expression that you can’t prevent. Most people won’t see it, but I will—I’ve learned how to see them. 

How has having that skill changed your life? 


Sometimes I know more than I would like to know, but you can’t turn it off. It’s like when you learn to read music, you hear music differently. You’ll never hear it like you heard it before you could read music.

My methods are now being used by everybody from Procter & Gamble to the State and Defense Departments: the primary thing people want to be able to understand using my techniques is what another person is feeling. I teach non-coercive methods of interviewing and surveillance of public places. Whether the person you’re talking to is a witness, an informant, or a perpetrator of something that’s already happened or is about to, you’re never going to get them to cooperate and open up to you unless you know how they’re feeling. 

For example, even second-generation Japanese rarely maintain eye contact with someone who’s an authority. That doesn’t mean that they’re lying; it’s just that they’re being respectful, and we teach people to be aware of those sensitivities. There are also feelings that the other person is having that they might not even know they’re having. One of the interesting things about a micro-expression is that it occurs not just with deliberate concealment, but it occurs with repression. And so you could see on someone’s face an emotion they’re not aware they’re feeling. They may not ever become aware of it, but you may be able to take a counter-move and better guard how you talk with them.

And what are your boundaries? Do you worry that your teaching might ever be abused? 

I get inquiries from the Chinese, the Iranians, and the Syrians on how to use my work, which I don’t respond to. I can’t know everything our government is doing, both because it’s too vast and because I don’t have a security classification, but, in the last three days, I’ve met people from secondary agencies who are using my work and they’re telling me they want to use it more because they feel they’re having such success with it in catching, as they put it, bad guys, people who are perpetrating crimes. 

The questions to ask here are manifold. How much hubris is there in any single academic scientist's definitive assessment of human expression? But if this became a general literacy, what kind of nightmarish, paranoid and distrustful society would be the result? How many subtle interactions would be reduced to mutually watchful expressions? 

It's human ingenuity that might well get us to this level of precision about the expressive humanity of those around us. But a wise and truly human-centred politics might start to ask: how far and no further?