{"id":3279,"date":"2022-01-17T19:13:03","date_gmt":"2022-01-17T19:13:03","guid":{"rendered":"https:\/\/mailinvest.blog\/index.php\/2022\/01\/17\/new-startup-shows-how-emotion-detecting-ai-is-intrinsically-problematic\/"},"modified":"2022-01-17T19:13:03","modified_gmt":"2022-01-17T19:13:03","slug":"new-startup-shows-how-emotion-detecting-ai-is-intrinsically-problematic","status":"publish","type":"post","link":"https:\/\/mailinvest.blog\/index.php\/2022\/01\/17\/new-startup-shows-how-emotion-detecting-ai-is-intrinsically-problematic\/","title":{"rendered":"New startup shows how emotion-detecting AI is intrinsically problematic"},"content":{"rendered":"

<\/a>\n<\/p>\n

\n
\n

Did you miss a session from the Future of Work Summit? Head over to our\u00a0Future of Work Summit on-demand library<\/a>\u00a0to stream.<\/em><\/p>\n

\n<\/div>\n

In 2019, a team of researchers published a meta-review of studies claiming a person\u2019s emotion can be inferred from their facial movements. They concluded<\/a> that there\u2019s no evidence emotional state can be predicted from expression \u2013 regardless of whether a human or technology is making the determination.<\/p>\n

\u201c[Facial expressions] in question are not \u2018fingerprints\u2019 or diagnostic displays that reliably and specifically signal particular emotional states regardless of context, person, and culture,\u201d the coauthors wrote. \u201cIt is not possible to confidently infer happiness from a smile, anger from a scowl, or sadness from a frown.\u201d<\/p>\n

Alan Cowen might disagree with this assertion. An ex-Google scientist, he\u2019s the founder of Hume AI<\/a>, a new research lab and \u201cempathetic AI\u201d company emerging from stealth today. Hume claims to have developed datasets and models that \u201crespond beneficially to cues of [human] emotions,\u201d enabling customers ranging from large tech companies to startups to identify emotions from a person\u2019s facial, vocal, and verbal expressions.<\/p>\n

\u201cWhen I got into the field of emotion science, most people were studying a handful of posed emotional expressions in the lab. I wanted to use data science to understand how people really express emotion out in the world, across demographics and cultures,\u201d Cowen told VentureBeat via email. \u201cWith new computational methods, I discovered a new world of subtle and complex emotional behaviors that nobody had documented before, and pretty soon I was publishing in the top journals. That\u2019s when companies began reaching out.\u201d<\/p>\n

Hume \u2014 which has ten employees and recently raised $5 million in funding \u2014 says that it uses \u201clarge, experimentally-controlled, culturally diverse\u201d datasets from people spanning North American, Africa, Asia, and South America to train its emotion-recognizing models. But some experts dispute the idea that there\u2019s a scientific foundation for emotion-detecting algorithms, regardless of the data\u2019s representativeness.<\/p>\n

\"Solutions<\/p>\n

\u201cThe nicest interpretation I have is that these are some very well-intentioned people who, nevertheless, are ignorant enough that \u2026 it\u2019s tech causing the problem they\u2019re trying to fix,\u201d Os Keyes, an AI ethics scientist at the University of Washington, told VentureBeat via email. \u201cTheir starting product raises serious ethical questions \u2026 [It\u2019s clear that they aren\u2019t] thoughtfully treating the problem as a problem to be solved, engaging with it deeply, and considering the possibility [that they aren\u2019t] the first person to think of it.\u201d<\/p>\n

Measuring emotion with AI<\/h2>\n

Hume is one of several companies in the burgeoning \u201cemotional AI\u201d market, which includes HireVue, Entropik Technology, Emteq, Neurodata Labs, Neilson-owned Innerscope, Realeyes, and Eyeris. Entropik claims its technology, which it pitches to brands looking to measure the impact of marketing efforts, can understand emotions \u201cby facial expressions, eye gaze, voice tonality, and brainwaves.\u201d Neurodata developed a product that\u2019s being used by Russian bank Rosbank to gauge the emotion of customers calling in to customer service centers.<\/p>\n

It\u2019s not just startups that are investing in emotion AI. In 2016, Apple acquired Emotient, a San Diego firm working on AI algorithms that analyze facial expressions. Amazon\u2019s Alexa apologizes and asks for clarification when it detects frustration<\/a> in a user\u2019s voice. Speech recognition company Nuance, which Microsoft purchased in April 2021, has demoed a product for cars that analyzes driver emotions from their facial cues. And Affectiva, an MIT Media Lab spin-out that once claimed<\/a> it could detect anger or frustration in speech in 1.2 seconds, was snatched up by Swedish company Smart Eye in May.<\/p>\n

The emotion AI industry is projected to almost double in size from $19 billion in 2020 to $37.1 billion by 2026, according to Markets and Markets. Venture capitalists, eager to get in on the ground floor, have invested a combined tens of millions of dollars in companies like Affectiva<\/a>, Realeyes<\/a>, and Hume. As the Financial Times reports<\/a>, film studios such as Disney and 20th Century Fox are using it to measure reactions to upcoming shows and movies. Meanwhile, marketing firms have tested the technology to see how audiences respond to advertisements for clients like Coca-Cola and Intel.<\/p>\n

The problem is that there exist few \u2013if any \u2014 universal markers of emotion, putting the accuracy of emotion AI into question. The majority of emotion AI startups base their work on psychologist Paul Ekman\u2019s seven fundamental emotions (happiness, sadness, surprise, fear, anger, disgust, and contempt), which he proposed in the early \u201970s. But subsequent research has confirmed the common-sense notion that there are major differences in the way that people from different backgrounds express how they\u2019re feeling.<\/p>\n

Factors like context, conditioning, relationality, and cultural influence the way people respond to experiences. For example, scowling \u2014 often associated with anger \u2014 has been found to occur less than 30% of the time on the faces of angry people. The expression supposedly<\/a> universal for fear is the stereotype for a threat or anger in Malaysia. Ekman himself later showed that there are differences between how American and Japanese students react to violent films, with Japanese students adopting \u201ca completely different set of expressions\u201d if someone else is in the room \u2014 particularly an authority figure.<\/p>\n

Gender and racial biases<\/a> are a well<\/a>\u2013documented<\/a> phenomenon<\/a> in facial<\/a> analysis<\/a> algorithms<\/a>, attributable to imbalances in the datasets used to train the algorithm. Generally speaking, an AI system trained on images of lighter-skinned people will perform poorly on people whose skin tones are unfamiliar to it. This isn\u2019t the only type of bias that can crop up. Retorio, an AI hiring platform, was found to respond differently to the same candidate in different outfits, such as glasses and headscarves. And in a\u00a02020 study<\/a> from MIT, the Universitat Oberta de Catalunya in Barcelona, and the Universidad Autonoma de Madrid, researchers showed that algorithms could become biased toward certain facial expressions, like smiling, which could reduce their recognition accuracy.<\/p>\n

A separate study<\/a> by researchers at the University of Cambridge and Middle East Technical University found that at least one of the public datasets often used to train emotion AI systems contains far more Caucasian faces than Asian or Black faces. More recent research<\/a> highlights the consequences, showing that that popular vendors\u2019 emotional analysis products assign more negative emotions to Black men\u2019s faces than white men\u2019s faces.<\/p>\n

Voices, too, cover a broad range of characteristics, including those of people with disabilities, conditions like autism, and who speak in other languages and dialects such as African-American Vernacular English (AAVE). A native French speaker taking a survey in English might pause or pronounce a word with some uncertainty, which could be misconstrued by an AI system as an emotion marker.<\/p>\n

Despite the technical flaws, some companies and governments are readily adopting emotion AI to make high-stakes decisions. Employers are using it to evaluate potential employees<\/a>\u00a0by scoring them on\u00a0empathy<\/a>\u00a0or\u00a0emotional intelligence<\/a>. Schools have deployed it to monitor students\u2019 engagement in the classroom<\/a> \u2014 and even while they do classwork at home<\/a>. Emotion AI has also been used to identify \u201cdangerous people<\/a>\u201d and tested at border control stops in the U.S., Hungary, Latvia, and Greece<\/a>.<\/p>\n

Training the algorithms<\/h2>\n

To mitigate bias, Hume says that it uses \u201crandomized experiments\u201d to gather \u201ca rich array\u201d of expressions \u2014 facial and vocal \u2014 from \u201cpeople from a wide range of backgrounds.\u201d According to Cowen, the company has collected more than 1.1 million images and videos of facial expressions from over 30,000 different people in the U.S., China, Venezuela, India, South Africa, and Ethiopia, as well as more than 900,000 audio recordings from over 25,000 people voicing their emotions labeled with people\u2019s self-reported emotional experiences.<\/p>\n

Hume\u2019s dataset is smaller than Affectiva\u2019s, which Affectiva once claimed was the largest of its kind with more than 10 million people\u2019s expressions from 87 countries. But Cowen claims that Hume\u2019s data can be used to train models to measure \u201can extremely wide range of expressions,\u201d including over 28 distinct facial expressions and 25 distinct vocal expressions.<\/p>\n

\"Solutions<\/p>\n

\u201cAs interest in accessing our empathic AI models has increased, we\u2019ve been preparing to ramp up access to them at scale. Thus, we will be launching a developer platform which will provide API documentation and a playground to developers and researchers,\u201d Hume said. \u201cWe\u2019re also collecting data and training models for social interaction and conversational data, body language, and multi-modal expressions which we anticipate will just expand use cases and our customer base.\u201d<\/p>\n

Beyond Mursion, Hume says it\u2019s working with Hoomano, a startup developing software for \u201csocial robots\u201d like Softbank Robotics\u2019 Pepper, to create digital assistants that deliver better recommendations by accounting for users\u2019 emotions. Hume also claims to have partnered with researchers at Mount Sinai and the University of California, San Francisco to see whether its models can pick up on symptoms of depression and schizophrenia \u201cthat no previous methods have been able to capture.\u201d<\/p>\n

\u201cA person\u2019s emotions broadly influence their behavior, including what they are likely to attend to and click on. Consequently, AI technologies like search engines, social media algorithms, and recommendation systems are already forms of \u2019emotion AI.\u2019 There\u2019s no avoiding it. So decision-makers need to worry about how these technologies are processing and responding to cues of our emotions and affecting their users\u2019 well-being, unbeknownst to their developers.\u201d Cowen said. \u201cHume AI is providing the tools needed to ensure that technologies are designed to improve their users\u2019 well-being. Without tools to measure cues to emotion, there\u2019s no way of knowing how an AI system is processing these cues and affecting people\u2019s emotions, and no hope of designing the system to do so in a manner that is consistent with people\u2019s well-being.\u201d<\/p>\n

Setting aside the fraught nature of AI to diagnose mental illness<\/a>, Mike Cook, an AI researcher at Queen Mary University of London, says that the company\u2019s messaging feels \u201cperformative\u201d and the discourse suspect. \u201c[T]hey\u2019ve clearly gone to great pains to talk about diversity and inclusion and stuff, and I\u2019m not going to complain that people are making datasets with more geographic diversity. But it feels a bit like it was massaged by a PR agent who knew the recipe for making your company look like it cares,\u201d he said.<\/p>\n

Cowen argues that Hume is more carefully considering the applications of emotion AI than competitors by establishing The Hume Initiative, a nonprofit \u201cdedicated to regulation empathic AI.\u201d The Hume Initiative \u2014 whose ethics committee includes Taniya Mishra, the former director of AI at Affectiva \u2014 has released regulatory guidelines that Hume says it\u2019ll abide by in commercializing its technologies.<\/p>\n

The Hume Initiative\u2019s guidelines, a draft of which was shared with VentureBeat, bans applications like manipulation, deception, \u201coptimizing for reduced well-being,\u201d and \u201cunbounded\u201d emotion AI. It also lays out constraints for use cases like platforms and interfaces, health and development, and education, for example requiring educators to ensure that the output of an emotion AI model is used to give constructive \u2014 but non-evaluative \u2014 feedback.<\/p>\n

Coauthors of the guidelines include Danielle Krettek Cobb, the founder of the Google Empathy Lab<\/a>; Dacher Keltner, a professor of psychology at UC Berkeley; and Ben Bland, who chairs the IEEE<\/a> committee developing standards for emotion AI.<\/p>\n

\u201cThe Hume Initiative began by listing all of the known use cases for empathic AI. Then, they voted on the first concrete ethical guidelines. The resulting guidelines are unlike any previous approach to AI ethics in that they are concrete and enforceable. They detail the uses of empathic AI that strengthen humanity\u2019s greatest qualities of belonging, compassion, and well-being, and those that admit of unacceptable risks,\u201d Cowen said. \u201c[T]hose using Hume AI\u2019s data or AI models are required to commit to using them only in compliance with The Hume Initiative\u2019s ethical guidelines, ensuring that any applications that incorporate our technology are designed to improve people\u2019s well-being.\u201d<\/p>\n

Reasons for skepticism<\/h2>\n

Recent history is filled with examples of companies touting their internal AI ethics efforts only to have those efforts fall by the wayside \u2014 or prove to be performative and ineffectual. Google infamously dissolved<\/a> its AI ethics board just one week after forming it. Reports have described Meta\u2019s (formerly Facebook\u2019s) AI ethics team, too, as largely\u00a0toothless<\/a>.<\/p>\n

It\u2019s often referred to as \u201cethics washing<\/a>.\u201d Put simply, ethics washing is the practice of fabricating or exaggerating a company\u2019s interest in equitable AI systems that work for everyone. A textbook example for tech giants is when a company promotes \u201cAI for good\u201d initiatives with one hand while selling surveillance tech to governments and corporations with the other.<\/p>\n

\"Solutions<\/p>\n

In a paper<\/a> by Trilateral Research, a technology consultancy based in London, the coauthors argue that ethical principles and guidelines do not, by themselves, help practically explore challenging issues such as fairness in emotion AI. These need to be investigated in-depth, they say, to ensure that companies don\u2019t implement systems in opposition to society\u2019s norms and values. \u201cWithout a continuous process of questioning what is or may be obvious, of digging behind what seems to be settled, of keeping alive this interrogation, ethics is rendered ineffective,\u201d they wrote. \u201cAnd thus, the settling of ethics into established norms and principles comes down to its termination.\u201d<\/p>\n

Cook sees flaws in The Hume Initiative\u2019s guidelines as written, particularly in its use of nebulous language. \u201cA lot of the guidelines feel performatively phrased \u2014 if you believe manipulating the user is bad, then you\u2019ll see the guidelines and go, \u2018Yes, I won\u2019t do that.\u2019 And if you don\u2019t care, you\u2019ll read the guidelines and go, \u2018Yes, I can justify this,’\u201d he said.<\/p>\n

Cowen stands by the belief that Hume is \u201copen[ing] the door to optimize AI for individual and societal well-being\u201d rather than short-term business goals like user engagement. \u201cWe don\u2019t have any true competitors because the other AI models available to measure cues of emotion are very limited. They focus on a very narrow range of facial expressions, completely ignore the voice, and have problematic demographic biases. These biases are woven into the data that AI systems are usually trained on. On top of that, no other company has concrete ethical guidelines for the use of empathic AI,\u201d he said. \u201cWe are creating a platform that centralizes the deployment of our models and offers users more control over how their data is used.\u201d<\/p>\n

But guidelines or no, policymakers have already begun to curtail the use of emotion AI technologies. The New York City Council recently passed a rule requiring employers to inform candidates when they\u2019re being assessed by AI \u2014 and to audit the algorithms every year. An Illinois law requires consent from candidates for analysis of video footage, and Maryland has banned the use of facial analysis altogether.<\/p>\n

Some vendors have proactively stopped offering or placed guardrails around their emotion AI services. HireVue announced<\/a> that it\u2019d stop using visual analysis in its algorithms. And Microsoft, which initially claimed its sentiment-detecting Face API could detect expressions across cultures, now notes in a disclaimer that \u201cfacial expressions alone do not represent the internal states of people.\u201d<\/p>\n

As for Hume, Cook\u2019s read is that The Hume Initiative \u201cmade some ethics documents so people don\u2019t worry about what [Hume is] doing.\u201d<\/p>\n

\u201cPperhaps] the biggest issue I have is I can\u2019t tell what they\u2019re doing. The part that\u2019s public \u2026 doesn\u2019t seem to have anything on it apart from some datasets they made,\u201d Cook said.<\/p>\n

\n

VentureBeat<\/h3>\n

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.<\/p>\n

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:<\/p>\n