Amazon.com Inc. is developing a wearable voice activated device that can recognize human emotions. The pulse gadget is described as a health and wellness product in internal documents reviewed by Bloomberg.
It’s a collaboration between Lab126, the hardware development group behind the Amazon Fire phone and the smart Echo speaker, and the Alexa voice software team.
Designed to work with a smartphone application, the device has microphones paired with software that can discern the user’s emotional state from the sound of his voice, according to the documents and a person familiar with the program. Eventually, the technology may be able to advise the user to interact more effectively with others, documents show.
It is not clear to what extent the project is or if it will ever become a commercial device. Amazon gives teams wide latitude to try products, some of which never come to market.
Work on the project, codenamed Dylan, was underway recently, according to the documents and the person, who requested anonymity to discuss an internal matter. A beta testing program is underway, said the person, although it is not clear whether the test includes hardware prototype, the software for detecting emotions or both.
Amazon declined to comment.
The notion of building machines capable of understanding human emotions has long been a landmark in science fiction, from Isaac Asimov’s stories to the android Data of Star Trek. Amid advances in machine learning and voice and image recognition, the concept has recently marched into reality.
Companies like Microsoft, Alphabet, Google and IBM, among a host of other companies, are developing technologies designed to derive emotional states from images, audio data, and other inputs. Amazon publicly discussed its desire to build a more realistic voice assistant.
The technology can help the company gain insight into potential health products or be used to better target advertising or product recommendations. The concept will likely add fuel to the debate over the amount and type of personal data collected by technology giants, who already collect hundreds of information about their customers.
Earlier this year, Bloomberg reported that Amazon has a team listening to and recording audio clips captured by Echo’s line of voice-activated loudspeakers.
An American patent filed in 2017 describes a system in which voice software uses vocal pattern analysis to determine how a user is feeling, distinguishing between “joy, anger, sadness, sadness, fear, repulsion, boredom, stress, or other states emotions.
The patent, released last year, suggests that Amazon could use knowledge of a user’s emotions to recommend products or otherwise tailor responses.
A diagram in the patent registry says that the technology can detect an abnormal emotional condition and shows a sneaking woman telling Alexa that she is hungry. The digital assistant, noticing that she has a cold, asks the woman if she would like a chicken soup recipe.
A second patent granted to Amazon mentions a system that uses techniques to distinguish the user’s speech from background noise. Amazon documents reviewed by Bloomberg say the wearable device will take advantage of this technology.
Amazon’s work on a wearable device underscores its ambitions to become a leading manufacturer of state-of-the-art speech recognition software and consumer electronics. The Echo smart speaker line and the built-in Alexa voice software have popularized the use of voice commands at home. The company also added voice control to television-branded streaming video devices as well as tablets.
But Amazon’s efforts to create smartphone software to rival Apple Inc. or Google have failed. So the company is trying to make Alexa ubiquitous in other ways. Bloomberg said earlier this year that Amazon was developing wireless headsets, similar to Apple’s AirPods, which should include Alexa voice software.
The company began distributing Echo Auto, a panel-mounted speaker and microphone system designed to pair with a smartphone, and says it has received 1 million pre-orders.
Amazon is also working on a home robot, Bloomberg reported last year. Codename “Vesta”, in honor of the Roman goddess of the home and family, the bot could be a kind of mobile Alexa, according to people familiar with the project. Robot prototypes can navigate the houses like a stand-alone car.