By Kathleen Berger, Executive Producer for Science and Technology

Whether it’s playing a holiday song or getting help with a holiday recipe, suddenly people are hit with online ads appealing to the very interests they vocalized. So, ‘Hey, Alexa, just how smart are smart speakers?’ Umar Iqbal doesn’t need to ‘ask’ Alexa for the answer. He launched a research investigation to find out. Iqbal is an assistant professor of computer science and engineering at the McKelvey School of Engineering at Washington University in St. Louis.

“They (smart speakers) are getting extremely popular. There are more than 95 million users in the U.S. alone. If you look at the practices of smart speaker platforms, like Amazon, there have been instances where they have shared users’ recordings with third party entities and they were able to listen to what users were talking about,” said Iqbal.

Iqbal said Amazon Echo, engaging people through Alexa, is the most popular of them all. So, he decided to find out what Alexa (Amazon Echo) is hearing and learning from people. The research team designed a study, measuring the collection, usage and sharing of Amazon Echo interaction data.

First, they created several personas with interests in specific categories and one control persona. Each persona interacted with a different Amazon Echo device. Then the researchers measured data collection by intercepting network traffic and gauged data usage by observing ads targeted to each persona on the web and on Echo devices.

“In terms of the actual experiments, everything was automated. We did not have a person sitting next to a smart speaker saying things,” he said. “We wrote programs that uttered commands automatically that the smart speakers were able to hear. And then when they responded back, we recorded their interaction.”

The automated interactions went on for a month. By measuring the data collection, the research team discovered the smart speaker data is used in ways you might not expect.

“And what ends up happening here is that all of that data goes to Amazon. And Amazon uses this data to infer certain characteristics about the user. And then they can potentially market them (users) products, or send advertisements to users. Users’ smart speaker data is being used to profile them.”

The team reported that as many as 41 advertisers sync or share their cookies — which are typically linked to personal information — with Amazon, and then those advertisers further sync their cookies with 247 other third parties, including advertising services. They also found that Amazon did not clearly disclose that users’ smart speaker interactions are used for profiling them for the purposes of ad targeting.

“So, when we were studying the Amazon Alexa ecosystem, we did not find clear statements on how smart speaker data is used. But once we released our findings and Amazon was made aware by a journalist, Amazon, later on, updated their disclosures and clearly mentioned that they use the users’ smart speaker interactions to infer their interests. And then they (Amazon) may use that data to target them advertisements,” Iqbal explained.

After this change in the Amazon ecosystem, Iqbal was approached by regulatory agencies, including the Federal Trade Commission.

“Some regulatory agencies, like Federal Trade Commission, in the U.S. have shown significant interest in our findings. And we have had public and private meetings with them. “