How Bezos solved an unexpected problem in advertising his smart speaker.

Tonight in the States will be the Super Bowl, the main television show of the year. Last year, 111.3 million Americans watched it live, almost every third. But the commercials in her breaks have almost become more important and more interesting than the game itself. They are a topic of conversation, they are quoted, ratings are made for them, and there is not even talk about switching the channel (in Canada they tried to ban videos because of the law, which caused protest from viewers who said they were “an integral part of the spectacle”).

Prices for one video this year start at $ 5 million. This is about twice as expensive as in normal prime time. The broadcasting company, NBC, sells them in a “package” with advertising at the next Olympics, and expects to earn $ 1.2 billion in this scheme. Amazon will spend the most on its video for the Super Bowl this year. Their video is the longest (90 seconds instead of the usual 30 seconds), and with a bunch of celebrities, including Bezos himself. Advertise, of course, Amazon Echo with an assistant Alex . That's just the advertising of all smart speakers, it turns out, there is one specific problem that somehow needs to be solved ...

According to experts, Amazon spent about $ 15- $ 20 million on a short video. Compared with the budget of some films! And the cast is appropriate. In the story, Alex, Amazon's virtual assistant, suddenly fell ill and lost her voice. Bezos decides to replace it with live celebrities. Chef Gordon Ramsay, swearing, gives advice to a teenager on how to make a sandwich. Singer Cardi B is trying to tell how many kilometers to Mars, and sings his songs instead of the ordered country music. And the actor Anthony Hopkins, who played in The Silence of the Lambs, scares the girl who wanted to call a friend.

17 million people have already seen ads on YouTube, even though they launched it only 4 days ago (in Russia, video from the Amazon channel does not show, a clone is inserted into the article).

But there is one problem. All this may be useless for anyone who already has a device with Alex at home. Their column will think that people from the TV are turning to it, it will turn on and will begin to carry out their commands. This is a nightmare.

Precedents have already been. In April 2017, the Burger King fast food chain creatively “warmed up” all users who had a Google Home column at home. In their video, an employee of the restaurant explained that in 15 seconds he would not have time to tell about all the charms of the chic "Whopper Burger". But he supposedly found how to do it! Having approached the camera, he says, “Okay, Google, what is the Whopper-burger?”. At this point, the Google columns and Android devices woke up, and began to read an article from Wikipedia about this burger (with the specially pre-modified Burger King, so that their product seemed even more appetizing).

Then Google had to urgently adjust the functionality of its smart assistant for 3 hours, and add annoying “dishonest” ads to the blacklist. It helped, but not for long. "Burger King" changed the voice acting in his video, and their advertising again earned. Despite the unclean methods, the campaign turned out to be very effective, and eventually won the Grand Prix at the Cannes Lions international advertising festival.

Bezos would not want to troll himself as Burger King managed to troll Google. Especially when it comes to the most expensive advertising on TV in a year.

Money, of course, he has. If he wants, he can buy at least all of American football. Recently, his fortune in one day rose by $ 6.5 billion - thanks to yet another Amazon rally. Now Bezos is ahead of billionaire number 2, Bill Gates, by more than $ 30 billion. He can afford to spend money. And, as you know, gives billions to a very specific hobby, like flying into space. But in business affairs, Bezos tries to optimize every penny. He has this fad. Devices with Alex are in tens of millions of American homes. Is it really tens of millions of viewers who, instead of watching his advertisements, will be in a panic listening to their suddenly awakened column?

However so. As we learned at Bloomberg, Amazon is confident that Alex will not turn on, even though she will be called 10 times in the video. All thanks to their newly developed locking system for certain frequencies.

They began to work on the problem before Echo entered the market. In September 2014, Amazon filed a patent application called "Sound Commands Filtering". It described the techniques that allow the voice assistant to “not wake up while watching a program designed for a large number of viewers (for example, during a sporting event.” This phrase from the patent describes the situation with the Super Bowl so precisely that you might think that Bezos planned all his advertising four years ago.

Patent technologies should, among other things, help prevent Amazon servers from being overloaded with millions of simultaneous commands. There are two of them. The first transmits to smart speakers a segment of sound from an advertisement before it is shown. Then the device will compare everything that it hears with this “acoustic print”, and if the sound is identical, it will refuse to turn on, realizing that this is sound from an advertisement, and not from the owner. The second method is to somehow encrypt the acoustic signal in the advertisement itself. So that the person did not distinguish him, and the column felt that the next command should be ignored.

In the end, apparently, Amazon chose the second option. So says reddit user Asphyhackr. A year ago, he conducted a study of this topic, "drove" the existing videos with Alex’s advertisement through the Audacity editor, and found that in all cases the audio had a significant decline in the range from 3,000 to 6,000 Hz. Sometimes these frequencies did not exist at all. Then he lost the voices of people asking Alex to wake up, but with a frequency of 5,200 Hz, and the speakers did not respond to the call. This is the very "acoustic imprint": the column sees a "hole" in the frequency spectrum, and understands that it is advertising.

But in fact it is still more difficult. In response to numerous questions, Amazon posted a blog in front of the Super Bowl, explaining its method:

When our 90-second video will be shown on Sunday evening, millions of Echo-devices will not accidentally turn on the word "Alex". This is possible thanks to the acoustic imprint technology developed by us, which is unique to each advertisement.

It turns out that teams of advertisers, scientists and engineers at Amazon are preparing for big events, such as advertising on the Super Bowl, and in each case decide how best to make Alex not respond if she is called. And if a large broadcast, where they say the team to the assistant, was not discussed in advance (for example, Alex will call one of the guests at the evening show), Amazon cloud servers immediately feel that something is wrong: too many requests with the same phrase . As a result, the first 10-20% of smart speakers in the country are included. And for the rest of the server, they manage to make an acoustic imprint “on the fly” and send it to the devices, warning that this is probably a false positive. That is, Amazon uses both methods from the patent. So that people can safely watch their favorite TV shows, and no one bothers them.

PS You can buy at Amazon and other American stores from Pochtoy.com. For delivery from the US parcel weighing 0.5 kg, we take from $ 8.99. And if you register and enter the code Geektimes, we will credit $ 7 to your account.