Patent filings have revealed that Facebook may be working on a controversial piece of tech which will allow it to trigger a user’s device (like a smartphone) to start recording audio whenever they hear inaudible messages hidden in television adverts. The patent spotted first by Metro, is titled “broadcast content view analysis based on ambient audio recording.” The patent describes a system in which an “ambient audio fingerprint or signature” that’s inaudible to humans could be embedded in broadcast content like a TV ad. When a hypothetical user is watching this ad, the audio fingerprint could trigger their smartphone or another device (like a smart speaker) to turn on its microphone, begin recording audio and share that recording with Facebook.
Gizmodo examined the patent and suggested that it is vague on the details of what exactly Facebook plans to do with the audio data. One scenario offered as an example in the patent document is where recorded ambient audio would be eliminated and the content playing on the broadcast would be identified, think of it as a Shazam for advertisements (h/t Nilay Patel for that example). Details about the user’s proximity to the audio, time of day, the identity of the Facebook user present would then be sent to the social media giant for further analysis and processing.
Facebook could then combine existing data it collects from its user’s browsing habits and content they share on the platform with this audio information to figure out which ads are the most effective. For example, if a user switches the channel as an ad comes on or walks away from the television, the ad might be labelled as ineffective. This data could then be served to advertisers as a performance metric.
Facebook says ‘we won’t use it’
This patent has some users concerned and a look at Facebook’s recent record on user privacy issues will tell you why. In fact, following the Cambridge Analytica disaster Facebook CEO Mark Zuckerberg testified before the US Congress where some lawmakers asked him if Facebook secretly recorded audio from users’ phones. Zuckerberg was quick to deny this and suggested that Facebook would never secretly record audio from its users.
Facebook has said it has no intention of ever actually implementing the technology described in the application. Allen Lo, Facebook’s Head of Intellectual Property has told multiple media outlets, “It is common practice to file patents to prevent aggression from other companies. Because of this, patents tend to focus on future-looking technology that is often speculative in nature and could be commercialized by other companies. The technology in this patent has not been included in any of our products, and never will be. As we’ve said before, we often seek patents for technology we never implement, and patent applications should not be taken as an indication of future product plans.”
While Lo’s take on patents isn’t wrong — they are often conceptual and never put into practice – the idea of Facebook even conceptualising such a piece of tech is justifiably worrying.
Silverpush has been there before
Fortunately, there is some regulatory precedence on this matter that might prevent Facebook from implementing the patent idea into a finished product. In March 2016, The US Federal Trade Commission issued warning letters to 12 application developers using an audio tracking code deployed by Gurgaon-based Silverpush. Addition of the Silverpush code allowed these apps to monitor consumers’ TV usage via audio beacons emitted by TV, which can’t be heard by the consumer, as long as the mobile phone remained in the same vicinity. Silverpush then used this information to provide advertising re-targeting on mobile.
In a statement, Jessica Rich, Director of the FTC’s Bureau of Consumer Protection had said that “These apps were capable of listening in the background and collecting information about consumers without notifying them,” and “Companies should tell people what information is collected, how it is collected, and who it’s shared with.”
Silverpush later informed MediaNama that the product in question had been discontinued.