Automatic Content Recognition for today’s TV experiences

With the rise of the multi-screen TV watching, Automatic Content Recognition is becoming an important technology for mobile TV apps to let users engage with its audible surroundings.

Automatic Content Recognition technologies are an effective way for broadcasters and operators to address the demand for a television experience that is Interactive, Personalized, and Social, allowing them to evolve towards the demands that come with today’s TV viewing habits.

Social Media and TV

If you are social media user while watching TV and you want to know what other viewers or friends are saying about that football match or talent show on TV, you likely grab you mobile and open Twitter or Facebook, enter a hashtag or web address to interact with the TV program itself or other views.

Automatic Content Recognition (ACR) technology, either based on watermarking or audio fingerprinting, will change TV engagement and interaction. Thanks to Automatic Content Recognition technology —like audio fingerprinting— your devices will be able to identify TV content and connect you other viewers watching the same content.

Added value to Advertisements

For example, if you are watching a TV show and an ad break comes in that is interesting for you, you could tag the advertisement with your phone based on ACR to get an exclusive coupon instantly downloaded to your smartphone. The goal is to add value to viewing experience with more relevance and interaction while helping advertisers with more conversion.

Automatic Content Recognition for synced advertising

Automatic Content Recognition (ACR) for contextual content

ACR is nothing new. As far back as 2002, Shazam created a service where you could identify music playing around you by dialling a phone number and holding your phone up to the source. The service would then send you a text message with the artist name and title of the track.

But now, ACR provides contextual awareness and more relevance to TV second screen applications. Although there are many applications, which are asynchronous, most of the suppliers of ACR believe that applications like IMDb or Twitter would be so much better if they could surface relevant data based on the content around you.

ACR for Second Screen Audio Sync

Synchronous Second Screen apps are being made for popular TV shows. ACR is used to synchronize time delay caused by buffering, time-shifted viewing or Video on-demand. Beatgrid Media delivers frame-accurate synchronization bringing apps to the moment or scene being consumed, which is far more relevant than the linear TV schedule.

In short, ACR in the form of Audio Fingerprinting can be used for syncing and identifying media content. The Audio Sync application of ACR allows coordinated display of information through a second screen, Identification allows the pairing of the exact moment of content with conversion tracking, ad targeting, and personalization. The advantage of audio fingerprinting versus content watermarking is that you make the content identifiable without the need to change or add something to the content.

ACR Technology applied for audio sync from broadcast to apps

Mass adoption and privacy concerns

Smart TV manufacturers like Samsung and LG have been using on-device Automated Content Recognition (ACR) for a while now, allowing these CE manufacturing get data about viewing behaviour, which is used by broadcasters or advertisers for synchronous TV experiences and increased relevance. This adoption by manufactures is criticized because it allows your viewing/listening data to be collected and analysed by which many feel is intrusive. While this may be the case, we’ve become used to the privacy trade-off. Giving up personal information to gain access to content is the norm.


Growing demand for ACR solutions

Since content is everywhere and media got more segmented due to people “multimedia tasking” cross-device, ACR is seeing an uptake in usage by broadcasters and advertisers to

  1. Connect users with content
  2. Measure audiences everywhere and
  3. Measure content and advertisement distribution more accurately


The Near future of ACR and Artificial Intelligence

Artificial Intelligence applied to ACR technology is where this gets exciting. Most ACR technologies will only recognize content that has been programmed to recognize (audio fingerprinted content) but when it will learn to recognize the Visual sights and Audio a person encounters around him day-to-day another world of opportunities will come alive. This can be based on audio ACR like city noises or the sound of driving a car or it can based on visual ACR like serving offers ads or coupon based on how many times you been exposed to brands in Youtube or Instagram browsing. It will open a world of deeper contextual media experiences.

Another Example:

You’re in the cinema and your movie is about to start and a trailer for the new Jurassic park film catches your attention. Imagine you then grab the IMDb app – which would contain ACR technology– and once you opened the app, it would directly know you are being exposed to the Jurassic park trailer and that you are a fan of adventure movies and if would automatically guide to related (premium) content or a ticket offer. Alternatively your smartwatch would be asking you if you’d like to follow this up after your movie. You tap “yes” and your watch goes back to sleep, allowing you to watch the movie. As soon as the movie finishes, your watch will tell you that the cinema has availability for the premiere and ask if you would like to buy tickets. The same is possible for music and concerts… There are other examples that go even deeper than which I will appoint in in one of the next blogs.

second screen technology beatgrid

Those advantages will need to convince the consumer to adapt audio and visual ACR in its life through different devices like smartphones or smart watches. People are only willing to provide data if applied in a way that is unobtrusive to the overall user experience.

ACR will be widely accepted

Automatic Content Recognition is still at the beginning of major adoption. It might be creepy, but ACR is here to stay and will soon be one of those technologies we take for granted.

ACR & Artificial Intelligence will make devices aware of their environment and learn from behavioral patterns to help the user in limitless ways and help the brands be more relevant every time.

If used smart it’s a Win –Win.

Contact Us

Tell us about yourself! And please tell us how we can help you.

EMOMENT wins Beatgrid's Automatic Content Recognition IBC hackfest challengeAudio Fingerprinting API