Google Introduces 'Emotion To Captions' With New Expressive Captions Feature
Google introduces Expressive Captions feature for Android to enhance accessibility. Expressive Captions uses AI to convey emotional context in spoken dialogue. The feature aims to make audio and video content more inclusive and engaging for all users.
This innovative software leverages artificial intelligence to help Deaf and hard-of-hearing individuals grasp the emotions conveyed in spoken dialogue.
Expressive Captions not only enables users to read spoken words but also provides a sense of the emotion behind them. According to Angana Ghosh, director of Android product management at Google, this update is significant as it allows individuals with hearing impairments to experience the emotional aspect of on-screen conversations.
Ghosh explained that Expressive Captions utilises AI on Android devices to interpret vocal attributes such as tone, volume, and even environmental sounds like crowd noise at events. By capturing these nuances, the feature goes beyond mere words to convey the full emotional context of the dialogue.
The development of Expressive Captions was a collaborative effort within Google, involving teams like DeepMind. Ghosh highlighted the complexity of the project, spanning several years and incorporating multiple AI models to analyse audio signals comprehensively.
Google's commitment to accessibility is evident through initiatives like Live Captions, introduced in 2019 to make media more inclusive for individuals with hearing challenges. Expressive Captions takes this a step further by providing context and emotion, enhancing the accessibility of audio and video content for all users.
Ghosh emphasised the importance of creating technology that benefits everyone, not just those with disabilities. By enhancing captions with emotional context, Google aims to improve the overall user experience and cater to a diverse audience, including the 70% of Gen Z who regularly use captions.
In response to feedback from the Deaf and hard-of-hearing community, Google recognised the need to capture the emotions and nuances present in audio content. By collaborating with experts in various fields, Google refined Expressive Captions to ensure that it reflects the context and emotional subtleties of spoken words accurately.
While Expressive Captions represents a significant advancement in accessibility technology, it is worth noting that professional captioners have long incorporated emotive metadata into closed captions. Despite this, Google's innovative approach to enhancing captions with emotional context marks a positive step towards more inclusive and equitable content consumption.
Ghosh expressed enthusiasm for the future of Expressive Captions, highlighting the team's dedication to creating a product that is both helpful and intuitive for users. By seeking feedback from diverse groups, including the Deaf and hard-of-hearing communities, Google aims to refine the feature further and make it truly beneficial for all users.
Google introduces Expressive Captions feature for Android to enhance accessibility.
Expressive Captions uses AI to convey emotional context in spoken dialogue.
The feature aims to make audio and video content more inclusive and engaging for all users.
Source: FORBES