Case Study: How AI can change a whole generation of deaf children's lives
We live in a time where technology allows us to do pretty much anything. But until recently, this innovation hasn’t been reflected in underrepresented groups.
A collaboration between Huawei, nonprofit European Union for the Deaf, Penguin, and animation gurus Aardman, StorySign is a D&AD Impact Pencil winner that shows how AI has the power to change lives.
For the first time, the needs of a whole generation of deaf children are being heard.
The problem
Literacy is a massive problem in the deaf community, stemming from an inability to learn letters phonetically, which results in a struggle to match words and sounds. Studies show that there are up to 32 million deaf children all over the world that are unable to read.
This has an effect not just on their education and social development, but the long term implications of this reach far into their futures, in a reduction of opportunities vocationally and fiscally, as well as impacting their health. New research shows that it takes 21 years for deaf children to catch up with their hearing peers.
What if there was a way to ensure this gap doesn’t get created in the first place and for deaf children all over the world to experience the magic of the written word in the same way as their hearing friends?
The idea
Huawei, the second-largest smartphone manufacturer in the world, prides itself on “pushing the limits of what is humanly possible”. Wanting to bring this mission statement to life in a way that had a positive impact on society whilst also improving brand perception and awareness, they challenged the team to identify a problem that could be solved by their technology.
“We decided the first execution for this had to have a social footprint and help solve a real problem”, said the team at FCB Inferno. “We looked at several things, but the scale of the problem of deaf illiteracy and how much we could help with mobile technology meant it bubbled to the top”.
Deaf literacy became the cultural insight and backdrop for the project, and soon plans were drawn up for an app that would allow deaf children to make the crucial link between printed pages and sign language, so deaf children and their parents can read and sign together.
It’s now the top-rated deaf app on iTunes, has had over 128 million in-app film views and has an estimated global reach of 1.5 billion children.
How the app works
Huawei’s image recognition technology was used so that the app could detect words and could even be used in low light - perfect for bedtime, whilst OCR enables greater accuracy. But to be able to make the connection between the words on the page and sign language itself, they needed a translator - an avatar.
Working with the Oscar-winning studio at Aardman, they created Star, whose every carefully considered detail was designed to make sure she would appeal to deaf children. For example, “she’s the age of an older sibling as children are proven to learn better from an older peer”. She wears a hearing aid so that it normalises and encourages their use and even her tom-boyish appearance has been chosen so that she appeals to both boys and girls.
“The most challenging part was the development of the UX,” said the team at FCB Inferno. “We worked with deaf charities and tested extensively at deaf schools and with deaf families. We wanted to create something that wasn’t simply an e-reader, as the biggest indicator of a child’s success at primary school is their parents’ engagement. So we knew we needed to create something that involved both parent and child, a physical book and StorySign to translate”.
Working with Penguin books allowed them to find a best selling book for each market so that it could be built around copies that families were likely to own already. So where Spot The Dog was used in English markets, Germany and Italy started with Peter Rabbit books.
From there, the library continued to grow, with 59 more books added in the past year, and a plan to keep on expanding. And it’s not just the book data that’s constantly being built upon. Soon the app will be launching in Brazillian, Indian and Chinese sign language.
Results
“Working with the children and seeing the incredible and immediate impact with the families has been incredible. Because 90% of deaf children are born to hearing parents, the vast majority of deaf children are growing up in a house where they don’t speak the same language as their parents. Providing something where parents and children can come together, and both learn has been amazing”.
To say StorySign has been a success story is an understatement. It’s now the top-rated deaf app on iTunes, has had over 128 million in-app film views and has an estimated global reach of 1.5 billion children.
It’s shown how mobile technology can tackle a real-world problem at scale, raised brand awareness and truly pushed the limits of what is humanly possible.
But above all, it’s changed a whole generation of deaf children’s lives by opening up a portal to a whole new world.
For 58years D&AD has stimulated and celebrated creative excellence in commercial creativity, annually inviting 250 of the world’s greatest creative practitioners to judge the year’s finest work. If you think you have a campaign or a project that deserves a D&AD Pencil, enter your work into the D&AD Awards and see if our judges agree.