Haleon harnesses Microsoft’s Seeing AI technology to make health product information more accessible for people who are blind or have low vision

Haleon harnesses Microsoft’s Seeing AI expertise to make well being product data extra accessible for people who find themselves blind or have low imaginative and prescient

Posted on


For Marc Powell, a seemingly easy journey to the shop to purchase remedy or different client well being merchandise is commonly an onerous, difficult endeavor.

Powell, who’s blind, normally should ask a retailer worker to assist discover what he wants after which learn the knowledge on the packaging to him, which might really feel intrusive. At house he’ll ask his accomplice, who has partial sight, for assist, however the print is usually too small for her to learn. In a pinch, he’ll scour the web or wait till a member of the family stops by and might learn aloud to him the knowledge he wants.

“I’m extremely reliant on any person else to let me know in regards to the product — what I’m meant to do with it, what number of tablets I can take or what it comprises,” says Powell, the accessibility innovation lead for the U.Ok.-based Royal Nationwide Institute of Blind Individuals (RNIB), who lives in Cambridge, England. “It’s loopy to say that, isn’t it?”

Marc Powell.

A brand new collaboration between Microsoft and international client well being firm Haleon goals to make well being product data extra accessible for people who find themselves blind or have partial imaginative and prescient, or people with low literacy. The businesses labored collectively to broaden the performance of Microsoft’s Seeing AI app to offer detailed audio data for greater than 1,500 Haleon merchandise within the U.S. and the U.Ok., together with manufacturers corresponding to Centrum, Sensodyne and Emergen-C. The initiative was launched in recognition of World Sight Day Oct. 13.

Utilizing Seeing AI, customers can now scan the bar code on Haleon merchandise and listen to the identical data that’s offered on the packaging, such because the product title, utilization directions and substances. When customers level a cellphone at a Haleon product, the Seeing AI app guides them to the bar code with a collection of intensifying beeps. After scanning the code, customers can navigate between sections to get the precise data they need, a characteristic that has been well-received by partially sighted individuals who tried out the performance earlier than its launch.

“We now have had nice suggestions from people who find themselves blind and have low imaginative and prescient,” says Tamara Rogers, international chief advertising officer at Haleon. “They actually worth the elevated independence that comes from being empowered to entry our product data themselves.”

Black and white head shot of Tamara Rogers, chief marketing officer for Haleon.
Tamara Rogers.

The brand new performance was developed not only for people who find themselves blind or have low imaginative and prescient, but additionally for many who have literacy challenges, Rogers says. “There’s a traditional well being trade saying which is, ‘All the time learn the label, all the time learn the directions,’ and for some, that simply isn’t attainable,” she says.

“This initiative will make Haleon merchandise extra accessible to people who find themselves blind or have low imaginative and prescient. It should additionally present extra independence to folks with low literacy ranges. It is a good way of with the ability to talk differently, audibly, moderately than asking folks to learn.”

Launched in 2017, Microsoft’s Seeing AI utility is an AI camera-based app that may learn textual content corresponding to paperwork or indicators, describe scenes, acknowledge forex and even establish buddies. Accessible in 19 languages, the app has grow to be a multipurpose device that’s serving to people who find themselves blind or have low imaginative and prescient navigate their every day lives. The newly enhanced performance helps customers within the U.S. and U.Ok. learn packaging particulars in English. Beforehand, there wasn’t a straightforward approach for Seeing AI to learn detailed data on packaging with out a person shifting a digicam across the textual content.

The collaboration with Haleon “is de facto vital and thrilling, as a result of Seeing AI has all the time been capable of acknowledge merchandise, however the problem has been gaining access to information that’s actually wealthy, and that comes from the producer,” says Saqib Shaikh, an engineering supervisor at Microsoft who led the crew that developed Seeing AI.

“Now you’ll be able to simply scan the barcode and listen to precisely what was written on the package deal. It’s immediately what the producer wished you to listen to. The knowledge is de facto correct, and also you’re getting it multi function place. You’re not having to scan all of the totally different sides of the package deal to search out the bits you need. It’s all proper there when you’ve scanned the barcode.

“Now you may give extra data, and reliable data,” Shaikh says. “Haleon is main the best way in doing this.”

Shaikh hopes extra firms will comply with Haleon’s lead and use expertise to offer audio data for people who find themselves blind or who’ve low imaginative and prescient, or these with literacy challenges. Doing so may present vital data not only for well being merchandise, he says, however for meals objects and numerous different issues folks use of their every day lives.

“I’d like to make each product on the market accessible so you’ll be able to simply scan it, whether or not by bar code or another future expertise, and know what it’s,” he says. “This information is on the market. It’s simply that every one the information’s in siloes. What we had been capable of do right here was, by the businesses having the desire, to interrupt down these siloes.”

Close-up of hands holding a box of Centrum in one hand and a phone showing the Seeing AI app in the other.
Utilizing the Seeing AI app, detailed audio data can be obtainable for detailed audio data for greater than 1,500 Haleon merchandise within the U.S. and the U.Ok.

Greater than 3 million folks in america and greater than 2 million within the U.Ok. live with sight loss, and thousands and thousands extra battle with poor literacy. A examine commissioned by Haleon discovered that 93 % of respondents didn’t suppose well being merchandise had been accessible sufficient, and nearly one in 5 mentioned they’d taken an incorrect dosage of a product as a result of they couldn’t learn the packaging.

Haleon noticed expertise as a device to deal with these points, Rogers says, however wished to construct on an current platform moderately than creating new expertise that people who find themselves blind or have low imaginative and prescient would want to undertake. Haleon approached Microsoft about utilizing the Seeing AI app to create audio guides for its merchandise, and the 2 firms labored intently collectively to construct out the brand new performance for Haleon’s bar codes. Microsoft developed an end-to-end pipeline that permits Haleon to manage the information for its client merchandise, which is saved in Azure, and replace data or add new objects as wanted.

Powell says to his information, Haleon — a by-product of GlaxoSmithKline created in July 2022 to concentrate on client well being merchandise — is the primary client well being firm to offer accessible data by Seeing AI for its merchandise.

“I believe this can be actually helpful,” he says. “This enables us as blind folks to have the identical degree of information and understanding as somebody who can learn the packaging. So it is a actually good step.”





Supply hyperlink

Leave a Reply

Your email address will not be published. Required fields are marked *