Apple to bring AI-powered VoiceOver, speech recognition and more accessibility features this year

Apple will roll out the VoiceOver, Magnifier, Voice Control, and Accessibility Reader, and more in the new OS updates later this year to iPhones, MacBooks, iPads, Apple TV and Vision Pro.

Smartphones, computers and tablets have become an integral part of the lives of most people, not only at the office and schools, colleges, but also at home.

These devices also come with advanced features to help people with disabilities use them with less hassle, but they are limited to a handful of brands. Among them, Apple stands at the forefront.

Compared to other platforms, Apple over the years has worked sincerely to bring more accessibility features to ensure the differently abled people make good use of iPhones, Mac computers, Watches, iPads and TV for office productivity, health tracking and entertainment.

Some of the popular features include eye tracking, Listen for Atypical Speech, Vocal Shortcuts and last year, it introduced all-new magnifier, braille experience, Live Listen, and Live Captions came to Apple Watches.

Now, it has announced to bring new features, including VoiceOver, a new Magnifier, Voice Control, Accessibility Reader, and more.

“Apple’s approach to accessibility is unlike any other. Now, with Apple Intelligence, we are bringing powerful new capabilities into our accessibility features while maintaining our foundational commitment to privacy by design,” said Tim Cook, Apple’s CEO.

Apple is bringing Apple Intelligence to Voice Over, Magnifier and Voice Control

For people who are blind or have low vision, VoiceOver and Magnifier can help them describe the onscreen information or their surroundings.

The Image Explorer in VoiceOver uses Apple Intelligence to give more detailed descriptions of images systemwide, including what’s in photographs, scanned bills, personal records, and other visual content. Users can also ask follow-up questions in their own words to get more visual information.

With Apple Intelligence integration, Magnifier can bring the same assistive exploration and visual description to a high-contrast interface designed for users who have low vision. Magnifier also works with the Action button, so users can quickly ask questions and get answers. And users can control the app itself with spoken requests, such as 'zoom in' or 'turn on flashlight'.

Apple Intelligence-powered Magnifier.

With Apple Intelligence, Voice Control will become more intuitive than ever. People with motor disabilities can use just natural language to navigate an iPhone and an iPad entirely by voice.

With a new flexible input, users can describe onscreen buttons and controls with natural language instead of memorising exact labels or numbers.

Accessibility Reader

This new Accessibility Reader is a customised reading feature, tailor-made for users with a wide range of disabilities, from dyslexia to low vision.

It also uses the on-device Apple Intelligence feature to read complex source material like scientific articles, handling text with multiple columns, images, and tables.

Accessibility Reader works on more complex source material like scientific articles, handling text with multiple columns, images, and tables.

On-demand summaries provide readers the option to get an overview of the article before diving into the details. And with new built-in translation, users can read text in their native language while retaining custom formatting, font, and colours.

Generated Subtitles for videos

While social media platforms and video-sharing applications now offer closed captions on videos, there is no built-in feature in devices that can intelligently generate subtitles for videos recorded on a phone or camera.

With the powered Apple Intelligence, on-device speech recognition feature will generate, subtitles privately and appear automatically for uncaptioned videos on iPhone, iPad, Mac, Apple TV, and Apple Vision Pro. The appearance of subtitles can be customised in the video playback menu or in Settings.

Vehicle Motion Cues, which was first introduced to iPhone and iPad, is now coming soon to visionOS. It will help reduce motion sickness for people who use Apple Vision Pro as a passenger in a moving vehicle. Vision Pro will also support face gestures for performing taps and system actions, plus a new way to select elements with one’s eyes while using Dwell Control.

Further, Apple is bringing much-awaited 'Larger Text' support to tvOS, so viewers who have low vision can increase the onscreen text size to be easier to read.

Apple is also launching the Name Recognition feature. It can notify users who are deaf or hard of hearing if someone says their name, and it works across more than 50 languages globally.

Wheelchair control on Apple Vision Pro

With a precision eye-tracking system on Apple Vision Pro, a new power wheelchair control feature offers users a responsive input method for compatible alternative drive systems. It will reduce the need for using joystick for moving around the house.

With Vision Pro, eye tracking doesn’t require frequent recalibration and works in a variety of lighting conditions.

Accessibility-centric MagSafe cover case for iPhone

Los Angeles-based designer Bailey Hikawa worked with individuals with a wide range of disabilities affecting grip, strength, and mobility to develop this MagSafe accessory.

The Hikawa Grip & Stand for iPhone.

The adaptive grip and stand help users hold the iPhone in the way that works best for them. The Hikawa Grip & Stand for iPhone is now available in three colours.

Get the latest news on new launches, gadget reviews, apps, cybersecurity, and more on personal technology only on DH Tech.