User Power with Apple LLM and Ferret

Ajax, Apple's massive language model, will power the new features, which aim to give important capabilities while ensuring privacy through on-device processing

The ability of Siri to examine and summarise text messages within the Messages app is one of the major enhancements anticipated in iOS 18

Apple Insider reports that Safari will have “Intelligent Search” to provide webpage summaries outside of Messages

Apple has been working on Ajax and talking to OpenAI and Google about powering some of its AI features, according to recent reports

Ferret is a multi-modal big language model that can follow instructions, focus on a targeted object, and understand its surroundings

It’s built for the now-standard AI use case of querying a device about the world, but it may also understand your screen

This has exciting accessibility implications that potentially revolutionise how you use your phone, Vision Pro, and smart glasses