To enable the search for landmarks in private photos, iOS compares image data with Apple servers. Opting out clearly does not reverse this.
The advanced photo analysis feature that Apple quietly introduced in iOS 18 and macOS 15 has come under criticism. After installing the current operating system version, iPhones, iPads and Macs send certain image data to Apple servers without being asked to make it possible to search for landmarks in the photo library. This is enabled by default, and Apple calls the feature “advanced visual search.” Developer Jeff Johnson complained that he had stumbled across a corresponding setting on his own iPhone, but had never requested such “enhancements” to photos stored locally on the device. Apple made the decision “silently and without consent.” Johnson’s post sparked a social media debate over Apple’s long-standing privacy practices, and has at times led to misleading media reports accusing the company of stealing users’ photos.
Apple mentions data protection caveats
Apple’s own summary of “all new features” in iOS 18 makes no mention of “advanced visual search.” The new feature is briefly mentioned in the Photos app’s privacy policy: “Your device privately compares your location in the photo to a global index held on Apple servers. Apple applies homomorphic encryption and differential privacy. In addition, an OHTTP relay is used that hides your IP address. “This means that Apple does not receive any information about the content of your photos,” the company promises there. An entry on Apple’s machine learning blog published in October explains how this is technically implemented:
Apple’s operating system now analyzes image content entirely locally on the device. This allows users to specifically search for certain image elements (“bicycle”) or scenes (“shore”). Previously, visual search provided additional information about the content of the image, trying to identify, for example, dog breeds, sights, plants, insects, etc. However, this only happens when you open the photo. For the first time, “enhanced visual search” now compares image data with a database on Apple servers. Homomorphic encryption allows calculations to be made using encrypted data, eliminating the need to decrypt the data on Apple’s servers.
Criticism for lack of external reviews
The feature also seems to work if photos are only stored locally on iPhone & Co and not synced via iCloud. One problem is that Apple doesn’t offer an opt-in for this. Users can turn this switch off later only if they know where it is (Settings > Apps > Photos > Advanced Visual Search in iOS/iPadOS 18, and the settings in the General tab of the Photos app in macOS).
The opt-out only applies to photos taken after that; the existing photo library has already been analyzed.
If you have “good intentions,” this is not the way to market a privacy-conscious product, cryptographer Matthew Green commented on Hacker News. Outside security researchers should be given a chance to verify Apple’s privacy commitments before enabling such a feature by default.
