Apple patents detail augmented reality device with advanced object recognition, POI labeling
At its core, the filing deals more with power efficient object recognition than it does visualization of AR data overlays. The former feature is perhaps the greatest technical barrier to wide adoption of AR. In particular, existing image matching solutions like those employed in facial recognition security systems consume a large amount of power and thus see limited real world use.
Apple's patent goes further than simple device control, however, as it details a power efficient method of monitoring a user's environment and providing information about detected objects. For example, the technology might be used as part of a guided tour app, scanning for and providing information about interesting objects as a visitor walks around a museum.
The document dives deep into initialization of optical tracking, or the initial determination of a camera's position and orientation. An integral facet of AR, initialization is considered one of the technology's most difficult hurdles to overcome.
Apple notes the process can be divided into three main building blocks: feature detection (feature extraction), feature description and feature matching. Instead of relying on existing visual computing methods, Apple proposes a new optimized technique that implements dedicated hardware and pre-learned data.
Interestingly, the invention notes depth images, which result in better object determination, require a two-camera setup. Coincidentally, Apple last year introduced a two-camera design with iPhone 7 Plus.
See the full story here: http://appleinsider.com/articles/17/01/31/apple-patents-detail-augmented-reality-device-with-advanced-object-recognition-poi-labeling