philip lelyveld The world of entertainment technology

Sensor Fusion for Learning-based Tracking of Controller Movement in Virtual Reality

These vision-based systems are unable to track controllers when they move out of the camera’s field-of-view (out-of-FOV). To overcome this limitation, we employ sensor fusion and a learning-based model. Specifically, we employ ultrasound sensors on the HMD and controllers to obtain ranging information. We combine this information with predictions from an auto-regressive forecasting model that is built with a recurrent neural network.

See the full story here: https://www.microsoft.com/en-us/research/publication/sensor-fusion-for-learning-based-tracking-ofcontroller-movement-in-virtual-reality/