VR and gesture tracking are already popular tools for controlling robots, but the new approach—combining them so that users can directly interact with large numbers of robots—is uniquely intuitive, the researchers say. The new system allowed untrained operators to corral 50 free-roaming virtual robots into three different areas in about 5 minutes, they report this week on the preprint server arXiv. In surveys afterward, users were broadly positive about the system.
Real world applications still face many challenges, including capturing enough data to create realistic simulations and transmitting them quickly enough to prevent the simulation from lagging. But the researchers say forthcoming 5G mobile communication technology could be the missing ingredient.