philip lelyveld The world of entertainment technology

Electric Fish In Augmented Reality Reveal How Animals “Actively Sense” the World Around Them

In a new study, NJIT and Johns Hopkins researchers have used augmented reality technology to alter this link and unravel the mysterious dynamic between active sensing movement and sensory feedback. The findings report that subtle active sensing movements of a special species of weakly electric fish — known as the glass knifefish (Eigenmannia virescens) — are under sensory feedback control and serve to enhance the sensory information the fish receives. The study proposes the fish use a dual-control system for processing feedback from active sensing movements, a feature that may be ubiquitous in animals.

Researchers say the findings, published in the journal Current Biology, could have implications in the field of neuroscience as well as in the engineering of new artificial systems — from self-driving cars to cooperative robotics.

“What is most exciting is that this study has allowed us to explore feedback in ways that we have been dreaming about for over 10 years,” said Eric Fortune, associate professor of biology, who led the study at NJIT. “This is perhaps the first study where augmented reality has been used to probe, in real time, this fundamental process of movement-based active sensing, which nearly all animals use to perceive the environment around them.”

“We’ve known for a long time that these fish will follow the position of their refuge, but more recently we discovered that they generate small movements that reminded us of the tiny movements that are seen in human eyes,” said Fortune. “That led us to devise our augmented reality system and see if we could experimentally perturb the relationship between the sensory and motor systems of these fish without completely unlinking them.

To investigate, the researchers placed weakly electric fish inside an experimental tank with an artificial refuge enclosure, capable of automatically shuttling back and forth based on real time video tracking of the fish’s movement. The team studied how the fish’s behavior and movement in the refuge would be altered in two categories of experiments: “closed loop” experiments, whereby the fish’s movement is synced to the shuttle motion of the refuge; and “open loop” experiments, whereby motion of the refuge is “replayed” to the fish as if from a tape recorder. Notably, the researchers observed that the fish swam the farthest to gain sensory information during closed loop experiments when the augmented reality system’s positive “feedback gain” was turned up — or whenever the refuge position was made to mirror the movement of the fish.  

See the full story here: https://news.njit.edu/electric-fish-augmented-reality-reveal-how-animals-actively-sense-world-around-them