New MIT tech lets you mess with objects in pre-recorded video
[Philip Lelyveld comment - this is amazing technology! It appears to also capture and fill in background information, although that may just be an illusion from the object not moving very far.]
The system, called "Interactive Dynamic Video" (IDV), needs less than five seconds of footage to track movement possibilities. It does so by analyzing how it shifts when intentionally jostled: In the video example below, a researcher slams the table on which a humanoid figure is resting, which lets the system see how it vibrates across different frequencies. Then it extrapolates how the item should behave when viewers reach in with their cursors and jostle the object in video.
See the full story here: https://www.engadget.com/2016/08/03/new-mit-tech-lets-you-mess-with-objects-in-pre-recorded-video/