According to Microsoft, the AI will rely on a camera placed on the roof of the Sister City hotel in New York City. “It can find and understand with better accuracy a wider range of objects, in much higher detail. So it doesn’t just find clouds, but denotes the density and type of cloud, whether cumulus or nimbus. And it won’t just find a bird, but will also distinguish an entire flock of birds.”
Based on what it detects, it will adapt the musician’s music accordingly. It will also be smart enough where long-term weather trends can also alter the music. It is a rather novel use of AI, but given that we’re still very much in the early stages of knowing what AI can and cannot do, it will be interesting to see how this application of AI could eventually be adapted for use in other areas as well.
If you’re interested in checking this AI experiment out, the music will be available in the lobby of the Sister City hotel in New York City, so if you’re in the area, it could be worth stopping by and checking it out.