philip lelyveld The world of entertainment technology

Google Debuts Deep Planning Network Agent with DeepMind

Google unveiled the Deep Planning Network (PlaNet) agent, created in collaboration with DeepMind, to provide reinforcement learning via images. Reinforcement learning uses rewards to improve AI agents’ decision-making. Whereas model-free techniques work by getting agents to predict actions from observations, agents created with model-based reinforcement learning come up with a general model of the environment leveraged for decision-making. In unfamiliar surroundings, however, agents must create rules from experience.

More specifically, PlaNet “leverages a latent dynamics model — a model that predicts the latent state forward, and which produces an image and reward at each step from the corresponding latent state — to gain an understanding of abstract representations such as the velocities of objects.”

Predictive image generation is the means whereby the PlaNet agent learns, a quick process that, “in the compact latent state space … only needs to project future rewards, not images, to evaluate an action sequence.” Hafner noted that, rather than a policy network, PlaNet “chooses actions based on planning.”

“For example,” he said, “the agent can imagine how the position of a ball and its distance to the goal will change for certain actions, without having to visualize the scenario. This allows us to compare 10,000 imagined action sequences with a large batch size every time the agent chooses an action. We then execute the first action of the best sequence found and replan at the next step.”

See the full story here: http://www.etcentric.org/google-debuts-deep-planning-network-agent-with-deepmind/