... Yudkowsky and Soares aren’t sure a supersmart AI would seek global domination, but admit there’s no way to predict the “preferences” it might develop. Its “alien mechanical mind” will possess “internal psychology,” that doesn’t correspond with ours. There’s no reason to expect that those preferences will include “happy, healthy people leading fulfilling lives,” they write. ...
Nonetheless, Yudkowsky and Soares argue it’s time for people who share their concerns to speak up. Write to lawmakers, vote for candidates who understand the issue, find and attend protests and, just as important, tell your friends and family about the dangers. You might get “strange looks,” Yudkowsky and Soares acknowledge, but if there’s even a small chance that they’re right, strange looks will be the least of our problems.
See the full story here: https://www.sfchronicle.com/entertainment/article/superintelligent-ai-books-risk-21021893.php