I recently read a long but very interesting
piece at
Aeon magazine about, among other things, the threat that artificial intelligence may pose to human survival. While the article contains some very intriguing ideas, I also thought some aspects of it seemed rather naive.
The story is built around a philospher attached to an Oxford think tank who studies far future potentials for humankind, so he's no dummy. With that said, however, both the article's writer and his subject seem somewhat confused in their focus at times.
For instance, they both seem to think of artificial intelligence in parameters largely defined by today's popular culture, particularly movies and books, and don't really distinguish it from a very sophisticated computer. As a result, they give AI a sort of maliciousness it quite likely will never have. They don't seem to consider that artificial intelligence can evolve as science and technology becomes more advanced.
When discussing how AI could come to care for humans, the writer says "AI might think that human happiness is a biochemical phenomenon. It might think that flooding your bloodstream with non-lethal doses of heroin is the best way to maximize your happiness."
A computer might do that, but AI most likely wouldn't, because AI is much more than a fancy computer. Arguably, with time, AI might will be developed to simulate compassion and other emotions, but even without it, AI isn't stupid.
The author apparently imagines that humans would stand by passively as AI took over the globe. He writes that AI "might plan out a sequence of cunning chess moves to insulate itself from resistance. Maybe it would surround itself with impenetrable defenses, or maybe it would confine humans in prisons of undreampt of efficiency."
That presumes humans would not monitor what AI is doing nor changes it made to its own architecture. That aside, whatever AI might do, it will always need a power source, and therein lies humanity's upper hand.
At other times, the writer seems not to have thought through some of his statements. He says at one point, for example, "there are good reasons for any species to think darkly of its own extinction." I think it's a pretty safe conclusion that only
homo sapiens think about their own extinction; no other species on earth is capable of that sort of intellectual abstraction. Animals think about survival, food, and procreation. They don't ponder the universe.
All in all, the article is very interesting, but I think the writer and his subject need to watch the
Terminator franchise a little less frequently.