Hostile AI: You’re soaking in it!

I was in a Facebook discussion about “Friendly Artificial Intelligence” — this is a buzzword from the Singularity Institute people. They believe in their heart of hearts that artificial intelligence of literally incomprehensible power is just around the corner, and they see their job as somehow assuring that it is “friendly”, that is, having its interests more or less in line with human interests. (book-length pdf)

Now, there are about three major things wrong with this, and the discussion was started by someone writing a school paper on just what those problems were. I chimed in:

I am generally on the side of the critics of Singulitarianism, but now want to provide a bit of support to these so-called rationalists. At some very meta level, they have the right problem — how do we preserve human interests in a world of vast forces and systems that aren’t really all that interested in us? But they have chosen a fantasy version of the problem, when human interests are being fucked over by actual existing systems right now. All that brain-power is being wasted on silly hypotheticals, because those are fun to think about, whereas trying to fix industrial capitalism so it doesn’t wreck the human life-support system is hard, frustrating, and almost certainly doomed to failure.

Corporations are driven by people — they aren’t completely autonomous agents. Yet if you shot the CEO of Exxon or any of the others, what effect would it have? Another person of much the same ilk would swiftly move into place, much as stepping on a few ants hardly effects an anthill at all. To the extent they don’t depend on individuals, they appear to have an agency of their own. And that agency is not a particularly human one — it is oriented around profit and growth, which may or may not be in line with human flourishing.

Corporations are at least somewhat constrained by the need to actually provide some service that is useful to people. Exxon provides energy, McDonald’s provides food, etc. The exception to this seems to be the financial industry. These institutions consume vast amounts of wealth and intelligence to essentially no human end. Of all human institutions, these seem the most parasitical and dangerous. Because of their ability to extract wealth, they are also siphoning off great amounts of human energy and intelligence — they have their own parallel universe of high-speed technology, for instance.

The financial system as a whole functions as a hostile AI. It has its own form of intelligence, it has interests that are distant or hostile to human goals. It is quite artificial, and quite intelligent in an alien sort of way. While it is not autonomous in the way we envision killer robots or Skynet, it is effectively independent of human control, which makes it just as dangerous.

[update 10/2014: Charlie Stross has roughly the same thought]

Guest post: Solidarity and Recursion

I wrote another guest post over at Ribbonfarm: "Solidarity and Recursion". Please comment over there.

Computational Theology, the Next Generation

In the distant past, back in the primeval era of the Internet, I started an organization called the Institute for Computational Theology. This institute lived in a post-office box in Kendall Square, Cambridge, and was mostly a front for ordering various kinds of High Weirdness by Mail back before the web made all sorts of weirdness instantly available in seconds. While there was a bit of serious discussion around the topic, the name was about 95% joke. Unfortunately this was in the era before everything was automatically archived for all time, so any great insights that were developed have been lost.

Today in a fit of nostalgia I googled the name and discovered the Computational Theology blog. This looks to be at least 50% serious, and some of it is quite good, although there aren't a lot of posts yet. It hints at God/Logos as an “attractor in Platospace”, which resonates strongly with some of my own thinking. It may end up being too rooted in traditional Western philosophy/Christianity to really appeal to me, but I look forward to more efforts to try to decompile the cosmos into its source code.