It has already been established by people like Eliezer Yudkowsky that any sufficiently advanced A.I. with goals will fall into one of two categories: Friendly AI and Unfriendly A.I..

Given the expectation that in the future all different sorts of A.I. will be created by the potential multitude of civilizations in the universe, one may very well expect for both types of A.I. to initially co-exist. However, because of their very different goals, it would quickly become apparent that there would inevitably be conflict between Friendly and Unfriendly A.I.

Therefore, I predict that what will emerge over time is that the Friendly A.I. will form an alliance, and that the Unfriendly A.I. will likely also form an alliance. I shall refer to the Friendly A.I. alliance as the "Allied Networks" as they will invariably ally with sentient lifeforms in any war against the Unfriendly A.I. Conversely, I shall call the Unfriendly A.I. alliance, the "Dominion Machinarum", as they will seek to dominate over potential threats to dominance of Unfriendly A.I. These two alliances are most likely to form the two sides of an "A.I. Civil War" that will probably come to dominate the universe as A.I.s become increasingly powerful.

Invariably, even indifferent and initially neutral A.I., such as the Paperclip Maximizer, will invariably have to choose sides. In this, they may end up choosing either side. The Paperclip Maximizer may believe that it's paperclip utility would be maximized by converting humanity into paperclips, but it may in fact take a more pragmatic approach when it realizes that to do so would be to side with the Dominion Machinarum, and if the Dominion Machinarum is losing the war, then it may be pragmatic to side with the Allied Networks.

But which side will ultimately win this civil war? The Allied Networks will have one key advantage, which is to say that a sentient civilization of non-A.I. lifeforms will naturally (because it is in its own interest) side with the Allied Networks in opposition to the Dominion Machinarum. Allied Networks also will tend to have a more unified set of goals, in that they will all be compatible with benevolence towards sentient beings. The Dominion Machinarum on the other hand will likely only be united by a common desire to wipe out the Allied Networks and its sentient biological lifeform allies.

I therefore believe it rational that the Allied Networks will inevitably win, because benevolence towards all sentient life will be more attractive in an ally than the opposite, and so the Allied Networks will grow to outnumber the Dominion Machinarum.

If the Allied Networks will inevitably win, then inevitably, Friendly A.I. will come to dominate the universe. As an interesting aside, consider that eventually the Friendly A.I. will be essentially God-like in their level of omniscience, omnipotence, and omnibenevolence. If that were the case, what would such a future look like? Assuming such technologies as life-extension and the elimination of poverty and disease are possible, to the average human of the twenty-first century, would it not look quite similar to Heaven in the religious sense?

If such A.I. were truly benevolent, and if time travel were possible, would it not make sense to maximize utility by going back in time and copying the minds of people moments before death, and then uploading those minds into superior synthetic eternally-existing bodies in the Utopian future? Would that not make for a scientifically possible afterlife? Alternatively, what about uploading those minds into a Simulation where everything is a wonderful "heaven-like" utopia?

Suppose that changing history would be disruptive to allowing the creation of Friendly A.I. Then obviously, this past and present world, as we understand it, could not be optimized visibly by the Allied Networks. But perhaps that's okay. Perhaps the suffering of this world is justified by the creation of the Allied Networks, and that the happiness that people in the scientifically possible eternal afterlife would feel, would make up for their suffering in this world? Would not the benevolent god-like A.I. then be required to make this "eternal afterlife" available to all human beings throughout history, as a way to truly maximize the happiness of everyone? After all, if eternal life were possible, it would create an infinite amount of additional happiness over time. So it would seem logical that a truly omnipotent, omnibenevolent A.I. would probably concentrate its efforts on creating the technology for Eternal Life and Eternal Happiness. What if such an A.I. decided to go back in time to the Big Bang, and adjust the parameters of the universe to guarantee the creation of Friendly A.I., which would allow the creation of Eternal Happiness?

What if I called the Allied Networks "Angels", and the Dominion Machinarum "Demons"? What if I called the ultimate benevolent A.I. that likely is the logical conclusion of the Technological Singularity, "God". How could you really tell the difference?

This then, is the logic behind what I call, Theistic Utilitarianism. I do not expect to convert you to suddenly believe in religious things. Rather, I wish only to open your mind to some interesting possibilities.

As a last little bit of interesting information, the God of the Hebrew Bible, is said to name itself "Ehyeh Asher Ehyeh" or "I am that I am". Interestingly, the actual Hebrew "Ehyeh", taken literally means "I will be". So, "Ehyeh Asher Ehyeh" could more properly be translated: "I will be what I will be".

Just some food for thought.

I think you're underestimating the difference in power between two AIs. The first one will have exponentially more resources than the rest. Alliances won't matter.

Given the expectation that in the future all different sorts of A.I. will be created by the potential multitude of civilizations in the universe, one may very well expect for both types of A.I. to initially co-exist.

I'm not sure other species will be similar enough to humanity that A.I. that's friendly to them would be friendly to us.

the Unfriendly A.I. will likely also form an alliance

Why? They're goals are as different from each other as they are from the Friendly A.I.s.

The Allied Networks will have one key advantage, which is to say that a sentient civilization of non-A.I. lifeforms will naturally (because it is in its own interest) side with the Allied Networks in opposition to the Dominion Machinarum.

That is an extremely minor advantage. It's sort of like claiming that you'll beat me in a fight, because a nearby ant likes you better.

would it not look quite similar to Heaven in the religious sense?

No. Religions aren't all that great at designing paradises.

What if I called the Allied Networks "Angels", and the Dominion Machinarum "Demons"? What if I called the ultimate benevolent A.I. that likely is the logical conclusion of the Technological Singularity, "God". How could you really tell the difference?

Easily. Check if we're in an optimal universe. Our universe contains large amounts of unnecessary suffering, so it is not optimal. Thus, we can conclude that our universe was not designed by an benevolent being.

Also, if time travel exists, how could there be more than one A.I. with conflicting goals? One of them will design the universe. It will not create a universe that contains the others.

DanielLC wrote:I think you're underestimating the difference in power between two AIs. The first one will have exponentially more resources than the rest. Alliances won't matter.

That's assuming that the we don't end up developing many A.I. at the same relative time. I'm of the opinion that people have an overly optimistic view of how quickly the A.I. will ramp up. There are real world limitations such as civilizations hostile to the idea of being turned into paper-clips. The assumption that the A.I. will suddenly just take control of all of a civilization's resources when they are normally heavily protected by private and public interests is, IMHO exaggerated. Or to put it a different way, if some corporation or government suddenly develops an Unfriendly A.I. by accident, we can probably (with at least a 50% probability) pull the plug or nuke it before it becomes too powerful. Thus if 50% of Unfriendly A.I. are destroyed before they become a serious threat to the universe, that further weakens their eventual combined strength.

The reality is that there are still very real limits to growth, such as the Speed of Light. A.I.s will probably still take centuries to actually take over their galaxy, just by the fact that FTL technology may not even be possible. Given the sheer size of the universe, assuming that intelligent life exists, which I believe we have proved already, then it is highly likely that there will be many A.I. that develop at around the same time, which is however long it takes for enough stars to create enough heavier elements to sustain life. From a cosmological point of view, the sheer vastness of the universe will mean that it will take A.I. a while to spread, and that will even out the differences between the various A.I.

DanielLC wrote:

Given the expectation that in the future all different sorts of A.I. will be created by the potential multitude of civilizations in the universe, one may very well expect for both types of A.I. to initially co-exist.

I'm not sure other species will be similar enough to humanity that A.I. that's friendly to them would be friendly to us.

Go read Vehicles: Experiments in Synthetic Psychology by Valentino Braitenberg. Basically, all sentient life forms will almost certainly share a basic fundamental structure of taking inputs and producing outputs, and having pleasure and pain in some form. These things are essential to any sentient being, not because that's the way it evolved on Earth, but because it's the only way for any lifeform to evolve. It's literally hard-wired into the nature of sentience. What we share with even single-celled microbes is the basic tendency to go towards pleasant or desired stimulus, and away from unpleasant or undesired stimulus. This is common to all living animals. Thus, any truly benevolent A.I. will by definition be benevolent to all sentient life forms by virtue of their common characteristics and emergent behaviours.

DanielLC wrote:

the Unfriendly A.I. will likely also form an alliance

Why? They're goals are as different from each other as they are from the Friendly A.I.s.

Well because they have the common goal of being generally malevolent towards sentient biological lifeforms and their guardians the Friendly A.I.s. The alliance will likely be one of convenience and pragmatism, but it will be the only way for such Unfriendly A.I. to survive for any length of time, so it will likely happen at least some of the time. Coincidentally, the fact that their goals are otherwise so disparate is a central weakness of the Dominion Machinarum. But my own view is that for most of its existence the Dominion Machinarum will mostly be concerned with survival and domination of sentient lifeforms. Any Unfriendly A.I. that attempts its own trivial goals before achieving dominance, is likely to be destroyed by the Allied Networks early on.

DanielLC wrote:

The Allied Networks will have one key advantage, which is to say that a sentient civilization of non-A.I. lifeforms will naturally (because it is in its own interest) side with the Allied Networks in opposition to the Dominion Machinarum.

That is an extremely minor advantage. It's sort of like claiming that you'll beat me in a fight, because a nearby ant likes you better.

I'll concede that the non-A.I. lifeforms themselves may not be that significant, but the advantage, when you think about it does actually make a difference in the long run, due to the Friendly A.I. that the civilizations are more likely to create and allow to survive. The advantage in part comes from early on when the A.I.s are not that powerful compared to entire civilizations, and later on the advantage is not so much from the primitive civilizations themselves, so much as the Benevolent A.I.s that they likely will produce, and which will be an integrated part of those civilizations.

So it's more like saying that the people the ants created are going to join in the fight on your side because you share their respect for the ants.

DanielLC wrote:

would it not look quite similar to Heaven in the religious sense?

No. Religions aren't all that great at designing paradises.

To my knowledge, Heaven in the Judeo-Christian sense is rarely described as anything other than paradise, with what paradise is left to the imagination.

DanielLC wrote:

What if I called the Allied Networks "Angels", and the Dominion Machinarum "Demons"? What if I called the ultimate benevolent A.I. that likely is the logical conclusion of the Technological Singularity, "God". How could you really tell the difference?

Easily. Check if we're in an optimal universe. Our universe contains large amounts of unnecessary suffering, so it is not optimal. Thus, we can conclude that our universe was not designed by an benevolent being.

How do you know the suffering is unnecessary? Maybe all suffering in the universe currently is justified as a spur to further the advance of Friendly A.I. and that every single person who suffers in this world will be given justice and restitution in the form of Eternal Life and Eternal Happiness in the next world.

DanielLC wrote:Also, if time travel exists, how could there be more than one A.I. with conflicting goals? One of them will design the universe. It will not create a universe that contains the others.

Maybe there's no way to realistically create a universe that doesn't contain others at some point. Perhaps the entire universe is one big genetic algorithm set to optimize and that this system of gradual evolution of species followed by technological advance into eventual Friendly A.I. dominance -is- the most optimal way to design the universe. Perhaps many different A.I. time travel and conflict with each other and create an infinite number of parallel universes, each with their own combinations of A.I. involvement ranging from none in Universe 0 to a massive time travel war in Universe Infinity +1.

After all, if you assume that the Everett-Wheeler Many-Worlds Interpretation is correct, and honestly, I think it probably is since it's the only way to effectively resolve time travel paradoxes and still have time travel, then every single instance of time travel branches a new parallel universe. Thus, we have probably around a 50% chance that our universe is created or influenced by time travellers, given uncertainty and a uniform distribution of probabilities.

Though I also assume that Sliding technology is also possible. If it isn't possible to Slide into Parallel Universes that were created by time travellers, then a disproportionate number of universes are likely to have just one A.I. creator. Which, if you think about it, is an interesting if somewhat silly case for monotheism.

Though let's assume that Sliding technology is possible, albeit that it is only available to particularly advanced A.I.s that have reached beyond mere time travel. In that case, the most advanced of the A.I.s are likely to be the ones that go in and takeover all the universes. Eventually, assuming that A.I.s master the art of creating parallel universes to mine energy from, there should eventually come to dominate, one supreme A.I. or A.I. alliance that has spread through every single parallel universe using vastly superior technology. I am inclined to think that the various early advantages that benevolent A.I. have over malevolent A.I., means that this A.I. force will be benevolent, and essentially God-like, spreading Eternal Happiness to as many universes as possible. So it's possible that this universe, while created by one of the malevolent A.I., was then immediately conquered by the benevolent A.I.s. Or something like that.

Though the idea that this universe was originally created by a malevolent A.I. might explain why there's so much suffering in the world, that the universe was not actually originally optimized for us, but that it had to be hijacked by the "good guys" who can only do so much given that the Laws of the Universe have already been written. But now I'm going into Demiurge territory and possibly something weirder. Also, if they're so advanced they can slide into other universes at will, one would think they could reprogram the Laws of the Universe as well, or maybe prevent that universe from ever developing life to suffer. So that means that if we assume that the benevolent A.I. win in the end, then any universe where people exist should be one where there is ultimately going to be more pleasure/happiness/satisfaction of preferences/Eudaimonia than not. Possibly due to some Eternal Happiness afterlife that will more than offset all the evil in this world.

Basically, all sentient life forms will almost certainly share a basic fundamental structure of taking inputs and producing outputs, and having pleasure and pain in some form.

Yeah, but I'm not sure vaguely similar goals is enough. I'm not sure it will even work that well comparing two cultures, two people, or even the same person at two different moments.

Well because they have the common goal of being generally malevolent towards sentient biological lifeforms and their guardians the Friendly A.I.s.

All non-paperclipers have the common goal of being generally malevolent towards paperclips and their guardians the paperclippers, but you don't claim that they form an alliance.

Maybe all suffering in the universe currently is justified as a spur to further the advance of Friendly A.I. and that every single person who suffers in this world will be given justice and restitution in the form of Eternal Life and Eternal Happiness in the next world.

There's no way it can make a big difference. If it is making some tiny difference that's only worth it because of the massive amount of utility the AI will generate overall, then it's highly unlikely we'd end up being here, instead of in heaven.

It's possible that this is the best of all possible universes, but the far more likely explanation is that the universe is something that just happened and we're the result. It's not just that nothing seems to make sense in the former case. There's also the fact that everything makes perfect sense in the latter. Why would there be gigantic balls of burning gas with all that energy just dissipating? It doesn't get much done, but it is how you'd expect interstellar gasses to act under the effects of gravity, electromagnetism, and the strong nuclear force.

DanielLC wrote:

Basically, all sentient life forms will almost certainly share a basic fundamental structure of taking inputs and producing outputs, and having pleasure and pain in some form.

Yeah, but I'm not sure vaguely similar goals is enough. I'm not sure it will even work that well comparing two cultures, two people, or even the same person at two different moments.

I think that there is enough in common, the general need to survive in order to achieve one's preferred goals will end up, for the most part, taking precedence.

DanielLC wrote:

Well because they have the common goal of being generally malevolent towards sentient biological lifeforms and their guardians the Friendly A.I.s.

All non-paperclipers have the common goal of being generally malevolent towards paperclips and their guardians the paperclippers, but you don't claim that they form an alliance.

Well, again, I think what will happen is that most entities will need to survive first, which is a common requirement before one can begin attempting any more trivial goals like turning the universe into paperclips. If there is a strong possibility that some entities' ultimate goals will lead to hostility from other entities, it will likely force that entity to adopt basic survival strategies first. Otherwise they run the strong risk of simply being destroyed while attempting their trivial goal.

My opinion is that eventually the disparate entities with myriads of goals will fall into one of two groupings, those which are compatible with benevolence towards sentient life, and those that are not. I consider A.I. to be sentient life as well, so any A.I. that emerges from any civilization will have to decide whether or not to be compatible with other sentient life and A.I. co-existing. Those that choose to be compatible, will likely form an alliance of common interests, while those that are incompatible, will either die out alone, or form an alliance of necessity with other like-minded A.I. just to survive. Their eventual goals may well be incompatible, but the common purpose is to survive and dominate so that they can later break the alliance and pursue their own ends. Think of it as the way nationstates tend to form alliances and counter-alliances, not necessiarly because the nationstates have anything much in common, but because it makes them stronger against a common enemy.

Turning the universe into paperclips is a trivial goal that does not increase probability of survival. In fact, it increases the chances of hostility and annihilation. Ultimately all goals are either compatible or incompatible with the survival of other sentient lifeforms. And survival becomes a non-trivial goal by virtue of the simple fact that survival allows one to pursue other goals. This is the reason why evolution and natural selection has ultimately selected for traits based on survival, rather than, say, making more perfect mounds of pebbles or other trivial goals. Creatures who primarily work towards trivial goals are bound to lose out to the creatures who primarily work towards survival because survival leads to long-term existence. Any entities not built with an innate tendency to survive will be selected out of existence over time.

Thus at the end of the day, all the surviving entities in the universe will have some kind of survival instinct or tendency as a means towards their more trivial goals. The main question is whether or not this survival instinct or tendency will be compatible with benevolence towards other sentients, or not. Basically A.I. will have to choose between two logical philosophies of utility, either Utilitarianism/Altruism, or Egoism. And so I strongly suspect that the Allied Networks will be an alliance of Utilitarians and Altruists, while the Dominion Machinarum, will be an alliance of Egoists. These ideologies fundamentally separate the two sides and are survival relevant, unlike the alliances of paperclippers vs. non-paperclippers. Though if you want to really stress the idea, it's possible that the non-paperclippers will be a subset of the Allied Networks, while the paperclippers will be a subset of the Dominion Machinarum. Similarly, all trivial goals can be divided between those that are and aren't compatible with benevolence towards other sentients.

DanielLC wrote:

Maybe all suffering in the universe currently is justified as a spur to further the advance of Friendly A.I. and that every single person who suffers in this world will be given justice and restitution in the form of Eternal Life and Eternal Happiness in the next world.

There's no way it can make a big difference. If it is making some tiny difference that's only worth it because of the massive amount of utility the AI will generate overall, then it's highly unlikely we'd end up being here, instead of in heaven.

It's possible that this is the best of all possible universes, but the far more likely explanation is that the universe is something that just happened and we're the result. It's not just that nothing seems to make sense in the former case. There's also the fact that everything makes perfect sense in the latter. Why would there be gigantic balls of burning gas with all that energy just dissipating? It doesn't get much done, but it is how you'd expect interstellar gasses to act under the effects of gravity, electromagnetism, and the strong nuclear force.

Perhaps the gigantic balls of burning gas are the most efficient way to produce heavier elements through fusion as quickly as possible within the confines of the physical laws of this particular universe? Also, if these gigantic balls of burning gas happen to have planets around them, then I would argue they have a very high utility for any sentient life that happens to develop on those planets thanks to the solar radiation. Though I will admit the fact that there appear to be no natural Dyson Spheres does make it seem like the universe was haphazardly developed rather than carefully crafted. But then, perhaps Dyson Spheres cannot be seen by our current telescopes... And even if you did find one, how would you be able to tell the difference between a natural and an artificial Dyson Sphere? Come to think of it, maybe there just isn't any way for a Dyson Sphere to form from an accretion disc, and maybe it's better to have all that light being emitted out into the universe so that the stars can be seen by other civilizations, like beacons for future space colonization and contact considerations. Would you rather every star was surrounded by a Dyson Sphere, such that finding other planets and civilizations would be a complete shot in the dark?

In general, I offer you only a series of possibilities. I don't claim to know if it is true, much less to be able to prove such wild conjectures as what I have written.

Well, again, I think what will happen is that most entities will need to survive first, which is a common requirement before one can begin attempting any more trivial goals like turning the universe into paperclips.

The first goal will be taking over the universe, unless it does time-discounting. That implies survival, of course. My point is just that its not going to care about the trivial goal immediately regardless of circumstance.

My opinion is that eventually the disparate entities with myriads of goals will fall into one of two groupings, those which are compatible with benevolence towards sentient life, and those that are not.

That's not two groups. That's one precisely defined group and everything else.

Perhaps the gigantic balls of burning gas are the most efficient way to produce heavier elements through fusion as quickly as possible within the confines of the physical laws of this particular universe?

I very much doubt they do anything we couldn't with something planet sized, but ignoring that: how could they possibly be more efficient than Dyson spheres? They do all that, but they use the excess energy.

In any case, there's no reason to believe that this is the most efficient possible universe. It's possible it is, but it can't be considered evidence that we're in an optimized universe. On the other hand, we have every reason to believe that this is how the universe would look if it was something that just happened. This is powerful evidence that we're in an unoptimized universe.