After watching a talk by Stuart Armstrong it struck me that I had drastically underestimated the force of the atronomical waste argument. Previously I had considered the utility of colonizing the Milky Way. However, in the light of his talk colonization of billions of galaxies did not seem very unlikely (say below a 0.01 probability). This adds a whole bunch of orders of magnitude.

Now it seems to me that even this estimate is an underestimate and rather than being a drastic underestimation it is an astronomical underestimation. The reason is that a standard total utilitarian should consider not only a few of the most likely scenarios and their values but also add together "a tail" of all other scenarios with a probability above 0 and their utilities. In my epistemic state prior to hearing about Armstrong's talk I should reasonbly at least have factored in, say a 0.1 probability of colonizing tens of galaxies, perhaps a 0.001 probability of colonizing billions of galaxies etc, since I had no reason to think that there are no possible ways to effectivize colonization. These estimates should then be updated when new evidence on ideas or methods for colonization are available.

Even spending a little time on this it seems like the expected utility provided by this "tail" would be enormous. For example, consider the possibility of travel across arbitrary distances (one interpretation of the "wormhole" idea in science fiction). Assuming space is infinite, If such a technology and the technology Armstrong describes in his talk is possible (self-replication von-neumann probes that can be powered by a star and use planets around the star to create massive solar power arrays) an exponential expansion that would last as long as the stars would be possible. After replicating a and harnessing the power of one star the probe could use the power to create, say 10, new probes and send through separate wormholes to distant parts of the universe. Assuming that stars will continue to shine for 10 billion years and that this cycle could be completed in 10 years (Armstrong calculated 40 years to strip mine mercury to construct an enormous solar powered mass driver to hurl billions of von neumann probes towards other galaxies at speeds close to c) then it follows that the number of probes (and thus, colonized galaxies) would increase tenfold every ten years and Bostrom's original Astronomical waste calculation should be increased by about *a billion orders of magnitude*.

I think it is fair to say that this number is not astronomical, but *astronomically astronomical*.

Is this scenario likely? Probably not. But I have a hard time justifying a subjective probability of less than, say, 1/10000 that such super-luminar travel and the other conditions are possible. So in a naive expected utility calculation the expected utility of reducing x-risk would still be approximately a billion orders of magnitude.

Also note that wormhole-travel is just one possible technology. Other, perhaps more far fetched, technologies that could lead to far greater utilities than this include time travel and finding some renewable source of energy and the creation of lab-universes.

Is this just another example of Pascal's mugging/wager? Perhaps. But I think that an important point is that in such a case we cannot draw any clear boundary between pascalian and non-pascalian cases. There is a continuum of more and more "pascalian" cases, starting with low probability/high impact actions such as eating vegan or donating money to effective altruism, the chance of affecting a moderate space expansion, a smaller affecting colonization of the milky way, an even smaller colonization of colonizing billions of galaxies, of colonizing astronomical numbers, of getting unbounded amounts of utility all the way up to infinite and higher-order-infinite outcomes.

(If you are a negative utilitarian an analogous argument could be stated about the astronomically astronomical importance of reducing future suffering).

Well this seems correct. Though unintuitive, the overwhelming moral consideration of a symmetric (as opposed to negative) hedonistic or preference utilitarian, is existential risk.

RyanCarey wrote:Though unintuitive, the overwhelming moral consideration of a symmetric (as opposed to negative) hedonistic or preference utilitarian, is existential risk.

It should be overwhelming for negative utilitarians as well, except that they have a clearer answer on the direction of their actions. As a symmetric hedonistic utilitarian, I still don't know whether I should assume that this probably quite different, nonhuman life would feel more good than bad, on average.

Hedonic Treader wrote:As a symmetric hedonistic utilitarian, I still don't know whether I should assume that this probably quite different, nonhuman life would feel more good than bad, on average.

This relates to what I think is the most interesting and overlooked feature of the existential risk argument. Contrary to what Bostrom seems to believe, we don't need to assume or argue that future sentients will have lives that are positive on the whole. Even if we are uncertain about the sign value of colonization, we might be in a position to know that this value, whether positive or negative, will be astronomical. Such knowledge might itself warrant the conclusion that concern for existential risk should dominate our thinking. However, the practical conclusion is not that we should do everything we can to reduce existential risk, but that we should do everything we can to know whether existential risk ought to be reduced or increased--and then act accordingly.

By the way, here's the talk Jesper refers to in his comment above.

Thanks for the link, Pablo. The talk is interesting throughout. The connection to the Fermi paradox seems highly relevant, since if it were indeed so (comparatively) easy to colonize far-away galaxies, we would expect others to have done it by now.

Without having much technical expertise on intergalactic travel, I would venture a guess: It will turn out that crossing these vast distances and times with functional technology will turn out to be impossible. So will FTL, by fundamental physical limitations. Life is hardy, and self-replication is a strong concept, but the probes can't self-replicate in intergalactic space - they would have to be functional for millions of years, traveling at such high speeds that any dust speck impact is highly disruptive. And then follow up with complex processes like deceleration, resource seeking and self-replication. This is even harder since all galaxies are targeted directly, i.e. there isn't even a self-replication step in jumping over a path of nearby galaxies. I would guess this is incredibly hard to do even between stars. And we don't have any proof of concept of complex technology that doesn't lose functionality over thousands of years, let alone hundreds of millions. This seems like the most down-to-earth explanation for the Fermi paradox.

Hedonic Treader wrote:Thanks for the link, Pablo. The talk is interesting throughout. The connection to the Fermi paradox seems highly relevant, since if it were indeed so (comparatively) easy to colonize far-away galaxies, we would expect others to have done it by now.

You are welcome. Actually Jesper is the one that one should thank. He sent me a link to this talk a few weeks ago.

Hedonic Treader wrote:This seems like the most down-to-earth explanation for the Fermi paradox.

This is an interesting speculation. However, a more parsimonious explanation is that we are the only intelligent, technologically-advanced civilization in the accessible universe. The only apparent evidence that intelligent life exists elsewhere is that it exists here, coupled with some principle about the uniformity of nature. But an observation selection effect guarantees that we will observe intelligent life, regardless of how prevalent it is in the rest of the cosmos. So, as far as I can see, there is actually no reason to believe that intelligent extraterrestrials exist. We haven't seen other aliens simply because there aren't any.

Pablo Stafforini wrote:But an observation selection effect guarantees that we will observe intelligent life, regardless of how prevalent it is in the rest of the cosmos. So, as far as I can see, there is actually no reason to believe that intelligent extraterrestrials exist.

Well, we have one reason: We know that intelligent life can develop naturally in our universe under conditions that are probably not unique. How rare earth really is, how rare self-replicating organic molecules really are, how rare the evolutionary steps to technological intelligence really are, is not certain - but the sheer number of potentially suitable star systems for this to happen still make it a plausible hypothesis. In contrast, we don't have a proof of concept of any complex self-replicating system that can survive functionally for millions of years in hostile environments without the regenerative intermediary step of self-replication, using outside resources and energy to counter local entropy. Given this, it isn't clear to me why I should assume the great filter is behind us.

Interesting idea. However, it seems it only helps against the Armstrong-Fermi paradox - not the original paradox concerning our own galaxy, or at least that it is less plausible for the intra-galaxy fermi.

Hedonic Treader wrote:Well, we have one reason: We know that intelligent life can develop naturally in our universe under conditions that are probably not unique. How rare earth really is, how rare self-replicating organic molecules really are, how rare the evolutionary steps to technological intelligence really are, is not certain - but the sheer number of potentially suitable star systems for this to happen still make it a plausible hypothesis.

It seems to me, however, that given how little we know about the process by which life originated and intelligent life evolved on Earth, this line of argument differs little from Eleanor Arroway's answer to the question of whether she thought there were intelligent extraterrestrials out there: "I don't know, but I guess I'd say if it is just us... seems like an awful waste of space." I mean, on what basis can we be warranted in saying that the conditions under which life emerged on Earth are "probably not unique". And if we are not warranted in saying this, is there any justification for saying that "the sheer number of potentially suitable star systems for this to happen still make [the existence of intelligent extraterrestrial life] a plausible hypothesis"?

I'm really not an expert on the probabilities of the respective filter steps. We have found an awful lot of extrasolar planets recently, and complex organic molecules have been confirmed in outer space. Evolution by natural selection seems a fairly robust principle once self-replicating molecules can store structural information. Maybe the conditions are so unlikely that it was really just a fluke that happened once in billions of galaxies. But this would imply a seriously low combined probability for the steps of intelligent life's emergence.

However, if it happens relatively rarely, we might be the only ones in the Milky Way, while everybody else is simply too far away to interact with us. There could be billions of civilizations, each isolated.

How probable is intergalactic travel? How probable is detectable (from other galaxies) engineering? If someone used radio waves 100 million light years away, would we even detect it over the noise?

Do you have informed numbers on the probabilities of where to assume the filter(s)?

I just found a video where Nick Bostrom outlines his position on the Fermi paradox. This is exactly what I myself believe (though I don't have any firm beliefs in this area).

I'm more skeptical. As minimal intelligence species, betting the unique in the galaxy, I assume a much stronger filter, in the future. In this century, before von Neumann probes.