Question

Why aren't animals motivated only by gradients of bliss or gradients of agony? Why do emotions have sometimes positive and sometimes negative valence?

Seeking vs. avoiding

The answer seems plausibly straightforward. Some things animals are supposed to avoid (fire, predators, sharp knives, etc.), and some things they're supposed to seek out (food, shelter, mates, etc.). Avoiding something is easier for a body to do by making that specific thing painful than by making everything else in the universe except that specific thing more pleasurable. Yew-Kwang Ng claims that emotions are metabolically costly, so the body aims to conserve on their use. A parallel argument goes for pleasure: It's easier to make the specific thing to seek pleasurable than to make everything but that thing painful.

That said, there are subtleties. When you get hungry, you don't just have a lack of pleasure, but you have an active discomfort. This is sort of like "make everything painful except eating." The same can be true for other pleasures, including addictions: Sometimes it's not that the drug makes you feel good -- it's that doing anything other than taking the drug makes you feel bad. So maybe it's not inconceivable after all to imagine life as gradients of discomfort that are more or less relieved by doing certain things. Probably depressed people do feel this way. And those with a high hedonic setpoint may be closer already to gradients of bliss, although there must be some exceptions for really severe pain.

Learning

Above I've been discussing motivation, which is probably the main function of our conscious emotions and so is likely most relevant. That said, we can also look at learning with positive and negative feedback.

It turns out there's a whole literature on this topic. For example, a number of references are cited in "The effects of positive versus negative feedback on information-integration category learning."

A long history of research has investigated the relative efficacy of positive and negative feedback. For example,
early two-choice discrimination-learning studies with rats found that punishment-only training caused faster learning than reward-only training (see, e.g., Hoge & Stocking, 1912; Warden & Aylesworth, 1926). The first human studies, which used simple two-choice rule-based category learning tasks, also found that negative feedback was more effective than positive feedback (see, e.g., Buss & Buss, 1956; Buss, Weiner, & Buss, 1954; Meyer & Offenbach, 1962). More recently, however, Frank, Seeberger, and O’Reilly (2004) reported that dopamine replacement medications reversed this effect in Parkinson’s disease patients (i.e., positive feedback became more effective than negative feedback). Several researchers hypothesized that the more commonly observed negative feedback advantage occurs because positive feedback is less informative than negative feedback, at least in two-choice tasks (Buchwald, 1962; Jones, 1961; Meyer & Offenbach, 1962). The idea is that negative feedback informs the participant that his or her hypothesis was incorrect and also signals which response was correct (i.e., the other response), whereas positive feedback signals only that the response was correct (i.e., the hypothesis might have been incorrect, but, by chance, the response was correct). With more than two categories, negative feedback loses some of this advantage. This information asymmetry hypothesis was supported by results from a four-category study that found no difference between negative and positive feedback (Buss & Buss, 1956).

It's interesting to observe how Popperian science is based on negative feedback (falsification).

Some more discussion from the world of animal training. "Dog Training: Positive Reinforcement vs. Alpha Dog Methods":

Sylvia-Stasiewicz admits results can come slower with purely positive reinforcement, but says the method has even saved so-called “death row dogs” who some thought impossible to rehabilitate.

"How to use Negative Reinforcement as a Clicker Trainer":

When clicker training was first being applied to horses, a lot of the training was done using only positive reinforcement and the model of dolphin training was the one most people used. This was true of dog training too. A lot of early dog training emphasized free shaping and the animal was at liberty and able to choose to participate in the training or not. This worked very well for training many types of behaviors but was not quite the same as training a horse to be ridden. Unless horse people wanted to throw out everything and start over, this meant horse people had the interesting challenge of figuring out how to combine a positive reinforcement based training system with a negative reinforcement based training system.

(I do not necessarily endorse these articles.)

Similar considerations about positive-vs.-negative feedback apply for human learning and working.

Hedonic treadmill

We should also remember the hedonic treadmill in this discussion. Even things that we seek out can become less positive due to habituation. As mentioned before, sometimes we seek things to relieve the discomfort of not having them.

A friend gave me these comments:

I don't think the piece addresses the issues of "suffering" and "happiness" but rather the mechanisms by which organisms respond to environmental stimuli, and modify their behavior and decision-making patterns based upon that experience. Agreed that some of those stimuli may be experienced as positive and some as negative, and that correlates for the most part with whether the stimulus is or is perceived to be harmful or beneficial. However, I would argue that the terms "suffering" and "happiness" are for the most part emotional states that don't necessarily strictly correlate with the actual benefit or detriment of a stimulus. I'm thinking that something like a desire to maintain physical and emotional stasis is driven by an emotional process that filters experiences through a complex web of individual physical response, learned behavior and cultural indoctrination, resulting in the apparent paradox that one person may snort coke and experience it as "happiness" while another will experience it as "suffering".

Nonetheless, the article is interesting and the line of inquiry that it follows is essential, I think, in attempting to describe the processes by which we learn behaviors and perceive experience.

I replied:

Thanks!

My response to your first paragraph is that we can explain the reason why the basic mechanisms of feeling bad vs. good arose via the need for avoidance vs. seeking. Once those mechanisms exist, they can then be combined in many ways to create all colors of the rainbow of experience using those building blocks.

The friend agreed with this.

It's interesting to observe that different ways of dying can vary astronomically in their painfulness, even though in evolutionary fitness cost, they're not that different. For example, burning to death feels much worse than freezing to death, and freezing to death feels much worse than dying of a heart attack in the night. Why the discrepancy?

I guess it's because most of our pain sensations aren't targeted toward avoiding death specifically but toward avoiding other unpleasant things. Contact with fire is one of the most deleterious things we can do per unit time, so the pain intensity needs to be really severe. Cold is less damaging, so the negative weight isn't as strong. Evolution doesn't want to discourage hunting in the snow if that's what's needed to bring home mammoth meat.

Presumably one of the main deterrents of death is not pain but fear: Avoiding the predator before it attacks you, avoiding the high cliff, avoiding diseased food or feces, etc. Once the pain starts, it may already be too late in some cases.

There are ways to die that aren't painful, like nitrogen/argon gas or overdose on painkiller medication. These obviously don't provoke pain because they weren't common ways to die in the ancestral environment. (Nitrogen is a painful way to die for burrowing mammals because anoxia was a more common cause of death for them.) Death by fire is much worse than most ways to die not because the evolutionary cost is greater but just because evolution had other reasons to make fire excruciating and no reason to tone down the magnitude of suffering when it was at the level to cause death.

Pain is there to help you learn from your mistakes. If there's too much pain, you would learn to put too much effort into avoiding it. If there's too little, you wouldn't learn to avoid it enough. If you die, you won't learn anything at all either way, and this is all irrelevant. As such, there's no significant evolutionary pressure in any direction for how painful dying is.

Agree with most of that.

DanielLC wrote:As such, there's no significant evolutionary pressure in any direction for how painful dying is.

Sure, but there are many cases where you're on the edge of death but can be motivated to avoid it because of pain. If your leg is being eaten by a lion, pain can cause you to run away and save yourself.

It's interesting to observe that different ways of dying can vary astronomically

Sure, but there are many cases where you're on the edge of death but can be motivated to avoid it because of pain. If your leg is being eaten by a lion, pain can cause you to run away and save yourself.

I think it's the opposite. Fear motivates you to run. Physical pain motivates you to avoid putting any stress on whatever hurts. In order to avoid this, one of the things adrenaline does is dull pain.

I've never really thought much about this. Animals being eaten alive will be full of adrenaline, so their deaths likely aren't as painful as you'd at first think. Does anyone know how much adrenaline dulls pain? Or, for that matter, how bad the emotional pain would be?

Why aren't animals motivated only by gradients of bliss or gradients of agony?

I think the key to this is knowing the difference between gradients of bliss and gradients of agony. After all, utility is only defined up to affine transformation. If you took a perfect utility maximizer and gave it a utility function that was always negative vs. one translated so that it's always positive, there'd be no difference.

DanielLC wrote:I think the key to this is knowing the difference between gradients of bliss and gradients of agony. After all, utility is only defined up to affine transformation. If you took a perfect utility maximizer and gave it a utility function that was always negative vs. one translated so that it's always positive, there'd be no difference.

If nonexistence is a state accessible to the agent, then it would want to stay alive with above-zero utilities but not with below-zero utilities. If you can reduce the utility of nonexistence along with everything else, then the behavior stays the same, but the hedonic value of nonexistence is pretty fixed at zero unless you have a weird way of regarding the situation.

If nonexistence is a state accessible to the agent, then it would want to stay alive with above-zero utilities but not with below-zero utilities.

I don't think nonexistence can really be said to have a hedonistic utility in any meaningful way. You can't decide you prefer unconsciousness to X, since you can't actually experience unconsciousness.

The average rate that happiness is felt during nonexistence is 0/0. You feel zero happiness for zero subjective time.

Ok, but you could prefer for the experience you're having to not happen. If so, it's below zero.

Ok, but you could prefer for the experience you're having to not happen.

You can't compare against experiencing nothing. You can compare against the experience of sitting in an empty room in which nothing is happening, but there's no reason that has to have zero hedonic utility.

If you experience something for which you think, "I would rather that moment have been cut out of my life (i.e., fast forward to a future point in my life)," that experience was below zero. No comparison with nonexistence needed.

(Ok, this definition ignores the problem that you might fast forward just out of impatience to get to the good parts, but it's at least approximate.)

You have to have some idea of what it means to experience less. It's just another way of asking if you'd rather die. Given that nobody has any real experience with dying, it's pretty difficult to make an informed decision.

Also, when you take time discounting into account, unless you will die very soon, it's just asking if it's worse than average.