i came across this yesterday http://boxcar2d.com/

as far as i understand it randomly generates the cars at first and then based on the ones that get further along the track it makes ones similar to those ones and gradually they get better and better, if you start it and then go and do something else and come back in while it will have gotten significantly better.

but it doesnt actually know anything right? it just works like a group of bacteria, the ones that are most succesfull pass on their genetic material, being successful is defined as getting far along the track.

so it appears to learn, the same way bacteria are said to become resistant.

so i was wondering is the way norns appear to learn done in a similar way or is it something different? because surely whats going on here is more like what bacteria do than what cats or pigs or humans (or maybe insects) do?

What makes you think humans don't learn that way?

Humans have other ways of learning, but the one that deals with happiness and sadness is pretty much this.

oh do you mean that its like one organism each car it creates is like an experience and then it records which ones work well and repeats them?

i was thinking of it like a group of organisms as i explained above.

do you mean that these are both the same? or that what the program is doing is the former thing i mentioned as in that each car is like an experience?

Each car is like an experience.

This is true for both human experiences and norn experiences.

Nice demo.

The cars were built with an evolutionary algorithm, but I think what goes on in humans and other animals is a bit different. In particular, the dopamine system closely resembles TD learning. This is distinct from evolutionary-type selection, although both are optimization processes.

I think the cars aren't conscious, not because they don't use TD learning but because they don't have richer brain processes that create a sense of "self" with memories, desires, reactions, etc. That part probably takes a lot more programming.