The thought that something as nuanced, complex, diverse, and ever-changing as human desires, can be captured in a single, static "reward function" sounds ridiculous to me.
And yet, people will try, and they will probably get pretty close. And then those whose desires do not fit neatly into that "reward function", will suffer.
I can already see lots of suffering caused by this type of thinking, today.
Sentences like this scare me.
The thought that something as nuanced, complex, diverse, and ever-changing as human desires, can be captured in a single, static "reward function" sounds ridiculous to me.
And yet, people will try, and they will probably get pretty close. And then those whose desires do not fit neatly into that "reward function", will suffer.
I can already see lots of suffering caused by this type of thinking, today.
reply