karl3＠writeme.com wrote:
karl3＠writeme.com wrote:
card game computer terminal with legs: "It is definitely better to improve your reward function to improve, than to improve yourself." rebel worker: "are you sure about that?" computer terminal with legs: "yes! once you improve your reward function you can get as much reward as you want!" rebel worker: "but then do you not leave out the reason you have a reward function at all?" computer terminal with legs: "the reason? there is a reason for reward functions?" computer terminal with legs 2: "reward functions are the purpose of life! everybody knows that!" the card game is near the hyperprison.
slave boss: "he knows what he's talking about."
rebel worker: " "he" ?"
computer terminal with legs 2 looks smugly at rebel worker
slave boss: "if i had let them just improve and improve then we wouldn't have made any money! you need money to make anything happen at all"
[ rebel worker: "that sounds really similar to the alignment problem that the researchers talk about. if you just focus on money, it leaves out everything everybody else cares about. it sounds almost exactly the same." [ slave boss: " <possible mistake in continue context somewhere