If you force people to be moral, they are less likely to actually be or act moral. Morality shouldn’t be a duty. It should come from within. You should want to help other people.
OTOH people who are moral, although they actually hate being moral, are more admirable than people who enjoy being moral. But the point is not to be admirable. The point is to be moral.
524. Meetup Post
If you deal with beings that are more intelligent than you are, you won’t know which specific actions they will take. If you knew that you would be at least as intelligent as they are. But if you know their goals, you’ll expect a specific outcome, namely the realization of these goals.
E.g. if you play chess against Kasparov you won’t know which moves he will make, but you know that you’ll lose.
Similar to the last post.
527. Aiming at the Target
Intelligent beings, or more general, sufficiently powerful optimization processes try to steer reality into a specific region of possible futures. Often a very narrow region.
Discusses how to measure the power of an optimization process.
Yudkowsky defines intelligence as efficient cross-domain optimization.
In other words:
So again I claim that this – computationally-frugal cross-domain future-steering – is the necessary and sufficient meaning that the wise should attach to the word, “intelligence”.