Category: Fun Theory

The Thing that I Protect – (Moral) Truth in Fiction?

The Thing That I Protect But still – what is it, then, the thing that I protect? Friendly AI?  No – a thousand times no – a thousand times not anymore.  It’s not thinking of the AI that gives me strength to carry on even in the face of inconvenience. So what is Yudkowsky’s highest

Continue Reading…

Category: Fun Theory

Three Worlds Collide

Three Worlds Collide (0/8)    (contains links to all parts) Set in a world where AI, nanotech and really advanced biotech is somehow impossible, Three Worlds Collide is a pretty cool sci-fi story about the encounter of alien races with differing values. (And I have to say: The story is a really great illustration of Yudkowsky’s

Continue Reading…

Category: Fun Theory

Fun Theory Conclusion: Post 31 – 34

(Mainly quotes. I didn’t want to comment all that much cause I discussed all of these issues before.) 31. Higher Purpose In today’s world, most of the highest-priority legitimate Causes are about large groups of people in extreme jeopardy.  (Wide scope * high severity.)  Aging threatens the old, starvation threatens the poor, existential risks threaten

Continue Reading…

Category: Fun Theory

Fun Theory: Post 24 – 30

24. Building Weirdtopia Yudkowsky invites the readers to write comments describing possible “weirdtopian” futures, i.e. worlds that are neither utopian nor dystopian, but pretty strange. 25. Justified Expectation of Pleasant Surprises We humans need hope. To get up every morning, we need to believe that there is at least a small chance that things will,

Continue Reading…

Category: Fun Theory

Fun Theory: Post 20 – 23

20. Emotional Involvement Can your emotions get involved in a video game?  Yes, but not much.  Whatever sympathetic echo of triumph you experience on destroying the Evil Empire in a video game, it’s probably not remotely close to the feeling of triumph you’d get from saving the world in real life.  I’ve played video games

Continue Reading…

Category: Fun Theory

Fun Theory: Post 14 – 19

14. Amputation of Destiny Yudkowsky describes a book by Ian Banks that is about a society, called the Culture, that consists of happy, intelligent, long-living humans, low-grade transhumanists, so to speak. But everything is controlled by Minds, superintelligent AIs. Yudkowsky calls this an amputation of destiny. We humans want to be the main players, we

Continue Reading…

Category: Fun Theory

Fun Theory: Post 11 – 13

11. Nonperson Predicates There is a subproblem of Friendly AI which is so scary that I usually don’t talk about it… …This is the problem that if you create an AI and tell it to model the world around it, it may form models of people that are people themselves.  Not necessarily the same person,

Continue Reading…

Category: Fun Theory

Fun Theory: Post 3 – 10

3. Complex Novelty In the book “Permutation City” by Greg Egan, apparently the favorite Sci-Fi book of Yudkowsky one of the main characters, Peer, modifies himself to find table-leg-carving utterly fascinating and enjoyable. Yudkowsky is horrified by this vision and thinks that… …at that point, you might as well modify yourself to get pleasure from

Continue Reading…

Category: Fun Theory

Fun Theory: Post 1 – 2

1. Prolegomena to a Theory of Fun Raise the topic of cryonics, uploading, or just medically extended lifespan/healthspan, and some bioconservative neo-Luddite is bound to ask, in portentous tones: “But what will people do all day?” They don’t try to actually answer the question.  That is not a bioethicist’s role, in the scheme of things. 

Continue Reading…