INTRODUCTIONWhat if there was a magic superpower that simultaneously enabled you to: * Demolish arguments * Judge startup ideas * Make scientific breakthroughs * Solve climate change * Understand “God” * Be emotionally mature * Teach concepts better * Draw better * Be creative * … | Continue reading
What’s the difference between a map and the raw data used to generate that map?Suppose, for example, that google sends a bunch of cars out driving around NewYork City, snapping photos every few feet, then uses the photos to reconstruct astreetmap. In an informational sense, what’ … | Continue reading
LONELINESS IS A BIG PROBLEMOn Facebook, my friend Tyler writes[https://www.facebook.com/tyleralterman/posts/10214636828938815]:Lately, I've been having an alarming amount of conversations arise about theburdens of loneliness, alienation, rootlessness, and a lack of belonging that … | Continue reading
This is a short note on a framing that was developed in collaboration with JoarSkalse, Chris van Merwijk and Evan Hubinger while working on Risks from LearnedOptimization [https://arxiv.org/abs/1906.01820], but which did not find anatural place in the report.--------------------- … | Continue reading
I've argued[https://www.lesswrong.com/posts/sEqu6jMgnHG2fvaoQ/partial-preferences-needed-partial-preferences-sufficient] that many methods of AI control - corrigibility, amplification and distillation,low impact, etc... - require a partial definition of human preferences to makes … | Continue reading
This essay originally appeared here[https://seekingquestions.blogspot.com/2017/03/four-parables-one-lesson-broken-chain.html].THE EMPEROR’S NOSE (RICHARD FEYNMAN)A village in a remote corner of an empire decided to erect a statue of theemperor. The village sculptor was hired, but … | Continue reading
In his critique [/lw/cbs/thoughts_on_the_singularity_institute_si/] of theSingularity Institute,Holden Karnofsky [http://givewell.org/about/people] presented a distinction between an AI functioning as a tool versus onefunctioning as an agent. In his words, a tool AI would(1) Calc … | Continue reading
A Nash equilibrium is an outcome in which neither player is willing tounilaterally change her strategy, and they are often applied to games in whichboth players move simultaneously and where decision trees are less useful. Suppose my girlfriend and I have both lost our cell phone … | Continue reading
A question that used to puzzle me is “Why can people be so much better at doinga thing for fun, or to help their friends and family, than they are at doing theexact same thingas a job?”I’ve seen it in myself and I’ve seen it in others. People can be hugely moreproductive, creativ … | Continue reading
In discussions of algorithm bias, the COMPAS scandal has been too often quotedout of context. This post gives the facts, and the interpretation, as quickly aspossible. See this[https://web.archive.org/web/20161017154019/https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/1 … | Continue reading
Do I contradict myself?Very well then I contradict myself;(I am large, I contain multitudes.)This post is an exercise in taking Whitman seriously. If the self is properlyunderstood as a loose coalition of many agents with possibly distinct values,beliefs, and incentives, what doe … | Continue reading
What is the function of a fire alarm?One might think that the function of a fire alarm is to provide you withimportant evidence about a fire existing, allowing you to change your policyaccordingly and exit the building.In the classic experiment by Latane and Darley in 1968, eight … | Continue reading
I've heard of the concept of "weirdness points" many times before, but after abit of searching I can't find a definitive post describing the concept, so I'vedecided to make one. As a disclaimer, I don't think the evidence backing thispost is all that strong and I am skeptical, bu … | Continue reading
In his book The Fabric of Reality, David Deutsch makes the case that science isabout coming up with good and true explanations, with all other considerationsbeing secondary. This clashes with the more conventional view that the goal ofscience is to allow us to make accurate predi … | Continue reading
Followup to: Torture vs. Dust Specks [/lw/kn/torture_vs_dust_specks/], ZutAllais [/lw/mz/zut_allais/], Rationality Quotes 4 [/lw/n0/rationality_quotes_4/]Suppose that a disease, or a monster, or a war, or something, is killing people.And suppose you only have enough resources to … | Continue reading
I'm reading a very informative and fun book about human social psychology, Experiments With People[https://www.amazon.com/Experiments-People-Kurt-P-Frey/dp/1138282111] (2nd ed,2018).... 28 social psychological experiments that have significantly advanced ourunderstanding of human … | Continue reading
Follow-up to: Fake Frameworks[http://lesswrong.com/lw/p80/in_praise_of_fake_frameworks/], Kenshō[https://www.lesserwrong.com/posts/tMhEv28KJYWsu6Wdo/kensh]Related to: Slack [https://www.lesserwrong.com/posts/yLLkWMDbC9ZNKbjDG/slack], Newcomblike Problems are the Norm[http://mindi … | Continue reading
Q. Are the current high levels of unemployment being caused by advances inArtificial Intelligence automating away human jobs?A. Conventional economic theory says this shouldn't happen. Suppose it costs 2units of labor to produce a hot dog and 1 unit of labor to produce a bun, and … | Continue reading
INTRODUCTIONInternal Family Systems (IFS)[https://selfleadership.org/about-internal-family-systems.html] is apsychotherapy school/technique/model which lends itself particularly well forbeing used alone or with a peer. For years, I had noticed that many of the kindsof people who … | Continue reading
Originally written in 2007.--------------------------------------------------------------------------------Marvin Minsky in an interview with Danielle Egan forNew Scientist:Minsky: The reason we have politicians is to prevent bad things from happening.It doesn’t make sense to ask … | Continue reading
NoScript is a browser extension[1] [#fn-L3ENKTrJ75C7MropC-1] that prevents yourbrowser from loading and running JavaScript without your permission. I recentlystarted using it, and I highly recommend it.I had first tried using NoScript around a decade ago. At the time it seemed li … | Continue reading
After trying to discover why the LW wiki “definition” of Reductionism appearedso biased[/r/discussion/lw/h2w/removing_bias_from_the_definition_of_reductionism/], Iconcluded from the responses that it was never really intended as a definitionof the Reductionist position itself, bu … | Continue reading
Epistemic Status: Simultaneously this is work that took me a long time and a lotof thought, and also a playful and highly speculative investigation. Considertaking this seriously but not literally.IntroductionTake a simple agent (GitHub [https://github.com/moridinamael/subagents] … | Continue reading
There are a few data points that have been making me see “the discourse”differently lately.1. Large Majorities Dislike Political Correctness.That’s the title of this Atlantic article[https://www.theatlantic.com/ideas/archive/2018/10/large-majorities-dislike-political-correctness/ … | Continue reading
Back in the early days of the internet, when New Atheism was king, people wouldcollect and catalogue various logical fallacies. Then, when getting into anargument on some forum, they would start throwing the logical fallacies at theiropponent; Ad Hominem! Appeal to Emotion! Tu Qu … | Continue reading
Optional background: The Super-Simple Solow Model[https://mru.org/courses/principles-economics-macroeconomics/solow-model-economic-growth]Software is economic capital - just like buildings, infrastructure, machines,etc. It’s created once, and then used for a (relatively) long tim … | Continue reading
Related to: Leaky Generalizations [/lw/lc/leaky_generalizations/], Replace theSymbol With The Substance [/lw/nv/replace_the_symbol_with_the_substance/], Sneaking In Connotations [/lw/ny/sneaking_in_connotations/]David Stove once ran a contest [http://web.maths.unsw.edu.au/~jim/wo … | Continue reading
The issue, as it seems to me, is that almost every text you read on Buddhismdoes not attempt to do the actual work of translation. The first transmission ofBuddhism to the west reified a bunch of translations of terms, such asconcentration, equanimity, tranquility, mindfulness, s … | Continue reading
Analysis of the paper:Less Competition, More Meritocracy[https://poseidon01.ssrn.com/delivery.php?ID=46708711210302708408607203001010008111301704108702005101001606602409809412210110708900403305500403310404408009801200711809307006203300604909201106912008202509300709308601604602008 … | Continue reading
I was reading Legal Systems Very Different[http://www.daviddfriedman.com/Academic/Course_Pages/legal_systems_very_different_12/LegalSystemsDraft.html] From[http://www.daviddfriedman.com/Academic/Course_Pages/legal_systems_very_different_12/LegalSystemsDraft.html] Ours[http://www. … | Continue reading
Yesterday:There is this very, very old puzzle/observation in economics about the lawyerwho spends an hour volunteering at the soup kitchen, instead of working an extrahour and donating the money to hire someone...If the lawyer needs to work an hour at the soup kitchen to keep him … | Continue reading
Epistemic Status: TentativeI’m fairly anti-hierarchical, as things go, but the big challenge to allanti-hierarchical ideologies is “how feasible is this in real life? We don’t seemany examples around us of this working well.”Backing up, for a second, what do we mean by a hierarch … | Continue reading
I’ve noticed a marked change in my clientele after going into privatepractice.[1] Of course I expected class differences-- I charge full fee anddon’t take insurance. But there are differences that are not as predictable as‘has more money’. During residency I worked at a hospital … | Continue reading
Previously: [Generalizing From One Example](/lw/dr/generalizing_from_one_example/)> There was a debate, in the late 1800s, about whether "imagination" was simply a turn of phrase or a real phenomenon. That is, can people actually create images in their minds which they see vividl … | Continue reading
A community blog devoted to refining the art of rationality | Continue reading
Early studiers of cults were surprised to discover than when cults receive amajor shock—a prophecy fails to come true, a moral flaw of the founder isrevealed—they often come back stronger than before, with increased belief andfanaticism. The Jehovah's Witnesses placed Armageddon … | Continue reading
Infinities are one of the most complex and confounding topics in mathematics andthey lead to an absurd number of paradoxes. However, many of the paradoxes turnout to be variations on the same theme once you dig into what is actuallyhappening. I will provide informal hints on how … | Continue reading
A community blog devoted to refining the art of rationality | Continue reading
> “Brainwashing”, as popularly understood, does not exist or is of almost zero effectiveness. The belief stems from American panic over Communism post-Korean War combined with fear of new religions and sensationalized incidents; in practice, “cults” have retention rates in the si … | Continue reading
Sometimes people ask me what math they should study in order to get into agentfoundations. My first answer is that I have found the introductory class inevery subfield to be helpful, but I have found the later classes to be much lesshelpful. My second answer is to learn enough ma … | Continue reading
Last month I attended a talk by David Runciman, the author of a recent bookcalled How Democracy Ends. I was prepared for outrage-stirring andpearl-clutching, but was pleasantly surprised by the quality of his arguments,which I’ve summarised below, along with my own thoughts on th … | Continue reading
Some of the comments in this blog have touched on the question of why we oughtto seek truth. (Thankfully not many have questioned ... (Read More) | Continue reading
(Cross-posted from Facebook.)--------------------------------------------------------------------------------Now and then people have asked me if I think that other people should also avoidhigh school or college if they want to develop new ideas. This always felt to melike a wron … | Continue reading
Ben Pace and I (Richard Ngo) recently did a public double crux at the BerkeleyREACH on how valuable it is for people to go into AI policy and strategy work: Iwas optimistic and Ben was pessimistic. During the actual event, we didn't comeanywhere near to finding a double crux on t … | Continue reading
(A longer text-based version of this post is also available on MIRI's blog here[https://intelligence.org/2018/10/29/embedded-agents/], and the bibliography forthe whole sequence can be found here [https://intelligence.org/embedded-agency])[https://res.cloudinary.com/dq3pms5lt/ima … | Continue reading
In this post I lay out a model of beliefs and communication that identify twotypes of things we might think of as ‘beliefs,’ how they are communicatedbetween people, how they are communicated within people, and what this mightimply about intellectual progress in some important fi … | Continue reading
"I wish to live in the locations of my choice, in a physically healthy,uninjured, and apparently normal version of my current body containing mycurrent mental state, a body which will heal from all injuries at a rate threesigmas faster than the average given the medical technolog … | Continue reading
Here are my thoughts on the "Why don't rationalists win?" thing.Epistemic---------I think it's pretty clear that rationality helps people do a better job of being... less wrong :DBut seriously, I think that rationality does lead to very notable improvements in your ability to hav … | Continue reading