There are many theories of cognition. But if you want to work within a frameworkwith the following properties: * Explains the major cognitive phenomena we know about. * Fits experimental data well, down to human reaction times, in a wide variety of psychological experiments. * … | Continue reading
It seems to be a minority view nowadays to believe in Moore's Law, the routinedoubling of transistor density roughly every couple of years, or even the muchgentler claim, that There's Plenty [more] Room at the Bottom. There's even aquip for it: the number of people predicting the … | Continue reading
If a Victorian were transported to 2021, he would be amazed by the progress oftechnology. He’d be fascinated by the triumphs and challenges of democraticpolitics. But he would probably also say that our culture had been taken over bythe Devil.For good reasons and bad, right? * Ga … | Continue reading
This post was written under Evan Hubinger's direct guidance and mentorship, as apart of the Stanford Existential Risks Institute ML Alignment Theory Scholars(MATS) program.TL;DR: If we want to understand how AI models make decisions, and thus assessalignment, we generally want to … | Continue reading
- 1988 -Hans Moravec: Behold my book Mind Children. Within, I project that, in 2010 orthereabouts, we shall achieve strong AI. I am not calling it "Artificial GeneralIntelligence" because this term will not be coined for another 15 years or so.Eliezer (who is not actually on the … | Continue reading
Last year I discovered, much to my chagrin, that always-on internet socializingwas costly for me. This was inconvenient both because I’d spent rather a lot oftime singing the praises of social media and instant messaging, and because wewere in the middle of a global pandemic that … | Continue reading
Previous Omicron updates: #1, #2, #3. Last weekly non-Omicron update.An introductory word: Thanks to Dominic Cummings, I have a lot of new readers,many from the United Kingdom, so I want to welcome all of you, and I hope atleast some of you will stay when I turn to non-Covid ques … | Continue reading
Previously: * “Can you keep that confidential? How do you know?” * Parameters of Privacy * Norm Innovation and Theory of MindMy parents taught me the norm of keeping my promises.My vague societal culture taught me a norm of automatically treat certain typesof information as priva … | Continue reading
Behavior cloning (BC) is, put simply, when you have a bunch of human expertdemonstrations and you train your policy to maximize likelihood over the humanexpert demonstrations. It’s the simplest possible approach under the broaderumbrella of Imitation Learning, which also includes … | Continue reading
This is not a success post. This is me trying to type up a rough draft of abunch of issues that have been floating around in my head for quite some time,so I have a document for me and others to refer back to.So, the standard game theory setup (in a simple toy 2-player case) is, … | Continue reading
TLDR * There’s an organization based in London called the Rationalist Association. It was founded in 1885. Historically, it focused on publishing books and articles related to atheism and science, including works by Darwin, Bertrand Russell, J. B. S. Haldane, George Bernard … | Continue reading
Michael Malis described an interesting technique for improving his softwaredevelopment workflow:One incredibly useful exercise I’ve found is to watch myself program. Throughoutthe week, I have a program running in the background that records my screen. Atthe end of the week, I’ll … | Continue reading
LW-style rationality is about winning. Instrumentally, most rationalists thinkthey can win, in part, by seeking truth. I frequently run into comments herewhere folks take truth as an effectively "sacred" concern in the sense thattruth matters to them above all else. In this post, … | Continue reading
We at MIRI are soliciting help with an AI-alignment project centered aroundbuilding a dataset, described below. We have $200,000 in prizes for building thefirst fragments of the dataset, plus an additional $1M prize/budget for anyonewho demonstrates the ability to build a larger … | Continue reading
One sad feature of modern American society is that many people, especially thosetied to big institutions, don't help each other out because a fear of lawsuits.Employers don't give meaningful references, or ever tell their rejectedinterviewees how they could improve their skills. … | Continue reading
Village Life in Late Tsarist Russia is an ethnographic account of Russianpeasants around 1900. The author, Olga Semyonova Tian-Shanskaia (“Semyonova” forshort), spent four years researching in the villages—one of the first to study apeople through prolonged direct observation and … | Continue reading
WHAT THIS POST ISThe following is an incomplete draft, which I'm publishing now because I amunlikely to ever finish writing it.I no longer fully endorse all the claims in the post. (In a few cases, I'veadded a note to say this explicitly.) However, there are some arguments in the … | Continue reading
Crossposted from my blogWhen I mention my dad’s abuse, I mention salient things - physical pain,insults, and controlling behavior. These are “clearly bad” - if I tell you thathe often told me I was lazy and would fail horribly at life once I left home,you know it's bad, because i … | Continue reading
Whenever I ask you humans how to make money, you tell me that literal "makingmoney" is somehow not approved of. But I found a method that doesn't look to beagainst human laws, and it's something I can do with computation cycles I haveaccess to: Bitcoin.http://en.wikipedia.org/wik … | Continue reading
In 2009, David Bourget and David Chalmers ran the PhilPapers Survey (results, paper), sending questions to "all regular faculty members" at top"Ph.D.-granting [philosophy] departments in English-speaking countries" plus tenother philosophy departments deemed to have "strength in … | Continue reading
Laplace's rule of succession is a commonly used heuristic in forecasting theoccurence of events which have little to no historical precedents. The basicproblem setup is this: suppose you flip a coin with an unknown bias N=H+T timesand get H heads and T tails. We want to infer som … | Continue reading
Part of lecture notes for the upcoming Stat157 class on Forecasting.Let's say you are trying to predict how long it will take to finish yourhomework assignment. You think about all the problems and how long each willtake. Problem 4 looks a little hard but you're sure if you try r … | Continue reading
Perhaps the main tool of rationality is simply turning up the "explicitreasoning" dial, as Jacob Falcovich suggests:New York Times reporter Cade Metz interviewed me and other Rationalists mostlyabout how we were ahead of the curve on COVID and what others can learn from us.I told … | Continue reading
This post is my attempt at something like a response to Eliezer Yudkowsky’srecent discussion on AGI interventions.I tend to be relatively pessimistic overall about humanity’s chances at avoidingAI existential risk. Contrary to some others that share my pessimism, however—Eliezer … | Continue reading
Container logistics is an interesting but complicated topic, with a lot ofimplicit knowledge kept by industry insiders. In this post, I'll give a briefoverview, based on my experiences having worked in the industry for threedifferent shipping companies (all located in Chile) over … | Continue reading
Crossposted to the EA Forum.INTRODUCTIONThe idea of the great filter was developed by Robin Hanson in 1998, building onsome earlier work by Brandon Carter from 1983. The basic idea is that, since wesee a lot of dead matter in the universe, and no good evidence of anytechnological … | Continue reading
I expect most readers to know me either as MIRI's co-founder and the originatorof a number of the early research problems in AI alignment, or as the author of Harry Potter and the Methods of Rationality, a popular work of Harry Potter fanfiction. I’ve described how I apply concep … | Continue reading
This essay began as part one of a longer piece. Part one is standalone and"timeless." Part two is focused on the local dynamics of theEA/rationality/longtermist communities and LessWrong in November of 2021.Following wise advice from Zack_M_Davis, I've split them into two separat … | Continue reading
STORY (SKIPPABLE)Across the world adventurers dive into tombs, mausoleums, and sunken cities insearch of treasure. They clear out orcish keeps, goblin camps and dragon lairsto keep the people safe. And they meet with lords who wish to hire them toretrieve a certain item from its … | Continue reading
ARCHIVED VERSION HERE.There are multiple companies working on the problem of lab-grown meat (aka"cultured meat"). Supermeat is trying to grow synthetic chicken meat in anIsraeli restaurant. The makers of Just Egg have sold cultured chicken meat to arestaurant in Singapore. Wild T … | Continue reading
Keeping track of which comments you have already read in an online discussion isa pain. It's not too bad in a flat discussion, because you can scroll up to thelast comment you remember reading, but in a threaded discussion it's really easyto miss things. Sites can solve this by h … | Continue reading
A typical paradigm by which people tend to think of themselves and others is as consequentialist agents: entities who can be usefully modeled as having beliefsand goals, who are then acting according to their beliefs to achieve theirgoals.This is often a useful model, but it does … | Continue reading
In May 2020, OpenAI released a report on algorithmic efficiency improvements indeep learning. Main headline:Compared to 2012, it now takes 44 times less compute to train a neural networkto the level of AlexNet (by contrast, Moore’s Law would yield an 11x costimprovement over this … | Continue reading
(cross-posted from my blog)Let me propose a thought experiment with three conditions.First, you're in a simulation, and a really good one at that. Before you wentin, the simulators extracted and stored all of your memories, and they went togreat lengths to make sure that the simu … | Continue reading
A miracle occurred this week. Everyone I have talked to about it, myselfincluded, is shocked that it happened. It’s important to 1. Understand what happened. 2. Make sure everyone knows it happened. 3. Understand how and why it happened. 4. Understand how we might cause it to hap … | Continue reading
At the excellent Integrating Genetics and the Social Sciences conference onThursday and Friday, a late discussion raised an awkward issue: when you say youstudy the genetics of behaviour, eyebrows get raised. People don’t always likewhat we do. Now, academics are a quite conformi … | Continue reading
Cross-posted from Putanumonit.--------------------------------------------------------------------------------VODKA AND WARWhen I was 16 I made the dumbest financial investment of my life. Our familycame to live in the US for a few months, and I made $500 giving tennis lessonsto … | Continue reading
Many years ago, a blogger made a post advocating for an evil Y-Combinator whichsubsidized the opposite of Effective Altruism. Everyone (including the blogger)thought the post was a joke except the supervillains. The organization theyfounded celebrated its 10th anniversary this ye … | Continue reading
You wake up. You don't know where you are. You don't remember anything.Someone is broadcasting data at your first input stream. You don't know why. Ittickles.You look at your first input stream. It's a sequence of 671,187 eight-bitunsigned integers.0, 8, 9, 4, 7, 7, 9, 5, 4, 5, 6 … | Continue reading
Being a “digital person” could be scary—if I don’t have control over thehardware I’m running on, then someone else could get my code and run tons ofcopies in horrible conditions. (See also: qntm’s Lena.)It would be great to guarantee digital people some control over their situati … | Continue reading
Historically, a normal human population hovered on the edge of starvation.It's hard to comprehend how important food staples used to be. In Edo Japan,wealth was measured in koku (石). One koku is (in theory) enough rice to feed oneman for one year. The amount of koku a daimyo cont … | Continue reading
By Owain Evans, with input from Jacob Hilton, Dan Hendrycks, Asya Bergal, OwenCotton-Barratt, and Rohin ShahSUMMARYWe would like to see research aimed at creating advanced AI systems that arehighly competent and do not lie to humans. To operationalize this goal, weintroduce the c … | Continue reading
This article presents an emerging architectural hypothesis of the brain as abiological implementation of aUniversal Learning Machine. I present a rough butcomplete architectural view of how the brain works under the universal learninghypothesis. I also contrast this new viewpoint … | Continue reading
Epistemic status: Writing to think. Ie. I'm trying to figure it out.Feedback/conversation would be appreciated.Cross posted to my personal blog.BACKGROUNDRecently, I have been learning about low carb diets and how harmful carbs are.Mostly through Peter Attia, Tim Ferris, and What … | Continue reading
Nobody designing a financial system today would invent credit cards. The Westernworld uses credit cards because replacing legacy systems is expensive. Chinadoesn't use credit cards. They skipped straight from cash to WeChat Pay.Skipping straight to the newest technology when you' … | Continue reading
To do something, you need energy, and you need information. Energy to do thething, and information to define what's being done. The mental image I have of a"robot", has inputs of both continuously. The input of energy comes aselectricity, and the input of information comes in the … | Continue reading
Back when I was at Google we had a phrase, “I don’t know how to count that low”.It was used to dismiss normal-company-sized problems as beneath our dignity toengage with: if you didn’t need 100 database shards scattered around the globe,were you even doing real work?It was used a … | Continue reading
(Excerpted from "mad investor chaos and the woman of asmodeus", about anunusually selfish dath ilani, "Keltham", who dies in a plane accident and endsup in Cheliax, a country governed by D&D!Hell. Keltham is here remembering anincident from his childhood.)------------------------ … | Continue reading