This post felt like it ended too soon? Seemed like the author never got around to explaining why he (she?) fell out with the rationalists
kenforthewin 2 hours ago [-]
It's a shame because I think there a lot of valid, thoughtful critiques one could level at the Rat/EA community; but this was not that.
danielhaven 1 hours ago [-]
They got to him before he could finish.
stevage 59 minutes ago [-]
I often have this experience with blog posts on HN. They do all the background, and once they get to the juicy bit, it ends.
gwd 53 minutes ago [-]
They did. The entire section starting with "Of course, despite the emphasis on reason — and the subtle implication that the out-group is more wrong — the rationalist community is susceptible to the same biases as anyone else" talks about the failure of the rationalist community to be any better than any other group.
Then this describes the actual break:
> By that time, my relationship with the skeptic community was already strained; in an earlier message to the same mailing list, I said that for all the claims of being enlightened, it’s the only place where stonings are still in fashion. Even so, when I saw the Damore email, I had a brief urge to respond. I wanted to accuse him of “motivated reasoning” — a purported fallacy of constructing an argument to advance a specific conclusion, in contrast to the divine, unmotivated reasoning of real rationalists. I bit my tongue, thought to myself “wow, what a stupid game to play”, and deleted the draft.
In other words, they had been dissatisfied with the whole rationalist way of arguing for a while; and this was the point where they finally felt like "This is actually just a game of words; and it's a dumb one I don't want to play any more." The implication being that after this they no longer considered themselves a rationalist, nor engaged in its form of arguing.
ETA: FWIW, I mean... hard things are hard? It's the normal state of the world that people think they've done the hard thing but are actually just finding excuses to do what they want. People like J.D. Vance decide to be a Catholic, but turns out Jesus' actual teachings are so counter-intuitive that he still doesn't understand them and does them badly. Actually being rational is a difficult, mostly solo activity.
Quitting rationalism (or Christianity, or ...) because most of the people around you are just cosplaying is like quitting chess because most of the people around you are just playing around and never get past 1200 ELO. Or like quitting weightlifting because most people around you talk about reps and whatever but never actually push themselves enough to get past their current plateau. Like, OK, that's their problem, but what's that got to do with you?
oh_my_goodness 12 minutes ago [-]
Yes. It's almost as if the author felt that long, long, long chains of explanations might be somehow undesirable.
actsasbuffoon 2 hours ago [-]
Yeah, that felt abrupt.
Leave the audience wanting more, I guess?
zahlman 48 minutes ago [-]
It comes across to me that there's a lot of culture warring in it that's irrelevant to the task of actually figuring out what's true. Bringing up Damore and "sealioning" within a few paragraphs is more than enough to establish for me that this is not about trying to have any kind of constructive discussion.
Aside from that, it's curious how I've been interested in these ideas (and following SSC) for like a decade and this is the first I've heard of the castle. But I do hear about the Against Malaria Foundation constantly from that direction, and I'm pretty sure they've received more than 15 million from self-styled "aspiring rationalists" and "effective altruists" (although these really aren't meant as identity labels and you're doing it wrong if you think of yourself that way; pg's "Keep your identity small" represents solid rationalist thinking).
nradov 2 hours ago [-]
Rationalism is just a secular religion, complete with all the usual arguments and schisms over minor points of dogma. These are not people worth taking seriously.
mrandish 15 minutes ago [-]
I agree with you about the tech-adjacent rationalist/EA community in the bay area (especially since around 2010). While that community originally coalesced around reason and evidence-based skepticism, those are uncontroversial ideas that've been around since the enlightenment.
Where the bay area rationalists went off the rails was in extrapolating far beyond those ideas to other things and then applying their new orthodoxy in rigidly dogmatic ways, eventually becoming 'a movement' with schisms, in-group/out-group policing, deference to influential members and ostracism for heterodox views.
zahlman 2 hours ago [-]
There isn't actually such a thing as "rationalism". This is a label applied from outside to those who are trying to apply the concepts of rationality, or those who try to get into rationality but don't actually understand it.
Yes, there are schisms. No, there is not dogma. There are arguments people don't want to rehash, and there are fundamental axioms of reasoning.
The entire point of the exercise is to establish that certain things are true, and that this is not because of faith, and that disbelieving them cannot make them false.
pdonis 1 hours ago [-]
> The entire point of the exercise is to establish that certain things are true
And any domain where you can do that is not the real world. The real world is messy and you can't prove things the way you can in nice, tidy, theoretical domains like math. And the kind of logic where you have fundamental axioms of reasoning and you can prove theorems is just a subset of math. In the real world, things don't exactly satisfy the axioms, and boundaries are fuzzy, not sharp. Trying to pretend otherwise, while it might not strictly be "dogma" in the usual religious sense, is just as counterproductive.
tcfhgj 19 minutes ago [-]
Trying to ignore reality doesn't sound rational.
s1artibartfast 5 minutes ago [-]
Im not sure what your criticism is directed at.
In the real world, we can study and predict things. We can study the weather and predict if it will rain tomorrow. We can asses how accurate our predictions have been in the past.
We can evaluate if it is true that smoking increases lung cancer, or that aspirin increases autism.
Sometimes we dont have good answers or predictions, and that too can be studied.
jml7c5 48 minutes ago [-]
>This is a label applied from outside
I don't think this is accurate. Most rationalists seem to self-describe as rationalists, and talk about rationalism as a "thing".
There is some dogma, sort of, in the form of The Sequences. (An extraordinarily goofy name that belies some of the mysticism-adjacent LARP that sought to treat the ideas as revolutionary. I suspect a 20 year old was behind it.) I say "sort of" dogma because people are aware that the studies referenced in there turned out not to replicate and have generally abandoned them. But you still see some references to them and vague reverence of them.
zahlman 44 minutes ago [-]
> I suspect a 20 year old was behind it.
Yudkowsky wrote them from 2006-2009, so he would have been in his late 20s.
> because people are aware that the studies referenced in there turned out not to replicate
The Sequences are about pure reasoning, not about results from "studies". They are very much about how to analyze studies and about the fact that you should care whether they replicate.
In fact, I can't even recall any such "studies referenced".
perching_aix 28 minutes ago [-]
We probably understand this word very differently, because as far as I'm concerned, rationality (rational behavior) is essentially an exercise in optimality evaluation, one that is thus strictly relative to some particular context. It is not just "facts and logic", and it is most certainly not universal, since universal optimality doesn't exist: https://en.wikipedia.org/wiki/No_free_lunch_theorem
> The entire point of the exercise is to establish that certain things are true, and that this is not because of faith
Taken at face value, that'd mean bridging the analog hole (https://en.wikipedia.org/wiki/Analog_hole), which is fundamentally impossible. Logic proves consistency, not truth, despite the many related terminologies suggesting otherwise. This is further skipping over how people segment the world different, even relative to themselves over time, which thus forms a coordination problem.
It reminds me to how HTTP is stateless. Yeah, it's stateless - all the state keeping has simply been laundered a layer above. It may sound tantalizing to position certain arguments as simply rational (i.e. correct/optimal), but all that ultimately does is sidestep the actual coordination part, which never really went away. There is still an underlying segmentation model being asserted, it is just now coated in self-righteousness, an illusion of rigor, drawing up a false dichotomy of "either you segment the world like we do, and then you're correct, or you're incorrect, and your segmentation is too".
You do realize you sound exactly like a cult member right? Like that's precisely how a scientologist would explain auditing to people outside the cult. Like this is "not dogma" just "fundamental axioms" that "is not because of faith, and that disbelieving them cannot make them false".
zahlman 1 hours ago [-]
> You do realize you sound exactly like a cult member right?
If you say so.
That doesn't change the fact that P(A|B) = P(B|A)P(A)/P(B).
lyu07282 10 minutes ago [-]
There is nothing I could possibly say to convince you otherwise, in the same EXACT way I couldn't convince a scientologist that their thetan levels are nonsense. It was never the argument that conditional probability or other mathematics is wrong, nobody is ever arguing that, but that is all you disciples ever argue against. You are all deeply uneducated and unserious people believing to "reason" yourselves to some higher "truth" on all kinds of broad topics using some WILDLY misappropriated and misunderstood mathematical principles.
bigbadfeline 44 minutes ago [-]
> That doesn't change the fact that P(A|B) = P(B|A)P(A)/P(B).
It didn't take long before a strawman got pulled out of nowhere. That's so typical, if one disagrees with the wild conjectures of the rationalists, which they call "axioms", they blame you for rejecting the Pythagorean Theorem or some such. Dishonesty is the defining feature of that "movement"
zahlman 23 minutes ago [-]
> It didn't take long before a strawman got pulled out of nowhere.
It's not a strawman. It's a central example of what I had in mind when I used the word "axiom".
> if one disagrees with the wild conjectures of the rationalists, which they call "axioms"
They don't refer to wild conjectures as axioms.
> they blame you for rejecting the Pythagorean Theorem or some such.
There is no "blame" going on here. Someone said I sounded like a cult member (which incidentally is insulting and inappropriate on HN; but I thought I would try to steer the discussion back on course) and I correctly explained why that perception is irrelevant. When I said
> The entire point of the exercise is to establish that certain things are true, and that this is not because of faith, and that disbelieving them cannot make them false.
That is, in fact, inarguably true. Bayes' Law is an example of that exact sort of thing. If you don't believe that conditional probabilities can be worked out that way, that won't change the fact that conditional probabilities can in fact be worked out that way; and it won't change the fact that in the long run I will be able to exploit your fallacy in games of chance.
The other poster gave no example of supposedly false or unsound axioms, so I cannot be strawmanning by failing to respond to them. My assertion is that such false or unsound axioms are not, in fact, in play. The burden of proof lies with those claiming their existence.
alephnerd 2 hours ago [-]
Very much this.
I've been to a couple rationalist events in the Bay during peak rationalism (originally with an open mind but eventually resorted to gawking after people couldn't handle "but why..." style questions and prodding which is the norm in my line of work and past academic experience) and it was truly weird seeing supposedly well educated people fall for such obvious cult-like traps.
In hindsight, I guess there might have been an overrepresentation of neurodivergence in the community as well, and for some rationalism's simple precepts simply worked - though arguably you can integrate axiomatic thinking and formal logic with traditional religion as well (eg. Nicomachean Ethics or Ashtadhyay).
I am a massive supporter for data-driven decision making (heck, much of my undergrad was touching this kind of stuff), but mental models are meant to be tools not a crutch, and overindexing on them causes one to lose sight of the forest when amongst trees.
I'd also refute the notion on HN that the movers and shakers of the industry are overrepresented in the community - I actually found the rationalist community to be somewhat marginal in the industry as a whole but overrepresented among a specific type of chronically online individual. Indoor climbing, pickleball, and creative arts (eg. Pottery) are much more representative.
smitty1e 2 hours ago [-]
It's not impossible that everything in the universe can be explained within the scope of human reason.
Life just seems to exceed that scope, is all.
ip26 1 hours ago [-]
It's like modeling a living cell in a computer building upon the basic maths of physics. Yes, it is a thing that is "possible". However it is not practicable to capture everything that matters and run that simulation.
bigbadfeline 10 minutes ago [-]
> It's like modeling a living cell in a computer building upon the basic maths of physics.
During COVID computer modeling of cell chemistry became an established mode of study, in-silico is now used before in-vitro with great results.
One has to be careful to not fall in the trap presented as an alternative to the rationalist one - it sounds like, "there's no truth" or "the truth is too hard, just give up and accept what you're told to accept".
The lack of truth isn't due to it being hard or unachievable, there's enough of it to solve all major problems, it's not being pursued because of conflicts of interest.
popalchemist 2 hours ago [-]
Indeed, it is literally an ideology.
api 2 hours ago [-]
[flagged]
lyu07282 2 hours ago [-]
Though much more destructive to society than Scientology, it's the cult that is underpinning Technofeudalism, these are people (almost) as vile as Ayn Rand.
api 1 hours ago [-]
I'd say rationalism and technofeudalism/neoreaction are adjacent more than directly linked. They seemed to develop in parallel around the same time and involved regulars on some of the same boards.
22 minutes ago [-]
sxp 2 hours ago [-]
> For one, their analyses almost always amounted to post hoc justifications of the things they liked.
"Almost always" is carrying a lot of weight here. Rationalists are ideologically diverse [1] when it comes to things like monogamy vs polyamory, left vs right, pro-AI vs anti-AI, nature vs nurture, racism vs anti-racism. The distribution is different from the general population, but many people have changed their minds about various aspects of their life after discussions and debates with rationalists.
> What’s less known is that the memo was originally sent to the skeptics@google.com mailing list, which I was a member of. It was a personal belief masquerading as a rationalist inquiry that tried to pull everyone into an unwanted workplace argument.
One of Damore's mistakes was opting for a high-decoupling route on a politically charged topic instead of a low decoupling route. That tends to end badly and he learned this the hard way.
That graph looks incredibly un-diverse. Assuming it's accurate, it's mostly childless white male Americans, who skew heavily towards being students.
lux-lux-lux 1 hours ago [-]
Well-educated young middle class white men, and with politics that generally flow entirely predictably from those demographics. The only real outlier is the openness to HBD/scientific racism stuff.
pavel_lishin 1 hours ago [-]
> The only real outlier is the openness to HBD/scientific racism stuff.
I'm not really sure that "openness to racism" is an outlier in a population of a bunch of straight white men.
lux-lux-lux 49 minutes ago [-]
They’re usually reluctant to admit it, though.
zahlman 42 minutes ago [-]
And?
The entire point is that true statements are true, and false statements are false, regardless of who does or doesn't utter them.
alephnerd 1 hours ago [-]
[dead]
xhevahir 1 hours ago [-]
> One of Damore's mistakes was opting for a high-decoupling route on a politically charged topic instead of a low decoupling route.
Why does the formation of correct beliefs, a topic people have been discussing for millennia, require such exotic jargon? It's things like this that give off cult vibes.
zahlman 40 minutes ago [-]
Because, as the incident with Damore amply demonstrated, trying to use common language to describe these concepts leads to enormous misunderstandings at best. (And sometimes people go out of their way to misinterpret you anyway.)
anon-3988 19 minutes ago [-]
> Another source of bias is that rationalists are necessarily selective in the arguments they want to have. In everyday life, they rely on intuition and anecdotes the same as anyone else; but if they want to debate your life choices, they will do so forcibly and under the guise of seeking scientific truth. The behavior is pervasive enough to have its own name: sealioning.
So they are not rationalist then? I would have assume a rationalist would be...well..rational. And that would mean to include emotions into discussion about human and people. This is my problem with the classical Consequentialism or Utilitarianism, they always discount emotions. I'm sorry, does emotion exists or not? Is it real? Then it should be brought into discussion.
If I am in a math conference to discuss the Riemann Hypothesis with my cunt ex-wife in the room, I would get effected. It does not really matter how rational I am, I have to consider that my brain will not be fair towards her ideas.
If my brain is wired to be racist and that I truly have whispers of racism in my head, that should be part of the discussion whether or not I am qualified for this position. You claim to NOT have any such inclination, AT ALL? Okay, how likely is that given the human nature, what percentage should I give that you are lying? That should included in the discussion.
All of this is too much to think about? That is also part of the calculation, perhaps we should utilize some shortcut.
I don't understand why people exclude things from the universe of things to consider.
ggm 2 hours ago [-]
This kind of furore happens more than you would think amongst nominatively deterministic rationalist groups. Randi broke with people and re-formed, Dawkins has unbelievably heated arguments over frankly nothing.
Maybe some of it is informed by the overlap with academia. Science has the aphorism "physics advances one funeral at a time" and certainly being on the outer has affected careers. Fred Hoyle was too argumentative, many of his ideas like panspermia have turned out to have elements of plausibility in them (like the cometary origins of water)
2 hours ago [-]
TheOtherHobbes 1 hours ago [-]
I was always amused that the rationalists threw themselves behind a professional trickster with absolutely no scientific credentials.
And Dawkins never had much of a research career.
My problem with the "skeptics" was that the paranormal is an easy target and politically it's punching down.
There are plenty of other irrationalities to consider - like how advertising and propaganda work, how cults and religions actually operate beyond "irrational bad", and how trad econ believes many things that are questionable at best - but the skeptics never dared go anywhere near those.
So skepticism turned into this kind of knockabout point-scoring media circus noise. And in the end the sneering and self-congratulatory condescension actually helped entrench modern conspiranoia instead of deflating it.
ggm 1 hours ago [-]
Randi was very good at debunking other tricksters. That bootstrapped a lot of respect.
pdonis 1 hours ago [-]
> Dawkins never had much of a research career.
Perhaps not, but I have found his popular books like The Selfish Gene and The Ancestor's Tale very informative.
Unfortunately, when he ventures outside the domain of his professional expertise, he doesn't do as well. For example, in The God Delusion, which is basically a book-length diatribe against orthodox religious belief, he comes out at one point with this:
Let me sum up Einsteinian religion in one more quotation from Einstein himself: “To sense that behind anything that can be experienced there is a something that our mind cannot grasp and whose beauty and sublimity reaches us only indirectly and as a feeble reflection, this is religiousness. In this sense I am religious.”
If we grant for the sake of argument that typical orthodox religious cant is BS, this is just as much BS. The phrase "let him who is without sin cast the first stone" comes to mind.
Planktonne 2 hours ago [-]
I have very little patience with the regular appearance of articles that follow the below flow.
1. I believe I'm very smart
2. I fell down a rabbit hole into a subculture filled with people who also think they are smart
3. Eventually I realised that they weren't that smart, and were actually most just a fad + spite (+ generally bigotry)
4. I worked this out because I'm very smart, and now I'm here to tell you, people who never fell for it, that that subculture isn't all it claims to be
5. Wow
It never, ever comes with the realisation that maybe the problem is the initial flawed belief and the smugness inherent in it.
cogman10 2 hours ago [-]
Falling for a cult is something that can happen to almost anyone. I think it's somewhat healthy for one to recognize they were in a cult and to share the experience of realizing that and getting out. It may come off as smug, but it's also a pretty vulnerable thing to do as you are admitting "I was duped".
The main smugness one can have is thinking they are immune to falling for a cult. Lot's of people do for lots of reasons. It's particularly likely when someone is in a vulnerable position in life (death, depression, major life change). Sometimes people are simply born into the cults and realize later that everything their family and community believes is a lie. Regardless, cults are pretty good at finding and "loving" people into the cult.
All this is to say that everyone should take a moment to question their core beliefs. Why do I believe this? Could I be wrong? How would I know I'm wrong? Questions I know I wish I asked and explored earlier in life. The honest answers, though, can be devastating if this part of your identity or family identity.
Planktonne 2 hours ago [-]
I fully agree with everything you've said, but crucially, I don't think this article, or the others like it, are actually recognising their own vulnerability.
The framing isn't 'I fell for this and I was wrong', but 'I am the first one f all of you to see the truth'; there's no humility, no recognition, just the exact same self-delusion, which means they'll fall into the next pit head-first as well.
cogman10 2 hours ago [-]
Fair enough. I just have a soft spot for people recognizing that they've been duped. It happened to me personally and I know it's a pretty difficult thing to go through. So I just tend to give a lot of grace even though the person might be an asshole as they deconstruct.
2 hours ago [-]
twic 2 hours ago [-]
> Timecube was an incoherent stream-of-consciousness website filled with bizarre cosmological claims.
Was? Timecube.net will outlive any and all faddy Silicon Valley cults.
vovavili 2 hours ago [-]
This seems like an awfully uncharitable and cherry-picked treatment of a community of well-intentioned and occasionally eccentric disagreeable online nerds. You're effectively just noticing that a community that actively aims to overcome bias is not free from bias itself, but no one within that community would ever deny that.
cogman10 2 hours ago [-]
I think the issue with the community is they take their eccentric beliefs as being gospel truths which can lead to truly unhinged and irrational beliefs and actions (see: the Zizians).
That's what can make the rationalist movement feel more like a religious cult and less like a bunch of eccentrics. As an outsider looking in, it seems like the entire movement is filled with just accepting priors with very little evidence for why those priors should be accepted.
FeepingCreature 2 hours ago [-]
Consider a community filled with 100 people; 95 of them who don't take their eccentric beliefs as gospel truths and 5 who do. Those 5 will inevitably become the public face of the community when they make the news. However there is very little that the 95 could have done to stop them.
joegibbs 23 minutes ago [-]
Doesn't it say something if 5% of a group are wacko lunatics if, for instance, a random sample of the rest of the population has only 0.5% wackos?
cogman10 2 hours ago [-]
> However there is very little that the 95 could have done to stop them.
How about the 95% stop having gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?
How is the 5% supposed to know that the 95% is actually larping belief? And how is an outsider who observes the 95% espousing their whacky beliefs supposed to know who is a member of the 95% and who is a member of the 5% (and if those percentages are accurate).
You can observe the same dynamic in any church. Everyone in the congregation will have different levels of belief, but when it comes to sharing a testimony you'll be hard pressed to distinguish between who the true believers are and who's just "faking it until you make it".
zahlman 2 hours ago [-]
> gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?
What on Earth are you referring to? What gatherings? What beliefs? What proclamations? What evidence of insincerity?
cogman10 1 hours ago [-]
> What gatherings?
Rationalist have groups meetup and conferences. For example [1]
> What beliefs?
It's pretty diverse and not uniformly held. Some of the more strange ones that seem to have pretty wide appeal are around AI safety. For example, Roko's basilisk.
> What proclamations? What evidence of insincerity?
I was responding to the comment above me that suggested that 95% don't actually hold rationalist beliefs and are instead simply having fun with it. Only a small 5% are actually sincere.
Okay, but those pieces don't establish your claim. You would need a Roko's basilisk-like concept that only 5% of the community agree with, and a "group meetup" or "conference" where "they all" (i.e.: also the 95%) "proclaim that their beliefs (i.e.: including the basilisk thing) are true".
If I observed such a gathering knowing that 5% of the people there have sincerely come to a particular conclusion based on the available evidence and the other 95% think the evidence doesn't support that conclusion, I would be extremely surprised to see everyone going around "larping belief". And I don't think you can actually show such a thing occurring.
cogman10 43 minutes ago [-]
Which claim exactly did I make which you are refuting? Please actually quote it.
> I would be extremely surprised to see everyone going around "larping belief". And I don't think you can actually show such a thing occurring.
I never claimed I could or that I observed such a thing. The 95/5% didn't come from me, it came from the person I responded to.
My point was that
- It's impossible to distinguish who's a true believer and who isn't.
- That the meetups of the rationalists re-enforce beliefs, particularly in true believers. Also that these beliefs of the rationalists can be pretty whacky and based on a jenga tower of priors.
- That there isn't "nothing" a rational person could do to dissuade someone from a whacky belief.
zahlman 32 minutes ago [-]
> Which claim exactly did I make which you are refuting? Please actually quote it.
When you said:
> How about the 95% stop having gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?
you implicitly claimed that "the 95%" indeed have such gatherings.
As far as I can tell, the gatherings that they have are not of that nature.
> My point was that It's impossible to distinguish who's a true believer and who isn't.
And my claim is that it's actually trivial to tell who's a true believer and who isn't, because the concept of "true believer" does not pertain to "rationality" (by construction!) but to fringe beliefs, and people who are doing rationality correctly (which I expect to be the large majority) simply don't profess the fringe beliefs (because doing so is inherently antithetical to rationality). Therefore, the fringe 5% are the 5% of people actually saying the fringe thing, and the 95% don't actually say the fringe thing. Trivial.
For example, you won't catch Scott Alexander demanding "donate to MIRI or the basilisk might reach back in time and torture you".
> That the meetups of the rationalists re-enforce beliefs, particularly in true believers. Also that these beliefs of the rationalists can be pretty whacky and based on a jenga tower of priors.
And you have done nothing to establish this, and I believe you're incorrect.
cogman10 15 minutes ago [-]
> you implicitly claimed that "the 95%" indeed have such gatherings.
> As far as I can tell, the gatherings that they have are not of that nature.
I literally cited you an example of such a gathering. Do you refute it or just deny that it exists?
> And my claim is that it's actually trivial to tell who's a true believer and who isn't, because the concept of "true believer" does not pertain to "rationality" (by construction!)
That's your mistake. The rationalists are irrational. Roko's basilisk, which I cited earlier, is an example of the sorts of ideas and beliefs they get up to which are irrational.
Your claim "This group is rational because they call themselves such" is unfounded. Trivially so. Just like the Christian Scientists aren't scientists just because it's part of their name.
> For example, you won't catch Scott Alexander demanding "donate to MIRI or the basilisk might reach back in time and torture you".
I never claimed universality. In fact, in response to you I explicitly said that the beliefs are diverse. Being able to cherry pick someone that's more rational than others and a part of the group doesn't prove that there aren't very influential members of the group who are generating a lot of fringe ideas. Specifically Eliezer Yudkowsky, the founder of LessWrong which is a core part of the rationality movement.
> And you have done nothing to establish this, and I believe you're incorrect.
I've established the meetups exist. I've established that a particularly nasty group came from these meetups, the Zizians. Specifically the founder got their start from the LessWrong forums and the "Center for Applied Rationality" [1] workshops (meetups).
The entire point of rationalism is to take your own beliefs seriously and follow them as if you were a robot.
(Except that they don't claim them to be "their own beliefs" but instead call them "priors".)
g-b-r 2 hours ago [-]
It's not the bias the problem, it's the conviction of being better and "less wrong" than the rest of us.
That, along with constantly calling yourself rational and being well-intentioned, easily leads to cultish beliefs to be pursued despite how wacky they seem at their face, because it was your superior brains that led you to them.
If we add the often good (or enormous) financial resources, it makes for a seriously dangerous cult.
zahlman 2 hours ago [-]
> It's not the bias the problem, it's the conviction of being better and "less wrong" than the rest of us.
There is not any such conviction. This is just your interpretation of a domain name, which is not intended that way.
> That, along with constantly calling yourself rational
This is explicitly not what you are supposed to do, and the point is explicitly not to create an identity around it.
There’s a wonderful essay about Objectivism called “The Unlikeliest Cult in History” that goes quite a bit into that. Although I’m still all for tearing down the systems of tribalism that form around mystical beliefs, it’s delusional to think that doing so will somehow result in universal peace and harmony or even a net increase in rational thinking.
twic 2 hours ago [-]
It's not just that they aren't free of bias. They appear to incubate truly wacky stuff - Leverage Research [1], the Zizians, all the AI/x-risk stuff. I am not the only person to have noticed this [2]. There's something about their approach that appears to actually make people more susceptible to nonsense, not less.
>There's something about their approach that appears to actually make people more susceptible to nonsense, not less.
It would be highly surprising if an open-access online community with the kind of slogans that might be particularly appealing to somewhat spectrumy individuals did not occasionally attract some eccentrics. The point I am trying to make here is that these are pretty fringe offshots and are far from the representative views of the community as a whole. Appeals to reason don't make any community immune to human nature. After all, their main place of gathering is named "less wrong", not "correct".
joegibbs 30 minutes ago [-]
The problem is the whole idea of working from first principles to try to discover the actual truth, rather than just going along with other ideas that other people have created through history. The benefit of going along with other peoples' ideas is that they've been tested out for hundreds or thousands of years and been found to not be that destructive.
If you start reasoning from first principles you can very easily make an incorrect assumption somewhere down the line that snowballs until you're talking about shrimp welfare, or making decisions based on the potential sextillion humans that exist in 1,000,000 AD, or planning to convert the entire universe into unthinking organic matter that can only experience bliss.
FeepingCreature 2 hours ago [-]
Raising existential risk from AI in the same sentence as "rationality demands we murder our landlord" is a bit weird a grouping.
zahlman 37 minutes ago [-]
Asking "why are there so many rationalist cults?" (as the URL does) in this context is kind of like asking "why are there so many mathematical cranks?", and then holding forth that this somehow demonstrates a fundamental problem with mathematics.
cindyllm 36 minutes ago [-]
[dead]
hollerith 29 minutes ago [-]
>actually make people more susceptible to nonsense
I think it is more that teenagers and very young adults and people high in the trait of "openness to new experiences" are more susceptible to nonsense and more attracted to new intellectual movements, of which the rationalist community is one.
Also, I expect a rationalist to be lower in conscientiousness than average (because the conscientious people are too busy to devote a lot of time to reading long intellectual texts) and low conscientiousness makes violence and other transgressive behavior more likely.
krapp 2 hours ago [-]
Regarding the Zizians, I recommend the Behind the Bastards series on them and the rationalist community. People involved with the rationalists will want to distance themselves from it of course, but the series makes a decent argument that the Zizians were simply the result of taking the ideology to its rational conclusion, that there is a definite throughline from one to the other in terms of philosophy and ideals.
I’ve interacted with a number of rationalists that are just as crazy as Ziz but managed to not implode with the help of money (either lucking into it via family/crypto or a tech job). My personal hot take is that if she’d been cis a number of people would still be alive.
mbgerring 1 hours ago [-]
The people who sincerely believe that humans should be turned into mulch for the Computer God are “well meaning”? Say more
2 hours ago [-]
jdw64 2 hours ago [-]
Every community has its biases, and everywhere there's a tendency for complex factors to combine and simplify into attractive patterns.
In the end, even I can't escape bias. And when I realize how ignorant I am, I feel like I have no choice but to live face to face with how pathetic I am.
Even if I'm aware of my bounded rationality and ignorance, there are still moments when I fail to recognize even that ignorance. I feel like I just have to live with the awareness of how contradictory a being I am
This is a pretty low-quality article that didn't give me the impression the author spent any time in the "rationalist community." It's just a list of the few mainstream controversies.
g-b-r 2 hours ago [-]
If you say so. I'm sure you're impartial about the matter.
throwawayffffas 2 hours ago [-]
> Of course, as a rational person, you wanted to side with the scientists; if nothing else, it seemed like the winning move.
The rational person does not engage with the argument. Because there is nothing to be gained from the argument.
The winning move is not to play.
Rendered at 00:25:54 GMT+0000 (Coordinated Universal Time) with Vercel.
Then this describes the actual break:
> By that time, my relationship with the skeptic community was already strained; in an earlier message to the same mailing list, I said that for all the claims of being enlightened, it’s the only place where stonings are still in fashion. Even so, when I saw the Damore email, I had a brief urge to respond. I wanted to accuse him of “motivated reasoning” — a purported fallacy of constructing an argument to advance a specific conclusion, in contrast to the divine, unmotivated reasoning of real rationalists. I bit my tongue, thought to myself “wow, what a stupid game to play”, and deleted the draft.
In other words, they had been dissatisfied with the whole rationalist way of arguing for a while; and this was the point where they finally felt like "This is actually just a game of words; and it's a dumb one I don't want to play any more." The implication being that after this they no longer considered themselves a rationalist, nor engaged in its form of arguing.
ETA: FWIW, I mean... hard things are hard? It's the normal state of the world that people think they've done the hard thing but are actually just finding excuses to do what they want. People like J.D. Vance decide to be a Catholic, but turns out Jesus' actual teachings are so counter-intuitive that he still doesn't understand them and does them badly. Actually being rational is a difficult, mostly solo activity.
Quitting rationalism (or Christianity, or ...) because most of the people around you are just cosplaying is like quitting chess because most of the people around you are just playing around and never get past 1200 ELO. Or like quitting weightlifting because most people around you talk about reps and whatever but never actually push themselves enough to get past their current plateau. Like, OK, that's their problem, but what's that got to do with you?
Leave the audience wanting more, I guess?
Aside from that, it's curious how I've been interested in these ideas (and following SSC) for like a decade and this is the first I've heard of the castle. But I do hear about the Against Malaria Foundation constantly from that direction, and I'm pretty sure they've received more than 15 million from self-styled "aspiring rationalists" and "effective altruists" (although these really aren't meant as identity labels and you're doing it wrong if you think of yourself that way; pg's "Keep your identity small" represents solid rationalist thinking).
Where the bay area rationalists went off the rails was in extrapolating far beyond those ideas to other things and then applying their new orthodoxy in rigidly dogmatic ways, eventually becoming 'a movement' with schisms, in-group/out-group policing, deference to influential members and ostracism for heterodox views.
Yes, there are schisms. No, there is not dogma. There are arguments people don't want to rehash, and there are fundamental axioms of reasoning.
The entire point of the exercise is to establish that certain things are true, and that this is not because of faith, and that disbelieving them cannot make them false.
And any domain where you can do that is not the real world. The real world is messy and you can't prove things the way you can in nice, tidy, theoretical domains like math. And the kind of logic where you have fundamental axioms of reasoning and you can prove theorems is just a subset of math. In the real world, things don't exactly satisfy the axioms, and boundaries are fuzzy, not sharp. Trying to pretend otherwise, while it might not strictly be "dogma" in the usual religious sense, is just as counterproductive.
In the real world, we can study and predict things. We can study the weather and predict if it will rain tomorrow. We can asses how accurate our predictions have been in the past.
We can evaluate if it is true that smoking increases lung cancer, or that aspirin increases autism.
Sometimes we dont have good answers or predictions, and that too can be studied.
I don't think this is accurate. Most rationalists seem to self-describe as rationalists, and talk about rationalism as a "thing".
There is some dogma, sort of, in the form of The Sequences. (An extraordinarily goofy name that belies some of the mysticism-adjacent LARP that sought to treat the ideas as revolutionary. I suspect a 20 year old was behind it.) I say "sort of" dogma because people are aware that the studies referenced in there turned out not to replicate and have generally abandoned them. But you still see some references to them and vague reverence of them.
Yudkowsky wrote them from 2006-2009, so he would have been in his late 20s.
> because people are aware that the studies referenced in there turned out not to replicate
The Sequences are about pure reasoning, not about results from "studies". They are very much about how to analyze studies and about the fact that you should care whether they replicate.
In fact, I can't even recall any such "studies referenced".
> The entire point of the exercise is to establish that certain things are true, and that this is not because of faith
Taken at face value, that'd mean bridging the analog hole (https://en.wikipedia.org/wiki/Analog_hole), which is fundamentally impossible. Logic proves consistency, not truth, despite the many related terminologies suggesting otherwise. This is further skipping over how people segment the world different, even relative to themselves over time, which thus forms a coordination problem.
It reminds me to how HTTP is stateless. Yeah, it's stateless - all the state keeping has simply been laundered a layer above. It may sound tantalizing to position certain arguments as simply rational (i.e. correct/optimal), but all that ultimately does is sidestep the actual coordination part, which never really went away. There is still an underlying segmentation model being asserted, it is just now coated in self-righteousness, an illusion of rigor, drawing up a false dichotomy of "either you segment the world like we do, and then you're correct, or you're incorrect, and your segmentation is too".
See this rhetorical landmine in action in a sibling thread: https://news.ycombinator.com/item?id=48986530
If you say so.
That doesn't change the fact that P(A|B) = P(B|A)P(A)/P(B).
It didn't take long before a strawman got pulled out of nowhere. That's so typical, if one disagrees with the wild conjectures of the rationalists, which they call "axioms", they blame you for rejecting the Pythagorean Theorem or some such. Dishonesty is the defining feature of that "movement"
It's not a strawman. It's a central example of what I had in mind when I used the word "axiom".
> if one disagrees with the wild conjectures of the rationalists, which they call "axioms"
They don't refer to wild conjectures as axioms.
> they blame you for rejecting the Pythagorean Theorem or some such.
There is no "blame" going on here. Someone said I sounded like a cult member (which incidentally is insulting and inappropriate on HN; but I thought I would try to steer the discussion back on course) and I correctly explained why that perception is irrelevant. When I said
> The entire point of the exercise is to establish that certain things are true, and that this is not because of faith, and that disbelieving them cannot make them false.
That is, in fact, inarguably true. Bayes' Law is an example of that exact sort of thing. If you don't believe that conditional probabilities can be worked out that way, that won't change the fact that conditional probabilities can in fact be worked out that way; and it won't change the fact that in the long run I will be able to exploit your fallacy in games of chance.
The other poster gave no example of supposedly false or unsound axioms, so I cannot be strawmanning by failing to respond to them. My assertion is that such false or unsound axioms are not, in fact, in play. The burden of proof lies with those claiming their existence.
I've been to a couple rationalist events in the Bay during peak rationalism (originally with an open mind but eventually resorted to gawking after people couldn't handle "but why..." style questions and prodding which is the norm in my line of work and past academic experience) and it was truly weird seeing supposedly well educated people fall for such obvious cult-like traps.
In hindsight, I guess there might have been an overrepresentation of neurodivergence in the community as well, and for some rationalism's simple precepts simply worked - though arguably you can integrate axiomatic thinking and formal logic with traditional religion as well (eg. Nicomachean Ethics or Ashtadhyay).
I am a massive supporter for data-driven decision making (heck, much of my undergrad was touching this kind of stuff), but mental models are meant to be tools not a crutch, and overindexing on them causes one to lose sight of the forest when amongst trees.
I'd also refute the notion on HN that the movers and shakers of the industry are overrepresented in the community - I actually found the rationalist community to be somewhat marginal in the industry as a whole but overrepresented among a specific type of chronically online individual. Indoor climbing, pickleball, and creative arts (eg. Pottery) are much more representative.
Life just seems to exceed that scope, is all.
During COVID computer modeling of cell chemistry became an established mode of study, in-silico is now used before in-vitro with great results.
One has to be careful to not fall in the trap presented as an alternative to the rationalist one - it sounds like, "there's no truth" or "the truth is too hard, just give up and accept what you're told to accept".
The lack of truth isn't due to it being hard or unachievable, there's enough of it to solve all major problems, it's not being pursued because of conflicts of interest.
"Almost always" is carrying a lot of weight here. Rationalists are ideologically diverse [1] when it comes to things like monogamy vs polyamory, left vs right, pro-AI vs anti-AI, nature vs nurture, racism vs anti-racism. The distribution is different from the general population, but many people have changed their minds about various aspects of their life after discussions and debates with rationalists.
https://docs.google.com/forms/d/e/1FAIpQLSf5FqX6XBJlfOShMd3U...
> What’s less known is that the memo was originally sent to the skeptics@google.com mailing list, which I was a member of. It was a personal belief masquerading as a rationalist inquiry that tried to pull everyone into an unwanted workplace argument.
One of Damore's mistakes was opting for a high-decoupling route on a politically charged topic instead of a low decoupling route. That tends to end badly and he learned this the hard way.
https://www.lesswrong.com/posts/7cAsBPGh98pGyrhz9/low-decoup...
I'm not really sure that "openness to racism" is an outlier in a population of a bunch of straight white men.
The entire point is that true statements are true, and false statements are false, regardless of who does or doesn't utter them.
Why does the formation of correct beliefs, a topic people have been discussing for millennia, require such exotic jargon? It's things like this that give off cult vibes.
So they are not rationalist then? I would have assume a rationalist would be...well..rational. And that would mean to include emotions into discussion about human and people. This is my problem with the classical Consequentialism or Utilitarianism, they always discount emotions. I'm sorry, does emotion exists or not? Is it real? Then it should be brought into discussion.
If I am in a math conference to discuss the Riemann Hypothesis with my cunt ex-wife in the room, I would get effected. It does not really matter how rational I am, I have to consider that my brain will not be fair towards her ideas.
If my brain is wired to be racist and that I truly have whispers of racism in my head, that should be part of the discussion whether or not I am qualified for this position. You claim to NOT have any such inclination, AT ALL? Okay, how likely is that given the human nature, what percentage should I give that you are lying? That should included in the discussion.
All of this is too much to think about? That is also part of the calculation, perhaps we should utilize some shortcut.
I don't understand why people exclude things from the universe of things to consider.
Maybe some of it is informed by the overlap with academia. Science has the aphorism "physics advances one funeral at a time" and certainly being on the outer has affected careers. Fred Hoyle was too argumentative, many of his ideas like panspermia have turned out to have elements of plausibility in them (like the cometary origins of water)
And Dawkins never had much of a research career.
My problem with the "skeptics" was that the paranormal is an easy target and politically it's punching down.
There are plenty of other irrationalities to consider - like how advertising and propaganda work, how cults and religions actually operate beyond "irrational bad", and how trad econ believes many things that are questionable at best - but the skeptics never dared go anywhere near those.
So skepticism turned into this kind of knockabout point-scoring media circus noise. And in the end the sneering and self-congratulatory condescension actually helped entrench modern conspiranoia instead of deflating it.
Perhaps not, but I have found his popular books like The Selfish Gene and The Ancestor's Tale very informative.
Unfortunately, when he ventures outside the domain of his professional expertise, he doesn't do as well. For example, in The God Delusion, which is basically a book-length diatribe against orthodox religious belief, he comes out at one point with this:
Let me sum up Einsteinian religion in one more quotation from Einstein himself: “To sense that behind anything that can be experienced there is a something that our mind cannot grasp and whose beauty and sublimity reaches us only indirectly and as a feeble reflection, this is religiousness. In this sense I am religious.”
If we grant for the sake of argument that typical orthodox religious cant is BS, this is just as much BS. The phrase "let him who is without sin cast the first stone" comes to mind.
1. I believe I'm very smart
2. I fell down a rabbit hole into a subculture filled with people who also think they are smart
3. Eventually I realised that they weren't that smart, and were actually most just a fad + spite (+ generally bigotry)
4. I worked this out because I'm very smart, and now I'm here to tell you, people who never fell for it, that that subculture isn't all it claims to be
5. Wow
It never, ever comes with the realisation that maybe the problem is the initial flawed belief and the smugness inherent in it.
The main smugness one can have is thinking they are immune to falling for a cult. Lot's of people do for lots of reasons. It's particularly likely when someone is in a vulnerable position in life (death, depression, major life change). Sometimes people are simply born into the cults and realize later that everything their family and community believes is a lie. Regardless, cults are pretty good at finding and "loving" people into the cult.
All this is to say that everyone should take a moment to question their core beliefs. Why do I believe this? Could I be wrong? How would I know I'm wrong? Questions I know I wish I asked and explored earlier in life. The honest answers, though, can be devastating if this part of your identity or family identity.
The framing isn't 'I fell for this and I was wrong', but 'I am the first one f all of you to see the truth'; there's no humility, no recognition, just the exact same self-delusion, which means they'll fall into the next pit head-first as well.
Was? Timecube.net will outlive any and all faddy Silicon Valley cults.
That's what can make the rationalist movement feel more like a religious cult and less like a bunch of eccentrics. As an outsider looking in, it seems like the entire movement is filled with just accepting priors with very little evidence for why those priors should be accepted.
How about the 95% stop having gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?
How is the 5% supposed to know that the 95% is actually larping belief? And how is an outsider who observes the 95% espousing their whacky beliefs supposed to know who is a member of the 95% and who is a member of the 5% (and if those percentages are accurate).
You can observe the same dynamic in any church. Everyone in the congregation will have different levels of belief, but when it comes to sharing a testimony you'll be hard pressed to distinguish between who the true believers are and who's just "faking it until you make it".
What on Earth are you referring to? What gatherings? What beliefs? What proclamations? What evidence of insincerity?
Rationalist have groups meetup and conferences. For example [1]
> What beliefs?
It's pretty diverse and not uniformly held. Some of the more strange ones that seem to have pretty wide appeal are around AI safety. For example, Roko's basilisk.
> What proclamations? What evidence of insincerity?
I was responding to the comment above me that suggested that 95% don't actually hold rationalist beliefs and are instead simply having fun with it. Only a small 5% are actually sincere.
[1] https://www.reddit.com/r/slatestarcodex/comments/1car63w/les...
If I observed such a gathering knowing that 5% of the people there have sincerely come to a particular conclusion based on the available evidence and the other 95% think the evidence doesn't support that conclusion, I would be extremely surprised to see everyone going around "larping belief". And I don't think you can actually show such a thing occurring.
> I would be extremely surprised to see everyone going around "larping belief". And I don't think you can actually show such a thing occurring.
I never claimed I could or that I observed such a thing. The 95/5% didn't come from me, it came from the person I responded to.
My point was that
- It's impossible to distinguish who's a true believer and who isn't.
- That the meetups of the rationalists re-enforce beliefs, particularly in true believers. Also that these beliefs of the rationalists can be pretty whacky and based on a jenga tower of priors.
- That there isn't "nothing" a rational person could do to dissuade someone from a whacky belief.
When you said:
> How about the 95% stop having gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?
you implicitly claimed that "the 95%" indeed have such gatherings.
As far as I can tell, the gatherings that they have are not of that nature.
> My point was that It's impossible to distinguish who's a true believer and who isn't.
And my claim is that it's actually trivial to tell who's a true believer and who isn't, because the concept of "true believer" does not pertain to "rationality" (by construction!) but to fringe beliefs, and people who are doing rationality correctly (which I expect to be the large majority) simply don't profess the fringe beliefs (because doing so is inherently antithetical to rationality). Therefore, the fringe 5% are the 5% of people actually saying the fringe thing, and the 95% don't actually say the fringe thing. Trivial.
For example, you won't catch Scott Alexander demanding "donate to MIRI or the basilisk might reach back in time and torture you".
> That the meetups of the rationalists re-enforce beliefs, particularly in true believers. Also that these beliefs of the rationalists can be pretty whacky and based on a jenga tower of priors.
And you have done nothing to establish this, and I believe you're incorrect.
> As far as I can tell, the gatherings that they have are not of that nature.
I literally cited you an example of such a gathering. Do you refute it or just deny that it exists?
> And my claim is that it's actually trivial to tell who's a true believer and who isn't, because the concept of "true believer" does not pertain to "rationality" (by construction!)
That's your mistake. The rationalists are irrational. Roko's basilisk, which I cited earlier, is an example of the sorts of ideas and beliefs they get up to which are irrational.
Your claim "This group is rational because they call themselves such" is unfounded. Trivially so. Just like the Christian Scientists aren't scientists just because it's part of their name.
> For example, you won't catch Scott Alexander demanding "donate to MIRI or the basilisk might reach back in time and torture you".
I never claimed universality. In fact, in response to you I explicitly said that the beliefs are diverse. Being able to cherry pick someone that's more rational than others and a part of the group doesn't prove that there aren't very influential members of the group who are generating a lot of fringe ideas. Specifically Eliezer Yudkowsky, the founder of LessWrong which is a core part of the rationality movement.
> And you have done nothing to establish this, and I believe you're incorrect.
I've established the meetups exist. I've established that a particularly nasty group came from these meetups, the Zizians. Specifically the founder got their start from the LessWrong forums and the "Center for Applied Rationality" [1] workshops (meetups).
[1] https://rationality.org/
(Except that they don't claim them to be "their own beliefs" but instead call them "priors".)
That, along with constantly calling yourself rational and being well-intentioned, easily leads to cultish beliefs to be pursued despite how wacky they seem at their face, because it was your superior brains that led you to them.
If we add the often good (or enormous) financial resources, it makes for a seriously dangerous cult.
There is not any such conviction. This is just your interpretation of a domain name, which is not intended that way.
> That, along with constantly calling yourself rational
This is explicitly not what you are supposed to do, and the point is explicitly not to create an identity around it.
[1] https://lydialaurenson.substack.com/p/the-inside-story-of-le...
[2] https://asteriskmag.com/issues/11/why-are-there-so-many-rati...
It would be highly surprising if an open-access online community with the kind of slogans that might be particularly appealing to somewhat spectrumy individuals did not occasionally attract some eccentrics. The point I am trying to make here is that these are pretty fringe offshots and are far from the representative views of the community as a whole. Appeals to reason don't make any community immune to human nature. After all, their main place of gathering is named "less wrong", not "correct".
If you start reasoning from first principles you can very easily make an incorrect assumption somewhere down the line that snowballs until you're talking about shrimp welfare, or making decisions based on the potential sextillion humans that exist in 1,000,000 AD, or planning to convert the entire universe into unthinking organic matter that can only experience bliss.
I think it is more that teenagers and very young adults and people high in the trait of "openness to new experiences" are more susceptible to nonsense and more attracted to new intellectual movements, of which the rationalist community is one.
Also, I expect a rationalist to be lower in conscientiousness than average (because the conscientious people are too busy to devote a lot of time to reading long intellectual texts) and low conscientiousness makes violence and other transgressive behavior more likely.
https://www.youtube.com/watch?v=9mJAerUL-7w
In the end, even I can't escape bias. And when I realize how ignorant I am, I feel like I have no choice but to live face to face with how pathetic I am.
Even if I'm aware of my bounded rationality and ignorance, there are still moments when I fail to recognize even that ignorance. I feel like I just have to live with the awareness of how contradictory a being I am
The rational person does not engage with the argument. Because there is nothing to be gained from the argument.
The winning move is not to play.