Toxic social norms don’t develop intentionally, nobody wants them to happen, they’re not written down, and nobody enforces them explicitly. (The intentional development of toxic social norms is otherwise known as founding a cult.) What happens is that there are positive social norms, like, “talking about epistemics and being curious about beliefs is cool,” or “try to be intentional about the positive impact you can have on the world.” These norms are great! But then, the group acts like a group, which is to say, people confer status depending on level of apparent adherence to values. This leads insecure people who completely depend on the group to over-identify with the set of values, to the extent that even slightly contrary actions become forbidden. Not forbidden in the like “we’ll arrest you” way, but in the like “everyone in the room immediately looks at you like you’re being rude if you talk about spirituality” way.
- No location available
- social norms, change blindness,

Sometimes—often—these forbidden thoughts/actions aren’t even contrary to the explicit values. They just don’t fit in with the implied group aesthetic, which is often a much stricter, more menacing guideline, all the more so because it’s a collective unwritten fiction. “Rationality is cool” becomes “rationality is the best framework” becomes “Rationalist and Rationalist-flavored stuff is a better use of your time than anything else” becomes “it’s uncool if you want to spend a lot of time doing stuff that has nothing to do with testable beliefs, or our favorite issues.” This is all unintentional and implicit.
- No location available
-

One sign of toxic social norms is if your behavior does deviate from the standard, you feel that the only way of saving face is through explaining your behavior via the group values. Like, if you watch the Bachelor all the time, and one of your smart peers finds out about that, you might find yourself hastily explaining that the series is enjoyable to you as an applied experiment in evolutionary psychology, when, in fact, you just like social drama because watching humans freak out is fun.
- No location available
-

Any set of principles, if followed too strictly and assigned too much social value, can become a weird evil super-ego that creeps into the deepest crevices of your psyche.
- No location available
-

Private guts are very important and arguably the thing that mostly guides people's behavior, but they are often also ones that the person can't justify. If a person felt like they should reject any beliefs they couldn't justify, they would quickly become incapable of doing anything at all.
- No location available
-

There is a phenomenon where the most visible people in a group are often the most involved, and are more likely to experience personal difficulties that drive extreme levels of involvement. Another term people use for this is "very online".
- No location available
- involvement,

It's good to acknowledge that unhealthy norms do exist in some corners of EA, but I worry that confusing "what EA looks like online" with "how EA actually is on a population level" might lead us to throw too many resources or too much concern at problems that aren't as widespread as they appear.
- No location available
-