Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them

!tags:: #lit✍/📰️article/highlights
!links:: altruism, doing good,
!ref:: Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them
!author:: 80000hours.org

=this.file.name

Book cover of "Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them"

Reference

Notes

Why knowing when you’re having a negative (counterfactual) impact is harder than it first seems

Fragile fields and where these risks are greatest

Ways to cause an unintended negative impact – from most to least obvious

1. You take on a challenging project and make a mistake through lack of expertise or poor judgement

Quote

Another common oversight is failing to appreciate how damaging interpersonal conflict can be and how hard it is to avoid. Interpersonal conflicts can harm a whole field by reducing trust and solidarity, which impedes coordination and makes recruitment much more difficult. Nobody wants to join a field where everybody is fighting with each other.An example from history: Dr. Ignaz Semmelweis realised in 1847 that cleaning doctors’ hands could save patients’ lives. His colleagues were at first willing to indulge his whim, and infection rates plummeted on his unit. But after a series of miscommunications and political conflicts within the hospital system, Semmelweis came to be regarded as a crank and was demoted. The practice of handwashing was abandoned, and thousands of patients died from infection over the next decades until later researchers proved him right. If he’d prioritised clear communication of his ideas and better relationships with his colleagues, the establishment might not have been so tragically late in realising that his ideas were correct.
- No location available
-

2. Reputational harm

Quote

Unfortunately, this is one reason why it can be costly to make your life countercultural or unusual in lots of ways that are unrelated to your project. If you fail and it turns out for example you were (as a made-up example) really into My Little Pony, whether that had anything to do with the success of your project or not, it makes it a much more interesting story, so it’s going to get more attention, and cause more damage to the reputation of the cause you were trying to support.
- No location available
-

Quote

For instance, imagine you’re excited about a totally new way to improve the world and want to start fundraising to get it off the ground, so you go and talk to all the millionaire donors who seem like they might be interested. Unfortunately, you haven’t really thought through objections to your ideas, and so repeatedly come across as naive. Those donors decide not to fund you, and are also less likely to take meetings with anyone else who wants to do something similar in the future.
- No location available
-

3. Resource diversion

Quote

When doing an impact evaluation of your project, it’s important to try to roughly compare your impact to the impact that would have been possible if the resources you used had gone to another impactful project they might have plausibly gone to. For instance, our readers who work on international development often use GiveDirectly as ‘baseline’ that could absorb a lot of funds.
- No location available
-

4. Locking in suboptimal choices

Quote

The decisions you make when you’re just starting out and know the least can stick around or even snowball out of control. This is for a number of reasons:Most people who know of you will only ever offer a tiny amount of attention, so it’s hard to change their first impression;Once the media has written about you, people will keep finding those articles, shaping future perceptions. In particular, journalists often draw from the work of earlier journalists.Terms are hard to change — we would struggle to abandon the term ‘effective altruism’ today even if we decided we didn’t like it;Once you define what you believe, you will tend to attract people who agree with that view, further entrenching it;People find it very hard to fire colleagues, change management structures, or abandon their strategy, so bad choices often carry on even once they’re known to be problematic.
- No location available
-

5. Crowding out

Quote

The other side of this problem is that it’s bad when qualified people successfully identify a gap but don’t take initiative because they think it’s already being handled by others — or they wait for somebody better to come along.
- No location available
-
- [note::What gaps am I not filling?]

Quote

We’re just pointing out that announcing the start of your project isn’t costless for the rest of your field, so it’s worth doing at least a bit of due diligence before moving ahead and potentially discouraging others. Get advice from people you trust to be honest about whether you’re a reasonable fit for the project you’re considering. Ask around to see if anybody else in your field has similar plans; maybe you should merge projects, collaborate, or coordinate on which project should move forward.
- No location available
-

Quote

Lastly, try to be honest with yourself about the likelihood that you’ll actually follow through with your plans. One of the most avoidable (and costly) forms of crowding out is when people announce a project but never really get it off the ground. For example, we’ve heard several people say they don’t want to start a local effective altruist group because one already exists, but then the existing group soon becomes neglected or entirely inactive.
- No location available
-

6. Creating other coordination problems

Quote

Larger groups are harder to coordinate than smaller ones. Whether you’re doing research, advocacy or dealing with outsiders, joining a field obligates your peers to invest time making sure you and they are in sync. Furthermore, a lot of coordination relies on high trust, and it’s hard to maintain trust in a larger or shifting group where you don’t have established relationships. Adding people to an area has some direct positive impact, but it also creates an extra cost in the form of more difficult coordination. This makes the bar for growing a cause (especially a small one) higher than it first seems.
- No location available
-
- [note::The cost of coordination]

How can you mitigate these risks?

1. Ideally, eliminate courses of action that might have a big negative impact

2. Don’t be a naive optimizer

Quote

Your ‘model’ of the situation probably spits out a very uncertain answer about what’s best (“known unknowns”). But then there’s also the chance your model itself is wrong, and you’re thinking about the situation entirely wrong, and this could be in ways you haven’t even considered (“unknown unknowns”). And beyond that, it’s uncertain how to even reason about these kinds of situations — there isn’t a single clearly accepted theory of what how to make decisions under uncertainty.So if you simply pick a goal that seems good to you, and aggressively pursue it, there are bound to be other important outcomes you’re ignoring.
- No location available
-

Quote

Some examples of being a naive optimiser:Trying to earn as much money as possible, so you can donate as much as possible (but ignoring issues around reputation, character, cooperative norms; as well as other pathways to doing good such as spreading important ideas and building career capital)Ignoring all considerations except for the one you think is most important (e.g. that there might be a lot of future generations)An issue for entrepreneurs is to get so convinced by the mission of their own organisation that they cut legal and ethical corners to make it succeed.
- No location available
-

Quote

There is no widely accepted solution to the problem of how to reason in the face of deep model uncertainty, but here are some ideas that make sense to us:Consider multiple models of the situation — many understandings of what matters, many potential outcomes, and many perspectives. This should include what conventional wisdom and other experts would say. You should actively seek out the best arguments against your approach. This gives you the best possible chances of spotting missing considerations.Take courses of action that either seem good according to many models, or that are very good on one perspective and roughly neutral on the others. In contrast, avoid courses of action that seem crazy on some reasonable perspectives.Be ambitious but not aggressive.
- No location available
- pluralism, rationality, strategy,

3. Have a degree of humility

Here are some more detailed tips on how to strike the balance:
Quote

Make a distinction between your “impressions” (what seems best to you) and your “all considered view” (what you believe after taking account of other people’s views). It’s important to develop your own impressions, so that you’re adding to the collective wisdom about what to do, but high-stakes action should generally be taken based on your all considered view.
- No location available
-

Quote

Put more weight on your own impressions the more reasons there are to trust your views (such as expertise or a track record), and when you can clearly point to information or values that you have which others don’t. If a random uninformed person disagrees with your project, that’s usually not worth worrying about.
- No location available
-

Quote

Gather lots of views. A large number of non-experts can easily be more accurate than a small number of experts.
- No location available
-

Quote

Weight someone’s views more by track record than superficial indicators of expertise. In many fields (e.g. most social science), experts aren’t much better at making predictions than random chance. The people with the best judgement are often informed generalists with the right mindset. Either way, try to evaluate how trustworthy people are based on their track record of making similar judgement calls.
- No location available
-

Quote

Weight the views of others by their strength, both in terms of the stakes and their degree of confidence. If even someone trustworthy believes your project is ‘meh,’ that’s not a big deal either way. If they’re convinced it’s very harmful, you should be a lot more cautious.
- No location available
-

Quote

Consider the stakes. In some contexts, like private intellectual discussion or doing small test projects, it’s OK to run with your wacky views. The greater the potential harms, the more important it is to consider a wide range of views.
- No location available
-

Quote

Don’t rely only on your own judgement for high-stakes decisions. For instance, if you’re the CEO of a larger project, it’s important to have a board and strong cofounders to check your most important decisions.
- No location available
-

Quote

If others think your project isn’t impactful (and haven’t thought about it much), that isn’t much reason to not do it. There will always be people who aren’t excited about your project.
- No location available
-

4. Develop expertise, get trained, build a network, and benefit from your field’s accumulated wisdom

Quote

When entering a field, consider starting out by working for an established organisation with the capacity to supervise your work. Learn about the field and make sure you understand the views of established players. The time this takes will vary a lot according to the area you’re working in. We generally think it makes sense to work in an area for at least 1-3 years before doing higher stakes or independently led projects, but in particularly complicated areas — like those that require lots of technical knowledge or graduate study — this can take longer.
- No location available
-

5. Follow cooperative norms

6. Match your capabilities to your project and influence

Quote

Try to match your capabilities to your degree of influence and the fragility of the problems you’re working on. As the stakes get higher, the more vetting, expertise and caution to bring.
- No location available
-

Quote

As your project becomes more influential and successful, it becomes more important to keep seeking out advisors and colleagues who will stand up to you. Unfortunately the opposite is often the case: as you get more successful, people will be less inclined to doubt you, and more worried about criticising you. To avoid this, you may need to purposely set up structures and processes to limit the role of your judgement.
- No location available
-

7. Avoid hard-to-reverse actions

Quote

Hiring, growing your funding or increasing your influence normally look like concretely good things from the perspective of your project in the short-term — being bigger means more impact — but it can be more ambiguous when you consider the field as a whole and the long-term effects.Rapid growth is hard to unwind, since it would mean firing people, so you get locked into the new, bigger state. More people are harder to coordinate, and require training and management, which is often in short supply. This leads to more of the issues we’ve covered, like unilateralism, breakdown of norms and errors of judgement. Going fast creates more reputation risks, which might have been avoided if you’d gone more slowly. (Ignoring these more diffuse harms is another example of naive optimization.)Of course, growing more slowly also has big costs; so it’s a question of balance. However, our sense is that the benefits of growth are usually more tangible than the costs, so people are more likely overestimate the value of growth than underestimate it.
- No location available
-
- [note::Potential impact, coordination/management costs, and reputational risks are all correlated with the size of any group.]


dg-publish: true
created: 2024-07-01
modified: 2024-07-01
title: Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them
source: hypothesis

!tags:: #lit✍/📰️article/highlights
!links:: altruism, doing good,
!ref:: Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them
!author:: 80000hours.org

=this.file.name

Book cover of "Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them"

Reference

Notes

Why knowing when you’re having a negative (counterfactual) impact is harder than it first seems

Fragile fields and where these risks are greatest

Ways to cause an unintended negative impact – from most to least obvious

1. You take on a challenging project and make a mistake through lack of expertise or poor judgement

Quote

Another common oversight is failing to appreciate how damaging interpersonal conflict can be and how hard it is to avoid. Interpersonal conflicts can harm a whole field by reducing trust and solidarity, which impedes coordination and makes recruitment much more difficult. Nobody wants to join a field where everybody is fighting with each other.An example from history: Dr. Ignaz Semmelweis realised in 1847 that cleaning doctors’ hands could save patients’ lives. His colleagues were at first willing to indulge his whim, and infection rates plummeted on his unit. But after a series of miscommunications and political conflicts within the hospital system, Semmelweis came to be regarded as a crank and was demoted. The practice of handwashing was abandoned, and thousands of patients died from infection over the next decades until later researchers proved him right. If he’d prioritised clear communication of his ideas and better relationships with his colleagues, the establishment might not have been so tragically late in realising that his ideas were correct.
- No location available
-

2. Reputational harm

Quote

Unfortunately, this is one reason why it can be costly to make your life countercultural or unusual in lots of ways that are unrelated to your project. If you fail and it turns out for example you were (as a made-up example) really into My Little Pony, whether that had anything to do with the success of your project or not, it makes it a much more interesting story, so it’s going to get more attention, and cause more damage to the reputation of the cause you were trying to support.
- No location available
-

Quote

For instance, imagine you’re excited about a totally new way to improve the world and want to start fundraising to get it off the ground, so you go and talk to all the millionaire donors who seem like they might be interested. Unfortunately, you haven’t really thought through objections to your ideas, and so repeatedly come across as naive. Those donors decide not to fund you, and are also less likely to take meetings with anyone else who wants to do something similar in the future.
- No location available
-

3. Resource diversion

Quote

When doing an impact evaluation of your project, it’s important to try to roughly compare your impact to the impact that would have been possible if the resources you used had gone to another impactful project they might have plausibly gone to. For instance, our readers who work on international development often use GiveDirectly as ‘baseline’ that could absorb a lot of funds.
- No location available
-

4. Locking in suboptimal choices

Quote

The decisions you make when you’re just starting out and know the least can stick around or even snowball out of control. This is for a number of reasons:Most people who know of you will only ever offer a tiny amount of attention, so it’s hard to change their first impression;Once the media has written about you, people will keep finding those articles, shaping future perceptions. In particular, journalists often draw from the work of earlier journalists.Terms are hard to change — we would struggle to abandon the term ‘effective altruism’ today even if we decided we didn’t like it;Once you define what you believe, you will tend to attract people who agree with that view, further entrenching it;People find it very hard to fire colleagues, change management structures, or abandon their strategy, so bad choices often carry on even once they’re known to be problematic.
- No location available
-

5. Crowding out

Quote

The other side of this problem is that it’s bad when qualified people successfully identify a gap but don’t take initiative because they think it’s already being handled by others — or they wait for somebody better to come along.
- No location available
-
- [note::What gaps am I not filling?]

Quote

We’re just pointing out that announcing the start of your project isn’t costless for the rest of your field, so it’s worth doing at least a bit of due diligence before moving ahead and potentially discouraging others. Get advice from people you trust to be honest about whether you’re a reasonable fit for the project you’re considering. Ask around to see if anybody else in your field has similar plans; maybe you should merge projects, collaborate, or coordinate on which project should move forward.
- No location available
-

Quote

Lastly, try to be honest with yourself about the likelihood that you’ll actually follow through with your plans. One of the most avoidable (and costly) forms of crowding out is when people announce a project but never really get it off the ground. For example, we’ve heard several people say they don’t want to start a local effective altruist group because one already exists, but then the existing group soon becomes neglected or entirely inactive.
- No location available
-

6. Creating other coordination problems

Quote

Larger groups are harder to coordinate than smaller ones. Whether you’re doing research, advocacy or dealing with outsiders, joining a field obligates your peers to invest time making sure you and they are in sync. Furthermore, a lot of coordination relies on high trust, and it’s hard to maintain trust in a larger or shifting group where you don’t have established relationships. Adding people to an area has some direct positive impact, but it also creates an extra cost in the form of more difficult coordination. This makes the bar for growing a cause (especially a small one) higher than it first seems.
- No location available
-
- [note::The cost of coordination]

How can you mitigate these risks?

1. Ideally, eliminate courses of action that might have a big negative impact

2. Don’t be a naive optimizer

Quote

Your ‘model’ of the situation probably spits out a very uncertain answer about what’s best (“known unknowns”). But then there’s also the chance your model itself is wrong, and you’re thinking about the situation entirely wrong, and this could be in ways you haven’t even considered (“unknown unknowns”). And beyond that, it’s uncertain how to even reason about these kinds of situations — there isn’t a single clearly accepted theory of what how to make decisions under uncertainty.So if you simply pick a goal that seems good to you, and aggressively pursue it, there are bound to be other important outcomes you’re ignoring.
- No location available
-

Quote

Some examples of being a naive optimiser:Trying to earn as much money as possible, so you can donate as much as possible (but ignoring issues around reputation, character, cooperative norms; as well as other pathways to doing good such as spreading important ideas and building career capital)Ignoring all considerations except for the one you think is most important (e.g. that there might be a lot of future generations)An issue for entrepreneurs is to get so convinced by the mission of their own organisation that they cut legal and ethical corners to make it succeed.
- No location available
-

Quote

There is no widely accepted solution to the problem of how to reason in the face of deep model uncertainty, but here are some ideas that make sense to us:Consider multiple models of the situation — many understandings of what matters, many potential outcomes, and many perspectives. This should include what conventional wisdom and other experts would say. You should actively seek out the best arguments against your approach. This gives you the best possible chances of spotting missing considerations.Take courses of action that either seem good according to many models, or that are very good on one perspective and roughly neutral on the others. In contrast, avoid courses of action that seem crazy on some reasonable perspectives.Be ambitious but not aggressive.
- No location available
- pluralism, rationality, strategy,

3. Have a degree of humility

Here are some more detailed tips on how to strike the balance:
Quote

Make a distinction between your “impressions” (what seems best to you) and your “all considered view” (what you believe after taking account of other people’s views). It’s important to develop your own impressions, so that you’re adding to the collective wisdom about what to do, but high-stakes action should generally be taken based on your all considered view.
- No location available
-

Quote

Put more weight on your own impressions the more reasons there are to trust your views (such as expertise or a track record), and when you can clearly point to information or values that you have which others don’t. If a random uninformed person disagrees with your project, that’s usually not worth worrying about.
- No location available
-

Quote

Gather lots of views. A large number of non-experts can easily be more accurate than a small number of experts.
- No location available
-

Quote

Weight someone’s views more by track record than superficial indicators of expertise. In many fields (e.g. most social science), experts aren’t much better at making predictions than random chance. The people with the best judgement are often informed generalists with the right mindset. Either way, try to evaluate how trustworthy people are based on their track record of making similar judgement calls.
- No location available
-

Quote

Weight the views of others by their strength, both in terms of the stakes and their degree of confidence. If even someone trustworthy believes your project is ‘meh,’ that’s not a big deal either way. If they’re convinced it’s very harmful, you should be a lot more cautious.
- No location available
-

Quote

Consider the stakes. In some contexts, like private intellectual discussion or doing small test projects, it’s OK to run with your wacky views. The greater the potential harms, the more important it is to consider a wide range of views.
- No location available
-

Quote

Don’t rely only on your own judgement for high-stakes decisions. For instance, if you’re the CEO of a larger project, it’s important to have a board and strong cofounders to check your most important decisions.
- No location available
-

Quote

If others think your project isn’t impactful (and haven’t thought about it much), that isn’t much reason to not do it. There will always be people who aren’t excited about your project.
- No location available
-

4. Develop expertise, get trained, build a network, and benefit from your field’s accumulated wisdom

Quote

When entering a field, consider starting out by working for an established organisation with the capacity to supervise your work. Learn about the field and make sure you understand the views of established players. The time this takes will vary a lot according to the area you’re working in. We generally think it makes sense to work in an area for at least 1-3 years before doing higher stakes or independently led projects, but in particularly complicated areas — like those that require lots of technical knowledge or graduate study — this can take longer.
- No location available
-

5. Follow cooperative norms

6. Match your capabilities to your project and influence

Quote

Try to match your capabilities to your degree of influence and the fragility of the problems you’re working on. As the stakes get higher, the more vetting, expertise and caution to bring.
- No location available
-

Quote

As your project becomes more influential and successful, it becomes more important to keep seeking out advisors and colleagues who will stand up to you. Unfortunately the opposite is often the case: as you get more successful, people will be less inclined to doubt you, and more worried about criticising you. To avoid this, you may need to purposely set up structures and processes to limit the role of your judgement.
- No location available
-

7. Avoid hard-to-reverse actions

Quote

Hiring, growing your funding or increasing your influence normally look like concretely good things from the perspective of your project in the short-term — being bigger means more impact — but it can be more ambiguous when you consider the field as a whole and the long-term effects.Rapid growth is hard to unwind, since it would mean firing people, so you get locked into the new, bigger state. More people are harder to coordinate, and require training and management, which is often in short supply. This leads to more of the issues we’ve covered, like unilateralism, breakdown of norms and errors of judgement. Going fast creates more reputation risks, which might have been avoided if you’d gone more slowly. (Ignoring these more diffuse harms is another example of naive optimization.)Of course, growing more slowly also has big costs; so it’s a question of balance. However, our sense is that the benefits of growth are usually more tangible than the costs, so people are more likely overestimate the value of growth than underestimate it.
- No location available
-
- [note::Potential impact, coordination/management costs, and reputational risks are all correlated with the size of any group.]