Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them
!tags:: #litâ/đ°ď¸article/highlights
!links:: altruism, doing good,
!ref:: Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them
!author:: 80000hours.org
=this.file.name
Reference
=this.ref
Notes
Why knowing when youâre having a negative (counterfactual) impact is harder than it first seems
Fragile fields and where these risks are greatest
Ways to cause an unintended negative impact â from most to least obvious
1. You take on a challenging project and make a mistake through lack of expertise or poor judgement
Another common oversight is failing to appreciate how damaging interpersonal conflict can be and how hard it is to avoid. Interpersonal conflicts can harm a whole field by reducing trust and solidarity, which impedes coordination and makes recruitment much more difficult. Nobody wants to join a field where everybody is fighting with each other.An example from history: Dr. Ignaz Semmelweis realised in 1847 that cleaning doctorsâ hands could save patientsâ lives. His colleagues were at first willing to indulge his whim, and infection rates plummeted on his unit. But after a series of miscommunications and political conflicts within the hospital system, Semmelweis came to be regarded as a crank and was demoted. The practice of handwashing was abandoned, and thousands of patients died from infection over the next decades until later researchers proved him right. If heâd prioritised clear communication of his ideas and better relationships with his colleagues, the establishment might not have been so tragically late in realising that his ideas were correct.
- No location available
-
2. Reputational harm
Unfortunately, this is one reason why it can be costly to make your life countercultural or unusual in lots of ways that are unrelated to your project. If you fail and it turns out for example you were (as a made-up example) really into My Little Pony, whether that had anything to do with the success of your project or not, it makes it a much more interesting story, so itâs going to get more attention, and cause more damage to the reputation of the cause you were trying to support.
- No location available
-
For instance, imagine youâre excited about a totally new way to improve the world and want to start fundraising to get it off the ground, so you go and talk to all the millionaire donors who seem like they might be interested. Unfortunately, you havenât really thought through objections to your ideas, and so repeatedly come across as naive. Those donors decide not to fund you, and are also less likely to take meetings with anyone else who wants to do something similar in the future.
- No location available
-
3. Resource diversion
When doing an impact evaluation of your project, itâs important to try to roughly compare your impact to the impact that would have been possible if the resources you used had gone to another impactful project they might have plausibly gone to. For instance, our readers who work on international development often use GiveDirectly as âbaselineâ that could absorb a lot of funds.
- No location available
-
4. Locking in suboptimal choices
The decisions you make when youâre just starting out and know the least can stick around or even snowball out of control. This is for a number of reasons:Most people who know of you will only ever offer a tiny amount of attention, so itâs hard to change their first impression;Once the media has written about you, people will keep finding those articles, shaping future perceptions. In particular, journalists often draw from the work of earlier journalists.Terms are hard to change â we would struggle to abandon the term âeffective altruismâ today even if we decided we didnât like it;Once you define what you believe, you will tend to attract people who agree with that view, further entrenching it;People find it very hard to fire colleagues, change management structures, or abandon their strategy, so bad choices often carry on even once theyâre known to be problematic.
- No location available
-
5. Crowding out
The other side of this problem is that itâs bad when qualified people successfully identify a gap but donât take initiative because they think itâs already being handled by others â or they wait for somebody better to come along.
- No location available
-
- [note::What gaps am I not filling?]
Weâre just pointing out that announcing the start of your project isnât costless for the rest of your field, so itâs worth doing at least a bit of due diligence before moving ahead and potentially discouraging others. Get advice from people you trust to be honest about whether youâre a reasonable fit for the project youâre considering. Ask around to see if anybody else in your field has similar plans; maybe you should merge projects, collaborate, or coordinate on which project should move forward.
- No location available
-
Lastly, try to be honest with yourself about the likelihood that youâll actually follow through with your plans. One of the most avoidable (and costly) forms of crowding out is when people announce a project but never really get it off the ground. For example, weâve heard several people say they donât want to start a local effective altruist group because one already exists, but then the existing group soon becomes neglected or entirely inactive.
- No location available
-
6. Creating other coordination problems
Larger groups are harder to coordinate than smaller ones. Whether youâre doing research, advocacy or dealing with outsiders, joining a field obligates your peers to invest time making sure you and they are in sync. Furthermore, a lot of coordination relies on high trust, and itâs hard to maintain trust in a larger or shifting group where you donât have established relationships. Adding people to an area has some direct positive impact, but it also creates an extra cost in the form of more difficult coordination. This makes the bar for growing a cause (especially a small one) higher than it first seems.
- No location available
-
- [note::The cost of coordination]
How can you mitigate these risks?
1. Ideally, eliminate courses of action that might have a big negative impact
2. Donât be a naive optimizer
Your âmodelâ of the situation probably spits out a very uncertain answer about whatâs best (âknown unknownsâ). But then thereâs also the chance your model itself is wrong, and youâre thinking about the situation entirely wrong, and this could be in ways you havenât even considered (âunknown unknownsâ). And beyond that, itâs uncertain how to even reason about these kinds of situations â there isnât a single clearly accepted theory of what how to make decisions under uncertainty.So if you simply pick a goal that seems good to you, and aggressively pursue it, there are bound to be other important outcomes youâre ignoring.
- No location available
-
Some examples of being a naive optimiser:Trying to earn as much money as possible, so you can donate as much as possible (but ignoring issues around reputation, character, cooperative norms; as well as other pathways to doing good such as spreading important ideas and building career capital)Ignoring all considerations except for the one you think is most important (e.g. that there might be a lot of future generations)An issue for entrepreneurs is to get so convinced by the mission of their own organisation that they cut legal and ethical corners to make it succeed.
- No location available
-
There is no widely accepted solution to the problem of how to reason in the face of deep model uncertainty, but here are some ideas that make sense to us:Consider multiple models of the situation â many understandings of what matters, many potential outcomes, and many perspectives. This should include what conventional wisdom and other experts would say. You should actively seek out the best arguments against your approach. This gives you the best possible chances of spotting missing considerations.Take courses of action that either seem good according to many models, or that are very good on one perspective and roughly neutral on the others. In contrast, avoid courses of action that seem crazy on some reasonable perspectives.Be ambitious but not aggressive.
- No location available
- pluralism, rationality, strategy,
3. Have a degree of humility
Here are some more detailed tips on how to strike the balance:
Make a distinction between your âimpressionsâ (what seems best to you) and your âall considered viewâ (what you believe after taking account of other peopleâs views). Itâs important to develop your own impressions, so that youâre adding to the collective wisdom about what to do, but high-stakes action should generally be taken based on your all considered view.
- No location available
-
Put more weight on your own impressions the more reasons there are to trust your views (such as expertise or a track record), and when you can clearly point to information or values that you have which others donât. If a random uninformed person disagrees with your project, thatâs usually not worth worrying about.
- No location available
-
Gather lots of views. A large number of non-experts can easily be more accurate than a small number of experts.
- No location available
-
Weight someoneâs views more by track record than superficial indicators of expertise. In many fields (e.g. most social science), experts arenât much better at making predictions than random chance. The people with the best judgement are often informed generalists with the right mindset. Either way, try to evaluate how trustworthy people are based on their track record of making similar judgement calls.
- No location available
-
Weight the views of others by their strength, both in terms of the stakes and their degree of confidence. If even someone trustworthy believes your project is âmeh,â thatâs not a big deal either way. If theyâre convinced itâs very harmful, you should be a lot more cautious.
- No location available
-
Consider the stakes. In some contexts, like private intellectual discussion or doing small test projects, itâs OK to run with your wacky views. The greater the potential harms, the more important it is to consider a wide range of views.
- No location available
-
Donât rely only on your own judgement for high-stakes decisions. For instance, if youâre the CEO of a larger project, itâs important to have a board and strong cofounders to check your most important decisions.
- No location available
-
If others think your project isnât impactful (and havenât thought about it much), that isnât much reason to not do it. There will always be people who arenât excited about your project.
- No location available
-
4. Develop expertise, get trained, build a network, and benefit from your fieldâs accumulated wisdom
When entering a field, consider starting out by working for an established organisation with the capacity to supervise your work. Learn about the field and make sure you understand the views of established players. The time this takes will vary a lot according to the area youâre working in. We generally think it makes sense to work in an area for at least 1-3 years before doing higher stakes or independently led projects, but in particularly complicated areas â like those that require lots of technical knowledge or graduate study â this can take longer.
- No location available
-
5. Follow cooperative norms
6. Match your capabilities to your project and influence
Try to match your capabilities to your degree of influence and the fragility of the problems youâre working on. As the stakes get higher, the more vetting, expertise and caution to bring.
- No location available
-
As your project becomes more influential and successful, it becomes more important to keep seeking out advisors and colleagues who will stand up to you. Unfortunately the opposite is often the case: as you get more successful, people will be less inclined to doubt you, and more worried about criticising you. To avoid this, you may need to purposely set up structures and processes to limit the role of your judgement.
- No location available
-
7. Avoid hard-to-reverse actions
Hiring, growing your funding or increasing your influence normally look like concretely good things from the perspective of your project in the short-term â being bigger means more impact â but it can be more ambiguous when you consider the field as a whole and the long-term effects.Rapid growth is hard to unwind, since it would mean firing people, so you get locked into the new, bigger state. More people are harder to coordinate, and require training and management, which is often in short supply. This leads to more of the issues weâve covered, like unilateralism, breakdown of norms and errors of judgement. Going fast creates more reputation risks, which might have been avoided if youâd gone more slowly. (Ignoring these more diffuse harms is another example of naive optimization.)Of course, growing more slowly also has big costs; so itâs a question of balance. However, our sense is that the benefits of growth are usually more tangible than the costs, so people are more likely overestimate the value of growth than underestimate it.
- No location available
-
- [note::Potential impact, coordination/management costs, and reputational risks are all correlated with the size of any group.]
dg-publish: true
created: 2024-07-01
modified: 2024-07-01
title: Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them
source: hypothesis
!tags:: #litâ/đ°ď¸article/highlights
!links:: altruism, doing good,
!ref:: Ways People Trying to Do Good Accidentally Make Things Worse, and How to Avoid Them
!author:: 80000hours.org
=this.file.name
Reference
=this.ref
Notes
Why knowing when youâre having a negative (counterfactual) impact is harder than it first seems
Fragile fields and where these risks are greatest
Ways to cause an unintended negative impact â from most to least obvious
1. You take on a challenging project and make a mistake through lack of expertise or poor judgement
Another common oversight is failing to appreciate how damaging interpersonal conflict can be and how hard it is to avoid. Interpersonal conflicts can harm a whole field by reducing trust and solidarity, which impedes coordination and makes recruitment much more difficult. Nobody wants to join a field where everybody is fighting with each other.An example from history: Dr. Ignaz Semmelweis realised in 1847 that cleaning doctorsâ hands could save patientsâ lives. His colleagues were at first willing to indulge his whim, and infection rates plummeted on his unit. But after a series of miscommunications and political conflicts within the hospital system, Semmelweis came to be regarded as a crank and was demoted. The practice of handwashing was abandoned, and thousands of patients died from infection over the next decades until later researchers proved him right. If heâd prioritised clear communication of his ideas and better relationships with his colleagues, the establishment might not have been so tragically late in realising that his ideas were correct.
- No location available
-
2. Reputational harm
Unfortunately, this is one reason why it can be costly to make your life countercultural or unusual in lots of ways that are unrelated to your project. If you fail and it turns out for example you were (as a made-up example) really into My Little Pony, whether that had anything to do with the success of your project or not, it makes it a much more interesting story, so itâs going to get more attention, and cause more damage to the reputation of the cause you were trying to support.
- No location available
-
For instance, imagine youâre excited about a totally new way to improve the world and want to start fundraising to get it off the ground, so you go and talk to all the millionaire donors who seem like they might be interested. Unfortunately, you havenât really thought through objections to your ideas, and so repeatedly come across as naive. Those donors decide not to fund you, and are also less likely to take meetings with anyone else who wants to do something similar in the future.
- No location available
-
3. Resource diversion
When doing an impact evaluation of your project, itâs important to try to roughly compare your impact to the impact that would have been possible if the resources you used had gone to another impactful project they might have plausibly gone to. For instance, our readers who work on international development often use GiveDirectly as âbaselineâ that could absorb a lot of funds.
- No location available
-
4. Locking in suboptimal choices
The decisions you make when youâre just starting out and know the least can stick around or even snowball out of control. This is for a number of reasons:Most people who know of you will only ever offer a tiny amount of attention, so itâs hard to change their first impression;Once the media has written about you, people will keep finding those articles, shaping future perceptions. In particular, journalists often draw from the work of earlier journalists.Terms are hard to change â we would struggle to abandon the term âeffective altruismâ today even if we decided we didnât like it;Once you define what you believe, you will tend to attract people who agree with that view, further entrenching it;People find it very hard to fire colleagues, change management structures, or abandon their strategy, so bad choices often carry on even once theyâre known to be problematic.
- No location available
-
5. Crowding out
The other side of this problem is that itâs bad when qualified people successfully identify a gap but donât take initiative because they think itâs already being handled by others â or they wait for somebody better to come along.
- No location available
-
- [note::What gaps am I not filling?]
Weâre just pointing out that announcing the start of your project isnât costless for the rest of your field, so itâs worth doing at least a bit of due diligence before moving ahead and potentially discouraging others. Get advice from people you trust to be honest about whether youâre a reasonable fit for the project youâre considering. Ask around to see if anybody else in your field has similar plans; maybe you should merge projects, collaborate, or coordinate on which project should move forward.
- No location available
-
Lastly, try to be honest with yourself about the likelihood that youâll actually follow through with your plans. One of the most avoidable (and costly) forms of crowding out is when people announce a project but never really get it off the ground. For example, weâve heard several people say they donât want to start a local effective altruist group because one already exists, but then the existing group soon becomes neglected or entirely inactive.
- No location available
-
6. Creating other coordination problems
Larger groups are harder to coordinate than smaller ones. Whether youâre doing research, advocacy or dealing with outsiders, joining a field obligates your peers to invest time making sure you and they are in sync. Furthermore, a lot of coordination relies on high trust, and itâs hard to maintain trust in a larger or shifting group where you donât have established relationships. Adding people to an area has some direct positive impact, but it also creates an extra cost in the form of more difficult coordination. This makes the bar for growing a cause (especially a small one) higher than it first seems.
- No location available
-
- [note::The cost of coordination]
How can you mitigate these risks?
1. Ideally, eliminate courses of action that might have a big negative impact
2. Donât be a naive optimizer
Your âmodelâ of the situation probably spits out a very uncertain answer about whatâs best (âknown unknownsâ). But then thereâs also the chance your model itself is wrong, and youâre thinking about the situation entirely wrong, and this could be in ways you havenât even considered (âunknown unknownsâ). And beyond that, itâs uncertain how to even reason about these kinds of situations â there isnât a single clearly accepted theory of what how to make decisions under uncertainty.So if you simply pick a goal that seems good to you, and aggressively pursue it, there are bound to be other important outcomes youâre ignoring.
- No location available
-
Some examples of being a naive optimiser:Trying to earn as much money as possible, so you can donate as much as possible (but ignoring issues around reputation, character, cooperative norms; as well as other pathways to doing good such as spreading important ideas and building career capital)Ignoring all considerations except for the one you think is most important (e.g. that there might be a lot of future generations)An issue for entrepreneurs is to get so convinced by the mission of their own organisation that they cut legal and ethical corners to make it succeed.
- No location available
-
There is no widely accepted solution to the problem of how to reason in the face of deep model uncertainty, but here are some ideas that make sense to us:Consider multiple models of the situation â many understandings of what matters, many potential outcomes, and many perspectives. This should include what conventional wisdom and other experts would say. You should actively seek out the best arguments against your approach. This gives you the best possible chances of spotting missing considerations.Take courses of action that either seem good according to many models, or that are very good on one perspective and roughly neutral on the others. In contrast, avoid courses of action that seem crazy on some reasonable perspectives.Be ambitious but not aggressive.
- No location available
- pluralism, rationality, strategy,
3. Have a degree of humility
Here are some more detailed tips on how to strike the balance:
Make a distinction between your âimpressionsâ (what seems best to you) and your âall considered viewâ (what you believe after taking account of other peopleâs views). Itâs important to develop your own impressions, so that youâre adding to the collective wisdom about what to do, but high-stakes action should generally be taken based on your all considered view.
- No location available
-
Put more weight on your own impressions the more reasons there are to trust your views (such as expertise or a track record), and when you can clearly point to information or values that you have which others donât. If a random uninformed person disagrees with your project, thatâs usually not worth worrying about.
- No location available
-
Gather lots of views. A large number of non-experts can easily be more accurate than a small number of experts.
- No location available
-
Weight someoneâs views more by track record than superficial indicators of expertise. In many fields (e.g. most social science), experts arenât much better at making predictions than random chance. The people with the best judgement are often informed generalists with the right mindset. Either way, try to evaluate how trustworthy people are based on their track record of making similar judgement calls.
- No location available
-
Weight the views of others by their strength, both in terms of the stakes and their degree of confidence. If even someone trustworthy believes your project is âmeh,â thatâs not a big deal either way. If theyâre convinced itâs very harmful, you should be a lot more cautious.
- No location available
-
Consider the stakes. In some contexts, like private intellectual discussion or doing small test projects, itâs OK to run with your wacky views. The greater the potential harms, the more important it is to consider a wide range of views.
- No location available
-
Donât rely only on your own judgement for high-stakes decisions. For instance, if youâre the CEO of a larger project, itâs important to have a board and strong cofounders to check your most important decisions.
- No location available
-
If others think your project isnât impactful (and havenât thought about it much), that isnât much reason to not do it. There will always be people who arenât excited about your project.
- No location available
-
4. Develop expertise, get trained, build a network, and benefit from your fieldâs accumulated wisdom
When entering a field, consider starting out by working for an established organisation with the capacity to supervise your work. Learn about the field and make sure you understand the views of established players. The time this takes will vary a lot according to the area youâre working in. We generally think it makes sense to work in an area for at least 1-3 years before doing higher stakes or independently led projects, but in particularly complicated areas â like those that require lots of technical knowledge or graduate study â this can take longer.
- No location available
-
5. Follow cooperative norms
6. Match your capabilities to your project and influence
Try to match your capabilities to your degree of influence and the fragility of the problems youâre working on. As the stakes get higher, the more vetting, expertise and caution to bring.
- No location available
-
As your project becomes more influential and successful, it becomes more important to keep seeking out advisors and colleagues who will stand up to you. Unfortunately the opposite is often the case: as you get more successful, people will be less inclined to doubt you, and more worried about criticising you. To avoid this, you may need to purposely set up structures and processes to limit the role of your judgement.
- No location available
-
7. Avoid hard-to-reverse actions
Hiring, growing your funding or increasing your influence normally look like concretely good things from the perspective of your project in the short-term â being bigger means more impact â but it can be more ambiguous when you consider the field as a whole and the long-term effects.Rapid growth is hard to unwind, since it would mean firing people, so you get locked into the new, bigger state. More people are harder to coordinate, and require training and management, which is often in short supply. This leads to more of the issues weâve covered, like unilateralism, breakdown of norms and errors of judgement. Going fast creates more reputation risks, which might have been avoided if youâd gone more slowly. (Ignoring these more diffuse harms is another example of naive optimization.)Of course, growing more slowly also has big costs; so itâs a question of balance. However, our sense is that the benefits of growth are usually more tangible than the costs, so people are more likely overestimate the value of growth than underestimate it.
- No location available
-
- [note::Potential impact, coordination/management costs, and reputational risks are all correlated with the size of any group.]