The Importance of GWWC Cohort Data

There are a few pieces of information that are required to properly analyze the value of Giving What We Can‘s membership.

They’re necessary for GWWC’s managers to evaluate different strategies. If GWWC was an object-level charity, we wouldn’t donate to it without knowing these numbers. And if GWWC were a public company, investors would not provide funding without such disclosure. As such, hopefully these metrics are already being collected internally, and publicly sharing them should not be very difficult, though very valuable. If not, GWWC should start collecting them!

GWWC already publishes the number of members it has at any given point and the total amount pledged. From this it’s easy to derive how many joined in any given year. However, it’s hard to judge what these people did later – how many fulfilled the pledge, and how much did they donate? Worse, this makes it hard to forecast the value of a new member, so we can’t tell how much effort we should put into extensive growth. As far as I can see (sorry if I just couldn’t find the data), we do not currently release the data required to make this analysis.

As part of it’s annual report, GWWC should release data on each cohort: how many of that cohort fulfilled the pledge by donating 10%; how many were ‘excused’ from donating 10% ( e.g. by being students); how many failed to abide by the pledge, donating less than 10% despite having an income; and how many did not respond.

Example Disclosure

In case it’s confusing what exactly I’m suggesting GWWC release, here’s an example (with totally made-up numbers). As part of it’s 2014 annual report, GWWC could report:

  • 2011 cohort:
    • Of the 107 who joined in 2011…
    • 75 donated over 10% in 2014
    • 15 were students and did not donate 10% in 2014
    • 10 had incomes but did not donate 10% in 2014
    • 7 could not be contacted in 2014
    • Total of $450,000 donated in 2014
  • 2012 cohort:
    • Of the 107 who joined in 2012…
    • 50 donated over 10% in 2014
    • 53 were students and did not donate 10% in 2014
    • 2 had incomes but did not donate 10% in 2014
    • 2 could not be contacted in 2014
    • Total of $300,000 donated in 2014
  • 2013 cohort:
    • etc.

While in the 2013 annual report, GWWC would have reported

  • 2011 cohort:
    • Of the 107 who joined in 2011…
    • 45 donated over 10% in 2013
    • 56 were students and did not donate 10% in 2013
    • 3 had incomes but did not donate 10% in 2013
    • 3 could not be contacted in 2013
    • Total of $250,000 donated in 2013
  • 2012 cohort:
    • Of the 107 who joined in 2012…
    • 16 donated over 10% in 2013
    • 89 were students and did not donate 10% in 2013
    • 1 had incomes but did not donate 10% in 2013
    • 1 could not be contacted in 2013
    • Total of $100,000 donated in 2013

This would allow us to see how each cohort matures of time, answering some very important questions:

  • How much is a member worth, after taking into account the risk of non-fulfillment?
  • How much does the value of a member differ with the discount rate we use?
  • How does the donation profile of a member change over time – does it rise as they progress in their career or fall as members drop out?
  • Are the cohorts improving or deteriorating in quality? Are the members who joined in 2012 more likely to still be a member in good standing in 2014 than they 2010 cohort were in 2012? Do they donate more or less?

There are some other numbers that might be nice to know, for example breaking the data down by age, sex, nationality, or even CEA employee vs non-employee, but it’s important not to impose too high a reporting burden.

Why this is not idle speculation

This might seem a bit ambitious. Yes, it would be nice if GWWC released this data. But is it really a pressing issue?

I think it is.

Bank problems: Extend and Pretend

Sometimes banks will make a series of bad loans – loans which are repaid at a significantly lower than expected rate, perhaps because the bank was trying to grow aggressively. When the first signs of this emerge, like people being late on payments, banks have two alternatives. The honest one is to admit there is a problem and ‘write down’ the loan – take a loss to profits. The perhaps less honest one is to extend and pretend – give the borrowers more time to repay and pretend to yourself/auditors/investors that they will come good in the end. This doesn’t actually create any value; it just delays the day of reckoning. Worse, it propagates bad information in the meantime, causing people to make bad decisions.

Unfortunately they neglected the Litany of Gendlin:

What is true is already so.
Owning up to it doesn’t make it worse.
Not being open about it doesn’t make it go away.
And because it’s true, it is what is there to be interacted with.
Anything untrue isn’t there to be lived.
People can stand what is true,
for they are already enduring it.
—Eugene Gendlin

GWWC: Dilutive Growth?

About a year ago, people were concerned that GWWC’s growth was slowing – only growing linearly, rather than exponentially. This would be pretty bad, and people were justifiably concerned. However, GWWC made a few changes with the aim of promoting growth. Most pertinently:

  • Allowing people to sign up online, rather than having to mail in a hand-signed paper form. This happened between April and June 2013.
  • Adjusting the pledge to become more cause-neutral, rather than just about global poverty. This happened late 2014.

GWWC signups labeled

source

The latter change was somewhat controversial, but I didn’t see much discussion of the former at the time.

The concern is that, though these measures have increased the number of members, they may have done so by reducing the average quality of members. Making it easier to join means more marginal people, with less attachment to the idea, can join. This is still good if their membership adds value, but they dilute the membership, which means we shouldn’t account for the average new member being signed up now as being equally valuable as the members who joined up in 2010. Additionally, the reduction in pomp and circumstance might reduce the gravitas of the pledge, making people take it less seriously and increase drop-out rates. If so, moving to paperless pledges might have reduced the value of sub-marginal members as well as diluting them.

The comparison with banks should be pretty clear – a bank that’s struggling to grow starts accepting less creditworthy applicants so it can keep putting up good short term numbers, but at the cost of reducing the long-run profitability. Similarly GWWC, struggling to grow, starts accepting lower quality members so it can keep putting up good short term numbers, but at the cost of reducing the long-run donations. This makes it harder to forecast the value of members, and might lead to over-investment in acquiring new ones.

This seems potentially a big risk, and it’s the sort of issue that this data would allow us to address. Of course, there are many other applications of the data as well.

And GWWC in fact has even stronger reasons than banks to report this data. The bank might be wary of giving information to its competitors, but GWWC has no such concerns. Indeed, if releasing more data makes it easier for someone else to launch a competing, better version of GWWC, all the better!

If you liked this you might also like: Happy 5th Birthday, Giving What We Can and GiveWell is not an Index Fund

Against Double-Counting Virtue: the Many Faces of Value

At times I fear that the highlight of my blogging career will be pointing out minor errors in Scott’s otherwise excellent posts. But his quality is so high that this is probably a commendable achievement anyway – or so I reassure myself. Anyway, now I’m back from the Midwest and have internet again, I can continue this sacred quest.

Recently, Scott wrote a post about how charity is a better mechanism than political activism for discharging any positive moral obligations we might have. As an aside, he points out that it’s credible that he actually has no such obligations, as his net impact on the world is positive anyway:

The marginal cost of my existence on the poor and suffering of the world is zero. In fact, it’s probably positive. My economic activity consists mostly of treating patients, buying products, and paying taxes. The first treats the poor’s illnesses, the second creates jobs, and the third pays for government assistance programs.

However, while I agree that ordinary productive members of society plausibly do not have such obligations, this argument involves double-counting. He basically does two positive things:

  1. Treating patientshttps://effectivereaction.wordpress.com/wp-admin/post.php?post=142&action=edit&message=1 in his job (imagine Scott does surgery)
  2. Buying things (say bread)

(We’ll ignore the bit about taxes.1)

Now consider Scottlynn, the farmer. She figures her net impact is positive as well. She basically does two things:

  1. Growing corn and turning it into bread
  2. Buying things (say surgery)

Yet now we have the same activities appearing in two people’s lists!  Scott considers both his buying from, and selling to, Scottlynn as his positive impact. Yet Scottlynn counts those exact same transactions as part of her positive impact!

This is because (ignoring for the moment the inherent dignity that comes from productive work) Scottlynn doesn’t really value being employed per say – she values it because it gets her money, which she can then use to buy surgery. Buy surgery… from Scott. He shouldn’t count his purchases as part of his positive impact; the only reason it’s good is because it will later on allow her to buy the other thing he counted as positive impact! To avoid double-counting, he needs to pick one or the other. This is similar to how we have multiple ways of measuring GDP – the production method, the income method and the expenditure method – but we can’t mix them together.

The money/price system serves three purposes; it rations scarce resources, it signals which need to be produced more, and it incentives their production. But it is only a means to an end; ultimately, what matters is the goods and services that are produced, and that they go to those who want them.

Some macroeconomic theories do suggest that there might be times when simply spending money is a good thing; for example, if you have cyclical under-utilization of resources due to sticky prices. But it is unclear if that’s actually ever really an issue; other equally plausible theories of macroeconomics say it’s not. And even if the Neo-Keynesians are right, their theory implies that sometimes (when you have cyclical over-utilization of resources) simply spending money is a bad thing! Over the long run the economy operates at above and below capacity roughly as often, so these two effects will cancel out.

I’m sure there are other positive non-charity things Scott does. For example, he’s probably pleasant to his coworkers, and he creates a lot of value through his blog. But buying stuff isn’t one of them.

 


  1. We’ll assume Scott works for the private sector, and otherwise ignore the government, though if you thought the government was very evil, or taxes very immoral this might change the conclusion. 

2014 charity choice: The Machine Intelligence Research Institute

This year, I’m donating to MIRI. Here’s a quick summary of the classic argument:

Even ignoring the risk of UFAI, I think that FAI may be one best ways of preventing run-away value drift from destroying all value in the future.

Happy 5th Birthday, Giving What We Can!

Giving What We Can recently celebrated its 5th birthday. It’s not much of a party if no-one external congratulates you, so here we go: Happy Birthday, GWWC!

It’s pretty impressive how much GWWC has grown since those early days. Here’s a chart of total membership, which I’ve put together from GWWC emails and liberal use of the internet archive. I’m sure they have better data (without gaps!) internally, but I’ve never seen this chart before. Notably, growth seems to have picked up since the fall of 2013. Did GWWC change their strategy at that point? (or their membership-counting-methodology?)

Lines going up are always good

Putting the same chart on a log scale, we can see that GWWC have actually done a reasonably good job of sustaining exponential growth.

Lines that go up on a log scale are even better!

Fitting a line of best fit to the chart, I estimate GWWC’s membership is growing 73.1% a year. Assuming 2% population growth, it will take just 30.25 years before all the world’s population are GWWC members. Taking over the world by the time I’m 58 sounds like pretty good going!

Happy Birthday, Giving What We Can!

GiveWell is not an Index Fund.

Someone1 on facebook recently asked

Can we think of donating to GiveWell- or GWWC-recommended charities as being the philanthropic analogue of investing in an index fund? In the sense that it may be possible in principle to do better, but it’s close to the best among readily available options, and almost everybody who tries to do better will do worse.

I think this analogy overlooks some important points about the underlying structure.

An Index contains all the stocks that satisfy some very broad criteria – for example, the S&P500 is basically “is very large US company”. Index funds invest in all the stocks in a specific index.2 Index funds try to do as well as the index on the whole – no better, no worse.

Active managers, on the other hand, by small subsets of the stocks in the index, and try to beat the index. That is, they try to buy stocks that will do better than the average stock in the index.

Part of the appeal of index funds is that many people think that all stocks have basically equally good prospects, ex ante, due to the Efficient Market Hypothesis.There are good reasons to think that free markets are in general very efficient.

Givewell looks at a large number of charities, and selects a very small number to recommend. In this way they’re much more like an active manager than an index fund. An index fund for charities would mean spreading out your donations between thousands of different charities.

There are important differences between the stock market and the charity market. Virtually any time you think you see an inefficiency in the stock market, you are probably wrong. But this does not apply to the charity market. Effective Altruists habitually and credibly claim that there are many orders of magnitude difference between the expected values of different charities (pdf). And the sorts of arguments about risk and diversification that motivate balanced portfolios of investments simply do not apply to charities; with charities, you should pick the best one and donate everything to it.3

Some perhaps Index Funds are Givewell are similar insomuchas they are both good things. But so is chocolate.

This analogy glosses over important differences, and potentially causes sloppy thinking. Just because two things are good doesn’t mean they are good for the same reasons or in the same way. Effective altruists would do well to understand that markets are not like charities. Others would do well to understand the there is no Efficient Charity Hypothesis!


  1. who will remain anonymous until they request otherwise. 
  2. technically some merely invest in representative sub-baskets, but S&P500 indexes probably buy all the constituents. 
  3. unless you are very rich. 

Laying down the Law

Recently Bill Barlow wrote an excellent piece in the Harvard Law Record, recommending law students go into corporate law so they could donate money to charity. Sima Atri wrote a response, to which Jeff Kaufman wrote a convincing rebuttal. I’m going to address one very small part of Atri’s article, which to be fair she probably didn’t put much thought into.

Because if you choose to go into Big Law and care about the poor and otherwise marginalized, giving your money to charity is the least you can do. I say this because you are not choosing to go into neutral, apolitical, work. None of us working in the legal profession are. Your firm, Bill, has represented JPMorgan Chase, a bank that backed thousands of predatory and racist loans and helped create the foreclosure crisis.

I think this attack on Bill’s firm is almost impressive in how mistaken it is. Specifically, I think
1) The banks were not at fault for what it did
2) What it did was not predatory
3) What it did was not racist
4) Even if 1,2,3) are all wrong, it’s still good to represent banks.
This is a lot to show, so lets begin.

1) The banks were not at fault.

Yes, banks lent to many poor people: subprime lending. But in part this was because the government forced them to make these loans, in an attempt to promote home-ownership. If anyone if to blame, it is the government, not the banks. Left to themselves, the banks would have preferred not to lend to such people, as they present higher risks. So it’s strange to blame the banks for this, who had little choice in the matter. The high default rates among minorities during the crisis was the result of government intervention, not the fault of the banks.

2) The banks were not predatory

Given 1), the worst we can really accuse the banks of is “only following orders” or being involuntarily predatory. But even this isn’t the case. Banks didn’t force anyone to borrow money. Having the opportunity to take out a loan is a benefit – it’s the reason we try to maintain good credit scores! As Arnold Kling noted, borrowers get a free option on rising house prices. Being given the option to borrow money is a good thing – you’re not taking advantage of people by giving them choices. (And stories about people not understanding that their repayments rates would rise don’t work, as the defaults frequently happened among people still paying the introductory rate.

Indeed, one part of the crisis was the so called ‘NINJA’ loans, where borrowers frequently defrauded the banks. It is hard to see how the banks can be taking advantage of people by being defrauded.

3) The banks were not racist

Indeed, JPMorgan Chase made many loans to minorities. As we noted in 1), left to their own devises, banks apparently made too few loans to minorities – they were accused of being racist for refusing to lend money. This was why the Community Reinvestment Act was passed. So it seems very strange to accuse them of being racist for making too many loans to minorities as well. Maybe it’s just impossible to win.

4) Even if 1,2,3) are all wrong, it’s still good to represent the banks.

Ok, so suppose 1),2) and 3) are all false; the banks are truly evil companies. Is it therefore bad to defend them in court? Well, the adversarial court system depends on their being lawyers available to defend the guilty. The ACLU often defends extremely unpopular causes, because it is in these cases that harmful precedents are most likely to be created. In the same way that anti-terrorist laws, originally aimed only at extremely dangerous and unpopular people are now being abused in much more mundane situations, so too will the extraordinary legal steps being used against the big banks one day be used against other targets.

I realize that this was just a small part of Atri’s article. But it’s worth commenting on anyway.

Concern for those we know not

Many social movements involve attempts to improve the welfare, rights or status of the movement’s own members. For example:

  • Nationalist Parties: try to support the people in the country, and are generally mainly supported by people in that country. It’s rare to very actively support another country’s nationalist movement, unless as a proxy in a war.
  • Labor Unions: at least initially, these were formed of working class people trying to benefit themselves.
  • Feminism: though there is considerable debate about the definition, this is generally considered to be about supporting women, and has over twice as many female supporters as male supporters. This is especially unusual when you realize that most social movements (including effective altruism) are primarily male.*

In other examples, the people in the movement are closely related to but distinct from the supposed beneficiaries:

  • Home-schooling: the parents who lobby for the legality of home-schooling are too old to benefit from it themselves, but do hope to benefit their children.
  • Soup Kitchens: people donating to soup kitchens probably have enough to eat themselves, but they hope to benefit others in their community.
  • Upper-class socialists, straight LGBT activists, male feminists and so on would also fit into this category.

Effective Altruism takes this one step further, however. Not only do most EAs care about people with little regard for nationality, most of our causes have beneficiaries extremely remote from ourselves:

  • Third World Hunger/Health: Virtually all EAs are part of the middle classes of the developed world. Few have ever been to Africa, and fewer still have ever met a beneficiary of GiveWell. Yet EAs continue to send large amounts of money to them, motivated only by abstract benevolence.
  • Animal Rights: Very few EAs have been to a factory farm, and the animals won’t reciprocate our concern. I guess everyone has seen animals in person, but rarely the intended beneficiaries.
  • Existential Risk: Here, the benefits mainly accrue to people so remote they don’t even exist yet.

I wonder if this is related to the typical backgrounds of effective altruists: physics, math and philosophy, all of which are extremely abstract, and rely on generalizing ideas from the specific to the general. Perhaps only those with a case of memetic immune disorder are capable of forgetting the original purpose of ethics was reciprocal altruism and kin selection, and instead generalize it to include people they have never met and never will.

I can think of only a few examples of other social movements with beneficiaries as remote:

  • Anti-Slavery in northern England: the Manchester cotton mill workers supporting abolition, even though they had never met a slave, and actually directly personally benefited from slavery.
  • The Pro-Life Movement: pro-life activists can hardly be said to be directly benefiting, and nor have they ever met an unborn child (though they may have seen ultrasounds, EAs have seen pictures of third world hunger). Pro-abortion people would argue that this case is almost identical to the Existential Risk case, as the beneficiaries aren’t yet people.

These examples do not seem to support my memetic immune disorder theory: Lancaster mill workers were not well-known for their educational level. But England as a whole was very well educated, and banned the slave trade for apparently largely altruistic reasons.

 


 

*I realize there is much debate on these points; some people argue that feminism is good for men, some that it is bad for women, and the YouGov article even argues there is little gender difference in support, though I think they have made the motte and bailey error. But you are welcome to substitute your own examples.

RCT as I say, not as I do

Randomized Controlled Trials (RCTs) are the gold standard in policy evaluation.

Say you’re investigating a third world development policy, like building schools, or installing water pumps, or distributing malaria-resistant bednets. A random sample of the villages in an area are selected to receive the policy. The other villages form the control group, and receive no special treatment. Metrics on various desiderata are recorded for each village, like income, lifespan and school attendance. By comparing these outcomes between villages with and without the intervention, we can judge whether it made a statistically significant difference.

RCTs give us strong evidence of a causal link between the intervention and the result – we assume there were no other systematic differences between the treatment and control villages, so we have good grounds for thinking the differences in outcome were due to the intervention.

This is a marked improvement over typical methods of evaluation. One such method is simply to not investigate results at all, because it seems obvious that the intervention is beneficial. But people’s intuitions are not very good at judging which interventions work. When Michael Kremer and Rachel Glennerster did a series of education RCTs in Kenya, all their best ideas turned out to be totally ineffective – plausible ideas like providing textbooks or teachers to schools had little impact. The one thing that did make a difference – deworming the children of intestinal worms – was not something you’d necessarily have expected to have the biggest impact on education. Our intuitions are not magic – there’s no clear reason to expect our to have evolved good intuitions into the effectiveness of developmental policies.

A common alternative is to give everyone the intervention, and see if outcomes improve. This doesn’t work either – outcomes might have improved for other reasons. Or, if outcomes deteriorated, maybe they would have been even worse without the intervention. Without RCTs, it’s very difficult to tell. Another alternative to RCTs is to compare outcomes for villages which had schools in the first place to those which didn’t, before you intervene at all, and see if the former have better outcomes. But then you can’t tell if there was a third factor that causes both schools and outcomes – maybe the richer villages could afford to build more schools.

The other main use of RCTs is in pharmaceuticals – companies that develop a new drug have to go through years of testing where they randomly assign the drug to some patients but not others, so we can be reasonably confident that the drug both achieves its aims and doesn’t cause harmful side effects.

One of the major criticisms of RCTs is that they are unfair, because you’re denying the benefits of the intervention to those in the control group. You could have given vaccinations to everyone, but instead you only gave them to half the people, thereby depriving the second half of the benefits. That’s horrible, so you should give everyone the treatment instead. This is a reasonably intelligent discussion of the issue.

But this is probably a mistake. Leaving aside the issue that it’s more expensive to give everyone the treatment than a subset (though RCTs do cost money to run), it’s a very static analysis. Perhaps in the short term giving everyone the best we have might produce the best expected results. But in the long term, we need to experiment to learn more about what works best. It is far better to apply the scientific method now and invest in knowledge that will be useful later than to cease progress on the issue.

Indeed, without doing so we could have little confidence that our actions were actually doing any good at all! Many interventions received huge amounts of funding, only for us to realize, years later, that they weren’t really achieving much. For example, for a while PlayPumps – children’s roundabouts that pumped drinking water – were all the rage, and millions of dollars raised, before people realized that they were expensive and inefficient. Worse, they didn’t even work as roundabouts, as the energy taken out of the system to pump the water meant they were no fun to play with.

Another excellent example of the importance of RCTs is Diacidem. Founded in 1965 by Lyndon Diacidem, it now spends $415 million a year, largely funded by the US government, on a variety of healthcare projects in the third world, where it deliberately targets the very poorest people. Given that total US foreign aid spending on healthcare is around $1,318 million, this is a very substantial program.

Diacidem have done RCTs. They did one with 3,958 people from 1974 to 1982, where they randomly treated some people but not others. The long time horizon and large sample size makes this an especially good study.

Unfortunately, they failed to find any improvement on nearly all of the metrics they used, and as they used a 5% confidence interval, you’d expect one to appear significant just by chance.

 “for the average participant, any true differences would be clinically and socially negligible… for the five general health measures, we could detect no significant positive effect… among participants who were judged to be at elevated risk [the intervention] has no detectable effect.

Even for those with low income and initial ill health, surely the easiest to help, they didn’t find any improvements in physical functioning, mental health, or their other metrics.

They did a second study in 2008, with 12,229 people, and the results were similar. People in the treatment groups got diagnosed and treated a lot more, but their actual health outcomes didn’t seem to improve at all. Perhaps most damningly,

“We did not detect a significant difference in the quality of life related to physical health or in self-reported levels of pain or happiness.”

Given that these two studies gave such negative results, you would expect there to be a lot more research on the effectiveness of Diacidem – if not simply closing it down. When there are highly cost-effective charities than can save lives with more funding, it is wrong to waste money on charities that don’t seem to really achieve anything instead. But there seems to be no will at all to do any further study. People like to feel like they’re doing good, and don’t like to have their charity criticized. Diacidem is political popular, so it’s probably here to stay.

Sound bad?

Unfortunately, things are far worse than that. Diacidem does not actually cost $415 million a year – in 2012, they spent over $415 billion, over 300 times as much as the US spends on healthcare aid. It wasn’t founded by Lyndon Diacidem, but by Lyndon Johnson (among others) Nor does it target the very poorest people in the third world – it targets people who are much better off than the average person in the third world.

The RCTs mentioned above are the RAND healthcare experiment and the Oregon healthcare experiment, with some good discussion here and here.

Oh, and it’s not actually called Diacidem – it’s called Medicaid.