Alan Dwarst has put out a new essay on the effectiveness of donating towards online veg ads. It's an update to his earlier essay on donating towards veg leaflets (that I looked at), and it's based primarily on survey data that Nick Cooney collected about the behavior of people who after clicking on Facebook ads for the Hidden Face of Food went on to click 'like' (7%) or request a Vegetarian Starter Pack (VSP) (1.5%).

I don't know how they contacted these people to give them the web survey: for the VSP they only optionally ask for your email and they say it's for a newsletter (so the people who give it might be more veg-inclined) and for 'like'ing I don't think they can do anything but post something to their followers which many won't see (in which case the claimed 10% response rate sounds way high.)

Update 2012-03-18: I made a followup post on the response rate.

Somehow, however, they sent the web survey to about 3% of the people who 'like'd their page and 40% of the people ordering a VSP. [1] The survey asked (details) about how people had changed the quantity of their consumption of several products: red meat, chicken, fish, eggs, dairy, meat-substitutes, and dairy-substitutes. The options were 'increased', 'stayed the same', 'decreased slightly', 'decreased significantly', 'no longer eat', and 'never ate'. First off, this asymmetry in the response, where only one category is for increase and three are for decrease worries me: it indicates that you should have decreased your consumption and they want to know how much. Similarly, they start with a "Since visiting the HiddenFaceOfFood.com website", which probably reminds people of the video HFoF showed them and elicits more anti-meat responses.

Even though they didn't ask "did this site make you go vegetarian", they give numbers for that: 26% of 'like'ers and 31% of VSPers. It looks like they calculated these by assuming that the people who stopped eating the three kinds of meat were all the same people [2] which is reasonable, though probably overstates it slightly.

Update 2012-03-08: Alan confirms this is what Nick did.

While sending a VSP surely has costs, that sounds like it's on the same order as the cost of getting a user to request one ($6.51: $5000 total cost for 768 VSP-requesters) so it probably doesn't change things too much.

Update 2012-03-09: Alan tells me that Nick told him they cost $0.20 to print and $0.28 to send, and it's all automatic so no time cost for him.

The biggest thing that worries me is that Facebook reports only 5330 'like's when they should have had 3524 in August 2011 alone (917 direct vegetarians at a 26% conversion rate for 'like'ers) and so should be up to at least 30K after spending ten times as much. I had thought this might just be that they were using many landing pages, but a site search shows that all their landing pages I checked pull the same Facebook comments and have the same 'like' count of 5330. Unless they're running multiple sites?

Update 2012-03-10: Alan tells me that having too few likes is because they've needed to refresh the page from scratch a few times to adjust settings. He remembers seeing ~20K likes recently.

The voluntary response nature of the survey is also worrying, but Alan's correction of assuming that everyone who didn't respond made no changes to their behavior is a clunky but effective way to eliminate that [3]. With these other issues, I'm not sure that $52 per new vegetarian really is "a very conservative maximum cost".

I'm also still interested in what fraction stay vegetarian.

(Even though I'm interested in this as an evaluation question, I'm not interested in this as a way of finding where to give because I don't value the suffering of animals.)

[1] They don't give sufficient numbers to determine this, but Alan says Nick tells him that they got a 10% response rate, and the survey has 44 respondents for 'like' and 60 for the VSP. The 'like' respondents were "Polled 0-5 months after 'liking' the page" while the VSP ones were "Polled 2-4 months after visiting the site". In the month of August, the only one they give numbers for, we can calculate they had 3524 'like'ers (917/.26==1339/.38) and 768 VSP-requesters (238/.31==300/.39). These are consistent with their 7% 'like' 1.5% VSP statistics, giving 51K total visitors (3524/.07==768/0.015) and a $0.10 CPC ($5000 total expenditure. Alan says Nick says costs have since gone up so this would now be about $6000, so a $0.12 CPC) Assuming that the "0-5 months" and "2-4 months" refer to the people they tried to contact and not just respondents, and that all months were the same, their 10% response rate indicates they sent surveys to 2.6% ((44*10)/(5*3524)) of their 'like'ers and 39% ((60*10)/(2*768)) of their VSPers. This makes me dubious about the response rate claim: did they really choose to survey these two groups so differently?

[2] Unless they had non-agreggated data to work off of too? But they did label this section "Raw Survey Data".

[3] Unless the 10% response rate figure isn't accurate; I'd like more data on that. If it turns out to be 20% for VSPs and 1% for 'like's then that changes things a lot. I've written to the author.