Effectiveness is difficult to define, but here’s some stats that should give some context.
When recruiting participants for the study published in the PNAS paper, our Facebook ads (see examples here) had on average a 2.8% (SD = 1.4%) click through rate. That is, of all the people who were shown an ad, 2.8% of people chose to click on the ad. Of those 2.8% that clicked on the ad, on average about 16% of them completed the survey (valid responses; i.e., cases where we could use the data for analyses).
Overall, we paid about 1,500,000yen in total in Facebook advertising fees. Taking into account that we ended up with just over 16,000 valid responses, that works out to be around 95yen per valid response.
Of course, there was a very big difference between countries when it came to cost per click, click through rates, and response rates. If you’re looking at these numbers and trying to estimate costs for your own study in one particular country, you’re very unlikely to get an accurate estimate. That’s because you can have a country whose cost-per-click is very low (e.g., Egypt in our case), but due to a very low response rate (e.g., Egypt in our case), the cost per valid response ends up higher than a country with a high cost-per-click (e.g., Japan in our case) but high response rate (e.g., Japan in our case).
Here’s an Excel sheet that breaks down all these things by country for our study. NOTE: This sheet was prepared in 2016 based on preliminary data. Country N’s may differ from the PNAS paper. It also doesn’t include South Korea data. Almost 6 years has passed since we did the first pilot using Facebook ads – since then costs may have increased. Also really important to note is that we used a fairly strong intrinsic incentive to encourage people to take part – an informal ‘analysis’ of their friendships and romantic relationships based on responses they provided in the survey (see the Behind the Scenes for more details). While we’re not experts on online survey motivation, we feel that graphic design, incentives, survey length, survey content, as well a myriad of other things will affect how well recruitment will work using Facebook ads. Take our experiences with a grain of salt 🙂