Annual Fundraiser: Checking Banner Results
Hey All–
We’ve been tracking a huge amount of data during this year’s fundraiser so we can better understand which messages work well and which don’t. We have two sets of banners that we set each day to run on all Wikipedia languages. Set one is the English version; set two is all non-English versions.
We have two sets because we want our banners to run globally only if they are translated…which can take some time and volunteer effort. This is why our non-English banners rotate slowly. However, with English banners, we can build a banner quickly and put it up to see how it does.
Let’s go into some detail on selecting a rotation. On December 3rd, our rotation and results (pp = Paypal, cc = credit card):
| % of Total | Site Notice | Payment | Number of | Total | Average | Highest | ||
| WP views | Type | Donations | Amount | Gift | Donation | |||
| 12/3/09 | 20% | 2009_Notice17 | pp | 210 | $4,933.75 | $23.49 | $100.00 | |
| 12/3/09 | cc | 156 | $5,894.76 | $37.79 | $500.00 | |||
| 12/3/09 | 20% | 2009_Notice18 | pp | 725 | $11,807.41 | $16.29 | $1,000.00 | |
| 12/3/09 | cc | 454 | $10,145.52 | $22.35 | $250.00 | |||
| 12/3/09 | 40% | 2009_Notice30_bold | pp | 504 | $11,023.15 | $21.87 | $250.00 | |
| 12/3/09 | cc | 389 | $14,468.07 | $37.19 | $1,000.00 | |||
| 12/3/09 | 20% | 2009_Notice36 | pp | 207 | $4,650.90 | $22.47 | $120.00 | |
| 12/3/09 | cc | 147 | $5,890.83 | $40.07 | $250.00 |
As you can see, we had three different notices running at 20% and one banner, taken from one of the better notices from 2008′s fundraiser, 2009_Notice30_bold running at 40%. It did well throughout last week.
As you can see, 2009_Notice18 pulled in a huge number of gifts despite only showing 20% of the time. Also, it had a significantly lower average gift…probably as a result of the message itself. Despite the low average gift, people seemed to really respond to the message…and donated lots.
We are wary of banner-fatigue and saturation, where users might be tired of seeing the same message, so we changed banners around for the next day.
Looking at December 4th, 2009:
| % of Total | Site Notice | Payment | Number of | Total | Average | Highest | ||
| WP views | Type | Donations | Amount | Gift | Donation | |||
| 12/4/09 | 20% | 2009_Notice17 | pp | 192 | $4,280.08 | $22.29 | $250.00 | |
| 12/4/09 | cc | 144 | $4,778.83 | $33.19 | $250.00 | |||
| 12/4/09 | 20% | 2009_Notice18 | pp | 611 | $9,511.88 | $15.57 | $250.00 | |
| 12/4/09 | cc | 390 | $9,390.74 | $24.08 | $500.00 | |||
| 12/4/09 | 20% | 2009_Notice30_bold | pp | 266 | $6,573.39 | $24.71 | $1,024.00 | |
| 12/4/09 | cc | 228 | $6,696.20 | $29.37 | $238.75 | |||
| 12/4/09 | 20% | 2009_Notice36 | pp | 205 | $4,399.75 | $21.46 | $166.53 | |
| 12/4/09 | cc | 162 | $5,018.47 | $30.98 | $250.00 | |||
| 12/4/09 | 20% | 2009_Notice40 | pp | 320 | $7,795.45 | $24.36 | $1,000.00 | |
| 12/4/09 | cc | 187 | $6,113.04 | $32.69 | $500.00 |
We introduced 2009_Notice40 (“Thanks, Wikipedia.”) to the mix and cut back on another. All five banners in rotation are at 20%. Two of the banners are greatly outperforming the others. We like what #40 is doing…but #18 is still rocking…1001 donations while the next closest is 507 donations (#40).
Again, we switched things up, removing #17, and adding 2009_Notice22, a similar, but opposite message to #18, which has been successful.
December 5th, 2009:
| % of Total | Site Notice | Payment | Number of | Total | Average | Highest | ||
| WP views | Type | Donations | Amount | Gift | Donation | |||
| 12/5/09 | 20% | 2009_Notice18 | pp | 518 | $8,207.09 | $15.84 | $250.00 | |
| 12/5/09 | cc | 314 | $6,866.38 | $21.87 | $250.00 | |||
| 12/5/09 | 20% | 2009_Notice22 | pp | 166 | $4,634.09 | $27.92 | $250.00 | |
| 12/5/09 | cc | 135 | $4,938.74 | $36.58 | $250.00 | |||
| 12/5/09 | 20% | 2009_Notice30_bold | pp | 272 | $6,989.63 | $25.70 | $250.00 | |
| 12/5/09 | cc | 197 | $8,005.98 | $40.64 | $1,000.00 | |||
| 12/5/09 | 20% | 2009_Notice36 | pp | 195 | $4,440.20 | $22.77 | $191.00 | |
| 12/5/09 | cc | 157 | $7,092.57 | $45.18 | $1,000.00 | |||
| 12/5/09 | 20% | 2009_Notice40 | pp | 279 | $5,881.04 | $21.08 | $250.00 | |
| 12/5/09 | cc | 168 | $6,259.51 | $37.26 | $1,000.00 |
This day is a fascinating look at our banners and our user population. Notice the results of #18 and #22…they are similar yet contrasting messages. #18 is an quote from a small dollar donation (USD 1.95), acknowledging the sincerity of the gift. #22 is a quote from a high dollar donation (USD 200), emphasizing that a big gift is a small matter.
Compare the numbers of gifts for the two banners:
#18: 832 gifts, $15073.47 total
#22: 301 gifts, $9572.83 total
Not close right? But look at the average gift sizes:
#18: $15.84 for Paypal, $21.87 for credit card
#22: $27.92 for Paypal, $36.58 for credit card
That’s quite a difference. What was it about the message that would account for that? Is it possible that our more affluent donors were more interested in #22, while other donors were affected by #18?
Post your thoughts below.
Rand Montoya
Head of Community Giving
Wikimedia Foundation
You are only measuring one side of the picture, of course. This is saying which banners get how much money – the positive side – but you also have to measure the negative side – which banners do people find the most offputting, and would discourage people from reading or contributing to Wikipedia. Not sure how you can do that, but I think you need to, in order to get teh full picture.
I would expect there to be a strong geographic dependency – both country-based and also sub-country-based – on which donation banner is most appealing. The ratio of paypal to credit card donations also has useful information, particularly on the 5th November. Note how those that use credit cards give a substantially higher average donation. However, interestingly, the highest donations of $1000+ don’t appear to depend on the payment mechanism. There is a rich set of statistical information here that can be used to determine which banners appeal to which types of people.
… that’s the statistician side of me talking. The other side of me wants to say “why didn’t you thank donors in the first sentence of your blog post?” :-) I also worry that there is too much focus on money, money, money here – yet that’s not the only way that people can donate. They could pledge their time (as a lot already do by helping put Wikipedia and its sister projects together), or content that they have but isn’t available (text, photographs, videos, etc.), or their abilities, etc. Not everyone can donate money to help spread knowledge freely, and not everyone can contribute that knowledge or their time, etc. – but everyone can do something.
I am not sure that averages and max donation is a sufficient metric. What would be interesting would be the median and maybe the 80th percentile or something like this, in order to cut outliers out. Single donations of 500+ $ are probably more often than not thought in advance and are not influenced by the actual banner that happens to be there at that very moment. So removing those outliers should give us much cleaner data on what works and what not… any chance to get access to the raw data? :)
Am I the only one, or is there something somewhat gross about engineering messages to extract the greatest possible donations? Maybe this is standard practice in fundraising, maybe it’s the right thing to do…but something about it doesn’t sit quite right. There’s something of a “we’ll say whatever, just hand over the cash” in it.
I mean, how far do you take it? Experiment with different formulations of the same message? Randomly insert different adjectives, and measure which is best? It’s a little close to what the most evil marketers do, paying consultants big bucks to tap directly into the human psychology for maximum profit…
I don’t know that I’m opposing it…just curious whether anyone else feels slightly grubby about it all.
Looking at December 5th: #22 is the worst message, and doesn’t raise bigger donations (cf. #30 medium donation): put it off. #36 doesn’t raise much, but it’s important to convey the “non-profit” message and to show the thermometer, so keep it.
Small donations are the way: add other messages like #18, or show it more often.
I really like the idea of quoting donor comments. Here’s a coupla good ones:
(I couldn’t put the complete addresses in because it looked like spam)
http://wikimediafoundation.org/wiki/Special:ContributionHistory?
then add
offset=1260762541#361327
offset=1260762541#361337
offset=1260762541#361301
offset=1260760755#361263
offset=1260760755#361251
I give a donation but felt that is unfair that I donate more and another not donate at all. I think it is better to show banners like #18, beacuse is do my donate more easily.
Thanks for your work
Yoni
Steve (at #4): I think it depends.
Even as people, when we ask others for something, we use a lot of well-honed machinery in doing so. As kids, we learn how (and how not) to ask for things. As adults, we’ve mainly forgotten the learning process while retaining the skill, so it seems more natural. But watching my nephew grow up reminds me how much practice has gone into it.
The difference for me is to what extent the message stays authentic. Do our numerical methods merely help us to be better at getting across what we want to say? Or do they lead to us saying things that we don’t really mean because it pays better?
So far, it all looks pretty authentic to me. I especially feel that way about the comments of donors; there’s no reason to think they’re lying about why they donate, and unlike us they don’t have a conflict of interest.
funny, have to say funny. A site that lets anyone add to anything, asking for money. Got to hand it to you wish I had the balls to have money and ask people to give me theirs. Funny world you people live in.
I like the message that encourages any donation, however small. It gets the point across that even if we don’t have much, that “this is our baby”. I remember the crummy old days of Encarta and having to buy new encyclopedia CDs every time something happened in the world. A resource that is ours, and when I say ours I mean everyone on the internet, and we can add entries as needed and change others if they’re incomplete. I donated ten dollars, and I suppose if things don’t go well enough I’ll donate ten more.
My comment in the donation jokingly noted that if I donated a nickel for every click onto a wikipedia page the fundraiser would be over! And it’s so true! Count the clicks you make, and let that factor into how much you think wikipedia is worth and donate accordingly.
The banners discussed in #4 (Steve) and #8 (William) are mostly (all?) donor quotes. There must have been thousands and thousands of these donor comments, and the fundraising team presumably picked about half a dozen they thought were “marketable” or whatever you want to call it.
It’s pretty similar to a book publisher printing (excerpts of) reviews on the back cover. Of course the publisher will only print positive reviews, and then only those excerpts of them that they think are most positive/marketable/likely to convince people to buy the book/whatever. People who read the back cover presumably know or at least expect this, but despite that this practice seems to work.
So what the fundraising folks are doing here (picking the most positive quotes we got and advertising with them) is nothing new. However, they have the luxury of seeing how well each quote works almost in real time and course-correct based on that data. This isn’t really feasible with books, but if it were (e-books?) I’m sure book publishers would do the same thing. They probably just wouldn’t be as transparent about it.
R.Stonecipher (#9), I guess you must be new to civilised society. Wikimedia is what we call a charity, and it relies on goodwill donations of both time and money in order to provide society with a valuable resource free of charge.
If you don’t think Wikimedia is delivering enough value to justify its income, or you don’t have the money to contribute, the fundraiser is not for you and nobody is condemning you for not donating.
But if you really have an issue with the fact that non-profit organisations have operating costs then you’re on your own.
@Will (#12) i would rather they have ad revenue and somehow profit share with the people who have helped build it (the volunteers not jimbo wales the guy who COFOUNDED wikipedia). of course, it probably wouldnt allow him to jet set around the world if he “gave back” to the community that has built wikipedia. why does wikipedia need a spokesperson? to garner more donations? i trust jimbo wales about as much as a politician.
@Steve (#4) i have to agree. asking for donations is one thing. i remember quite a while back jimbo wales saying wikipedia would never have ads. lately as i see banners more frequently with “personal appeals” that sound like begging, ive thought that this was the equivalent of an advertisement. now this concept on this page is the marketing behind that advertising. im bothered by these “banner results” simply for that reason. if you are a charity, you dont compel people to give.
There is a cumulative effect to the banners and requests as well. Changing the banner on one single day is not a stand alone event. People (at least those that use wikipedia daily) are also responding to the ongoing message and may give today based on yesterday’s message. Maybe the desire to give is based on one message one day but the means (such as pay day or some other event) may force waiting until another day to actually donate. There is not a direct correlation between each message and each donation.
Wiki has become a proven winner and used by millions with reliable information and up to date
facts. I donated because I pay for things every day that don’t provide me with half of what
Wiki gives me.
The personal appeal from the founder is what got me!! It just showed me the person behind wiki… and the philosophy of it I guess
@yoni bar ilan. Why is it unfair^ If this resource is valuable to you, I dont think itd be unfair to donate. + the whole point is that not everybody is in the same situation. Not everyone CAN donate… so everyone gives what they can to keep it free.
Sure, there are people who have and dont give. But generally, the thought follows the above.
I totally support what Wikipedia is doing, and the insight into the banner compaign is interesting.
Chad (#13) Not sure I understand, in paragraph 1 you say prefer Wikipedia should have ad revenue (if contributors can get some), and in paragraph 2 you criticize Wikipedia for running personal appeals because they are almost like ads?
As for charities not compelling money, of course. The IRS *compels* people to give them money. Wikipedia, like other charities, is asking its users for money (and people who don’t use Wikipedia don’t see these requests). There is a significant difference.
I donated because of #18, and in an amount around equal to the average. I think that what makes Wikipedia truly great is many people making small contributions (even if WP also operates a bit by the Pareto Principle, I doubt that it would have had the impact it had or the popularity it does without its reputation as a source of knowledge from many).
Charities must compel people to give. Corporations don’t have to compel, because they are charging for their services. Wikipedia does not charge for its services, and thus has no way to collect revenue other than convincing someone to give it money. As a result, it must use some means for drumming up support. It’s no different, in any way, from the pledge drives that take place on public broadcasting stations (both TV and radio) in the States.
The truth is, people hate banner ads because they’re annoying and get in the way, and say that they hate them because they promote commercialism or something of that nature. These banner pledge notices also get in the way and are thus annoying, but they don’t compromise the non-commercial nature of Wikipedia and indeed are directly because of that nature.
However, having given money, it would be nice for there to be an option to turn off the banners. I’m not planning on giving a second time, especially since it’s not going to result in the ads disappearing anyway.
The reason for relying on (many small) Donations instead of Corporate support / Advertisement lies of course in the question of independence. If you don’t want to go the way of the mass media, which always have to consider the interests and whishes of their corporate customers (and owners) when publishing you need to be free of “market money”. Customer is King, if you don’t want to have a king, you need to have no customers.
Donation therefore is the “royal way” for Wikipedia to stay up, running and independent. The guys and girls at Wikimedia have the job to get as many free, unbound resources for the Wiki-Project as possible. To do this Job sincere of course includes the use of marketing strategies to optimizie the intake. Wikimedia doesn’t force anyone to give, it simply attempts to convince people that a donation for Wikipedia is a good use of their money. What Rand Montoya presents us here is simply the process of optimizing Wikipedias ability to convince. We look for the best arguments and use those instead of the less convincing. I really appreciate the open and transparent way here, to show the whole community also the backoffice of this effort to finance one of the most important Internet-Projects.
In practical terms, I support Jordans suggestion: It should be easily possible for everyone to turn off the Donation-Banners in one or two clicks. We shouldn’t nag people, which already made a decision (to give or not to give).