Fundraising/Chapters/WMDE

From Meta, a Wikimedia project coordination wiki

This page is for discussion and informational purposes about fundraising tests from Wikimedia Germany. Discussion should occur on the discussion page. If want to leave a note in German, the German portal site might be a better place. Till Mletzko (WMDE) (talk)

Newest Test request[edit]

Completed tests[edit]

Test 1: more than thousands words...[edit]

13.9.2012, 10:00-12:30 UTC

Wikipedia is held and guarded by the amazing support of our donors. In our appeals to Wikipedia readers we try to make this aware. Can we also communicate this successfully by using pictures instead of just relying on text? This was the question for our first test this year. Therefore we changed the layout of the landing page profoundly. Equally intended was to make the site less text heavy and more appealing. What do you think about the new design? Do you like it, do you have critical remarks? We are happy about every comment!

The results of the test show that we still need to improve the new layout. Both versions of the new landing page performed not as good as the old one. It can be that the reason for this lies in the necessity to expand the box with Jimmys appeal and in one case as well the donation form. Perhaps the additionally needed clicks have the effect of a barrier. But all in all the results are not very selective and the difference to the default landing page is only in one case statistically significant, according to ABBA (here und here). In addition, we noticed afterwards that the new layout didn't fit in small screens and appeared cutted. So all in all we see potential in improving the new layout to create a more compelling landing page.

Banner Landingpage Visitors Sum Amount Average Numbers
per View
Sum
per View
% Women
Jimmy Jacket default layout 11136 266 5365,74 20,17 0,02389 0,481837 22,95
Jimmy Jacket globe_form_expandable 11281 235 4582,00 19,50 0,02083 0,406170 22,81
Jimmy Jacket globe_big_form 11052 208 4142,12 19,91 0,01882 0,374785 20,00

--TobiasSchumann(WMDE) (talk) 09:55, 19 September 2012 (UTC)[reply]


Test 2: donation sums[edit]

20.9.2012, 08:00-10:00 UTC

In this test we focused on the donation form. We analysed the effect of different donation sums given in the form. What we can see clearly is that total amount and average sum are effected positively, although we didn't check statistical significance, yet. Equally interesting are the similar donation numbers which give us confidence that the changes in donation numbers don't effect the will to support.

For analysis we used amount_50 this time. By doing so, the effect of rare high donation sums is diminished. What we see is that the different donation amount and average sums in the test are not due to singular outliers.

Banner Landingpage Visitors Sum Amount Average Numbers
per View
Sum
per View
Sum_50 Sum_50
per View
Jimmy Jacket default 7740 190 3748,34 19,73 0,02455 0,484282 3148,34 0,40676
Jimmy Jacket Sum_A 7832 167 4727,77 28,31 0,02132 0,603648 3740 0,47753
Jimmy Jacket Sum_B 7719 192 7167,50 37,33 0,02487 0,928553 4022,5 0,52112

--TobiasSchumann(WMDE) (talk) 13:23, 21 September 2012 (UTC)[reply]


Test 3: closer to the reader[edit]

Wednesday, 10.10.2012, 8:00-10:00 UTC

These results tell a clear story! We tested WMF’s facts and drop down banners and they obiously have a strong power to motivate people to donate. Comparing the drop down to the default personal appeal banner we saw a 185 % increase in donation numbers and a 55% higher total sum. The approach of reaching more people by putting the message into the banner and shifting the point of decision to donate before the banner click seems to work very well. 20% of the people clicking on the facts banner really donated – a level of “productive clicks” never seen before by us. On the other hand the average donation of the new banners appears to be smaller. The reason could be the sentence directly asking for 5 €, an effect we wondered about last year as well. Finally what really needs to be figured out now is what happens to the circa 500 people deciding to donate in the drop down banner and finally don’t do it.


Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
control / appeal Jimmy default 846300 10038 296 7485,50 25,29 0,01186 0,0003498 0,02949 0,008845 0,745716 25,00
facts default 847900 3722 785 12050,99 15,35 0,00439 0,0009258 0,21091 0,014213 3,237773 15,77
dropdown default 843300 1398 846 11611,52 13,73 0,00166 0,0010032 0,60515 0,013769 8,305808 14,64

--TobiasSchumann(WMDE) (talk) 11:49, 11 October 2012 (UTC)[reply]

Test 4: green is the new blue[edit]

Tuesday, 16.10.2012, 8:00-10:00 UTC


From previous WMDE und WMF tests we know that colours of campagin elements play a major role in communication. This finding holds for the new facts banner as well. WMF found out that a blue version outperformed grey. Since by a somewhat strange regularity the colour green often worked best in our tests we wanted to see how it would be this time. And the 'rule' applies again: The green banner lead to a 27 % increase in donation numbers compared to control and 18 % to blue.

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
drop control grey default 869200 1095 675 10022,19 14,85 0,00126 0,0007766 0,61644 0,011530 9,152685 16,20
drop green default 838500 1465 856 12890,00 15,06 0,00175 0,0010209 0,58430 0,015373 8,798635 17,14
drop blue default 849800 1268 725 9928,00 13,69 0,00149 0,0008531 0,57177 0,011683 7,829653 16,95

--TobiasSchumann(WMDE) (talk) 11:08, 17 October 2012 (UTC)[reply]

Test 5: Jimmy and the facts[edit]

Tuesday, 23.10.2012, 9:30 - 11:30 UTC


The facts banner is very successful but unfortunately it is not personalized. It is impossible to realize when personal appeals on the landing page change and someone new tells his /her story. But for the dramaturgy of the campaign this is crucial. So we tried to merge the appeal banner with the facts. Sadly, our layouts did not work out. Facts_green and Photo_green have identical click rates but more people donate through the facts banner. Since landing pages are similar more people are convinced through the facts banner to donate even before they click. Interesting to see is that the Jimmy photo with green background seems to be less successfull than the one with white background, although it's not clear wether the photo or the design or (most probably) the combination of both explains the different outcome of the two foto banners.


Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
Average_100
control facts_green default 898800 3364 602 8540,90 14,19 0,00374 0,0006698 0,17895 0,009503 2,538912 14,10
facts/photo_green default 912300 3196 413 5954,50 14,42 0,00350 0,0004527 0,12922 0,006527 1,863110 14,41
facts/photo_green_white default 917600 2323 348 6569,09 18,88 0,00253 0,0003793 0,14981 0,007159 2,827848 15,88

--TobiasSchumann(WMDE) (talk) 11:15, 25 October 2012 (UTC)[reply]


Test 6: new appeals[edit]

Tuesday, 30.10.2012, 09:10-12:10 UTC

This time we were happy to test this years first appeals that are not from Jimmy. Three appeals showed different perspectives on Wikipedia. Gerd Seidel, fotographer and Wikipedian, explained his participation and why he thinks volunteering is so important. The donor Gunther Tutein described why he donated last year and what feelings he had. WMDE Project Managerin Elly Köpf introduced education projects of the chapter.

Unfortunately, we did not succeed in beating Jimmy. Nevertheless the results are interesting for us. The best performing appeal was the one of Gunther. Is it because readers can identify with his perspective and the feelings he describes strongly? That could be an explanation. Looking at donations per lp we can see that his appeal was best performing (leaving Jimmy aside). At a glance on the click-through-rates you can see the incredible attention that Elly's banner received. Twice as much people clicked on her banner than on Jimmy's, but her appeal convinces readers the least. All in all, the personal appeals of this test (including Jimmy) did nowhere reach as much donations as the facts banners in previous test.

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy appeal Jimmy 1121600 11045 255 5525,36 21,67 0,00985 0,0002274 0,02309 0,004926 0,500259 20,54
Gunther appeal Gunther 1119600 9733 137 3428,50 25,03 0,00869 0,0001224 0,01408 0,003062 0,352255 17,09
Elly appeal Elly 1105900 23871 104 2676,00 25,73 0,02159 0,0000940 0,00436 0,002420 0,112103 25,88
Gerd appeal Gerd 840100 8976 84 2286,00 27,21 0,01068 0,0001000 0,00936 0,002721 0,254679 21,33

--TobiasSchumann(WMDE) (talk) 09:59, 2 November 2012 (UTC)[reply]


Test 7: reformulated banner texts[edit]

Thursday, 1.11.2012, 10:00-12:00 UTC

One of the key elements of the success of the facts banner is the important phrase "If everyone donated 5€ ..." Unfortunately, the sentence also affects the average donation which in turn decreases significantly. In this test we tried out different banner texts to avoid that effect. Simply replacing "5€" by "a small contribution" didn't bring a solution - while the average donation increased only slightly, the donation per impression descreased by 13%. Based on findings from a donor focus group and donor communication in general, we modified the banner textes more profoundly. For example by adding the mission statement of Wikimedia and the fact that Wikipedia doesn't have any commercial interests. One of the text alternatives (new text 2) performed very well. Even with "5€" replaced by "a small contribution" the donation per impression rate is practically the same as in the unmodified facts banner. Furthermore, the average donation increases so that the new version of the facts banner outperforms the older one.

Here is the (somewhat crude) translation of the successful new banner text (since it is not translated in the banners linked in the chart below):

  • Wikipedia is non-profit and doesn't charge fees. It's the #6 website in the world. We don't pursue commercial interests. Our goal is the free access to knowledge for everybody.
  • Google and Yahoo have thousands of servers and staff. We have 641 servers and less than 200 employees. To protect our independence, we'll never run ads.
  • 450 Millionen people use Wikipedia every month. For a lot of them it has become a naturally part of every day life. If everyone made a small contribution, our fundraiser would be done in an hour.


Banner Landingpage impressions clicks visitors sum amount average click-
through-
rate
numbers
per imp.
numbers
per click
numbers
per View
sum
per imp.
sum
per view
average
_100
control default 364320 4141 821 469 6354,50 13,55 0,011366 0,0012873 0,1132577 0,57125 0,017442 7,739951 13,23
"small contribution" default 484990 5851 906 513 7996,27 15,59 0,012064 0,0010578 0,0876773 0,56623 0,016487 8,825905 15,59
new text 1 default 489940 5649 647 370 6797,22 18,37 0,011530 0,0007552 0,0654983 0,57187 0,013874 10,505750 17,56
new text 2 default 480040 7263 1022 601 10749,90 17,89 0,015130 0,0012520 0,0827482 0,58806 0,022394 10,518493 17,30

--TobiasSchumann(WMDE) (talk) 14:34, 6 November 2012 (UTC)[reply]


Test 8: layout donation form and landing page[edit]

Tuesday, 6.11.2012, 10:30-13:30 UTC

The new drop down banner modifies the whole donation process. With that banner the decision to donate is directly made on the expanded banner and before reaching the landing page. We were wondering why still 40% of the people that reach the landing page quit the process and don't donate. Our assumption was that the old layouts of landing page and donation form distract people because first the Jimmy appeal is displayed again and second people have to think about the donation amount again because now they see recurrent donations are possible as well. To reduce the drop out rate, we modified the donation form inside the banner by adding amount, frequency and payment methods. The donation process continues on the landing page where donors need to fill in their personal data. We tested different layouts of the landing page to see if they can reduce the drop outs.

By the modification of the donation form we could reduce the drop outs on the landing page significantly but at the same time less people reached the landing page, that is less people filled out the donation form inside the banner. Obviously, filling out the form in the old banner version cannot be interpreted as a strong decision to donate. Rather some people just want to see how the donation process goes on and to decide later. In the new banner version these people don't fill out the form any more. All in all the modification of the donation form seems to have a slightliy positive effect - more people donate per click - but the difference is not statistically significant.

Concerning the new and very different layouts of the landing page we surprisingly did not see any impact on donation numbers. But we can learn from it that distraction by layout does not seem to be the reason for a drop out.

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
amount
pro
lpvisitor
average
_100
form_A ctrl default 825700 8994 1007 481 8233,17 17,12 0,010893 0,0005825 0,0534801 0,47766 0,009971 8,175938 16,39
form_B default 719200 7703 608 432 7751,63 17,94 0,010711 0,0006007 0,0560820 0,71053 0,010778 12,749391 17,06
form_B clear 587900 6521 531 368 5973,77 16,23 0,011092 0,0006260 0,0564331 0,69303 0,010161 11,250038 16,32
form_B photo 704500 7679 593 425 6790,72 15,98 0,010900 0,0006033 0,0553457 0,71669 0,009639 11,451467 15,78
form_B 5 facts 712300 7667 600 422 7301,54 17,30 0,010764 0,0005924 0,0550411 0,70333 0,010251 12,169233 16,35

--TobiasSchumann(WMDE) (talk) 12:56, 9 November 2012 (UTC)[reply]


Test 9: call for action[edit]

Thursday, 8.11.2012, 10:30-13:45 UTC

In this test we analyzed the effect of the call for action inside the donation form. We tested four different phrases against a form without a call. Surprisingly there was no real difference between the groups. Only mentioning our most common and our average donation sum seems to increase the number of donations but in return to decrease the average donation. The differences in the groups were not statistically significant so we decided not to change our donation form.

Here is the translation of the phrases:

  • fact_ctrl: If everyone would give a small contribution our fundraiser would be over in one hour.
  • fact_please help: Please donate to keep Wikipedia free, independent and ad free.
  • fact_5/23: 5€ is the most common donation. 23€ is the average.
  • fact_give: How much is Wikipedia worth to you? Today you can say thank you.


banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
amount
pro
lpvisitor
average
_100
fact_ctrl 5_facts 725200 7019 537 388 5960,60 15,36 0,009679 0,0005350 0,0552785 0,72253 0,008219 11,099814 15,23
fact_noline 5_facts 630300 5954 437 327 5925,74 18,12 0,009446 0,0005188 0,0549211 0,74828 0,009401 13,560046 16,44
fact_pleasehelp 5_facts 746700 6901 532 385 6210,29 16,13 0,009242 0,0005156 0,0557890 0,72368 0,008317 11,673477 15,74
fact_5/23 5_facts 757000 7171 575 417 5845,63 14,02 0,009473 0,0005509 0,0581509 0,72522 0,007722 10,166313 14,02
fact_give 5_facts 858600 8012 609 436 7200,66 16,52 0,009331 0,0005078 0,0544184 0,71593 0,008387 11,823744 16,52

--TobiasSchumann(WMDE) (talk) 17:43, 16 November 2012 (UTC)[reply]

Test 10: facts-banner texts II[edit]

Tuesday, 13.11.2012, 10:30-14:15 UTC

With the begin of the campaign we saw a need to retest two aspects which had narrow outcomes in our previous tests. We reviewed first the decision for the drop down banner version and second the substitution of "5€" by "a small contribution" in the banner text. Furthermore we tried an alternative no_google version.

We are happy that our old test results were confirmed by this new test. More people click on the drop down banner version and this results in more donations as well. Writing "5€" instead of "a small contribution" generates more donations but decreases the average donation so much that our current version performes better. Our new banner text version unfortunately wasn't successful.

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
amount
pro
lpvisitor
average
_100
no_google 5_facts 941400 8442 550 362 6887,50 19,03 0,008967 0,0003845 0,0428808 0,65818 0,007316 12,522727 17,65
no_google2 5_facts 1068000 8597 472 294 4813,83 16,37 0,008050 0,0002753 0,0341980 0,62288 0,004507 10,198792 16,37
no_google_5€ 5_facts 1064200 8772 669 450 6025,32 13,39 0,008243 0,0004229 0,0512996 0,67265 0,005662 9,006457 13,06
no_drop_no_google 5_facts 948800 xxxx 4041 311 5634,50 18,12 0,00426 0,0003278 xxxx 0,07696 0,005939 1,394333 17,80

--TobiasSchumann(WMDE) (talk) 17:43, 16 November 2012 (UTC)[reply]


Test 11: call for action[edit]

Wednesday, 14.11.2012; 17:15-20:45 UTC

In our last test we saw how important the dropdown-tab is. Much more people click on the dropdown version of the facts banner than the non-drop down. In this test we wanted to use the attention which the tab obviously receives. We put different calls for donation inside the tab and tried a different tab layout. Furthermore we tested a modification of donation numbers indicated in our donation form.

The last one didn't have a positive effect. But both versions of a long call to action (tab_1 and tab_2: 'Please help to keep Wikipedia free and independent') did perform better than our control. So we changed our current campaign banner according tab_2.

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
amount
pro
lpvisitor
average
_100
ctrl 5_facts 838700 5974 394 281 5198,46 18,50 0,007123 0,0003350 0,0470372 0,71320 0,006198 13,194061 18,50
tab_1 5_facts 836000 xxxx 432 301 5184,82 17,23 xxxx 0,0003600 xxxx 0,69676 0,006202 12,001898 xxxx
tab_2 5_facts 874300 6915 437 306 5186,14 16,95 0,007909 0,0003500 0,0442516 0,70023 0,005932 11,867597 16,78
tab_3 5_facts 830300 5625 429 277 5451,51 19,68 0,006775 0,0003336 0,0492444 0,64569 0,006566 12,707483 18,89
10/20/30 5_facts 845000 6127 355 260 4736,79 18,22 0,007251 0,0003077 0,0424351 0,73239 0,005606 13,343070 18,22

--TobiasSchumann(WMDE) (talk) 14:07, 19 November 2012 (UTC)[reply]


Test 12: layout banner: bold text[edit]

Thursday, 15.11.2012; 14:00-17:00 UTC

A simple but effective test. We tested bold versus regular text. It looks like bold text increases attention: More people click on the banner, more people donate.

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
amount
pro
lpvisitor
average
_100
regular 5_facts 1785800 12556 661 419 7722,00 18,43 0,007031 0,0002346 0,0333705 0,63389 0,004324 11,682300 18,07
bold 5_facts 1805900 14232 716 476 8515,49 17,89 0,007881 0,0002636 0,0334458 0,66480 0,004715 11,893142 17,57

--TobiasSchumann(WMDE) (talk) 14:07, 19 November 2012 (UTC)[reply]


Test 13: alternative wording facts banner III (Google/Yahoo/FB)[edit]

Friday, 16.11.2012; 11:30 am -02:30 pm UTC

We started the campaign with a facts banner that compares Wikipedia with other top sites but doesn't explicitly mention google and yahoo. In this test we tried google/yahoo and google/facebook plus another different wording. Explicitly refering to google etc. seems to increase donation numbers on a small level (+9%), but the difference is not statistically significant (chi-squared test). This let us to the decision not to change the banner text.

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
amount
pro
lpvisitor
average
_100
ctrl 5_facts 702500 4753 271 194 4169,50 21,49 0,006766 0,0002762 0,0408163 0,71587 0,005935 15,385609 21,49
go_yahoo 5_facts 705800 5203 293 213 3944,50 18,52 0,007372 0,0003018 0,0409379 0,72696 0,005589 13,462457 17,81
go_fb 5_facts 802100 6030 323 237 4276,00 18,04 0,007518 0,0002955 0,0393035 0,73375 0,005331 13,238390 18,04
no_go2 5_facts 808900 5869 297 184 4059,51 22,06 0,007256 0,0002275 0,0313512 0,61953 0,005019 13,668384 20,43

--TobiasSchumann(WMDE) (talk) 14:44, 25 November 2012 (UTC)[reply]


Test 14 & 15: banner colour & length Jimmy appeal[edit]

test 14, Friday, 16.11.2012, 5:30-7:45 pm UTC and Test 15 am Monday, 19.11.2012, 01:40-03:30 pm UTC

Beeing convinced to donate after reading the text on the facts banner - is there still a need for the long and much space occupying Jimmy appeal? We tested WMF's condensed version of the appeal inside the drop down and saw that this had a negative effect on donation numbers.

Instead the main focus of test 14 and 15 was the colour of the banner. In a first test 4 green won against blue and grey. In the light of the success of blue in WMF's tests we wanted to retest that. Now blue won against green in test 14. This contradictory result puzzled us. A possible reason could be that banners are not allocated to one single user but instead every reader sees all banners by clicking through Wikipedia. (Bucketing function in Central Notice was not available yet.) By that it is possible that banner impacts influence each other and the whole test is biased. Since 2/3 of the banners were green in test 14 the more infrequent blue alternation created perhaps more attention and by that more clicks. To avoid the chance of such an effect we tried another testing configuration. In four successive 30 minutes slots we only displayed one banner, alternating green and blue. Afterwards we summed up the results of the time periods. Now it looks like the two colours do not have such a strong difference as expected, but green did a bit better. Still on methodological thin ground we decided to take green.

test 14

banner landingpage impressions clicks
banner
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
amount
pro
imp.
green/long 5_facts 684100 5125 237 4179,00 17,63 0,007492 0,0003464 0,0462439 0,006109
blue/long 5_facts 689900 5633 255 5062,33 19,85 0,008165 0,0003696 0,0452690 0,007338
green/short 5_facts 695000 5249 203 3652,00 17,99 0,007553 0,0002921 0,0386740 0,005255

test 15

banner landingpage impressions clicks
banner
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
amount
pro
imp.
green 5_Fakten 1375900 8503 216 4230,01 19,58 0,006180 0,0001570 0,0254028 0,003074
blue 5_Fakten 1395200 8431 195 3556,24 18,24 0,006043 0,0001398 0,0231289 0,002549

--TobiasSchumann(WMDE) (talk) 14:44, 25 November 2012 (UTC)[reply]

Test 16: appeal inside banner[edit]

Monday, 19.11.2012; 6:00-8:00 pm UTC

In test 14 we saw the importance of the unshortened Jimmy appeal instead of a condensed version. Now we wanted to know if it is possible to replace Jimmy's appeal by another. We ran Jimmy against our donor Gunther Tutein. The outcome is very promising. Gunther's appeal nearly performed as good as Jimmy's although in a personal appeal banner test their difference in donation numbers was very high. More tests with different appeals are needed now.

On the occation of the anniversary of 1.500.000 articles in the german language version of Wikipedia we had a thank-you banner on for a short time. It featured a call for donations but it didn't work good.

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
amount
pro
lpvisitor
average
_100
facts_Jimmy 5_facts 945900 5686 265 196 4026,00 20,54 0,006011 0,0002072 0,0344706 0,73962 0,004256 15,192453 18,91
facts_Gunther 5_facts 909600 5512 241 185 3947,00 21,34 0,006060 0,0002034 0,0335631 0,76763 0,004339 16,377593 19,71
article record 5_facts 433100 1074 56 32 620,00 19,37 0,002480 0,0000739 0,0297952 - - - 19,37

--TobiasSchumann(WMDE) (talk) 14:44, 25 November 2012 (UTC)[reply]


Test 17: Jimmy personal appeal vs. facts-banner[edit]

Tuesday 20.11.2012; 09:30 - 12:00 am UTC

After the first week of the campaign we saw a decrease in donations stronger than expected. We wanted to know if it is now possible to convince more people with the Jimmy appeal banner. For the first time we successfully used the new bucketing function of central notice. Indeed donation numbers increased, so we decided to continue the campaign with the Jimmy appeal (difference in numbers/imp. statistically significant, chi-squared test: p=0,00009). In the first test of Jimmy against the facts (test 3) the latter increased donations by 185%. Now Jimmy outperformed the facts by increasing donations by more than 30%. Thats remarkable. It looks like there are really two target groups and a big share of the people being responsive to our facts banner donated during the first week.

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
amount
pro
lpvisitor
average
_100
facts_green 5_facts 1374800 8242 398 271 5593,00 20,64 0,005995 0,0001971 0,0328804 0,68090 0,004068 14,052764 18,63
Jimmy appeal Jimmy 1364300 x 15464 367 8492,00 23,14 0,01133 0,0002690 x 0,02373 0,006224 0,549146 22,32

--TobiasSchumann(WMDE) (talk) 17:09, 26 November 2012 (UTC)[reply]


Test 18: layout Jimmy banner[edit]

Wednesday, 22.11.; 1:45-3:15 pm UTC

Today we tested a new layout of the Jimmy appeal banner trying to generate more attention with a coloured background and white text. This didn't work well. The new layout has a lower click rate and hence less donations. (difference statistically significant; chi-squared test: numbers/imp.: p=0,028, clics/imp.: p=2,2e-16)

banner landingpage impressions visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
LP
amount
pro
imp.
Jimmy ctrl Jimmy appeal 978200 9323 155 3292,50 21,24 0,00953 0,0001585 0,01663 0,003366
Jimmy green Jimmy appeal 972800 7121 118 2566,62 21,75 0,00732 0,0001213 0,01657 0,002638

--TobiasSchumann(WMDE) (talk) 17:09, 26 November 2012 (UTC)[reply]


Test 19 & 20: integrating the facts on LP[edit]

Test 19: Wednesday, 21.11., 3:15 pm - Thursday, 22.11., 9:00 am UTC; Test 20: Thursday, 22.11., 3:00-5:00 pm UTC

We tried to integrate the facts banner on our landing page this time. This indeed increased the conversion rate. An alternative wording of the facts didn't work well in test 20.

test 19

banner landingpage impressions visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
per
LP
amount
per
imp.
Jimmy default 5619200 52544 1030 22368,94 21,72 0,00935 0,0001833 0,01960 0,003981
Jimmy facts 5660000 52409 1120 26269,10 23,45 0,00926 0,0001979 0,02137 0,004641

numbers per LP statistically significant (chi-squared-t., p=0,04323)

test 20

banner landingpage impressions visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
per
LP
amount
per
imp.
Jimmy facts_ctrl 1232400 10984 184 4166,01 22,64 0,00891 0,0001493 0,01675 0,003380
Jimmy facts_alt 1236800 10938 161 3473,36 21,57 0,00884 0,0001302 0,01472 0,002808

numbers per LP not statistically significant (chi-squared-t., p=0,2267)

--TobiasSchumann(WMDE) (talk) 17:09, 26 November 2012 (UTC)[reply]


test 21: 2nd round of new appeals[edit]

Friday, 23.11., 11:10 am- 01:30 pm UTC

Today we tested a second round of non-Jimmy appeals. We had an appeal of Wikipedia author Harald Krichel describing his captivation since he first discovered the online encyclopedia. Wikpedia donor Solveig Wehking explains the importance of free accessible knowledge. Wikimedia fundraiser Till Mletzko compares Wikipedia with a classic library to illustrate the uniqueness of the project. We did not manage to create an appeal which beats Jimmy's but we've learned a lot for future improvements.

banner landingpage impressions visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
LP
amount
pro
imp.
average
_100
Jimmy appeal Jimmy 515900 4274 99 2005,00 20,25 0,00828 0,0001919 0,02316 0,003886 20,25
Till appeal Till 583000 5368 74 1838,00 24,84 0,00921 0,0001269 0,01379 0,003153 24,84
Solveig appeal Solveig 583300 5001 57 2235,00 39,21 0,00857 0,0000977 0,01140 0,003832 23,42
Harald appeal Harald 493100 4562 58 1057,00 18,22 0,00925 0,0001176 0,01271 0,002144 18,22

--TobiasSchumann(WMDE) (talk) 10:57, 28 November 2012 (UTC)[reply]


Test 22, 23 & 24: yellow facts banner[edit]

In this test round wir evaluated the impact of a yellow version of the facts banner. The direct comparison of the new colour against the green facts (test 22) as well as against the Jimmy banner (test 23) revealed a clear increase in donations. In the light of this result - were we wrong to run the Jimmy Banner the last days? If we look closer to the results the answer is no. While the yellow banner increased donations/imps by 123,6% compared to the green facts the augmentation compared to the Jimmy banner only amount to 87,3%. Hence the Jimmy banner still performed better than the green facts banner.

Reflecting on the rise of donations by the Jimmy appeal in the beginning of the week we were wondering if there are two different target groups that are each more or less responsive either to the facts or to the personal appeal. If it would be like this, would it be possible to appeal to both groups at the same time? We tried to do this (test 24) by showing one group of readers both Jimmy and facts alternating each other (group B). The comparison group only saw the yellow facts banner (group A). Unfortunately this did not work out. Group B did perform worse than group A.


friday, 23.11.; test 22: 1:30-4:00 pm UTC; test 23: 4:00-5:30 pm UTC; test 24: 5:30-8:15 pm UTC


test 22

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
facts_yel 5_facts 768400 6485 328 263 5257,44 19,99 0,008440 0,0003423 0,0405551 0,80183 0,006842
facts_green 5_facts 777500 4160 173 119 3356,00 28,20 0,005350 0,0001531 0,0286058 0,68786 0,004316

difference in numbers per imp. statistically significant (chi-squared-t., p=7.221e-14)

test 23

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
Jimmy appeal Jimmy 714600 5677 329 223 3949,00 17,71 0,00794 0,0003121 0,0392813 0,67781 0,005526
facts_gelb 5_facts 726300 x 6260 121 2321,00 19,18 0,00862 0,0001666 x 0,01933 0,003196

difference in numbers per imp. statistically significant (chi-squared-t., p=1.596e-08)

test 24

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
pro
click
numbers
pro
LP
amount
pro
imp.
group A facts_yellow 5_facts 1193800 9279 602 414 9473,63 22,88 0,007773 0,0003468 0,0446169 0,68771 0,007936
group B aggregated results 1200900 4819 6127 341 7394,54 21,68 0,008875 0,0002840 x x 0,006157
facts_yellow 5_facts 603100 4819 288 196 3768,54 19,23 0,007990 0,0003250 0,0406723 0,68056 0,006249
Jimmy appeal Jimmy 597800 x 5839 145 3626,00 25,01 0,00977 0,0002426 x 0,02483 0,006066

difference in numbers per imp. bucket A-B statistically significant (chi-squared-t., p=0,006169)

--TobiasSchumann(WMDE) (talk) 10:57, 28 November 2012 (UTC)[reply]


Test 25, 26 & 27: new design facts banner[edit]

Today we tested WMF's new best performing facts banner with and without sticky function. As we can see in the results the pure modification of the layout does not increase donations (test 25). Interesting to see is the stable donations/imp. while click rate and conversion after clicking the banner (numbers per click) vary a great deal. Test 26 shows the impact of the sticky function which leads to a huge increase in donations per impressions. We were wondering if formating the text in blocks like we had it before would be easier to read and thereby convince more people to donate. But as test 27 reveals continuous text performes better in this banner layout.


test 25, 26.11., 1:15 - 3:30 pm UTC

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
per
click
numbers
per
LP
amount
per
imp.
average
_100
ctrl facts_yelw 5_facts 1480200 9011 345 214 4174,77 19,51 0,006088 0,0001446 0,0237488 0,62029 0,002820 19,51
var facts_yelw 5_facts 1501700 2821 334 219 4367,35 19,94 0,001879 0,0001458 0,0776320 0,65569 0,002908 19,94


test 26 , Monday 26.11., 3:30 - 6:00 pm UTC

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
per
click
numbers
per
LP
amount
per
imp.
average
_100
facts_yelw_ctrl 5_facts 1655300 10616 454 311 7213,83 23,20 0,006413 0,0001879 0,0292954 0,68502 0,004358 22,23
facts_yelw_sticky 5_facts 1434400 7718 887 624 14304,51 22,92 0,005381 0,0004350 0,0808500 0,70349 0,009972 21,32

difference donation/imp. statistically significant (chi-squared-test, p-value < 2.2e-16 )

difference click/Imp. statistically significant (chi-squared-test, p-value < 2.2e-16 )

difference donation/click statistically significant (chi-squared-test, p-value < 2.2e-16 )


test 27 , Tuesday 27.11., 12:00 am - 01:45 pm UTC

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
per
click
numbers
per
LP
amount
per
imp.
average
_100
running text 5_facts 830900 13140 526 383 7311,07 19,09 0,015814 0,0004609 0,0291476 0,72814 0,008799
text blocks 5_facts 816100 13018 428 295 5923,05 20,08 0,015951 0,0003615 0,0226609 0,68925 0,007258

difference donation/imp. statistically significant (chi-squared-test, p-value = 0.00165 )

--TobiasSchumann(WMDE) (talk) 12:21, 30 November 2012 (UTC)[reply]


Test 28: appeals inside facts banner: Jimmy, Gunther, Till[edit]

Tuesday, 27.11., 1:45-4:00 pm UTC

Another time we tested different appeals inside the drop down facts banner. Against Jimmy we ran our current best performing texts from Wikipedia donor Gunther and from Till. Jimmy was ahead another time but the difference was not so big (donation/click ~ -15%). For the purpose of having a more diversified campaign we decided to replace Jimmy's appeal by the one of Till.

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
per
click
numbers
per
LP
amount
per
imp.
Jimmy 5_facts 1107000 18898 663 484 11005,50 22,74 0,017071 0,0004372 0,0256112 0,73002 0,009942
Till 5_facts 1112400 18970 593 411 7890,00 19,20 0,017053 0,0003695 0,0216658 0,69309 0,007093
Gunther 5_facts 1112600 18940 619 407 8411,02 20,67 0,017023 0,0003658 0,0214889 0,65751 0,007560

--TobiasSchumann(WMDE) (talk) 12:21, 30 November 2012 (UTC)[reply]


Test 29 & 30: Who is talking?[edit]

These tests focused on the current non-Jimmy appeal in our banner. We analyzed if explicitly mentioning costs would have an effect on donation conversion. Leaving the relevant sentence out seems to have a small negative impact (test 29). Since the appeal is kind of non-personal (it has no biographic component) we could alter the person who speaks. Interestingly in this case it did practically not make a difference who is talking (test 30). However according to the result it could be that the percentage of female donors rises when the speaker is also a woman.

test 29, Wednesday, 28.11., 1:00-3:00 pm UTC

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
per
click
numbers
per
LP
ctrl 5_facts 905400 13511 475 331 6430.50 19.13 0.014923 0.0003656 0.02450 0.69684
without_costs 5_facts 912100 13323 440 315 7624.06 21.35 0.014607 0.0003454 0.02364 0.71591

difference donation/click not statistically significant (chi-squared-test, p-value=0.6477)


test 30, Wednesday, 28.11., 2:00-11:00 pm UTC

banner landingpage impressions clicks
banner
visitors
lp
donations amount average click-
Through-
Rate
numbers
per
imp.
numbers
per
click
numbers
per
LP
% women
Till 5_facts 1736700 31829 959 694 16802.61 24.21 0.018327 0.0003996 0.02180 0.72367 21.38
Alice 5_facts 1994900 35740 1105 774 17175.25 22.19 0.017916 0.0003880 0.02166 0.70045 24.29
Pavel 5_facts 1980100 36148 1058 733 14918.02 20.35 0.018256 0.0003702 0.02028 0.69282 22.30
Elly 5_facts 1757800 31780 986 699 14908.77 21.33 0.018079 0.0003977 0.02199 0.70892 27.35

--TobiasSchumann(WMDE) (talk) 12:21, 30 November 2012 (UTC)[reply]


Test 31: wording facts banner: dear readers[edit]

Thursday, 29.11., 10:00 am - 03:00 pm UTC

In this test we can see the effect of clear adressing readers. We tried WMF's text modification of the facts banner starting the text with "dear Wikipedia readers" and finishing it with a "thank you". With the new text the donation rate (numbers/imp) improved by 18%.

Banner
Tracking
Landing-Page Impressions Banner
Clicks
Donations Sum Average Click
Drop
Imp
Donation
Imp
Donation
Click
Drop
Visitors
LP
Donation
LP
Visitor
Amount
Imp
ctrl 5_facts 2057500 28137 674 14027.00 20.81 0.013675 0.0003276 0.02395 995 0.67739 0.0068175
dear readers 5_facts 2046900 28728 791 16356.02 20.68 0.014035 0.0003864 0.02753 1134 0.69753 0.0079906

difference in numbers/imp. statistically significant (p-value = 0.001456)

--TobiasSchumann(WMDE) (talk) 17:20, 7 December 2012 (UTC)[reply]


Test 32: Scroll-Down-Icon[edit]

Thursday, 29.11.2012, 3:00-7:30 pm UTC

Sometimes it's remarkable to see how small changes have big effects. In this test we re-integrated the small symbol in the tab of the banner which indicates the drop down of the banner after clicking instead of opening another page. The symbol has a strong effect: the click rate increases by 35%. Not all of them do actually donate, but still numbers/imp increased by 13%.

Banner
Tracking
Landing-Page Impressions Banner
Clicks
Donations Sum Average Click
Drop
Imp
Donation
Imp
Donation
Click
Drop
Amount
Imp
ctrl 5_facts 1840000 30927 752 16412.84 21.83 0.016808 0.0004087 0.02432 0.0089200
icon 5_facts 1813500 41087 842 18885.76 22.43 0.022656 0.0004643 0.02049 0.0104140

difference in numbers/imp. statistically significant (p-value = 0.01095)

difference in clicks/imp. statistically significant (p-value < 2.2e-16)

--TobiasSchumann(WMDE) (talk) 17:20, 7 December 2012 (UTC)[reply]


Test 33: caution! you're redirected[edit]

Friday, 30.11., 2:30-6:00 pm UTC

During the donation process there is a 30% drop out rate after clicking the donation button inside the banner and being forwarded to the landing page. This is a result of fake donations and transactions that fail, but still it's a very high rate. We tried to decrease that rate in test 8 by various changes to the landing page layout which didn't work. Now we thought donors are perhaps simply cautious by the redirect to an unknown landing page. So we integrated an animated notice which appears after clicking the donation button and explains the transmission to the ssl encrypted landing page for security reasons. Unfortunately this didn't change the drop out rate at all. At least we know sure that the transimssion to the landing page is not the problem.

Banner Landing-Page Impressions Banner
Clicks
Donations Sum Average Click
Drop
Imp
Donation
Imp
Donation
Click
Drop
Amount
Imp
ctrl 5_facts 1317700 27244 644 15553.04 24.15 0.020675 0.0004887 0.02364 0.0118032
ssl notice 5_facts 1326600 26887 619 13942.21 22.52 0.020268 0.0004666 0.02302 0.0105097

--TobiasSchumann(WMDE) (talk) 17:20, 7 December 2012 (UTC)[reply]


Test 34, 35 & 37: wording + speaker of appeal[edit]

Over long time periods in this tests we tried some small changes in the appeal text. We changed the wording of the request for donate (test 34) and added focused on the usefulness of Wikipedia for readers (test 35). Both tests didn't bring much change to the conversion. Furthermore we altered the speaker of the (unpersonal) appeal. It didn't make a difference if Till or Pavel appeared as author. Even changing the role of Pavel (staffer vs. CEO) practically had no effect.


test 34,Friday, 30.11., 07:00 pm UTC - Sunday, 2.12., 01:30 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Donations Sum Average Click
Drop
Imp
Donation
Imp
Donation
Click
Drop
Amount
Imp
ctrl 5_facts 3266600 87677 1904 42850.45 22.51 0.026840 0.0005829 0.02172 0.0131178
var_2 5_facts 3747700 99776 2187 48507.85 22.18 0.026623 0.0005836 0.02192 0.0129434
var_3 5_facts 3766500 99841 2194 50402.21 22.97 0.026508 0.0005825 0.02197 0.0133817
var_1 5_facts 3287500 87253 1928 44210.40 22.93 0.026541 0.0005865 0.02210 0.0134480

test 35, Sunday, 2.12., 01:30 pm UTC - Monday, 3.12., 01:45 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Donations Sum Average Click
Drop
Imp
Donation
Imp
Donation
Click
Drop
Amount
Imp
ctrl 5_facts 5849500 131660 2588 60299.82 23.30 0.022508 0.0004424 0.01966 0.0103085
var_1 5_facts 5911200 132308 2638 59400.66 22.52 0.022383 0.0004463 0.01994 0.0100488

test 36, Monday, 3.12., 06:15 pm UTC - Monday, 4.12., 10:30 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Donations Sum Average Click
Drop
Imp
Donation
Imp
Donation
Click
Drop
Amount
Imp
% Women
Till 5_facts 1766100 42743 840 19087.72 22.72 0.024202 0.0004756 0.01965 0.0108078 25.37
Pavel_CEO 5_facts 1738100 42567 815 17461.23 21.42 0.024491 0.0004689 0.01915 0.0100462 25.24
Pavel_staff 5_facts 1777200 43077 797 19646.53 24.65 0.024239 0.0004485 0.01850 0.0110548 22.69

all test are not statistically significant

--TobiasSchumann(WMDE) (talk) 17:01, 14 December 2012 (UTC)[reply]


Test 36 & 43: testing sticky function[edit]

The stickyness of our current banner wasn't yet well implemented: the banner "jumps" in the top position when scrolling down instead of sticking to the top all the time. In this tests we checked a new stable sticky version. We had to test this twice - a second time for a very long period to be sure - because we were supprised by the results.

It looks like readers react in different ways to the different banner behaviours. While click rate increases by ~6-8% in the jumping version, a higher percentage of people clicking the banner actually donates in the stable version (+ ~10%). It seems like the old version generates more attention by kind of popping up. However these are no productive clicks. Despite statistically significant differences in click rate and donation/click all in all both banner versions perform almost identically in donation/impression. Because the new version has greater usability we decided to continue with it.

test 35, Monday, 3.12., 1:45-6:15 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop
Imp
Visitor
per
Banner-Click
Donation
Imp
Donation
Click
Drop
Donation
LP
Visitor
ctrl 5_facts 1912200 37550 903 628 14793.68 23.56 0.019637 0.02405 0.0003284 0.01672 0.69546
sticky_new 5_facts 1940300 35669 923 601 13498.50 22.46 0.018383 0.02588 0.0003097 0.01685 0.65114

test 43, Sunday, 9.12., 1:30 pm - Tuesday, 11.12., 3:45 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop
Imp
Visitor
per
Banner-Click
Donation
Imp
Donation
Click
Drop
Donation
LP
Visitor
ctrl 5_facts 12612500 211079 5718 3977 87053.16 21.89 0.016736 0.02709 0.0003153 0.01884 0.69552
sticky_new 5_facts 12854800 199427 5907 4080 94801.39 23.24 0.015514 0.02962 0.0003174 0.02046 0.69071

click-rate: -7,30% ; p-value < 2.2e-16

numbers/imp: +0,66%; p-value = 0.7691


--TobiasSchumann(WMDE) (talk) 17:01, 14 December 2012 (UTC)[reply]


Test 38 & 42: wording text facts banner[edit]

Since a lot of text variations of the appeal text inside the facts drop down banner didn't make a real difference, in this two tests we tried small changes of the banner text itself. Not mentioning the lack of fees seems to reduce click rate and the will to donate on a small scale (test 38). Explicitly refering to costs does improve conversion (test 42): Although we left out a sentence pointing out the importance of Wikipedia in everyday live (for reasons of banner size) more people donated in the modified version (numbers/imp: + ~7%).

test 38, Tuesday, 4.12., 10:30 am - 02:15 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop
Imp
LP-
Visitor
Imp
Visitor
per
Banner-Click
Donation
Imp
Donation
Click
Drop
Donation
LP
Visitor
ctrl 5_facts 1510200 26277 750 515 11470.05 22.27 0.017400 0.000497 0.02854 0.0003410 0.01960 0.68667
fees 5_facts 1542200 25423 710 460 9694.95 21.08 0.016485 0.000460 0.02793 0.0002983 0.01809 0.64789

click rate: statistically significant (p-value = 5.918e-10)

test 42, Friday, 7.12., 12:20 am - Sunday, 9.12., 1:30 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop
Imp
Donation
Imp
Donation
Click
Drop
Donation
LP
Visitor
ctrl 5_facts 9069600 166528 4906 3708 83480.55 22.51 0.018361 0.0004088 0.02227 0.75581
costs 5_facts 9035800 166669 5253 3940 91454.85 23.21 0.018445 0.0004360 0.02364 0.75005

numbers/imp: statistically significant (p-value = 0.004679)

--TobiasSchumann(WMDE) (talk) 17:20, 7 December 2012 (UTC)[reply]


Test 39: blue link[edit]

Tuesday, 4.12., 2:15 - 6:15 pm UTC

To highlight the well performing sentence "If everyone made a small contribution..." we tested a layout modification of WMF formatting the phrase blue and as a link. This does improve conversion rate (numbers/imp) by 21% while click rate seems to decrease a bit.

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop
Imp
Donation
Imp
Donation
Click
Drop
Donation
LP
Visitor
ctrl 5_facts 1658800 31490 712 484 10360.99 21.41 0.018984 0.0002918 0.01537 0.67978
link 5_facts 1653800 30078 849 584 13420.51 22.98 0.018187 0.0003531 0.01942 0.68787
  • click-rate: p-value = 8.042e-08
  • numbers/imp.: p-value = 0.001872

--TobiasSchumann(WMDE) (talk) 17:01, 14 December 2012 (UTC)[reply]


Test 40 & 41: appeal Harald Krichel: text + photo[edit]

In this test we evaluated the conversion of different text variations of the appeal of Wikipedia editor Harald Krichel. We are very happy that the results enable us to continue the campaign showing an editor appeal for a longer period of time inside the facts banner. All in all the appeals of Pavel (control) and Harald performed on the same level. One text version of Haralds appeal performed slightly better, but the results are not statistically significant. (Although the test lasted nearly 24 hours the samples are too small for such minor differences.) In a further test we wanted to see which impact the personalization by a picture of the speaker has. Inside the facts banner in this test leaving the portrait of Harald away did not have an effect on donation conversion.


test 40, Tuesday, 4.12., 6:15 pm - Wednesday, 5.12., 4:00 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
LP-
Visitor/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
Pavel_ctrl 5_Fakten 2504300 44977 1303 945 19281,92 20,40 0,017960 0,000520 0,02897 0,0003774 0,0210107 0,72525 0,007700
Harald_1 (lang) 5_Fakten 2770600 49827 1489 1066 23455.49 22.00 0.017984 0.000537 0.02988 0.0003848 0.02139 0.71592 0.0084659
Harald_2 (kurz) 5_Fakten 2763300 49477 1448 1004 20418,48 20,34 0,017905 0,000524 0,02927 0,0003633 0,0202923 0,69337 0,007389
Harald_ctrl 5_Fakten 2517500 45238 1320 914 19171,12 20,97 0,017969 0,000524 0,02918 0,0003631 0,0202043 0,69242 0,007615
  • difference numbers/click not statistically significant (p-value = 0.6826)


test 41, Wednesday, 5.12., 6:15 pm - Thursday, 6.12., 3:30 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
LP-
Visitor/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
photo 5_facts 4618200 77639 2383 1721 36460.45 21.19 0.016812 0.000516 0.03069 0.0003727 0.02217 0.72220 0.0078949
no_photo 5_facts 4642200 77766 2488 1753 38080.87 21.72 0.016752 0.000536 0.03199 0.0003776 0.02254 0.70458 0.0082032

--TobiasSchumann(WMDE) (talk) 11:25, 27 December 2012 (UTC)[reply]


Test 44 & 50: banner layouts personal appeals[edit]

In these two tests we analyzed different ways to present personal appeals in banners. We know from the second week of our campaign that there was a point, where more pepole were convinced to donate by the personal appeal of Jimmy than the facts banner with which we started the campaign (test 17). Of course by improvements in banner design this result was reversed (test 23). After weeks of facts banners - why should't personal appeals gain more attention and more donations by now?

To find out we tested our current facts banner which features the appeal of Wikipedia editor Harald Krichel inside against a personal appeal banner of Harald. Compared to the old inline banners our new appeal banner has a new design (top, yellow, sticky). Still, it did not work out. While the personal appeal indeed gets more attention (click rate +16%), less people are convinced to donate (numbers/imp. -53%).

Since we couldn't be sure about our new appeal banner layout (which e.g. has a smaller picture) we tested it against the classic inline banner. Here we can see that our layout modifications really improved the appeal banner. The old inline banner received far less clicks (-40%) and didn't convince as many people to donate (numbers/imp. -21%).


test 44, Tuesday, 11.12., 3:45 pm - 8:15 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
LP-
Visitor/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
facts 5_facts 986200 15736 422 297 6370.99 21.45 0.015956 0.000428 0.02682 0.0003012 0.01887 0.70379 0.0064601
pa_top 5_facts 1001200 18564 241 142 3978.01 28.01 0.018542 0.000241 0.01298 0.0001418 0.00765 0.58921 0.0039732
  • difference click-rate: statistically significant (p-value < 2.2e-16)
  • difference numbers/imp: statistically significant (p-value = 4.127e-14)


test 50, Friday, 14.12., 9:30-11:45 am UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
LP-
Visitor/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
pa_top 5_facts 726300 11214 241 139 4490.00 32.30 0.015440 0.000332 0.02149 0.0001914 0.01240 0.57676 0.0061820
pa_inline 5_facts 809100 0 7446 123 4193.00 34.09 0.000000 0.009203 0.00000 0.0001520 0.00000 0.01652 0.0051823
  • difference click-rate: statistically significant (p-value < 2.2e-16)
  • difference numbers/imp: statistically significant (p-value = 0.06229)

--TobiasSchumann(WMDE) (talk) 11:25, 27 December 2012 (UTC)[reply]


Test 45 & 52 : fundraising video, inside facts banner[edit]

In this year's campaign we wanted to illustrate as many perspectives on Wikipedia as possible and therefore also let readers speak in a donation appeal. Another goal was to try new ways to communicate. Both we intended to achieve by producing a fundraising video for the campaign. In this two-minute clip Wikipedia readers of different age, background and interest describe their experiences with the online encyclopedia and explain why they like it so much. After that Wikimedia Deutschland CEO Pavel Richter in a short appeal asks for suppport.

Given the lack of experience with video appeals we were very curious about the test results. Now we are happy to see: inside the facts banner the video increased donations per banner click by 12% compared to the current control facts banner with a personal appeal of Wikipedia editor Harald Krichel. Hence, after one week with an editor appeal inside, we continue to show the video inside the facts banner. Since for the success of a video length is a crucial factor we were wondering if a shorter version would perform better than the relative long lasting 2 minute clip. In another test we analyzed the effect of a condensed version which only consists of the appeal of Pavel. This short version improved donations per bannerclick by another 7,7%. Still, this result doesn't hold generally since it is a strong influencing factor that the video is combined with the facts banner.


test 45, Wednesday, 12.12., 1:45-5:45 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Donation/
Imp
Donation/
Click-Drop
Amount/
Imp
ctrl 5_facts 1606000 24412 528 349 7732.74 22.16 0.015200 0.0002173 0.01430 0.0048149
video 5_facts 1596200 25077 611 403 8798.28 21.83 0.015710 0.0002525 0.01607 0.0055120
  • donation/click: p-value = 0.1067 (chi-squared-test)


test 50, Friday, 14.12., 6:30 pm - Saturday, 15.12., 1:00 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
video_long 5_facts 2783500 48063 1319 1012 21565.91 21.31 0.017267 0.0003636 0.02106 0.76725 0.0077478
video_short 5_facts 2794800 47875 1296 936 19848.08 21.21 0.017130 0.0003349 0.01955 0.72222 0.0071018
  • donation/click: p-value = 0.09847 (chi-squared-test)


--TobiasSchumann(WMDE) (talk) 16:26, 9 January 2013 (UTC)[reply]


Test 46, 47, 48 & 49: facts banner vs. video banner[edit]

One reason of showing video and personal appeals is to win attention especially from new target groups. Although we already had a multifaceted campaign with different kind of appeals this all took place inside the facts banner so that nobody realizes the change without clicking on the banner. With our campaign video, we hoped to receive high attention so that we tested a new banner design announcing the video against the facts banner (test 46). Unfortunately, the results werde disappointing: donation/impression was reduced by 70%. The new banner did't even receive more clicks.

During three further test we aimed to improve the performance of the video banner on ground of our previous findings. We strenghened the direct appeal to the readers and added the facts-text into the unfolded banner (test 47). Than we changed the video still inside the banner, altered the banner wording and modified the position of the facts-text inside the banner (test 48). In a final step we added two explicit calls to donate (test 49). Step by step we managed to improve the video banner so that it now received much more attention (e.g. clicks) than the facts banner. Although we did not achieve to reach the level of success of the facts banner. A decline of 27,9% in donation/imp. of the optmized video banner (compared to the facts banner) in test 48 also means an improvement by more than 100% compared to the first video banner version. This gives us confindence to achieve further improvements of non-facts banners.


test 46, Wednesday, 12.12., 6:00-7:45 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Donation/
Imp
Donation/
Click-Drop
Amount/
Imp
facts_video 5_facts 622200 10595 247 170 3517.88 20.69 0.017028 0.0002732 0.01605 0.0056539
video_1 5_facts 642000 10218 91 52 1335.00 25.67 0.015916 0.0000810 0.00509 0.0020794


test 47, Thursday, 13.12., 11:00 am - 1:00 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Donation/
Imp
Donation/
Click-Drop
Amount/
Imp
facts_video 5_facts 719400 9654 253 167 3365.22 20.15 0.013420 0.0002321 0.01730 0.0046778
video_2 5_facts 717400 9868 142 72 1738.50 24.15 0.013755 0.0001004 0.00730 0.0024233


test 48, Thursday, 13.12., 3:15 am - 6:15 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Donation/
Imp
Donation/
Click-Drop
Amount/
Imp
facts_video 5_facts 1091600 16435 372 264 5533.50 20.96 0.015056 0.0002418 0.01606 0.0050692
video_3 5_facts 1066300 20898 305 186 3805.50 20.46 0.019599 0.0001744 0.00890 0.0035689
  • donations/imp: difference statisticaly significant (p-value = 0.0006062, chi-squared-test)


test 49, Friday, 14.12., 11:30 am - 2:30 pm UTC

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Donation/
Imp
Donation/
Click-Drop
Amount/
Imp
facts_video 5_facts 1010400 13345 348 250 5663.00 22.65 0.013208 0.0002474 0.01873 0.0056047
video_4 5_facts 969000 15139 266 169 3478.33 20.58 0.015623 0.0001744 0.01116 0.0035896

--TobiasSchumann(WMDE) (talk) 16:26, 9 January 2013 (UTC)[reply]


Test 51: personal editor appeals[edit]

Friday, 14.12., 2:30-6:30 pm UTC

In this third group of new personal appeals featuring three Wikipedia editors we illustrated the wide age sprectrum of the volunteering encyclopedians. Beeing 86 years old, Dr. Peter Cueppers already edited Wikipedia more than 19.000 times. In his appeal he describes how he began contributing to the free encyclopedia. That it is possible to get involved with Wikipedia at a young age is shown by Tobias Klenze who started editing at the age of 16. As a crucial reason he states his motivation to help others and he emphasizes the voluntariness of the community as a precioius characteristic of Wikipedia. With a full time employment and a family Gereon Kalkuhl needs to take the time to write about topics like his favourite field chess. In his appeal he points out the high relevance of free access to knowledge for all parts of the society.

We are very happy about the results of this test. The appeal of Dr. Cueppers came as close to the success of Jimmy's appeal as no other text from this year (-7,8% donation/imp). Also the appeal of Tobias performed quite well (-24,54% donation/imp) whereas the one of Gereon didn't convince readers as good (-42,9% donation/imp). A closer look at the results reveals that the reasons for good performance once again lie in a high click rate e.g. attention that non-Jimmy appeal banners often are receiving. While the appeal texts of Gereon and Tobias have a similiar conversion (donation/click) far more people click on the banner of Tobias (+31% compared to Jimmy). One possible explanation could be that Tobias and Dr. Cueppers with their age irritate assumptions about the typical Wikipedia editor and therefore create interest. Still it is not only the high click rate (+33% compared to Jimmy) but also a good performance of the appeal text that explains the success of Dr. Cueppers.


Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
LP-
Visitor/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
Jimmy 5_facts 563400 10911 211 153 3425.00 22.39 0.019366 0.000375 0.01934 0.0002716 0.01402 0.72512 0.0060792 27.74
Tobias 5_facts 648900 16415 241 133 4492.56 33.78 0.025297 0.000371 0.01468 0.0002050 0.00810 0.55187 0.0069233 26.32
Dr.Cueppers 5_facts 571200 14715 227 143 3433.00 24.01 0.025762 0.000397 0.01543 0.0002504 0.00972 0.62996 0.0060102 25.00
Gereon 5_facts 645300 12710 158 100 2740.00 27.40 0.019696 0.000245 0.01243 0.0001550 0.00787 0.63291 0.0042461 19.10

--TobiasSchumann(WMDE) (talk) 16:26, 9 January 2013 (UTC)[reply]


NOTE: Meanwhile we successfully ended the 2012 fundraiser. Our more than 60 tests played a significant role in achieving our campaign goal. Still the results and 
reportings of some tests are missing. To supply a full documentation of the campaign we will continue to post test results in the coming days and weeks.


Test 53 & 54: 5€ banner[edit]

To have a strong message for the final run of the campaign we tested the most successful banner text from last year ("If everyone donated 5€..."). Instead of beeing part of the facts text we wanted to see the effect as isolated sentence on the banner (whereat the facts text was displayed on the banner after clicking on it) (test 53). Interestingly the text had the same effect as last year: decreasing the click rate (-27%) but increasing donation rate (+64%). Great! It's the first time that the facts banner was beaten by another banner.

In the following test we analysed which appeal works best inside the "5€ banner": the campaign video or the appeals of Dr. Cueppers, Pavel or Jimmy (test 54). With small differences the best performing appeal was the one of Pavel, the least one that of Jimmy. But unfortunately the results are not statistically significant.


test 53, Saturday, 15.12., 1:00 - 4:00 pm UTC

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
facts banner 5f 822200 13578 346 257 5625.30 21.89 0.016514 0.02548 0.0003126 0.01893 0.74277 0.0068418 21.50
5€ banner 5f 840400 10111 563 432 8449.93 19.56 0.012031 0.05568 0.0005140 0.04273 0.76732 0.0100547 22.47
  • click-rate: -27.15% (p-value < 2.2e-16)
  • don/click: +125.70% (p-value < 2.2e-16)
  • don/imp: +64.44% (p-value = 1.756e-10)


test 54, Saturday, 15.12., 4:00 - 9:00 pm UTC

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
5€_Video 5f 789000 9610 531 403 7271.00 18.04 0.012180 0.05525 0.0005108 0.04194 0.75895 0.0092155 21.90
5€_Pavel 5f 799500 9711 584 446 8563.41 19.20 0.012146 0.06014 0.0005578 0.04593 0.76370 0.0107110 24.32
5€_Cueppers 5f 713400 8551 492 369 6273.00 17.00 0.011986 0.05754 0.0005172 0.04315 0.75000 0.0087931 20.97
5€_Jimmy 5f 701700 8432 458 332 5144.50 15.50 0.012017 0.05432 0.0004731 0.03937 0.72489 0.0073315 24.91
  • appeal_Pavel: don/imp: +9.21% (p-value = 0.1993)


Test 55 & 56: Layout banner of Dr. Cueppers' appeal[edit]

To analyse different ways to present the very successfull appeal of Dr. Cueppers on a banner was the aim of these two tests. At first we tried out two personal appeal banners with one featuring a quote to hopefully gain interest of the readers (test 55). Unfortunately the quote had no effect at all.

We then once again tested a personal appeal banner against a facts banner with the appeal inside the drop out. The result shows that more readers clicked on the personal appeal banner but a smaller percentage actually donated. Still the result is not as bad since the degree of the decline of donation rate (donation/imp) is lesser than in test 44 (-32% compared to -53%).


test 55, Monday, 17.12., 3:00 - 6:00 pm UTC

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
pa_Cueppers_ctrl 5f 1254300 30691 457 276 6190.11 22.43 0.024469 0.01489 0.0002200 0.00899 0.60394 0.0049351 22.04
pa_Cueppers_cite 5f 1267500 31018 451 275 6294.00 22.89 0.024472 0.01454 0.0002170 0.00887 0.60976 0.0049657 22.32
  • don/imp: -1.38% (p-value = 0.8686)


test 56, Tuesdeay, 18.12., 11:15 am - 2:00 pm Uhr

Banner Landing-Page Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
LP-
Visitor/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
facts_Cuepp 5_facts 1033100 19338 492 366 8517.62 23.27 0.018718 0.000476 0.02544 0.0003543 0.01893 0.74390
pa_Cuepp 5_facts 1056100 23810 441 254 6366.95 25.07 0.022545 0.000418 0.01852 0.0002405 0.01067 0.57596
  • click-rate: + 20,45% (p-value < 2.2e-16)
  • don/click: -43,63% (p-value = 7.569e-13)
  • don/imp: -32,12% (p-value = 1.814e-06)


Test 57: new personal appeals[edit]

Tuesday, 18.12., 2:00 - 5:45 pm UTC

A bunch of new interesting stories and perspecitvse are told in the new appeals we tested today. Robin Müller already as a shool child began to participate in the largest free encyclopedia, Christoph Meineke as a mayor and Wikipedian explains the importance of Wikipedia in his small town and donor Cornelia Dietz writes about quality and helpfulness of the content. As in the other appeal tests the click rates increased compared to Jimmy's appeal but less people donated. While Cornelia recieved the highest attention (click rate +33%) the appeal of Robin performed best in total (don/imp -27%).

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
Jimmy 5f 697200 13532 262 160 4040.55 25.25 0.019409 0.01936 0.0002295 0.01182 0.61069 0.0057954 26.43
Robin Müller 5f 805600 20080 261 134 4214.00 31.45 0.024926 0.01300 0.0001663 0.00667 0.51341 0.0052309 22.88
Christoph Meineke 5f 812700 18809 255 121 3326.00 27.49 0.023144 0.01356 0.0001489 0.00643 0.47451 0.0040925 25.23
Cornelia Dietz 5f 706900 18319 194 104 4378.18 42.10 0.025915 0.01059 0.0001471 0.00568 0.53608 0.0061935 20.93
  • Robin Müller: click-rate: +28.42%; don/click: -43.54%; don/imp: -27.52% (p-value = 0.005769)
  • Christoph Meineke: click-rate: +19.24%; don/click: -45.57%; don/imp: -35.13% (p-value = 0.0002949)
  • Cornelia Dietz: click-rate: +33.52%; don/click: -51.97%; don/imp: -35.89% (p-value = 0.0003719)


Test 58: facts banner: appeals of Pavel and Dr.Cueppers[edit]

Tuesday, 18.12., 7:00 pm - Wednesday, 19.12., 12:15 am UTC

Given the success of the appeal of Dr. Cueppers and the good performance of Pavel's appeal in combination with the 5€ banner we wanted to see which of both works better in combination with the facts banner. With an increase of 10% in donation rate the appeal of Dr. Cueppers does.

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
facts_Pavel 5f 3300300 61264 1650 1174 25596.74 21.80 0.018563 0.02693 0.0003557 0.01916 0.71152 0.0077559 25.47
facts_Cueppers 5f 3293000 60851 1747 1285 28382.53 22.09 0.018479 0.02871 0.0003902 0.02112 0.73555 0.0086190 25.58
  • don/imp: +9.71% (p-value = 0.02180)


Test 59: merging facts and personal appeal[edit]

Thursday, 20.12., 11:45 am - Friday, 12.12., 10:00 am UTC

What an exciting test. We are still working to optimze the banner layout for personal appeals. So far the problem was that we only were able to show personal appeals within the drop down banner. Hence we were unable to let readers know when we have changed the appeals because the facts stay the same all the time. But personal appeal banners were not as successful as the facts.

What we have learned from tests with the fundraising video is that moving the facts text into the drop down block has a positive effect. Our aim with this test was to make this changing more visible. For that, we changed the layout in terms of colour (red) and text (white with a 3D-effect). Due to this we were able to increase the success of the personal appeal banner profoundly. With a decrease of the donation rate by 6,7 % it now performes nearly on the same level as the facts banner - which is a sucess compared to the performance of the personal appeal banner in previous tests with decreases up to 53% (Test 44 and Test 56). With our constant work on optimizing the banner designs we were happy to find succeed in finding a decent way to present personal appeals.

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
facts_Cueppers 5f 4558900 84225 2157 1592 35548.10 22.33 0.018475 0.02561 0.0003492 0.01890 0.73806 0.0077975 22.89
pa_Cueppers_red 5f 4614300 95496 2246 1503 36993.23 24.61 0.020696 0.02352 0.0003257 0.01574 0.66919 0.0080171 23.20
  • click-rate: +12.02% (p-value < 2.2e-16)
  • don/click: -16.73% (p-value = 2.700e-07)
  • don/imp: -6.72% (p-value = 0.05285)


--TobiasSchumann(WMDE) (talk) 15:35, 10 June 2013 (UTC)[reply]


Test 60: more personal appeals[edit]

Friday, 21.12., 10:00 am - 5:00 pm UTC

How you became a Wikipedian on Christmas and why you can get into coding at the age of 12 - those were topics in our new round of personal appeals. Dirk Franke, a digital encyclopdian from the very early days, tells us how he perceived the start of Wikipedia. Jens Ohling and Denny Vrandečić are two programmers working for WikiData. Both tell us what their motives are while programming for Wikipedia.

The outcome of this test is similar to other personal appeal tests: a better click rate but worse donation rate compared to Jimmy. But we were very happy about how close we came with the appeals of Dirk (4 % lower donation per imp) and Jens (8 % lower donation per imp).

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
Jimmy 5f 802300 13831 393 259 6039.70 23.32 0.017239 0.02841 0.0003228 0.01873 0.65903 0.0075280 28.13
Dirk Franke 5f 775800 15601 366 239 4803.00 20.10 0.020110 0.02346 0.0003081 0.01532 0.65301 0.0061910 21.95
Jens Ohlig 5f 683300 12727 286 203 5420.79 26.70 0.018626 0.02247 0.0002971 0.01595 0.70979 0.0079333 21.74
Denny Vrandečić 5f 700900 12137 257 167 4317.50 25.85 0.017316 0.02117 0.0002383 0.01376 0.64981 0.0061599 16.08
  • Dirk Franke: click-rate: +16.65%; don/click: -18.21%; don/imp: -4.56% (p-value = 0.6019)
  • Jens Ohlig: click-rate: +8.04%; don/click: -14.84%; don/imp: -7.97% (p-value = 0.3753)
  • Denny Vrandečić: click-rate: +0.45%; don/click: -26.54%; don/imp: -26.19% (p-value = 0.002122)


Test 61 & 62: personal appeal banners: Dr. Cueppers, Dirk Franke, Jens Ohlig[edit]

With these two tests we compared the best performing personal appeals from besides Jimmy so far. The outcome shows that the "senior Wikipedian" Peter Cüppers performs way better than the young Wikimedia progammers.


test 61: Friday, 21.12., 03:15 pm - Saturday, 22.12., 11:30 am UTC

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
pA_Cueppers 5f 3028500 65157 1533 1059 26708.93 25.22 0.021515 0.02353 0.0003497 0.01625 0.69080 0.0088192 20.70
pA_Dirk 5f 3027600 61302 1350 905 20761.10 22.94 0.020248 0.02202 0.0002989 0.01476 0.67037 0.0068573 21.69
  • click-rate: -5.89% (p-value < 2.2e-16), don/click: -9.15% (p-value = 0.03222), don/imp: -14.52% (p-value = 0.0005225)


test 62, Saturday, 22.12., 11:30 am - Sunday, 23.12., 12:00 am UTC

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
pA_Cueppers 5f 4308900 90930 2011 1455 36632.47 25.18 0.021103 0.02212 0.0003377 0.01600 0.72352 0.0085016 20.92
pA_Jens 5f 4362400 82325 1769 1287 33033.07 25.67 0.018871 0.02149 0.0002950 0.01563 0.72753 0.0075722 19.54
  • click-rate: -10.57% (p-value < 2.2e-16), don/click: -2.29% (p-value = 0.5398), don/imp: -12.64% (p-value = 0.0004124)


Test 63: 5€ banner: Dirk Franke vs. Dr. Cueppers[edit]

Sunday, 23.12., 12:00 am - Wednesday, 26.12., 12:30 am UTC

The personal appeal banners have been up for the last three days now. Since christmas days have never been very good days in terms of donations, we did another longer test with the very effective 5 Euro-Banner with the appeals of Peter Cüppers and Dirk Franke. Both appeals perform about the same, with Cueppers performing about 3,5 % better. Interestingly, we were able to get a large sample resulting in a significant result even though the difference is pretty small. The second interesting observation is the interaction between the banner text and the appeal. We know from other tests that appeals don´t work for themselves but in combination with the banner text. In this test, we see that the banner text lowers the difference of the impact of the two appeals which has always been a lot bigger in previous tests.

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
5€_Cueppers 5f 13694900 152822 6840 5307 100731.12 18.98 0.011159 0.04476 0.0003875 0.03473 0.77588 0.0073554 19.07
5€_Dirk 5f 13680500 152959 6641 5112 99742.54 19.51 0.011181 0.04342 0.0003737 0.03342 0.76976 0.0072909 18.04
  • don/click: -3.77% (p-value = 0.04656)
  • don/imp: -3.57% (p-value = 0.0633)


Test 64 & 65: personal appeal Dr.Cueppers: 5€ banner, personal appeal banner, stickyness[edit]

Before we wanted to end the campaign with a 5-day-personal-appeal marathon of every one who participated with their story so far, we ran a couple of last tests. We compared the effect of Peter Cueppers' appeal in the 5 Euro banner and in the personal appeal banner (Test 64). Basically, both perform as in a the previous comparable test 53 but with a different amplitude. While in Test 53 the 5 Euro banner had a 27 % lower click rate, now we see a decrease of 51 %. The same with the donation rate (+64% vs. +26%). What we don´t know is whether the reason for this is an effect of "banner fatique" or a difference in the facts/appeal banner.

In the second test (65) we focused on the effect of the sticky function. Again, we see an outcome comparable to a previous test with a different magnitude. Certainly, the sticky function provokes lower click rates along with a boosting of the donation rate (Test 26: +128%; Test 65: +68%).


test 64, Wednesday, 26.12., 12:30 am - 2:45 pm UTC

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
5€_Cueppers 5f 699000 7289 310 227 4424.60 19.49 0.010428 0.04253 0.0003247 0.03114 0.73226 0.0063299 23.83
pA_Cueppers 5f 670600 14355 227 173 4060.00 23.47 0.021406 0.01581 0.0002580 0.01205 0.76211 0.0060543 23.78
  • don/imp: p-value = 0.02225


test 65, Wednesday, 26.12., 2:45 pm - Thursday, 27.12., 1:45 pm UTC

Banner Landing-
Page
Impressions Banner
Clicks
Visitors
LP
Donations Sum Average Click
Drop/
Imp
Visitor/
Banner-Click
Donation/
Imp
Donation/
Click-Drop
Donation/
LP-Visitor
Amount/
Imp
% Women
sticky 5f 4952600 98570 1853 1388 35532.50 25.60 0.019903 0.01880 0.0002803 0.01408 0.74906 0.0071745 22.28
non_sticky 5f 5669000 45621 1305 947 26481.50 27.96 0.008047 0.02861 0.0001670 0.02076 0.72567 0.0046713 20.18
  • don/imp: p-value < 2.2e-16


--TobiasSchumann(WMDE) (talk) 09:05, 11 June 2013 (UTC)[reply]