Jump to content

Fundraising 2011/Local testing/DE

From Meta, a Wikimedia project coordination wiki

Newest test requests

[edit]

that´s it!

Completed Tests

[edit]

Test 24: Jimmy-Banner-Test

[edit]

Tuesday, 3.1.2012, 10:35 - 13:05 UTC, 100%, anons only

This is it, our last test of the campaign. With this final one, we had been looking for a successful banner for the last two days. We tested the urgent Jimmy appeal and the 5€-banner with and without donation progress meter. To stress out urgency and the forthcoming end of the campaign we tried a completely new banner copy: "Only 2 more days to go. Still € 100.000 needed. Please help!" It worked very well. Together with the 5€-banner which includes the donation meter it was the most successful one. We decided to run the countdown-banner because of the higher donation sum in the end.

The test results clearly show that urgency and the near realization of the goal of the campaign are highly motivating to make a donation. In our first tests of the progress meter we did not have a result as clear as this. That leads to the conclusion that the proximity of the goal is very much important for the efficacy of the donation meter. Another result is similiar to previous tests: Banners with a direct call to donate gain less clicks than the personal appeals but in the end reach more donations. However this is probably very much affected by the the actual phase of the campaign. --TobiasSchumann(WMDE) 12:49, 4 January 2012 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy jacket 5€ Jimmy jacket LP 1238196 3945 187 3168,00 16,94 0,00319 0,0001510 0,04740 0,002559 0,803042 26,28
Jimmy jacket urgent Jimmy jacket LP 1237881 6979 136 2747,00 20,20 0,00564 0,0001099 0,01949 0,002219 0,393609 30,58
Jimmy jacket countdown Jimmy jacket LP 1236686 4385 246 5221,00 21,22 0,00355 0,0001989 0,05610 0,004222 1,190650 25,62
Jimmy spacey 5€ thermo Jimmy spacey LP 1237454 4277 246 4696,45 19,09 0,00346 0,0001988 0,05752 0,003795 1,098071 29,72

Test 23: Re-Testing Landing Page Picture

[edit]

Wednesday, 28.12. 2011, 11:00 - 13:30 UTC, 100%, anons only

Are our tests only snapshots limited in significance or are the results universally applicable? Can we trust our results and are they valid over a longer period of time? Bearing in mind that we rely on these results when designing and conducting the campaign these questions are of high relevance. We were really curious how this test from Wednesday would end: We simply repeated an old test of November. In test 10 we analyzed the importance of personalization of the landing page. We found out that a picture of the speaker of the appeal improves the efficacy and increases the donation sum. Wikimedia UK afterwards tried that test as well but came to no clear outcome. Moreover, the WMF in 2010 tested a picture in the appeal and had a negative result. Indeed, there were some good reasons to re-test that test. So what is the outcome? We breathe a sigh of relief and now have more certainty about the validity of our test results. Like in the first test of the personalization of the landing page, this time integrating a picture generated more donations. There are now good reasons to believe that our results are generally applicable. --TobiasSchumann(WMDE) 12:33, 29 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy Royal Tannenbaum without photo/banner 1132856 4964 89 2202,00 24,74 0,00438 0,0000786 0,01793 0,001944 0,443594 25,97
Jimmy Royal Tannenbaum top banner 1134581 4975 86 2737,00 31,83 0,00438 0,0000758 0,01729 0,002412 0,550151 28,00
Jimmy Royal Tannenbaum top photo 1132765 4873 107 3183,00 29,75 0,00430 0,0000945 0,02196 0,002810 0,653191 21,98


Test 22: Banner-Test: different pictures 5€-Banner

[edit]

Tuesday, 27.12. 2011, 14:30 - 17:00 UTC, 100%, anons only

This test was about a fine adjustment of the successful 5€-banner. We wanted to know if we could raise the relatively low click-rate by changing the speaker or simply the picture of him. Thats why we tested the actual banner against an alternative photo of Jimmy and the successful speakers Brandon and Ryan. In fact the new speakers moved more people to click, whereas the alternation of Jimmy's picture made no difference. Unfortunately, the conversion of Brandon and Ryan was pretty low. Very interesting to see once again: not only the number of clicks is important. How successful the 5€-banner in general is, you can realize by comparing the numbers per impressions in all tests of the last months: with the new banner the value of that rate could be increased tenfold. That is a value we had been seeing only during the first two weeks of the campaign and before it started. --TobiasSchumann(WMDE) 12:45, 28 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy Jacket 5€ thermo Jimmy Jacket LP 843239 2519 186 3703,00 19,91 0,00299 0,0002206 0,07384 0,004391 1,470028 17,11
Ryan 5€ thermo Ryan LP 836324 2731 145 2139,65 14,76 0,00327 0,0001734 0,05309 0,002558 0,783468 15,97
Brandon 5€ thermo Brandon LP 837329 2678 165 3185,00 19,30 0,00320 0,0001971 0,06161 0,003804 1,189320 18,60
Jimmy Spacey 5€ thermo Jimmy Spacey LP 836889 2358 183 3508,00 19,17 0,00282 0,0002187 0,07761 0,004192 1,487701 22,98

Test 21: Jimmy-Banner: 5€ vs. tax-deductibility

[edit]

Tuesday, 27.12. 2011, 09:00 - 10:45 UTC, 100%, anons only

As in our previous test this time we tried to find out if a succecful banner of WMF from last year's campaign would work as well in Germany: a reminder of the possibility to make a tax-deductible donation in 2011. Although the new banner did quite good the difference to the actual 5€-banner is profound. The 5€-banner another time reached a conversion of 10%, generated twice as much donations and had a much higher donation sum. --TobiasSchumann(WMDE) 12:45, 28 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy tax Jimmy LP 812486 2345 89 2617,00 29,40 0,00289 0,0001095 0,03795 0,003221 1,115991 15,38
Jimmy 5€ thermo Jimmy LP 812486 1996 201 4090,12 20,35 0,00246 0,0002474 0,10070 0,005034 2,049158 13,21


Test 20: Jimmy-Banner-Test: 5€ vs. urgent vs. thermometer

[edit]

Thursday, 22.12.2011, 09:30-11:40 UTC, 100%, anons only

We definitely found the banner for the coming Christmas holidays. It's quite rare that a test has as distinct results as this one. We tested two different aspects of the banner. First, we wanted to know if a very powerful banner text of WMF last year campaign would work in germany as well. We tried the text "If every reader donates 5€ our campaign would end today" and compared it's effect with the "urgent appeal". On the other hand we added the donation progress meter as a variation to each of both banners to find out if the thermometer would function as a strengthener. All banners had an identical landing page.

The results are exciting: Although less people clicked the 5€-banner - the click-rate was the lowest we have ever measured in a test - these people were very much convinced to make a donation. 10-12% of the visitors on the landing page made a donation, a by far unparalleled conversion rate! It's no wonder that the average donation is much smaller, but all in all the new banner text leads to an incredible augmentation of donation numbers of 100%. Less clear is the effect of the progress meter. In one case it could improve the efficacy, in the other not. What is clear and interesting is that in both cases less people clicked the banner. But these are more likely to donate as the higher conversion rate shows. --TobiasSchumann(WMDE) 11:32, 23 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
if everyone 5€ Jimmy LP 700640 2181 226 3929,56 17,39 0,00311 0,0003226 0,10362 0,005609 1,801724 22,56
if everyone 5€ thermo Jimmy LP 700900 1788 222 3949,00 17,79 0,00255 0,0003167 0,12416 0,005634 2,208613 13,14
urgent Jimmy LP 700858 2940 96 2238,00 23,31 0,00419 0,0001370 0,03265 0,003193 0,761224 15,48
urgent thermo Jimmy LP 700161 2259 107 2174,00 20,32 0,00323 0,0001528 0,04737 0,003105 0,962373 21,74

Test 19: test of various appeals

[edit]

Wednesday, 21.12.2011, 10:30 - 15:30 UTC, 100%, anons only

Great! This test has a really terrific result as we think. For the first time an appeal out of the german community was the winner of the test, even though with a small advance. Katrin, a donor and commited in the wikimedia-universe, wrote an appeal about the importance of Wikipedia which we already tested a couple of weeks ago. Her appeal worked quite good, but her banner only moved a small amount of people to click. This proved the weakness of the old-type banner style with people in front of a white background. So this time we tried it with a new picture, shooted in the botanical garden, of course. Not suprising, the new banner had a much higher click-rate and outperformed the other banners so that Katrin's appeal got the most donations and the highest donation sum. For another time we remarkably can see how good the new concept of photographs works. However Katrin's appeal did not have a good conversion. The appeal of Ryan Kaldari, which already outperformed Jimmy' text in our tests, once again proved his power and had the best conversion rate. Finally the result is not very distinctive so that strictly speaking Katrin and Ryan are on the same level. We decided to put Katrin's appeal live to maintain the campaign diverse and exciting. Unfortunately the appeal of Ward Cunningham - we think it's amazing he wrote one - was not as good as Katrin and Ryan even though all appeals worked quite good. --TobiasSchumann(WMDE) 16:49, 22 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Alan green LP Alan 1856858 6598 53 1938,00 36,57 0,00355 0,0000285 0,00803 0,001044 0,293725 20,00
Ward green LP Ward 1861560 7275 70 2750,00 39,29 0,00391 0,0000376 0,00962 0,001477 0,378007 20,31
Katrin green LP Katrin 1856880 12284 88 2888,00 32,82 0,00662 0,0000474 0,00716 0,001555 0,235103 17,57
Ryan green LP Ryan 1859754 7565 84 2205,00 26,25 0,00407 0,0000452 0,01110 0,001186 0,291474 20,55

Test 18: Jimmy's urgent appeal

[edit]

Tuesday, 20.12.2011, 11:00 - 15:00 UTC; 100%, anons only

Since we know that the days around Christmas and New Year are very important for the fundraiser we wanted to see, if a small strengthener can improve the effect of the banner. Would an "urgent appeal" bring an increase in click rate compared to the "personal appeal"? The answer is yes. In fact, this small modification in the banner text lead to a real difference. Considerably more people clicked on the banner and made a donation. --TobiasSchumann(WMDE) 16:49, 22 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy Jacket urgent LP Jimmy Jacket urgent 3180155 15803 228 4965,24 21,78 0,00497 0,0000717 0,01443 0,001561 0,314196 30,26
Jimmy Jacket LP Jimmy Jacket 3184272 12663 166 3011,00 18,14 0,00398 0,0000521 0,01311 0,000946 0,237779 26,67

Test 17: German editor and donor appeals Round 2 + Ryan Kaldari

[edit]

Thursday, 15.12, 12:45 - 16:30 UTC, 100%, anons only

After we have not found a new editor appeal to beat Jimmy the day before we thought to do another round of testing. Especially, since we had a new appeal of ED Pavel Richter and a new picture of Ryan (finally in green). So we were excited, if we would end up with a new banner after the test. And so we did. The results are: Pavel and Attila had good clicks but very low conversion rates. Ryan did extremely good. His appeal already worked pretty good during his first test, but with the new banner he certainly reached Jimmy level. He had a similiar conversion rate and moreover outperformed Jimmy in clicks and donation numbers. Of course we decided to go on with Ryan and leave him up for the coming days. But around christmas we will have another round of Jimmy banners (probably) to head towards a good years end.

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Pavel LP Pavel 1583377 9340 70 1799,00 25,70 0,00590 0,0000442 0,00749 0,001136 0,192612 19,67
Ryan LP Kaldari 1582892 12253 123 3317,50 26,97 0,00774 0,0000777 0,01004 0,002096 0,270750 29,81
Attila LP Attila 1583427 10459 51 1595,00 31,27 0,00661 0,0000322 0,00488 0,001007 0,152500 15,91
Jimmy Jacket LP Jimmy Jacket 1585317 8158 78 2148,00 27,54 0,00515 0,0000492 0,00956 0,001355 0,263300 30,43

Till Mletzko (WMDE) 14:48, 16 December 2011 (UTC)[reply]

Test 16: German editor and donor appeals

[edit]

Wednesday, 14.12., 10:00-14:00 UTC, 100 % for anons

For the first time this year we were happy to be able to test author- and donor appeals from Germany. With Kurt a long time wikipedian and former president of the board of WMDE asks for a donation and talks about what fascinates him about Wikipedia. Corinna and Katrin speak as donors about their personal relationship to the online encyclopedia and the importance it has to them. The appeals were quite diverse in content, equally their results were different. Unfortunately none of the appeals reached the level of Jimmy’s appeal. What the test reveals once again is that new banners receive a high attention and therefore generate a much higher click-rate compared to the banner of Jimmy. The exception is the banner of Katrin which is in an old banner layout showing the speaker cut out on a white background. This remarkably goes along with the findings about the importance of the picture and the efficiency of the new type of photographs in natural surroundings. We hope to improve the click-rate of Katrins appeal significantly with a new picture and by that find an alternative to Jimmy’s appeal since her conversion is quite solid. --TobiasSchumann(WMDE) 09:59, 16 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Katrin appeal Katrin 1647441 7712 61 1798,00 29,48 0,00468 0,0000370 0,00791 0,001091 0,233143 26,09
Corinna appeal Corinna 1643034 10369 63 2090,00 33,17 0,00631 0,0000383 0,00608 0,001272 0,201562 24,56
Kurt appeal Kurt 1646670 11034 53 1492,76 28,17 0,00670 0,0000322 0,00480 0,000907 0,135287 18,18
Jimmy Jacket appeal Jimmy 1646024 7379 95 3172,00 33,39 0,00448 0,0000577 0,01287 0,001927 0,429869 22,22

Test 15: Text vs Video

[edit]

Tuesday, 13.12.2011, 12:10-16:00, 100 % for anons

Video or text - what works better in persuading people to make a donation fo free knowledge? An exciting question that we tried to answer last summer by testing on the WMDE-run search portal wikipedia.de. Due to low traffic numbers of wikipedia.de the amount of donations was very low. Thats why our results were not very significant. But taken as mere indications the results were instructive. It didn't make a difference if video or text were displayed on the landing page. On the contrary, a video instead of a banner on wikipedia.de convinced a lot more readers to donate - we had a converson rate of 6%!

On Tuesday we tested that video again, thanks to the high traffic with a larger sample. How did we approach? We used three Jimmy banners, while two different banners linked to the video landing page. Here are the results:

  • Unfortunately the video didn't work on the landing page. The conversion rate is considerably lower than the text landing page.
This can have different reasons. Either videos don't work in general or particularly this video was not effective. Contrary to the old tests the content of video and text are not identical, that is comparability is limited. We know the current text appeal is a very effective one.
  • The click-rate show clearly the potential of videos
Obviously the inpact of the banners was very different. One of the two Jimmy banners linked to the video-page had a play button on it. By klicking on it no video started but the reader was directed to the landing page. This banner received by far most clicks. Reflecting on the low conversion rate we assume that the redirection to the landing page instead of running the video produced disappointment. But the click-rates remarkably show the attention a video directly on Wikipedia would receive.
  • The line "Please read", which we cut of in the video banners, is very important to generate klicks.
A comparison of the two banners without playbutton is very interesting. The huge differences in click-rates underline the results of former WMF-tests about the importance of the text line on the banner.

--TobiasSchumann(WMDE) 13:56, 14 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy jacket Jimmy text 2310573 11609 124 3008,00 24,26 0,00502 0,0000537 0,01068 0,001302 0,259109 35,45
Jimmy playbutton Jimmy video 2301875 19394 81 1941,00 23,96 0,00843 0,0000352 0,00418 0,000843 0,100082 27,54
Jimmy no playbutton Jimmy video 2305326 9625 50 1420,00 28,40 0,00418 0,0000217 0,00519 0,000616 0,147532 33,33

Test 14: Author Appeals

[edit]

Friday, 09.12.2011, 12:30-16:30 Uhr, 100 % for anons


The third test in one week - more than ever. This friday we tested two new editor appeals, compared to Jimmy and Brandon. It is very obvious: the new banners have higher click-rates and therefore much more visitors on the landing page. That shows clearly that new banners receive high attention and that people are curious and want to read the fresh appeals. However, a much smaller amount of landing page visitors actually donates compared to the appeal of Jimmy. Given his high conversion rate it is clear that his texts works much better than the new appeals. But fortunately the appeal of Susan generated a very high average sum. Together with the high click rate of her appeal she is on par with Jimmy. An effect of high attention that runs out fast. Thats why we decided to put her banner live, but only for a short period of time. Today we already switched banners again to run the Jimmy appeal. --TobiasSchumann(WMDE) 16:50, 12 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Susan Susan appeal 1403390 12132 95 3085,00 32,47 0,00864 0,0000677 0,00783 0,002198 0,254286 28,24
Brandon woods Brandon appeal 1403021 8739 77 2505,00 32,53 0,00623 0,0000549 0,00881 0,001785 0,286646 17,19
Karthik Karthik appeal 1403799 12589 63 1988,00 31,56 0,00897 0,0000449 0,00500 0,001416 0,157916 19,23
Jimmy Jacket Jimmy appeal 1404663 8255 120 2951,00 24,59 0,00588 0,0000854 0,01454 0,002101 0,357480 34,34

Test 13: Jimmy Banner

[edit]

Thursday, 08.12.2011, 14:50-18:50 UTC, 100 % for anons.

Since our German editor appeals were not ready yet, we decided to have a Jimmy-Banner-test. We wanted to see which one turns out to be best in order to have the right Jimmy banner to kick off when everything else drops. The results are a little bit confusing. More clicks on the new banner, but no big differences in the donation numbers. Royal Tannenbaum and Jacket had the best conversion rate, while the CtR-winner was Kevin Spacey. Please notice that Jacket had some high donations - which we did not exclude -, so the donation sum in that one should be read carefuly. What we can learn here is that we are able to turn on new Jimmy-banners without risking loosing donations. But see for yourself:

Banner Landingpage Impressions Visitors Numbers Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per Visitor
Sum
per Imp.
Sum
per Visitor
% Women
Royal Tannenbaum current LP 1936389 12892 153 3007,00 19,65 0,00666 0,0000790 0,01187 0,001553 0,233245 25,74
Kevin Spacey current LP 2008481 14980 153 3122,00 20,41 0,00746 0,0000762 0,01021 0,001554 0,208411 29,10
Jacket current LP 2009227 14554 169 4217,13 24,95 0,00724 0,0000841 0,01161 0,002099 0,289757 30,46

Till Mletzko (WMDE) 10:20, 9 December 2011 (UTC)[reply]

Test 12: layout donation form

[edit]

Wednesday, 07.12.2011, 10:00-16:00 UTC; on 100% for anons

It's done! A test we had on our list for quite a while - it' about colors. Before we conduct more tests on banners and appeals already this week we tried to improve the layout of our landing page again. We analysed different colors of the donation form. Amazingly it had an effect on the amount of donations coming in. Compared to the variants in red (our default), blue and orange the donation form in green worked best. --TobiasSchumann(WMDE) 17:34, 8 December 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Brandon woods red form 2634236 15822 132 3323,12 25,18 0,00601 0,0000501 0,00834 0,001262 0,210032 24,14
Brandon woods orange form 2638177 16844 143 4285,50 29,97 0,00638 0,0000542 0,00849 0,001624 0,254423 26,67
Brandon woods blue form 2633606 16539 136 4530,00 33,31 0,00628 0,0000516 0,00822 0,001720 0,273898 19,49
Brandon woods green form 2631397 16492 179 5653,00 31,58 0,00627 0,0000680 0,01085 0,002148 0,342772 22,50



Test 11: Jimmy vs. non-Jimmy

[edit]

30.11.2011, 12:00 UTC - 18:15 UTC, 100 % for anons

We have had Jimmy up for nearly the first two weeks. Since we saw a decline in donations we decided to run a non-Jimmy-appeal. Unfortunately, our German specific appeals are not ready yet. Fortunately, there are already great appeals out there we can adapt. So we did. We ran Jimmy against Ryan Kaldari, Brandon Harris and Alan Sohn. The result are interesting: Brandon did the best in CtR but the worst in conversion (but had the best donation sum). Alan, who had the least klicks, had the best conversion rate. So we decided to run Brandon 100 % and we actually made the best day within the last week. But see for yourself:

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy royal tannenbaum Aufruf Jimmy 2142694 15294 184 5376,00 29,22 0,00714 0,0000859 0,01203 0,002509 0,351510 27,78
Kaldari Aufruf Kaldari 2141460 15718 203 5402,00 26,61 0,00734 0,0000948 0,01292 0,002523 0,343682 20,47
Brandon woods Aufruf Brandon 2141181 27522 315 8416,29 26,72 0,01285 0,0001471 0,01145 0,003931 0,305802 20,22
Alan Aufruf Alan 2143790 13975 189 5208,00 27,56 0,00652 0,0000882 0,01352 0,002429 0,372665 20,13

I would like to emphasize three points to that test. 1. We were able to revert a declining donation income by changing the appeal. 2. A non-Jimmy-appeal works great in Germany aswell and 3. Brandon did not have the best conversion rate. This is why we will have to make the next step to more complex testing.

Till Mletzko (WMDE) 09:16, 2 December 2011 (UTC)[reply]

Test 10: landing page layout - top banner II

[edit]

Thursday, 24.11.2011, 14:30-17:30 UTC, 3 banners together on 100%

This time testing was again about the layout of the landing page. In a previous test we analysed wether a banner on the landing page affects the donation process negatively or not. The result was not distinctive. But it looked like the banner increases donations slightly. That was contradictory to our assumptions because appeal and donation form are moved downwards by the use of the banner. This can be a problem on small laptop screens. Our conclusion was: personalization of the landing page could be very successful. This time we wanted to test exactly that. We tested our default layout against a landing page with banner and a landing page with a picture of Jimmy added to the appeal. It worked really well! The personalised landing page with the photo really improved the conversion rate and by that the number and amount of donations. Due to this test we were able to change our landing page for better again. --TobiasSchumann(WMDE) 12:16, 25 November 2011 (UTC)[reply]

Banner Landingpage Impressions Visitors Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
royal tannenbaum top photo 1699056 13053 194 5054,01 26,05 0,00768 0,0001142 0,01486 0,002975 0,387191 25,99
royal tannenbaum top banner 1696215 13011 136 3036,00 22,32 0,00767 0,0000802 0,01045 0,001790 0,233341 23,26
royal tannenbaum without photo/banner 1694102 12961 149 3788,00 25,42 0,00765 0,0000880 0,01150 0,002236 0,292261 23,19

Test 9: landing page layouts

[edit]

Friday, 18.11.2011; 16:15-19:15 UTC ; both banners together on 100%

After several test on Banners and donation forms this test focussed on the layout of the whole landing page. The goal was to find out, if a more simple design brings more clarity to the site and by that a higher donation rate. On the right side of the landing page we have several infoboxes (the so called tower) in our default layout. In the test we let them out and brought the information to other pages, accessible by the modified navigation. We tested the new layout against our current landing page. The winner is: the new layout. But the result is not very distinct and should be treated with caution. --TobiasSchumann(WMDE) 15:21, 24 November 2011 (UTC)[reply]

Banner Landingpage Impressions Besucher Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
royal tannenbaum with tower 1892034 24612 361 7642,75 21,17 0,01301 0,0001908 0,01467 0,004039 0,310529 22,39
royal tannenbaum no tower 1890619 24557 391 8562,95 21,90 0,01299 0,0002068 0,01592 0,004529 0,348697 21,45

Test 8: banner test

[edit]

Tuesday, 15.11.2011, 17:30-19:30 UTC; both banner running 100%

Just having started the campaign the testing continues. When we heard from the success of the new Jimmy-Banner in tests of WMF and WMFR we decided to check immediately if it has the same impact on german Wikipedia. The new banner seemed promising with its increase in click-rate and donation amount. The test results are exciting. The new banner lead to a one hundred percent increase in donation amount compared to the old Jimmy-Banner. Obviously, the reason for this lies in the dramatic augmentation of the click rate, which is twice as high. With this outcome we of cource decided to run the new banner as our current campaign banner. --TobiasSchumann(WMDE) 12:36, 16 November 2011 (UTC)[reply]

Banner Landingpage Impressions Besucher Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
crossed arms default 1.594.843 17.955 277 5859,10 21,15 0,01126 0,0001737 0,01543 0,003674 0,326321 20,15
royal tannenbaum default 1.595.965 36.019 605 12.180,51 20,13 0,02257 0,0003791 0,01680 0,007632 0,338169 19,40


Test 7: appeal test

[edit]

11.11.2011, 10-12 UTC, Two Jimmy-appeals running 100%

This was the last test before the beginning of the Fundraiser. As a last preperation we wanted to know which of two different Jimmy-appeals works better. The banners were visible for 100% of anons. The new appeal stretches out the difference of Wikipedia compared to other succsessful internet sites like google. In tests of the WMF this appeal was very effective. After the test it is clear now that our old appeal works as successful as the new one. Both appeals have virtually the same results. So now we know that we have two promising appeals to work with in the next weeks. --TobiasSchumann(WMDE) 11:45, 15 November 2011 (UTC)[reply]


Banner Landingpage Impressions Besucher Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
crossed arms Aufruf new 1.502.516 13.447 319 6680,61 20,94 0,00895 0,0002123 0,02372 0,004446 0,496810 21,32
crossed arms Aufruf old 1.505.269 13.589 329 6870,60 20,88 0,00903 0,0002186 0,02421 0,004564 0,505600 25,84


Test 6: donation ask string

[edit]

07.11.2011, 14:30-16:30 UTC - Jimmy 100 %

Last monday we had our sixth test with Jimmy running 100 % for two hours for anons. We had three donation forms with different ask strings. We wanted to see if the amount and numbers of donation suggestions have an impact on the outcome. And it does! Our current ask string with five options beat out the other ones. We can now confirm the test conclusion from WMF last year that the more options you offer, the more people donate. In another test we will see if (more) higher amounts lead to an higher average donation.

Till Mletzko (WMDE) 15:15, 9 November 2011 (UTC)[reply]

Banner Landingpage Impressions Besucher Sum Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
crossed arms 3 amounts 1.345.338 16.924 165 4198,15 25,44 0,01258 0,0001226 0,00965 0,003121 0,248059 23,03
crossed arms 5 amounts 1.346.105 16.749 253 6313,10 24,95 0,01244 0,0001879 0,01511 0,004690 0,376924 22,53
crossed arms 4 amounts 1.349.501 16.894 207 5097,72 24,63 0,01252 0,0001534 0,01225 0,003777 0,301747 18,36

Test 5: radiobutton vs. dropdown

[edit]

31.10.2011, 13:00 - 15:00 UTC - Jimmy 100%

We ran our fifth test on Monday 31st. For two hours we had the crossed-arm-Jimmy-Banner up leading to two different landingpages. One LP had our current dropdown-donationform while the other had a new radiobutton-donationform. With the new form we optimized the usability and the donation process. Unfortunatley, the results are not very enlightening. We had more donations with the radiobutton-form but a higher donation sum with the dropdown-form. Next week, we want to see in another test if changing the donation ask string in the radiobutton-form has an effect on the donation sum.

Banner Landingpage Impressions Vistors Numbers Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
crossed arms Radio-Button 1.527.107 19.145 321 6.105,40 19,02 0,01254 0,0002102 0,01677 0,003998 0,318903 18,69
crossed arms Drop-Down 1.526.563 19.218 299 6.868,22 22,97 0,01259 0,0001959 0,01259 0,004499 0,357385 21,07

Test 4: Banner-Layout

[edit]

26.10.2011, Tuesday 10:15 - 16:15 UTC, all Banners with 25%

Banner Landingpage Impressions Visitors Numbers Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
crossed arms default layout 2.442.011 30272 576 11.611,76 20,16 0,01240 0,0002359 0,01903 0,004755 0,383581 21,35
white banner default layout 2.442.512 27752 436 8324 19,09 0,01136 0,0001785 0,01571 0,003408 0,299942 24,54
black banner default layout 2.444.291 17192 306 6500,88 21,24 0,00703 0,0001252 0,01780 0,002660 0,378134 23,53
green banner default layout 2.444.670 17429 305 6750 22,13 0,00713 0,0001248 0,01750 0,002761 0,387286 22,62

On Oct. 26th we tested the fourth time. This one was a performance test in order to check our server-infrastructure. The test lasted for six hours on 100% during daytime. The results show us that we are well prepared for the coming fundraiser from a tech perspective while the donation side also looks pretty good. We used the performace test for Jimmy-banners aswell. We were searching for an opener and we might have found it. The crossed-arms Jimmy performed the best (in generell the white banners did better than the other colours). But take a look at the stats yourself. If you have questions, go ahead: till.mletzko[at]wikimedia.de

--TobiasSchumann(WMDE) 17:09, 31 October 2011 (UTC)[reply]


Test 3: Top-Banner vs. without Banner

[edit]

18.10.2011, Tuesday 12:00 - 14:00 UTC, both Banners with 50%

Banner Landingpage Impressions Visitors Numbers Amount Average Click-
Through-
Rate
Numbers
per Imp.
Numbers
per View
Sum
per Imp.
Sum
per View
% Women
Jimmy Black Landingpage A
without Top-Banner
1.475.728 14.967 203 4174,77 20,43 0,01014 0,0001376 0,01356 0,002811 0,277128 21,67%
Jimmy Black Landingpage B
with Top-Banner
1.481.728 14.970 215 4582,76 21,32 0,01010 0,0001451 0,01436 0,003093 0,306130 17,21%


We had the third test on Oct. 18th. We had the Jimmy Banner up for two hours going to two different LP 100%. One of the LP had the Jimmy Top-Banner, the other one had none. In addition, we activated the CC-donation-option. The test was very successful from a techical and content perspective. Even though the differences in the absolut numbers are not that high, we can say that not using the top-banner on the LP has no big influence. Therefore we will not have it for the coming fundraiser to imcrease usability for smaller screens. On top of that, we added another testing variable: gender! We know that only a small part of our donors are female despite the fact that a lot more women using Wikipedia. We are convinced that raising the number of female donors is one of the major reasons for the success of the coming fundraiser. This is why the gender category will be one more thing to analyse and test.

TobiasSchumann(WMDE) 07:30, 21 October 2011 (UTC)[reply]

Impressions and rates added TobiasSchumann(WMDE) 09:36, 26 October 2011 (UTC)[reply]


Test 2: Top-Banner vs. without Banner

[edit]

10.10.2011; Monday between 12:05-13:05 UTC, beginning with 25% each, going up to 50% each after 20 minutes

Banner Landingpage Impressions View Click-Through-Rate Numbers Direct Debit Sum Direct Debit Numbers Money Transfer Sum Money Transfer Numbers PayPal Sum PayPal Numbers Total Sum Total
Euro
Average Amount per View Amount per Impression
Jimmy Black Landingpage A without Top-Banner 362.507 5203 0,0143 13 285 28 740
Jimmy Black Landingpage B with Top-Banner 361.612 5142 0,0142 14 320 18 465
40 605 113 2415 21,37 0,2334 0,0033

Results: this test was not really successful. We had a major bug in the donation form, which we discovered after the test was over. Due to the bug we had a lot of direct-debit-donors that jumped off right before they finished the donation and (even worse) some finished donations containing false adress information. We fixed that bug a couple of hours later (in the meantime we had to put down the donation site). In addition to that PayPal recently changed the information that they used to send us per default. Because of that we are currently not able to track the PayPal-donations to one or the other Landingpage. This is why the numbers above are not reliable and are not linked entirely to the tested Landingpages. I think we will have to repeat that test configuration in the coming weeks. The good thing about PayPal is that we will have a PayPal-iFrame set up in the coming days with which we can gather adress and tracking information on our side.

Till Mletzko (WMDE) 11:55, 12 October 2011 (UTC)[reply]

Test 1: Performance Test for new wiki-donationform

[edit]

30.09.2011; Friday between 12:05-13:05 UTC

Banner LandingPage Impressions Views Numbers Direct Debit Sum Direct Debit Numbers Money Transfer Sum Money Transfer Numbers PayPal Sum PayPal Numbers Total Sum Total
Euro
Average Click-Through-Rate Amount per View Amount per Impression
Jimmy Black Landingpage with new wiki-based donation form 97.311 2.332 13 265 9 350 12 275 34 890 26,18 0,0240 0,3816 0,0091

WMDE had its first test on de.wikipedia.org last week. We ran the black Jimmy banner for one hour between 12:05-13:05 UTC. We started with 5%, moved quickly to 10% and ended up with 20% for the second 30 minutes for anonymous users only. We tested our new wiki-based donation form with some alterations regarding the payment method money transfer. The test was quite successful despite some technical (e.g. Piwik) and minor functional deficits that we will correct for next time. The new donation form worked pretty well and now we are able to test different donation forms a lot easier. In addition to that we are now able to gather adress information from money transfer donation before the possible donor leaves our landingpage. Donations via money transfer usually does not contain adress information, so in the past we were neither able to send them a donation receipt nor cultivate them. But one has to keep in mind that we only gather adress information with our new donation form. Therefore those "donations" are in fact donation intentions. We can only say after some time if the donation intention became a real donation on our account. Till Mletzko (WMDE) 09:16, 5 October 2011 (UTC)[reply]