# Banner Test - "Static VS Fader"

For additional documentation on the testing methodology please see the following pages:

## Banners

#### Result

Test Time: 2010-12-29 14:10:00 UTC - 2010-12-29 15:50:00 UTC
Sampling Interval = 2 minutes
Testing Interval = 20 minutes
Total Number of Samples per Class = 50

TEST RESULT INCONCLUSIVE

(*) The rate of donations per banner impression over a fixed time interval.

(**) The rate of amount50 per banner impression over a fixed time interval. Amount50 is the dollar amount raised from donations initiated under a given banner where all donations of more than \$50 are recorded as \$50 donations. This counters the skewing effect of outlier donations.

## Data Analysis

This section analyzes and interprets the results of the tests. It should be noted that the data is analyzed over a period at least as large as the full testing period and that the testing period was chosen based on the period of time where significant hits and donations were observed.

### Data Consistency and Cleaning

Analyzing the above plots the donation and impression data appear to be quite regular over the interval 2010-12-29 14:10:00 UTC - 2010-12-29 15:50:00 UTC. Therefore, two minute intervals will be used for sampling over this period as a source for the paired t-test to assess confidence in the winner.

### Modelling and Hypothesis Testing

"Static Banner" won in each case for donations/impression and amount50/impression with increases of 5.11% and 1.79% respectively. The student's t-test was used to assess confidence over each metric and the confidence in the winner for donations/impression and amount50/impression is at least 75.0% and 90.0% respectively. This is a very slight and non-significant win for the "Static Banner". Both banners effectively performed the same.

TOTAL DONATIONS "Static banner": 365

TOTAL AMOUNT50* RAISED "Static banner": \$5293.97
TOTAL AMOUNT50* RAISED "Fader banner": \$5501.81

* AMOUNT50 indicates the total amount raised where all donations greater than \$50 are taken to be a donation of \$50.

DONATIONS PER IMPRESSION:

Between 75.0% and 90.0% confident about the winner.

Fader VS Static -- 2010-12-29 14:10:00 - 2010-12-29 15:50:00

item 2  = Static banner

The winner Static banner had a 5.11% increase.

interval	mean1		mean2		stddev1		stddev2

0		0.00036		0.00032		0.00022		0.00005
1		0.00033		0.00034		0.00017		0.00003
2		0.00030		0.00035		0.00020		0.00016
3		0.00029		0.00030		0.00013		0.00013
4		0.00028		0.00031		0.00017		0.00008

Overall Parameters:

mean1		mean2		stddev1		stddev2
0.00031		0.00033		0.00018		0.00010

AMOUNT50 PER IMPRESSION:

Between 90.0% and 95.0% confident about the winner.

Fader VS Static -- 2010-12-29 14:10:00 - 2010-12-29 15:50:00

item 2  = Static banner

The winner Static banner had a 1.79% increase.

interval	mean1		mean2		stddev1		stddev2

0		0.00566		0.00411		0.00243		0.00186
1		0.00501		0.00526		0.00440		0.00029
2		0.00455		0.00533		0.00341		0.00294
3		0.00409		0.00455		0.00227		0.00086
4		0.00395		0.00441		0.00128		0.00154

Overall Parameters:

mean1		mean2		stddev1		stddev2
0.00465		0.00473		0.00295		0.00175

## Endnotes

1. Campaign = "20101228JA075"
2. "Static Banner" utm_source = "20101227_JA061_US"