Research:Article Creation Workflow/Data and metrics

This page describes the data and metrics plan for the Article Creation Workflow (ACW) feature.

Test phases

• During phase 1 we will focus on the usage of the landing page to understand how many among those users who are directed to the landing page enter a specific funnel or successfully complete them. We will also measure the effectiveness of of ACW at getting anonymous users to register an account.
• During phase 2 we will study the quality and survival of pages created via different ACW funnels.

Bucketing

We will display the new experimental ACW landing page to a small sample (1%) of all users trying to create a page by following a redlink (whether coming from (1) a search result page, (2) another article containing a redlink or (3) the interstitial displayed when typing a URL with a non-existent page name). We will redirect the remaining 99% to the default edit screen.

Data

All data for test phase 1 are obtained via the clicktracking extension. The full specifications of the data collected are here.

Metrics

Entry points

Impressions of the ACW landing page as a function of the redlink referral:

• search
• browse
• url

Clickthrough rates

Volume of clicks per impression of the ACW landing page:

• log in (anonymous only)
• sign up (anonymous only)
• article wizard
• create draft (registered only)
• create directly (registered only)

Conversion rates

Ratio of successfully completed edits per impression of the edit screen via the following funnels:

• create draft
• create directly

Notes

• We will use as a baseline the conversions of users directed to the default edit screen.
• The order of the elements on the landing page will be randomized (with the exception of the exit button which will be always at the bottom).