Research talk:Account creation UX

Don't panic

Before anyone asks... this is our prep work for any tests we might run on the account creation process. It's not happening yet. Steven Walling (WMF) • talk 22:47, 17 April 2012 (UTC)

Requiring an e-mail address

Hi. Even as an experiment, I don't believe it's acceptable to require an e-mail address to create an account. I suppose experiments could nudge in that direction, but an e-mail address must not be made mandatory. --MZMcBride (talk) 02:49, 26 July 2012 (UTC)

Why? Most major websites require email these days, and the notion that you can't be anonymous because you have a confirmed email address has been disproven by Wikimedians time and again. Steven Walling (WMF) • talk 05:45, 26 July 2012 (UTC)

I say "don't require an e-mail address" and your response is "well, others do it and it's trivially easy to bypass as a protection measure"? Would you like to try again? --MZMcBride (talk) 16:16, 29 July 2012 (UTC)

Reducing amount of text in account creation process

I think it's a very noble goal to reduce the amount of text that currently weighs down the account creation process. You'll likely run into resistance from people who feel like the rules must be made explicit, though. w:MediaWiki:Fancycaptcha-createaccount has been trimmed significantly a few times (and it almost stuck, it looks like), but invariably an admin comes along and undoes the reduction of text. This is, of course, putting aside the issue of system message abuse and mis-use. Sigh. --MZMcBride (talk) 02:56, 26 July 2012 (UTC)

Ultimately, we may just remove that message from the interface if people cannot be trusted to avoid adding too much cruft to it. Considering the name, I'm not really sure why it has stuff not related to err... fancy CAPTCHAs? Anyway, the original account creation tests convinced the interested people to at least agree to a temporary test that removed such text, and I am confident that through the power of data from A/B tests we can combat the notion that the wall of text really prevents "bad accounts". Steven Walling (WMF) • talk 05:44, 26 July 2012 (UTC)

Asterisks

In redesigning the login form, should definitely use the form style guide, in particular the section about displaying fields. It says "no asterisks" pretty forcefully. --MarkTraceur (talk) 17:58, 31 July 2012 (UTC)

"Create Your Free Account"

I think there are two things at play in this proposed heading for the registration page--the personal nature of "your account" and the issue of price with "free". I'd like to suggest separating the two, maybe doing a four-bucketed test, to see if one or the other helps more, then maybe we can further explore that dimension in other trials. If both changes are made at once, no insight is gained as to which is more helpful! --MarkTraceur (talk) 18:07, 31 July 2012 (UTC)

We can get to this level of fine-grained testing later perhaps, but in the meantime, reminding people that they are in fact creating an account which is free as in beer is pretty standard. Steven Walling (WMF) • talk 19:26, 31 July 2012 (UTC)

Email address

This is definitely a contentious point. I would propose the following:

1. Try putting an optional email form on the registration page first, see if it gets used
2. Try making a simple link from post-login to add-email, see if that helps
3. Then, and only then, consider requiring email for random sample of users (but if they object, maybe put a "why?" link that will give the option to remove the requirement)

This seems like it's in the category of "things of which there should be extensive testing before anything is done that cannot be undone easily". --MarkTraceur (talk) 18:15, 31 July 2012 (UTC)

1 already exists and 3 would be how we would test this. Steven Walling (WMF) • talk 19:19, 31 July 2012 (UTC)

Notes from S

Some notes migrated from mediawiki.org, courtesy S.

"Conversion"

The key metric is conversion: users who navigate to the account creation page but fail to complete the action vs. users who successfully create an account on English Wikipedia.

Approximately 200 new accounts are created in English Wikipedia every hour. This volume of new accounts created gives us ample opportunity to test many separate, small changes in hourly or daily tests.

"Defining Blocks"

After successful account creation, Patrollers and bots find usernames against policy and block them.

from research topic $16, We want to relate how many blocks during the campaign and whether the user went through A/B/control (or none),

To do this we would have to log username rather than the hash. But user is making herself well-known by registering.

This information won't be directly provided by extension code,

nice to haves...

• The experiment will not track failures within the account creation.

Following links out of AC?

Track people following sidebar links like Help:Contents during account creation.

Where does Account creation traffic come from?

TODO: Need to verify experiment makes sense for all incoming traffic flows:

Mainly from "Create account" on every single page

There used to be only a "Login" on every page, and from there uses can Create an account. On this flow, CustomUserAccount (Lennart's project) may appear

Templates for shared IPs say "Create an account" and login.

Any anonymous edit has a login / create account banner.

How to track where they go after?

Maybe we leave something in ArticleSaveComplete for *all* experiments one that works for tracking all experiments. Ryan thinks there could be a table of hash, event name, and whether in earlier experiment. Not at first.

It should be trivial to add new tracking codes to the custom user config file. However, MediaWiki doesn't parse < a > elements; therefore, the only way to track link-specific events is to wrap them in a span or div and assign an ID to the wrapper. All changes will be prototyped and subject to code review.

We will additionally have any squid filters reviewed before deployment. Emery does not currently have disk space issues [1] and the experiment should not generate more than 1K events per hour, which it can easily handle. As long as other admins don't mess around with the markup of the AC templates, it's safe to enable clicktracking and collect the data we need.

ACC

No information about w:en:WP:ACC? Although I have created ~900 accounts, nearly no account has gained autoconfirmation (I had checked the accounts regular)... Moreover: is there a "newsletter" which can be posted at my talkpage (at enwp) if any updates happen? mabdul 07:59, 3 August 2012 (UTC)

I agree it's important, but it's farther downstream from the direct process, and we want to focus on one thing at a time. It may end up that CAPTCHA improvements being discussed may reduce the need for the requests process (such as by making them comply with accessibility requirements). In the meantime, our first priority is the actual registration pages and the welcome afterwards, since that is where the majority of people end up. Steven Walling (WMF) • talk 21:05, 3 August 2012 (UTC)

Yes and no: I agree that this is a part after the actual account creation and should or better saying doesn't have to be high priority, but actually there are many users requesting accounts (and I'm not talking about the blocked IPs (esp. the Checkuser and Schools-blocks), or students through the EP, or diabled persons) - I'm talking about users requesting an account and doing 'nothing' with the account. (Actually I have created 1039 accounts through ACC; and 181 not created mostly because of UPol) I'm just curios: I have watchlisted them and after a time of ~2months I check the accounts before I remove them from my watchlist and check if the had edited: nobody (with a few exceptions) do editing. My 'normal' understanding about creating an account (and the biggest need of) is the editing and it is just.... say... strange... Through our "Welcomer bot" the user gets welcomed (in my case) with the en:Template:Welcome-personal (a user preferences in the ACC tool), and thus getting the most needed links; But that doesn't change my overall feeling that the users who have (let) created an account but doing nothing with it... mabdul 01:20, 4 August 2012 (UTC)

You might be interested in https://toolserver.org/~DarTar/reg2/. The gap between registered accounts and those that have edited has been with us for years. We definitely need to tackle that particular issue, but our first step in dealing with how we can interest people in contributing post-registration will still come after basic fixes to the experience registering. Steven Walling (WMF) • talk 21:36, 4 August 2012 (UTC)

Steven, it might be worth expanding a bit on the rationale of the experiment as we did for R:PEF] --DarTar (talk) 00:36, 4 October 2012 (UTC)

Analysis requirements

(copied here from main page, these are now covered by the overall metrics)

Priority

• Conversions for each bucket, as a percent of pageviews
• Number of users indefintiely blocked from each bucket $n$ days after conversion
  • (Optional) Number of users indefinitely blocked from each bucket using one of the following comments in the block log:

1. {{Uw-botublock}}
2. {{UsernameBlocked}}
3. {{Softerblock}}
4. {{Causeblock}}
5. {{UsernameHardBlocked}}
6. {{Spamusername}}
7. {{Vaublock}}

Results

So I reread the results quickly and is my impression correct that the results were mixed and not really significant? mabdul 00:20, 30 March 2013 (UTC)

No. It's easy to get that impression when you see changes of 1-4%. But two pieces of context are important here, when evaluating whether that's successful or not in performing its core function (assisting people trying to sign up)...

1. Especially on the scale of 3-4,000 signups a day for English Wikipedia, it is extremely difficult to get gains on the order of double digits when it comes to signups. Even small startups consider a 1% gain well enough to run with a redesign in A/B testing.
2. The results all pass tests for statistical significance, meaning that even if we're only talking a few hundred more signups a week, that the likelihood this was due to random chance etc. (despite being small) is extremely low.

We did factor the lack of huge gains in to our decision about precisely what feature set we want in a permanent version (current description of that, at a high level and with mockups). We're going to go ahead with the basic clean up of the form, and we'll also be introducing some previously untested enhancements to deal with issues like our frustrating CAPTCHA system. But there are tons of other changes (some minor, some major) that we have deferred based the modest gains we made so far. Steven Walling (WMF) • talk 23:13, 31 March 2013 (UTC)