|This research page is kept for historical interest. Any policies mentioned may be obsolete. |
Please visit the research homepage for current research-related contents and discussions.
You can review the most recent Teahouse metrics report at Research:Teahouse/Metrics.
- 1 August Metrics
- 2 Highlights
- 3 July Metrics
- 4 Highlights
- 5 April 24th metrics report
- 6 April 2nd metrics report
- 6.1 Activity Metrics
- 6.2 Engagement Metrics
- 6.3 Retention Metrics
- 7 March 25th Report
- 8 March 12th report
- 9 March 5th report
Cumulative activity metrics for the period February 27th through August 26th are presented below. The metrics report from the pilot period is available here.
Participation :: overview
The Teahouse is busier than it's ever been. Although participation by new editors dropped off somewhat in the ~2 months following the end of the pilot period on May 26th, participation rebounded in mid-late July and has since increased substantially. Since July 23rd, an average of 46 new editors have participated per week, an increase of 53% over the pilot period. Two likely factors contributing to this increase are:
- hard work by Teahouse hosts, particularly in the quality of help they provide and their commitment to new editor outreach
- the implementation of several automated outreach features: automatic invitations of all new editors listed on the invitee reports, and new links to Teahouse in experimental deployments such as the New pages feed and AFT-5.
|total editors (on any WP:Teahouse/* page)||1250|
|joined & participated since pilot||12|
|total new editors (< 100 edits at first visit)||741|
Participation :: Q&A board
New editor participation in the Q&A board has risen sharply since the implementation of automatic invites. Since July 23rd, the average number of questions asked has increased 29% over the pilot period. The difference is even more stark in comparison to the three weeks immediately prior to automatic invitation (7.3 questions per day versus 4.8).
The Q&A board is highly interactive. 53% of questioners post a response (usually a follow up question, or a simple 'thank you') to the thread they started, and 22% of guests have asked more than one question. Despite the recent increase in questions per day, Teahouse host have maintained a high quality of service. Questions are still usually answered in about 30 minutes, and each question receives an average of 3 responses (not including any responses by the original questioner). The number of newcomers who respond to someone else's question remains low, at about 11%.
|average time to first response||1:06 hours|
|median time to first response||32 minutes|
|% of guests who respond to their own question||53%|
|questions per guest||1.6|
|responses per question||3|
|responses by new editors||10|
Participation :: Guests page
Guest profile creation has increased markedly since 7/23, up 53% per week from the pilot period. The implementation of automatic invites seems to have contributed to the number of new editors who introduce themselves at the Teahouse, even if they don't have an immediate question to ask.
|total new editor profiles||274|
Retention :: Visitors vs. non-visitors
In order to determine whether participating in the Teahouse has a positive effect on an editor's likelihood of continuing to edit Wikipedia and on their level of editing activity, we analyzed the editing behaviors of a sample of editors who were invited to the Teahouse during the pilot period, identified through either the Invitee report or the Articles for Creation process. Roughly half of these editors subsequently visited the Teahouse, the other half did not. To control for the possibility that new editors who did not visit had simply not seen the invitation, we excluded from our sample those who did not make at least one edit to Wikipedia after the date of invitation.*
Our main findings from this analysis are highlighted below.
- Teahouse guests made more edits on average. (216.7 versus 74.4) than editors who were invited to the Teahouse, but did not visit.
- Teahouse guests also subsequently edited more articles than invited non-visitors, even after controlling for whether the non-visitors had made at least 1 edit the day after receiving the invitation.
- Teahouse guests made more edits to talk namespaces. A higher proportion of guest edits were to talk namespaces (user_talk, Wikipedia_talk, article_talk): 35% on average for Teahouse visitors, versus 16% for non-visitors.
|Group||sample size||total edits||articles edited||% talk namespace edits|
* This sample differs somewhat from the sample used in our previous analysis. Specifically, we have removed the 'Control A' group from this analysis, due to the difficulty of making meaningful comparison between editors who were intentionally selected by Teahouse hosts as good candidates for invitation (invitees) and a random sample of highly active new editors. Our previous findings showed that both invitees and invited visitors showed greater retention and higher productivity than this group, so we are not concerned that excluding this sample invalidates our results.
Basic activity metrics for the month of July 2012 are presented below.
- questions per week has increased slightly, up to 43 per week from 36 in June
- 12 new hosts joined the Teahouse in July, up from 4 new hosts in June
- both questions and profiles increase noticeably after 7/23. Possible result of automated invites
- most other activity measures are roughly equivalent to the numbers for June
Overall participation in the Q&A board is in line with previous months, resulting in an average of 5.8 questions per day. However, since the implementation of automatic invites on July 24th, the average number of questions per day has increased to 8.4, compared with an average of 4.8 for the period July 1 - 23.
|guests - new editors (< 100 edits)||80||41.67%|
|guests - veteran editors (> 100 edits)||82||42.71%|
|guests - IP addresses||30||15.63%|
|hosts participating per week||13|
|guests participating per week||48|
|questions from new editors||77||42.54%|
|questions from veteran editors||62||34.25%|
|questions from IP addresses||30||16.57%|
|mean time to first response||1:02 hours|
|median response time||28 minutes|
|mean questions per guest||1.4|
|mean responses per question||2.6|
|responses by new editors||10|
|questions per week||43|
|questions per day||5.8|
Guest profile creation was slow for the first three weeks of July. However, the implementation of automatic invites seems to have contributed to a jump in the number of guests who create profiles. Since July 24th, when HostBot began sending out automated invitations to between 50 and 70 every day, 15 new editors have created profiles on the Guests page, a 300% rate increase from the weeks prior.
|profiles per week||7|
|total new profiles created||30|
April 24th metrics report
This section contains data related to activity on WP:Teahouse/Questions and WP:Teahouse/Guests, as well as activities related to inviting new editors to Teahouse. All data reported in this section was gathered between 2/27 and 4/24/2012.
Teahouse receives most of its visitors through direct invitation. Visitors who have not received a direct invitation seem to be more likely to ask a question on Teahouse/Questions than to create an intro box on the Teahouse/Guests page. For this analysis, a "new editor visitor" is defined as a user who either created their account in 2012 or who had made fewer than 100 edits under that account at the time they edited Teahouse/Questions or Teahouse/Guests.
|new editor visitors (invited)||136|
|new editor visitors (not invited)||57|
|total new editor visitors||193|
|new editor visitors (invited)||139|
|new editor visitors (not invited)||13|
|total new editor visitors||152|
Overall, the Teahouse has received an average of about 6 questions per day since February 27th, although this number varies significantly day-by-day and week-by-week, and has increased notably in recent weeks. It is common, but not the norm, for Teahouse visitors to ask more than one question. Questions are answered rapidly, with the average response time of about 1.5 hours and a median response time of half an hour.
|average questions per user*||1.5|
|% editors who asked more than one question*||22%|
|average questions per week||41.75|
|Time between question and first response|
|average response time||1:28:06|
|median response time||0:30:27|
* excludes questions asked by Teahouse hosts, WMF staff members and 1 disruptive editor
A content analysis of 100 questions each from Teahouse/Questions and Help Desk was conducted by a member of the research team. This analysis, which compared the types of questions asked in these two help forums over the same time period, suggests several intriguing differences between the help needs of new editors and the help needs of the general Wikipedia population:
- Teahouse guests asked more questions about rules of editing (such as questions about reliability of sources, notability and conflict of interest)
- Teahouse guests asked more (both technical and non-technical) questions about adding sources in general
- Teahouse guests asked fewer technical questions (such as how to create a template, or move a page) in general
- Teahouse guests asked more questions related to finding work to do, such as identifying articles in need of editing or WikiProjects looking for new members
Q&A page activity by host
This table contains the number of edits made to Teahouse/Questions by editors who have identified themselves as Teahouse hosts. Number of edits is a proxy for number of answers to questions. However, these numbers will be slightly skewed for editors who have also made changes to the layout of the Q&A page. For instance, the number of apparent "answers" by users Jtmorgan, Heatherawalls and Writ Keeper may be slightly inflated for this reason.
|Host name||# revisions to Teahouse/Questions|
|Bill william compton||5|
|total guest intro boxes created||172|
|average intros created per week||21.5|
Invitations to new editors
Invitees by source
|invitee identified through||# invite templates sent||# invitees who visited Teahouse (Guests page, Q&A, or both)||% invitees who visited Teahouse||# who edited /Questions||# who edited /Guests|
|New editor contrib||2205||97||4.40%||47||64|
|India Education Program||15||0||0.00%||0||0|
Templated invites sent by host
|host name||invites sent|
|Worm That Turned||12|
|Bill william compton||7|
Impact of email invites
We are interested in determining whether new editors who receive an email invite in addition to an invite template are more likely to ask a question or create an intro box at the Teahouse. In order to test this, we collected a sample of editors who were invited through the Teahouse invitee database reports, the new editor contrib filter or the feedback dashboard. We find that editors who received an email invite are roughly twice as likely to show up at Teahouse than editors who only received an invite template on their talk page.
|invitees not emailed||2562|
|# invitees who showed up||141|
|template + email invite, showed up||51|
|template only invite, showed up||90|
|% emailed invitees who showed up||7.08%|
|% not emailed invitees who showed up||3.51%|
Engagement and retention metrics
In order to begin to understand whether visiting the Teahouse has a positive impact on subsequent editing behaviors or editor retention, we analyzed the outcomes for three groups of editors in three different 'conditions'--two different control groups and an experimental group--drawn from editors who joined between mid-February and mid-April 2012.
- Control A: 75 new editors who met the requirements to be included on the Teahouse invitee report but whose usernames were not included on that report, who were not invited through any other forum, and who did not find their way to the Teahouse on their own.
- Control B: 74 new editors who were invited through the feedback dashboard, the Teahouse invitee reports and the new editor contribs filter, but did not ask a question on the Teahouse Q&A page or create an introduction box on the Teahouse/Guests page.
- Experimental: 75 new editors who were invited through the feedback dashboard, the Teahouse invitee reports and the new editor contribs filter and who did edit the Teahouse/Questions and/or Teahouse/Guests pages.
The table below displays aggregate data related to subsequent editing activity by members of the three different experimental groups. These measures were generated using a custom-built analytics platform used by the Wikimedia Foundation's Global Development department in order to assess the impact of outreach initiatives on subsequent editing behavior and editor retention. We use these measures to analyze whether Teahouse visitors engage in certain positive editing activities, such as whether they edit the article namespace, how many articles they edit, and whether their edits persist over time or are subsequently reverted or deleted.
Teahouse visitors (Experimental group) are more likely to edit the article namespace after being invited, and show a higher average number of edits to the article namespace, a larger volume of article namespace contribution, and a higher degree of edit persistence than either of the control groups.
Editors who receive a Teahouse invite template but do not subsequently visit the Teahouse (Control B) exhibit higher performance than Control A, but still do not edit as actively or productively as the Experimental group. This is likely to be an artifact of an expected "invitation bias" on the part of Teahouse hosts, which is discussed further in the next section.
A table key with detailed descriptions of each measure is provided below.
|GroupID||Editors with >0 NS0 edits||Avg Global Edits||avg bytes added (by editor)||avg bytes survived (by editor)||articles created/edited|
- Editors with >0 NS0 edits: editors in this sample who made at least one edit to the article namespace after their invite date*.
- Avg Global Edits: the average number of global edits, per editor
- avg bytes added (by editor): the average number of bytes added per group, post-invite
- avg bytes survived (by editor): the average number of bytes added which had not been reverted or deleted at the time the analysis was run
- articles edited: total number of different article namespace pages edited (or created) by group members, post-invite
Level of recent activity
In order to determine whether Teahouse visitors were active at a higher rate than non-visitors, we measured how many members of each group had made at least one edit to any namespace between April 20th and April 27th. Each of our three sample groups contained editors who were invited (or, in the case of Control A, met the criteria for invitation) between February 27th and April 10th, the minimum time elapsed time between invitation and the time period in which we analyzed follow-up activity was 10 days. The invitation date distributions between the three groups was similar.
Our findings for this round of analysis are comparable to the proportions we found last time around. As expected, invitees who did not visit Teahouse (Control B) are still active at a higher rate than Control A (12% versus 5%). This is likely the result of an expected selection bias among Teahouse hosts: they (appropriately) tend to invite new editors who show promise, and avoid inviting new editors who show obvious tendencies towards vandalism or who they judge to be less engaged--in other words, Teahouse hosts may avoid inviting new editors who they believe are likely to be blocked, or whose editing behavior in their first few days otherwise indicates that they may be unlikely to stick around. This selection bias would likely result in the phenomenon seen below: the editors from Control B continue to be active at a higher rate than editors from control group A, which consists a random, unscreened sample of new editors from the same sources.
However, new editors who were invited and who subsequently visited the Teahouse (Experimental group) were much more likely to remain active weeks later than Control B (28% versus 12%), consistent with our previous analysis. Although our sample is still relatively small and more nuanced retention measures are still needed, this week's confirmatory findings (coupled with the other subsequent editing activity data presented directly above) provides additional evidence that Teahouse is having a positive impact on retention by:
- encouraging sustained contribution from editors who would otherwise have stopped editing, and
- providing key support infrastructure for 'high quality' newcomers who are already intent on sticking around, but who face many social and technical challenges as they integrate themselves into the community.
|'||total editors in sample||# who made at least 1 edit between 4/20 and 4/27||% who made at least 1 edit between 4/20 and 4/27|
April 2nd metrics report
We report a variety of data related to the activity on WP:Teahouse, the experience of new editors who participated, and the impact of participation on editor retention. All data below refer to on-wiki activity that took place between 2/27/2012 and 4/2/2012. Additional analyses and updates to these existing analyses will be posted here in late April or early May, as the Teahouse pilot wraps up.
In this section we report metrics related to activity on WP:Teahouse and its sub-pages, as well as activities related to identifying and inviting new editors to Teahouse. All data reported in this section was gathered between 2/27 and 4/2/2012.
- Overall, we show a steady level of activity on Teahouse/Questions (the q & a board) and Teahouse/Guests (the guests intro page).
- More than half (61%) of new editors who ask a question on the q & a board return to thank the answering host, ask for followup or clarification, or ask an additional question (26%).
- Most new editors are identified through a few major sources: the Teahouse invitee reports, the new editor contribs filter and the feedback dashboard. Of these, the invitee reports and the feedback dashboard demonstrate a slightly higher 'hit rate', but in general hit rate is fairly low, hovering between 3% and 6% for these three sources through the pilot period so far.
The total number of new editors who visited WP:Teahouse between 2/27 and 4/2. All invitees who visited are counted. Editors who were not invited and whose accounts were > 1 month old or who had > 100 edits at the time of their visit are not counted, which may lead to under-reporting.
|newbie visitors who were invited||87|
|newbie visitors who were not invited||7|
|total newbie visitors||94|
|newbie visitors who were invited||79|
|newbie visitors who were not invited||43|
|total newbie visitors||122|
|total questions asked||199|
|total unique newbie visitors to Teahouse (either asked a question or made a guest intro)||164|
Many new editors visited Teahouse more than once. Hosts use a special Talkback template to let questioners know that their question has been answered, which increases the possibility that a questioner will return to Teahouse. Many visitors also returned to ask another question.
|average number of visits||2.950819672|
|num returning visitors||74|
|% returning visitors||61%|
|average questions per visitors||1.55|
|visitors who asked multiple questions||32|
|% vistors who asked multiple questions||26%|
Invitees by source
Invitees and invited visitors to WP:Teahouse listed by where they were found. "unknown" means that an invite template was transcluded on the new editors talk page, but their name was not recorded in the invite tracking spreadsheet. "Other" indicates that the inviting host specified "Other" as the invite location in the invite tracking spreadsheet.
|invited through||number of invites sent||invited visitors to WP:Teahouse (/Guests, /Questions, or both)||% invitees who visited|
|New editor contrib||1938||79||4%|
|AfC help desk||35||0||0%|
|India Education Program||15||0||0%|
|WP project signup||2||0||0%|
|AFC help desk||1||0||0%|
|Chzz\'s talk page||1||0||0%|
Invitees by host
Total invite templates transcluded since 2/27, by inviting host. "unknown" means that an invite template was transcluded on the new editors talk page, but the invite was not recorded in the invite tracking spreadsheet.
|host name||invites sent|
|Worm That Turned||11|
|Bill william compton||7|
Below we present a selection of questions and answers drawn from a survey of ~1500 Teahouse invitees and (new editor) visitors between February 27th and March 15th. This survey was active between 3/20 and 4/2/2012. We received a total of 160 responses (including partially complete surveys) and 124 completed responses.
- Satisfaction with the Teahouse Q&A board and Guests' page were overwhelmingly positive (86%/78% 'satisfied' or 'very satisfied')
- Overall satisfaction with Teahouse was roughly two-thirds 'neutral' and one-third 'positive', with very little 'negative' experience reported
- respondents especially appreciated the speed with which questions were answered, and the welcoming and helpful attitude of the answering hosts
- roughly 27% of respondents were female, noticeably higher than the average proportion for en.wiki
Q&A page satisfaction
Prompt: How satisfied were you with the quality of the answer you received at Teahouse/Questions?
Guests page satisfaction
Prompt: How satisfied were you with the experience of creating your guest page introduction?
Prompt: How satisfied were you with your overall experience on Teahouse?
Negative experience with Teahouse
Prompt: Was there anything in particular you disliked about your Teahouse experience? Tell us about it.
|I think an invitation to edit would have been nice; but, Wikipedia is an encyclopedia, and an invitation to social networking seems a bit weird.|
|didn\'t help me with what i wanted to do|
|Having my question ignored. Getting told, by a \"host,\" that my comments, due to extreme frustration with bad information on Wikipedia, were \"mean.\"|
Positive experience with Teahouse
Prompt: Was there anything in particular you liked about your Teahouse experience? Tell us about it.
|It was welcoming to us as new members of Wikipedia.|
|Most of the help I received was useful.|
|It\'s nice to have a way like this to have my questions answered|
|it was far more social and less confrontational than anywhere else on Wikipedia|
|The opinionated information may sometimes be political, although, it makes a great advantage to apply information and medicine to the media world.|
|The idea is great|
|How friendly and helpful people are on helping you out.|
|I like the name, and I really like that the language is inviting. It is unlike many of the \"Help\" pages that read more like a technical document with the occasional \"we\'re here to help each other\" message. Teahouse overall is very inviting and proves to be quite useful.|
|Promptitude in my (two) questions being answered.|
|The quick solution to my problem was great.|
|The response to my question really helped!|
|A good way of making me feel welcome on joining the site|
|It is nice that this exists. It was very nice and interesting to see the work that the Teahouse host had done in Wikipedia :)|
|Teahouse experience... That wasn\'t much, except that I had a good time introducing myself :) The only other thing that I want to mention is that I don\'t really know how to upload an avatar. I could have copied one from the list on the introduction page, but I want my own photo... so I ended up having no avatars instead, which is pretty sad. :(|
|I got my question answered even though it was one I should have been able to figure out myself.|
|I really liked how it was a new project in shot and was helping newcomers on Wikipedia|
|the answers are very understanding|
|I was just a newcomer in Teahouse and kind of an inexperienced guest|
|While I was nervous about posting a question, the open and friendly nature of the responses I received helped make me feel more comfortable with seeking help from others in Wikipedia.|
|The response was almost immediate and very kind. I feel the Teahouse is a great place for new Wikipedians to get orientations|
|The teahouse was a warming experience that got me over my fear of making major edits. Everyone in the tea house seemed to be very understanding and I don\' t think I noticed any kind of disrespect or mean responses at all like a lot of us were afraid of getting when joining Wikipedia and editing. Thanks!|
Prompt: What are some of the things are you are excited about doing on Wikipedia?
|My motivation is contribute for increment of free information to all|
|Getting knowledge and reliable knowledge :)|
|Translate articles into Spanish.|
|Contributing to certain articles of interest to me.|
|question needs rephrasing, could be read in several ways|
|Creating new articles, editing articles in topics that are of interest to me.|
|Editing, reading and writing articles|
|Looking forward to help increase quality of articles on Football, particularly USA, Canada, Greece and Spanish-speaking countries|
|Adding to knowledge, and correcting erroneous or unbalanced articles|
|Editing articles, reading articles, reading historical articles.|
|don\'t know, building an encyclopedia|
|Editing articles. Maybe not excited, more interested.|
|getting good grades|
|helping make articles better!|
|The sandbox = Independence. Showing the truth and countervandalism identity to keep track on many anonymous IP users while in disguise.|
|Editing, and contributing to what I think is one of the best inventions of 21st century.|
|Being able to contribute my knowledge to others|
|Simply to help have all knwledge in one palce, for free.|
|Centralizing all the world\'s knowledge.|
|editing to make the site better for all who use it|
|Editing and improving articles.|
|(should it be \"What are some things you are...Wikipedia?\") I\'m excited about creating new articles!!! I\'ve already created one, and I\'m looking forward to editing ones that are already here. I\'ve even fixed a couple of links on established pages. I\'m also excited to be apart of WikiProject Cats (because I love them... even though I can\'t hug \'em all).|
|contributing, wherever I can; there seem to be areas I could help expand on once I\'m more familiar with the commons.|
|The opportunity to edit in order to leave an existing article more accurate and interesting than one found it; to put into public domain what you already knew and had citation for; and to right inaccuracies when one sees them|
|writing articles, contributing|
|Because so many people use Wikipedia as a source of information, I am excited about promoting neutrality and fact-based claims that address a variety of topics.|
|Developing pages with other users.|
|I look forward to adding more matierial when I have more time to do it.|
|Just helping to contribute to the huge collected knowldege and doing my bit where needed.|
|Connecting certain interests to the information all ready out there, but not linked.|
|^Terrible grammar to this question! Correcting things like this is always nice :)|
|Adding accurate information and updates.|
|Copyediting. Maybe a bit of article editing (mostly developing underdeveloped articles that relate to my coursework).|
|I enjoy writing and I hope some of my inputs will be useful to future readers. I\'d like to help with site maintenance in due course as time allows and I learn more|
|Animal biology. Lots of topics that need work. Human biology much more thoroughly treated.|
|Editing pages to spread knowledge|
|EDITTING! (And by doing so, I won\'t need to go shopping with my mom! :D)|
|Editing and reading about new articles|
|I am a pretty casual editor. I just happened across some information that I knew was incorrect and corrected it.|
|the good feeling of expressing on a an prestigious as also genuinely intellectual forum|
|reviewing/edite content on U.S. history|
|Someday beating ClueBot NG to the punch. No, but actually I am researching for an article.|
|Not sure I understand the question which is semantically quite odd, but I like to add paragraphs to Wikipedia to share my (little) knowledge I have about some limited subjects... I do so in Dutch, French and English|
|Adding new pages for people who are notable but who don\'t have pages yet|
|Editing and using it to kill spare time|
|Finding out new things through research, especially on subjects I am passionate about.|
|I was happy.|
|Making new articles that aren\'t in here and learning Wikicode|
|Getting the information right.|
|I was excited to get my own userpage and have other editors respond to me and help build and give away a free encyclopedia that anyone can edit|
|getting answer to my question on the talk page of \"Bill Adler\"|
|Clearing facts up and making links between pages.|
|Actually, I have been impressed with Wikipedia since I started researching here.|
|editing and giving reliable citations|
|I like posting articles|
|I\'m excited to translate a few pages and help fill in the blanks in information on other language Wikipedias.|
|i would like to add photo\'s to my page but they were not accepted and I don\'t know why|
|Improving the quality of the content of topics of which I am familiar with; create new articles|
|I enjoy just surfing around, and if something catches my fancy I see if I can expand on it.|
|Just editing articles but id like to translate japanese|
|promoting facts and building a library representative of all people|
|Improving info on subjects of my research|
|I am interested in adding information in areas where I have an interest and have something to contribute. I try to correct errors of grammar and usage whenever I encounter them. In technical articles (current computer technology, e.g.) I find that while the technical material presented seems to be up to date, the writing tends to have more errors, something that does not surprise me.|
|I intend to use Wikipedia to improve my own writing, and to be able to proofread better.|
|Aspiring to Inspire before I Expire|
|Just editing in general. I didn\'t realize how much I liked editing.|
|Reading and correcting articles|
|Reading history, geography|
Prompt: Teahouse is still under development, and we're planning to add new features soon. Which of the features below would be most helpful to you? (select all that apply)
|1||Live Chat with Teahouse hosts||55||52%|
|2||Live Chat with other Teahouse guests||35||33%|
|3||A list of Wikiprojects that are looking for new members.||74||70%|
|4||Links to other \'Help\' resources, such as tutorials, videos, and how-to pages.||67||64%|
|5||More/better host and guest profiles||26||25%|
|6||Other (please suggest new features below, or use this space to elaborate on ideas you\'ve checked in the list above)||18||17%|
|Other (please suggest new features below, or use this space to elaborate on ideas you\'ve checked in the list above)|
|Online Jobs for the needy ones and Competitions for all the Users.|
|disregard the idea of another WP:HD|
|None. I\'m not really interested in socializing with Wikipedia editors using Wikipedia as a social hango-out.|
|The ability to become a recognised expert on particular subject matter|
|Teahouse should be put in a more obvious place on wikipedia.|
|Groups within groups? I think that\'s where the Wikiprojects come in but it would be nice to have subgroups for lady librarians or catlovers or music editors.|
|I like the idea of a live chat, but just wanted to add that it might make more since than the Q&A page. This way, editors can receive more immediate feedback.|
|I would LOVE a list of wikiprojects looking for new members. A lot of the ones I considered joining/joined are pretty much dead, and I don\'t know which other ones to look at.|
|Cool ways you can customize your user page. I have learned a lot from these.|
|They all sound pretty good to me. I doubt I will be a frequent flyer so don\'t give my opinion too much weight.|
|I think the experience is too fresh for me to be ablle to apply my mind|
|Not quite interested.|
|Tea house should have a place where editors cool off and talk about there day helping (like \"the waste bucket\" of Wikipedia)|
|Time is an issue for me, so I will need to withold this response for now.|
|Maybe a way to get help/answers to questions regarding info posted on particular wikipedia page. For example, as a new member, I\'m not good at editing, but I know of several sites that have false info that needs to be changed. Info I have can be verified.|
|Above all, the Wikiprojects list. Too many projects make it unclear or difficult to figure out just how active or inactive they may be.|
|Easier interface (using a format similar of that of social media websites) to interact with other members|
|At the moment I do not have anything to contribute to Teahouse, but I\'m hoping in the future that when I make contributions to an article that I will be able to find someone (through Teahouse) who will take note of my changes and give me a critique.|
Prompt: Teahouse is intended to be a friendly, supportive environment for all new Wikipedians, but we make a special effort to welcome female editors, who are underrepresented within the Wikipedia community. If you're comfortable telling us, what is your gender?
|1||I am female||33||27%|
|2||I am male||78||64%|
|3||I\'d rather not say||10||8%|
Although Teahouse has only been live for a little over a month, we do have some preliminary data addressing the question of whether Teahouse visitors tend to continue editing Wikipedia at a higher rate than editors who did not visit Teahouse. The table below lists three groups of editors who share the following characteristics:
- the created a user account between 2/26 and 3/17
- they made at least 10 edits within their first 24 hours
- they were not blocked from editing before their 10th edit
Some of these editors ("invited") were presented as 'potential invitees' on the Teahouse invitees database report, and were subsequently invited by Teahouse hosts via a talk page invite template, an email, or both. Of these, a small number ("visited" group) subsequently visited the Teahouse. A third group ("control") met the criteria for invitation, but were excluded from the invitees database report and thus Teahouse hosts were not given the opportunity to invite them.
In order to determine whether Teahouse visitors were active at a higher rate than non-visitors, we measured how many members of each group had made at least one edit between March 26th and April 2nd. As expected, "invited" visitors were active at a higher rate than the control group (12% versus 7%). This is likely the result of a selection bias among Teahouse hosts: they (appropriately) tend to invite new editors who show promise, and avoid inviting new editors who show obvious tendencies towards vandalism or who don't seem as engaged--in other words, they may avoid inviting new editors who are likely to be blocked, or who are more likely to leave, resulting in a higher proportion of subsequently active editors than the control group, which includes a wider variety of new editors.
However, new editors who were invited and who visited the Teahouse were much more likely to remain active weeks later than even their "invited" fellows (38% versus 12%). Although these numbers are small, and there may be additional selection biases at work, this is an early encouraging sign that Teahouse may be drawing editors in who would otherwise leave sooner and also providing key support infrastructure for 'high quality' newcomers who are already intent on sticking around, but who face many social and technical challenges as they integrate themselves into the community.
|group||total||number active in past week||% active in past week|
|control - met criteria for invitation, but not listed on database reports page||318||21||7%|
|invited - listed on db report (through 3/17)||266||33||12%|
|visited - Teahouse after db report invite||13||5||38%|
March 25th Report
Invitations by host
As usual, Rosiestep and SarahStierch lead the pack, invite-wise. Great work, you two!
The "unknown host" user here is intended to account for a large number of invite templates which were found through an API query, but which had not been included on the invite tracking spreadsheet.
|host||number of invites sent|
|Worm That Turned||8|
|Bill william compton||7|
Invitations by source
These are the places hosts are looking to find new users to invite. Since the database reports have been out of commission off and on for the last week or so(!), there have been relatively fewer newbies invited through that channel. However, database reports, new editor contribs and feedback dashboard remain the richest sources of newbies: new editors recruited from the feedback dashboard, the database reports and the new editor contribs page continue to show up at the Teahouse at the highest rates. Keep watching these sources!
|source||number attendees||hit rate (attendees/total invitees)|
|New editor contrib||67||4%|
As above, "unknown" in the table below accounts for a large number of invite templates which were found through an API query, but which had not been included on the invite tracking spreadsheet.
|invite source||number of invites sent|
|New editor contrib||1715|
|AfC help desk||27|
|Global Ed Collab||22|
|India Education Program||15|
|WP project signup||2|
|Chzz's talk page||1|
Overview of attendees
Total attendees, and another note on emailing
The following table summarizes the number of invited guests to WP:Teahouse.
As you can see from this table, emailing guests remains an effective strategy for getting them to show up. Guests who are emailed are significantly more likely to subsequently introduce themselves or ask a question than one who is not. Keep emailing!
|invited guests who showed up (Q&A, Guests page, or both)||117|
|invited guests who showed up (where emailed status is known)||111|
|invited guests (where emailed status is known)||2671|
|invitees who were not emailed||2188|
|invitees who were emailed||588|
|attendees (invitees who showed up) who were emailed||39|
|invitees who were emailed, but didn\'t show up||549|
|% Teahouse invitees who were emailed||22%|
|% Teahouse attendees who were emailed||35%|
Newbies who participated in WP:Teahouse
Because it's not easy to discern whether someone editing Teahouse pages is in fact a new user (as opposed to, say, a passing Wikipedian or an unofficial 'host'), editors were counted as 'newbies' if they they met one of the following two conditions:
- their account was less than 1 month old at the time they edited a Teahouse page, or
- they had fewer than 100 edits to their username at the time they edited a Teahouse page
Summary: A good number of Q&A participants are finding the Teahouse without being explicitly invited (or at least those invites aren't being recorded as such). However, new users appear to be much less comfortable introducing themselves on the 'guests' page when they have not been explicitly invited to Teahouse. This suggests to me that the 'welcoming' aspect of explicit invites puts guests more at ease, and may increase the perception that they are part of a community.
|total questions asked||154|
|invited q&a participants||46|
|other newbie q&a participants (not explicitly invited)||46|
|total newbies on q&a||105|
|'other' Teahouse/Questions participants*||103|
|invited Teahouse/Guests participants||68|
|Teahouse/Guests participants - not invited||8|
|total newbies on Teahouse/Guests||76|
* This 'other' category is a catchall for a bunch of different kinds of editors, from those who are performing 'host' duties but have not included themselves on the Teahouse hosts' list,r to relative newbies who were not explicitly invited but found the Teahouse and asked ask a question, to many other Wikipedians of all levels of experience who dropped by to ask or to answer a question.
March 12th report
Basic stats - this week
40 new editors edited the Teahouse Q&A page this week. Invitees who received emails were about twice as likely to participate than those who were invited by talk page message alone.
|Q&A guests (invited)||14|
|Q&A guests (not invited)||26|
|email hit rate*||3%|
|no email hit rate||1%|
*percent of guests who were emailed who actually showed up.
Guests invited per host
Rosiestep and SarahStierch invited the most guests!
|invites by host||'|
|Bill william compton||6|
Invitees per invite location
Most guests were invited through the New editor contrib page, followed by the Database report.
|invites by location||'|
|New editor contrib||610|
|AfC help desk||16|
Basic Stats - since launch
Q&A Board guests, by invite location
Looking at the proportion of invitees who actually showed up, the Database report and Feedback Dashboard seem to be the best place to find new editors to invite. This suggests that hosts should focus on inviting editors from these locations, rather than focusing so heavily on the New editor contribs page.
Many guests who were not invited by hosts showed up at the Teahouse Q&A board. Many of these guests were students who are currently studying Wikipedia and were invited to participate by their Online Ambassadors or by J-Mo. We can expect the number of student participants to decline in coming weeks.
|questioners by location||'||hit rate*|
|New editor contrib||16||3%|
|wasn't invited - came anyway||33|
* percent of people invited through that location who asked a question on the Q&A board.
March 5th report
I wanted to share some number from our first week. These are not only interesting to look at, they can help us spot trends and identify strategies that are working. I'm posting it here for now because this is where the action is, but if someone wants to make a 'reporting' page for hosts and throw it up there, that would be totally fine with me.
First, invites! Here are the hosts who invited users to Teahouse (at least those who noted it on the "Invite Tracking" spreadsheet). Congrats to Rosie and Sarah for their contribution here!
|host||# invites sent|
Next, I wanted to give some overall metrics on invitees. These are already out of date, as we've had several new questions and intros since I ran the numbers yesterday. The overall count isn't that impressive yet, but in general we're seeing a steady acceleration in the number of newbies per day.
|how many newbies were invited?||694|
|how many showed up?||30|
|how many posted questions?||18|
|how many created an intro box?||18|
|how many did both?||6|
|how many were not invited at all (or at least not tracked on the tracking sheet), but showed up anyway?||4|
As you can see, we sampled from a variety of different locations. Some locations seem offer more bang for your buck than others. In particular, the database reports and the new editor contrib filter seem like the most promising places to look for newbies to invite.
|where were the newbies found?||total invites sent||number who showed up||% who showed up|
|new editor contrib filter||335||18||5%|
|Global Ed Collab||46||0||0%|
To email or not to email?
I wanted to mention one thing I saw in the data in particular: emailing users who have the email this user feature really seems to work. Consider this: we only emailed 102 users. However, of the 30 newbies who showed up, 7 had received an email. That's almost 25%! So it looks like using the "Email this user" feature in addition to using a talk page template is a good strategy for reaching out to newbies, if they have it enabled (the database report page shows which ones do). And remember that if you use the "email this user" feature, your email will not be shown.
That's it for now. Any questions for me? How are things going in general? Any 'feature requests' (I've already got the header for the invite templates on my to-do list)?. Keep up the good work! - J-Mo 20:12, 5 March 2012 (UTC)
- I suggest a slightly different calculation:
- 102 people were emailed; 7 showed up (just under 7%)
- 592 people were contacted via user talk pages [694-102]; 23 showed up (just under 4%)
- So emails appear to be roughly twice as effective as user talk pages. Unfortunately, there are two assumptions in that conclusion that may not be valid:
- (1) New editors who enable email are similar to those who do not. (The only way to test this assumptions is to take a group of editors who have enabled email, and send half of them an email and half a user talk page invitation. It's not clear that was done. Alternatively, one could look at the 592 people contacted via user talk pages, and see what the response rate was for those with email enabled.)
- (2) Those receiving an email invitation came from seven places where newbies were found, and in the same proportion, as those getting user talk page invitations. (Why is this critical? If emails were sent, for example, only to new editors found via the database report and the new editor contrib filter, then the expected response rate would be between 5 and 7%, and getting a 7% response would not indicate that email is better. Again, a systematic approach - start with a homogenous group, say only those with email enabled and found via the new editor contrib filter, and then split it into halves, randomly - is the only way to be comfortable with the conclusion that email is better.) -- John Broughton (♫♫) 00:17, 10 March 2012 (UTC)
- Hi John, thanks for your input. I don't have a hypothesis either way about (1) or enough data to begin measuring it accurately if I did, and we definitely don't have a large enough sample yet to figure out whether or not (2) is true. There are other potential confounds too, of course. As there always are. :) I'd love to chat with you about them, if I see you at #wikipedia-teahouse. cheers, J-Mo Talk to Me Email Me 15:41, 13 March 2012 (UTC)