Meta:Training/Wiki Metrics/Training feedback

LauraHale (talk)

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

What did you find most helpful?

Link to metric page.

What, if anything, was confusing for you?

Looking at the sample for project name space and seeing just en listed on the sample. It was confusing. I had to search and think how English Wikinews should be processed given this is not consistent from one thing to another. (Stats.grok.se uses en.n for example and that was my first guess.)

(Response: Thanks for your feedback, the namespace numbers do vary across projects and the links provided there are to serve as a reminder to be conscious in your report requests to indicate namespace numbers specific to your cohorts project(s). Unfortunately, I do not think those tables are necessarily available across all projects, and while some are consistent, like namespace 0, others are totally different. I will bring it up as a potential UI improvement area, but most likely, due to the need for a table for each project, this will not change any time soon as there is a backlog of priorities they are still working toward, but they may be able to eventually figure out something more helpful for quicker reference JAnstee (WMF) (talk) 19:06, 4 September 2013 (UTC))

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

More clear understanding this was a beta, and this was designed mostly for English Wikipedia. I 100% get this is beta and very potentially cool but seeing only the two metrics made it feel like data analysis that is important was missing. :(

(Note: This is not the original beta version that they are building from, it is thus only "beta" in it's extension to the "program leaders" user group, rather. It is being released piecemeal as they build in the functions from the beta system "User Metrics API" to allow our user group to have access as soon as possible rather than waiting for the full system capacity - you can very much expect to watch it grow each week, check now and you'll see already the pages created metric as well as the start and end date parameter options for each metric; and most likely, before next week, you will also see the survival metric. Also, while the system defaults to en wiki, it was very much designed for all projects. However, program leaders are a new user group that we are purposely developing the system toward now, and once the full beta functions are built into WikiMetrics we will continue to grow the metrics available for user cohorts. What I do notice about your metric interests are that they are content metrics rather than user metrics, and that area, while an important aim, was not what the User Metrics API model was built for, and thus, those expansions will take longer to be identified, designed, and slated for development - it will be important that program leaders share those metric interests with the PE&D team so that we can best represent those interests in the ongoing development and prioritization of features JAnstee (WMF) (talk) 19:06, 4 September 2013 (UTC))

Would you recommend this training module to other program leaders?

Not at this point because the tool does not feel developed enough yet to provide useful data. When I opened the csv output after having imported four cohorts, I had no idea which cohort I was actually looking at as the output did not give me any clues regarding that.

Any other comments?

It is a very good start.

Bdcousineau (talk)

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

What did you find most helpful?

That you took the time to do this at all, thanks! It gives me lots to think about going forward.

What, if anything, was confusing for you?

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

Did I miss the part about how the Wikimedia Foundation will use this information?

Would you recommend this training module to other program leaders?

Yup.

Any other comments?

Here's hoping this works for those of us just beginning to build our community. I'm at a loss how to follow up with new editors who take an editing training and then disappear. Perhaps it's a momentum thing, and in fact my geographic region is still at the "birth-stage".

Even though this region is unaffiliated, it seems useful to do this just to get the skills.

Thanks for the training!

Kippelboy (talk)

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

What did you find most helpful?

Videos were useful

What, if anything, was confusing for you?

• I expected to see a visual report, not a CSV output file

(Response: No automated visual reporting is planned yet for the WikiMetrics, but the csv and JSON will function with the LIMN to do such visualization of the data and we will most likely be working toward that option JAnstee (WMF) (talk) 18:49, 4 September 2013 (UTC))

• At the moment, It doesn't work for non English Wikis. I tried to upload a cohort from cawiki and system told me: Invalid Username. When I did the same for enwiki, it worked (I don't have a bugzilla account, yet)

(Response: This should not be the case, did you check that those are the correct usernames in the cawiki as well as enwiki - sometimes different usernames are registered for different projects, also, they have found problems with underscores and trailing spaces that invalidate usernames, but that does not seem like it would be if you used the same csv file for both cohort creations - were all user indicated as invalid? Please let me know these further details so that I may pass along from here or on the mailing list under support tab within WikiMetrics. JAnstee (WMF) (talk) 18:49, 4 September 2013 (UTC))

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

The Wikipedia:Namespace numbers, probably including the table directly in the Wikimedtrics tools would be much easier. I didn't understood at first time what number was I supposed to write until i checked the link.

(Response: Thanks for your feedback, the namespace numbers do vary across projects and the links provided there are to serve as a reminder to be conscious in your report requests to indicate namespace numbers specific to your cohorts project(s). Unfortunately, I do not think those tables are necessarily available across all projects, but some are consistent, like namespace 0, while others are totally different. I will bring it up as a potential UI improvement area, but most likely, due to the need for a table for each project, this will not change any time soon as there is a backlog of priorities they are still working toward, but they may be able to eventually figure out something more helpful for quicker reference JAnstee (WMF) (talk) 18:49, 4 September 2013 (UTC))

Would you recommend this training module to other program leaders?

Yes.

Any other comments?

Is good to start soft with a selected group of people.

Daria Cybulska (WMUK) (talk)

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

What did you find most helpful?

Explanations of how to format usernames was particularly clear

What, if anything, was confusing for you?

How editable a cohort is. Can I add to it if another participant sends me his username later? Can I add another namespace to be analysed if I find out the workshop focused on different language Wikipedias?

(Response:Currently, there are no cohort management functions implemented but it is in the plan to allow you to add or remove cohort members, not sure about reassigning project to an unspecified username list. Namespaces may be specified in each report request, as many as apply within a particular project, separated by commas.JAnstee (WMF) (talk) 18:38, 4 September 2013 (UTC))

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

How to understand the results you are getting

Who has access to that data (Answer: Only you for now, eventually, WMF will also along with possible optional sharing capability within country/chapters JAnstee (WMF) (talk) 18:38, 4 September 2013 (UTC))

Would you recommend this training module to other program leaders?

Yes if they are intending on using the Wiki Metrics

Any other comments?

A couple of comments on the Wiki Metrics itself:

• On the Cohort description page, it would help to have buttons with explanations on what is meant by 'Name', 'Default project' etc - so that one does not have to refer back to the training page, but can have a reminder within the Wiki Metrics

(Response: Because we are only one use case for programs, this does not make sense to include in the system itself necessarily, but I will ask if the basic model reference for reminder, could be included as an example perhaps JAnstee (WMF) (talk) 18:38, 4 September 2013 (UTC))

• Pedantic - if the cohorts are organised alphabetically, it would be easier to name them year-month-day first and then the rest, so they get ordered chronologically. I can just picture a bit of a messy list once there are a lot of cohorts entered.

(Response: I think that the order of the name components may be flexible, just need to have the identity components - we should discuss this further with the larger user group JAnstee (WMF) (talk) 18:38, 4 September 2013 (UTC))

• Can't I change the default project for the cohort once I set it up? E.g. I want to run a report to see if the attendees have been editing pages in a different project - seems I need to go through the whole cohort creation process again?

(Response: Yes, you have to create a new cohort for each project as it is now - this is why I recommend not specifying project in csv but waiting until cohort creation so that you can use the same csv file at least JAnstee (WMF) (talk) 18:38, 4 September 2013 (UTC))

• It's a good training - would make sense to have a link to it from the 'support' tab within Wiki Metrics page

(Good idea for sure JAnstee (WMF) (talk) 18:38, 4 September 2013 (UTC))

Note: Thanks for your specific feedback. I hope you can join our next hangout planned for tomorrow JAnstee (WMF) (talk) 18:38, 4 September 2013 (UTC)

OR drohowa (talk)

I was searching for information on tracking Wikimedia Commons uploads. Is there such a function on Metrics?

I also think more info on the CSV file with usernames would be helpful for people.

Susannaanas (talk)

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

Note: The comments are also on the Metrics itself and the PED evaluation requirements.

What did you find most helpful?

1. The existence of the training
2. Step by step instructions
3. Illustrations

What, if anything, was confusing for you?

...

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

This did not help me track the survival of editors. I might have understood the concept in the end, but I wasn't able to complete the tasks I needed to.

1. Timespan from a specified point in time
2. User-friendly input of time
3. Get rid of sunsets unless they are illustrative. For me they weren't.

Would you recommend this training module to other program leaders?

Definitely

Any other comments?

Please synchronize reports with reporting needs. If editor retention is required in moths, have an input box for months. Or have the user upload a cohort and timespan & select a report and the Metrics system would do the rest: create the lists of new editors, pages created, pages edited, images uploaded, editor retention. It is insane to gather the data by manually clicking pages and gathering the data with pen and paper.

I think naming the cohorts in this manner is systematic, but should be made automatically, if such accuracy is required:

1. Select event type
2. Select project
3. Input location
4. Insert date

Liz (talk)

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

What did you find most helpful?

Well, it was awfully brief.

What, if anything, was confusing for you?

"Opt-in at in-person events is required" - I don't this is standard ethnographic practice. You need to inform people that you are collecting their data (yes, even usernames) for use in an educational project...or if not educational, one that will help Editors.

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

Again, it was too short. It could have gone into more depth by providing examples, for instance.

Would you recommend this training module to other program leaders?

Well, as a first step .

Any other comments?

I think you should do more of these and more detailed.

Beat Estermann (talk)

Any other comments?

Concerning the usability of the tool (and not necessarily the instructions):

• Monthly time series didn't work for me (the numbers in the output were incorrect)
• There is no indication as to how to produce reports across several Wikipedia projects (different language versions, Commons, Wikidata) - does this functionality exist?
• The indication of the namespaces should be explained in a more obvious way - not just by providing a few links.

Thanks for your feedback Beat. I'm just going to provide some quick answers, as our team is knee deep in a report right now.

1. The issues with monthly times series is something you should report to analytics. The Wiki Metrics site has a link to reporting bugs (or just go through Bugzilla) for that. Sadly that's out of control of Jaime and myself.
2. You can't produce cross-project reports at this time. (Not sure if we ever will be able to.)
3. Thanks for the final tip, we will be redoing the training in the near future and will use your input. SarahStierch (talk) 21:42, 19 November 2013 (UTC)

92.229.93.196

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

What did you find most helpful?

...

What, if anything, was confusing for you?

...

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

...

Would you recommend this training module to other program leaders?

...

Any other comments?

...

Liang (WMTW) (talk)

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

What did you find most helpful?

I'm glad to know there is a quantity-based analysis that I can join.

What, if anything, was confusing for you?

Nothing

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

Some examples / real case to show that collecting these data really make the projects improved.

Would you recommend this training module to other program leaders?

Yes, after I translated this into Chinese.

Any other comments?

Interesting, thank you.

Alex Wang (WMF) (talk)

Note: It is your choice to answer any of the following question prompts, either anonymously, or with your name. If you are logged in, you will be known, if you are logged out, you will be anonymous.

What did you find most helpful?

Very clear process instructions.

What, if anything, was confusing for you?

Still not totally clear on the terminology.

What, if anything, did you feel was missing from this overview for your orientation to Wiki Metrics?

Some questions that came up:

1. Is there any benefit to using usernames vs. user ids? or vice versa?
2. To clarify, if the same users are editing multiple projects, you need to create a cvs file for each project?
3. If you choose JSON for your output, how long does that link stay good for?

Would you recommend this training module to other program leaders?

Yes, definitely!

Any other comments?

Hi Alex Wang (WMF)! - Thanks so much for your questions/comments. Here are answers to your questions below. I've also gone ahead and made a few changes to the training so these are more clear:

#Is there any benefit to using usernames vs. user ids? or vice versa?

The main benefit is that user ids use only numbers, which makes it easier to locate a user in the database. Symbols can be sometimes tricky for the system to read, but Dan from analytics says that they've fixed this.

#To clarify, if the same users are editing multiple projects, you need to create a cvs file for each project?

No. You create one CSV per cohort, but you add the project name to the cohort name on wikimetrics. (not the filename). I think I've made this a bit more clear in the training.

#If you choose JSON for your output, how long does that link stay good for?

30 days. I've made this edit to the training so its more clear

--EGalvez (WMF) (talk) 17:44, 6 March 2014 (UTC)

...