Contraband

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search

Contraband is a tool for Wikimedia event organizers for tracking developer activity. This tool is currently a work in progress.

Background information[edit]

Wikimedia has numerous tools to gather mw:Development statistics, one of them being Bitergia's analytics tool. This tool provides useful information and is convenient for community managers who are familiar with its every know-how. However, this tool is cumbersome to use for others as it requires too many steps to obtain statistics for a topic and there is a bit of learning curve to get comfortable with the tool. For example event's scholarship committee reviewers who need developer contribution statistics while reviewing applications for Wikimedia events juggle between different platforms like Github, Gerrit, Phabricator to view developer activity for deciding on an applicant.

Proposed workflow[edit]

The Contraband tool is currently a work in progress and aims to give a sneak peek into a developer’s contributions on Wikimedia platforms: Gerrit, Phabricator, and Github. The event organizers can log in to the app and perform three steps to retrieve the results.

  1. Organizers type in a list of users with their Wikitech/Gerrit, MediaWiki/Phabricator, and Github usernames or provide the same data in a CSV format.
  2. Organizers choose to filter the data by timestamp, the status of the commit (merged, abandoned, declined), and project name.
  3. Displays data in a tabular format with the ability to sort the data by username or activity.

Any user will be able to use this tool, but the event organizers will have slightly more advantage and will be able to authenticate and gain access to additional features such as search history, uploading the data in CSV format, etc. As of now, the plan is to allow one of the authenticated users to validate the newly registered user.

The idea for the Contraband tool is inspired from AWMD stats that generates monthly statistics of technical contributors to Wikimedia projects from Africa.

This tool will be available for use on Toolforge.

Mockups & wireframes[edit]

Mockups[edit]

a) Enter a list of usernames
b) Provide a filter
c) Display user contributions in tabular format
d) Display user contributions in graphical format

Wireframes[edit]

a) Input usernames
b) Provide advanced filters
c) Display the fetched result in a tabular format
d) Display the graphs and timeline of user contributions
e) Show User activity for a certain timeframe in tabular format

Technical implementation[edit]

Fetch Gerrit and Phabricator contributions of a user[edit]

For Gerrit contributions, all changesets, new, merged, and abandoned will be considered. For Phabricator, all tasks authored by and assigned to a user will be considered. The visualization will look as shown in mockup a).

[WIP Section] The solutions to both the above questions is not proposed by me. It is the way how wikimedia.bitergia.io gets the statistics. I am just following it’s method.

For retrieving contributions, here are some identified solutions:

ElasticSearch[edit]

Phabricator API[edit]

Request URL: https://phabricator.wikimedia.org/conduit/method/maniphest.search

Notes:

  • It is currently only possible to fetch "authored and assigned" and not "authored or assigned." For development purpose, only authored count will be considered. For production use, an API request to fetch both authored and assigned will be performed separately, and then their responses will be merged.
  • The output of the above API search will be paginated for which API requests need to be continued till all the results have been fetched.

Gerrit API[edit]

Request URL: https://gerrit.wikimedia.org/r/changes/?q=owner:rammanoj&o=DETAILED_ACCOUNTS

Request Payload: None

Note: All the above objects need to be added to get a count of all contributions.

Retrieve user contributions for different dates and times[edit]

The visualization will look as shown in mockup b).

ElasticSearch

Request URL for fetching contributions from Gerrit: GET gerrit/_search?filter_path=took,hits.total,aggregations

Request URL for Phabricator: GET maniphest/_search?filter_path=took,hits.total,aggregations. For Phabricator and Gerrit, same API Calls will be performed via the following APIs:

Display user activity in a tabular format[edit]

The visualization will look as shown in mockup e).

ElasticSearch

Request URL: GET gerrit/_search

Request URL for Phabricator: GET maniphest/_search?filter_path=took,hits.total,aggregations. For Phabricator and Gerrit, same API Calls will be performed via the following APIs:

Benefits of using ElasticSearch over Gerrit and Phabricator APIs[edit]

With Phabricator API, there is a limit of 100 objects per call that cannot be performed in a parallel manner. With Bitergia contribution count can be easily fetched, the response is retrieved much quickly and that too with a single API request. This would also allow displaying data to the user in realtime.

Contact[edit]

This tool is being developed by Rammanojpotla as part of Google Summer of Code 2019 under the mentorship of Suchakra Sharma and Srishti sethi. Keep an eye on the technical updates on Phabricator: https://phabricator.wikimedia.org/T220254.