Community Tech/Development

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search

This is a page for Community Tech developers to share information about coding conventions, workflows, etc. This is intended to supplement the manual on MediaWiki.org, and applies to all of the projects listed on Community Tech/Maintenance.

Development workflow[edit]

This section describes our development workflow, based around our sprint board on Phabricator and code review on both Gerrit and Github. Once a ticket is assigned to someone, it stays with them until it's closed.

Working on a ticket:[edit]

  1. Before taking a new ticket to work on, make sure that no progress can be made on anything you're already working on.
  2. Take a ticket from the top of the Ready column (this column is manually kept arranged in priority order).
  3. Assign it to yourself, and move it to the In Development column.
  4. Work on the ticket, and upload patches to Gerrit or Github as appropriate. When a patch is ready for review, make sure there's a link to it on the ticket (Gerrit does this automatically; you have to do it manually for Github; and in both cases it might be good to make a comment about the patch), and move the ticket to the Needs Review/Feedback column.
  5. After receiving feedback that suggests changes, move the ticket back to In Development (even if you're not going to be working on it right away; this makes it obvious to other reviewers that they don't need to look at this yet). (See below for the process for a patch that passes review and is merged.)

Reviewing others' work:[edit]

  1. Work down the Needs Review/Feedback column from top to bottom.
  2. For each ticket, do whatever is required to review the patch (see mw:Gerrit/Code review for more about this process; that page is about Gerrit, but we apply similar principles on Github).
    • If the patch is not ready for merging, leave comments and mark it as such (this is a similar process on Gerrit and Github).
    • If it is ready, merge it (and for Github patches, leave a comment on the ticket)
    • The assignee of the ticket will move the ticket to the Product/QA review column.
    • The assignee is responsible for getting the change deployed to the relevant environment(s) for QA.

Communication expectations[edit]

The Phabricator ticket should be updated in any of the following situations:

  • A Pull Request is created on Github (note that no comment is required for Gerrit patches because Gerrit-bot will post on the ticket automatically).
  • When the ticket moves to QA there needs to be an explicit explanation about how the ticket can be QA'd (i.e. where to find it in staging, production, etc.).
  • When work for the ticket has been deployed a comment should be added to specify that.

In addition, the CHANGELOG.md file should be updated (following the guidelines of keepachangelog.com), listing anything added, changed, deprecated, removed, fixed, or that are security vulnerabilities. Merge conflicts can be resolved by reviewers when required (rather than waiting for the patch author to do it).

Github workflow[edit]

Many tools and bots are hosted on Github, outside of the WMF's Gerrit patch review system. The intent of this process is to enable work on projects hosted on GitHub to be reviewed and merged quickly when colleagues may not be receiving notifications automatically, as they would through the integrated Phabricator/Gerrit/git review system.

Our Github projects should have the following tags: WIP (#cc317c) Ready for review (#33aa3f).

Process[edit]

  • Developer forks the project in question.
  • Developer may handle branches as they like, although they should strongly consider starting a branch for each task worked on with a descriptive name.
  • When work is complete, submit a Pull Request to the main repository.
  • After all Continuous Integration tests pass for the PR, add relevant people as reviewers.
  • Each PR should contain a single commit, amended as many times as is required (this is to match the Gerrit model, and results in a clean history for the repository).
  • After a PR is merged, the person who merged it is responsible for making a comment on the Phabricator ticket.
  • The process for ticket management should be paid attention to as automatic notification between Github and Phabricator don't yet work. We have to rely on humans to communicate with each other on Phabricator to ensure the status is correctly reflected.

Tools Deployment workflow[edit]

The Community Tech team works on several tools that live outside Wikimedia's general deployment environment, like Wikimedia Cloud Services. These tools require having a separate deployment workflow. Each tool has two accounts: the main "production" instance, and a 2nd "staging" account of the same name with (usually) -test appended.

For tools whose code is hosted on Github, the deployment process is:

  1. "Staging" instance is kept up to date with the current master branch. The latest code is pulled into the staging instance as soon as a PR is merged (by a cronjob running every 10 minutes).
  2. When the state of staging instance is ready (post QA and Product Review) we "cut" a release:
    1. Add an annotated version tag to the repo (in the SemVer format of major.minor.patch)
    2. A script running on production will automatically update to the latest version — updating the production instance

Expectations from investigation tickets[edit]

Investigations done by Engineers should cover the requested topic and lead towards some actionable direction; either a solution proposed, or a conclusion about the feasibility of the feature investigated.

Expected preliminary results[edit]

Until this stage, the ticket is "In progress". When the preliminary results are written, it should move to "In Review"

Preliminary results should include:

  • Summary of the investigation itself; what was checked, what should be considered, the challenges taken into consideration, etc.
  • Conclusion of the preliminary results
  • Available options to take forward
  • If possible -- a recommendation for a next step.

After the preliminary results are posted, the team can review the results and discuss on the ticket. Once a decision of next-steps is made, it should be documented.

Expected next steps[edit]

At this stage, a new ticket should be created, and the investigation ticket is resolved.

  • When an investigation is done and some agreement is made, it is to be summarized
  • A new actionable ticket should be created (and linked from the investigation)
  • The results (along with a brief summary) should be copied into the investigation ticket's description so it is not lost in comments.