IRC office hours/Office hours 2014-06-18

Chat on MediaWiki Release Management RFP: Q&A with applicants
18 June 2014
2000 - 2100 UTC
01:11 <greg-g> #startmeeting MediaWiki Release Management RFP IRC Office Hour (Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE | | https://meta.wikimedia.org/wiki/IRC_office_hours))

01:11 <wm-labs-meetbot> Meeting started Wed Jun 18 20:01:11 2014 UTC and is due to finish in 60 minutes.  The chair is greg-g. Information about MeetBot at http://wiki.debian.org/MeetBot.

01:11 <wm-labs-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.

01:11 <wm-labs-meetbot> The meeting name has been set to 'mediawiki_release_management_rfp_irc_office_hour__channel_is_logged_and_publicly_posted__do_not_remove_this_note_____https___meta_wikimedia_org_wiki_irc_office_hours__'

01:17 <greg-g> #topic intro

01:24 <greg-g> Hello and welcome to the IRC discussion with the two MediaWiki release management applicants.

01:31 <greg-g> There are two applicants this time around. "Mark y Markus" and "The Consortium".

01:35 <greg-g> #link https://www.mediawiki.org/wiki/Release_Management_RFP/2014/Mark_y_Markus_LLC

01:38 <greg-g> and

01:43 <greg-g> #link https://www.mediawiki.org/wiki/Release_Management_RFP/2014/Consortium

01:50 * greg-g waits for the bot to catch up?

02:03 <greg-g> maybe it's silent on links

02:04 <greg-g> moving on

02:14 <greg-g> I hope everyone has had a chance to read through the proposals before this meeting. :)

02:37 <greg-g> btw, who here is here to ask questions (ie: not part of either applicantion)?

02:45 * mwalker raises hand

02:51 * greg-g knows many will lurk and energize when a topic that interests them comes up

03:11 * legoktm has questions too

03:14 <greg-g> cool

03:32 <greg-g> Then let's get going with the M&M proposal, want to have as much time for questions as possible

03:36 <greg-g> #topic Mark y Markus

03:40 <greg-g> First, hexmode and mglaser, please introduce yourselves.

04:06 <hexmode> I'll go

04:27 <hexmode> I've been active in the technical community since 2010

04:46 <hexmode> as WMF and afterwards working with individuals and companies

05:04 <hexmode> in the past year I've worked with mglaser on release management

05:16 <greg-g> (for the record, you can both introduce yourselves at the same time, to save time, this goes doubly for "The Consortium" since there's more of them ;) )

05:23 <mglaser> k

05:42 <mglaser> I'm working with wikis in enterprises since 2007

05:49 <mglaser> MediaWikis

06:18 <mglaser> Since 2010 I'm involved with Wikimedia

06:58 <greg-g> (let me know when you're done)

07:02 <mglaser> There are some extensions I maintain(ed): BlueSpice, Windows Azure Storage, a Windows package for MediaWiki

07:05 <mglaser> and others

07:07 <hexmode> I don't know what else to say here except to point you to the rfp.

07:11 <greg-g> :)

07:23 <hexmode> (last years has more info about me)

07:24 <mglaser> with hexmode, I did the releases of MediaWiki in the last year

07:25 <greg-g> that's perfect, just wanted to give you a moment to give context of who you are :)

07:51 <mglaser> More details about both of us can be found in the proposal

07:53 <mglaser> so: done

07:57 <hexmode> :)

07:57 <greg-g> awesome, ok, well, let's get on with the questions

08:00 <greg-g> Questions for them?

08:16 <greg-g> (I have some if no one speaks up ;) )

08:25 <hexmode> Maybe start us off?

08:33 <mwalker> you address tarball releases in your rfp; but I'm curious about any work you're doing with packaging

08:54 <hexmode> mwalker: I've been in touch with Debian and Fedora devs

08:56 <mwalker> specifically; the ohter proposal says they're going to work on install instructions for parsoid and other popular but complicated things

09:13 <hexmode> coordinating with them about security releases

09:27 <mglaser> mwalker, there's a lot of discussion going on about which distribution formats to use or not

09:48 <mglaser> e.g. vagrant, debian packages or installer scripts

10:00 <hexmode> mwalker: yeah, and parsoid does need to be packaged.  I started on an RPM for a client (not this work) but haven't returned to it

10:27 <mglaser> We think it is important to keep mediawiki easy enough to installl that even people that are not sysops can test and use it

10:28 <hexmode> mwalker: I will soon, though, because $CLIENT wants to deploy VE in the next couple of monnths

10:36 <greg-g> so, "it's complicated"? ;)

11:00 <hexmode> greg-g: packaging is targeting to larger, non-shell users

11:02 <mglaser> to that end, I had a lot of discussions, e.g. with Gabriel Wicke, about a proper packaging format / or fprmats

11:10 <mwalker> but; do you see that effort being mostly on your side of things? or is that a WMF responsibility?

11:15 <hexmode> people who can maintain their own infrastructureer

11:28 <hexmode> mwalker: WMF already has a lot of packaging they do

11:32 <mglaser> greg-g, yeah, it will become complicated as more components come in

11:35 <mglaser> such as parsoid

11:35 * greg-g nods

11:40 <hexmode> mwalker: we'll reuse wher appropiratew

12:06 <hexmode> mwalker: but a lot more needs to be done to make packaging work for non-WMF orgs

12:13 <mglaser> mwalker, I think its a joint effort

12:30 <hexmode> +1

12:44 <hexmode> Ori started mw-vagrant, for example

12:58 * YuviPanda isn't sure vagrant can be considered similar to a 'package' of any sort, though

13:00 <hexmode> I'm trying to build on that for Redhat installations

13:17 <mwalker> there's also a difference between taking the lead (like ori did) or us following along and helping

13:22 <greg-g> to throw in a potential curve ball, we know from experience that what the WMF uses to deploy to hundreds of servers is not what someone else will use on a shared host or even on a 5 vps 'cluster', who will take the torch and be the point person for making sure that use case is addressed effectively?

13:29 <hexmode> YuviPanda: right, but it can be thought of as a start to deploying

13:38 <mglaser> YuviPanda it's more about how to get MediaWiki to the people who want to use it

14:06 <mglaser> If vagrant is a viable way of making it easy to start off with a MediaWiki, that's one way.

14:10 <bawolff> So in last year's rfp - https://www.mediawiki.org/wiki/Release_Management_RFP/2013/NicheWork_and_Hallo_Welt!#Problems_with_MediaWiki_release_managment you listed a whole bunch of problems and "initial improvements" you wanted to make. How'd that go?

14:20 <YuviPanda> hmm, right. but vagrant, as it is now, is very much a dev environment. lots of things that'd make sense in a production environment are turned off, and lots of things that are terrible ideas in a prod environment are turned on...

14:21 <mglaser> I feel we will not have "the one and holy technology", though

14:26 <gwicke> greg-g: that needs to be considered early on in the design process; it's not something you can bolt on easily

14:35 <hexmode> greg-g: the release team on this contract should take the lead of adapting the work to non-wmf users

14:55 <greg-g> gwicke: so are you volunteering? ;)

15:00 <mglaser> YuviPanda, you're right there. That's why we have not made a final decision on this

15:08 <gwicke> we are giving that a lot of thought already

15:28 <YuviPanda> mglaser: do the maintainers of mw-vagrant know that you guys are potentially considering it as a starting point for deployment/packaging?

15:45 <YuviPanda> I'm also not sure how that'll work, though. 'MW as a virtual appliance'?

15:45 <hexmode> YuviPanda: absolutely, but using the vagrant work to iterate would be better than going from scratch

15:46 <greg-g> to gwicke but generally: right, but something like this needs a champion or it'll just be forgotten

16:06 <gwicke> greg-g: we have it in our goals for the service team

16:20 <hexmode> YuviPanda: packaging, no. deployment, yes

16:21 <greg-g> hexmode: mglaser ^^

16:23 <gwicke> this includes a) packaging and b) design to scale down

16:35 * greg-g nods

16:38 <hexmode> \o/

16:47 <greg-g> ok, so it seems like a team effort with gwicke doing much of the torch holding

16:48 <hexmode> I like that

16:59 <greg-g> moving on :)

17:03 <YuviPanda> hexmode: ah, hmm. makes sense. do make sure that people like ori and bd808 (and perhaps me?) are in the loop though, whatever you guys decide to build on top of vagrant

17:06 <greg-g> In the budget section (thanks for that! it's good to have that break down), you have a line for "advocate third-party interests" at 4 hours/week. That one seems very nebulous (there is no definitive definition of what what entails, exactly, in the proposal); can you explain a bit about what you see fitting into that line item vs the first three line items in the "cost of organising (sic ;) ) a user group" section?

17:19 <legoktm> [01:14:11 PM]  <bawolff>	 So in last year's rfp - https://www.mediawiki.org/wiki/Release_Management_RFP/2013/NicheWork_and_Hallo_Welt!#Problems_with_MediaWiki_release_managment you listed a whole bunch of problems and "initial improvements" you wanted to make. How'd that go? <-- I'd also like to hear an answer to this.

17:22 <gwicke> something like packaging needs to be a wmf-wide effort

17:36 <gwicke> we'll focus on service packaging, partly driven by our own needs as well

17:37 <mglaser> bawolff, legoktm : we addressed this in our proposal

17:42 <greg-g> ok, bawolff's question first if you haven't already started answering

17:52 <mglaser> Maintainn two major releases per year: done

18:02 <gwicke> I haven't heard much from platform on core packaging however

18:13 <greg-g> completed vs ongoing work sections from this year's RFP

18:21 <gwicke> or is this something the release team would like to tackle?

18:34 <mglaser> Continuous integration: We have integrated the make release script with Jenkins, it's triggered on a git tag

18:44 <hexmode> greg-g: yeah, that section is meant to address bawolff's q.

18:45 <mglaser> tarballs are automatically built and tested

18:50 <greg-g> gwicke: let's not get bogged down in this topic, but yes, A) platform will respond work on that with you and B) release team will also be involved

19:05 <hexmode> bawolff: is there something in the 12 month recap you want more info on?

19:22 <mglaser> Work with extension developers: Reached out to them at various conferences, SMWCon, Hackathons, Wikimania, etc

19:25 <gwicke> greg-g: that's good to hear

19:59 <bawolff> More just a status report. There's a big list of things, what areas didn't work out, and why? What have you learned trying to do those things. What are you planning to do differently should you get the contract again?

20:01 <bawolff> etc

20:01 <mglaser> Lasting relationships with Open Source organisations: Mark is in touch with Mozilla and Debian packagers, and others afaik

20:34 <greg-g> I have 3 questions related to bawolff's:

20:35 <greg-g> What were the hardest things you had to do this past year? How will you make them easier?

20:38 <greg-g> • What were the easiest things you did this past year?

20:40 <greg-g> • What was the biggest surprise from this past year?

20:44 <legoktm> mglaser: well you listed 4 bullet points on https://www.mediawiki.org/wiki/Release_Management_RFP/2013/NicheWork_and_Hallo_Welt!#Problems_with_MediaWiki_release_managment (what bawolff linked), and I'm trying to figure out which of those you guys made progress on.

20:51 <hexmode> bawolff: I'll try to do quick bullet points for you after greg-g's

21:16 <hexmode> greg-g: easiest -- writing emails about releases

21:35 <greg-g> :)

22:04 <mglaser> greg-g: easiest: go to conferences and talk to third party users/developers

22:14 <mglaser> (because I like this)

22:23 <greg-g> :)

22:28 <hexmode> greg-g: biggest surprise for me -- how much work needed to be done on getting people we needed done

22:43 <hexmode> ex: waiting on the tarball server

22:46 <mglaser> hardest: my first release, when I found that I have about half of the permissions needed.

22:54 <mglaser> (now resolved)

23:47 * greg-g nods

23:54 * legoktm repeats self...

23:56 <legoktm> [01:20:45 PM]  <legoktm>	 mglaser: well you listed 4 bullet points on https://www.mediawiki.org/wiki/Release_Management_RFP/2013/NicheWork_and_Hallo_Welt!#Problems_with_MediaWiki_release_managment (what bawolff linked), and I'm trying to figure out which of those you guys made progress on.

24:03 <hexmode> items on "problems list" from last year to follow

24:36 <hexmode> "Skinning sucks" -- it still sucks.  Not really a release mgmt issue, but still something we want to fix

25:20 <gwicke> greg-g: sorry to drag you back to packaging: so who will take responsibility for packaging core?

25:27 <bawolff> So you mentioned last year: "Our first attempt at fundraising will be done with Kickstarter to fund the development of a better skin system for MediaWiki. We'll set up a skin exchange and publicize it as well."

25:30 <greg-g> gwicke: off-topic for now, please

25:31 <bawolff> did that happen?

25:37 <hexmode> "Web-based config" -- still needed.  Much more of a release issue.  No significant progress b/c of the issues with actually putting out a release reliably.

26:09 <mglaser> legoktm, these are issues that should be addressed by a functioning 3rd party community. In the first year, we were mainly focussed on the actual release process, to make this as tight as possible.

26:10 <greg-g> bawolff: no

26:36 <hexmode> "spam" -- still an issue.  Not much work, but SimpleAntiSpam is included now so only started.

26:37 <greg-g> bawolff: 'it's complicated' given trademarks and such

26:49 <greg-g> so, 6 minute warning

26:50 <gwicke> greg-g: so is this a retrospective only?

26:53 <mglaser> as hexmode said, these are still issues and will continue to be unless we have a functioning third party community. So that's what we want to focus on this year.

26:59 <greg-g> gwicke: no, but we have only 6 minutes left and I have other questions

27:09 <bawolff> greg-g: What does trademarks have to do with it? There's no need for it to be officially Wikimedia branded

27:09 <greg-g> we can't bogart this topic everywhere

27:12 <legoktm> hexmode: "Not really a release mgmt issue, but still something we want to fix" <-- A bit confused. Last year you put it under a heading "Problems with MediaWiki release managment", but now its not?

27:45 <gwicke> at some point we should try to come up with a coherent distribution strategy

27:45 <hexmode> "Help system" -- some work has been done on mw.o that could be used, but none to put in tarball yet.

27:52 <greg-g> ok, it seems to be getting a bit nit-picky on what things were done/how much/and what category they're in, but, let's go bigger picture for the last 4 minutes

27:59 <greg-g> gwicke: yes, this is not the venue

28:08 <greg-g> ok, reset discussion....

28:10 <hexmode> legoktm: yes, my view changed

28:17 <mglaser> bawolff: its hard to raise money for MEdiaWiki development when you can't use the word MediaWiki

28:22 <greg-g> mwalker: go ahead

28:29 <bawolff> oh right, that's trademarked too

28:31 <legoktm> ok, thanks for clarifying

28:41 <bawolff> yeah, I could see that being an issue

28:46 <gwicke> greg-g: I think it should be very much part of the release process

29:10 <greg-g> gwicke: we both agree, but there are other issues to talk about now, sorry, end of discussion on it (for now, not for ever) we are time boxed here

29:34 <hexmode> 1m?

29:36 <mglaser> Marybelle, can we continue the budget question on MW.o?

29:41 <greg-g> I thought mwalker had one

29:47 <mwalker> you requested 75k last year, with three major focus areas. As an end user, I only saw improvement in the release process -- but can you describe the work you've done for supporting mediawiki users?

30:16 <mwalker> e.g. did the funds / time actually get split evenly between all your tasks that you were going to focus on?

30:31 <hexmode> mwalker: we underestimated how much time release management would take

30:44 <mglaser> mwalker: the release took more time than we initially thought

31:07 <greg-g> can you give an estimated hours/week it took?

31:20 <greg-g> (The Consortium, we'll start at the end of this question)

31:30 <mglaser> ?

31:32 <mglaser> sorry

31:35 <hexmode> 24 total is on the low end

31:43 <greg-g> 24hour each week?

31:47 <greg-g> s

32:05 <greg-g> (to clarify)

32:10 <mglaser> Producing a tarball takes about 4-8 hours.

32:17 <hexmode> yes, I think that is a conservative amount considering all the communication, etc

32:19 <mglaser> backporting not included

32:29 <mglaser> there's stuff around this.

32:37 <mglaser> such as automation

32:41 <mglaser> additional testing

32:45 <hexmode> but

32:55 <hexmode> now that we have added automation

33:02 <hexmode> we expect it to take less

33:08 <mglaser> +1

33:20 <hexmode> still, there are things that we don't have automated

33:27 <hexmode> e.g. signing tarballs

33:40 <greg-g> so, on a release week, you spent 30+ hours on just that? (given the probable fluctuation, other weeks being shorter)

33:57 <Vulpix> hexmode: and branching extensions on new major releases https://bugzilla.wikimedia.org/show_bug.cgi?id=64157

34:14 <mglaser> greg-g, no

34:40 <mglaser> it depends on the amount of backports

34:51 <mglaser> I'd say 16 hhrs

35:02 <mglaser> average

35:04 <greg-g> mobile-reportcard.wmflabs.org

35:05 <greg-g> er

35:09 <greg-g> 16:31 <   hexmode> 24 total is on the low end

35:14 <greg-g> (bad paste)

35:18 <hexmode> also making sure we pick up bug reports from, say, support desk, etc

35:37 <greg-g> cool, ok, that helps

35:47 <greg-g> so, let's continue this thread on that talk page that Marybelle started

35:51 <greg-g> thanks hexmode and mglaser !

35:55 <hexmode> :)

35:57 <hexmode> yw

36:02 <greg-g> I'm going to push us on to the next group, sorry :)

36:07 <mglaser> thanks greg-g and everyone in this discussion

36:13 <greg-g> #topic The Consortium

36:20 <greg-g> Those members of The Consortium present, please do introduce yoursevles (concurrently).

36:28 <Skizzerz> Hi

36:28 <Emufarmers> Hi!

36:32 <MatmaRex> hi

36:34 <ashley> rar!

36:40 <greg-g> (one-liner intros probably)

36:45 <Skizzerz> (Isarra isn't with us today)

36:51 <greg-g> nicks to names are useful ;)

36:54 <Qcoder00> Agenda?

37:02 <Emufarmers> I'm Emufarmers (Benjamin Lees).  I've been working with MediaWiki and providing support to people using it for...a while now.

37:45 <MatmaRex> i'm a wikipedian since 2007 (but i've always been focusing on technical matters rather than writing the articles), and a mediawiki developer for about two years now, with +2 rights. (Bartosz Dziewoński here.)

37:47 <Skizzerz> I'm Skizzerz (Ryan Schmidt), I started working with mediawiki in late 2007, developing custom extensions and providing a ton of end user support. A short list of things I did are available on the RFP as well as my mw.org user page

38:10 <ashley> I'm Jack Phoenix, I do stuff and have been doing for a fair while; more info can be found on my MW.org page for those interested in my areas of expertise, extensions/skins I've developed and so on: https://www.mediawiki.org/wiki/User:Jack_Phoenix

38:21 <greg-g> cool, thanks

38:27 <greg-g> alright, questions?

38:34 <greg-g> I have one to start...

38:39 <greg-g> In your RFP, I felt like the word "we" was being used interchangeably to mean "The Consortium members" and "third-party users/sys-admins/devs". Example:

38:44 <greg-g> (in the "For system administrators" section) "In order to serve our user communities, we need:" vs "Schedules should be consistent so we can plan our own upgrades accordingly."

38:44 <Vulpix> what is "The Consortium"?

38:59 <greg-g> is that intentional or otherwise?

39:04 <Skizzerz> Vulpix: what we are calling the 5 of us submitting the RFP

39:20 <gwicke> https://www.mediawiki.org/wiki/Release_Management_RFP/2014/Consortium

39:26 <greg-g> thanks gwicke :)

39:40 <Vulpix> thanks Skizzerz

39:40 <greg-g> #link https://www.mediawiki.org/wiki/Release_Management_RFP/2014/Consortium

39:47 <Skizzerz> probably accidental. The first example refers to us as the team, the latter seems to refer to end users (which does include us since all of us are involved with 3rd party wikis to some extent)

39:50 <mwalker> if I can jump on greg's question -- we sort of have scheduled releases already; the last thurs of every month -- how will you improve

40:25 <mwalker> or are you just identifying that it's something you'll need to keep doing?

41:13 <greg-g> Skizzerz: to continue that thread, I like to be pedantic at times (we're all wikimedians in our own little ways), and I wonder if that will be confusing when your advocating on behalf of third-parties vs what 'the consortium as contracter' needs

41:14 <MatmaRex> mwalker: yes, the current schedule is nice, assuming it is kept. we're probably going to continue doing this.

41:36 <greg-g> s/your/you're/ #pedanticfail

42:38 <gwicke> I have another question in case this one is answered

42:44 <Skizzerz> On the flip side, if a single month did not have any significant number of bugfixes that were deemed good to backport, we may hold off for that month, as upgrading costs 3rd party sysadmins time and energy, and if we are delivering an update with only middling returns, they may decide to skip it entirely (in which case, what is the point?)

43:31 <MatmaRex> greg-g: yeah, that was an oversight, since our team is full of third-party users too :) we should probably correct this

43:40 <Skizzerz> greg-g: If you'd like I can go through and clarify what all of the "we"'s are sometime after this meeting to make it more clear :)

43:40 * greg-g nods

44:10 <greg-g> either way on that, Skizzerz, I think my bigger point is what I'm more thinking about (how you'll advocate for others vs yourselves)

45:14 <gwicke> What's your plan for making the installation & maintenance of full-featured mediawiki (with caching, tidy, VE, parsoid etc) easier?

45:20 <greg-g> :)

45:45 <Skizzerz> greg-g: We (The Consortium) are all 3rd party users, and a lot of our goals as contractors align with pain points that we have experienced administering 3rd party wikis/wiki farms in the past. Does that answer your question?

46:05 <mwalker> you dont have to address this in response to gwickes question; but it is related: "if that question is answered -- how do you plan on improving the release notes / regression notices? Something that's traditionally been a very hard problem for the foundation"

46:29 <greg-g> Skizzerz: I think so :)

46:36 <bawolff> It wasn't that hard a problem in the svn days...

46:37 <greg-g> Skizzerz: it's a nuanced thing, I think

47:25 <MatmaRex> mwalker: well most of all, somebody should actually review the changes coming in, at least briefly. right now adding the release notes is an annoying process for developers as well (constant merge conflicts), so it's very easy to forget about them

47:29 <gwicke> I see 'outreach' in your plan -- who is going to do the work?

48:13 <greg-g> MatmaRex: would that be something the consortium could do?

48:15 <MatmaRex> mwalker: i've previously written some simple tools to make adding release notes less painful, and i'm hoping to continue working on that

48:29 <MatmaRex> greg-g: yes

48:44 <greg-g> MatmaRex: eg, reading over the merged changes per wmfXX branch, consolidating that into the point releases or major releases

48:52 * greg-g nods

49:10 <MatmaRex> such problematic changes are often not very hard to spot if you're looking for them, but are very easy to forget about

49:16 <Skizzerz> gwicke (in response to the "full-featured" question): We can package standalone extensions in the tarball similarly to how we are doing now, but that is only half of the story. Something I'd love to see is an install script that is capable of installing dependencies for packaged extensions, so you can package VE, check the "I want to install this" box, and the script tries to get parsoid set u

49:16 <Skizzerz> p, etc. (prompting the user with instructions if it is unable to do that)

49:35 <Skizzerz> Another avenue is package repositories, such as debian and red-hat

49:58 <Skizzerz> so you can apt-get install mediawiki-visualeditor and it'd set up node.js, Parsoid, and VE with sane default configs

50:34 <greg-g> gwicke: is it feasible for a single host (vps or real machine) to host all the peices (namely just parsoid, ve, and mwcore)?

50:57 * greg-g goes down tangent, should probably stick to topic

51:03 <gwicke> greg-g: sure

51:08 <greg-g> You have two related (in my eyes) points in the "What we propose to do" section:

51:11 <Skizzerz> sure, I've partially done it (was writing a guide, then left off on it)

51:11 <greg-g> "Thorough testing before releases: multiple PHP versions, different operating systems and databases, and against common extensions" and "Development of automated testing infrastructure"

51:12 <gwicke> my $3/month VPS certainly can

51:12 <MatmaRex> greg-g: i can answer, definitely yes :)

51:16 <greg-g> and From my experience, that's a ton of work for non-full time individuals. What experience does your team have on automated testing? Do you have experience with Jenkins (how much?) and selenium?

51:28 <greg-g> gwicke: :)

51:56 <chrismcmahon> greg-g: as we discovered recently VE also needs the Cite extension and TemplateData (and some other Template stuff?) at minimum

52:21 <greg-g> chrismcmahon: :) but those aren't separate services, at least0

52:22 <greg-g> -0

52:39 <gwicke> Skizzerz: given that we already have packages for services like parsoid -- are you going to build the installer packages and/or work on improving core packaging?

52:41 <James_F> chrismcmahon: Actually, that's just the way it's currently packaged; we could split it up if there's third party demand to not use e.g. the citation stuff.

53:02 <James_F> greg-g: We're about (2–3 months) to add the citoid service as an optional extra, BTW.

53:04 <greg-g> The consortium: to keep you on track/topic: see the question from me re Jenkins/Selenium :)

53:10 * James_F shuts up.

53:12 <greg-g> James_F: oh right

54:42 <greg-g> (6 minute warning)

54:51 <MatmaRex> greg-g: re testing, i think we've admittedly only worked with jenkins as "end-users" until now (having it comment on our patches :) ), i have a bit of experience with ruby and selenium

55:21 <greg-g> ok, so put another way: how will you go about doing the 'thorough testings before releases"

55:40 <greg-g> manual is well, manual and prone to miss things unless you are a pro at it. :)

55:52 <Skizzerz> gwicke: there's my initial comment about making the tarball package capable of handling stuff like that (likely to be realized via composer, although I haven't given the fine details a whole lot of thought yet). I feel it is very important that the end user have multiple avenues of installing something, as not everyone will have SSH access and apt-get/yum. Thus, having a system in place that c

55:52 <Skizzerz> an set all of that up from a web UI would be very beneficial to those users, and isn't a point that the WMF has already made a lot of progress in, I believe

56:38 <greg-g> (thanks for your full thought answers, Skizzerz, seriously)

57:11 <gwicke> Skizzerz: you can't set up fully-featured MW on a shared host normally

57:13 <Skizzerz> now that I have that out of my way, I've fiddled with jenkins a bit (I run my own instance at https://jenkins.skizzerz.net to run builds for a C++ project a couple of friends and I are working on), so I'm at least familiar with the admin side of it

57:18 <greg-g> I don't have any other questions for now, unless more on the testing pre releases can be said by the team, so if others have short/quick questions in the last 4 minutes...

57:31 <greg-g> Skizzerz: cool

57:34 <Skizzerz> if that host provides node.js already installed (sadly I'm unaware of any that do), I don't see why you couldn't

57:53 <MatmaRex> greg-g: i'm not sure what kind of answer you're looking for, unit tests and integration tests (like crawling a wiki with a lot of extensions installed and looking for fatals) are the usual approach and that's what we're planning

57:56 <gwicke> my question was specifically about the growing group of users that have a cheap VPS with root

58:05 <legoktm> greg-g: manual testing isn't perfect, but I think it should still be able to catch stuff like https://bugzilla.wikimedia.org/show_bug.cgi?id=60054

58:06 <gwicke> and have the ability to run everything & the kitchen sink

58:10 <gwicke> if there's good packaging

58:15 <gwicke> so far we dont' support them

58:22 <bawolff> You mention documentation. Currently we have very little documentation on scaling up mediawiki (e.g. How to set up slave databases, How to set up varnish with htcp purges, etc). Is that something you guys plan to work on, or do you plan to concentrate on small users (Which is arguably most of our third party user base)

58:24 <legoktm> (as a random example)

58:27 <greg-g> MatmaRex: right, so going with that, do you/your team, feel comfortable doing it?

58:52 <greg-g> legoktm: agreed, it isn't one XOR the other

58:57 <MatmaRex> yup

59:06 <greg-g> MatmaRex: cool, thanks

59:12 <MatmaRex> greg-g: yup

59:16 <Skizzerz> good instructions go a long way (I've started an incomplete guide for VE at https://mediawiki.org/wiki/User:Skizzerz/VisualEditor ), and easy install scripts go even further. For example we can ship an install.sh as part of VE that the user can run to grab all of the dependencies and set it up automagically :)

59:29 <Skizzerz> er

59:30 <Skizzerz> mislink

59:47 <Skizzerz> https://www.mediawiki.org/wiki/User:Skizzerz/VE_on_1.23

00:13 <greg-g> 1 minute left officially, we can officially end the meeting but you all can stay and chat for as long as you can.

00:28 <Skizzerz> any last questions? :)

00:36 <Skizzerz> oh, bawolff had one

00:39 <gwicke> Skizzerz: so was that a 'no' ?

00:41 <gwicke> ;)

01:56 * greg-g waits for Skizzerz to answer bawolff

02:23 <csteipp> Consortium: for collaboration with package maintainers, do you guys have contact with the current debian/redhat/etc maintainers?

02:32 <Skizzerz> bawolff: I believe having good step-by-step guides is very beneficial to sysadmins who may not be entirely familiar with the mediawiki ecosystem. The issue with such guides is they quickly get out of date, so it would require going back to them and updating every so often. Things that are considered "scaling" can also benefit small users as well, for example rate limiting features are not avai

02:32 <Skizzerz> lable unles Memcached is used

03:19 * greg-g now waits for an answer to csteipp's question

03:27 <greg-g> (there's no one after us in here, so we can go long)

03:44 <bawolff> Well different scaling is useful to different groups. You have to be pretty big to need master/slave db. Caching is useful to just about everybody

04:06 <Skizzerz> We are not going to only focus on small users, however. Some of us are in a unique position to actually have access to and experience running larger farms (ShoutWiki for instance), and assisting others in bridging the gap from "small wiki" to something larger is something we'd be able to do

05:36 <gwicke> large farms are less interesting for distribution IMO

05:37 <greg-g> anyone on the consortium have contact with the current debian/redhat/etc mediawiki maintainers?

05:50 <greg-g> (rephrasing/asking csteipp's question)

06:04 <Skizzerz> gwicke: In my opinion, there are four different "classes" of third-party users: shared hosting, single wikis on dedicated/VPS, "clustered" wikis/wiki alliances, and wiki farms. Our efforts would mostly focus on the first two groups, as they are the most likely to make use of the tarball packages

06:19 <gwicke> also, do you have experience with packaging yourself?

06:28 <MatmaRex> csteipp: no. we're hoping they'll be happy to accept our contributions without a need to have a "contact" inside ;)

06:36 <greg-g> :)

07:08 <greg-g> ok, I'm going to officially end this, but please do keep chatting if you have time and more questions.

07:11 <greg-g> Thanks all!

07:25 <Skizzerz> Thanks!

07:29 <greg-g> Thanks to the applicants and those who asked questions, especially. Great discussions.

07:35 <greg-g> #endmeeting