User:Lahariyaniyathi/Responses to FDC

Mike Peel

Question

@Lahariyaniyathi: It is an interesting metric, but I'm not sure I see how you would make use of it. Is the aim to maximise it, or minimise it? Would a metric looking at the numbers of pages affected by this work be more meaningful? Thanks. Mike Peel (talk) 17:59, 23 April 2017 (UTC)

Answer

CIS-A2K would like to thank you for the question. We have introduced this metric to set right a historical anomaly of sub-standard content being present on Wikimedia projects that A2K team considered as Focus Language Areas. The outcome of the metric is not to 'maximise bytes deleted' but to bring to notice articles/content that should be improved. We are hoping to report minimal numbers against this metric but explain (both in quantifiable terms and qualitative terms) what changes have been made to the content under this metric. The performance of this metric is inversely proportional to the efficacy of our Content Enrichment and Skill Building Initiatives. Such an arrangement will act as an internal check for our programmatic activities as well.

As a metric our aim is to minimise it but as a Quality Improvement Measure we would like to maximise it. Historically CIS-A2K has received critical feedback regarding the content that was introduced to our FLA projects both via content enrichment activities and WEP initiatives. Over the past few years we have introduced several measures to maintain the quality of content. This metric is inspired by the initiative of Telugu Wikipedians attempt to rectify the damage caused by machine translated articles housed on Telugu Wikipedia.

Risker

Question

Questions from FDC members Risker[edit]

Thank you very much for the opportunity to review this proposal. I find the focus on leveraging the more technical skills of the community to be interesting, in particular. A few questions:

You indicate that you will be adding two new positions, but you say in Question 4 under table 4 that you anticipate spending approximately 58,000 USD for staffing in the coming year. That is a dramatic decrease from the current year (approx. 102,000 USD), and doesn't match what you post in your work plan, which indicates staffing at approx. 86,000 USD. Could you please verify what your actual expected staffing expenses will be for the period covered by this grant.

Answer

Dear Risker,

Many thanks for taking the time to engage with our proposal. We apologise for the confusions and attempt to answer your queries:

We will be adding two positions but these are not new positions. These positions have been indicated in the previous proposal. Community Advocate position for Kannada Language has been left vacant from November 2016 and the current intern working for Bridging Gender Gap has a contract till June 2017.

The dramatic decrease is because of the exit and merging of a few team positions. New recruits are not being recruited at the same salary level as the past team members (For eg: 2016-17 Projection for Program Officer Communication USD 14,555.03 2017-18 Projection for Program Officer Communication is USD 8,910). Also a Program Officer has been promoted to the position of Program Manager after the incumbent Program Manager exited the team. As this was an internal promotion and no new candidate has been hired to fill the position A2K team has been able to reduce its staffing costs.

Our staffing expenses will be 57,982.68 USD as mentioned under Table 4 of the proposal page. The work plan page takes into account both the staffing cost and the honorarium provided to Community Advocates(57,982.68+27,855) working across 4 languages (Kannada, Marathi, Odia and Telugu) and 1 thematic area (Metrics and evaluation) as well. However the honorarium provided to Community Advocates is considered as Volunteer Support and not staffing costs to provide autonomy for their work.

Question

If this expense is lower than the previous year, please advise how you will be increasing staffing while lowering staffing costs.

In a similar vein, your total expenses in Table 8 are lower than the amount you are requesting in this grant. Please advise if there is any information that needs to be corrected here.

Our total expenses as indicated in Table 8 $168,637.68 (Rs1,12,42,512.1) does not include Institutional Development Fee which is $18,000 (Rs12,00,000). We have not included this as there was no scope for this item in Table 8. After adding Institutional Development fee of $18,000 our total project grant stands at $186,637.68. The excess $ 6,637.68 is in-kind support that CIS shall be generating for the A2K team.

Thanks, Risker (talk) 14:54, 23 April 2017 (UTC)

Answer

Our total expenses as indicated in Table 8 $168,637.68 (INR. 11,242,512.1) does not include Institutional Development Fee which is $18,000 (INR 1,200,000). We have not included this as there was no scope for this item in Table 8. After adding Institutional Development fee of $18,000 our total project grant stands at $186,637.68. The excess 6,637.68^{USD or INR} is in-kind support that CIS shall be generating for the A2K team.

Mike Peel

Question

Thanks for submitting this proposal! I have a few questions: Are the numbers in Table 2 correct? They seem to show that you've already spend all of the previous grant, with some months still left to go? This is quite different from the numbers in the progress report, which showed you were underspending. I also don't follow how the projected amount at the end of the grant can be less than the actual amount spent by 1 March 2017, or is the projected amount excluding in-kind donations

Answer

Dear Mike Peel,
CIS-A2K is immensely grateful for your comments and questions as they allow us to clarify on matters of importance.

The numbers on Table 2 are correct. They reflect the numbers as per the first installment of the grant ($89340.47) that A2K team has received. We are glad to inform you that during the time of drafting the proposal (21 March, 2017) we had utilised 100% of the first installment of the grant. We received the second instalment of the grant on 29 March, 2017 ($63831.75). The under-spending as reported during the Progress report (27 January 2017)was when A2K team had not yet organised its marquee events (TTT and MWT 2017 20-26 February, 2017). With these events and other programmatic activities we have utilised 100% of our 1st instalment of the grant.

You are absolutely correct that the 'porojected amount at the end of the grant can not be less than the actual amount spent'. The projected amount is excluding in-kind support that can be calculated only after the completion of this year’s programmatic activities. We have mentioned the calculation of in-kind support as a footnote for further clarification.

Question

On staffing, there is an increase in 1 Program Officer without any note in the explanation of changes. Also, in last year's plan you were saying 10 staff members, whereas the current number here is 9. Can you comment on these please?

Answer

Apologies for not explaining the changes under Table 4. A2K team would like to recruit a Program Officer for Technical Operations during the July 2017 to June 2018 plan period. This position was active until November, 2016 after which the position became vacant. A2K would initiate the process to fill this position for the upcoming grant.

In the last year’s plan we mentioned 10 but during the current plan period a Program Officer has been promoted to the position of Program Manager after the incumbent Program Manager exited the team. As this was an internal promotion it resulted in merging of two positions and hence the decrease in staff numbers.

Question

In both this year's proposal and last year's you've had very ambitious targets with folios uploaded to Wikisource, but at Q2 in this year you're at 3,700 of the 200,000 target, and you're aiming for 100,000 this year. How realistic is this target? (Don't get me wrong, 3,700 folios uploaded is fantastic anyway!)

Answer

There has been an initiative with Maharashtra Granthottejak Sangh where CIS-A2K had initiated digitisation of 1000 books pertaining to Maratha History, Public Life, Social Customs and Religious Practices. These books have been fully scanned and digitised, and we are currently under the process of verifying copyright issues and prioritisation of books to be uploaded with the help of the community members. This process has taken a lot more time than we anticipated and hence the poor performance against the metric. We are confident that this community consultation will be over soon and uploading of the folios will begin.

Liam

1. As with the section above, I too am confused about how the "value of bytes deleted" metric works. I appreciate that you are trying to ensure not just improved quantity on-wiki, but improved quality too. But, can you explain:
   1. How you arrived at the goal/target of deleting "3,583,120" bytes (primarily in Marathi and Telugu WPs)? This is an extremely specific number.

      Deletion of bytes as a quality improvement practice on Telugu Wikipedia came out of Google Translate article prioritization exercise. More than 1,900 articles in Telugu Wikipedia have maintenance tags and need to be improved - most of these are machine translated articles that were created between 2009-11. Telugu Wikipedians found that the project had produced many articles which were of huge size and of poor quality Due to the huge size of the articles attempts to correct each and every line in order to improve them proved to be a demanding task.
      For example, A Google translated article about conspiracy theory on Telugu Wikipedia which initially had 127,078 bytes was considered as one of the articles that needed to be improved. The improvement in this case was deletion of 114,901 bytes. The article now stands at 12,000+ bytes. Another example is of an article about Kumara Mangalam Birla which had 17,491 bytes and after deletion stands at 8,491 bytes.
      A2K team has been engaged with scanning of 1000 books with Maharasthra Granthottejak Sangh and the scanning activity has been completed. For the upcoming grant period our efforts will focus on uploading these pages on Marathi Wikisource. During our test run we have found that OCR conversion of the scanned page generates around 20kb of junk data (OCR for Indic languages does not give 100% output and is dependent on the type of print, quality of scan and age of the book amongst many other conditions). A2K team intends to upload 100,000 pages on Wikisource during the next plan period and hence the 2MB metric.
      We have added values from all of our plan verticals to achieve the number ‘3,583,120’ Bytes

      • Bridging Gender Gap: 3,000
      • Kannada: 45,000
      • Marathi: 2,000,000
      • Odia: 30,000
      • Skill Development Initiatives: 2,048
      • Telugu: 1,500,000
   2. In the same way that it is hard for an affiliate to claim obvious responsibility for any specific bytes added to a Wikipedia, how will you define which bytes deleted from these Wikis are appropriate to count in your report, and which are "organic" deletions?

      For any bytes deletion activity there is a three step process that A2K team would be following along with the support of community members.

      • Identification of stub articles: by employing cohort analysis we shall be selecting stub/junk articles that were created by editors initiated by A2K or in the activities organised by A2K.
      • Community Consultation: A consultation to arrive at the course of action for each article. Identified article can be deleted, identified article can be improved and editors who have created or worked on this article will be given further training.
      • Deletion of articles: A2K team would request the admins to delete articles that have not been improved despite various efforts

      The bytes that are deleted as per the community consultation and approval are the ‘organic deletions’ and these are the only bytes that will be reported. It is the aim of the A2K team to minimise this metric as it allows us to work towards improving the existing content. However with the increasing instances of non NPOV content finding its way on Indic language Wikipedia projects we believe that this is an important quality ensuring measure to be followed.

   3. Do you consider more deleted bytes to be a "better" or "worse" result than your target number? why?

      A2K considers 'more' deleted bytes as a worse result or a disappointing outcome. We feel this way because it would mean that A2K took the easy way out by deleting the content than investing in improving the content, increasing the skills of editors, providing better understanding of Wikipedia policies and making better references and resources available to editors.

2. With regards to the "footprint tracker" metric:
   1. If this is a measure of outreach events that are a) organised by community members and b) only those about outreach; this means that any events organised by A2K and anything (organised by anyone) focused on supporting the existing community - are not counted?

      That's correct. No activity of A2K team in places/institutions where there has been an earlier initiative will be counted once a year in this metric. However the same is not true for Wikimedians. One of the aims of TTT is for the participants to conduct effective outreach activities and a prerequisite for participation in TTT is to be an active Wikimedian. We have observed that TTT participants are eager in conducting Wikimedia Orientation and Outreach activities in not only their native town and states but also explore inter state and inter district possibilities of conducting such events.

      If a new event (Wikidata, Wikispeech, Wikivoyage) related to new projects is conducted by A2K team, such an event will be counted under this metric. But an edit-a-thon or a Wikisource sprint with a partner institution involving community members will not be counted as per this metric.
      A2K will support all existing community members in conducting outreach events and will count the event under Footprint Tracker only if

      1. Community member provides consent to be included in this metric
      2. If there had been no prior activity at that venue.

   2. Furthermore, since "no one time activity will be included" yet you will "only count one footprint per city per project irrespective of the frequency of follow-up event" - does that mean that in order for something to count according to this metric, it has to be run more-than-once but will only appear in the statistics once, unless it is run in multiple cities?

      Yes. A2K team has been adhering the WMF directive of ‘discouraging one time activities’ and hence under this metric A2K will not be including exploratory meetings, meetups without a follow-up plan or celebratory events. If the same event is taken up by different communities at different intervals we shall be including that event under this metric.

3. I do not understand the way you have laid-out your SMART goals chart (This image). The proposal asks you to "list your goals and objectives for each program" and to ensure that each of the goals is S.M.A.R.T. - but the words listed in the middle of this graphic are sometimes conceptual (e.g. "outreach" or "reach" [which is listed twice] or "sustainable collaboration"), sometimes named programs (e.g. "Women in Red" or "Hyperlocal QRpedia" or "heritage walks") and sometimes quantitative targets (e.g. "1200 encyclopedic images" or "involving at least 65 active wikipedians"). These are all grouped horizontally by thematic area (Content, Skill, Instutional, GLAM) and grouped Vertically by each part of the SMART acronym. But... I don't understand how to read this graphic. For example "Facts matter" appears at the intersection of "GLAM" and "Measurable"; or "open assessments" appears at the intersection of "institutional partnerships" and "achievable". Neither "Facts Matter" nor "Global Guidance" are mentioned anywhere else in the document. Can you explain how I should use this graphic to understand how each of your projects indeed have SMART goals; and why the different words in the graphic were placed where they are?

   Apologies for the confusion, it was our intention to provide the FDC members and global Wikimedians an overview of our program structure via the above-mentioned graphic. A2K team has attempted to depict SMART programs instead of representing SMART goals of our programs. We sincerely apologise for the confusion. The conceptual terms were used to avoid lengthy text and repetitive terms, the usage of quantitative targets was to reflect our commitment toward RoI.
   The graphic was to be read from Right to Left with the following ontological construct: Through this specific initiative (sustainable collaboration), a measurable activity (Wikimedian in Residence) would shape up a program that will achieve (resource exchange) for the A2K team and shall result in movement relevant goals (increasing reach)in a time bound fashion (of initial level of collaboration during the program period).
   On closer inspection and multiple readings we have realised that the placement of these terms specifically and the overall table have not been easy to read and understand. A2K team apologises for the dense table and for not attaching a glossary or a footnote explaining the idea behind this graphic.
   We are attaching herewith two visualisations of SMART goals of our programs and SMART programs (as an earlier version has already been submitted in the proposal). Goals of the A2K team for the current proposal are in alignment with our strategy document submitted to WMF. Our proposal for July 2017-June 2018 have the following goals:

   • Cultivating new editors
   • Empowering the existing community
   • Content generation
   • Skill building and general service to the movement.

     

     A suitable caption goes here.

     

     A suitable caption goes here.

4. With regards to your "Global Engagement activities: I definitely appreciate that you are trying to find successful programs from other Wikimedia Chapters (and indicating in the summary table which chapter inspired you for each) and replicate them. However, I don't see how you have the resources to achieve the things that you're proposing. Example 1: the WikiMOOC (which you've said was inspired by WM-France and is described here by them has been a multi-year project with dozens of people involved and taking >2.000 hours of work - and is described in extensive detail in this section of WM-FR's current Annual Plan application. Your proposal description on this topic is three sentences long. Example 2: Your Technical Wishes project is also described briefly and without a clear allocation of resources (time, volunteers, staff, budget). By Comparison, WM-Germany (which you've specified as your source of inspiration for this project) as a whole software development department for this activity (as described near the bottom of this section in their Annual Plan. WM-Sweden has a project like this. I certainly understand that A2K is not proposing to do anything like on the same scale as those chapters, and that's fine!, but could you point to some more detail about how you are going to do those things with the resources you are asking for?

Thank you for your response on the Global Engagement part of the proposal. As you have rightly identified, the Global Engagements plan is an effort to learn from and adapt successful lessons from global chapters. To this effect, we have picked programs that are most transferable and well-suited for localisation to the Indic language communities. Since this is a fledgling attempt, we would urge you to keep in mind that our targets as well as our scale are made out accordingly.

With regard to Example 1, the aim of the proposed WikiMOOC is to work in conjunction with our Wikipedia Education Program in order to create a central repository of Wikipedia editing resources for the students. While performing its role of being an online course, this also enables us to demonstrate the platform and its use during our presence-based events. While catering to students on the WEP, this can also be used for display and distribution to the general public to raise awareness regarding the basics of contributing to Wiki projects.

With regard to Example 2, the Technical Wishes project will be a one-time event as it runs parallel to our Request page. This will surround our flagship event, MediaWiki Training. We would request participants from across the country to send in the specific technical bugs or errors they would like to see fixed(wishes fulfilled) during the event. These will then be collaboratively addressed during the sessions.

The attempt to work with pre-existing programs run by CIS-A2K is keeping in mind our limitations with regard to staff, budget and scale. We understand these plans are ambitious and we have created a safety net with regard to time and volunteers by combining these new projects with the ones that already exist. Please let us know if there is anything else. Regards.

Laurentius

1. You made and interesting choice of grantee-defined metrics; they point in the direction of a focus on quality and structured activities. The way in which they are defined or you plan to work with them is not completely clear to me, but I see that there are already other questions addressing those points.

   Thanks Laurentius for engaging with our Proposal page and Work plan. A2K is attempting to build a sustainable communities across Indic community projects and hence our choice of tracking footprints and bytes deleted as grantee defined metrics.
   We have provided our response to the questions raised by other FDC members regarding the grantee defined metrics. Please let us know if there are any other questions or observations. ~~~~

2. Thanks for the metrics histograms, they make things visually clear. I was wondering:
   • Marathi is higher on participants and newly registered, but lower on content pages; Telegu is the opposite. What is the reason of that difference? (that might be for exemple a difference in the activities, or maybe Telegu is a more mature community and you focus more on content and less on bringing in new people?)

   There are two reasons for this structural difference:

• As you have rightly pointed out Telugu community is a more mature community and we are focusing on improving and consolidating content enrichment activities. With the Marathi community our aim is to gather a critical mass of users, readers and editors before we launch into content generation activities.
• Due to the non availability of Community Advocate from June 2016-December 2016, A2K could not follow-up on its earlier outreach activities and lost some steam. We will need to have regather our base and hence the difference. ~~~~

• Bytes deleted apply almost exclusively to Marathi and Telegu - why? - Laurentius (talk) 08:16, 25 April 2017 (UTC)

A2K team has always worked closely with the communities while formulating new plans and proposing activities. Of the four language communities that we work closely Telugu community has already embarked upon a cleaning drive of junk articles and Marathi community members have expressed their wish to do the same. However the consensus on Kannada Wikipedia and Odia Wikipedia is towards improving the articles rather than deleting them. Hence a large section of the Bytes deleted metric is coming from Telugu and Marathi language plans. A2K team will however attempt to identify articles/content across all verticals of its engagement and improve such articles/content. As a last resort these articles might be deleted and the resulting bytes will be indicated against this metric. Thanks and regards. ~~~~

Table 2

Financials for the current funding period
Revenues or expenses	Planned (budgeted)		Actual, until one month before the proposal due date		Projected
	Currency requested	US dollars	Currency requested	US dollars	Currency requested	US dollars
Revenues (from all sources)	INR 10,500,000	153,195	INR 6,123,045	89,335.22	INR 10,700,000	156,113
Expenses	INR 10,500,000	153,195	INR 5,386,537	78,589.57	INR10,700,000	156,113