Grants talk:Project/FLG/History of Quebec and French-speaking North America

From Meta, a Wikimedia project coordination wiki

Feedback/First thoughts[edit]

Hey @Mathieugp: and team! This proposal provides some great first outlines of the work. I have a couple of open questions:

  • First-- can you clarify a bit what work has already been happening at the FLG in relationship to Wikimedia? It would be useful for reviewers of the grant to understand how this role could build on a history of doing this well.
  • Second -- residencies typically include a substantial component beyond event coordination (batch uploads to commons/wikidata and/or facilitating online competitions/campaigns). The current grant only seems to be focused on a limited number of events. Can you describe a little bit deeper what the 2 month event cycles will include? (will it be multiple trainings? Will it include more campaign-like activities? Etc.) Do you plan on doing something with data or media content that helps organize the contribution?
  • Third -- outcomes seem to be built around specific numbers of people with training, and around content. However, the expected impact of the project seems to be more focused on the development of a community interested in French-Speaking North American history on Wikimedia projects. Could you evaluate this broader goal through other ways? (I.e. training professionals at x number of organizations, who can facilitate events on their own). Part of what is persuasive about the framing of the grant is the intention to engage a network of participants': how can you make this a more central part of the evaluation?
  • Additionally, a salary of 40,000 seems a bit low for a full year of income in a major metropolitan area in North America -- is this being designed as part time employment? Or are you planning on framing it as a fellowship? It may be worth documenting any funding that is coming in-kind from the organization or allied organizations (including for example, any contribution to running the events, supplements of the salary for the residence, money spent on MathieuGP's supervision of the role, etc).

I am commenting as my role as a GLAM strategist for the foundation, and am happy to talk through/explore these comments more if you have other questions. Astinson (WMF) (talk) 17:32, 9 November 2018 (UTC)Reply[reply]

@Mathieugp: pinging correctly Astinson (WMF) (talk) 17:33, 9 November 2018 (UTC)Reply[reply]
Thanks for the feedback! I will answer the four points briefly right away and come back next week with more details:
1. Our main Wikimedia projects (listed here) have involved mostly training (on the first Tuesday of each month since 2014 + other occasions), but also uploading media (image, audio, text), improving articles, and publishing articles (outside Wikipedia) to demystify our favourite encyclopedia for the benefit of history teachers (listed here).
2. Yes, the tasks of our Wikimedian-in-Residence will go beyond just event coordination. I will come back with more detailed scenarios next weeks.
3. Yes, we can certainly try to come up with a way to quantify the evaluation of our broader goal of building an active community.
4. Our Wikimedian-in-Residence will work the same schedule as the rest of the team, so 32 hours per week (8AM to 5PM, Monday through Thursday).
-- Mathieugp (talk) 20:08, 9 November 2018 (UTC)Reply[reply]
@Mathieugp: Thanks! Looking forward to some of those specifics working their way into the grant :) Astinson (WMF) (talk) 20:53, 13 November 2018 (UTC)Reply[reply]
@Astinson (WMF): It is done! I have modified the grant project based on your feedback and changed its status from DRAFT to PROPOSED. Mathieugp (talk) 20:35, 14 November 2018 (UTC)Reply[reply]

Eligibility comments[edit]

Hi Mathieugp, this is Chris Schilling, the program officer for Project Grants. Thanks for your proposal to start a Wikimedian-in-Residence program with the Fondation Lionel-Groulx. I wanted to bring your attention to the eligibility criteria concerning equipment purchases that are eligible for funding:

Equipment to be shared amongst Wikimedia community members (cameras, laptops, book scanner, wifi hotspot/dongle, etc.) with a plan for use and content integration
Small scale equipment purchases for Wikimedia affiliates to be shared among multiple volunteers. If no affiliate is available to host the equipment, a single active volunteer in good standing may serve as the coordination point for shared equipment.

...and what is not eligible for funding:

Personal equipment, including books, and equipment for non-Wikimedia affiliates

Given the activities you've listed in the proposal, I think I have a fair idea of how the equipment (TV, conference cam, laptop/accessories) would be used to facilitate content integration. More generally, I also consider this to be a small scale equipment request. However, it is not clear who would own the equipment during and after the grant, and how it would be made available and accessible to other volunteers in the movement. While a formal, finalized plan on this need is not required at this stage, I would like to hear how you are thinking about this. Thanks, I JethroBT (WMF) (talk) 03:39, 8 December 2018 (UTC)Reply[reply]

The Fondation Lionel-Groulx would own the equipment. The primary use of the equipment would be training. It could also be used, like you suggest, for contribution events. Our first choice would be to keep the equipment at the Fondation Lionel-Groulx after the grant as we have every intention of continuing to provide training and organize contribution events in 2020, 2021, and beyond. That being said, we could also play safe and take the written engagement to donate the equipment to Wikimedia Canada were the Fondation Lionel-Groulx to stop its wikimedia projects in the future for whatever reasons (who knows what future boards of directors could decide). What do you think? --Mathieugp (talk) 15:37, 10 December 2018 (UTC)Reply[reply]
@Mathieugp: While it is acceptable for the equipment to be physically housed and kept at Fondation Lionel-Groulx (for the sake of convenience for this project and other possible future works), ownership would either need to be maintained by Wikimedia Canada, or if they are unable, by another affiliate, or failing that, by yourself as an individual. Please start by contacting Wikimedia Canada as soon as you are able to gauge their capacity to do so. I JethroBT (WMF) (talk) 19:18, 10 December 2018 (UTC)Reply[reply]
@I JethroBT (WMF): Understood. I will clarify this point with Wikimedia Canada and come back here with an answer ASAP. --Mathieugp (talk) 19:28, 10 December 2018 (UTC)Reply[reply]
Good day, on behalf of Wikimedia Canada, I can say that we can own the equipment. It would be physically kept at the Fondation Lionel-Groulx for the duration of the project. After that, we would keep it where it makes most sense, since this equipment could be used for other activities organized by the chapter and the volunteers in Canada. Thanks, Amqui (talk) 15:24, 16 January 2019 (UTC)Reply[reply]
@Amqui: Thank you for this message. I will inform the Project Grants Committee of this arrangement regarding the equipment. I JethroBT (WMF) (talk) 18:20, 22 January 2019 (UTC)Reply[reply]

Eligibility confirmed, round 2 2018[edit]

This Project Grants proposal is under review!

We've confirmed your proposal is eligible for round 2 2018 review. Please feel free to ask questions and make changes to this proposal as discussions continue during the community comments period, through January 2, 2019.

The Project Grant committee's formal review for round 2 2018 will occur January 3-January 28, 2019. Grantees will be announced March 1, 2018. See the schedule for more details.

Questions? Contact us.

--I JethroBT (WMF) (talk) 19:19, 10 December 2018 (UTC)Reply[reply]

Some questions for clarification[edit]

This is an interesting proposal, and I have a couple questions about it:

  • In your statement, "significantly improved the quality of 40 articles in the French-language version of Wikipedia through the organization of four major contribution events that will involve 80 participants" do you mean any 40 articles or those that specifically involve the History of Quebec and French-speaking North America?
  • Does French-speaking North America also include articles or populations located outside Canada, such as in New Orleans, New York (i.e., the U.N.), or Haiti?
  • When the project ends, do you envision this proposed Wikimedian in Residence role to end, or will you seek funding to continue this work?
  • Do you have any thoughts on how you can determine or track if "at least 10 to 15% of the 80 participants to our events will continue to contribute from time to time" after this project ends?
  • Can you describe what will be involved in the "March-April : (training of our Wikimedian-in-Residence)"?
  • Does the budget for the Wikimedian in Residence include any funds for benefits or taxes or the like (not sure how this works with staff and insurance and such in your area)? The budget also seems fairly low for a major city like Montreal. Will this be enough for somebody who has the experience you require for the work you plan?
  • How will you measure and evaluate the success of each of your monthly events and then how will you integrate those lessons learned into improving the next events?
  • Will you engage in other Wikimedians in Residence as part of this initiative to share ideas and also contribute to the global movement of others working in similar areas?

I look forward to your ideas on these questions. --- FULBERT (talk) 14:11, 29 December 2018 (UTC)Reply[reply]

I want to make sure you saw this Mathieugp. --- FULBERT (talk) 22:26, 30 December 2018 (UTC)Reply[reply]
Yes, I saw. I will get back to you ASAP. Probably tomorrow. :-) --Mathieugp (talk) 00:21, 31 December 2018 (UTC)Reply[reply]
Thanks Mathieugp. —- FULBERT (talk) 12:42, 31 December 2018 (UTC)Reply[reply]
OK, so here are the answers to the questions above:
1. We mean 40 articles on the History of Quebec and French-speaking North America.
2. The most extensive definition of French-speaking North America includes parts of Canada, the USA and the Antilles. However, in the literature, the focus is often on continental North America, so mainly Canada and the USA. In the USA, it includes not only New Orleans, but all of former French Louisiana, and the parts of the New England States where many French-speaking Quebecers migrated in the 19th and 20th centuries (see Quebec diaspora, Franco-Americans).
3. We envision this proposed Wikimedian-in-Residence will continue and we will seek new funding.
4. We will collect usernames during our events. One way would be to provide a post-event survey to try to determine who intends to continue to contribute and later on (let's say 6 months after) verify what people really did. But I am open to other suggestions.
5. Our Wikimedian-in-Residence will mainly learn - straight from me - how to train new Wikimedians, Commoners and Wikisourcers. He/she will ultimately be training one-on-one, in small groups and larger groups during our events.
6. The $40,000 USD in revenue for the Wikimedian-in-Residence means about $54,000 CAD. The person will work 4 days a week and the Fondation Lionel-Groulx closes for a whole month during the summer. We are confident this will be considered a generous payment. We do not expect to find a particularly skilled individual, and that's why we planned two months of training. Our preferred option for payment is a simple one-year contract between the Wikimedian-in-Residence and the Fondation Lionel-Groulx (similar to my own contract). The Wikimedian-in-Residence will charge the Fondation Lionel-Groulx for sales taxes, but as a charity the State will refund the Fondation Lionel-Groulx.
7. We will measure participation to events and provide a post-event survey to get some feedback. Our goals for content improvement will involve pre-evaluating each target article for weaknesses and areas of improvement. For example, we would have article A (let's say a biography) pre-evaluated as needing more reference notes, better iconography, maybe a new section on mentions in popular culture, etc. In the French-language Wikipedia we can add a todo list in the Talk page of an article, so we are likely to take advantage of that feature to track improvements. Experience from the first event will help us establish what todo items are or aren't too difficult/time-consuming for participants.
8. Sure! Is there a mailing list or a forum for Wikimedians-in-Residence?
Mathieugp (talk) 17:33, 31 December 2018 (UTC)Reply[reply]
Thank you for your replies Mathieugp. If funded, you may want to explore the Wikimedians in Residence Exchange Network. --- FULBERT (talk) 20:21, 5 January 2019 (UTC)Reply[reply]

Aggregated feedback from the committee for History of Quebec and French-speaking North America[edit]

Scoring rubric Score
(A) Impact potential
  • Does it have the potential to increase gender diversity in Wikimedia projects, either in terms of content, contributors, or both?
  • Does it have the potential for online impact?
  • Can it be sustained, scaled, or adapted elsewhere after the grant ends?
(B) Community engagement
  • Does it have a specific target community and plan to engage it often?
  • Does it have community support?
(C) Ability to execute
  • Can the scope be accomplished in the proposed timeframe?
  • Is the budget realistic/efficient ?
  • Do the participants have the necessary skills/experience?
(D) Measures of success
  • Are there both quantitative and qualitative measures of success?
  • Are they realistic?
  • Can they be measured?
Additional comments from the Committee:
  • This proposal sees Wikipedia (and other projects) as a site where Québecois and French-speaking North America history/content should be hosted. As such, it fits with the strategic direction of knowledge as a service. There is some potential for impact with regards to improved quality of content and increased engagement of new editors. I would like to see more planning and thinking around sustainability of the work with regards to what will happen to the WiR role after grant funding has ended.
  • Truth is French-related articles on are very centered on Europe. It's a good thing this projects also aims for French diaspora outside Quebec. The most significant impact will be, in my opinion, engaging with new institutions to work with on further projects but clearly this project doesn't plan on this because it's totally dependent on further funding when finished.
  • This project appears it will fill a gap in Wikimedia coverage, one that does not appear to be intentional, though one that nevertheless is real and needs to be fixed. This is consistent with knowledge equity.
  • Nothing particularly innovative here as we have seen many WiR and editing event requests. However I see this as more of an iterative project that seeks to build on what FLG has already been doing (namely the monthly workshops). Given this context, it would be good to have more info from the applicant on what has/has not been working in terms of their involvement with Wikimedia to date. Most of the metrics provided seem to relate to the events (e.g. content improved through events; # of participants) but this also seems like it should only be a small component of the WiR’s role.
  • Clear and real targets are set to measure the outcome, however they are very low figures regarding the investment.
  • This iterates on a solution that has already been shown to work elsewhere. That this will result in an experienced Wikimedian passing this onto a new person and then guiding that person to continue spreading this forward is also something the Movement has learned can be effective.
  • Salary looks fine to me. Regardless of the eligibility of FLG to take ownership of the equipment I do question why the equipment is necessary. Travel also seems high and no details were provided to justify the amount. Applicant is experienced and the organization has existing relationships and good connections (e.g. to Wikimedia Canada and the national library). I think scope is achievable but if anything I’d actually like to see something more ambitious: What will the WiR be doing outside of organizing and providing support for the events? How will the WiR increase engagement among the history/heritage community and around contributing to Wikimedia projects?
  • I'm concerned regarding this 'training time' of the Wikimedian-In-Residence, who will spend more than 200 hours doing so. If this project is aimed to work with institutions why not trying to work with Wikimedians already with expertise on the matter thus eliminating some hours of this 'training'. Not sure on who is going to decide who are they going to contract to do so. Concerned regarding the volunteer who will give more than 200 hours in training in the space of two months, although I would say he/she has enough experience to do so.
  • This does not seem to be an overly ambitious project, so it should be able to be done given the time limits, budget, and skills provided.
  • Quite a few endorsements. Target community (outside of Wikimedia) is history and heritage professionals.
  • Considering they've been asked by staff to contact WM-Canada regarding the fate of the computer it's clear this project lacks communication with the local chapter as well they don't have any specific plan for on-line communities found in Wikimedia spaces as well as social networks.
  • The community seems to be aware of this project and seems supportive of it. As it seeks to be more inclusive of a population that has been less involved in the past, it has the promise of active engagement on a variety of levels given there will be a dedicated resource for it.
  • This seems like an experienced applicant from an organization with the necessary experience and relationships to take on this work. However, it’s not clear to me why a full WiR position is required, or how FLG will apply what is has learned from its past events to this undertaking. The proposal frames this project as a diversity/knowledge equity initiative, but while francophones are a language minority group within North America, French is still one of the larger Wikipedias and North America is over-represented among global users. For these reasons, I would prioritize this project less than others. In order to support, I would want to see more details around the WiR's position (particularly plans around engaging the history/heritage community) or a decreased budget that reflects the WiR as only a part-time (2 day a week perhaps?) role.
  • I'm not confident in the success of Wikimedian-in-Residence program in which the organisation (here the Fondation Lionel-Groulx) is not willing to pay a significant part of the project.
  • Considering this project is in the strategic plan for the Foundation Lionel-Groulx (private) and not in any affiliates plan, this foundation should be providing most of the funding for this project as it falls in their strategic engaging. Moreover expensive material will be bought with a very short-sighted use, future use is not planned more than storing it for further use, which we don't know. I don't think it's worth it for four-or-so events.
  • The only change I recommend is that they also consider writing some new articles representative of this population, rather than only improving 40 articles. That is something I think is missing from this proposal and hope gets included.

This proposal has been recommended for due diligence review.

The Project Grants Committee has conducted a preliminary assessment of your proposal and recommended it for due diligence review. This means that a majority of the committee reviewers favorably assessed this proposal and have requested further investigation by Wikimedia Foundation staff.

Next steps:

  1. Aggregated committee comments from the committee are posted above. Note that these comments may vary, or even contradict each other, since they reflect the conclusions of multiple individual committee members who independently reviewed this proposal. We recommend that you review all the feedback and post any responses, clarifications or questions on this talk page.
  2. Following due diligence review, a final funding decision will be announced on March 1st, 2019.
Questions? Contact us.

I JethroBT (WMF) (talk) 19:32, 6 February 2019 (UTC)Reply[reply]

@I JethroBT (WMF): Is it appropriate to reply to some of the above comments at this point in the process? It think I could clarify many of the above points. --Mathieugp (talk) 14:52, 7 February 2019 (UTC)Reply[reply]
@Mathieugp: Yes, if there is any feedback you would like to respond to, you are welcome to do so here and I will be sure to bring this to the attention of the Project Grants Committee. I JethroBT (WMF) (talk) 17:46, 7 February 2019 (UTC)Reply[reply]
@I JethroBT (WMF): Noted. Thanks! :-) --Mathieugp (talk) 18:47, 7 February 2019 (UTC)Reply[reply]

The comments from the above preliminary assessment that were not entirely positive pointed to some 7 issues that I chose to label this way: FLG's contribution to the project; future funding of the project; the WiR role definition; FLG's digital projects manager role; travel expenses; the lack of new contents creation; communication with Wikimedia Canada (WMC). Here are the clarifications I wish to provide on these:

  1. FLG's contribution to the project. While the FLG cannot hire a full-time WiR (32 hours per week) on its own or through its partners at the moment, hence the grant application, it is nevertheless contributing a great deal in support to the project: a workplace (with Internet, printers, office spaces, conference room, kitchen, library, etc.), administrative support by a paid employee, project coordination by another paid employee, training, supervision and support of the WiR by a contract worker (me), access to the network of institutions and associations making up the history/heritage milieu in French-speaking North America, and experience in conducting Wikimedia projects since 2014.
  2. Future funding of the project. Assuming the success of the FLG's first full-time WiR experiment, there are good reasons to be confident that it will be possible to obtain new funding either from our current partners or from new partners. Our current partners[1] include important players like Desjardins (finance), Québecor (media, telecom), Fondation J.A. DeSeve (charity), and they are currently funding some of our other projects that will eventually be over. Not only that, we will have a whole year to approach new partners. Long story short: it is realistic to think that in the short future the FLG will be able to stop depending fully on the Wikimedia Foundation to pays its full-time WiR.
  3. WiR role definition: the FLG sees the role of its proposed WiR exactly as stated in the grant application. It will be a person dedicated to a) organizing contribution events, and b) training and supporting individuals and organizations. There will be four major contribution events and dozens of training or support sessions throughout the year. The WiR will also work alongside the FLG's digital projects manager for general outreach. 32 hours a week is what we evaluated to be adequate for such of work load.
  4. FLG's digital projects manager role. This person (me) is already a de facto part-time WiR at FLG since 2014. Once supported by a full-time official WiR, I will be able to focus on the more long-term objective stated in the FLG's 2018-2019-2020 Action Plan, which is to "build a community of institutional partners and individual volunteers willing to work concretely" on the development of "high quality contents on the history of Québec and French-speaking North America in Wikipedia, Wikisource, and Commons". This will involve, as stated in the grant application, general outreach and communication. To attain this long-term and central objective, I will be working closely with the FLG's chief executive and his administrative assistant to seek new funding.
  5. Travel expenses. They would have been labeled more precisely as travel/hosting/space rental expenses. Theses expenses will occur when the WiR will travel Quebec's vast territory for on-site training or contribution events. The WiR will be accompanied by the digital projects manager from time to time, especially at first.
  6. New contents creation. It is true that creating new articles could have been included as part of the project. The reasons they were not included is that I personally feel Wikipedia is not short of stubs, and new stubs are not very difficult to create. I think it is more challenging but more valuable to improve already existing articles. I also believed that the grant review process would recognize the greater value of improving existing articles over creating new stubs. That being said, maybe a mix between creating new contents (25%?) and improving already existing contents (75%?) is a more balanced (and less challenging) approach? We can be flexible on that account. What do you think?
  7. Communication with WMC. One comment concluded that the "project lacks communication with the local chapter" since the FLG has been "asked by staff to contact WM-Canada regarding the fate of the computer". This conclusion is not accurate. Not only do I personally know several of WMC board members, (not just through Wikipedia or other electronic means), but the FLG has been collaborating with WMC on several concrete projects since 2014. As I believe it is visible from the discussion I had with @I JethroBT (WMF): concerning the ownership of the computer equipment, I was simply initially unaware that ownership by the FLG was not an option. The options I learned were owning it myself or have WMC own it. Since I did not see the point of owning the said equipment myself, I thought it best to let WMC have ownership of it.

--Mathieugp (talk) 16:11, 14 February 2019 (UTC)Reply[reply]

Round 2 2018 decision[edit]

Congratulations! Your proposal has been selected for a Project Grant.

The committee has recommended this proposal and WMF has approved funding for the full amount of your request, $49,000.00 USD

Comments regarding this decision:
The committee supports funding these efforts to organize events and trainings to improve content related to Québec and French-speaking North America in conjunction with the Fondation Lionel-Groulx, and appreciate the experience of the applicant and community engagement efforts associated with this project.

Prior to finalizing a contract, we ask that you provide a job description specifically outlining the activities of the WiR role (we are now making this request of all WiRs).

Please note that we consider funding for WiR activities to be short-term. Grant funding that the Wikimedia Foundation provides for WiRs is not intended to support ongoing workflows, but to leverage the partnership to build a sustainable platform that ensures outcomes long after the WiR has completed their service. Their work should secure long-term outcomes that do not depend on ongoing grant funding.

Next steps:

  1. You will be contacted to sign a grant agreement and setup a monthly check-in schedule.
  2. Review the information for grantees.
  3. Use the new buttons on your original proposal to create your project pages.
  4. Start work on your project!

Upcoming changes to Wikimedia Foundation Grants

Over the last year, the Wikimedia Foundation has been undergoing a community consultation process to launch a new grants strategy. Our proposed programs are posted on Meta here: Grants Strategy Relaunch 2020-2021. If you have suggestions about how we can improve our programs in the future, you can find information about how to give feedback here: Get involved. We are also currently seeking candidates to serve on regional grants committees and we'd appreciate it if you could help us spread the word to strong candidates--you can find out more here. We will launch our new programs in July 2021. If you are interested in submitting future proposals for funding, stay tuned to learn more about our future programs.

I JethroBT (WMF) (talk) 14:59, 1 March 2019 (UTC)Reply[reply]


Judging from the final report, this project turned out to be extremely expensive: some 30 articles improved, and less than 100 new users (how many active editors?), for 50 k$?! Nemo 19:36, 22 February 2020 (UTC)Reply[reply]

Hello Nemo bis! The final report of our 2019-2020 project is currently a draft and will be submitted to be under review probably in mid-March 2020. We are certain that by then we will have met or exceeded all our goals. For reference, the metrics that are in the WMF approved proposal are: 40 articles significantly improved, 60 new Wikimedians, 2 new history teachers bringing student contributions in the classroom, 5 new organizations inspired by our work and organizing other events afterwards. A focus on significantly improving already-existing articles was set instead of a focus on creating an impressive number of new article stubs (that Wikipedia does not need in my opinion). This meant choosing quality over quantity. As expected, significantly improving articles is hard (and super time consuming!), so in my opinion we were right to set a conservative figure of "only" 40 articles in total. That being said, while the creation of new article was not one of the chosen metrics, new stubs were nevertheless created during the project (around 15 so far, likely about 20 very soon) that will need to be counted in addition to the 40 significantly improved articles. In any case, I invite you to come back in about a month when the final report will be under review. Best Regards! :-) Mathieugp (talk) 21:37, 22 February 2020 (UTC)Reply[reply]
Thank you for the answer. Yes sorry, I didn't manage to comment the goals before and I'm commenting the results too early because I may forget to look back later.
I don't dispute the focus on quality nor the difficulty in expanding articles, but I suggest that for such a project maybe that's not the biggest impact you've had, so I'm not sure it's wise to focus on those numbers. Maybe it's not the number of good articles or the words added but the audience they reach, or something else. Speaking of which, I've created a pagepile for those 71 titles for usage on TreeViews, maybe you find the pageview statistics useful. I'm not mistaken they have a median of 150 visits in the last month.
Maybe the real value lies elsewhere, for instance those "60 new wikimedians" if they include many new active editors, or maybe the two teachers if they became very active. (For instance I know a couple teachers in Italy who teach Wikimedia wikis to hundreds of students every year as part of their job; if I could pay 50 k€ to "clone" them, I probably would!) Nemo 16:13, 23 February 2020 (UTC)Reply[reply]
Thanks for the good and useful feedback Nemo bis! We will surely take a close look at the pagepile you created. You are right, teachers who become very active are the best value! That is the main reason why our new grant proposal for 2020-2021 shifts the focus on academia. All the best! :-) -- Mathieugp (talk) 16:37, 23 February 2020 (UTC)Reply[reply]