| @metadata | | sourceLanguage | "en" |
|---|
| priorityLanguages | |
|---|
| allowOnlyPriorityLanguages | true |
|---|
| description | "video in Introduction to Partnership Building (computer-graded exercises) - Basics of partnership building for Wikimedia volunteers, with computer-graded exercises." |
|---|
| label | "WikiLearn - Introduction to Partnership Building (computer-graded exercises) - video: Picking indicators; Joint metrics" |
|---|
|
|---|
| display_name | "Picking indicators; Joint metrics" |
|---|
| subtitle-120-5910-1 | "Now, we already discussed indicators." |
|---|
| subtitle-6090-11460-2 | "We discussed the sort of the building blocks of evaluation are those indicators that tell" |
|---|
| subtitle-11460-17670-3 | "us whether this particular action has contributed to the desired change in a" |
|---|
| subtitle-17670-20250-4 | "certain indicator, in a certain measure of success." |
|---|
| subtitle-22350-26100-5 | "And sometimes those indicators, again, are not ours to choose." |
|---|
| subtitle-26100-28860-6 | "They are dictated to us by an external funder." |
|---|
| subtitle-30750-37470-7 | "But when we get to choose, we should always measure indicators that are tied to outcomes." |
|---|
| subtitle-37590-45150-8 | "The indicators should somehow show us progress towards an outcome, and almost" |
|---|
| subtitle-45150-47970-9 | "anything can be quantified in one way or another." |
|---|
| subtitle-50400-57210-10 | "I say almost anything, even if it is not inherently a quantity, not inherently" |
|---|
| subtitle-57210-63990-11 | "countable. We can create countable indicators of it by surveys, for example." |
|---|
| subtitle-64170-71010-12 | "Your satisfaction with this course is not a number, but I use surveys to get a quantified" |
|---|
| subtitle-71460-74190-13 | "indicator of your satisfaction with this course." |
|---|
| subtitle-74670-82350-14 | "That's an example of deriving a quantified metric from an essentially non quantifiable" |
|---|
| subtitle-83220-86670-15 | "sense, feeling, impression, etc." |
|---|
| subtitle-86670-88590-16 | "and it has obvious downsides." |
|---|
| subtitle-89100-94710-17 | "That number, if the number ends up being, whatever, 4.6 out of five, that's probably" |
|---|
| subtitle-94710-96870-18 | "good. But it doesn't tell me really how you feel." |
|---|
| subtitle-97350-103530-19 | "It just tells me that broadly more people ranked it five or four than people who ranked" |
|---|
| subtitle-103530-104730-20 | "it one or two." |
|---|
| subtitle-104880-110010-21 | "But it doesn't give me deep insight into what you feel about the course." |
|---|
| subtitle-110040-116760-22 | "So the last bullet here says, that quantitative indicators do tell us the" |
|---|
| subtitle-117120-119160-23 | "progress we're making against our goals." |
|---|
| subtitle-119160-123090-24 | "Because, remember, our goals were designed to be measurable." |
|---|
| subtitle-123510-129240-25 | "I refer you back to Assignment seven, our theory of change, but qualitative indicators," |
|---|
| subtitle-129240-134730-26 | "indicators that aren't numbers, that are open ended, questions that are interviews." |
|---|
| subtitle-135270-142320-27 | "These can give us a deeper understanding of things like motivations, concerns, examples," |
|---|
| subtitle-142320-144270-28 | "stories, testimonials." |
|---|
| subtitle-144270-149010-29 | "And those could be crucial to understanding the quantitative results." |
|---|
| subtitle-149190-155610-30 | "To understanding why we're getting 2.1 here, whereas last year we were getting 4.7." |
|---|
| subtitle-156630-160140-31 | "The fact that we are getting 2.1 doesn't explain why." |
|---|
| subtitle-160560-167070-32 | "So qualitative research and qualitative indicators are definitely part of a good" |
|---|
| subtitle-167070-169290-33 | "evaluation, to be clear." |
|---|
| subtitle-171210-175360-34 | "But broadly, the qualitative indicators will tell us why." |
|---|
| subtitle-175620-177240-35 | "Will give us the insight." |
|---|
| subtitle-178050-182160-36 | "So here are some examples of measures of success in the movement, and some of them are" |
|---|
| subtitle-182160-184740-37 | "very common and shared across many, many projects." |
|---|
| subtitle-185280-191550-38 | "For example, number of participants, or number of newly registered users, or number" |
|---|
| subtitle-191550-193710-39 | "of content pages created or improved." |
|---|
| subtitle-193740-200460-40 | "These are very frequent indicators in a lot of project plans, in a lot of partnerships" |
|---|
| subtitle-200460-202260-41 | "and a lot of grant proposals." |
|---|
| subtitle-203080-206360-42 | "But there are many other indicators that we could have." |
|---|
| subtitle-206380-210430-43 | "For example, the number of female editors who are admins." |
|---|
| subtitle-210640-216430-44 | "That's an interesting indicator for a certain kind of question around diversity and" |
|---|
| subtitle-216430-224950-45 | "participation. The number of policy processes influenced can be a metric," |
|---|
| subtitle-225880-232420-46 | "the whether and how much institutions incorporate open knowledge in their programs." |
|---|
| subtitle-232750-235180-47 | "And again, that's not a number." |
|---|
| subtitle-235180-240730-48 | "So we need to find ways to measure it, or at least to capture it qualitatively through" |
|---|
| subtitle-240730-244960-49 | "stories and testimonials, etc., etc.." |
|---|
| subtitle-245860-247540-50 | "The tone of discussion on Wikimedia platforms." |
|---|
| subtitle-248410-250660-51 | "Has the tone become more friendly and open?" |
|---|
| subtitle-251600-254180-52 | "How do we know? How do we know?" |
|---|
| subtitle-254300-256120-53 | "Probably through a survey of some kind." |
|---|
| subtitle-256130-259970-54 | "We need kind of an aggregate opinion of people who have been participating in" |
|---|
| subtitle-259970-264140-55 | "discussions to tell us if the tone has improved, for example." |
|---|
| subtitle-264170-270020-56 | "So there are many, many possible measures of success, and some of them may be unique to" |
|---|
| subtitle-270020-271160-57 | "your projects." |
|---|
| subtitle-271190-275930-58 | "In other words, don't always just take the ones you saw in another grant proposal." |
|---|
| subtitle-275960-280100-59 | "Really think about what you need to measure for your outcomes." |
|---|
| subtitle-280100-284270-60 | "And it's possible that you need a metric that nobody ever had to measure before." |
|---|
| subtitle-284720-285920-61 | "That's absolutely fine." |
|---|
| subtitle-289730-295250-62 | "Again, the most important thing is your measures of success should always be" |
|---|
| subtitle-295250-298520-63 | "connected to the outcomes that you want to achieve." |
|---|
| subtitle-299180-302570-64 | "Don't just adopt measures of success from somewhere else." |
|---|
| subtitle-303110-309170-65 | "Now, ideally, we want those measures of success to be joint, to be things that both" |
|---|
| subtitle-309170-312140-66 | "partners care about ideally." |
|---|
| subtitle-312290-320780-67 | "If both our group and the partner share some of these indicators or metrics, we are both" |
|---|
| subtitle-320780-327170-68 | "looking at the same number, but we're both looking at the same gauge to see if we're" |
|---|
| subtitle-327170-330530-69 | "doing well or not. At least [for] some of our goals or outcomes," |
|---|
| subtitle-330530-333230-70 | "we are literally tracking the same thing." |
|---|
| subtitle-333230-335060-71 | "We and the partner." |
|---|
| subtitle-335920-340120-72 | "And this contributes to what I called "shared reality" last time." |
|---|
| subtitle-340440-345760-73 | "This idea that we're looking at the same things and we won't be surprised by the" |
|---|
| subtitle-345760-349180-74 | "other's results or the other's perspective." |
|---|
| subtitle-350360-352340-75 | "It also improves accountability." |
|---|
| subtitle-352340-358190-76 | "If we're both measuring the same thing, even if they're also measuring other things we" |
|---|
| subtitle-358190-363050-77 | "don't care about. For example, page views to their Facebook account." |
|---|
| subtitle-363260-364880-78 | "That might be something the partner cares about." |
|---|
| subtitle-365210-368930-79 | "We, Wikimedia, don't directly care about their Facebook page." |
|---|
| subtitle-369290-370880-80 | "That's a relevant metric for them." |
|---|
| subtitle-373590-378540-81 | "And when we have such joint metrics, they provide a really solid foundation to build on" |
|---|
| subtitle-378540-380220-82 | "for making any change." |
|---|
| subtitle-380460-383400-83 | "If we both look at something and we see, "you know what?" |
|---|
| subtitle-383400-388740-84 | "This needle isn't moving, this thing is not improving!", then it's very easy to agree," |
|---|
| subtitle-388740-390420-85 | "yes, we need to change something here." |
|---|
| subtitle-390420-396360-86 | "We need to rework it, change an approach, or maybe even abandon it." |
|---|
| subtitle-396360-401400-87 | "Sunset it. Like let it come to a halt." |
|---|
| subtitle-403050-408870-88 | "For example, the number of attendees to the editing workshop at the museum." |
|---|
| subtitle-409170-415350-89 | "That can be a joint metric because it advances both our goals of training editors" |
|---|
| subtitle-415350-420300-90 | "and the museum's goal of holding relevant public events, for example." |
|---|
| subtitle-420930-427740-91 | "So for these different goals, we may have this shared metric or shared indicator." |
|---|
| subtitle-428430-436920-92 | "We're both tracking this number towards our different but compatible goals." |
|---|
