Talk:Future Audiences

From Meta, a Wikimedia project coordination wiki

/July 2023

The following Wikimedia Foundation staff monitor this page:

In order to notify them, please link their username when posting a message.
This note was updated on 04/2024

Announcing monthly Future Audiences open "office hours"[edit]

Hi all, we're going to be kicking off a monthly hour-long "Future Audiences" video call – this will be a space where anyone interested in this work can come learn, ask questions, and give input. This month, I'll be sharing some early results from the ChatGPT plugin experiment, as well as research on social video app creators that may inform some future experimentation on these platforms.

More details:

Looking forward to seeing/hearing from you, and if you can't make it this time, no worries – planning to make this a regular monthly meeting. MPinchuk (WMF) (talk) 16:02, 27 July 2023 (UTC)[reply]

@Waltercolor, @Natalia Ćwik (WMPL), @Lydia Pintscher (WMDE), @Grzegorz Kopaczewski (WMPL), @Klara Sielicka-Baryłka (WMPL), @Bertux, @Sandizer, @Frank Schulenburg, @MJL, @Jklamo, @Sdkb, @Frostly, @Rtnf, @Count Count, @Fuzheado, @Shani (WMF), @Soni, @Theklan, @Heike Gleibs (WMDE), @Tarkowski, @AyourAchtouk, @Bluerasberry, @Adithyak1997, @Psubhashish, @Sobaka, @Alalch E., @Dyork, @Mathglot, @DancingPhilosopher, @Stevesuny, @Oceanflynn, @Kasyap
Pinging to let you know about this meeting next week (see above)! Also, that is a lot of user pinging And I know not everyone keeps up with Meta, so please do get in touch via email (futureaudiences(_AT_)wikimedia.org) and indicate your interest in participating in this and future calls, so that I can just send a quick email update to the group to let you know about next month's call. Thank you, and looking forward to your thoughts/questions! MPinchuk (WMF) (talk) 16:46, 27 July 2023 (UTC)[reply]
  • UPDATE: Thanks to all who attended the call! Links to the slides, recording, and notes can all be found here. I'll go back over the notes and if there were any questions that weren't covered, will add them here and do my best to answer in the next few days. Hope to see you next month! MPinchuk (WMF) (talk) 19:49, 3 August 2023 (UTC)[reply]
  • Questions asked but not covered in the meeting:
  1. Are there plans for/progress with a WMF TikTok account as part of this work? We are planning to discuss this with our Communications team in September. A WMF account could be helpful in experimentation, but there are lots of important considerations before proceeding.
  2. Are there plans for further (new or assembly of existing) research as part of this work? Content formats, knowledge influencers, subject matter, platform usage? Yes, we definitely want to conduct more research on both how people interact with AI assistants for learning/general knowledge, and to learn more about how and where global youth learn. Ideas/suggestions for research questions are always appreciated!
  3. Question: I saw a post by a market researcher who claimed that young people start at TikTok, but when they want more information, they travel from that platform to other (written) platforms on the Internet. How can we be sure what young people actually want? If we added more videos or entertaining content to Wikipedia, then to what extent would that satisfy users? Excellent question and something we're digging into more deeply. The initial results of testing prototypes of more media-rich content (from an admittedly small sample of around a dozen young people) suggested that they preferred the less visually-appealing, more static content because it was more readable. This is something to keep an eye on and continue to investigate as we get a larger sample of user feedback. But yes, we shouldn't assume that just because some experiences (e.g. short video) work on other platforms that they'll translate well to our projects/context.
  • Question: one of the beauties of Wikipedia is the Talk page, and this is where a lot of potential engagement lies. Any thoughts on how we might surface this aspect of the tool in things like conversational AI? Great question – the short answer is, we haven't thought too much about this yet, but I'd love to talk more and hear ideas if anyone has them!
Ideas/suggestions:
  1. Focus on conveying originality/authority in media contributions and all content when more and more content is being synthesized. (Provenance of media metadata; use AI to merge source information, find duplicates and identify sources; use AI to create audiovisual narratives based on well-sourced information)
  2. Re the work on the approach to AI: I am wondering about the sense of interactivity that ChatGPT offers with its slight pause and “thinking” that gives it a sense of human-ness and that perception that it’s happening in real time is part of the level of interest. (Ironic, since WP is entirely created by humans…) While this is a totally inaccurate perception, it provides a sense of interactivity that is much more dynamic than Wikipedia’s pages of static comment.
  • Idea: This may be pointing toward a future recommendation about providing different options about how people read WP than the straightforward single option about how people edit WP. The bars for both of these uses are different but their current approach is the same.
MPinchuk (WMF) (talk) 20:12, 10 August 2023 (UTC)[reply]

Environmental impact of ChatGPT plugin (and other AI tools)[edit]

Hi – I hope that I am in the right place to ask this question: I read that currently available AI tools such as ChatGPT use absurd amounts of energy, both during the training of the algorithm and for generating outputs. Therefore, I am wondering if and how this also applies to our new ChatGPT plugin (and any other AI-related Wikimedia 'ventures'). Has someone already started looking into this issue? How will this be factored into the WMF environmental sustainability report? Thank you, Gnom (talk) 23:59, 1 August 2023 (UTC)[reply]

Hi @Gnom! I'm the Director of ML for Wikimedia. That is a really good question. You are right to point out that inference with large models at ChatGPT (although they aren't saying how large, but it is definitely large) is expensive in terms of energy. That said, the best way to think about would be that our plug-in doesn't use ChatGPT, instead ChatGPT uses our plug-in, meaning the only additional energy usage would be from the plug-in itself. Put another way, our plug-in doesn't actually have any AI or ML in it, it simply searches Wikipedia using the regular search API for the most relevant articles and provides those to ChatGPT. The plug-in itself is only a few hundred lines of code takes little resources to run. It would easily run on your laptop without any issues. We also haven't purchased any additional servers or anything to run the plug-in, so any energy impact is negligible for this experiment.
If this experiment was a big success and folks wanted to scale up then we would have to add dedicated servers, at which point we should factor it into our environmental reporting. CAlbon (WMF) (talk) 19:29, 3 August 2023 (UTC)[reply]
Thank you for the quick response, @CAlbon (WMF). Gnom (talk) 07:49, 5 August 2023 (UTC)[reply]

The next monthly Future Audiences video call[edit]

Hi all, this is announcement of the next monthly hour-long Future Audiences video call – a space where anyone interested in this work can come learn, ask questions, and give input on the work the Future Audiences team are doing around AI or social media experiments, or talk about their own related initiatives. Please feel free to turn up – we're happy for anyone who wants to come talk or listen to us. You don't need to have done anything specific within these fields, the main requirement is to be interested enough in the topic that you want to spend time in the call.

More details:

Slides, recordings and notes from previous meetings can be found at Future Audiences/Community discussions, and you can read more about questions asked at the last meeting above. We're looking forward to seeing/hearing from you. /Johan (WMF) (talk) 15:26, 30 August 2023 (UTC)[reply]

Ping Waltercolor, Natalia Ćwik (WMPL), Lydia Pintscher (WMDE), Grzegorz Kopaczewski (WMPL), Klara Sielicka-Baryłka (WMPL), Bertux, Sandizer, Frank Schulenburg, MJL, Jklamo, Sdkb, Frostly, Rtnf, Count Count, Fuzheado, Shani (WMF), Soni, Theklan, Heike Gleibs (WMDE), Tarkowski, AyourAchtouk, Bluerasberry, Adithyak1997, Psubhashish, Sobaka, Alalch E., Dyork, Mathglot, DancingPhilosopher, Stevesuny, Oceanflynn, Kasyap, who have previously showed interest in these calls. /Johan (WMF) (talk) 15:40, 30 August 2023 (UTC)[reply]
This is two days from now! Johan (WMF) (talk) 09:35, 12 September 2023 (UTC)[reply]

FYI: The presentation slides, the video recording (part 1 and part 2, and notes are now available. LWyatt (WMF) (talk) 11:03, 15 September 2023 (UTC)[reply]

Getting content creators to cite Wikipedia[edit]

@MPinchuk (WMF) and @LWyatt (WMF), following up from the call earlier, I had a few thoughts on getting content creators to cite Wikipedia that I wanted to share.

Fundamentally, this is a question of incentives, since people are never going to do something en masse unless they find the pros of it outweigh the cons. Currently, the main incentives that come to mind for citing work overall (not just from Wikipedia) are:

Pros

  • It increases audience trust in your content by making it seem well-researched.
  • In some cases it reduces the chance that the entity you're building your work off of gets mad at you.

Cons

  • It's very time-consuming
  • Depending on the quality of the source, some may use it to discredit your content.
  • Depending on how you use the source, some people in your audience may judge you for ripping it off (e.g. think you're lazy).

The way that this tends to work out in practice is that some of the highest-quality content cites sources, since they value audience trust and have the capacity to make citations. This is despite YouTube/TikTok having no native way to do citations, making it a clunky process. I like the approach of the Real Science YouTube channel, which has numbers pop up in the corner every time a fact is mentioned, which then leads to a link to the source in the video description. But beyond that realm, most content isn't citing sources, since they either don't have the capacity to do so or think it would reflect badly on them if they did.

Efforts to get people to cite Wikipedia could try to minimize any of the cons. For the time-consumption one, the clear path there would be to get YouTube or TikTok to introduce tools to make it easier to cite sources. This would also likely increase the number of viewers who would check out those sources (which is a plus for us as a source but not for the platforms if it takes people away from them).

For the quality issue, the big problem here is the decades-old refrain in everyone's mind about "Wikipedia is not a reliable source." Marketing to change cultural attitudes about using Wikipedia would be the (admittedly difficult) path there. It's complicated by the fact that, yes, Wikipedia is not a reliable source. Personally, when I'm sharing something, sometimes in addition to citing the direct source of information, I'll also give "discovery credit," i.e. this is the person/aggregator/etc. that led me to discover the original source. That's ideally what we'd want content creators to be giving Wikipedia, and to be ultimately citing Wikipedia references rather than Wikipedia itself. But if getting people to cite sources is hard, getting them to offer discovery credit on top of that is even harder/even less common currently.

Lastly, for the ripping off concern, Wikipedia is also rather vulnerable. Because it's so easy to access, I think many people may perceive citations to it as meaning that a content creator didn't dig very deep. This has resulted in a situation where many people are embarrassed to say they used Wikipedia, since they're worried it'd make others think they're lazy. Part of the solution here might be to work to educate the world about Wikipedia's free content license.

I hope all that is helpful, and looking forward to further discussion! Thanks for hosting the monthly calls; it's really helpful to be able to touch base through them. Cheers, {{u|Sdkb}} talk 23:05, 14 September 2023 (UTC)[reply]

Yes indeed, these are definitely kinds of reasons that content-creators of 'edu-tainment' material say when asked about this issue. And on top of what you've said, there's also the issue that we all want people to go onwards and continue their research from the things that we cite. Which, when they do reference those academic publications and primary source material (a good thing!) they are [probably] less willing to acknowledge that they discovered those sources via Wikipedia's footnotes. It's more impressive to say "according to this 1973 patent application...." than "according to a footnote about a 1973 patent application in the Wikipedia article about this topic..."
So the question is: How do we encourage meaningful (not begrudging/forced) attribution of where video content creators are getting their ideas from - because we know they are reading Wikipedia articles as part of their research - in a way that is helpful to their audience and technically easy for them to do. It's something the Future Audience program is going to experiment with over the coming year. Keyword 'experiment' - not big/expensive/long-term investments but little tests to gather real-world proof of concept data. LWyatt (WMF) (talk) 11:25, 15 September 2023 (UTC)[reply]

The next monthly Future Audiences video call: October 19, 14:00 UTC[edit]

Hi all, this is announcement of the next monthly hour-long Future Audiences video call – a space where anyone interested in this work can come learn, ask questions, and give input on the work the Future Audiences team are doing around AI or social media experiments, or talk about their own related initiatives. Please feel free to turn up – we're happy for anyone who wants to come talk or listen to us. You don't need to have done anything specific within these fields, the main requirement is to be interested enough in the topic that you want to spend time in the call.

More details:

Agenda:

  • 1. AI & attribution presentation & Q&A (30 min)
  • 2. Engaging youth on third-party video platforms:
  • a) Ikusgela presentation (15 min)
  • b) WMF TikTok experiment (15 min)
  • c) Wrap-up discussion/Q&A (10–15 min or as needed)

Slides, recordings and notes from previous meetings can be found at Future Audiences/Community discussions, and you can read more about questions asked at the last meeting above. We're looking forward to seeing/hearing from you. Johan (WMF) (talk) 16:43, 9 October 2023 (UTC)[reply]

The next monthly Future Audiences video call: December 14, 15:00 UTC[edit]

Hi all, this is announcement of the next monthly hour-long Future Audiences video call – a space where anyone interested in this work can come learn, ask questions, and give input on the work the Future Audiences team are doing around AI or social media experiments, or talk about their own related initiatives. Please feel free to turn up – we're happy for anyone who wants to come talk or listen to us. You don't need to have done anything specific within these fields, the main requirement is to be interested enough in the topic that you want to spend time in the call.

More details:

Agenda:

  • A collaborative mapping of possible strategies the Wikimedia movement could pursue to ensure that Wikimedia content and communities continue to thrive in the face of changing technology and user behavior trends
  • A brief recap of some conversations on this topic that began at WikiConference North America last month, then collectively map out and discuss the key features, risks, and opportunities of several potential strategies
  • This mapping will inform the annual Wikimedia Foundation External Trends update

To register contact: futureaudiences(_AT_)wikimedia.org – we'll send you a link to the call.

Slides, recordings and notes from previous meetings can be found at Future Audiences/Community discussions, and you can read more about questions asked at the last meeting above. We're looking forward to seeing/hearing from you. Johan (WMF) (talk) 13:00, 7 December 2023 (UTC)[reply]

Ending the Wikipedia ChatGPT plugin experiment[edit]

Report of the ChatGPT experiment (PDF)

Hi, I'm Mike, a Wikimedia Foundation product manager who has been working on the Wikipedia plugin on ChatGPT.

In July 2023, the WMF Future Audiences team launched a ChatGPT plugin as an experiment in learning more about the rapidly evolving Artificial Intelligence (AI) landscape, accelerated by OpenAI’s release of their ChatGPT Large Language Model (LLM). This experiment has been fruitful in helping us learn more about how we fit into this new technology. The end date for this experiment is 2 Feb 2024.

Below is a brief overview of the project background and lessons learned from the experiment.

Background

The release of ChatGPT in late 2022 was a landmark in providing an easy-to-use LLM capable of understanding and generating natural language, often performing with seemingly near-human levels of proficiency. With its rapid increase in usership, ChatGPT had the potential to become a new way for most people to get information online, potentially putting WMF projects like Wikipedia at great risk from lower traffic to our websites, where much of our donations and contributions occur. While posing risks to our existing volunteer recruitment and fundraising models, LLMs also offer the potential to increase the variety of channels and processes for reading and editing Wikipedia content.

In that context, in March 2023 the WMF Future Audiences team was invited by OpenAI to develop and release an official Wikipedia ChatGPT plugin on its new “plugin” platform, which gave a handful of organizations (e.g., Kayak, Wolfram Alpha) the ability to more narrowly tailor the ChatGPT experience. The experimental Wikipedia plugin we built allowed ChatGPT subscribers to get information directly from Wikipedia via ChatGPT, which was instructed to only use Wikipedia content to generate an answer, and with appropriate citation(s).

. Our goal in releasing the plugin was to answer two primary questions:

  • User behavior. Is there a paradigm shift towards ChatGPT (or other LLM-based chat assistants) being the primary means of getting information online?
  • LLM tech. How well does ChatGPT (or other comparable LLMs) generate relevant and accurate text for information seeking purposes?

Lessons learned

ChatGPT has not become the new information seeking paradigm (yet?).

After 6 months of being active, we have concluded that the ChatGPT has not significantly affected the public’s usage of the internet as a general purpose tool. Instead, we are seeing specialization of LLMs for specific tasks, as well as this technology being used to augment traditional and familiar information seeking forms, such as search. It will be good to continue to keep an eye on these trends, though the initial urgency has diminished.

We also noticed that ⅔ of users are from N. America and Europe (based on logging data), as well as being middle-ish aged men (based self-reported survey data). At this time, it doesn’t appear that chat assistants are reaching new audience demographics beyond what we typically expect to see in many of our editor communities. We should note, though, that the Wikipedia ChatGPT plugin required users to have a paid ChatGPT account, which poses a barrier of entry for the same people who have historically been underrepresented. As OpenAI shifts to being more for-profit, these fees will continue to be obstacles for wider access to Wikipedia content through ChatGPT.

LLMs

While there continue to be many problems with LLMs, including the ability to unpredictably hallucinate facts and/or reproduce biases in its input data, we have found that they are generally quite good at retrieving and summarizing Wikipedia content as answers to natural language queries. Based on a small-scale internal audit of historic queries, we found our ChatGPT plugin’s responses to be generally relevant and accurate, and are confident that with more iteration, relevance and quality could continue to improve over time.

LLMs are likely here to stay in some form, and offer a powerful toolset for well-defined tasks. We would recommend continuing to further explore using them where appropriate.

Based on self-reported survey data, we also found that users generally had more trust in ChatGPT’s responses when it was clear that the answers were coming from Wikipedia. Crucially, this was not just the content itself, but when our “brand” was associated with the answer. This implies that Wikipedia’s name and direct attribution have social capital, and we should strongly consider how to better take advantage of this fact as a sign of both knowledge integrity in a world of misinformation, and as a financially significant brand asset.

Winding down

We are ending the Wikipedia ChatGPT experiment primarily due to being now able to answer our primary questions – as a reminder, our main objectives on the WMF Future Audience team is to run quick, cheap, experiments to learn about near-term trends, not to build fully featured and permanent products.

This timing also coincides with OpenAI’s move away from the plugin marketplace for ChatGPT, and towards no/low-code customizable GPTs (i.e. AI chat assistants focused on specific tasks). This shift has made our plugin in its current form inaccessible to new users and largely redundant. While we could repurpose this functionality towards being a GPT, we don’t believe we would learn significantly more beyond how to create a product within the OpenAI ecosystem. This may be a good place for WMF to eventually have/maintain a product, and we will continue to keep an eye on developments there, but the urgency for this is not high at the moment.

For further information about the plugin’s code and functionality see this repo. To learn about and ask questions about the WMF’s Future Audiences project visit its homepage on Meta wiki. MPham (WMF) (talk) 15:44, 2 February 2024 (UTC)[reply]

Besides being a paid feature, another reason for the skew in user demographics may be that this is not a Wikimedia hosted platform, but a platform where users may not feel safe for various privacy and integrity reasons. ♥Ainali talkcontributions 20:39, 7 February 2024 (UTC)[reply]

The next monthly Future Audiences video call: February 15, 15:00 UTC[edit]

Hi all, this is announcement of the next monthly hour-long Future Audiences video call – a space where anyone interested in this work can come learn, ask questions, and give input on the work the Future Audiences team are doing around AI or social media experiments, or talk about their own related initiatives. Please feel free to turn up – we're happy for anyone who wants to come talk or listen to us. You don't need to have done anything specific within these fields, the main requirement is to be interested enough in the topic that you want to spend time in the call.

More details:

Agenda: Conclusion of ChatGPT plugin experiment, and what comes next.

To register contact: futureaudiences(_AT_)wikimedia.org – we'll send you a link to the call.

Slides, recordings and notes from previous meetings can be found at Future Audiences/Community discussions, and you can read more about questions asked at the last meeting above. We're looking forward to seeing/hearing from you. Johan (WMF) (talk) 13:00, 7 December 2023 (UTC)[reply]

Recording (& transcript)
This meeting has now occurred.
The topics covered were discussed in the meeting (video embedded here) at the following times:
  • @00:00 - Welcome, context of 'future audiences'
  • @14:40 - Conclusion of ChatGPT plugin experiment (report & slides);
  • @35:00 - introduction of "citation needed" browser extension experiment
  • @55:00 - invitation to "AI Sauna" event, Helsinki.
Sincerely, LWyatt (WMF) (talk) 17:44, 15 February 2024 (UTC)[reply]

How to propose a new domain/audience[edit]

Hi: I'm new with this Future Audiences so sorry if asking something obvious. For some time to know I'm working conceptualizing a couple domains which I see has enormous potential and with a place in the Wikimedia Movement. The question is: which is the procedure to present and develop them?

Thanks. —Ismael Olea (talk) 16:00, 8 March 2024 (UTC)[reply]

Hey Olea: Do you mean like a new wiki project? Johan (WMF) (talk) 00:29, 22 March 2024 (UTC)[reply]
Ideally, yes. For the moment I just want to spread a new way (for us) to describe knowledge using linked open data models. I would share some technical details but I'm actively writing them to submit a proposal for a funding call on 1st April. I'll do later. For the moment I only have this almost useless testimonial description. This particular proposal focuses in learning, but the approach can apply to any other domain.
They key point in this conversation is, as far I understand, this approach would define a new audience with a quality change, models, as we have done with metadata in Wikidata. But maybe I missed something. —Ismael Olea (talk) 10:17, 22 March 2024 (UTC)[reply]
This is a major conversation much bigger than the Future Audiences team. I'm honestly not sure about the best way to approach it. I would say "a new proposal on Meta", but they rarely tend to get that many eyes. I'd suggest talking about it in the community first, gaining support. Maybe in Wikipedia Weekly, the Facebook group, on Telegram, or perhaps wikimedia-l, the mailing list? Johan (WMF) (talk) 12:25, 12 April 2024 (UTC)[reply]

The next monthly Future Audiences video call: April 18, 14:00 UTC[edit]

Hi all, this is announcement of the next monthly hour-long Future Audiences video call – a space where anyone interested in this work can come learn, ask questions, and give input on the work the Future Audiences team are doing around AI or social media experiments, or talk about their own related initiatives. Please feel free to turn up – we're happy for anyone who wants to come talk or listen to us. You don't need to have done anything specific within these fields, the main requirement is to be interested enough in the topic that you want to spend time in the call.

More details:

Agenda:

To register contact: futureaudiences(_AT_)wikimedia.org – we'll send you a link to the call.

Slides, recordings and notes from previous meetings can be found at Future Audiences/Community discussions, and you can read more about questions asked at the last meeting above. We're looking forward to seeing/hearing from you. Johan (WMF) (talk) 12:28, 12 April 2024 (UTC)[reply]

Pings for people who have indicated their interest on the Future Audiences page: Aisha Khatun, Newton Kamau (AI), User:Waltercolor, Natalia Ćwik (WMPL), Lydia Pintscher (WMDE), Grzegorz Kopaczewski (WMPL), Klara Sielicka-Baryłka (WMPL), Bertux, Sandizer, Frank Schulenburg, MJL, Jklamo, Sdkb, Frostly, Rtnf, Count Count, Fuzheado, Shani Evenstein Sigalov, Soni, Theklan, Heike Gleibs (WMDE), Tarkowski, AyourAchtouk, Bluerasberry, Adithyak1997, Psubhashish, Sobaka, Alalch E., Dyork, Mathglot. Johan (WMF) (talk) 12:46, 12 April 2024 (UTC)[reply]
More pings for users who have signed up: DancingPhilosopher, Stevesuny, Oceanflynn, Kasyap, Waldyrious, Doc Taxon, Lofhi, SJ, Danny Benjafield (WMDE), Hfordsa, Baltakatei, CorraleH, Doctorxgc, Lebron jay, Benoît Prieur, Alphama, Susannaanas, Zache, VisbyStar, Joalpe, Chlod, Ismael Olea, Txtdgtl (WMMX), SCP-2000, Hammunculs. Johan (WMF) (talk) 12:46, 12 April 2024 (UTC)[reply]
@Johan (WMF). Thanks for the ping. There might be a problem with the info, as on the title it states April 18 and on the text December 14 (time stamp also leads to December 14). Anyway, I cannot make it unfortunately, as this event falls during the Wikimedia Summit. Will te call be recorded? Thank you. Joalpe (talk) 14:28, 12 April 2024 (UTC)[reply]
Thanks, fixed by Sdkb – I was so sure I had fixed the link, but I think I changed it in the window where I copied the template from, not the one where I posted. My apologies.
It will be recorded! Johan (WMF) (talk) 15:49, 12 April 2024 (UTC)[reply]
I could not attend, sorry :-/ —Ismael Olea (talk) 08:40, 13 April 2024 (UTC)[reply]

Another heads-up: AI call on May 16[edit]

We're also having a conversation in May, we're having a general AI call related to the annual plan. The page doesn't have a lot of information yet as we've just created it, but do sign up for that, too, if you're interested! Johan (WMF) (talk) 12:36, 12 April 2024 (UTC)[reply]

AI:JA! Thanks Johan. –SJ talk  14:42, 14 April 2024 (UTC)[reply]
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Talk:Future_Audiences&oldid=26602954"