ウィキメディア・エンタプライズ
ウィキメディア財団職員ならびに契約職員はボランティアのコミュニティと共にこのページの内容の保守に取り組みます。 |
Wikimedia Enterprise | |
---|---|
enterprise.wikimedia.com | |
プロジェクト | |
技術的事項 | |
Wikimedia Enterpriseはウィキメディア財団の部門横断型の新しいサービスです。enterprise.wikimedia.comでご利用可能です。このサービスの目的はウィキメディアのコンテンツの大容量での商業転用者のためのサービスを構築することです。このサービスの告知は2021年3月に(blogpost、 WIRED article)で行われました。2021年10月、 (Press release, OpenFutures article)始まりました。
ウィキメディアのコンテンツを他のコンテキストで再利用し、より高速で包括的、信頼性が高く、安全なデータサービスを大規模に提供したい組織に焦点を当てています。 ウィキメディア・エンタープライズは、ウィキメディアの読者のユーザー・エクスペリエンスを、私たち独自のWebサイトを超えて向上させることを目指しています。 また、コンテンツをより多くの人に見つけられるようにします。 さらに自己資金サービスを通じて、プロジェクトのデータを最も再利用する組織が容易に情報の典拠を確認したり、内容を検証したりできるよう向上させます。
閲覧や編集という一般的な使用方法以外では、ウィキメディアのデータを使用するには非常に高い障壁があります。 これは、ウィキメディアのコンテンツが機械にとってセグメント化して理解するのが難しいためであり、そのことがウィキメディア・プロジェクトのデータが私たち自身のエコシステムを超えて到達する範囲やそれが及ぼす影響の規模を制限しています。
運動戦略の勧告の中には、「私たちの運動の持続可能性を高める」と「ユーザーの経験を向上」というものがあり、それぞれの勧告の中には次のようなことが推奨されています。 「パートナーシップと収入によって、収入創出と自由な知識普及の両方のための新しい機会を模索。例えば、… 企業レベルのAPIを構築すること」そして「大規模ユーザーと協力しながら、ウィキメディアのAPI Suiteを、より包括的、確実、安全、高速にしていく。…そして、コンテンツの再利用者の属性と検証性の認識と容易さを改善する」
It is well known that a few massive companies use our projects' data. Those companies recognize that without the Wikimedia projects, they would not be able to provide as rich or reliable an experience to their own users. There has long been a feeling among community members that these companies should do more to reinvest in the Wikimedia communities for the benefits they gain from the content and resources they use.
This led to the idea of developing a new approach that is more sustainable in the long term and provides a much clearer relationship between Wikimedia and enterprise users. Most financial benefit for Wikimedia would likely only come from a very small handful of heavy for-profit users, and would feed back into the Wikimedia movement.
As this idea developed, it became clear there is a responsibility to democratize our data for organizations that do not possess the resources of these largest users, to ensure we are leveling the playing field and helping to foster a healthy internet without reinforcing monopolies. The benefits of such a service shouldn't just be for startups or alternatives to the internet giants, but also for universities and university researchers; archives and archivists; along with the wider Wikimedia movement.
概要
Wikimedia Enterprise’s focus is on businesses that reuse our content, typically at a large scale—e.g., integrated into knowledge graphs, search, voice assistants, maps, news reporting, community tools, third party applications, and full-corpus research studies. Augmenting Wikimedia's many datasets to put structure behind our unstructured content will allow all our content reusers to meet their individual requirements while also setting us up to build new tools and services in the future, available to everyone. Reusers of our content are looking for three critical components:
- Frequency: Regular current snapshots of Wikimedia projects
- Reliability: Dependable, accessible infrastructure
- Quality: a “best last revision”
Emphasizing a self-funding set of specific use cases allows the Wikimedia API team to focus on volunteers, teams, and organizations looking to access (and, most importantly, interact with) our data sets. This includes the majority of community editing tools, which will be out of scope for this service. For more information on improvements to the existing Wikimedia APIs see the service page on the "API Gateway" initiative.
Program Goals:
- Content: Make more of our movement's content available in consistent machine-readable formats, freely available for all researchers and re-users.
- Resource-load: Reduce the need for high-intensity site-scraping by the highest-frequency and highest-volume reusers, which currently target our production servers.
- Fundraising: Provide a clearer and more consistent way for the largest re-users to reinvest derived benefits back to the movement, instead of making occasional altruistic donations that vary in size.
コミュニティ
Contact the team if you would like to arrange a conversation about this service with your community, at a time and meeting software platform of your choice.
Past public meetings:
-
2021年3月 #1
-
2021年3月 #2
-
2021年4月
-
2022年6月
-
2023年2月
...and also at the EMWCon Spring 2021 conference (video); March and July 2022 Strategic Wikimedia Affiliates Network (SWAN) meetings; the May 2021 Wikimedia Clinic; and at Wikimania 2021.
Following are the introduction paragraphs for a much more detailed Community essay.
Libre and Gratis are the two meanings of “free,” commonly phrased as free as in speech, or free as in beer.
Wikimedia projects are, have always been, and will always remain libre. The principles of free cultural works mean that anyone can use Wikimedia without restriction, including commercially. As a movement, we embrace this. It is why we reject ‘non-commercial’ licenses, as they would limit the kinds of reuse possible. And it is why we consider commercial reuse an important means of distributing knowledge to audiences.
Equally, Wikimedia projects are, have always been, and will always remain gratis. The ability to freely access the knowledge available across all Wikimedia projects has always been core to the mission of the Foundation and the movement. We provide this access not only to individuals visiting our websites but also programmatically to machines so that our content can be repurposed in other environments. The full corpus of Wikimedia content always has been, and will continue to be, made available for reuse in various forms (including but not limited to database dumps, APIs, and scraping) at no cost.
As a result, our content is often repurposed by for-profit organizations that rely on it to support their business models, and which consequently earn revenue from it. Outside of voluntary corporate donations to the Wikimedia Foundation, the movement has never received benefits from any of this revenue through return investment. In acknowledgement of this, under the heading of Increase the sustainability of our movement the Movement Strategy process asked the Wikimedia Foundation to explore, among other things, “enterprise-level APIs...models for enterprise-scale for-profit reusers, taking care to avoid revenue dependencies or other undue external influence in product design and development.” Furthermore, under the heading Improve User Experience, a further recommendation stated, "Make the Wikimedia API suite more comprehensive, reliable, secure, and fast, in partnership with large scale users where that aligns with our mission and principles, to improve the user experience of both our direct and indirect users, increase the reach and discoverability of our content and the potential for data returns, and improve awareness of and ease of attribution and verifiability for content reusers."
The Enterprise project team is developing a new resource aimed at for-profit content reusers, who have product, service, and system requirements that go beyond what we freely provide. Use of this offering will not be required for for-profit content reuse; companies can continue to use the current tools available at no cost. All Enterprise API revenue will unequivocally be used to support the Wikimedia mission—for example, to fund Wikimedia programs or help grow the Wikimedia Endowment.
This project represents a new kind of activity at the Foundation. The project is at a very early stage that should be considered a learning period. We will have successes, we will make mistakes, and we will need to adapt our strategies. The team is committed to listening, engaging, and where possible, integrating the feedback we get on our work. This document is organic and is reflective of the team's current thinking; we are attempting to document as much work as possible in the open. Up until now, our work has been shaped by a series of initial interviews with community members, Wikimedia Foundation Board and staff, researchers, and reusers.
...continue to read the rest of the Community essay. See also the FAQ and Principles.
Given the nature of the service, primary decision making for it will rest with the Wikimedia Foundation. We are seeking community input, in particular from the technical community and those who have been involved in the strategy process, throughout the lifetime of the service. Technical feedback has been gathered from colleagues at the Wikimedia Foundation, industry and research partners, technical partners across the movement, and with the broader technical communities via Phabricator. Input into the funding development side of the service will follow a similar pattern. We will continue gathering input via research interviews and focus groups, as well feedback here on Meta as per our principles.
Technical
For full information about the product work, see the documentation homepage on MediaWiki.org and the API documentation subpage.
Over time, the "product" being offered will grow and improve. This information is accurate as of February 2023.
オンデマンドAPI
High-volume reusers that use an infrastructure reliant on the EventStream platform depend on services like RESTBase to pull HTML from page titles and current revisions to update their products. High-volume reusers have requested a reliable means to gather this data, as well as structures other than HTML when incorporating our content into their KGs and products.
Wikimedia Enterprise On-demand API, at release, will contain:
- A commercial schema
- SLA
リアルタイムAPI
High-volume reusers currently rely heavily on the changes that are pushed from our community to update their products in real time, using EventStream APIs to access such changes. High-volume reusers are interested in a service that will allow them to filter the changes they receive to limit their processing, guarantee stable HTTP connections to ensure no data loss, and supply a more useful schema to limit the number of api calls they need to make per event.
Enterprise Realtime API, at release, will contain:
- Filtering of events by Project or Revision Namespace
- Guaranteed connections
- Commercially useful schema similar* to those that we are building in our On-demand API and Snapshot API
- SLA
*We are still in the process of mapping out the technical specifications to determine the limitations of schema in event platforms and will post here when we have finalized our design.
スナップショットAPI
For high volume reusers that currently rely on the Wikimedia Dumps to access our information, we have created a solution to ingest Wikimedia content in near real time without excessive API calls (On-demand API) or maintaining hooks into our infrastructure (Realtime).
Enterprise Snapshot API, at release, will contain:
- 24-hour JSON[1], Wikitext, or HTML compressed dumps of "text-based" Wikimedia projects
- A hourly update file with revision changes of "text-based" Wikimedia projects
- SLA
- ↑ JSON dumps will contain the same schema per page as the On-demand API.
Access
There are several methods to obtain access to the Enterprise API datasets:
- Paid
- Realtime API (Batch and Streaming) and daily dump file in NDJSON format through the dedicated Enterprise API product website https://enterprise.wikimedia.com/
- Free
- Creating an account via the Enterprise API product website includes 10,000 on-demand API requests (including the Structured Contents endpoint) and monthly snapshot API file in NDJSON format at no cost.
- An update of the Enterprise API data is provided for all every two weeks at https://dumps.wikimedia.org/other/enterprise_html/
- Snapshot API + Realtime (Batch) via Data services, available to anyone with a Wikimedia cloud services account.
- Those who have a non-commercial and mission-relevant use-case, which cannot be fulfilled by existing free-access APIs/dumps etc, can request ongoing access to the paid service at no cost.
チーム
The Wikimedia Foundation staff who work specifically on this project:
Business
Product
-
Christopher Petrillo
Product Management -
Stephanie Delbecque
Product Management
Engineering
-
Ricardo Esquito
Software Engineering -
Luvo Dlulisa
Software Engineering -
Ruairi O'Donnell
Software Engineering -
Ehi Enabulele
Software Engineering
Program
Names in bold indicate management.
Many people from different teams also contribute significantly, including from the WMF Legal, Engineering, Partnerships, Design, Communications teams etc. Additional contract work provided by: Speed & Function are providing engineering support; PartnerHero provide customer support services; Vuurr are assisting our sales process; and Super Natural Design are the designers of the project website.
Governance
The board of the LLC overseeing the project are Ex officio from Wikimedia Foundation leadership, representing their Wikimedia Foundation staff roles. This includes the Chief Advancement Officer Lisa Seitz-Gruwell; General Counsel Stephen LaPorte; Chief Product and Technology Officer Selena Deckelman; and Lane Becker who serves as the LLC's president. The LLC is subject to the governance of the Wikimedia Foundation Board of Trustees as described at the Wikimedia Foundation Board Statement on Wikimedia Enterprise revenue principles.
All reports and official documents of the LLC are published on a dedicated Wikimedia Enterprise page on the Wikimedia Foundation Governance website. For convenience, annual reports are also linked here:
- 2022年 (calendar)
- 2022年-2023年 (fiscal)
Press
Commercial launch - October 2021 | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Wikimedia Foundation Press release Of particular note:
|
First customers - June 2022 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Press Release Of particular note:
|
関連項目
- API:Main page – MediaWiki Action API documentation
- Wikitech: Data Services portal – A list of community-facing services that allow for direct access to databases and dumps, as well as web interfaces for querying and programmatic access to data stores.
- Enterprise hub – a page for those interested in using the MediaWiki software in corporate contexts.
- MediaWiki Stakeholders group – an independent affiliate organisation that advocates for the needs of MediaWiki users outside the Wikimedia Foundation, including for-profit enterprises.
- Enterprise MediaWiki Conference – an independent conference series for that community.