Machine learning models/Production/Arabic Wikipedia article topic
Model card | |
---|---|
This page is an on-wiki machine learning model card. | |
Model Information Hub | |
Model creator(s) | Aaron Halfaker (User:EpochFail) and Amir Sarabadani |
Model owner(s) | WMF Machine Learning Team (ml@wikimediafoundation.org) |
Model interface | Ores homepage |
Code | drafttopic Github, ORES training data, and ORES model binaries |
Uses PII | No |
In production? | Yes |
Which projects? | Arabic Wikipedia |
This model uses article text to predict the likelihood that the article belongs to a set of topics. | |
Motivation
[edit]How can we predict what general topic an article is in? Answering this question is useful for various analyses of Wikipedia dynamics. However, it is difficult to group a very diverse range of Wikipedia articles into coherent, consistent topics manually.
This model, part of the ORES suite of models, analyzes an article to predict its likelihood of belonging to a set of topics. Similar models (though not necessarily with the same performance level or topics, are deployed across about a dozen other projects. There is also a language agnostic article topic model.
This model may be useful for high-level analyses of Wikipedia dynamics (pageviews, article quality, edit trends) and filtering articles.
Users and uses
[edit]- high-level analyses of Wikipedia dynamics such as pageview, article quality, or edit trends — e.g. How are pageview dynamics different between the physics and biology categories?
- filtering to relevant articles — e.g. filter articles only to those in the music category.
- definitively establishing what topic an article pertains to
- automated editing of articles or topics without a human in the loop
This model is a part of ORES, and generally accessible via API. It is used for high-level analysis of Wikipedia, platform research, and other on-wiki tasks.
Example API call:https://ores.wikimedia.org/v3/scores/arwiki/1234/articletopic
Ethical considerations, caveats, and recommendations
[edit]- This model was trained on data that is now several years old (from mid-2020). Underlying data drift may skew model outputs.
- This model uses word2vec as a training feature. Word2vec, like other natural language embeddings, encodes the linguistic biases of underlying datasets — along the lines of gender, race, ethnicity, religion etc. Since Wikipedia has known biases in its text, this model may encode and at times reproduce those biases.
- This model has highly variable performance across different topics — consult the test statistics below to get a sense of inter-topic performance.
Model
[edit]Performance
[edit]Test data confusion matrix:
Test data confusion matrix | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Test data sample rates:
Test data sample rates | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Test data performance:
Test data performance | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Implementation
[edit]Model architecture |
---|
{
"type": "GradientBoosting",
"params": {
"scale": false,
"center": false,
"labels": [
"Culture.Biography.Biography*",
"Culture.Biography.Women",
"Culture.Food and drink",
"Culture.Internet culture",
"Culture.Linguistics",
"Culture.Literature",
"Culture.Media.Books",
"Culture.Media.Entertainment",
"Culture.Media.Films",
"Culture.Media.Media*",
"Culture.Media.Music",
"Culture.Media.Radio",
"Culture.Media.Software",
"Culture.Media.Television",
"Culture.Media.Video games",
"Culture.Performing arts",
"Culture.Philosophy and religion",
"Culture.Sports",
"Culture.Visual arts.Architecture",
"Culture.Visual arts.Comics and Anime",
"Culture.Visual arts.Fashion",
"Culture.Visual arts.Visual arts*",
"Geography.Geographical",
"Geography.Regions.Africa.Africa*",
"Geography.Regions.Africa.Central Africa",
"Geography.Regions.Africa.Eastern Africa",
"Geography.Regions.Africa.Northern Africa",
"Geography.Regions.Africa.Southern Africa",
"Geography.Regions.Africa.Western Africa",
"Geography.Regions.Americas.Central America",
"Geography.Regions.Americas.North America",
"Geography.Regions.Americas.South America",
"Geography.Regions.Asia.Asia*",
"Geography.Regions.Asia.Central Asia",
"Geography.Regions.Asia.East Asia",
"Geography.Regions.Asia.North Asia",
"Geography.Regions.Asia.South Asia",
"Geography.Regions.Asia.Southeast Asia",
"Geography.Regions.Asia.West Asia",
"Geography.Regions.Europe.Eastern Europe",
"Geography.Regions.Europe.Europe*",
"Geography.Regions.Europe.Northern Europe",
"Geography.Regions.Europe.Southern Europe",
"Geography.Regions.Europe.Western Europe",
"Geography.Regions.Oceania",
"History and Society.Business and economics",
"History and Society.Education",
"History and Society.History",
"History and Society.Military and warfare",
"History and Society.Politics and government",
"History and Society.Society",
"History and Society.Transportation",
"STEM.Biology",
"STEM.Chemistry",
"STEM.Computing",
"STEM.Earth and environment",
"STEM.Engineering",
"STEM.Libraries & Information",
"STEM.Mathematics",
"STEM.Medicine & Health",
"STEM.Physics",
"STEM.STEM*",
"STEM.Space",
"STEM.Technology"
],
"multilabel": true,
"population_rates": null,
"ccp_alpha": 0.0,
"criterion": "friedman_mse",
"init": null,
"learning_rate": 0.1,
"loss": "deviance",
"max_depth": 5,
"max_features": "log2",
"max_leaf_nodes": null,
"min_impurity_decrease": 0.0,
"min_impurity_split": null,
"min_samples_leaf": 1,
"min_samples_split": 2,
"min_weight_fraction_leaf": 0.0,
"n_estimators": 150,
"n_iter_no_change": null,
"presort": "deprecated",
"random_state": null,
"subsample": 1.0,
"tol": 0.0001,
"validation_fraction": 0.1,
"verbose": 0,
"warm_start": false,
"label_weights": {}
}
}
|
Output schema |
---|
{
"title": "Scikit learn-based classifier score with probability",
"type": "object",
"properties": {
"prediction": {
"description": "The most likely labels predicted by the estimator",
"type": "array",
"items": {
"type": "string"
}
},
"probability": {
"description": "A mapping of probabilities onto each of the potential output labels",
"type": "object",
"properties": {
"Culture.Biography.Biography*": {
"type": "number"
},
"Culture.Biography.Women": {
"type": "number"
},
"Culture.Food and drink": {
"type": "number"
},
"Culture.Internet culture": {
"type": "number"
},
"Culture.Linguistics": {
"type": "number"
},
"Culture.Literature": {
"type": "number"
},
"Culture.Media.Books": {
"type": "number"
},
"Culture.Media.Entertainment": {
"type": "number"
},
"Culture.Media.Films": {
"type": "number"
},
"Culture.Media.Media*": {
"type": "number"
},
"Culture.Media.Music": {
"type": "number"
},
"Culture.Media.Radio": {
"type": "number"
},
"Culture.Media.Software": {
"type": "number"
},
"Culture.Media.Television": {
"type": "number"
},
"Culture.Media.Video games": {
"type": "number"
},
"Culture.Performing arts": {
"type": "number"
},
"Culture.Philosophy and religion": {
"type": "number"
},
"Culture.Sports": {
"type": "number"
},
"Culture.Visual arts.Architecture": {
"type": "number"
},
"Culture.Visual arts.Comics and Anime": {
"type": "number"
},
"Culture.Visual arts.Fashion": {
"type": "number"
},
"Culture.Visual arts.Visual arts*": {
"type": "number"
},
"Geography.Geographical": {
"type": "number"
},
"Geography.Regions.Africa.Africa*": {
"type": "number"
},
"Geography.Regions.Africa.Central Africa": {
"type": "number"
},
"Geography.Regions.Africa.Eastern Africa": {
"type": "number"
},
"Geography.Regions.Africa.Northern Africa": {
"type": "number"
},
"Geography.Regions.Africa.Southern Africa": {
"type": "number"
},
"Geography.Regions.Africa.Western Africa": {
"type": "number"
},
"Geography.Regions.Americas.Central America": {
"type": "number"
},
"Geography.Regions.Americas.North America": {
"type": "number"
},
"Geography.Regions.Americas.South America": {
"type": "number"
},
"Geography.Regions.Asia.Asia*": {
"type": "number"
},
"Geography.Regions.Asia.Central Asia": {
"type": "number"
},
"Geography.Regions.Asia.East Asia": {
"type": "number"
},
"Geography.Regions.Asia.North Asia": {
"type": "number"
},
"Geography.Regions.Asia.South Asia": {
"type": "number"
},
"Geography.Regions.Asia.Southeast Asia": {
"type": "number"
},
"Geography.Regions.Asia.West Asia": {
"type": "number"
},
"Geography.Regions.Europe.Eastern Europe": {
"type": "number"
},
"Geography.Regions.Europe.Europe*": {
"type": "number"
},
"Geography.Regions.Europe.Northern Europe": {
"type": "number"
},
"Geography.Regions.Europe.Southern Europe": {
"type": "number"
},
"Geography.Regions.Europe.Western Europe": {
"type": "number"
},
"Geography.Regions.Oceania": {
"type": "number"
},
"History and Society.Business and economics": {
"type": "number"
},
"History and Society.Education": {
"type": "number"
},
"History and Society.History": {
"type": "number"
},
"History and Society.Military and warfare": {
"type": "number"
},
"History and Society.Politics and government": {
"type": "number"
},
"History and Society.Society": {
"type": "number"
},
"History and Society.Transportation": {
"type": "number"
},
"STEM.Biology": {
"type": "number"
},
"STEM.Chemistry": {
"type": "number"
},
"STEM.Computing": {
"type": "number"
},
"STEM.Earth and environment": {
"type": "number"
},
"STEM.Engineering": {
"type": "number"
},
"STEM.Libraries & Information": {
"type": "number"
},
"STEM.Mathematics": {
"type": "number"
},
"STEM.Medicine & Health": {
"type": "number"
},
"STEM.Physics": {
"type": "number"
},
"STEM.STEM*": {
"type": "number"
},
"STEM.Space": {
"type": "number"
},
"STEM.Technology": {
"type": "number"
}
}
}
}
}
|
https://ores.wikimedia.org/v3/scores/arwiki/1234/articletopic
Output:
Example output |
---|
{
"arwiki": {
"models": {
"articletopic": {
"version": "1.3.0"
}
},
"scores": {
"1234": {
"articletopic": {
"score": {
"prediction": [
"Culture.Philosophy and religion"
],
"probability": {
"Culture.Biography.Biography*": 0.0018775771968361505,
"Culture.Biography.Women": 0.0022559171278080403,
"Culture.Food and drink": 0.02911020063825202,
"Culture.Internet culture": 0.00013396466832571897,
"Culture.Linguistics": 0.0002016623350064864,
"Culture.Literature": 0.01630206474244034,
"Culture.Media.Books": 0.0019739903099149016,
"Culture.Media.Entertainment": 0.005776333050595081,
"Culture.Media.Films": 0.0005357906321037945,
"Culture.Media.Media*": 0.01873311881808571,
"Culture.Media.Music": 0.0006598789501761831,
"Culture.Media.Radio": 0.0008711772320624666,
"Culture.Media.Software": 0.00045306840360182074,
"Culture.Media.Television": 0.004563663906527891,
"Culture.Media.Video games": 0.00014238944031148178,
"Culture.Performing arts": 0.00031857030709468275,
"Culture.Philosophy and religion": 0.9915133947766572,
"Culture.Sports": 0.0032536645394025034,
"Culture.Visual arts.Architecture": 0.003171696122120992,
"Culture.Visual arts.Comics and Anime": 0.0030041266177882604,
"Culture.Visual arts.Fashion": 0.008690660472179179,
"Culture.Visual arts.Visual arts*": 0.015999157238911713,
"Geography.Geographical": 0.0059701879622301715,
"Geography.Regions.Africa.Africa*": 0.20140713747028768,
"Geography.Regions.Africa.Central Africa": 0.0005659981357134575,
"Geography.Regions.Africa.Eastern Africa": 0.004211961235703654,
"Geography.Regions.Africa.Northern Africa": 0.012708751878181768,
"Geography.Regions.Africa.Southern Africa": 0.0006662932204457024,
"Geography.Regions.Africa.Western Africa": 0.0005140083680583202,
"Geography.Regions.Americas.Central America": 0.0016675740255396078,
"Geography.Regions.Americas.North America": 0.0062834206734540595,
"Geography.Regions.Americas.South America": 0.0006909355854571832,
"Geography.Regions.Asia.Asia*": 0.01339558309316005,
"Geography.Regions.Asia.Central Asia": 0.0006353555918342474,
"Geography.Regions.Asia.East Asia": 0.001418334017189674,
"Geography.Regions.Asia.North Asia": 0.000575145968948881,
"Geography.Regions.Asia.South Asia": 0.003160871781527632,
"Geography.Regions.Asia.Southeast Asia": 0.002178204210680102,
"Geography.Regions.Asia.West Asia": 0.014274257947124423,
"Geography.Regions.Europe.Eastern Europe": 0.0013954670948905467,
"Geography.Regions.Europe.Europe*": 0.011614035833480698,
"Geography.Regions.Europe.Northern Europe": 0.0018661121742356134,
"Geography.Regions.Europe.Southern Europe": 0.007193423419921128,
"Geography.Regions.Europe.Western Europe": 0.0034747000735290425,
"Geography.Regions.Oceania": 0.0014825584617553515,
"History and Society.Business and economics": 0.009040864250628715,
"History and Society.Education": 0.0034818655626133728,
"History and Society.History": 0.015335759410860367,
"History and Society.Military and warfare": 0.0030978850596784535,
"History and Society.Politics and government": 0.029165387410026597,
"History and Society.Society": 0.031041994032148346,
"History and Society.Transportation": 0.00022066416987785007,
"STEM.Biology": 0.005349680305325293,
"STEM.Chemistry": 0.0027381500340171645,
"STEM.Computing": 0.0006550726961037265,
"STEM.Earth and environment": 0.0008943785135649393,
"STEM.Engineering": 0.0013233564634350919,
"STEM.Libraries & Information": 7.807094663224667e-05,
"STEM.Mathematics": 0.002929987189174986,
"STEM.Medicine & Health": 0.061891530372885306,
"STEM.Physics": 0.0026454984695803902,
"STEM.STEM*": 0.098436433397069,
"STEM.Space": 0.001050419037453933,
"STEM.Technology": 0.004580541565463146
}
}
}
}
}
}
}
|
Data
[edit]Licenses
[edit]- Code: MIT license
- Model: MIT license
Citation
[edit]Cite this model card as:
@misc{
Triedman_Bazira_2023_Arabic_Wikipedia_article_topic,
title={ Arabic Wikipedia article topic model card },
author={ Triedman, Harold and Bazira, Kevin },
year={ 2023 },
url={ https://meta.wikimedia.org/wiki/Machine_learning_models/Production/Arabic_Wikipedia_article_topic }
}