Jump to content

Picturesque

From Meta, a Wikimedia project coordination wiki

Picturesque is a comprehensive web-based tool suite hosted at Picturesque Toolforge that facilitates image management, discovery, and contribution across the Wikimedia ecosystem (Wikimedia Commons, Wikidata, and Wikipedia).

The platform provides a unified interface for photographers, contributors, and GLAM (Galleries, Libraries, Archives, and Museums) institutions to work with freely licensed educational images.

System Architecture

[edit]

Core Components

[edit]

The Picturesque platform is organized into seven primary functional modules:

  • Image Creation Tools
  • WikiVerse Image Management
  • Third-Party Image Import
  • Image Discovery and Mapping
  • Maintenance and Cleanup
  • GLAM Analytics
  • Metadata Management

Technical Requirements

[edit]
  • User Account: Wikimedia Commons and/or Wikidata account required for most operations
  • Authentication: OAuth integration for tool permissions
  • Browser: Modern web browser with JavaScript enabled
  • Network: Stable internet connection for API calls and image transfers

Module 1: Image Creation Tools

[edit]

WikiShootMe

[edit]

Purpose: Identify nearby Wikidata items lacking images and facilitate on-location photography.

Functionality:

  • Geographic search based on user location or specified coordinates
  • Item-specific search by Wikidata ID or name
  • Real-time filtering of items without images
  • Direct upload interface to Commons

Use Cases:

  • Street photography for monuments and buildings
  • Documenting local landmarks
  • Targeted photography missions

API Integration:

  • Wikidata Query Service (SPARQL)
  • Wikimedia Commons Upload API
  • Geolocation services

PictureThis

[edit]

Purpose: Search and upload images for specific Wikidata items.

Features:

  • Full-text search across Wikidata items
  • Proximity-based discovery
  • Image upload wizard
  • Automatic metadata population

Technical Workflow:

  1. Query Wikidata for items matching search criteria
  2. Filter results by image presence (P18 property)
  3. Present upload interface with pre-filled metadata
  4. Upload to Commons via API
  5. Update Wikidata item with image property

Google Drive Integration (Gdrive)

[edit]

Status: Potentially deprecated/unstable

Purpose: Bulk import from personal Google Drive storage.

Technical Notes:

  • Requires Google OAuth authorization
  • Direct API bridge between Google Drive and Commons
  • May require manual metadata entry
  • Check tool status before use

Module 2: Images in the WikiVerse

[edit]

WD-FIST (Wikidata – Find Images for Similar Topics)

[edit]

Purpose: Cross-language image discovery for Wikidata items.

Algorithm:

  1. Accept input list of Wikidata items or Wikipedia articles
  2. Query all language editions of Wikipedia
  3. Extract images used in articles about the same subject
  4. Filter for Commons-hosted images
  5. Present candidates for addition to Wikidata

Input Methods:

  • Direct Wikidata item IDs (Q-numbers)
  • Wikipedia article titles
  • PetScan query results
  • Manual lists

Output:

  • Candidate images with usage statistics
  • One-click addition to Wikidata items
  • Batch processing capabilities

File Candidates

[edit]

Purpose: Topical image discovery and transfer from Flickr and Commons.

Supported Categories:

  • People (biographical items)
  • Species (taxonomic items)
  • Buildings and structures
  • Geographic locations
  • Cultural artifacts

Workflow:

  1. Select topic category
  2. Browse items without images
  3. View candidate images from Commons/Flickr
  4. Transfer Flickr images to Commons (if needed)
  5. Add images to Wikidata items

Technical Components:

  • Flickr API integration
  • Commons API for image search
  • Wikidata edit API
  • License verification system

WD4WP (Wikidata for Wikipedia)

[edit]

Purpose: Bridge the gap between image-rich Wikidata and image-poor Wikipedia articles.

Detection Logic:

  • Scans Wikipedia articles without images
  • Checks corresponding Wikidata items for P18 property
  • Evaluates image suitability for article context
  • Provides one-click insertion

Parameters:

  • Language selection
  • Article category filtering
  • Quality thresholds
  • License compatibility checks

FIST (Legacy Tool)

[edit]

Purpose: Multi-source image discovery for Wikipedia articles.

Data Sources:

  • Sister Wikipedia language editions
  • Wikimedia Commons categories
  • Third-party repositories (where applicable)

Note: Older tool with potentially limited maintenance. Consider using WD-FIST for newer implementations.

Module 3: Third-Party Image Import

[edit]

License Verification Framework

[edit]

All import tools implement mandatory license checks:

  • Creative Commons licenses (CC0, CC-BY, CC-BY-SA)
  • Public Domain declarations
  • Government works (jurisdiction-specific)
  • Explicit permission documentation

Flickr2Commons

[edit]

Purpose: Single image or collection import from Flickr.

Features:

  • License auto-detection
  • Metadata preservation (EXIF, IPTC)
  • Bulk collection transfer
  • Author attribution tracking

Technical Process:

  1. Authenticate with Flickr API
  2. Verify license compatibility
  3. Download high-resolution version
  4. Generate Commons metadata template
  5. Upload with proper categorization

Supported Licenses:

  • CC0 (Public Domain)
  • CC-BY 2.0/3.0/4.0
  • CC-BY-SA 2.0/3.0/4.0
  • US Government Works

FlickrFree

[edit]

Purpose: Stream monitoring for newly uploaded free images on Flickr. Update Frequency: Real-time or near-real-time

Use Case: Rapid acquisition of timely images (events, news, nature)

Caution: Manual review required for Commons suitability:

  • Educational value assessment
  • Out-of-scope filtering (personal photos)
  • Quality evaluation
  • Duplicate checking

Mapillary to Commons

[edit]

Purpose: Extract street-level photography from Mapillary platform.

Technical Specifications:

  • Supports Mapillary's CC-BY-SA license
  • Geolocation data preservation
  • Image sequence extraction
  • Panorama support

Ideal For:

  • Street furniture documentation
  • Architectural surveys
  • Transportation infrastructure
  • Urban planning documentation

Space Agency Import Tool

[edit]

Purpose: Bulk import from space agency repositories.

Supported Agencies:

  • NASA (Public Domain for US-produced content)
  • ESA (varies by mission)
  • JAXA (varies by mission)
  • Roscosmos (varies)

Metadata Handling:

  • Mission information
  • Instrument data
  • Capture date and time
  • Object identification
  • Astronomical coordinates

Twitter to Commons

[edit]

Purpose: Import openly licensed images from Twitter/X.

Requirements:

  • Explicit license statement in tweet or profile
  • Author authorization verification
  • High-resolution version availability

Limitations: Requires manual license verification due to Twitter's default copyright.

Geograph to Commons

[edit]

Purpose: Import from Geograph geographic photo collections.

Coverage: Primarily UK, Ireland, Germany

Features:

  • Grid reference preservation
  • Geographic categorization
  • Photographer attribution
  • License compatibility checking

CommonsHelper

[edit]

Purpose: Transfer freely licensed images from local Wikipedia projects to Commons.

Rationale:

  • Centralize media storage
  • Enable cross-project usage
  • Preserve upload history
  • Maintain attribution

Process:

  1. Identify local files with free licenses
  2. Check for Commons duplicates
  3. Transfer with complete history
  4. Update local file pages
  5. Replace local usage with Commons link

URL to Commons

[edit]

Purpose: Universal import tool for any web-accessible image.

Warning: Highest risk for license violations.

Mandatory Steps:

  1. Verify explicit free license on source page
  2. Document license evidence
  3. Provide source URL
  4. Confirm author identity
  5. Add detailed permission information

Best Practice: Use specialized tools when available instead of URL importer.

Module 4: Existing and Missing Images

[edit]

WikiMap

[edit]

Purpose: Geographic visualization of image coverage gaps.

Visualization Types:

  • Heatmaps of coverage density
  • Pin maps of missing images
  • Category-filtered views
  • Time-series analysis

Export Formats:

  • GeoJSON
  • KML
  • CSV with coordinates
  • Static map images

Use Cases:

  • Planning photography expeditions
  • Identifying underrepresented regions
  • Coordinating community photo walks
  • GLAM partnership targeting

CommonsCoverage

[edit]

Purpose: Global overview of Commons geographic coverage.

Metrics:

  • Images per region
  • Geographic distribution
  • Category coverage
  • Temporal trends

Analysis Features:

  • Comparative regional statistics
  • Category-specific maps
  • Quality assessment overlays

Wikinity

[edit]

Purpose: Alternative mapping interface for image gaps.

Distinguishing Features:

  • 3D terrain visualization
  • Mobile-optimized interface
  • Offline map caching
  • Route planning for photo missions

Bldrwnsch

[edit]

Purpose: Process German Wikipedia image requests.

Functionality:

  • Aggregates requests from de.wikipedia.org
  • Categorizes by topic and location
  • Tracks fulfillment status
  • Links to requester discussions

Language: Primarily German interface

Module 5: Maintenance Tools

[edit]

Commons Image Feed

[edit]

Purpose: RSS/Atom feed generation for category monitoring.

Configuration Options:

  • Category selection
  • Update frequency
  • Image size in feed
  • Metadata inclusion level

Applications:

  • Blog integration
  • External monitoring systems
  • Automated processing pipelines
  • Quality control workflows

Commons-aiuser

[edit]

Purpose: Track contributions from individual uploaders.

Metrics:

  • Upload history
  • Category distribution
  • Quality ratings
  • License usage patterns

Use Cases:

  • Contributor recognition
  • Quality assessment
  • Potential copyright issues
  • Upload pattern analysis

File Siblings

[edit]

Purpose: Detect similar or related images.

Technology:

  • Perceptual hashing
  • Visual similarity algorithms
  • Metadata comparison
  • Filename pattern matching

Applications:

  • Duplicate detection (near-duplicates)
  • Image series identification
  • Quality comparison (keep best version)
  • Derivative work tracking

File Duplicates

[edit]

Purpose: Exact duplicate detection.

Method: Cryptographic hash comparison (MD5, SHA-1)

Actions:

  • Flag for deletion
  • Merge metadata
  • Redirect usage to primary file
  • Update file descriptions

Bulk Download

[edit]

Purpose: Batch download from Commons.

Parameters:

  • Category selection
  • File type filtering
  • Date range
  • Size limitations
  • Maximum file count

Use Cases:

  • Offline processing
  • Machine learning datasets
  • Backup operations
  • External analysis

Unused Files

[edit]

Purpose: Identify Commons files not used on any Wikimedia project.

Detection Criteria:

  • No transclusions on Wikipedia
  • No Wikidata usage (P18, P154, P41, etc.)
  • No sister project usage
  • Age threshold (e.g., >30 days)

Actions:

  • Propose for deletion
  • Improve categorization
  • Add to relevant galleries
  • Link to Wikidata items

Multi-language Descriptions

[edit]

Purpose: Generate localized descriptions for Commons files.

Process:

  1. Extract existing description
  2. Machine translate to target languages
  3. Apply language-specific templates
  4. Present for human review
  5. Bulk update file pages

Supported Languages: All major Wikipedia languages

Add Information Template

[edit]

Purpose: Apply {{Information}} template to newly uploaded files.

Template Fields:

  • Description
  • Source
  • Date
  • Author
  • Permission
  • Other versions
  • Categories

Automation Level: Semi-automated with manual review

No Information Tool

[edit]

Purpose: Find files lacking proper {{Information}} template.

Prioritization:

  • Upload date (oldest first)
  • Usage frequency (most used first)
  • Category importance
  • Uploader activity (active users first)

Usual Suspects

[edit]

Purpose: Identify common cleanup issues.

Detection Categories:

  • Copyright violations
  • Low quality images
  • Improperly categorized files
  • Duplicate uploads
  • Misleading filenames
  • Missing source information

Module 6: GLAM Tools

[edit]

BaGLAMa

[edit]

Purpose: Measure reach and impact of GLAM partnerships.

Metrics:

  • Page views for articles using GLAM images
  • Geographic distribution of viewers
  • Temporal trends
  • Comparative analysis across collections

Reporting Features:

  • Monthly/annual reports
  • CSV export
  • Visualization graphs
  • Top articles ranking

Use Cases:

  • Partnership reporting
  • Grant applications
  • Impact assessment
  • Community engagement metrics

GLAMorous

[edit]

Purpose: Track usage of GLAM images across Wikimedia projects.

Analysis Dimensions:

  • Project distribution (Wikipedia, Wiktionary, etc.)
  • Language distribution
  • Article topic categories
  • Usage context (infobox, gallery, article body)

Output Formats:

  • Interactive web dashboard
  • Spreadsheet export
  • JSON API
  • Embeddable widgets

TreeViews

[edit]

Purpose: Monthly view statistics for category hierarchies.

Features:

  • Recursive category traversal
  • Aggregated statistics
  • Month-over-month comparison
  • Subcategory breakdown

Performance: Handles large category trees efficiently through caching

GLAMorgan

[edit]

Purpose: BaGLAMa variant with enhanced customization.

Additional Features:

  • Custom date ranges
  • Multiple category comparison
  • Advanced filtering options
  • Automated email reports

Module 7: Metadata Management

[edit]

Image Annotator

[edit]

Purpose: Create and edit structured image annotations.

Annotation Types:

  • Rectangular regions
  • Named entities (people, objects, locations)
  • Wikidata entity linking
  • Descriptive text

Technical Implementation:

  • Canvas-based drawing interface
  • JSON-LD metadata format
  • Wikidata entity search integration
  • Revision history tracking

Use Cases:

  • People identification in photographs
  • Object tagging for search
  • Educational annotations
  • Accessibility descriptions

Commons Mass Description

[edit]

Purpose: Bulk metadata editing for multiple files.

Workflow:

  1. Select files by category, uploader, or list
  2. Define template or pattern
  3. Preview changes
  4. Apply batch edit
  5. Monitor edit queue

Safety Features:

  • Preview mode
  • Rollback capability
  • Edit conflict detection
  • Rate limiting

Supported Operations:

  • Description updates
  • Category additions/removals
  • Template parameter changes
  • License updates

Locator Tool

[edit]

Purpose: Add or correct geographic coordinates for images.

Interface:

  • Interactive map (OpenStreetMap base)
  • Drag-and-drop positioning
  • Coordinate entry (decimal degrees, DMS)
  • Bulk processing for image series

Metadata Output:

  • {{Location}} template
  • Structured data (P1259 coordinates)
  • Heading direction (for directional photos)
  • Altitude information

Integration: Writes directly to Commons file pages and Wikidata

Module 8: Query Tools

[edit]

Pre-filled Query System

[edit]

Purpose: Provide ready-to-use queries for common tasks.

Example: Wikipedia Requested Photographs

Query Chain:

  1. PetScan identifies pages in request categories
  2. Extracts Wikidata items from article links
  3. WDFIST finds candidate images
  4. Presents results with upload/add options

Customization: Users can modify parameters while preserving query logic

SPARQL Query Builder

[edit]

Purpose: Construct custom Wikidata queries for missing images.

Interface Components:

  • Type (P31) Selector: Dropdown or autocomplete for item classes
  • Multiple type selection (AND/OR logic)
  • Subclass traversal options
  • Location Filter (Optional): Geographic constraint (within radius), administrative division, or coordinates input

Generated Query Structure:

SELECT ?item ?itemLabel WHERE {
  ?item wdt:P31 wd:Q... .          # Type constraint
  MINUS { ?item wdt:P18 ?image }   # No image exists
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en" }
}

Tool Integration: Launch results directly in:

  • Wikidata Query Service
  • WikiShootMe
  • File Candidates
  • Custom tools via API

Authentication and Permissions

[edit]

OAuth Implementation

[edit]

Authorization Flow:

  1. User initiates tool action requiring permissions
  2. Redirect to Wikimedia OAuth consent page
  3. User grants specific permissions
  4. Tool receives access token
  5. Token stored for session (not persistent)

Permission Scopes:

  • Read user information
  • Edit Wikidata items
  • Upload files to Commons
  • Edit Commons file pages

Security Best Practices:

  • Tokens expire after session
  • Per-tool authorization (granular permissions)
  • Explicit user consent for each scope
  • Audit log of tool actions

Account Requirements

[edit]

Some tools may require minimum account age or edit count:

  • Account age: 4+ days
  • Edit count: 10+ edits
  • Autoconfirmed status

Rationale: Prevent abuse and spam

API Architecture

[edit]

Wikidata Query Service

[edit]

Endpoint: https://query.wikidata.org/sparql

Rate Limits:

  • 60 seconds maximum query execution
  • Concurrent query limits per IP
  • Result size limits

Optimization Strategies:

  • Use SERVICE wikibase:label efficiently
  • Limit result sets with LIMIT clause
  • Add timeouts to prevent hanging queries

Wikimedia Commons API

[edit]

Base URL: https://commons.wikimedia.org/w/api.php

Key Endpoints:

  • action=upload: File upload
  • action=edit: File page editing
  • action=query&prop=imageinfo: Image metadata
  • action=categorymembers: Category contents

Rate Limiting:

  • Strict limits for anonymous users
  • Higher limits for logged-in users
  • Bot flags for approved high-volume operations

MediaWiki Action API

[edit]

Documentation: API:Main page

Common Parameters:

  • format=json: Response format
  • action=: Specify API action
  • token=: CSRF token for edits

Best Practices

[edit]

Metadata Consistency

[edit]

Goal: Ensure metadata quality and uniformity across Wikimedia Commons and Wikidata.

Guidelines:

  • Always use standard templates ({{Information}}, {{Location}}, {{License}})
  • Prefer structured data statements over unstructured text
  • Maintain consistent date formats (ISO 8601)
  • Use canonical Wikidata items for locations, authors, and subjects
  • Add multilingual labels and descriptions when possible

File Naming

[edit]

Principles:

  • Use descriptive, human-readable filenames
  • Avoid non-ASCII characters unless required
  • Use underscores instead of spaces
  • Include disambiguation where necessary (e.g., “Paris_Bridge_2023.jpg”)
  • Avoid version numbers; rely on file history

Licensing and Attribution

[edit]

Best Practices:

  • Verify that all uploaded media comply with Commons’ licensing policy
  • Provide full attribution for authors and sources
  • Prefer Creative Commons licenses (CC BY, CC BY-SA)
  • Ensure license templates are correctly formatted and categorized

Categorization

[edit]

Recommendations:

  • Place files in the most specific categories available
  • Avoid redundant parent categories
  • Ensure every file has at least one topical and one geographic category
  • Use Commons’ category tree navigation for validation

Tool Interaction

[edit]

Workflow Integration:

  • Validate data consistency when passing between tools (e.g., PetScan → WD-FIST → Commons)
  • Use the “preview” or “dry-run” mode before committing edits
  • Check tool logs for error reports
  • Avoid parallel batch edits to reduce conflict risk

Troubleshooting

[edit]

Common Issues

[edit]
Issue Cause Resolution
Missing OAuth permission Token expired or not granted Reauthorize via OAuth login
API rate limit reached Too many rapid requests Add delay between requests or use bot account
Incorrect category results Cached PetScan data Clear cache or modify query parameters
File upload failure Network or API timeout Retry after short delay or check file size
Duplicated metadata Concurrent edits Use rollback or re-sync tool

Debugging Tools

[edit]
  • Commons API Sandbox: Test API requests interactively
  • WDQS Interface: Validate SPARQL queries
  • Toolforge Logs: Check tool runtime errors
  • Wikidata RecentChanges: Monitor structured data updates

Reporting Bugs

[edit]
  • Submit issues to the tool’s Phabricator page
  • Include reproduction steps, timestamps, and screenshots
  • Mention related API queries or OAuth scopes
  • Tag with appropriate project (e.g., Commons-tools, Wikidata)

Performance Optimization

[edit]

Query Optimization

[edit]
  • Limit SELECT variables to required fields
  • Use filters (FILTER()) early in query chains
  • Use subqueries for large datasets
  • Avoid regex filters on labels when possible
  • Use Wikibase “hint:Prior” for faster entity retrieval

API Efficiency

[edit]
  • Use generator parameters for multi-page requests
  • Request JSON format for compact responses
  • Implement caching for static metadata
  • Use batch editing endpoints
  • Respect API retry-after headers

Tool Integration Optimization

[edit]
  • Minimize data redundancy between tools
  • Schedule heavy queries during off-peak hours
  • Use asynchronous or queued processing
  • Share intermediate results through temporary storage

Data Formats

[edit]

Structured Data on Commons (SDC)

[edit]

Schema Overview:

  • Entity-based (Mediainfo namespace)
  • Each statement has a property (e.g., P180) and a value
  • Supports qualifiers and references
  • Fully compatible with Wikidata entities

Common Properties:

Property Meaning
P180 Depicts
P571 Date of creation
P170 Creator
P275 License
P1259 Coordinates
P7482 Digital representation of

JSON Format

[edit]

Used for:

  • API responses
  • Structured data export
  • Tool interoperability

Structure Example:

{
  "title": "File:Example.jpg",
  "metadata": {
    "author": "John Doe",
    "license": "CC BY-SA 4.0",
    "depicts": ["Q42", "Q146"]
  }
}

CSV Format

[edit]

Usage:

  • Bulk export for analysis
  • GLAM reporting
  • Category metrics

Columns Typically Include:

  • File name
  • URL
  • Upload date
  • Categories
  • Views
  • Description

Advanced Features

[edit]

Automation Pipelines

[edit]

Concept: Connect tools for automated workflows.

Example Pipeline:

  1. PetScan selects target items
  2. WD-FIST suggests image matches
  3. QuickStatements adds image links to Wikidata
  4. GLAMorgan tracks usage metrics

Advantages:

  • Time-saving automation
  • Reduced human error
  • Repeatable workflows

API Integration via Toolforge

[edit]

Purpose: Unified hosting and API gateway for Wikimedia tools.

Capabilities:

  • Shared database access
  • OAuth management
  • Scheduler for recurring jobs
  • Secure data handling

Batch Processing Scripts

[edit]

Tools:

  • Pywikibot
  • CommonsHelper
  • QuickStatements API

Example Workflow:

for file in commons_files:
    upload_to_commons(file)
    add_metadata(file)
    link_to_wikidata(file)

Community Integration

[edit]

Collaboration Channels

[edit]

User Roles

[edit]
Role Responsibilities
Uploader Contributes media to Commons
Curator Reviews metadata and categories
Bot Operator Maintains large-scale automation
GLAM Partner Provides institutional media collections
Developer Builds and maintains tool infrastructure

Governance and Ethics

[edit]
  • Ensure transparency in automated edits
  • Respect copyright and attribution standards
  • Avoid biased data selection
  • Encourage open collaboration
  • Report misuse or policy violations promptly

Conclusion

[edit]

Summary: The Wikimedia Commons and Wikidata ecosystems rely on a broad suite of interconnected tools that facilitate media management, metadata curation, and knowledge linking. From uploaders to GLAM professionals, contributors can leverage these utilities to improve discoverability, accuracy, and cultural representation of digital assets.

Key Takeaways:

  • Integration between Commons and Wikidata enhances media value
  • Automation tools streamline repetitive tasks
  • Community oversight ensures data quality
  • Open APIs enable innovation and collaboration


Final Note: This documentation should serve as both a reference and a roadmap for Wikimedia contributors seeking to understand and optimize the tools that power a global open media repository.