Picturesque
Picturesque is a comprehensive web-based tool suite hosted at Picturesque Toolforge that facilitates image management, discovery, and contribution across the Wikimedia ecosystem (Wikimedia Commons, Wikidata, and Wikipedia).
The platform provides a unified interface for photographers, contributors, and GLAM (Galleries, Libraries, Archives, and Museums) institutions to work with freely licensed educational images.
System Architecture
[edit]Core Components
[edit]The Picturesque platform is organized into seven primary functional modules:
- Image Creation Tools
- WikiVerse Image Management
- Third-Party Image Import
- Image Discovery and Mapping
- Maintenance and Cleanup
- GLAM Analytics
- Metadata Management
Technical Requirements
[edit]- User Account: Wikimedia Commons and/or Wikidata account required for most operations
- Authentication: OAuth integration for tool permissions
- Browser: Modern web browser with JavaScript enabled
- Network: Stable internet connection for API calls and image transfers
Module 1: Image Creation Tools
[edit]WikiShootMe
[edit]Purpose: Identify nearby Wikidata items lacking images and facilitate on-location photography.
Functionality:
- Geographic search based on user location or specified coordinates
- Item-specific search by Wikidata ID or name
- Real-time filtering of items without images
- Direct upload interface to Commons
Use Cases:
- Street photography for monuments and buildings
- Documenting local landmarks
- Targeted photography missions
API Integration:
- Wikidata Query Service (SPARQL)
- Wikimedia Commons Upload API
- Geolocation services
PictureThis
[edit]Purpose: Search and upload images for specific Wikidata items.
Features:
- Full-text search across Wikidata items
- Proximity-based discovery
- Image upload wizard
- Automatic metadata population
Technical Workflow:
- Query Wikidata for items matching search criteria
- Filter results by image presence (P18 property)
- Present upload interface with pre-filled metadata
- Upload to Commons via API
- Update Wikidata item with image property
Google Drive Integration (Gdrive)
[edit]Status: Potentially deprecated/unstable
Purpose: Bulk import from personal Google Drive storage.
Technical Notes:
- Requires Google OAuth authorization
- Direct API bridge between Google Drive and Commons
- May require manual metadata entry
- Check tool status before use
Module 2: Images in the WikiVerse
[edit]WD-FIST (Wikidata – Find Images for Similar Topics)
[edit]Purpose: Cross-language image discovery for Wikidata items.
Algorithm:
- Accept input list of Wikidata items or Wikipedia articles
- Query all language editions of Wikipedia
- Extract images used in articles about the same subject
- Filter for Commons-hosted images
- Present candidates for addition to Wikidata
Input Methods:
- Direct Wikidata item IDs (Q-numbers)
- Wikipedia article titles
- PetScan query results
- Manual lists
Output:
- Candidate images with usage statistics
- One-click addition to Wikidata items
- Batch processing capabilities
File Candidates
[edit]Purpose: Topical image discovery and transfer from Flickr and Commons.
Supported Categories:
- People (biographical items)
- Species (taxonomic items)
- Buildings and structures
- Geographic locations
- Cultural artifacts
Workflow:
- Select topic category
- Browse items without images
- View candidate images from Commons/Flickr
- Transfer Flickr images to Commons (if needed)
- Add images to Wikidata items
Technical Components:
- Flickr API integration
- Commons API for image search
- Wikidata edit API
- License verification system
WD4WP (Wikidata for Wikipedia)
[edit]Purpose: Bridge the gap between image-rich Wikidata and image-poor Wikipedia articles.
Detection Logic:
- Scans Wikipedia articles without images
- Checks corresponding Wikidata items for P18 property
- Evaluates image suitability for article context
- Provides one-click insertion
Parameters:
- Language selection
- Article category filtering
- Quality thresholds
- License compatibility checks
FIST (Legacy Tool)
[edit]Purpose: Multi-source image discovery for Wikipedia articles.
Data Sources:
- Sister Wikipedia language editions
- Wikimedia Commons categories
- Third-party repositories (where applicable)
Note: Older tool with potentially limited maintenance. Consider using WD-FIST for newer implementations.
Module 3: Third-Party Image Import
[edit]License Verification Framework
[edit]All import tools implement mandatory license checks:
- Creative Commons licenses (CC0, CC-BY, CC-BY-SA)
- Public Domain declarations
- Government works (jurisdiction-specific)
- Explicit permission documentation
Flickr2Commons
[edit]Purpose: Single image or collection import from Flickr.
Features:
- License auto-detection
- Metadata preservation (EXIF, IPTC)
- Bulk collection transfer
- Author attribution tracking
Technical Process:
- Authenticate with Flickr API
- Verify license compatibility
- Download high-resolution version
- Generate Commons metadata template
- Upload with proper categorization
Supported Licenses:
- CC0 (Public Domain)
- CC-BY 2.0/3.0/4.0
- CC-BY-SA 2.0/3.0/4.0
- US Government Works
FlickrFree
[edit]Purpose: Stream monitoring for newly uploaded free images on Flickr. Update Frequency: Real-time or near-real-time
Use Case: Rapid acquisition of timely images (events, news, nature)
Caution: Manual review required for Commons suitability:
- Educational value assessment
- Out-of-scope filtering (personal photos)
- Quality evaluation
- Duplicate checking
Mapillary to Commons
[edit]Purpose: Extract street-level photography from Mapillary platform.
Technical Specifications:
- Supports Mapillary's CC-BY-SA license
- Geolocation data preservation
- Image sequence extraction
- Panorama support
Ideal For:
- Street furniture documentation
- Architectural surveys
- Transportation infrastructure
- Urban planning documentation
Space Agency Import Tool
[edit]Purpose: Bulk import from space agency repositories.
Supported Agencies:
- NASA (Public Domain for US-produced content)
- ESA (varies by mission)
- JAXA (varies by mission)
- Roscosmos (varies)
Metadata Handling:
- Mission information
- Instrument data
- Capture date and time
- Object identification
- Astronomical coordinates
Twitter to Commons
[edit]Purpose: Import openly licensed images from Twitter/X.
Requirements:
- Explicit license statement in tweet or profile
- Author authorization verification
- High-resolution version availability
Limitations: Requires manual license verification due to Twitter's default copyright.
Geograph to Commons
[edit]Purpose: Import from Geograph geographic photo collections.
Coverage: Primarily UK, Ireland, Germany
Features:
- Grid reference preservation
- Geographic categorization
- Photographer attribution
- License compatibility checking
CommonsHelper
[edit]Purpose: Transfer freely licensed images from local Wikipedia projects to Commons.
Rationale:
- Centralize media storage
- Enable cross-project usage
- Preserve upload history
- Maintain attribution
Process:
- Identify local files with free licenses
- Check for Commons duplicates
- Transfer with complete history
- Update local file pages
- Replace local usage with Commons link
URL to Commons
[edit]Purpose: Universal import tool for any web-accessible image.
Warning: Highest risk for license violations.
Mandatory Steps:
- Verify explicit free license on source page
- Document license evidence
- Provide source URL
- Confirm author identity
- Add detailed permission information
Best Practice: Use specialized tools when available instead of URL importer.
Module 4: Existing and Missing Images
[edit]WikiMap
[edit]Purpose: Geographic visualization of image coverage gaps.
Visualization Types:
- Heatmaps of coverage density
- Pin maps of missing images
- Category-filtered views
- Time-series analysis
Export Formats:
- GeoJSON
- KML
- CSV with coordinates
- Static map images
Use Cases:
- Planning photography expeditions
- Identifying underrepresented regions
- Coordinating community photo walks
- GLAM partnership targeting
CommonsCoverage
[edit]Purpose: Global overview of Commons geographic coverage.
Metrics:
- Images per region
- Geographic distribution
- Category coverage
- Temporal trends
Analysis Features:
- Comparative regional statistics
- Category-specific maps
- Quality assessment overlays
Wikinity
[edit]Purpose: Alternative mapping interface for image gaps.
Distinguishing Features:
- 3D terrain visualization
- Mobile-optimized interface
- Offline map caching
- Route planning for photo missions
Bldrwnsch
[edit]Purpose: Process German Wikipedia image requests.
Functionality:
- Aggregates requests from de.wikipedia.org
- Categorizes by topic and location
- Tracks fulfillment status
- Links to requester discussions
Language: Primarily German interface
Module 5: Maintenance Tools
[edit]Commons Image Feed
[edit]Purpose: RSS/Atom feed generation for category monitoring.
Configuration Options:
- Category selection
- Update frequency
- Image size in feed
- Metadata inclusion level
Applications:
- Blog integration
- External monitoring systems
- Automated processing pipelines
- Quality control workflows
Commons-aiuser
[edit]Purpose: Track contributions from individual uploaders.
Metrics:
- Upload history
- Category distribution
- Quality ratings
- License usage patterns
Use Cases:
- Contributor recognition
- Quality assessment
- Potential copyright issues
- Upload pattern analysis
File Siblings
[edit]Purpose: Detect similar or related images.
Technology:
- Perceptual hashing
- Visual similarity algorithms
- Metadata comparison
- Filename pattern matching
Applications:
- Duplicate detection (near-duplicates)
- Image series identification
- Quality comparison (keep best version)
- Derivative work tracking
File Duplicates
[edit]Purpose: Exact duplicate detection.
Method: Cryptographic hash comparison (MD5, SHA-1)
Actions:
- Flag for deletion
- Merge metadata
- Redirect usage to primary file
- Update file descriptions
Bulk Download
[edit]Purpose: Batch download from Commons.
Parameters:
- Category selection
- File type filtering
- Date range
- Size limitations
- Maximum file count
Use Cases:
- Offline processing
- Machine learning datasets
- Backup operations
- External analysis
Unused Files
[edit]Purpose: Identify Commons files not used on any Wikimedia project.
Detection Criteria:
- No transclusions on Wikipedia
- No Wikidata usage (P18, P154, P41, etc.)
- No sister project usage
- Age threshold (e.g., >30 days)
Actions:
- Propose for deletion
- Improve categorization
- Add to relevant galleries
- Link to Wikidata items
Multi-language Descriptions
[edit]Purpose: Generate localized descriptions for Commons files.
Process:
- Extract existing description
- Machine translate to target languages
- Apply language-specific templates
- Present for human review
- Bulk update file pages
Supported Languages: All major Wikipedia languages
Add Information Template
[edit]Purpose: Apply {{Information}} template to newly uploaded files.
Template Fields:
- Description
- Source
- Date
- Author
- Permission
- Other versions
- Categories
Automation Level: Semi-automated with manual review
No Information Tool
[edit]Purpose: Find files lacking proper {{Information}} template.
Prioritization:
- Upload date (oldest first)
- Usage frequency (most used first)
- Category importance
- Uploader activity (active users first)
Usual Suspects
[edit]Purpose: Identify common cleanup issues.
Detection Categories:
- Copyright violations
- Low quality images
- Improperly categorized files
- Duplicate uploads
- Misleading filenames
- Missing source information
Module 6: GLAM Tools
[edit]BaGLAMa
[edit]Purpose: Measure reach and impact of GLAM partnerships.
Metrics:
- Page views for articles using GLAM images
- Geographic distribution of viewers
- Temporal trends
- Comparative analysis across collections
Reporting Features:
- Monthly/annual reports
- CSV export
- Visualization graphs
- Top articles ranking
Use Cases:
- Partnership reporting
- Grant applications
- Impact assessment
- Community engagement metrics
GLAMorous
[edit]Purpose: Track usage of GLAM images across Wikimedia projects.
Analysis Dimensions:
- Project distribution (Wikipedia, Wiktionary, etc.)
- Language distribution
- Article topic categories
- Usage context (infobox, gallery, article body)
Output Formats:
- Interactive web dashboard
- Spreadsheet export
- JSON API
- Embeddable widgets
TreeViews
[edit]Purpose: Monthly view statistics for category hierarchies.
Features:
- Recursive category traversal
- Aggregated statistics
- Month-over-month comparison
- Subcategory breakdown
Performance: Handles large category trees efficiently through caching
GLAMorgan
[edit]Purpose: BaGLAMa variant with enhanced customization.
Additional Features:
- Custom date ranges
- Multiple category comparison
- Advanced filtering options
- Automated email reports
Module 7: Metadata Management
[edit]Image Annotator
[edit]Purpose: Create and edit structured image annotations.
Annotation Types:
- Rectangular regions
- Named entities (people, objects, locations)
- Wikidata entity linking
- Descriptive text
Technical Implementation:
- Canvas-based drawing interface
- JSON-LD metadata format
- Wikidata entity search integration
- Revision history tracking
Use Cases:
- People identification in photographs
- Object tagging for search
- Educational annotations
- Accessibility descriptions
Commons Mass Description
[edit]Purpose: Bulk metadata editing for multiple files.
Workflow:
- Select files by category, uploader, or list
- Define template or pattern
- Preview changes
- Apply batch edit
- Monitor edit queue
Safety Features:
- Preview mode
- Rollback capability
- Edit conflict detection
- Rate limiting
Supported Operations:
- Description updates
- Category additions/removals
- Template parameter changes
- License updates
Locator Tool
[edit]Purpose: Add or correct geographic coordinates for images.
Interface:
- Interactive map (OpenStreetMap base)
- Drag-and-drop positioning
- Coordinate entry (decimal degrees, DMS)
- Bulk processing for image series
Metadata Output:
- {{Location}} template
- Structured data (P1259 coordinates)
- Heading direction (for directional photos)
- Altitude information
Integration: Writes directly to Commons file pages and Wikidata
Module 8: Query Tools
[edit]Pre-filled Query System
[edit]Purpose: Provide ready-to-use queries for common tasks.
Example: Wikipedia Requested Photographs
Query Chain:
- PetScan identifies pages in request categories
- Extracts Wikidata items from article links
- WDFIST finds candidate images
- Presents results with upload/add options
Customization: Users can modify parameters while preserving query logic
SPARQL Query Builder
[edit]Purpose: Construct custom Wikidata queries for missing images.
Interface Components:
- Type (P31) Selector: Dropdown or autocomplete for item classes
- Multiple type selection (AND/OR logic)
- Subclass traversal options
- Location Filter (Optional): Geographic constraint (within radius), administrative division, or coordinates input
Generated Query Structure:
SELECT ?item ?itemLabel WHERE {
?item wdt:P31 wd:Q... . # Type constraint
MINUS { ?item wdt:P18 ?image } # No image exists
SERVICE wikibase:label { bd:serviceParam wikibase:language "en" }
}
Tool Integration: Launch results directly in:
- Wikidata Query Service
- WikiShootMe
- File Candidates
- Custom tools via API
Authentication and Permissions
[edit]OAuth Implementation
[edit]Authorization Flow:
- User initiates tool action requiring permissions
- Redirect to Wikimedia OAuth consent page
- User grants specific permissions
- Tool receives access token
- Token stored for session (not persistent)
Permission Scopes:
- Read user information
- Edit Wikidata items
- Upload files to Commons
- Edit Commons file pages
Security Best Practices:
- Tokens expire after session
- Per-tool authorization (granular permissions)
- Explicit user consent for each scope
- Audit log of tool actions
Account Requirements
[edit]Some tools may require minimum account age or edit count:
- Account age: 4+ days
- Edit count: 10+ edits
- Autoconfirmed status
Rationale: Prevent abuse and spam
API Architecture
[edit]Wikidata Query Service
[edit]Endpoint: https://query.wikidata.org/sparql
Rate Limits:
- 60 seconds maximum query execution
- Concurrent query limits per IP
- Result size limits
Optimization Strategies:
- Use SERVICE wikibase:label efficiently
- Limit result sets with LIMIT clause
- Add timeouts to prevent hanging queries
Wikimedia Commons API
[edit]Base URL: https://commons.wikimedia.org/w/api.php
Key Endpoints:
action=upload: File uploadaction=edit: File page editingaction=query&prop=imageinfo: Image metadataaction=categorymembers: Category contents
Rate Limiting:
- Strict limits for anonymous users
- Higher limits for logged-in users
- Bot flags for approved high-volume operations
MediaWiki Action API
[edit]Documentation: API:Main page
Common Parameters:
format=json: Response formataction=: Specify API actiontoken=: CSRF token for edits
Best Practices
[edit]Metadata Consistency
[edit]Goal: Ensure metadata quality and uniformity across Wikimedia Commons and Wikidata.
Guidelines:
- Always use standard templates ({{Information}}, {{Location}}, {{License}})
- Prefer structured data statements over unstructured text
- Maintain consistent date formats (ISO 8601)
- Use canonical Wikidata items for locations, authors, and subjects
- Add multilingual labels and descriptions when possible
File Naming
[edit]Principles:
- Use descriptive, human-readable filenames
- Avoid non-ASCII characters unless required
- Use underscores instead of spaces
- Include disambiguation where necessary (e.g., “Paris_Bridge_2023.jpg”)
- Avoid version numbers; rely on file history
Licensing and Attribution
[edit]Best Practices:
- Verify that all uploaded media comply with Commons’ licensing policy
- Provide full attribution for authors and sources
- Prefer Creative Commons licenses (CC BY, CC BY-SA)
- Ensure license templates are correctly formatted and categorized
Categorization
[edit]Recommendations:
- Place files in the most specific categories available
- Avoid redundant parent categories
- Ensure every file has at least one topical and one geographic category
- Use Commons’ category tree navigation for validation
Tool Interaction
[edit]Workflow Integration:
- Validate data consistency when passing between tools (e.g., PetScan → WD-FIST → Commons)
- Use the “preview” or “dry-run” mode before committing edits
- Check tool logs for error reports
- Avoid parallel batch edits to reduce conflict risk
Troubleshooting
[edit]Common Issues
[edit]| Issue | Cause | Resolution |
|---|---|---|
| Missing OAuth permission | Token expired or not granted | Reauthorize via OAuth login |
| API rate limit reached | Too many rapid requests | Add delay between requests or use bot account |
| Incorrect category results | Cached PetScan data | Clear cache or modify query parameters |
| File upload failure | Network or API timeout | Retry after short delay or check file size |
| Duplicated metadata | Concurrent edits | Use rollback or re-sync tool |
Debugging Tools
[edit]- Commons API Sandbox: Test API requests interactively
- WDQS Interface: Validate SPARQL queries
- Toolforge Logs: Check tool runtime errors
- Wikidata RecentChanges: Monitor structured data updates
Reporting Bugs
[edit]- Submit issues to the tool’s Phabricator page
- Include reproduction steps, timestamps, and screenshots
- Mention related API queries or OAuth scopes
- Tag with appropriate project (e.g.,
Commons-tools,Wikidata)
Performance Optimization
[edit]Query Optimization
[edit]- Limit SELECT variables to required fields
- Use filters (
FILTER()) early in query chains - Use subqueries for large datasets
- Avoid regex filters on labels when possible
- Use Wikibase “hint:Prior” for faster entity retrieval
API Efficiency
[edit]- Use
generatorparameters for multi-page requests - Request JSON format for compact responses
- Implement caching for static metadata
- Use batch editing endpoints
- Respect API retry-after headers
Tool Integration Optimization
[edit]- Minimize data redundancy between tools
- Schedule heavy queries during off-peak hours
- Use asynchronous or queued processing
- Share intermediate results through temporary storage
Data Formats
[edit]Structured Data on Commons (SDC)
[edit]Schema Overview:
- Entity-based (Mediainfo namespace)
- Each statement has a property (e.g., P180) and a value
- Supports qualifiers and references
- Fully compatible with Wikidata entities
Common Properties:
| Property | Meaning |
|---|---|
| P180 | Depicts |
| P571 | Date of creation |
| P170 | Creator |
| P275 | License |
| P1259 | Coordinates |
| P7482 | Digital representation of |
JSON Format
[edit]Used for:
- API responses
- Structured data export
- Tool interoperability
Structure Example:
{
"title": "File:Example.jpg",
"metadata": {
"author": "John Doe",
"license": "CC BY-SA 4.0",
"depicts": ["Q42", "Q146"]
}
}
CSV Format
[edit]Usage:
- Bulk export for analysis
- GLAM reporting
- Category metrics
Columns Typically Include:
- File name
- URL
- Upload date
- Categories
- Views
- Description
Advanced Features
[edit]Automation Pipelines
[edit]Concept: Connect tools for automated workflows.
Example Pipeline:
- PetScan selects target items
- WD-FIST suggests image matches
- QuickStatements adds image links to Wikidata
- GLAMorgan tracks usage metrics
Advantages:
- Time-saving automation
- Reduced human error
- Repeatable workflows
API Integration via Toolforge
[edit]Purpose: Unified hosting and API gateway for Wikimedia tools.
Capabilities:
- Shared database access
- OAuth management
- Scheduler for recurring jobs
- Secure data handling
Batch Processing Scripts
[edit]Tools:
- Pywikibot
- CommonsHelper
- QuickStatements API
Example Workflow:
for file in commons_files:
upload_to_commons(file)
add_metadata(file)
link_to_wikidata(file)
Community Integration
[edit]Collaboration Channels
[edit]- Commons Village Pump
- Wikidata Project Chat
- Phabricator Bug Tracker
- Mailing Lists
- Wikimedia Discord and Telegram channels
User Roles
[edit]| Role | Responsibilities |
|---|---|
| Uploader | Contributes media to Commons |
| Curator | Reviews metadata and categories |
| Bot Operator | Maintains large-scale automation |
| GLAM Partner | Provides institutional media collections |
| Developer | Builds and maintains tool infrastructure |
Governance and Ethics
[edit]- Ensure transparency in automated edits
- Respect copyright and attribution standards
- Avoid biased data selection
- Encourage open collaboration
- Report misuse or policy violations promptly
Conclusion
[edit]Summary: The Wikimedia Commons and Wikidata ecosystems rely on a broad suite of interconnected tools that facilitate media management, metadata curation, and knowledge linking. From uploaders to GLAM professionals, contributors can leverage these utilities to improve discoverability, accuracy, and cultural representation of digital assets.
Key Takeaways:
- Integration between Commons and Wikidata enhances media value
- Automation tools streamline repetitive tasks
- Community oversight ensures data quality
- Open APIs enable innovation and collaboration
Final Note:
This documentation should serve as both a reference and a roadmap for Wikimedia contributors seeking to understand and optimize the tools that power a global open media repository.