Wikidata/Berlin summit 2012/fetch, cache, purge

fetch/cache/purge

You can view the presentation slides.

Quick overview of how changes will be propagates through the system and how things will be cached. Covers both what has been implemented and what we're considering to do for the next phase.

By Jeroen De Dauw

Creating items:

item storage: repo

• blob in page/revision table (primary); requires changes in MW core
• wb_items table used for id increment
• extra tables for querying (secondary)

item storage: plans?

• properties?
• query handling?
• not sure yet how to handle queries
• ideas welcome: which technologies to look at

propagation of changes

• from the repo to clients
• maintenance script polls to see if there's something new, and launches hooks

item storage: client:

• table with copy of the relevant items
• for convenience, links items directly w IDs

Q: how do the clients check is there's something new; is there a feed? use the job queue? A: table works like Recent changes; use the database table, and implement on top of that if needed;

People interested in item storage: Come talk to Jeroen!