Proposal for new cache architecture
From Meta, a Wikimedia project coordination wiki
this page proposed a caching architecture which removes squid, and puts caching into http servers.
user request
-> frontend-cache-server
(view request) and (a filecache exist for the page)?
YES: serve static content; return.
NO: continue
-> normal-apache
render page; store in file cache; return.
frontend-cache-server is NFS server for filecache, and runs Apache. uses following rewrite rules:
RewriteEngine on
RewriteMap filecache prg:/usr/local/bin/ctwiki/rewrite-for-fcache.pl
RewriteCond %{HTTP_COOKIE} ^$
RewriteCond %{HTTP_HOST} ^(.*)$
RewriteRule ^/w/(.*)$ ${filecache:%1 $1} [L]
RewriteRule ^/w/(.*)$ /w/index.php?title=$1 [L,QSA]
rewrite-for-fcache.pl is a perl script:
#! /usr/bin/perl
$| = 1;
use MD5;
use URI::Escape;
$c = new MD5;
$fs = "/home/wikipedia/htdocs";
$fcache = "/fcache/";
while (<>) {
chop($_);
my ($host, $path) = split / /;
$md5 = $c->hexhash($path);
$result = $fcache . $host . '/' . substr($md5, 0, 1) . '/' .
substr($md5, 0, 1) . substr($md5, 1, 1) . '/' . uri_escape($path) . ".html";
if (-f $fs.$result) {
print "$result\n";
} else {
print "/w/index.php?title=uri_escape($path)\n";
}
}