Proposal for new cache architecture

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search

this page proposed a caching architecture which removes squid, and puts caching into http servers.

user request

 -> frontend-cache-server
      (view request) and (a filecache exist for the page)?
        YES: serve static content;  return.
         NO: continue
      -> normal-apache
           render page;  store in file cache;  return.

frontend-cache-server is NFS server for filecache, and runs Apache. uses following rewrite rules:

RewriteEngine on
RewriteMap filecache prg:/usr/local/bin/ctwiki/rewrite-for-fcache.pl
RewriteCond %{HTTP_COOKIE} ^$
RewriteCond %{HTTP_HOST} ^(.*)$
RewriteRule ^/w/(.*)$ ${filecache:%1 $1} [L]
RewriteRule ^/w/(.*)$ /w/index.php?title=$1 [L,QSA]

rewrite-for-fcache.pl is a perl script:

#! /usr/bin/perl
$| = 1;
use MD5;
use URI::Escape;
$c = new MD5;
$fs = "/home/wikipedia/htdocs";
$fcache = "/fcache/";
while (<>) {
        chop($_);
        my ($host, $path) = split / /;
        $md5 = $c->hexhash($path);
        $result = $fcache . $host . '/' . substr($md5, 0, 1) . '/' .
                substr($md5, 0, 1) . substr($md5, 1, 1) . '/' . uri_escape($path) . ".html";
        if (-f $fs.$result) {
                print "$result\n";
        } else {
                print "/w/index.php?title=uri_escape($path)\n";
        }
}