[Radiant] 12,000 articles in a Radiant site?

[Daniel Sheppard]

> As Aitor suggests, it sounds like this site could really use some 
> caching that reminisces of Rails' page caching model.

The radiant caching model is actually pretty damn quick - not raw-apache quick, but pretty damn close. Look through the archives for
the actual figures from my benchmarks, but my system could push out somewhere around 100 req/sec using radiant's cache compared to
600 req/sec for raw apache. In the grand scheme of things, that performance difference is negligible (from what little data I can
find, 100 req/sec would be more than adequate to handle a slashdotting/digging). Performance is about half that if you don't use
mod_x_sendfile (I advise using mod_x_sendfile).

> Now, I imagine one of the largest slowdowns on your site is 
> going to be 
> Page.find_by_url because it is recursive.

Page.find_by_url isn't actually that big of a problem anymore. You'll get a db hit for each level of your page heirachy, but that
doesn't actually add up to much - again look through the archives for the benchmarks, but I don't think that it starts impacting on
your performance until you get quite deep. Also, radiant's caching does not hit the db at all - only non-cached requests have to
deal with find_by_url.

Radiant used to have a problem with serving up a wide list of pages, but unless you use something that changes the url scheme (ie.
ArchivePage) that is no longer a problem - if you do use something like ArchivePage, then you can fix it up by customising the
find_by_url method in that page to look up child pages directly (which I should really get around to doing in the core ArchivePage).

My site (www.thegroggysquirrel.com) is currently serving ~800 pages, all images (~421 of various sizes) are also served through
radiant's caching. My main issue with that many pages is that I have a very wide list of pages (most pages are direct children of
either /comics or /articles), so the admin interface takes a while to load that list up. The highest traffic the site's had to deal
with is 2,000 visitors a day, but it did that without hiccup (I'm convinced that my bandwidth will give out before radiant does).

>  Radiant could 
> really be sped 
> up by caching the URL in the database, which would reduce lookup time 
> for most pages, and having the recursive method as a 
> fallback.  If you 
> want, we can hash over the design of this optimization 
> together (perhaps 
> with John and Daniel) and apply it to the core.

I'm fairly sure that the speedup there is negligible now that Page.find_by_url isn't so bad.... I think I might have benchmarked
that to see whether it was worth doing work on and it came up fine.

Dan.