I also have mad respect for those who are fortunate enough to have enough time to work on community projects — many of us do not have that time available

as for describing the setup — that really is not so necessary — you can replicate the same results in dev/production or thin/mongrel/whatever — of course they have different performance impacts — I’m not here to measure web frameworks — you can see the same results in whatever environment you choose

the “important realization” here was the fact that everywhere you go you see people claiming that templating engines add no noticeable performance hit and I find that to be highly untrue — I still find web frameworks to be highly useful for things such as routing so

“Caching the whole page not only removed the page rendering time (with a majority of time probably spent in haml/erb/etc)” — you hit the nail on the head right here — “the majority of the time spent in haml/erb” — this is what I was bitching about

as for the 20 pages/second I think I must have been looking at something else as it’s much higher than that on production now (much higher on dev as well)

actually as for the unicorn and such — that was obviously a tongue in cheek reference to the last 200 or so projects that I have ran across being described like that — I’m not fond of projects being dressed up to compensate for their lack of technical merit

It would be more useful if you talked about your testing setup. You make no mention at all of how you are hosting the various application servers under test in your ‘benchmark’. Thin, Mongrel, Webbrick, Apache, Nginx, production mode, dev mode? These choices have important performance implications.

I’m not sure what the important realization here is. You say that you gained a 20x speedup when you cached the entire page vs. serving a page from a ‘templating engine’. Was this surprising? I’m surprised it wasn’t more since you have effectively removed Merb/Sinatra/Etc from the critical path entirely and you might as well be serving a static file. Caching the whole page not only removed the page rendering time (with a majority of time probably spent in haml/erb/etc) but the DB and routing time as well. And you only got to 20 pages/sec after caching the whole page? There is something wrong with your env if you can only do 20 pages/sec on a fully cached page as you described. Care to elaborate?

I think the lesson for *any* site is to cache as aggressively as you can, not avoid templating/markup tools that make you more productive as a developer.

I found it funny how you make fun of people who use the word ‘tasty’ to describe code… right after your pretty pictures of Unicorns. Who was it BTW who said that Haml was “unicorns sprinting across rainbows shitting shooting stars when it comes to performance”. I guess I missed that writeup.

Thanks for taking the time to comment. I apologize if this article looks like it is pissing on haml — that’s not really my intention — it’s pissing on templating engines in general.

The benchmark as you have correctly pointed out does not isolate the templating engine which may be somewhat misleading — however benchmarking only templating is also completely irrelevant as the majority of users would be using it within the context of a webapp. Therefore, having access to benchmarks of only templating does not impart anything of value to me.

The benchmark is trivial to replicate — pick your web app — I don’t care which. Power up apache benchmark or httpperf and bench several piece of static content with no templating — now do the EXACT same piece but use either erb/haml. You will notice a significant drop in requests/second.

I also do not have an answer to this — we still use haml over erb over static html.