To tell you the truth I usually am ctrl-D’ng (mutt mass deleting) all threads from this mailing list since I just do not have time to read them all. This of course caught my eye though and I’ve been waiting for a day to set aside some time to write up an article on it. So that day is today and this is that article.

These guys make a TON of claims and the description page is chockful of buzzwords describing what exactly OrientDB is. Let’s take a quick look.
Features

• Written in Java
• 40k inserts/second on commodity hardware
• Document/Graph Based DataBase
• Supports Schema-less, schema-full, schema-mixed modes
• Security Profiling for users/roles Based Access
• Supports SQL as a query language
• ACID compliant with support for transactions
• Less than 400k Footprint for Full Server (embeddable)

It is truthly very easy to get started unlike other solutions out there. It unfortuantely has a XML config file but the default one that comes with the release I downloaded (0.9.13) clocks in at only 22 lines and is fairly readable. It listens for binary connections on port 2424 by default and a built-in web interface on 2480 by default. The package I downloaded came with jars ready to go so there was no bullshit to configure or anything but if you grab sources ant should work things out for you.

Example:

svn checkout https://orient.googlecode.com/svn/trunk/ orient
export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.20/
chmod a+x build.sh
./build.sh

Like any good startup script it proudly displays it’s ascii logo.
It really is quite a nice little web interface I have to admit. For those of you who have/want to care about licenses it is under the Apache 2.0 License. There are currently 12 members on it’s google groups mailing list.

Web Interface
The web interface has a RESTful interface that supports GET, PUT, POST, and DELETE operations. This is really cool because then you can do all your requests with javascript and pull down result sets with json. Being able to put this crap behind a load balancer allows you to scale like a motherfucker when it comes to reads, which is one reason why other databases that have RESTful interfaces caught on so quickly. It has a built in profiler that really rocks out and I firmly believe all software should have built in shit like this — it is just common sense for applications like this. There are tabs right next to it that show you configurations, connections and and database pools — cool stuff indeed.

OrientDB Profiler

The schema view is pretty cool as well as you can see all of your classes (tables) , their record count and their cluster count. Physical clusters make for fast IO, whereas logicals save space but are slower. Of course in the day and age of pocket terrabyte storage drives — who the hell needs more space? We need ram and cpu damnit!

Security
I must say that out of all the NoSQL solutions out there OrientDB really fucking shines with it’s security built-in from the get-go — almost every other system out there has added it on as an *optional* feature — as if security should ever be treated that way — it’s like modern day programmers for some reason still do not know that 14 year old kids have scripts that scan the net for open things like this — some mid-20s kids have those exact same scripts.

Apparently the underlying data structure is a “RB+Tree” which as the authors point out is a mix of Red-Black and B+ Tree algos — this they assert give you the best mix of balancing, speed, and 50% less memory usage.

All in all this is a pretty nice offering and I might need to play with again here in the future. I have a couple million xml documents I’m thinking about transforming to json. What are your thoughts on this fellow readers?

Dependencies are a big problem here but so are the tests. With the ruby community’s ever so crushing pride about various ways to accomplish a task you usually wind up having to either pick between the options of using a LOT of plugins/dependencies or not using any. Also, you know you have problems when after spending the better part of a morning upgrading your dev environment you can’t run your tests.

So we started looking at different web application frameworks. We have a rather large existing codebase in java already to do some text parsing but xml config files and the ever-growing concern of java developers to deal with encapsulation in the most fucked up manner imaginable leads us having java with a sour taste in our mouths. Scala on the other hand has been what I have deemed “what java SHOULD have been”. Seriously, that language is so elegant compared to it’s predecessor that they should just scrap everything and move on from that point forward. The JVM is really solid shit but java itself — ehh…

Anyways on to the meat of this quick article. Why do I like play?

• class reloading
• easy setup
• fucking awesome test runner
• great documentation && active community
• NO xml config files
• use with scala

Class Reloading

Do I even need to explain this one? Having the ability to reload your classes on the fly without having to go through configuration hell is so ridiculously helpful it’s insane. Your development productivity will gain leaps and bounds and with integrated auto-testing you are assured a winner.

Easy Setup

It does not get easier than a:

wget http://download.playframework.org/releases/play-1.0.2.1.zip
unzip play*
sudo cp -R play* /usr/bin/play
play new my_new_app
play run

Fucking Awesome Test Runner

I need to attach these screenshots to show you the awesomeness of this.

Test Index

Selenium Test Runner

Great Documentation && Active Community

When you first boot up Play (and it boots FAST) you will find that you have all the basics to start hacking up the HelloWorld into whatever you wish. A simple click of the mouse gives you your environment settings (which show you everything from paths to thread info, to monitors). Play also has a cool little route built in http://127.0.0.1:9000/@documentation that has screencasts, sample apps, faqs, and pretty much everything you need to know to start playing around.

Admittedly there were only 12 users online in IRC earlier today when I wrote it but the mailing list has ~30 emails/day with active development/discussion.

NO XML Config Files

Yes, that’s right ladies and gentlemen — no goddamn config files written in fucking XML. WTF! Your main config file, application.conf is just your basic java properties set — no need to get all crazy and declare everything under the sun.

What’s the routes look like?

# Routes
# This file defines all application routes (Higher priority routes first)
# ~~~~
 
# Home page
GET     /                                       Application.index
 
# Map static resources from the /app/public folder to the /public path
GET     /public/                                staticDir:public
 
# Catch all
*       /{controller}/{action}                  {controller}.{action}

yeh….. they are that simple looking

Use with Scala

There is a scala module that I highly recommend you download and use — oh and if you like scala — you can get on the akka bandwagon as well. It’s also highly recommended.

Do yourself a favor and Play Now!

Ruby FanBoy

1) class reloading
2) dynamic typing OR a sweet ass type inference engine

I could not stress these two points more. I do NOT want to have to recompile/reload my classes every time I make a simple edit to the code — having this with ruby/merb has just made tremendous strides in how fast we can flesh out features. I also HATE having to declare types for my methods, variables, etc. I think it’s a waste of time and good code should not have to do that (especially if you are using type inference like all good languages do).

So eventually I came across lift which I thought sucked right out since it likes maven/xml. Seriously, xml files were NOT meant to be configuration files — goto hell!

However, recently on another front my colleague and I have already been looking at languages that run on the JVM for another project for the same website. We’ve been looking into scalability concerns with big data and have both concluded that we want/need the power of the JVM but really do not like java particularly too much. This has led us to look at the two up and comers: Scala and Clojure. Recently I got ahold of the pragprog book and started flipping through it’s pages. One section in particular caught my eye. The author in a matter of 5-10 lines or so was able to open up a file, scrape some xml from the web, parse it and then save it. I was like — WHAT THE FUCK!? In java the exact same code would have had to have gone through the verbosity monster making machine and clock in at 20-30 lines easily.

Motherfucking <3

This was enough for me to seriously start thinking about re-writing our main java project. This particular project has 10 (count them ten) antlr grammars and quite a few supporting classes along with CLI wrappers to said classes. We have quite a few test suites for each class thankfully otherwise this project would probably not be so possible without seriously killing it.

I first started converting one of our main classes over. Replace the for loops, drop the semicolons, take out the public identifiers, etc. Let’s try to compile. Oh noes! We don’t have access to these methods anymore. This is where javap comes in handy when you are converting from java –> scala. Javap will show you what the REAL class name or method name is for each language. Keep in mind my original intention was to just get a scala class to compile and use it FROM java. Now I’m on route to be using everything from scala but it really doesn’t matter — as long as you have class files and as long as those class files have access to the right methods/variables you are set.

Let’s say we have a seat class written in scala:

package com.bluffware;
 
class Seat() {
  var id:Int = 0
  var number:Int = 0
  var seat_id:Int = 0
  var sitout:String = ""
  var position:String = ""
  var button:String = ""
  var player:String = ""
  var amount:String = ""
 
  // might want to add utg, cutoff later on..
}

You might think that from java you can do a simple:

seat = Seat.new();
seat.number = 2;
myContainer.seats += seat;

Well you can’t cause javap shows that there is NO access to number.

If you are accessing this class from java you’ll need to do the following:

seat = Seat.new();
seat.number_\$eq(2);
myContainer.seats().\$plus\$eq(seat);

The escapes are for antlr — in normal java it’d just be _$eq(var)

Really though, the only time when you are going to have to do this is if you
have a java class that you need to be compiled through java — if you can convert
it to scala go ahead and do so.

Depending on the size and scope of your project it might make sense to convert
a bit to scala and do it piece-meal and then have other classes that access the scala
class to change the way they use it.

Really in our project now the only thing that uses this is our antlr grammars which
I’m going to be at sometime finally finishing the scala antlr target.

Anyways, if you have not tried out scala yet and you are a java dev — what are you
waiting for!? It’s easier than you think — make sure you have some tests for a small
class you’d like to convert then start doing it!

I’ve also been in the process of doing some serious benchmarking lately against webservers and other things while managing to piss people off when I start quoting scripture from the mountaintop regarding performance — which is the one thing I pay most attention to on whatever project I am working on. “Fast enough” is not faster or fastest.

This led me to examine what exactly I’m trying to do on various web projects and what I need to do to get it done. Primarily I want to serve up an application via the web to the world. To this end I want an environment that is either ridiculous fast at compiling/loading classes or I want an interpreter. Furthermore this environment should be able to serve up web pages. My first search started re-analyzing common types of web servers.

Types of WebServers

• CGI
• FastCGI
• Native Code
• Your Own WebServer

Let’s review them real quick:

CGI
CGI is probably the first thing people used to turn to when they want to serve up a script written in some language but do NOT want to write the webserver themselves since the grammar of HTTP is not the prettiest thing (from what I’ve been told).

Your typical cgi script is EXTREMELY easy to do.
Here’s an example:

First edit your /etc/apache/apache.conf like so:

ScriptAlias /cgi-bin/ /www/cgi-bin/

Now just put this simple two line ruby script into
that directory …

#!/usr/bin/ruby
 
puts "Content-type: text/html\n\n"
puts "test"

… and change the permissions.

sudo chmod -R 755 /www/cgi-bin/
sudo chown -R www-data:www-data /www/cgi-bin/
sudo /etc/init.d/apache/restart

Now you can navigate to your url at http://127.0.0.1/cgi-bin/test.rb and see a webpage!

Of course, it’s not very fast ….

ab -c 1 -n 100 http://127.0.0.1/cgi-bin/test.rb
Requests per second:    150.45 [#/sec] (mean)
 
ab -c 10 -n 100 http://127.0.0.1/cgi-bin/test.rb
Requests per second:    435.64 [#/sec] (mean)
 
ab -c 100 -n 100 http://127.0.0.1/cgi-bin/test.rb
=> Requests per second:    421.97 [#/sec] (mean)

For this reason a lot of people do not use straight up CGI in production environments anymore.

FastCGI

FastCGI is ‘faster’ than CGI since it loads the interpreter once and therefore does not have to keep loading it up everytime a page is requested. In reality it can be just as slow though if you have to load a ton of different components like notable ruby frameworks merb and rails. Other than this there really is not much difference between CGI and FastCGI — so it’s out of our game as well.

Native Code
This is what we are going to be looking at. Native code can take your requested webfile — parse it and return the contents immediately — this is the fastest way outside of creating your own webserver and having to consult Beej (cause I’m assuming you are writing your language in something fast like C right??? Right!?? Typically when people choose this option they are looking at options like phusion passenger where they make their own nginx or apache module. This is retarded fast.

Your Own WebServer
There are very good reasons for not rolling your own web server. There are quite a few edge cases in HTTP you need to support and various browsers have ‘weird’ ways of handling things. This is not to even mention that you are going to leave out quite a few people who would rather use apache or nginx. However, ruby’s webrick would be an example of ‘rolling your own’.

So let’s take a second look at writing an apache module (cause let’s face it — nginx is cool but apache is adopted everywhere else).

Compiling your Module

You’ll want to grab modules/experimental/mod_example.c from an apache 2.2 directory. You obviously don’t need the majority of the crap in it. I’ve included the most important part here:

/* This example just takes a pointer to the request record as its only 
* argument */
static int webit_handler(request_rec *r)
{
 
        /* We decline to handle a request if hello-handler is not the value 
         * of r->handler */
        if (strcmp(r->handler, "webit-handler")) {
                return DECLINED;
        }
 
        /* The following line just prints a message to the errorlog */
        ap_log_error(APLOG_MARK, APLOG_NOERRNO|APLOG_NOTICE, 0, r->server,
        "mod_webit: %s", "Loaded to kick ass!");
 
        /* We set the content type before doing anything else */
        ap_set_content_type(r, "text/html");
 
        /* If the request is for a header only, and not a request for 
         * the whole content, then return OK now. We don't have to do 
         * anything else. */
        if (r->header_only) {
                return OK;
        }
 
        /* Now we just print the contents of the document using the 
         * ap_rputs and ap_rprintf functions. More information about 
         * the use of these can be found in http_protocol.h */
        ap_rputs("<HTML>\n", r);
        ap_rputs("Hello world\n", r);
        ap_rprintf(r, "%s\n", parseThatShit(r->filename));
        ap_rputs("</HTML>\n" ,r);
 
        /* We can either return OK or DECLINED at this point. If we return 
        * OK, then no other modules will attempt to process this request */
        return OK;
}

Your handler is what you want to focus in on. It receives a request and at that
point you can do whatever the hell you want with it. I have chosen here to take
whatever file the user gave to me and sent it off to parseThatShit which returns
a charstar.

You can compile it with the following:
(I have chosen c99 for better or for worse — I’ll leave your discriminatory comments up to
the internet to judge.)

sudo apxs2 -c -i -Wc,-std=c99 -a mod_webit.c 
sudo /etc/init.d/apache/restart

Let’s setup apache to accept this wonderfulness:
(make sure your module was installed in the same path as mine)

LoadModule webit_module /usr/lib/apache2/modules/mod_webit.so
 
Listen 82
 
<VirtualHost 127.0.0.1:82>
    SetHandler webit-handler
    ServerName 127.0.0.1
    DocumentRoot /home/feydr/random-hacking/webit/www/views
</VirtualHost>

Don’t forget that the log files are you friend — if nothing happens check the logs! It’s probably because you didn’t compile with debugging support and you haven’t spent enough time in gdb land. Let’s go segfaults! Duhm, duh duh duhm!

 tail -f /var/log/apache2/error.log
*** stack smashing detected ***: /usr/sbin/apache2 terminated
[Wed Mar 03 17:13:35 2010] [notice] child pid 318 exit signal Segmentation fault (11)
[Wed Mar 03 17:13:42 2010] [notice] mod_webit: Loaded to kick ass!
*** stack smashing detected ***: /usr/sbin/apache2 terminated
[Wed Mar 03 17:38:50 2010] [notice] mod_webit: Loaded to kick ass!
[Wed Mar 03 17:38:50 2010] [notice] mod_webit: Loaded to kick ass!

Lex and Yacc
Now to the meat of this article.

Lex allows you to define all your tokens. Yacc will allow you to put your tokens into meaningful functions.

Here’s a simple lex file:

%{
#include <stdio.h>
#include "y.tab.h"
%}
%%
 
var                     return TOKVAR;
puts                    return TOKPUTS;
\"                      return QUOTES;
times                   return TIMES;
do                      return DO;
end                     return END;
'.'                     return PERIOD;
[-+()/*\n]              return *yytext;
[0-9]+                  yylval=atoi(yytext); return NUMBER;
[a-zA-Z]*               yylval=strdup(yytext); return WORD;
=                       return EQUALS;
[ \t]+                  /* ignore whitespace */;
%%

Our yacc file is not so simple but it should be a bit more readable. Let’s take a look at some of it:

Our main:

main(int argc, char *argv[])
{
  if(argc > 1) {
    parseFile(argv[1]);
  } else {
    repl = 0;
    printf("Webit Version %s", VERSION);
    putPrompt();
 
   /* yyparse is what actually will start parsing your file */ 
   yyparse();
  }
}
 
/* ........ */
 
/* here we see that we have various commands that we can do */
commands: /* empty */
        | commands command
        ;
 
command:
        var_assign
        |
        statement
        |
        string_assign
        |
        do_loop
        |
        puts_var
        ;
 
/* ..... */
 
/* variable assignment */
var_assign:
        TOKVAR WORD EQUALS WORD '\n'
        {
          int x = firstEmpty();
          /* debug mode only
            printf("assigned %s to %s\n", $4, $3);
          */
          char *name = strdup($2);
          char *value = strdup($4);
          putPrompt();
        }
        ;

our Makefile

all:
        lex webit.l
        yacc -d webit.y
        cc lex.yy.c y.tab.c -std=c99 -o blah
 
clean:
        rm -rf y.tab.h blah lex.yy.c y.tab.c

Really making your own programming language the way you want to program and with performance is easy. Maybe you shouldn’t rush to use it on production servers but shit — it sure beats the hell out of putting up with crappy software in an age where programming is considered to be just including other people’s crappy software.

I immediately focused my attention with mytop trying to find slow queries. Found quite a few and fixed them. I then started to cache out some of the problem areas and that is when I started to notice something very very unusual.

This particular application uses the merb framework which is sadly fast becoming ghostware with no promise from the powers that be to continue to integrate it into the upcoming rails 3 which was due out a year ago. Apologies, that I don’t have time myself to dedicate to this framework — so I can’t be too harsh, after all it is free software.

You see, we use datamapper as an ORM layer to our database and datamapper collections don’t fit well into memcached since they have procs that can’t be marshalled correctly — if I am wrong about this PLEASE LET ME KNOW.

Caching

So the typical way of caching your shit is to just call your finder from a helper in your view and cache the view. Simple enough I suppose — sounds like it should work. I then proceeeded to fragment cache everything I could.

When I was looking at items that would expire my cache (like a new blogpost upon a page that lists blog entries) I noticed that caching the entire page versus caching just a fragment or a partial of a page was noticeably faster. Now, we aren’t talking like “oh wow, that’s faster”… we are talking “HOLY FUCKING SHIT — that’s fucking faster than shit!”

Templating Performance

This drew me to the realization that makes the article’s purpose. Contrary to popular opinion — templating is a HUGE performance issue.

By caching our sql we gained 5-10X speedups (1 r/s to 5-10 r/s). By caching the entire damn page we went to 20 requests/second — a 20X speedup! That’s double what our best sql/fragment caching could do.

I needed to confirm this because everyone has told me that sql is by far a worse performance killer than templating engines. Well, my benchmarks say otherwise.

Let’s look at some of those benchmarks real quick. These were extremely simple to do and I invite you to replicate them — it’ll take 2 minutes of your time to see for yourself.

I benched all the requests using Apache Benchmark and they look like this:

Actually that is Alchesay, Apache was named cause back in the day when I had nothing to do but beat up on *nix boxen apache had more holes than the town whore.

feydr@mhu:~$ ab -c1 -n100 http://127.0.0.1:4567/hi

Everytime I used a haml template or put the equivalent HTML into the controller as a string it would look like this:

%h1
  Hello World
%ul
  %li
    Blah 1
  %li
    Blah 2
  %li
    Blah 3
  %li
    Blah 4

Note: I’m not trying to pick on haml — I really enjoy it — I just don’t want to be delluded that it’s all unicorns sprinting across rainbows shitting shooting stars when it comes to performance — because it is not — furthermore that kind of attitude in the community really aggravates me.

As a side note — anyone who uses the words like ‘tasty’ to describe code needs to go back to their frontpage bullshit.

So what’s the conlusion to this testing? Tell markup languages to take a hike and only use css/html? Nope, we still use haml as I hate looking at html now because of it.
What I ended up doing was caching everything I could up front and then combining multiple renders myself.

Solution

I basically decided to stub out portions of my views that I knew were not easily cacheable like the user layout found on hulu:

Some Code:

You can see I stub out “{begin-login}” with whatever I’m going to replace it with later. This really is not necessary
and I could prob. save a few cycles by not doing this but whatever.

 
    # user specific non-cacheable
    # needs 2 renders
    if session.authenticated? then
 
      @cache = MMCACHE.clone
      begin
        @welcomehome = @cache.get("/welcome/homein")
      rescue
        @posts = Blogpost.all(:order => [:created_at.desc], :limit => 4)
 
        # render our base layout without user-specific stuff
        @welcomehome = render :layout => 'cachein'
        @cache.set("/welcome/homein", @welcomehome, 0)
      end
      @cache.quit
 
      # render user-specific stuff if logged in and tack it on
      @loginshit = render :template => 'layout/_logout', :layout => false
      @welcomehome = @welcomehome.gsub("\"{begin-login}\"", @loginshit)
 
    # non-logged in user -- should only take 1 render
    # we have a special section for a logged in user that 
    # shows their inbox and other things
    else
 
      @cache = MMCACHE.clone
      begin
        @welcomehome = @cache.get("/welcome/homeout")
      rescue
        @posts = Blogpost.all(:order => [:created_at.desc], :limit => 4)
 
        @welcomehome = render :layout => 'cacheout'
        @cache.set("/welcome/homeout", @welcomehome, 0)
      end
      @cache.quit
 
    end
 
    #output final render
    @welcomehome

This could probably be cleaned up a bit more and be stuffed into a helper but you get the drift.

I did not include any fetch_partial (merb-cache helper methods) benchmarks here but to suffice to say I gained 20 requests/second by NOT using them — YMMV.

Today’s rant/diatribe brought to you by:

• Roxy Music – Take a Chance With Me
• Kid Cudi – Pursuit of Happiness
• Blue Scholars – 50K Deep

So sorry, no crazy new utility to solve templating performance problems other than this hack but this does lay the foundation for more serious research in this category.

Let me know how you solve your templating performance problems in the comments below.

  ENV['LC_CTYPE'] = 'en_US.UTF-8'
  IO.popen("java -cp \"#{classpath}\" com.bluffware.BluffParse #{xtra}"
                +handfile) do |f|
    @xml = f.read
  end

Of course once we started getting some users this quickly got retarded fast.
So I went ahead and wrote my first servlet in tomcat to grab the hand history. Only problem is that I had no clue how the hell I was supposed to know who was accessing our webservice since tomcat lived on a different server from merb and the cookies had a different domain. This turned out not to be a huge problem.

JRUBY to the Rescue!

JRuby has a sweet java<-->ruby bridge called RedBridge. This allows us to execute ruby code in java which allows us to get to our beloved cookies.
I wrote something like this to get our sessions going:

    // return merb session id from cookies
    public Integer getSession(Cookie[] cookies) {
        String userid = "0";
 
        if(cookies != null) {
          for(int i=0; i<cookies.length; i++) {
            Cookie cookie = cookies[i];
 
            // base64 decode, then un-marshall ruby style...
            // finally figure out what to do with our session secret key
            if(cookie.getName().equals("_session_id")) {
              if(!cookie.getValue().equals("")) {
                ScriptingContainer container = 
               new ScriptingContainer(LocalVariableBehavior.PERSISTENT);
                //container.setWriter(out);
                container.runScriptlet("require 'base64'; " +
                              "blah = Marshal.load(Base64.decode64(\""
                              + cookie.getValue() + "\"))[\"user\"];");
                userid = container.get("blah").toString();
              }
            }
 
          }
        }
 
        return Integer.parseInt(userid);
    }

All was good in the land of poker analysis until one of my users messaged me one day saying some of my hands had ended up in his account. WTF!??! Then after accepting that he was not lying I indeed experienced seeing one of my own hands pop up as someone else’s.

I started putting trace statements all across the code trying to figure out what was going on where.

System.out.println("is it here?");
//.....
System.out.println("it must be here...");
// etc...

then watching it with one of my more favorite tools, tail.

tail -f /usr/local/tomcat/logs/catalina.out

after scanning it for a bit I could not find anything out of the ordinary; then I deployed
some new servlet code without restarting tomcat

cp -R myservlet/ /usr/local/tomcat/webapps/.

and lo and behold, after posting a hand my cookie had changed! the key to figuring
this and most problems like this out is to trigger the error and I had now found out
by deploying the app without restarting I could do so.

it turned out that my cookie was changing somewhere INSIDE the ruby code.
I popped onto the jruby website and noticed that my servlet was not
thread safe.
I checked in with the friendly folks at #jruby on freenode and they confirmed that this was more than likely the problem since it was a local var in a multi-threaded app.

So I changed my offending line to this:

ScriptingContainer container = 
new ScriptingContainer(LocalContextScope.THREADSAFE, LocalVariableBehavior.PERSISTENT);

I started triggering the same event like mad and it works now! Yeh, JRuby! Yeh, for docs!