Birds Eye View of Node.js

I’m sure you’ve all seen/heard stuff about using real-time websockets — this guy used node.js but a lot of ppl don’t know that there is so much more to this framework than meets the eye.

First off, it has no locks, is non-blocking in as many places as possible and uses very little memory. Oh yeh, pretty much everything has a callback, following an evented model. Some POSIX operations return a ‘promise’ which signal a ‘success’ or ‘error’. Node is ridiculously powerful as you can see by reading over the api. There is so much here to do and really allows you to do some ‘system-level’ style of programming with a whole slew of POSIX style (compliant?) system calls

Concurrency without Multi-Threading?
Most of the built in hard-core concurrency comes from the evented nature of the framework. Being able to immediately respond to incoming requests while something is off reading from a database provides much needed clock cycles. Right now it appears there is no straight-forward way of utilizing multiple cores other than putting a load balancer of sorts in front and sending over unix sockets or something but there are plans to including forking in the future so that different processes may emerge taking advantage of the cores and utilize message passing to talk to each other. You can of course fork new processes using the system calls available.

Apparently the developers of node are looking towards using web workers available in HTML5 which will also alleviate some concurrency issues.

So who uses it? Is it production quality yet? Well, you can always ask the good folks over at Plurk.com (twitter for japan) who run it w/over 100k concurrent users — apparently this runs on one box with 8 proccesses…. uhhhh…

… maybe I should go spend some time in japan … I keep finding all sorts of shit I like from there

Hello World in Node.js

var sys = require('sys');
sys.puts('hello!');

so first off we include the sys lib then we do our puts statement .. pretty simple
let’s take a look at a

Simple Node.js HTTP Server

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(8124, "127.0.0.1");
console.log('Server running at http://127.0.0.1:8124/');

yeh… it doesn’t really get too hard but this shows the evented nature of node — shit is retardo fast

Speed Demon Benches

ab -c 100 -n 1000 http://127.0.0.1:8124/
Requests per second:    6792.23 [#/sec] (mean)

… but it’s not just for HTTP

if you have any need for networking this is a badass framework to be using. It uses a microkernel design by including modules that you need rather than a bunch of shit you *might* use — big win in my favor..even down to sys/http modules.

you can find a list of supporting modules here

Binary Data

I’m including this section here because when I was looking for information on sending bytes here and there I found lots of articles on the internet saying this was not easily done with node.js — this IS EASILY DONE now. You CAN make a buffer with binary data now and NOT have to specify ‘utf8′, ‘ascii’, ‘base64′, etc…the ‘net2 branch’ that you might see referenced to on the net is already merged in and the api docs are current though can be rather confusing after reading older documentation found around…so ignore that old shit — so, just to clarify, if you are sending binary data through some random socket you can do that now.

Want to see an example of this mysterious byte buffer?

  Buffer = require('buffer').Buffer;
  buf = new Buffer(8);
  for(i =0; i < 8; i+=1) { buf[i] = 0; }

very interesting that you can do this now since javascript doesn’t like binary data that well

buffer.copy does a memcpy() …. begging to get owned by some hackers — remember to sanitize your data

Downloading/Installing

before doing anything else go ahead and grab the latest (as of this writing):

wget http://nodejs.org/dist/node-v0.1.102.tar.gz
./configure && make && sudo make install

woohoo! simple build process ftw

want to use express? it’s pretty cool, like sinatra and it’ll allow you to get to know the most tubular npm tool.

let’s check our version because all of this code is bleeding edge stuff

feydr@mhu:~$ node -v
v0.1.102

also, let’s grab npm first:

git clone http://github.com/isaacs/npm
cd npm; sudo make install

now we can install express and it’s supporting modules that we want:

sudo npm install jade
sudo npm install less
sudo npm install connect
sudo npm install express

playing with it:

mkdir test/; cd test;
express # generates a new app
node app.js # to start the server
# switch to a diff. window
curl 127.0.0.1:3000

speed?
much slower than native node.js but hey, it takes a lot of pain away
it’s still slightly retarded fast compared to say rails and django

ab -c 100 -n 1000 http://127.0.0.1:3000/
Requests per second:    1200.16 [#/sec] (mean)

Support && References
shit tons of ppl on the #node.js irc channel

Need something to show the boss — some slides

My mind is literally wheeling with the crazy shit that this framework allows you to do… particularly remotely…

Last thing before you leave, watch this.

If you are like me you probably have to be working in several different languages at any given time and monitoring more than a handful of daemons. Sometimes your tests do not catch crucial items that they should be catching. As my loyal readers might already know I do not consider myself as knowledgeable about the JVM and java as I should be. These past couple of days were a lesson in humliity when it comes to garbage collection on the JVM.

I grew up learning how to write code using C from man pages managing memory on my own. Malloc and free were my friends and I would pride myself on running valgrind on a paritcular piece of code passing through it’s tests.

The JVM of course uses a managed memory model with it’s garbage collector. This means no more memory leaks right? Wrong.

Example:

Let’s suppose you have an object you instantiated called Blah. Blah gets used but is never de-referenced nor does it have a way to tell the GC that it is a candidate for removing. (You never actually remove an object — you just tell the garbage collector that is is ready to be removed.

” Note the wording: Just because an object is a candidate for collection doesn’t mean it will be immediately collected.” Candidate for Removal

What happens after you instantitate 300k of these Blah objects that use 80 bytes a pop? You just lost 22 meg that you won’t regain until you kill that process!

So to cut to the chase I had a tomcat servlet that kept dying about once a week and I didn’t allocate any time to look at it what the problem was until now as I could just restart it and everything would be peachy. I finally decided to look at it.

This is what I did.

Get a Heap Dump

First, you need to obtain a heap dump — without the heap dump you are having better luck at the casino. Chances are if you are crashing cause of an OOM memory error you won’t be able to get your heap dump from the production site — we don’t really want to screw with that anyways.

So, let’s duplicate our results on our dev box. Like I said before, we were looking at a tomcat servlet that was not working so in theory we should be able to hit our servlet with enough requests to trigger some heap growth.

I wrote a quick little script in ruby to slam our servlet, here it is:

#!/usr/bin/ruby
 
require 'rubygems'
require 'uri'
require 'net/http'
require 'cgi'
require 'timeout'
require 'open-uri'
require 'monitor'
 
File.open('b.txt', 'r') do |f| @filetopost = f.read end
@params = {'text' => CGI::escapeHTML(@filetopost), 'submit' => 'Submit'}
 
def pingsite
  500.times do |i|
    x = Net::HTTP.post_form(URI.parse('http://127.0.0.1:8080/upload/process'), @params)
    puts i
  end
end
 
threads = []
(1..20).each do |i|
  threads << Thread.new do
    pingsite
  end
end
 
threads.each do |t|
  t.join
end

This essentially as you might have guessed posts the contents of a text file through a form to our servlet for processing (text into xml). It makes 10k requests (500 at 20 threads).

BEFORE you run this you’ll want to hop into jconsole. JConsole will allow you to watch the action live as you see your heap grow and be garbage collected. You’ll see that the faster the heap grows the faster the garbage collector will start to run so your graph will start to have it’s lines closer together. What you are wanting to do here is just make the graph curve upward — that’s it.

Once you have a graph that looks like this:

we are ready for the next step. This graph actually grew up into the 400s and was much more pronounced by the time my 10k requests were done — all you really need to do is show proof that there is a growth — the more the merrier as it should help you drill down more easily to determine what is piling up.

Dump the Heap

Find out the process we need to dump:

ps aux | grep tomcat

dump it:

jmap -dump:live,format=b,file=heap.bin 6608

The dump will need to be initiated by the same user owning the tomcat process or someone with perms for it (tomcat6, root, joebob, whatever).

Great! Let’s take a quick look at the size

ls -lh heap.bin

200 meg! WTF!?

Get Relevant Stats

Ok, now let’s hop into jvisualvm (visualvm). I went ahead and changed the perms on the dump file so I could run it as my own user. Once in visualvm we want to go to our classes view in the top right hand corner. From here we can sort by # of instances or % of how much the heap is towards the object. I focused on instance count because that seemed like something that would let me know what was up. I noticed that at least one thing was wrong here. I had something like 300k prepared statement instances being built — what the fuck?!

Fix the Problem

So I went back to my code and did a .close() on them.

        // note this is scala
        pStmt.executeUpdate()
 
        pStmt.close()
 
        if(stmt != null) {
          stmt.close()
        }

Measure Again to Ensure the Problem is Fixed

After this was done we compile our jar, throw it back to our servlet, restart tomcat and run through our bulk POSTing again. The theory here is that we should be able to have a noticeable impact our instance count for prepared statements and have a noticeably different graph as well. I did not know that simply copying your webapp over to the root directory was not enough — you NEED to restart tomcat to get accurate results as items in the PermGen space will be duplicated if you simply copy a new webapp over (and after doing that a couple of times you’ll find a nice little PermGen OOM error in your catalina log files.)

Here’s the new graph:

The new heap size is :20 meg!!

Well, we might not have squashed all of our bugs but I’d bet good money that this servlet doesn’t die again in the next week or so.

Other Profiling Tools I Used but Did not Like:

jhat: I tried this out and it seems like it is supposed to be used on remote servers, however, there was absolute NO thought put into making it a nice UI to use by a human and it didn’t particularly lend itself well to a computer looking at information so frankly I think it’s a piece of shit.

Memory Analyzer: Speaking of piece of shit, this rotten crap came with Eclipse (now, I have my own opinions about how horrible eclipse is to begin with so maybe that’s just my prejudice rubbing off) but this kept crashing every fucking time I tried to load the heap. I tried passing in the arguments -Xms128m -Xmx1024m and all but it just did not like me. Fuck it — I got the job done regardless.

Anyways, I hope this information was somewhat useful for someone else — while documentation exists on the internet for this it wasn’t the most readily available info.

Many people utilize the DOM based parsing methods available in easy to use libraries throughout most languages but few ever need to utilize the power of SAX. Let me invite you to the party!

We’ve all been there — we need to parse an xml document so we pull out some xpath expression and get to work on grabbing our little bit of data from our document.

token = REXML::XPath.first(doc, "/books/title/")

This works great and is easy to use no matter your language of choice for MOST things. Now, let’s pretend this particular document is 10 meg — fuck that, let’s pretend it’s 100 meg — actually let’s get real and call it 2.5 gig of xml.

No, it won’t if you are using DOM parsing since that is based upon building a tree. That is if you are willing to wait for the fucker to even load itself into the tree — forget the memory usage.

I ran into this problem earlier today — memory hit a little over what the document size was in htop but the cpu just kept burning while the world was turning. I eventually killed it and contemplated splitting the xml apart — of course … I was going to run into the exact same problem if I continued to utilize the old worn out DOM parsing methods.

This is when I was turned onto SAX parsing from some random mailing list plea for help. All I need to know I learnt from man pages and mailing lists. I had ran across SAX in the past but was not too excited that I had to prepare a language JUST to parse my document rather than pass it some tried and true xpath — basically I was used to parsing html DOM — NOT million line xml documents. I had read that it was faster in the past but my feeble attempts at comparing documents of the 200k size were just an exercise in futility to find any meaning in using SAX.

Now let me address the audience in what happened. That little job I killed that was bitch-smacking one of my cores around for over 10 minutes and using over 10% of 6gig of memory did the EXACT same extraction under 10 seconds utilizing something like 0.1% of memory and only 70%ish CPU. I, to use the phrase slightly, shit a fucking brick!

Want to see an utterly contrived example of pulling 40+k urls from a sitemap which tied up my cores for-fucking-ever using old style DOM parsing but was done in less than 10 seconds with SAX?

require 'rexml/document'
require 'rexml/streamlistener'
require 'mysql'
include REXML
 
MY = Mysql.new("host", "luser", "assword", "dbname")
 
class ParseThatShit
  include REXML::StreamListener
  def tag_start(*args)
  end
 
  def text(data)
    return if data =~ /^\w*$/
    return if data =~ /\d\.\d/
    st = MY.prepare("insert into links (name) values (?)")
    st.execute(data)
    st.close
  end
end
 
pts = ParseThatShit.new
xmlfile = File.new("big-ass-sitemap.xml")
Document.parse_stream(xmlfile, pts)

Remember boys and girls — this Example is just an example to show you the power of SAX — does it take more LOC than a 3 line xpath block? Yeh, but so does your mother.

Use the power of SAX and drink the spice while you slip into the higher dimension of parsing power!