Hiro Nakamura is a level-3 programmer, from the television show “Heros” that one day discovers he can manipulate space and time. Hiro, however is not the best thing to come out of the Tokyo software industry.

Recently I’ve been spending the last couple of months trying to learn how to scale our databases for the massive flux of data we expect once our site goes live. To this end we’ve tried all sorts of different opporunities. They have all left us with a rotten feeling in our stomach that most of our past year’s work is indeed worthless if we can’t scale our site accordingly.

Enter TokyoCabinet. TokyoCabinet is an extremely powerful key-value hash-based database management system straight out of Japan where top engineers collaborate to build mixi, the famous Japanese social networking site ala Facebook.

How do I install this?
Installation is extremely easy. Let’s run through it real quick shall we?

First off let’s install bzip2 and their headers — you probably have these already but I’m
assuming you are starting off fresh somewhere.

1

sudo apt-get install libbz2-dev

Now let’s grab your files from sourceforge:

1
2

wget http://tokyocabinet.sourceforge.net/tokyocabinet-1.4.7.tar.gz
wget http://tokyocabinet.sourceforge.net/rubypkg/tokyocabinet-ruby-1.20.tar.gz

unpackage them:

1
2

tar xzf tokyocabinet-1.4.7.tar.gz
tar xzf tokyocabinet-ruby-1.20.tar.gz

install tokyocabinet:

1
2

cd tokyocabinet-1*
./configure && make && make install

install the tokyocabinet ruby gem

1
2
3
4
5

cd tokyocabinet-r*
ruby extconf.rb
make
sudo make install
sudo ldconfig

test to make sure everything went ok

1
2
3

/home/user$ irb
irb(main):001:0> require 'tokyocabinet'
=> true

Now, let’s bench your system!
Grab this file (http://gist.github.com/64391):

git clone git://gist.github.com/64391.git

NOTE: do NOT install the ffi version if you want speed. I installed the ffi version yesterday and only achieved around 5555 req/second. Today I installed the real version and achieved 28,000 inserts/second and 48,000 selects/second. This was for the table database — which if you are trying to emulate your mysql RDBMS, you’ll want to choose this.

The very first thing you’ll want to after hopping out of bed and grabbing that first cup of coffee is to take your retard capsules (buy one, get one free!)

Next tell yourself that today is going to be shitty because you stubbed your toe, your hangover is starting up and you just realized that you are out of gas.

Now, goto work, add your ssh-keys, crank up screen and pick up where you left off yesterday. You left of knowing that your timer code was all fucked up and it was stopping you from reading in a very important value from a message queue. With this in mind discount ALL possibilities that it could be anything else (for example that your jobs from the message queue have their own timeouts).

Now using your super 1337 spacetime googling skillz scour the interwebs for clues to your lack of knowledge on this predicament. Throw in stuff like

1
2

System.out.println("got here!");
System.out.println("def. am not getting here!");

.

Good. Got that done?

Next, now that you know it’s this function that is causing your pain read up on it and starting writing example code that explains how YOU think it should work proving to yourself that those who created the api obviously missed some fatal design flaw and you are just now discovering it. (yeh, after you root hundreds of boxen when you were younger and gobbled retard pills like it was candy you think you are always the superior one in these matters)

Spend the rest of the day bemoaning how life sucks and how nothing happens and then go home.

Start consuming massive quantities of booze and shut everything out of your life except some Desktop Tower Defense.

Your day is now officially fucked! Congratulations!

Sounds like this is a security related article, doesn’t? Ahh, and right you are — only it’s not the security article that you thought it was. Rather this is here to point out the reasoning why being a professional programmer should only be left to those who hack in their spare time.

I have been looking at houses recently with the expectation of acquiring one. This has presented to me many educational opportunities. One of these was learning all about the process of appraisals and assessments — that is how much does the State wish to bribe you from your livelihood? Fortunately for us this information is public knowledge and easily attainable at your local
county assesor’s or collector’s webpage.

After learning how to use mechanize and hpricot well enough I was able to scrape large portions of my friendly Boone County’s webpage. I then proceeded to start working on other neighboring counties and large cities in the magnificient state of Missouri. This is where I learned once again that those who hack for money will ALWAYS be of lesser quality programmers than those who code because they like to. I’m not saying that the coder who codes for money is less intelligent — I’ve met many programmers that have what my girlfriend terms “google-brain”; however their ways of thinking about things are:

• inferior
• unimaginative

Let me show you proof:

1
2
3
4
5
6
7
8
9

<form action="searchProperty.asp" method="post"
<input type="submit" value="Next  &gt;&gt;" name="next">
<input type="hidden" value="SELECT * FROM qryProperty 
  WHERE FullAddress LIKE '%HALL%' ORDER BY ADDRESS" 
  name="sql"/>
<input type="hidden" value="1" name="lowcount">
<input type="hidden" value="10" name="uppcount">
<input type="hidden" value="next" name="dir">
</form>

Had I been even 5-6 years younger than I am now the State of Missouri would probably be in shambles…I mean, this is just a 14-19 yo hacker’s wet dream. I found this CASUALLY looking at the css so I could scrape the page — NOT hack it.