Have you ever wondered how to make IO faster?
We have this dilemma on the messaging world where we need to provide a framework where users are able to send thousands messages per second and we have found our way on doing it, and we are quite excited about it.
In our tests we have been able to send 25,000 persistent messages per second being very close to the the limits of the disk not consuming a lot of CPU on this process, on a computer with 2 QuadCores and a SAS/SCSI disk which is a very modest workstation. And about 10,000 messages/sec on my crappy Laptop because of my slow disk.
We have implemented a very generic JNI layer in top of AIO where you send a Java Callback Interface on every write, and you get informed when the data is on the disk.
With that we avoided requiring to send a sync on every commit we perform. We can just wait for the commit completion so a transaction commit will not affect other transactions.
This JNI layer is also using DMA (Direct Memory Access) when sending the data to the disk, so the CPU will be free doing other things while the data is being written.
At this point we are focusing on making this libaio layer very performatic and stable on JBoss Messaging and depending on the demand we might do it as a separate project very easily since it is ready.
Consider this blog just a teaser. I can't wait the release of JBoss Messaging Alpha showing these numbers so everyone will be able to produce these numbers, and we are almost there. It's amazing to be part of this!
If you want a very fast and reliable Java Messaging System, JBoss Messaging 2 will be your only choice :-)
2 comments:
it's very interesting. for a university research project we developed an application with standard jms on jboss 4.05 for tracking users with bluetooth devices (cellular phones) and the bottleneck was writing messages on database.
the mdb pooling go down because messages db writing is very slow.
i'm curious to know JBoss Messaging 2 improvements.
I should have written this on the original post.
I need to thank Jeff Moyer, the current project lead or maintainer of libaio.
Without his help I wouldn't be able to finish this module.
Post a Comment