Friday, December 9, 2016

50K persistent messages per second on my laptop with Artemis

This is a guide to how to make your producers and consumers fly with Artemis


Many users have been using message systems that will favor performance instead of full guarantees of writes.

This is something we have been doing for years with the Artemis codebase, including Artemis and previous versions of HornetQ.

This is a small tutorial to do such thing with Artemis, pretty simple and quite powerful.

This will make sender to be 100% asynchronous, So you never block the producer and you can easily make thousands messages / second.

Even still the guarantees are quite high as the message will be persisted at the journal within milliseconds.

So, here is how you do it:

  • Disable Block on Persistent Messages:

There are two ways you can do that:

I - Disable blockOnDurableSend through a property:


ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory();
factory.setBlockOnDurableSend(false);



II - Disable blockOnDurableSend through the URI for the connection Factory:

ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("tcp://localhost:61616?blockOnDurableSend=false);


  • Send non Transactionally:

A transaction means a hard sync on the world. Make a round trip to the server and sync on disk.
If you send a regular Persistent Message, asynchronously, then you get a large throughput on your producers.


Here is a full example:
package org.apache.activemq.artemis.cli.commands.messages;

import javax.jms.Connection;
import javax.jms.Destination;
import javax.jms.MessageProducer;
import javax.jms.Session;

import org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory;
import org.apache.activemq.artemis.jms.client.ActiveMQDestination;

public class FastProducer {

   public static void main(String arg[]) {
      try {
         ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("tcp://localhost:61616?blockOnDurableSend=false");
         Connection connection = factory.createConnection();
         Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
         Destination queue = ActiveMQDestination.createDestination("queue://TEST", ActiveMQDestination.QUEUE_TYPE);
         MessageProducer producer = session.createProducer(queue);

         long time = System.currentTimeMillis();

         for (int i = 0; i < 100000; i++) {
            producer.send(session.createTextMessage("hello guys"));
         }

         System.out.println("Send message in " + (System.currentTimeMillis() - time) + " milliseconds");

         connection.close();
      }
      catch (Exception e) {

      }
   }
}

With this little example here I am able to send 100K messages in less than 2 seconds using my laptop.



Send message in 1772 milliseconds



And we are still improving things. Watch out for more improvements over the next month :)

Tuesday, September 13, 2016

Artemis 1.4.0 Released

Artemis 1.4.0 has been released.


This is an epic release:

- Paging had a lot of improvements. you can have a max-size for the entire broker before the destinations will start to page.

- The producers will block when the disk is beyond a % limit. (Nice feature :) )

- When you kick clients, consumers will disconnect immediately (unless you configured reconnects on core protocol).


Check the sub project page:
http://activemq.apache.org/artemis/download.html


Onward 1.5.0 now. I don't want to spoil the surprise but I am thinking of some nice things, that I will start adding JIRAs towards it :)

Tuesday, September 6, 2016


I am making a quick video on how to download and run an Artemis Broker.

On this video you will be able to hear me with my beautiful Brazilian-English accent.







Saturday, September 3, 2016

Epic change on Artemis 1.4.0 Coming up

I just sent an epic pull request towards ActiveMQ Artemis today.https://github.com/apache/activemq-artemis/pull/749

After this Artemis will monitor disk usage, and block or fail producers.

There is now a global limit for the server's memory before address start into page mode. Before you would need to specify the limit per address. Now you can have a global limit.


Besides this, artemis is getting into a nice shape for 1.4.0. I'm really excited about it coming up this week, and I should post a video or something this week on how to start a server.

Friday, May 14, 2010

Pointers in JNI / C++

When writing native code for Java (JNI), It's a common behaviour to store the pointer as long in a field in Java, and do a type-cast conversion on the native code back to the original pointer.


I did that on the native module on HornetQ, however this seems to eventually fail.

when using different bit sets as the target compilation (32 bits for instance) this seems to fail when you move back and forth.


The best way for that would be to use a Native Byte Buffer. This way you don't need to worry about conversions. You can just use a pointer back and forth:


Example:


In your Java class, you create this signature:

public class SomeClass
{
public native ByteBuffer initPointer();
}


And in your C++ code:


JNIEXPORT jobject JNICALL Java_SomeClass_initPointer
(JNIEnv * env, jobject obj)
{
return env->NewDirectByteBuffer(myPointer, 0); // size = 0, you don't want anyone to change it
}




Later, any C++ code can get back the pointer by doing:


return (MyClass *) env->GetDirectBufferAddress(myPointer);




I know the word pointer makes this a bit sour, but maybe it will be helpful to someone some day :-)

Wednesday, September 2, 2009

HornetQ and Maven

Andy Taylor has written a nice blog on how to use HornetQ on Maven. Very nice article:


http://hornetq.blogspot.com/2009/09/hornetq-simple-example-using-maven.html

Tuesday, August 11, 2009

Best Practice on Junit: Always set your fields to null on tearDown

This was a surprise to me today. Just wanted to share it thourgh a small blog entry, so maybe others are also going through the same thing.

Say you have this test:


public class MyTest extends TestCase
{

Server server;

public void setUp()
{
server = new Server();
}

public void testOne() throws Exception
{
System.gc();
Thread.sleep(1000);
}

public void testSecond() throws Exception
{
System.gc();
Thread.sleep(1000);
}

public void testThird() throws Exception
{
System.gc();
Thread.sleep(1000);
}
}


JUnit will hold one instance of MyTest for every method being executed. So if Server is a heavy weight object you will end up with three instances in the memory, until all the test methods on this class are executed.

So, as a best practice on unit tests aways set big objects such as server, connections or anything lilke that to null on a junit test.