Introduction to Chronicle Queue

1. Overview

Chronicle Queue persists every single message using a memory-mapped file. This allows us to share messages between processes.

It stores data directly to off-heap memory, therefore, making it free of GC overhead. It is designed for providing low-latency message framework for high-performance applications.

In this quick article, we will look into the basic set of operations.

2. Maven Dependencies

We need to add the following dependency:

<dependency>
    <groupId>net.openhft</groupId>
    <artifactId>chronicle</artifactId>
    <version>3.6.4</version>
</dependency>

We can always check the latest versions hosted by the Maven Central with the link provided before.

3. Building Blocks

There are three concepts characteristic for Chronicle Queue:

• Excerpt –  is a data container
• Appender – appender is used for writing data
• Trailer – is used for sequentially reading data

We’ll reserve the portion of memory for read-write operations using Chronicle interface.

Here is the example code for creating an instance:

File queueDir = Files.createTempDirectory("chronicle-queue").toFile();
Chronicle chronicle = ChronicleQueueBuilder.indexed(queueDir).build();

We will need a base directory where the queue will persist records in memory-mapped files.

ChronicleQueueBuilder class provides different types of queues. In this case, we used IndexedChronicleQueue which uses the sequential index to maintain memory offsets of records in a queue.

4. Writing to the Queue

To write the items to a queue, we’ll need to create an object of ExcerptAppender class using Chronicle instance. Here is example code for writing the messages to the queue:

Here is example code for writing the messages to the queue:

ExcerptAppender appender = chronicle.createAppender();
appender.startExcerpt();

String stringVal = "Hello World";
int intVal = 101;
long longVal = System.currentTimeMillis();
double doubleVal = 90.00192091d;

appender.writeUTF(stringValue);
appender.writeInt(intValue);
appender.writeLong(longValue);
appender.writeDouble(doubleValue);
appender.finish();

After creating the appender, we will start the appender using a startExcerpt method. It starts an Excerpt with the default message capacity of 128K. We can use an overloaded version of startExcerpt to provide a custom capacity.

Once started, we can write any literal or object value to the queue using a wide range of write methods provided by the library.

Finally, when we’re done with writing, we’ll finish the excerpt, save the data to a queue, and later to disc.

5. Reading from the Queue

Reading the values from the queue can easily be done using the ExcerptTrailer instance.

It is just like an iterator we use to traverse a collection in Java.

Let’s read values from the queue:

ExcerptTailer tailer = chronicle.createTailer();
while (tailer.nextIndex()) {
    tailer.readUTF();
    tailer.readInt();
    tailer.readLong();
    tailer.readDouble();
}
tailer.finish();

After creating the trailer, we use the nextIndex method to check if there is a new excerpt to read.

Once ExcerptTailer has a new Excerpt to read, we can read messages from it using a range of read methods for literal and object type values.

Finally, we finish the reading with the finish API.

6. Conclusion

In this tutorial, we gave a brief introduction to the Chronicle Queue and its building blocks. We saw how to create a queue, write and read data. Using it offers many benefits including low latency, durable interprocess communication (IPC) as well as no Garbage Collection overhead.

The solution provides data persistence through memory mapped files – with no data loss. It also allows concurrent read-writes from multiple processes; however, writes are handled synchronously.

As always, all code snippets can be found over on GitHub.