1. Overview
In this article, we'll make a simple comparison between two well-known Java frameworks, Spring Boot and Quarkus. At the end of it, we'll have a better understanding of the differences and similarities between them, as well as some particularities.
Also, we'll perform some tests to measure their performance and observe their behavior.
2. Spring Boot
Spring Boot is a Java-based framework focusing on enterprise applications. It connects all Spring projects and helps to accelerate developers' productivity by offering many production-ready integrations.
By doing this, it reduces the amount of configuration and boilerplate. Furthermore, thanks to its convention over configuration approach, which automatically registers default configurations based on the dependencies available at the classpath in the runtime, Spring Boot considerably reduces the time-to-market for many Java applications.
3. Quarkus
Quarkus is another framework with a similar approach as the above-mentioned Spring Boot, but with an additional promise of delivering smaller artifacts with fast boot time, better resource utilization, and efficiency.
It's optimized for cloud, serverless, and containerized environments. But despite this slightly different focus, Quarkus also integrates well with the most popular Java frameworks.
4. Comparison
As mentioned above, both frameworks have great integration with other projects and frameworks. However, their internal implementations and architectures are different. For example, Spring Boot offers web capabilities in two flavors: blocking (Servlets) and nonblocking (WebFlux).
On the other hand, Quarkus also offers both approaches, but unlike Spring Boot, it allows us to use both blocking and non-blocking approaches simultaneously. Moreover, Quarkus has the reactive approach embedded in its architecture.
For that reason, to have a more exact scenario in our comparison, we'll use two entirely reactive applications implemented with Spring WebFlux and Quarkus reactive capabilities.
Also, one of the most significant features available in the Quarkus project is the ability to create native images (binary and platform-specific executables). So, we'll also include both native images in the comparison, but in the case of Spring, native image support is still in the experimental phase. To do this, we need the GraalVM.
4.1. Test Applications
Our application will expose three APIs: one allowing the user to create a zip code, the other to find the information of a particular zip code, and lastly, querying zip codes by city. These APIs were implemented using both Spring Boot and Quarkus entirely using the reactive approach, as already mentioned, and using a PostgreSQL database.
The goal was to have a simple sample application but with a little more complexity than a HelloWorld app. Of course, this will affect our comparison as the implementation of things like database drivers and serialization frameworks will influence the result. However, most applications are likely to deal with those things as well.
So, our comparison doesn't aim to be the ultimate truth about which framework is better or more performant, but rather a case study that will analyze these particular implementations.
4.2. Test Planning
In order to test both implementations, we'll use JMeter to perform the test and its metrics report to analyze our findings. Also, we'll use VisualVM to monitor the applications' resource utilization during the execution of the test.
The test will run for 5 minutes, where all APIs will be called, starting with a warmup period and after increasing the concurrent users until reaching 1,500 of them. We'll begin to populate the database during the first seconds, and then the queries kick-off, as shown below:
All the tests were performed on a machine with the following specifications:
Although not ideal because of the lack of isolation from other background processes, the test only aims to illustrate the proposed comparison. It's not the intention to provide an extensive and detailed analysis of the performance of both frameworks, as already mentioned.
5. Findings
The developer experience was great for both projects, but it's worth mentioning that Spring Boot has better documentation and more material that we can find online. Quarkus is improving in this area, but it's still a little behind.
In terms of metrics, we have:
With this experiment, we could observe that Quarkus was nearly twice as fast as Spring Boot in terms of startup time both in JVM and native versions. The build time was also much quicker. In the case of native images, the build took 9 minutes (Quarkus) vs. 13 minutes (Spring Boot), and the JVM builds took 20 seconds (Quarkus) vs. 39 seconds (Spring Boot).
The same was observed looking at the size of the artifacts, where, once again, Quarkus took the lead by producing smaller artifacts: native images with 75MB (Quarkus) vs. 109MB (Spring Boot), and JVM versions with 4KB (Quarkus) vs. 26MB (Spring Boot).
However, regarding other metrics, the conclusions are not straightforward. So, let's take a deeper look at some of them.
5.1. CPU
If we focus on the CPU usage, we'll see that the JVM versions consume more CPU at the beginning during the warmup phase. After that, the CPU usage stabilizes, and the consumption becomes relatively equal to all the versions.
Here are the CPU consumptions for Quarkus in JVM and Native versions, in that order:
5.2. Memory
Regarding memory, it's even more complicated. First, it's clear that the JVM versions of both frameworks reserve more memory for the heap. Still, it is also true that Quarkus reserves less memory from the start and the same holds for memory utilization during startup.
Then, looking at the utilization during the test, we can observe that the native images seem not to collect the memory as efficiently or as frequently as the JVM versions. It may be possible to improve this by tweaking some parameters. Nevertheless, in this comparison, we only used the default parameters.
Therefore, no changes were made to GC, JVM options, or any other parameters.
Let's have a look at the memory usage graphs:
(Spring Boot JVM)
(Quarkus JVM)
(Spring Boot Native)
(Quarkus Native)
Quarkus seemed to consume fewer resources in terms of memory, despite having higher spikes during the test.
5.3. Response Time
Lastly, regarding response times and the number of threads used in the peak, Spring Boot seems to have the advantage here. It was able able to handle the same load using fewer threads while also having better response times.
The Spring Boot Native version has shown better performance in this case. But let's look at the response time distribution of each version of the application:
(Spring Boot JVM)
Despite having more outliers, the Spring Boot JVM version had the best evolution over time, most likely due to the JIT compiler optimizations:
(Quarkus JVM)
(Spring Boot Native)
(Quarkus Native)
Quarkus showed itself to be powerful in terms of low resource utilization. However, at least in this experiment, Spring Boot showed better throughput and responsiveness. Despite that, both frameworks were able to handle all the requests without any errors.
Not only this, but their performance was pretty similar, and there wasn't a significant disparity between them.
5.4. Connecting the Dots
All things considered, both frameworks proved to be great options when it came to implementing Java applications.
The native apps have shown to be fast and to have low resource consumption, being excellent choices for serverless, short-living applications and environments where low resource consumption is critical.
On the other hand, the JVM apps seem to have more overhead but excellent stability and high throughput over time, being ideal for robust, long-living applications.
6. Conclusion
In this article, we compared the Spring Boot and Quarkus frameworks and their different deployment modes, JVM and Native. We also looked at other metrics and aspects of those applications.
As usual, the code of the test application and scripts used to test them are available over on GitHub.
