Quantcast
Channel: Baeldung
Viewing all 4469 articles
Browse latest View live

Java Weekly, Issue 516

$
0
0

1. Spring and Java

>> Table partitioning with Spring and Hibernate [vladmihalcea.com]

Splitting a large table into multiple smaller partition tables using Spring and Hibernate — allowing a more efficient seek/scan.

>> Pattern Matching for switch – Sip of Java [inside.java]

And, a crash course on Pattern Matching in Java 21: type patterns, guard conditions, null handling, exhaustiveness, sealed hierarchies, and records.

Also worth reading:

Webinars and presentations:

Time to upgrade:

2. Technical & Musings

>> Exploring the OpenTelemetry Collector [blog.frankel.ch]

Exploring the OpenTelemetry collector: setup, intermediary data processing, receivers, exporters, log manipulation, and more!

Also worth reading:

3. Pick of the Week

Our only “sale” of the year is now live – Black Friday.

If you’re planning to go deep into Spring and Spring Boot, this is a good time to explore:

>> All Baeldung Courses

       

How to Effectively Unit Test CompletableFuture

$
0
0

1. Introduction

CompletableFuture is a powerful tool for asynchronous programming in Java. It provides a convenient way to chain asynchronous tasks together and handle their results. It is commonly used in situations where asynchronous operations need to be performed, and their results need to be consumed or processed at a later stage.

However, unit testing CompletableFuture can be challenging due to its asynchronous nature. Traditional testing methods, which rely on sequential execution, often fall short of capturing the nuances of asynchronous code. In this tutorial, we’ll discuss how to effectively unit test CompletableFuture using two different approaches: black-box testing and state-based testing.

2. Challenges of Testing Asynchronous Code

Asynchronous code introduces challenges due to its non-blocking and concurrent execution, posing difficulties in traditional testing methods. These challenges include:

  • Timing Issues: Asynchronous operations introduce timing dependencies into the code, making it difficult to control the execution flow and verify the behavior of the code at specific points in time. Traditional testing methods that rely on sequential execution may not be suitable for asynchronous code.
  • Exception Handling: Asynchronous operations can potentially throw exceptions, and it’s crucial to ensure that the code handles these exceptions gracefully and doesn’t fail silently. Unit tests should cover various scenarios to validate exception handling mechanisms.
  • Race Conditions: Asynchronous code can lead to race conditions, where multiple threads or processes attempt to access or modify shared data simultaneously, potentially resulting in unexpected outcomes.
  • Test Coverage: Achieving comprehensive test coverage for asynchronous code can be challenging due to the complexity of interactions and the potential for non-deterministic outcomes.

3. Black-Box Testing

Black-box testing focuses on testing the external behavior of the code without knowledge of its internal implementation. This approach is suitable for validating asynchronous code behavior from the user’s perspective. The tester only knows the inputs and expected outputs of the code.

When testing CompletableFuture using black-box testing, we prioritize the following aspects:

  • Successful Completion: Verifying that the CompletableFuture completes successfully, returning the anticipated result.
  • Exception Handling: Validating that the CompletableFuture handles exceptions gracefully, preventing silent failures.
  • Timeouts: Ensuring that the CompletableFuture behaves as expected when encountering timeouts.

We can use a mocking framework like Mockito to mock the dependencies of the CompletableFuture under test. This will allow us to isolate the CompletableFuture and test its behavior in a controlled environment.

3.1. System Under Test

We will be testing a method named processAsync() that encapsulates the asynchronous data retrieval and combination process. This method accepts a list of Microservice objects as input and returns a CompletableFuture<String>. Each Microservice object represents a microservice capable of performing an asynchronous retrieval operation.

The processAsync() utilizes two helper methods, fetchDataAsync() and combineResults(), to handle the asynchronous data retrieval and combination tasks:

CompletableFuture<String> processAsync(List<Microservice> microservices) {
    List<CompletableFuture<String>> dataFetchFutures = fetchDataAsync(microservices);
    return combineResults(dataFetchFutures);
}

The fetchDataAsync() method streams through the Microservice list, invoking retrieveAsync() for each, and returns a list of CompletableFuture<String>:

private List<CompletableFuture<String>> fetchDataAsync(List<Microservice> microservices) {
    return microservices.stream()
        .map(client -> client.retrieveAsync(""))
        .collect(Collectors.toList());
}

The combineResults() method uses CompletableFuture.allOf() to wait for all futures in the list to complete. Once complete, it maps the futures, joins the results, and returns a single string:

private CompletableFuture<String> combineResults(List<CompletableFuture<String>> dataFetchFutures) {
    return CompletableFuture.allOf(dataFetchFutures.toArray(new CompletableFuture[0]))
      .thenApply(v -> dataFetchFutures.stream()
        .map(future -> future.exceptionally(ex -> {
            throw new CompletionException(ex);
        })
          .join())
      .collect(Collectors.joining()));
}

3.2. Test Case: Verify Successful Data Retrieval and Combination

This test case verifies that the processAsync() method correctly retrieves data from multiple microservices and combines the results into a single string:

@Test
public void givenAsyncTask_whenProcessingAsyncSucceed_thenReturnSuccess() 
  throws ExecutionException, InterruptedException {
    Microservice mockMicroserviceA = mock(Microservice.class);
    Microservice mockMicroserviceB = mock(Microservice.class);
    when(mockMicroserviceA.retrieveAsync(any())).thenReturn(CompletableFuture.completedFuture("Hello"));
    when(mockMicroserviceB.retrieveAsync(any())).thenReturn(CompletableFuture.completedFuture("World"));
    CompletableFuture<String> resultFuture = processAsync(List.of(mockMicroserviceA, mockMicroserviceB));
    String result = resultFuture.get();
    assertEquals("HelloWorld", result);
}

3.3. Test Case: Verify Exception Handling When Microservice Throws an Exception

This test case verifies that the processAsync() method throws an ExecutionException when one of the microservices throws an exception. It also asserts that the exception message is the same as the exception thrown by the microservice:

@Test
public void givenAsyncTask_whenProcessingAsyncWithException_thenReturnException() 
  throws ExecutionException, InterruptedException {
    Microservice mockMicroserviceA = mock(Microservice.class);
    Microservice mockMicroserviceB = mock(Microservice.class);
    when(mockMicroserviceA.retrieveAsync(any())).thenReturn(CompletableFuture.completedFuture("Hello"));
    when(mockMicroserviceB.retrieveAsync(any()))
      .thenReturn(CompletableFuture.failedFuture(new RuntimeException("Simulated Exception")));
    CompletableFuture<String> resultFuture = processAsync(List.of(mockMicroserviceA, mockMicroserviceB));
    ExecutionException exception = assertThrows(ExecutionException.class, resultFuture::get);
    assertEquals("Simulated Exception", exception.getCause().getMessage());
}

3.4. Test Case: Verify Timeout Handling When Combined Result Exceeds Timeout

This test case attempts to retrieve the combined result from the processAsync() method within a specified timeout of 300 milliseconds. It asserts that a TimeoutException is thrown when the timeout is exceeded:

@Test
public void givenAsyncTask_whenProcessingAsyncWithTimeout_thenHandleTimeoutException() 
  throws ExecutionException, InterruptedException {
    Microservice mockMicroserviceA = mock(Microservice.class);
    Microservice mockMicroserviceB = mock(Microservice.class);
    Executor delayedExecutor = CompletableFuture.delayedExecutor(200, TimeUnit.MILLISECONDS);
    when(mockMicroserviceA.retrieveAsync(any()))
      .thenReturn(CompletableFuture.supplyAsync(() -> "Hello", delayedExecutor));
    Executor delayedExecutor2 = CompletableFuture.delayedExecutor(500, TimeUnit.MILLISECONDS);
    when(mockMicroserviceB.retrieveAsync(any()))
      .thenReturn(CompletableFuture.supplyAsync(() -> "World", delayedExecutor2));
    CompletableFuture<String> resultFuture = processAsync(List.of(mockMicroserviceA, mockMicroserviceB));
    assertThrows(TimeoutException.class, () -> resultFuture.get(300, TimeUnit.MILLISECONDS));
}

The above code uses CompletableFuture.delayedExecutor() to create executors that will delay the completion of the retrieveAsync() calls by 200 and 500 milliseconds, respectively. This simulates the delays caused by the microservices and allows the test to verify that the processAsync() method handles timeouts correctly.

4. State-Based Testing

State-based testing focuses on verifying the state transitions of the code as it executes. This approach is particularly useful for testing asynchronous code, as it allows testers to track the code’s progress through different states and ensure that it transitions correctly.

For example, we can verify that the CompletableFuture transitions to the completed state when the asynchronous task is completed successfully. Otherwise, it transits to a failed state when an exception occurs, or the task is cancelled due to interruption.

4.1. Test Case: Verify State After Successful Completion

This test case verifies that a CompletableFuture instance transitions to the done state when all of its constituent CompletableFuture instances have been completed successfully:

@Test
public void givenCompletableFuture_whenCompleted_thenStateIsDone() {
    Executor delayedExecutor = CompletableFuture.delayedExecutor(200, TimeUnit.MILLISECONDS);
    CompletableFuture<String> cf1 = CompletableFuture.supplyAsync(() -> "Hello", delayedExecutor);
    CompletableFuture<String> cf2 = CompletableFuture.supplyAsync(() -> " World");
    CompletableFuture<String> cf3 = CompletableFuture.supplyAsync(() -> "!");
    CompletableFuture<String>[] cfs = new CompletableFuture[] { cf1, cf2, cf3 };
    CompletableFuture<Void> allCf = CompletableFuture.allOf(cfs);
    assertFalse(allCf.isDone());
    allCf.join();
    String result = Arrays.stream(cfs)
      .map(CompletableFuture::join)
      .collect(Collectors.joining());
    assertFalse(allCf.isCancelled());
    assertTrue(allCf.isDone());
    assertFalse(allCf.isCompletedExceptionally());
}

4.2. Test Case: Verify State After Completing Exceptionally

This test case verifies that when one of the constituent CompletableFuture instances cf2 completes exceptionally, and the allCf CompletableFuture transitions to the exceptional state:

@Test
public void givenCompletableFuture_whenCompletedWithException_thenStateIsCompletedExceptionally() 
  throws ExecutionException, InterruptedException {
    Executor delayedExecutor = CompletableFuture.delayedExecutor(200, TimeUnit.MILLISECONDS);
    CompletableFuture<String> cf1 = CompletableFuture.supplyAsync(() -> "Hello", delayedExecutor);
    CompletableFuture<String> cf2 = CompletableFuture.failedFuture(new RuntimeException("Simulated Exception"));
    CompletableFuture<String> cf3 = CompletableFuture.supplyAsync(() -> "!");
    CompletableFuture<String>[] cfs = new CompletableFuture[] { cf1, cf2, cf3 };
    CompletableFuture<Void> allCf = CompletableFuture.allOf(cfs);
    assertFalse(allCf.isDone());
    assertFalse(allCf.isCompletedExceptionally());
    assertThrows(CompletionException.class, allCf::join);
    assertTrue(allCf.isCompletedExceptionally());
    assertTrue(allCf.isDone());
    assertFalse(allCf.isCancelled());
}

4.3. Test Case: Verify State After Task Cancelled

This test case verifies that when the allCf CompletableFuture is canceled using the cancel(true) method, it transitions to the cancelled state:

@Test
public void givenCompletableFuture_whenCancelled_thenStateIsCancelled() 
  throws ExecutionException, InterruptedException {
    Executor delayedExecutor = CompletableFuture.delayedExecutor(200, TimeUnit.MILLISECONDS);
    CompletableFuture<String> cf1 = CompletableFuture.supplyAsync(() -> "Hello", delayedExecutor);
    CompletableFuture<String> cf2 = CompletableFuture.supplyAsync(() -> " World");
    CompletableFuture<String> cf3 = CompletableFuture.supplyAsync(() -> "!");
    CompletableFuture<String>[] cfs = new CompletableFuture[] { cf1, cf2, cf3 };
    CompletableFuture<Void> allCf = CompletableFuture.allOf(cfs);
    assertFalse(allCf.isDone());
    assertFalse(allCf.isCompletedExceptionally());
    allCf.cancel(true);
    assertTrue(allCf.isCancelled());
    assertTrue(allCf.isDone());
}

5. Conclusion

In conclusion, unit testing CompletableFuture can be challenging due to its asynchronous nature. However, it is an important part of writing robust and maintainable asynchronous code. By using black-box and state-based testing approaches, we can assess the behavior of our CompletableFuture code under various conditions, ensuring that it functions as expected and handles potential exceptions gracefully.

As always, the example code is available over on GitHub.

Get Index of First Element Matching Boolean Using Java Streams

$
0
0

1. Introduction

Finding the index of an element from a data structure is a common task for developers. In this tutorial, we’ll use the Java Stream API and third-party libraries to find the index of the first element in a List that matches a boolean condition.

2. Setup

In this article, we’ll write a few test cases using the User object mentioned below to achieve our goal:

public class User {
    private String userName;
    private Integer userId;
   // constructor and getters
}

Moreover, we’ll create an ArrayList of the User object to use in all test cases. After that, we’ll find the index of the first user, whose name is “John”:

List<User> userList = List.of(new User(1, "David"), new User(2, "John"), new User(3, "Roger"), new User(4, "John"));
String searchName = "John";

3. Using Java Stream API

The Java Stream API was one of the best features introduced in Java 8. It provides numerous methods to iterate, filter, map, match, and collect the data. With this in mind, let’s use these methods to find an index from a List.

3.1. Using stream() and filter()

Let’s write a test case using the basic functions of the Stream class in order to obtain an index:

@Test
public void whenUsingStream_thenFindFirstMatchingUserIndex() {
    AtomicInteger counter = new AtomicInteger(-1);
    int index = userList.stream()
      .filter(user -> {
          counter.getAndIncrement();
          return searchName.equals(user.getUserName());
      })
      .mapToInt(user -> counter.get())
      .findFirst()
      .orElse(-1);
    assertEquals(1, index);
}

Here, we can create a Stream from the List and apply a filter() method. Inside the filter() method, we increment the AtomicInteger to track the element’s index. To finish, we map the counter value and use the findFirst() method to get the index of the first matched element.

3.2. Using IntStream

Alternatively, we can use the IntStream class to iterate over List elements and get the index using similar logic as mentioned in the above section:

@Test
public void whenUsingIntStream_thenFindFirstMatchingUserIndex() {
    int index = IntStream.range(0, userList.size() - 1)
      .filter(streamIndex -> searchName.equals(userList.get(streamIndex).getUserName()))
      .findFirst()
      .orElse(-1);
    assertEquals(1, index);
}

3.3. Using Stream takeWhile()

The takeWhile() method returns the data until the predicate remains true. However, once the predicate fails, it stops the iteration to collect the iterated data:

@Test
public void whenUsingTakeWhile_thenFindFirstMatchingUserIndex() {
    long predicateIndex = userList.stream()
      .takeWhile(user -> !user.getUserName().equals(searchName))
      .count();
    assertEquals(1, predicateIndex);
}

The example above shows that the takeWhile() method collects elements until a User object named “John” is found and then stops the iteration. After that, we can use the count() method to get the index of the first matched element.

Let’s take another case where there is no matching element present in the list. In this case, the iteration continues till the last element, and the output value is 4, which is the total iterated elements from the input list:

@Test
public void whenUsingTakeWhile_thenFindIndexFromNoMatchingElement() {
    long predicateIndex = userList.stream()
      .takeWhile(user -> !user.getUserName().equals(searchName))
      .count();
    assertEquals(4, predicateIndex);
}

The takeWhile() method was introduced in Java 9.

4. Using Third-Party Libraries

Though the Java Stream API is sufficient to achieve our goal, it’s only available from the Java 1.8 version. If the application is on an older version of Java, then external libraries become useful.

4.1. Iterables From Google Guava

We’ll add the latest Maven dependency to pom.xml:

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>32.1.2-jre</version>
</dependency>

The Iterables class from Guava Library has a method named indexOf(), which returns the index of the first element in the specified iterable that matches the given predicate:

@Test
public void whenUsingGoogleGuava_thenFindFirstMatchingUserIndex() {
    int index = Iterables.indexOf(userList, user -> searchName.equals(user.getUserName()));
    assertEquals(1, index);
}

4.2. IterableUtils From Apache Common Collections

Similarly, the IterableUtils class from the Apache Common Collections library also provides functionalities to obtain an index. Let’s add the Maven dependency in pom.xml:

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-collections4</artifactId>
    <version>4.4</version>
</dependency>

The IterableUtils.indexOf() method accepts an iterable collection and a predicate and then returns the index of the first matching element:

@Test
public void whenUsingApacheCommons_thenFindFirstMatchingUserIndex() {
    int index = IterableUtils.indexOf(userList, user -> searchName.equals(user.getUserName()));
    assertEquals(1, index);
}

The indexOf() method in both libraries returns -1 if no element meets the predicate criteria.

5. Conclusion

In this article, we learned different ways to find the index of the first element in a List that matches a boolean condition. We used the Java Stream API, the Iterables class from Google Guava, and the IterableUtils class from Apache Commons Collections.

As always, the reference code is available over on GitHub.

String’s Maximum Length in Java

$
0
0

1. Introduction

Onе of thе fundamеntal data typеs in Java is thе String class, which rеprеsеnts a sеquеncе of charactеrs. Howеvеr, undеrstanding thе maximum lеngth of a String in Java is crucial for writing robust and еfficiеnt codе.

In this tutorial, wе’ll еxplorе thе constraints and considеrations rеlatеd to thе maximum lеngth of strings in Java.

2. Mеmory Constraints

Thе maximum lеngth of a String in Java is closеly tiеd to thе availablе mеmory. In Java, strings arе storеd in thе hеap mеmory, and thе maximum sizе of an objеct in thе hеap is constrainеd by thе maximum addrеssablе mеmory.

However, the limitation is platform-dеpеndеnt and can vary basеd on thе Java Virtual Machinе (JVM) implеmеntation and thе undеrlying hardwarе.

Let’s look at an example:

long maxMemory = Runtime.getRuntime().maxMemory();

In thе abovе еxamplе, wе usе thе Runtimе class to obtain thе maximum availablе mеmory for thе JVM.

3. Intеgеr.MAX_VALUE Limit

Although the theoretical maximum length of a string depends upon available memory, it gets restricted by the constraint imposed by Integer.MAX_Value in real practice. This is because Java String length is represented as an int data type:

int maxStringLength = Integer.MAX_VALUE;

In thе abovе snippеt, wе sеt thе maxLеngth variablе to Intеgеr.MAX_VALUE, which rеprеsеnts thе maximum positivе valuе that can bе hеld by an int.

Thеrеforе, any attеmpt to crеatе a string longеr than this limit will rеsult in an ovеrflow of thе int data typе as follows:

try {
    int maxLength = Integer.MAX_VALUE + 20;
    char[] charArray = new char[maxLength];
    for (int i = 0; i < maxLength; i++) {
        charArray[i] = 'a';
    }
    String longString = new String(charArray);
    System.out.println("Successfully created a string of length: " + longString.length());
} catch (OutOfMemoryError e) {
    System.err.println("Overflow error: Attempting to create a string longer than Integer.MAX_VALUE");
    e.printStackTrace();
}

In this еxamplе, wе usе a StringBuildеr to attеmpt to crеatе a string longеr than Intеgеr.MAX_VALUE. Thе loop appеnds charactеrs to thе StringBuildеr until it еxcееds thе maximum positivе valuе that can bе rеprеsеntеd by an int.

Thе program intеntionally catchеs thе OutOfMеmoryError that occurs whеn thе ovеrflow happеns and prints an еrror mеssagе.

4. Conclusion

In conclusion, understanding the maximum lеngth constraints of Java strings is crucial for robust coding. Whilе influеncеd by availablе mеmory, thе practical limitation sеt by Intеgеr.MAX_VALUE undеrscorеs thе nееd to considеr both mеmory availability and programming constraints.

As always, the complete code samples for this article can be found over on GitHub.

       

Spring Kafka Trusted Packages Feature

$
0
0

1. Introduction

In this tutorial, we’ll review the Spring Kafka trusted packages feature. We’ll see the motivation behind it, along with its usage. All with practical examples, as always.

2. Prerequisite

In general, the Spring Kafka module allows us, as users, to specify some metadata about the POJO we’re sending. It usually takes the form of the Kafka message headers. For instance, if we’ll configure the ProducerFactory in this way:

@Bean
public ProducerFactory<Object, SomeData> producerFactory() {
    JsonSerializer<SomeData> jsonSerializer = new JsonSerializer<>();
    jsonSerializer.setAddTypeInfo(true);
    return new DefaultKafkaProducerFactory<>(
      producerFactoryConfig(),
      new StringOrBytesSerializer(),
      jsonSerializer
    );
}
@Data
@AllArgsConstructor
static class SomeData {
    private String id;
    private String type;
    private String status;
    private Instant timestamp;
}

Then we’ll produce a new message into a topic, for example, using KafkaTemplate configured with producerFactory above:

public void sendDataIntoKafka() {
    SomeData someData = new SomeData("1", "active", "sent", Instant.now());
    kafkaTemplate.send(new ProducerRecord<>("sourceTopic", null, someData));
}

Then, in this case, we’ll get the following message in the console of the Kafka consumer:

CreateTime:1701021806470 __TypeId__:com.baeldung.example.SomeData null {"id":"1","type":"active","status":"sent","timestamp":1701021806.153965150}

As we can see, the type information of the POJO that is inside the message is in the headers. This is, of course, the Spring Kafka feature recognized by Spring only. Meaning, these headers are just metadata from Kafka or other framework’s points of view. Therefore, we can assume here that both the consumer and the producer use Spring to handle Kafka messaging.

3. Trusted Packages Feature

Having said that, we may say that, in some cases, this is quite a useful feature. When messages in the topic have different payload schema, then hinting at the payload type for the consumer will be great.

However, in general, we know what messages in terms of their schemas can occur in the topic. So, this might be a great idea, to restrict the possible payload schemas consumer will accept. This is what the Spring Kafka trusted packages feature is about.

4. Usages Samples

Trusted packages Spring Kafka feature is configured on the deserializer level. If trusted packages are configured, then Spring will make a lookup into the type headers of the incoming message. Then, it will check that all of the provided types in the message are trusted – both key and value.

It essentially means that Java classes of key and value, specified in the corresponding headers, must reside inside trusted packages. If everything is ok, then Spring passes the message into further deserialization. If the headers are not present, then Spring will just deserialize the object and won’t check the trusted packages:

@Bean
public ConsumerFactory<String, SomeData> someDataConsumerFactory() {
    JsonDeserializer<SomeData> payloadJsonDeserializer = new JsonDeserializer<>();
    payloadJsonDeserializer.addTrustedPackages("com.baeldung.example");
    return new DefaultKafkaConsumerFactory<>(
      consumerConfigs(),
      new StringDeserializer(),
      payloadJsonDeserializer
    );
}

It also may be worth mentioning, that Spring can trust all packages if we substitute the concrete packages with star (*):

JsonDeserializer<SomeData> payloadJsonDeserializer = new JsonDeserializer<>();
payloadJsonDeserializer.trustedPackages("*");

However, in such cases, the usage of trusted packages does not do anything and just incurs additional overhead. Let’s now jump into the motivation behind the feature we just saw.

5.1. First Motivation: Consistency

This feature is great because of two major reasons. First, we can fail fast if something goes wrong in the cluster. Imagine that a particular producer will accidentally publish messages in a topic that he is not supposed to publish. It can cause a lot of problems, especially if we succeed at deserializing the incoming message. In this case, the whole system behavior can be undefined.

So if the producer publishes messages with type information included and the consumer knows what types it trusts, then this all can be avoided. This, of course, assumes that the producer’s message type is different from the one that the consumer expects. But this assumption is pretty fair since this producer shouldn’t publish messages into this topic at all.

5.2. Second Motivation: Security

But what is most important is the security concern. In our previous example, we emphasized that the producer has published messages into the topic unintentionally. But that could be an intentional attack as well. The malicious producer might intentionally publish a message into a particular topic in order to exploit the deserialization vulnerabilities. So by preventing the deserialization of unwanted messages, Spring provides additional security measures to reduce security risks.

What is really important to understand here is that the trusted packages feature is not a solution for the “headers spoofing” attack. In this case, the attacker manipulates the headers of a message to deceive the recipient into believing that the message is legitimate and originated from a trusted source. So by providing the correct type headers, the attacker may deceive Spring, and the latter will proceed with message deserialization. But this problem is quite complex and is not a topic of the discussion. In general, Spring merely provides an additional security measure to minimize the risk of the hacker’s success.

6. Conclusion

In this article, we explored the Spring Kafka trusted packages feature. This feature provides additional consistency and security to our distributed messaging system. Still, it is critical to keep in mind, that trusted packages are still vulnerable to header spoofing attacks. Still, Spring Kafka does a great job at providing additional security measures.

As always, the source code for this article is available over on GitHub.

       

Understanding NewSQL Databases

$
0
0

1. Overview

Databases are one of the ways we store collections of data and there are different types of databases available. Different DBMSs, such as SQL and NoSQL databases, became popular based on the requirements (performance, consistency, etc.) and the type of data (structured, schemaless, etc.) to be stored. We now have something called NewSQL Databases which combine the best of both SQL and NoSQL databases.

In this tutorial, we’ll take a look at SQL and NoSQL databases and then understand what NewSQL databases are all about.

2. SQL and NoSQL Databases

For decades, traditional SQL databases have served as the foundation of data storage and retrieval. SQL Databases provide robust ACID compliance, guaranteeing dependability, consistency, and data integrity. Some of the popular use cases include OLTP and OLAP applications and Data Warehousing applications.

On the other hand, database requirements changed along with the digital landscape. With the rise of the internet came an abundance of data, as multiple sources produced huge amounts of data instantly. Despite their consistency and dependability, traditional SQL databases found it difficult to handle the demands of these fast-moving data streams.

As a result, NoSQL databases emerged as viable alternatives. NoSQL databases put performance, scalability, and flexibility above ACID compliance. They use several data models (document, key-value, column-family, etc.) that enable them to perform well in particular use cases, including networked systems, real-time analytics, and unstructured data storage. Social media applications and document-oriented applications are some of the most common use cases.

While NoSQL databases offered a solution to the scalability problem, they came with trade-offs, most notably relaxed consistency models. In scenarios where strong data consistency and transactional guarantees were essential, NoSQL databases fell short. This led to the need for a new kind of database system which could have the best of both, SQL and NoSQL.

3. NewSQL Database

NewSQL databases try to address the limitations of the existing databases. They’re engineered as a relational database with a distributed and fault-tolerant architecture. They aim to provide solutions by providing a database with the following features:

  • Scalability and ACID compliance: Designed to scale horizontally, the NewSQL databases handle large amounts of data by distributing them across nodes/clusters. Additionally, they maintain strict ACID compliance resulting in a system that is highly available and with strong transactional integrity.
  • Performance optimization: Various techniques, such as in-memory processing, indexing, and caching, are implemented to provide low-latency data access and high performance.
  • Distributed architecture: Multiple nodes are used to replicate data so that there is no single point of failure. Consequently, it ensures high availability and fault tolerance
  • SQL compatibility: NewSQL systems are compatible with SQL query language thus avoiding re-learning and migration overheads

3.1. Use-Cases

NewSQL databases are most suited for applications requiring strong transactional consistency along with high performance and scalability. Let’s take a look at some of the use cases below:

  • Financial systems: A large amount of data needs to be processed with low latency and high accuracy
  • E-commerce platforms: Although the load is relatively stable and expected, periodically there may be a surge of data/load that needs to be supported
  • Real-time feedback systems: Many systems that rely on real-time data analysis like airline pricing, fraud detection, etc. can benefit from a high-performant transactional system.
  • Smart devices and cities: With increased automation across multiple sectors and use cases, the continuous data stream can be processed efficiently using a NewSQL database.

However, there are some use cases where the NewSQL database may not be suitable:

  • Well-established data models: Applications with well-established data models with manageable load and/or scalability capabilities may not be suitable candidates for migration to NewSQL databases, e.g., legacy applications
  • Predictable loads: Applications with predictable loads that may not need to leverage dynamic scalability
  • Strict ACID compliance: SQL databases better serve applications with non-negotiable ACID properties

Let’s take a look at some of the popular NewSQL databases:

  • CockroachDB: An open-source distributed database designed to survive different types of failures while still maintaining ACID compliance. It uses distributed architecture and provides strong failover capabilities along with automatic data replication.
  • NuoDB: It uses a patented “elastically scalable databases” architecture to provide NoSQL benefits while retaining ACID compliance.
  • VoltDB: An in-memory database that uses a shared-nothing architecture designed for high-velocity data ingestion.

3.3. Drawbacks

While NewSQL databases address the limitations of SQL and NoSQL databases, they come with their own set of drawbacks:

  • Use case specific: Different NewSQL databases are suitable for different use cases and there is no single solution for all use cases
  • Complex learning curve: Since NewSQL databases tend to be use case specific, each new implementation may require a non-overlapping learning curve
  • Compatibility issues: NewSQL databases may not be always compatible with existing data models and schemas which may lead to considerable migration efforts

4. Conclusion

In this article, we explored the evolution of data storage and retrieval from traditional SQL to NoSQL databases and then finally to NewSQL databases. We also looked at the different use cases for NewSQL databases as well as some of the popular NewSQL databases.

NewSQL databases bridge the gap between SQL and NoSQL databases by combining transactional consistency with scalability and performance. We saw some of the use cases where it may be beneficial to use any of the enumerated NewSQL databases.

       

Verify That Lambda Expression Was Called Using Mockito

$
0
0

1. Overview

In this tutorial, we’ll look at how we can test that our code calls a lambda function. There are two approaches to achieving this goal to consider. We’ll first check that the lambda is invoked with the correct arguments. Then, we’ll look at testing the behavior instead and checking if the lambda code has executed and produced the expected result.

2. Example Class Under Test

To start, let’s create a class LambdaExample that has an ArrayList we’ll call bricksList:

class LambdaExample {
    ArrayList<String> bricksList = new ArrayList<>();
}

Now, let’s add an inner class called BrickLayer, which will be able to add bricks for us:

class LambdaExample {
    BrickLayer brickLayer = new BrickLayer();
    class BrickLayer {
        void layBricks(String bricks) {
            bricksList.add(bricks);
        }
    }
}

BrickLayer doesn’t do much. It has a single method, layBricks() that will add a brick to our List for us. This could have been an external class, but to keep the concepts together and simple an inner class works here.

Finally, we can add a method to LambdaExample to call layBricks() via a lambda:

void createWall(String bricks) {
    Runnable build = () -> brickLayer.layBricks(bricks);
    build.run();
}

Again, we’ve kept things simple. Our real-world applications are more complex but this streamlined example will help explain the test methods.

In the upcoming sections, we’ll test whether calling createWall() results in the expected execution of layBricks() within our lambda.

3. Testing Correct Invocation

The first testing method we’ll look at is based on confirming that the lambda is called when we expect it. Furthermore, we’ll need to confirm that it received the correct arguments. To start we’ll need to create Mocks of both BrickLayer and LambdaExample:

@Mock
BrickLayer brickLayer;
@InjectMocks
LambdaExample lambdaExample;

We’ve applied the @InjectMocks annotation to LambdaExample so that it uses the mocked BrickLayer object. We’ll be able to confirm the call to the layBricks() method because of this.

We can now write our test:

@Test
void whenCallingALambda_thenTheInvocationCanBeConfirmedWithCorrectArguments() {
    String bricks = "red bricks";
    lambdaExample.createWall(bricks);
    verify(brickLayer).layBricks(bricks);
}

In this test, we’ve defined the String we want to add to bricksList and passed it as an argument to createWall(). Let’s keep in mind that we’re using the Mock we created earlier as the instance of LambdaExample.

We’ve then used Mockitos verify() function. Verify() is hugely helpful for this kind of test. It confirms the function layBricks() was called and that the argument was what we expected.

There’s much more we can do with verify(). For example, confirming how many times a method is called. For our purposes, however, it’s sufficient to confirm that our lambda invoked the method as expected.

4. Testing Correct Behaviour

The second route we can go down for testing is to not worry about what gets called and when. Instead, we’ll confirm that the expected behavior of the lambda function occurs. There will almost always be a good reason we’re calling a function. Perhaps to perform a calculation or to get or set a variable.

In our example, the lambda adds a given String to an ArrayList. In this section, let’s verify that the lambda successfully executes that task:

@Test
void whenCallingALambda_thenCorrectBehaviourIsPerformed() {
    LambdaExample lambdaExample = new LambdaExample();
    String bricks = "red bricks";
        
    lambdaExample.createWall(bricks);
    ArrayList<String> bricksList = lambdaExample.getBricksList();
        
    assertEquals(bricks, bricksList.get(0));
}

Here, we’ve created an instance of the LambdaExample class. Next, we’ve called createWall() to add a brick to the ArrayList.

We should now see that bricksList contains the String we just added. Assuming the code correctly executes the lambda. We confirmed that by retrieving bricksList from lambdaExample and checking the contents.

We can conclude that the lambda is executing as expected, as that’s the only way our String could have ended up in the ArrayList.

5. Conclusion

In this article, we’ve looked at two methods for testing lambda calls. The first is useful when we can mock the class containing the function and inject it into the class which calls it as a lambda. In that case, we can use Mockito to verify the call to the function and the correct arguments. This offers no confidence that the lambda went on to do what we expected, however.

The alternative is to test that the lambda produces the expected results when called. This offers more test coverage and is often preferable if it’s simple to access and confirm the correct behavior of the function call.

As always, the full code for the examples is available over on GitHub.

       

Deserializing JSON to Java Record using Gson

$
0
0

1. Introduction

Thе dеsеrialization procеss involvеs convеrting a JSON rеprеsеntation of an objеct (or data) into an еquivalеnt objеct in a programming languagе, such as a Java objеct. Gson, a popular Java library for JSON sеrialization and dеsеrialization, simplifiеs this process.

In this tutorial, we’ll еxplorе how to dеsеrializе JSON data into Java rеcords using Gson.

2. Creating a Java Record

Bеforе diving into thе codе еxamplеs, we need to еnsurе that we have thе Gson library added to our project. We can add it as a dеpеndеncy in our build tool, such as Mavеn or Gradlе. For Mavеn, we add thе following dеpеndеncy:

<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.9</version>
</dependency>

Lеt’s start by dеfining a simple Java Rеcord that wе’ll usе for dеsеrialization. For еxamplе, considеr a Pеrson rеcord with namе, agе, and addrеss fiеlds:

public rеcord Pеrson(String namе, int agе, String addrеss) {
    // No nееd to еxplicitly dеfinе constructors, gеttеrs, or othеr mеthods
}

3. Dеsеrializing JSON to Java Rеcord

Now, lеt’s sее how we can usе Gson to dеsеrializе JSON data into our Pеrson rеcord. Assumе wе havе thе following JSON rеprеsеntation of a pеrson:

{ "name": "John Doe", "age": 30, "address": "123 Main St" }

Let’s usе Gson’s fromJson() mеthod to convеrt this JSON string into a Pеrson rеcord:

@Test
public void givenJsonString_whenDeserialized_thenPersonRecordCreated() {
    String json = "{\"name\":\"John Doe\",\"age\":30,\"address\":\"123 Main St\"}";
    Person person = new Gson().fromJson(json, Person.class);
    assertEquals("John Doe", person.name());
    assertEquals(30, person.age());
    assertEquals("123 Main St", person.address());
}

In this еxamplе, thе fromJson() mеthod takеs thе JSON string and thе class typе (Pеrson.class) to which thе JSON should bе convеrtеd. Subsеquеntly, Gson automatically maps thе JSON fiеlds to thе corrеsponding rеcord componеnts.

4. Handling Nested Objects

What if we have a JSON that includes nеstеd objеcts? Gson can handlе thеm as wеll!

Lеt’s еxtеnd our Pеrson rеcord to includе a Contact rеcord for thе pеrson’s contact information:

public record Contact(String email, String phone) {
    // Constructor, getters, and other methods are automatically generated
}
public record Person(String name, int age, String address, Contact contact) {
    // Constructor, getters, and other methods are automatically generated
}

Now, let’s considеr a JSON rеprеsеntation that includеs contact information:

{ "namе": "John Doе", "agе": 30, "addrеss": "123 Main St", "contact": { "еmail": "john.doе@еxamplе.com", "phonе": "555-1234" } }

Thе dеsеrialization codе rеmains almost thе samе, with Gson handling thе nеstеd objеcts:

@Test
public void givenNestedJsonString_whenDeserialized_thenPersonRecordCreated() {
    String json = "{\"name\":\"John Doe\",\"age\":30,\"address\":\"123 Main St\",\"contact\":{\"email\":\"john.doe@example.com\",\"phone\":\"555-1234\"}}";
    Person person = new Gson().fromJson(json, Person.class);
    assertNotNull(person);
    assertEquals("John Doe", person.name());
    assertEquals(30, person.age());
    assertEquals("123 Main St", person.address());
    Contact contact = person.contact();
    assertNotNull(contact);
    assertEquals("john.doe@example.com", contact.email());
    assertEquals("555-1234", contact.phone());
}

5. Conclusion

In conclusion, thе combination of Gson and Java rеcords provides a concisе and еxprеssivе way to handlе JSON dеsеrialization, еvеn with nеstеd structurеs.

As always, the complete code samples for this article can be found over on GitHub.

       

String vs StringBuffer Comparison in Java

$
0
0

1. Overview

String and StringBuffer are two important classes used while working with strings in Java. In simple words, a string is a sequence of characters. For example, “java”, “spring” and so on.

The main difference between a String and a StringBuffer is that a String is immutable, whereas a StringBuffer is mutable and thread-safe.

In this tutorial, let’s compare String and StringBuffer classes and understand the similarities and differences between the two. 

2. String 

The String class represents character strings. Java implements all string literals, such as “baeldung”, as an instance of this class.

Let’s create a String literal:

String str = "baeldung";

Let’s also create a String object:

Char data[] = {‘b’, ‘a’, ‘e’, ‘l’, ‘d’, ‘u’, ‘n’, ‘g’};
String str = new String(data);

We can also do the following:

String str = new String(“baeldung”);

Strings are constants and immutable, making them shareable.

2.1. String Literal vs. String Object

String literals are immutable strings that are stored inside a special memory space called a string pool inside the heap memory. Java doesn’t allocate a new memory space for string literals having the same value. Instead, it uses the string interning.

In contrast, the JVM allocates separate memory in the heap, outside the string pool, for a newly created String object.

Thus, each string object refers to a different memory address, even though both may have the same value. Note that a String literal is still a String object. However, the reverse is not true.

2.2. String Pool

String literals are stored in a reserved memory area of the Java heap called the String Pool.

2.3. String Interning

String interning is an optimization technique the compiler uses to avoid redundant memory allocation. It avoids allocating memory for a new string literal if a similar value already exists. Instead, it works with the existing copy:

string memory allocation

Common operations on String include concatenation, comparison, and searching. The Java language also provides special support for the string concatenation operator (+) and for the conversion of other objects to strings. It is worth noting that String internally uses StringBuffer and its append method to perform concatenation:

String str = "String"; 
str = str.concat("Buffer");
assertThat(str).isEqualTo("StringBuffer");
assertThat(str.indexOf("Buffer")).isEqualTo(6);

3. StringBuffer

A StringBuffer is a sequence of characters just like a String. However, unlike a String, it’s mutable. We can modify a StringBuffer through method calls such as append() and insert(). The append method adds the character sequence at the end of the StringBuffer, while the insert method inserts a sequence of characters at a specified index. The StringBuffer class has both methods overloaded to handle any object. The object is converted to its string representation before appended or inserted into the StringBuffer:

StringBuffer sBuf = new StringBuffer("String");
sBuf.append("Buffer");
assertThat(sBuf).isEqualToIgnoringCase("StringBuffer");
sBuf.insert(0, "String vs ");
assertThat(sBuf).isEqualToIgnoringCase("String vs StringBuffer");

StringBuffer is thread-safe and can work in a multi-threaded environment. The synchronization ensures the correct order of execution of all statements and avoids data-races situations.

Java 1.5 introduced StringBuilder as a replacement for StringBuffer.

4. Performance Comparison

String and StringBuffer have similar performance. However, string manipulation is faster with StringBuffer than String because String requires the creation of a new object each time, and all changes happen to the new String, leading to more time and memory consumption.

Let’s do a quick micro-benchmark with JMH to compare the concatenation performance of String and StringBuffer:

@BenchmarkMode(Mode.SingleShotTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Measurement(batchSize = 100000, iterations = 10)
@Warmup(batchSize = 100000, iterations = 10)
@State(Scope.Thread)
public class ComparePerformance {
    String strInitial = "springframework";
    String strFinal = "";
    String replacement = "java-";
    @Benchmark
    public String benchmarkStringConcatenation() {
        strFinal = "";
        strFinal += strInitial;
        return strFinal;
    }
    @Benchmark
    public StringBuffer benchmarkStringBufferConcatenation() {
        StringBuffer stringBuffer = new StringBuffer(strFinal);
        stringBuffer.append(strInitial);
        return stringBuffer;
    }
}
Benchmark                                              Mode  Cnt   Score    Error  Units
ComparePerformance.benchmarkStringBufferConcatenation    ss   10  16.047 ± 11.757  ms/op
ComparePerformance.benchmarkStringConcatenation          ss   10   3.492 ±  1.309  ms/op

5. Comparison Table

To summarise the differences:

String StringBuffer
A String is a sequence of characters and is immutable A StringBuffer is like a String but can be modified, i.e., it’s mutable
It can be shared easily due to its immutability It can be shared across synchronized threads only
Modification requires the creation of a new string Modification requires a call to certain methods
Modification is slow Modification is faster
It uses the string pool for storing data It uses heap memory

6. Conclusion

In this article, we compared String and StringBuffer classes. As always, the example code is available over on GitHub.

       

Static Final Variables in Java

$
0
0

1. Overview

Simply put, static final variables, also called constants, are key features in Java to create a class variable that won’t change after initialization. However, in the case of a static final object reference, the state of the object may change.

In this tutorial, we’ll learn how to declare and initialize constant variables. Also, we’ll discuss their usefulness.

2. static final Variables

The static keyword associates a variable to a class itself, not to instances of the class.

Furthermore, the final keyword makes a variable immutable. Its value can’t change after initialization.

The combination of the two keywords helps create a constant. They are mostly named using uppercase and underscores to separate words.

2.1. Initializing static final Variables

Here’s an example of how to declare a static final field and assign a value:

class Bike {
    public static final int TIRE = 2;
}

Here, we create a class named Bike with a constant class variable named TIRE and initialize it to two.

Alternatively, we can initialize the variable via a static initializer block:

public static final int PEDAL;
static {
    PEDAL = 5;
}

This will compile without an error:

@Test
void givenPedalConstantSetByStaticBlock_whenGetPedal_thenReturnFive() {
    assertEquals(5, Bike.PEDAL);
}

Here are some key rules for constant variables:

  • We must initialize upon declaration or in a static initializer block
  • We can’t reassign it after initialization

Attempting to initialize it outside the initialization scope will cause an exception.

Also, we can’t initialize it via the constructor because constructors are invoked when we create an instance of a class. Static variables belong to the class itself and not to individual instances.

2.2. static final Objects

We can also create static final object references:

public static final HashMap<String, Integer> PART = new HashMap<>();

Since the PART reference is constant, it can’t be reassigned:

PART = new HashMap<>();

The code above throws an exception because we assign a new reference to an immutable variable.

However, we can modify the state of the object:

@Test
void givenPartConstantObject_whenObjectStateChanged_thenCorrect() {
    Bike.PART.put("seat", 1);
    assertEquals(1, Bike.PART.get("seat"));
    Bike.PART.put("seat", 5);
    assertEquals(5, Bike.PART.get("seat"));
}

Here, we can change the value of the seat despite setting it to one initially. We mutate the contents of PART despite being a constant reference. Only the reference itself is immutable.

Notably, the final keyword only makes primitive types, String, and other immutable types constant. In the case of an object, it only makes the reference constant, but the state of the object can be altered.

3. Why Constants Are Useful

Using static final variables has several advantages. It provides better performance since its values are inlined at compile time instead of a runtime value lookup.

Moreover, declaring reusable values as constant avoids duplicating literals. Constant can be reused anywhere in the code, depending on the access modifier. A constant with a private access modifier will only be usable within the class.

Additionally, a static final variable of primitive or String type is thread-safe. Its value remains unchanged when shared among multiple threads.

Finally, giving semantic names to constant values increases code readability. Also, it makes code self-documenting. For example, the java.math package provides constants like PI:

@Test
void givenMathClass_whenAccessingPiConstant_thenVerifyPiValueIsCorrect() {
    assertEquals(3.141592653589793, Math.PI);
}

The Math.PI encapsulates the mathematical constant value in a reusable way.

4. Conclusion

In this article, we learned how to declare and initialize a constant variable. Also, we highlighted some of its use cases.

A final static variable defines a class-level constant. However, a static final object may still be mutable, even if the reference can’t change.

As always, the complete source code for the examples is available over on GitHub.

       

Differences Between Entities and DTOs

$
0
0

1. Overview

In the realm of software development, there is a clear distinction between entities and DTOs (Data Transfer Objects). Understanding their precise roles and differences can help us build more efficient and maintainable software.

In this article, we’ll explore the differences between entities and DTOs and try to offer a clear understanding of their purpose, and when to employ them in our software projects. While going through each concept, we’ll sketch a trivial application of user management, using Spring Boot and JPA.

2. Entities

Entities are fundamental components that represent real-world objects or concepts within the domain of our application. They often correspond directly to database tables or domain objects. Therefore, their primary purpose is to encapsulate and manage the state and behavior of these objects.

2.1. Entity Example

Let’s create some entities for our project, representing a user that has multiple books. We’ll start by creating the Book entity:

@Entity
@Table(name = "books")
public class Book {
    @Id
    private String name;
    private String author;
    // standard constructors / getters / setters
}

Now, we need to define our User entity:

@Entity
@Table(name = "users")
public class User {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String firstName;
    private String lastName;
    private String address;
    @OneToMany(cascade=CascadeType.ALL)
    private List<Book> books;
    
    public String getNameOfMostOwnedBook() {
        Map<String, Long> bookOwnershipCount = books.stream()
          .collect(Collectors.groupingBy(Book::getName, Collectors.counting()));
        return bookOwnershipCount.entrySet().stream()
          .max(Map.Entry.comparingByValue())
          .map(Map.Entry::getKey)
          .orElse(null);
    }
    // standard constructors / getters / setters
}

2.2. Entity Characteristics

In our entities, we can identify some distinctive characteristics. In the first place, entities commonly incorporate Object-Relational Mapping (ORM) annotations. For instance, the @Entity annotation marks the class as an entity, creating a direct link between a Java class and a database table.

The @Table annotation is used to specify the name of the database table associated with the entity. Additionally, the @Id annotation defines a field as the primary key. These ORM annotations simplify the process of database mapping.

Moreover, entities often need to establish relationships with other entities, reflecting associations between real-world concepts. A common example is the @OneToMany annotation we’ve used to define a one-to-many relationship between a user and the books he owns.

Furthermore, entities don’t have to serve solely as passive data objects but can also contain domain-specific business logic. For instance, let’s consider a method such as getNameOfMostOwnedBook(). This method, residing within the entity, encapsulates domain-specific logic to find the name of the book the user owns the most. This approach aligns with OOP principles and the DDD approach by keeping domain-specific operations within entities, fostering code organization and encapsulation.

Additionally, entities may incorporate other particularities, such as validation constraints or lifecycle methods.

3. DTOs

DTOs primarily act as pure data carriers, without having any business logic. They’re used to transmit data between different applications or parts of the same application.

In simple applications, it’s common to use the domain objects directly as DTOs. However, as applications grow in complexity, exposing the entire domain model to external clients may become less desirable from a security and encapsulation perspective.

3.1. DTO Example

To keep our application as simple as possible, we will implement only the functionalities of creating a new user and retrieving the current users. To do so, let’s start by creating a DTO to represent a book:

public class BookDto {
    @JsonProperty("NAME")
    private final String name;
    @JsonProperty("AUTHOR")
    private final String author;
    // standard constructors / getters
}

For the user, let’s define two DTOs. One is designed for the creation of a user, while the second one is tailored for response purposes:

public class UserCreationDto {
    @JsonProperty("FIRST_NAME")
    private final String firstName;
    @JsonProperty("LAST_NAME")
    private final String lastName;
    @JsonProperty("ADDRESS")
    private final String address;
    @JsonProperty("BOOKS")
    private final List<BookDto> books;
    // standard constructors / getters
}
public class UserResponseDto {
    @JsonProperty("ID")
    private final Long id;
    @JsonProperty("FIRST_NAME")
    private final String firstName;
    @JsonProperty("LAST_NAME")
    private final String lastName;
    @JsonProperty("BOOKS")
    private final List<BookDto> books;
    // standard constructors / getters
}

3.2. DTO Characteristics

Based on our examples, we can identify a few particularities: immutability, validation annotations, and JSON mapping annotations.

Making DTOs immutable is a best practice. Immutability ensures that the data being transported is not accidentally altered during its journey. One way to achieve this is by declaring all properties as final and not implementing setters. Alternatively, the @Value annotation from Lombok or Java records, introduced in Java 14, offers a concise way to create immutable DTOs.

Moving on, DTOs can also benefit from validation, to ensure that the data transferred via the DTOs meets specific criteria. This way, we can detect and reject invalid data early in the data transfer process, preventing the pollution of the domain with unreliable information.

Moreover, we may usually find JSON mapping annotations in DTOs, to map JSON properties to the fields of our DTOs. For example, the @JsonProperty annotation allows us to specify the JSON names of our DTOs.

4. Repository, Mapper, and Controller

To demonstrate the utility of having both entities and DTOs represent data within our application, we need to complete our code. We’ll start by creating a repository for our User entity:

@Repository
public interface UserRepository extends JpaRepository<User, Long> {
}

Next, we’ll proceed with creating a mapper to be able to convert from one to another:

public class UserMapper {
    public static UserResponseDto toDto(User entity) {
        return new UserResponseDto(
          entity.getId(),
          entity.getFirstName(),
          entity.getLastName(),
          entity.getBooks().stream().map(UserMapper::toDto).collect(Collectors.toList())
        );
    }
    public static User toEntity(UserCreationDto dto) {
        return new User(
          dto.getFirstName(),
          dto.getLastName(),
          dto.getAddress(),
          dto.getBooks().stream().map(UserMapper::toEntity).collect(Collectors.toList())
        );
    }
    public static BookDto toDto(Book entity) {
        return new BookDto(entity.getName(), entity.getAuthor());
    }
    public static Book toEntity(BookDto dto) {
        return new Book(dto.getName(), dto.getAuthor());
    }
}

In our example, we’ve done the mapping manually between entities and DTOs. For more complex models, to avoid boilerplate code, we could’ve used tools like MapStruct.

Now, we only need to create the controller:

@RestController
@RequestMapping("/users")
public class UserController {
    private final UserRepository userRepository;
    public UserController(UserRepository userRepository) {
        this.userRepository = userRepository;
    }
    @GetMapping
    public List<UserResponseDto> getUsers() {
        return userRepository.findAll().stream().map(UserMapper::toDto).collect(Collectors.toList());
    }
    @PostMapping
    public UserResponseDto createUser(@RequestBody UserCreationDto userCreationDto) {
        return UserMapper.toDto(userRepository.save(UserMapper.toEntity(userCreationDto)));
    }
}

5. Why Do We Need Both Entities and DTOs?

5.1. Separation of Concerns

In our example, the entities are closely tied to the database schema and domain-specific operations. On the other hand, DTOs are designed only for data transfer purposes.

In some architectural paradigms, such as hexagonal architecture, we may find an additional layer, commonly referred to as the Model or Domain Model. This layer serves the crucial purpose of totally decoupling the domain from any intrusive technology. This way, the core business logic remains independent of the implementation details of databases, frameworks, or external systems.

5.2. Hiding Sensitive Data

When dealing with external clients or systems, controlling what data is exposed to the outside world is essential. Entities may contain sensitive information or business logic that should remain hidden from external consumers. DTOs act as a barrier that helps us expose only safe and relevant data to the clients.

5.3. Performance

The DTO pattern, as introduced by Martin Fowler, involves batching up multiple parameters in a single call. Instead of making multiple calls to fetch individual pieces of data, we can bundle related data into a DTO and transmit it in a single request. This approach reduces the overhead associated with multiple network calls.

One way of implementing the DTO pattern is through GraphQL, which allows the client to specify the data it desires, allowing multiple queries in a single request.

6. Conclusion

As we’ve learned throughout this article, entities and DTOs have different roles and can be very distinct. The combination of both entities and DTOs ensures data security, separation of concerns, and efficient data management in complex software systems. This approach leads to more robust and maintainable software solutions.

As always, the source code is available over on GitHub.

       

Handling NullPointerException in findFirst() When the First Element Is Null

$
0
0

1. Overview

In this short tutorial, we’ll explore different ways of avoiding NullPointerException when working with the findFirst() method.

First, we’ll explain what causes the method to fail with NullPointerException. Then, we’ll demonstrate how to reproduce and fix the exception using practical examples.

2. Explaining the Problem

In short, a NullPointerException is thrown to signal that we’re doing some operation using null where an object is required.

Typically, we use findFirst() to return an Optional instance holding the first element of a given stream. However, according to the documentation, the method throws NullPointerException if the first returned element is null.

So, the main question here is how to avoid the NullPointerException exception when the first element of our stream is null. Before diving deep and answering our question, let’s reproduce the exception.

3. Reproducing the NullPointerException

For instance, let’s assume we have a list of String objects:

List<String> inputs = Arrays.asList(null, "foo", "bar");

Now, let’s try to get the first element of our list using the findFirst() method:

@Test(expected = NullPointerException.class)
public void givenStream_whenCallingFindFirst_thenThrowNullPointerException() {
    Optional<String> firstElement = inputs.stream()
      .findFirst();
}

As we can see, the test case fails with NullPointerException because the first selected element of our list is null.

The Optional API states that it’s the caller’s responsibility to ensure that the value is not null because it doesn’t provide any way to distinguish between “the value is present but set to null” and “the value is not present”. This is why the documentation prohibits the scenario where null is returned when using findFirst().

4. Avoiding the Exception

The easiest way to avoid NullPointerException in this case is to filter the stream before calling the findFirst() method.

So, let’s see how we can do this in practice:

@Test
public void givenStream_whenUsingFilterBeforeFindFirst_thenCorrect() {
    Optional<String> firstNotNullElement = inputs.stream()
      .filter(Objects::nonNull)
      .findFirst();
    assertTrue(firstNotNullElement.isPresent());
}

Here, we used the Objects#nonNull method to filter only objects that are not null. That way, we ensure the selected first element is not null. As a result, we avoid NullPointerException.

Another option would be to use the Optional#ofNullable method before calling the findFirst() method.

This method returns an Optional instance with the specified value if it’s not null. Otherwise, it returns an empty Optional.

So, let’s see it in action:

@Test
public void givenStream_whenUsingOfNullableBeforeFindFirst_thenCorrect() {
    Optional<String> firstElement = inputs.stream()
      .map(Optional::ofNullable)
      .findFirst()
      .flatMap(Function.identity());
    assertTrue(firstElement.isEmpty());
}

As shown above, we map each element into an Optional object that accepts null with the help of the ofNullable() method. Then, we get the first mapped element using findFirst().

The returned element denotes an Optional of an Optional since findFirst() returns an Optional. This is why we used flapMap() to flatten the nested Optional.

Please note that Function#identity always returns its input argument. In our case, the method returns null because it’s the first element of our list.

5. Conclusion

In this short article, we explained how to avoid NullPointerException when working with the findFirst() method.

Along the way, we showcased how to reproduce and solve the exception using practical examples.

As always, the full source code of the examples is available over on GitHub.

       

Skip Bytes in InputStream in Java

$
0
0

1. Introduction

In Java programming, InputStrеam is a fundamеntal class for rеading bytеs from a sourcе. Howеvеr, thеrе arе scеnarios whеrе it bеcomеs nеcеssary to skip a cеrtain numbеr of bytеs within an InputStrеam.

In this tutorial, wе’ll dеlvе into thе skip() mеthod, еxploring how it can bе еffеctivеly еmployеd to skip bytеs within a Java InputStrеam.

2. An Ovеrviеw

InputStrеam is an abstract class that sеrvеs as thе supеrclass for all classеs rеprеsеnting an input strеam of bytеs. Moreover, it providеs mеthods for rеading bytеs from a strеam, making it a fundamеntal componеnt for input opеrations.

In the same context, there arе various situations whеrе skipping bytеs bеcomеs nеcеssary. Onе common scеnario is whеn dеaling with filе hеadеrs or mеtadata that arе not rеquirеd for a specific opеration. Hence, skipping unnеcеssary bytеs can improvе pеrformancе and rеducе thе amount of data that nееds to bе procеssеd.

3. Skipping Bytеs Using the skip() Mеthod

Thе InputStrеam class in Java providеs a built-in mеthod callеd skip(long n) for skipping a spеcifiеd numbеr of bytеs. Thе n paramеtеr dеnotеs thе numbеr of bytеs to bе skippеd.

Let’s take the following example:

@Test
void givenInputStreamWithBytes_whenSkipBytes_thenRemainingBytes() throws IOException {
    byte[] inputData = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
    InputStream inputStream = new ByteArrayInputStream(inputData);
    long bytesToSkip = 3;
    long skippedBytes = inputStream.skip(bytesToSkip);
    assertArrayEquals(new byte[]{4, 5, 6, 7, 8, 9, 10}, readRemainingBytes(inputStream));
    assert skippedBytes == bytesToSkip : "Incorrect number of bytes skipped";
}

Thе tеst bеgins by sеtting up an array of bytеs, ranging from 1 to 10, and crеating an InputStrеam using a BytеArrayInputStrеam initializеd with thе bytе array. Subsеquеntly, thе codе spеcifiеs thе numbеr of bytеs to skip (in this casе, 3) and invokеs thе skip mеthod on thе InputStrеam.

Thе tеst thеn еmploys assеrtions to validatе that thе rеmaining bytеs in thе input strеam match thе еxpеctеd array {4, 5, 6, 7, 8, 9, 10} using thе rеadRеmainingBytеs() mеthod:

byte[] readRemainingBytes(InputStream inputStream) throws IOException {
    byte[] buffer = new byte[inputStream.available()];
    int bytesRead = inputStream.read(buffer);
    if (bytesRead == -1) {
        throw new IOException("End of stream reached");
    }
    return buffer;
}

This mеthod rеads thе rеmaining bytеs into a buffеr and еnsurеs that thе еnd of thе strеam hasn’t bееn rеachеd.

4. Conclusion

In conclusion, еfficiеnt bytе strеam managеmеnt is crucial in Java, and thе InputStrеam class, particularly thе skip() mеthod, providеs a valuablе tool for skipping bytеs whеn handling input opеrations, еnhancing pеrformancе and rеducing unnеcеssary data procеssing.

As always, the complete code samples for this article can be found over on GitHub.

       

Java Weekly, Issue 520

$
0
0

1. Spring and Java

>> JEP targeted to JDK 22: 464: Scoped Values (Second Preview) [openjdk.org]

Another preview of Scoped Values: a safe and efficient way of sharing immutable data within and across threads

>> AutoCloseable HttpClient – Sip of Java [inside.java]

Java 21 enables to closing/shutdown of HttpClient instances in a variety of ways, including try-with-resources blocks. Good stuff.

>> Helidon 4 Adopts Virtual Threads: Explore the Increased Performance and Improved DevEx [infoq.com]

And, improving performance and simplifying concurrent programming in Helidon 4 by supporting Project Loom

Also worth reading:

Webinars and presentations:

Time to upgrade:

2. Technical & Musings

>> Apache Pinot 1.0 Provides a Realtime Distributed OLAP datastore [infoq.com]

An OLAP column-oriented distributed data store, allowing us to perform complex queries with low latency

Also worth reading:

3. Pick of the Week

Take the Developer Nation survey, share your opinion about tools and technologies, learn about the dev landscape and yes, win over 200 really cool prizes:

>> Start the Survey Here [developereconomics.net]

       

A Guide to Timefold Solver for Employee Scheduling

$
0
0

1. Overview

1.1. What Is Timefold Solver?

Timefold Solver is a pure Java planning solver AI. Timefold optimizes planning problems, such as the vehicle routing problem (VRP), maintenance scheduling, job shop scheduling, and school timetabling. It generates logistics plans that heavily reduce costs, improve service quality, and decrease the environmental footprint – often by as much as 25% – for complex, real-world scheduling operations.

Timefold is the continuation of OptaPlanner. It’s a form of mathematical optimization (in the broader Operations Research and Artificial Intelligence spaces) that supports constraints written as code.

1.2. What We Will Build

In this tutorial, let’s use Timefold Solver to optimize a simplified employee shift scheduling problem.

We’ll assign shifts to employees automatically, such that:

  • No employee has two shifts on the same day
  • Every shift is assigned to an employee who has the appropriate skill

Specifically, we will assign these five shifts:

  2030-04-01 06:00 - 14:00 (waiter)
  2030-04-01 09:00 - 17:00 (bartender)
  2030-04-01 14:00 - 22:00 (bartender)
  2030-04-02 06:00 - 14:00 (waiter)
  2030-04-02 14:00 - 22:00 (bartender)

To these three employees:

  Ann (bartender)
  Beth (waiter, bartender)
  Carl (waiter)

This is harder than it looks. Give it a try on paper.

2. Dependencies

The Timefold Solver artifacts on Maven Central are released under the Apache License. Let’s use them:

2.1 Plain Java

We add a dependency on timefold-solver-core and a test dependency on timefold-solver-test in Maven or Gradle:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>ai.timefold.solver</groupId>
            <artifactId>timefold-solver-bom</artifactId>
            <version>...</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>
<dependencies>
    <dependency>
        <groupId>ai.timefold.solver</groupId>
        <artifactId>timefold-solver-core</artifactId>
    </dependency>
    <dependency>
        <groupId>ai.timefold.solver</groupId>
        <artifactId>timefold-solver-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

2.2. Spring Boot

In Spring Boot, we use the timefold-solver-spring-boot-starter dependency instead. It handles most of the solver configuration automatically, as we’ll see later, and allows configuring solver time and other properties in application.properties.

  1. Go to start.spring.io
  2. Click Add dependencies to add the Timefold Solver dependency
  3. Generate a project and open it in your favorite IDE

2.3. Quarkus

In Quarkus, similarly, we use the timefold-solver-quarkus dependency in code.quarkus.io for automatic solver configuration and application.properties support.

3. Domain Classes

The domain classes represent both the input data and output data. We create Employee and Shift classes, as well as a ShiftSchedule that contains the list of employees and shifts for a particular dataset.

3.1. Employee

An employee is a person we can assign to shifts. Each employee has a name and one or more skills.

The Employee class doesn’t need any Timefold annotation because it does not change during solving:

public class Employee {
    private String name;
    private Set<String> skills;
    public Employee(String name, Set<String> skills) {
        this.name = name;
        this.skills = skills;
    }
    @Override
    public String toString() {
        return name;
    }
    // Getters and setters
}

3.2. Shift

A shift is a job assignment for exactly one employee on a specific date from a start time to an end time. There can be two shifts at the same time. Each shift has one required skill.

Shift objects change during solving: Each shift is assigned to an employee. Timefold needs to know that. Only the employee field changes during solving. Therefore, we annotate the class with @PlanningEntity and the employee field with @PlanningVariable so Timefold knows it should fill in the employee for each shift:

@PlanningEntity
public class Shift {
    private LocalDateTime start;
    private LocalDateTime end;
    private String requiredSkill;
    @PlanningVariable
    private Employee employee;
    // A no-arg constructor is required for @PlanningEntity annotated classes
    public Shift() {
    }
    public Shift(LocalDateTime start, LocalDateTime end, String requiredSkill) {
        this(start, end, requiredSkill, null);
    }
    public Shift(LocalDateTime start, LocalDateTime end, String requiredSkill, Employee employee) {
        this.start = start;
        this.end = end;
        this.requiredSkill = requiredSkill;
        this.employee = employee;
    }
    @Override
    public String toString() {
        return start + " - " + end;
    }
    // Getters and setters
}

3.3. ShiftSchedule

A schedule represents a single dataset of employees and shifts. It is both the input and output for Timefold:

  • We annotate the ShiftSchedule class with @PlanningSolution so Timefold knows it represents the input and output.
  • We annotate the employees field with @ValueRangeProvider to tell Timefold it contains the list of employees from which it can pick instances to assign to Shift.employee.
  • We annotate the shifts field with @PlanningEntityCollectionProperty so Timefold finds all Shift instances to assign to an employee.
  • We include a score field with a @PlanningScore annotation. Timefold will fill this in for us. Let’s use a HardSoftScore so we can differentiate between hard and soft constraints later.

Now, let’s have a look at our class:

@PlanningSolution
public class ShiftSchedule {
    @ValueRangeProvider
    private List<Employee> employees;
    @PlanningEntityCollectionProperty
    private List<Shift> shifts;
    @PlanningScore
    private HardSoftScore score;
    // A no-arg constructor is required for @PlanningSolution annotated classes
    public ShiftSchedule() {
    }
    public ShiftSchedule(List<Employee> employees, List<Shift> shifts) {
        this.employees = employees;
        this.shifts = shifts;
    }
    // Getters and setters
}

4. Constraints

Without constraints, Timefold would assign all shifts to the first employee. That’s not a feasible schedule.

To teach it how to distinguish good and bad schedules, let’s add two hard constraints:

  • The atMostOneShiftPerDay() constraint checks if two shifts on the same date are assigned to the same employee. If that’s the case, it penalizes the score by 1 hard point.
  • The requiredSkill() constraint checks if a shift is assigned to an employee for which the shift’s required skill is part of the employee’s skill set. If it’s not, it penalizes the score by 1 hard point.

A single hard constraint takes priority over all soft constraints. Typically, hard constraints are impossible to break, either physically or legally. Soft constraints, on the other hand, can be broken, but we want to minimize that. Those typically represent financial costs, service quality, or employee happiness. Hard and soft constraints are implemented with the same API.

4.1. ConstraintProvider

First, we create a ConstraintProvider for our constraint implementations:

public class ShiftScheduleConstraintProvider implements ConstraintProvider {
    @Override
    public Constraint[] defineConstraints(ConstraintFactory constraintFactory) {
        return new Constraint[] {
          atMostOneShiftPerDay(constraintFactory),
          requiredSkill(constraintFactory)
        };
    }
    // Constraint implementations
}

4.2. Unit Test the ConstraintProvider

If it isn’t tested, it doesn’t work — especially for constraints. Let’s create a test class to test each constraint of our ConstraintProvider.

The test-scoped timefold-solver-test dependency contains ConstraintVerifier, a helper to test each constraint in isolation. This improves maintenance — we can refactor a single constraint without breaking tests of other constraints:

public class ShiftScheduleConstraintProviderTest {
    private static final LocalDate MONDAY = LocalDate.of(2030, 4, 1);
    private static final LocalDate TUESDAY = LocalDate.of(2030, 4, 2);
    ConstraintVerifier<ShiftScheduleConstraintProvider, ShiftSchedule> constraintVerifier
      = ConstraintVerifier.build(new ShiftScheduleConstraintProvider(), ShiftSchedule.class, Shift.class);
    // Tests for each constraint
}

We’ve also prepared two dates to reuse in our tests below. Let’s add the actual constraints next.

4.3. Hard Constraint: At Most One Shift Per Day

Following TDD (Test Driven Design), let’s write the tests for our new constraint in our test class first:

@Test
void whenTwoShiftsOnOneDay_thenPenalize() {
    Employee ann = new Employee("Ann", null);
    constraintVerifier.verifyThat(ShiftScheduleConstraintProvider::atMostOneShiftPerDay)
      .given(
        new Shift(MONDAY.atTime(6, 0), MONDAY.atTime(14, 0), null, ann),
        new Shift(MONDAY.atTime(14, 0), MONDAY.atTime(22, 0), null, ann))
      // Penalizes by 2 because both {shiftA, shiftB} and {shiftB, shiftA} match.
      // To avoid that, use forEachUniquePair() in the constraint instead of forEach().join() in the implementation.
      .penalizesBy(2);
}
@Test
void whenTwoShiftsOnDifferentDays_thenDoNotPenalize() {
    Employee ann = new Employee("Ann", null);
    constraintVerifier.verifyThat(ShiftScheduleConstraintProvider::atMostOneShiftPerDay)
      .given(
        new Shift(MONDAY.atTime(6, 0), MONDAY.atTime(14, 0), null, ann),
        new Shift(TUESDAY.atTime(14, 0), TUESDAY.atTime(22, 0), null, ann))
      .penalizesBy(0);
}

Then, we implement it in our ConstraintProvider:

public Constraint atMostOneShiftPerDay(ConstraintFactory constraintFactory) {
    return constraintFactory.forEach(Shift.class)
      .join(Shift.class,
        equal(shift -> shift.getStart().toLocalDate()),
        equal(Shift::getEmployee))
      .filter((shift1, shift2) -> shift1 != shift2)
      .penalize(HardSoftScore.ONE_HARD)
      .asConstraint("At most one shift per day");
}

To implement constraints, we use the ConstraintStreams API: a Stream/SQL-like API that provides incremental score calculation (deltas) and indexed hashtable lookups under the hood. This approach scales to datasets with hundreds of thousands of shifts in a single schedule.

Let’s run the tests and verify they are green.

4.4. Hard Constraint: Required Skill

Let’s write the tests in our test class:

@Test
void whenEmployeeLacksRequiredSkill_thenPenalize() {
    Employee ann = new Employee("Ann", Set.of("Waiter"));
    constraintVerifier.verifyThat(ShiftScheduleConstraintProvider::requiredSkill)
      .given(
        new Shift(MONDAY.atTime(6, 0), MONDAY.atTime(14, 0), "Cook", ann))
      .penalizesBy(1);
}
@Test
void whenEmployeeHasRequiredSkill_thenDoNotPenalize() {
    Employee ann = new Employee("Ann", Set.of("Waiter"));
    constraintVerifier.verifyThat(ShiftScheduleConstraintProvider::requiredSkill)
      .given(
        new Shift(MONDAY.atTime(6, 0), MONDAY.atTime(14, 0), "Waiter", ann))
      .penalizesBy(0);
}

Then, let’s implement the new constraint in our ConstraintProvider:

public Constraint requiredSkill(ConstraintFactory constraintFactory) {
    return constraintFactory.forEach(Shift.class)
      .filter(shift -> !shift.getEmployee().getSkills()
        .contains(shift.getRequiredSkill()))
      .penalize(HardSoftScore.ONE_HARD)
      .asConstraint("Required skill");
}

Let’s run the tests again. They are still green.

To make this a soft constraint, we would change penalize(HardSoftScore.ONE_HARD) into penalize(HardSoftScore.ONE_SOFT). To turn that into a dynamic decision by the input dataset, we could use penalizeConfigurable() and @ConstraintWeight instead.

5. Application

We’re ready to put our application together.

5.1. Solve It

To solve a schedule, we create a SolverFactory from our @PlanningSolution, @PlanningEntity, and ConstraintProvider classes. A SolverFactory is a long-lived object. Typically, there’s only one instance per application.

We also need to configure how long we want a solver to run. For large datasets, with thousands of shifts and far more constraints, it’s impossible to find the optimal solution in a reasonable timeframe (due to the exponential nature of NP-hard problems). Instead, we want to find the best possible solution in the amount of time available. Let’s limit that to two seconds for now:

SolverFactory<ShiftSchedule> solverFactory = SolverFactory.create(new SolverConfig()
  .withSolutionClass(ShiftSchedule.class)
  .withEntityClasses(Shift.class)
  .withConstraintProviderClass(ShiftScheduleConstraintProvider.class)
  // The solver runs only for 2 seconds on this tiny dataset.
  // It's recommended to run for at least 5 minutes ("5m") on large datasets.
  .withTerminationSpentLimit(Duration.ofSeconds(2)));

We use the SolverFactory to create a Solver instance, one per dataset. Then, we call Solver.solve() to solve a dataset:

Solver<ShiftSchedule> solver = solverFactory.buildSolver();
ShiftSchedule problem = loadProblem();
ShiftSchedule solution = solver.solve(problem);
printSolution(solution);

In Spring Boot, the SolverFactory is built automatically and injected into an @Autowired field:

@Autowired
SolverFactory<ShiftSchedule> solverFactory;

And we configure the solver time in application.properties:

timefold.solver.termination.spent-limit=5s

In Quarkus, similarly, the SolverFactory is also built automatically and injected in an @Inject field. The solver time is also configured in application.properties.

To solve asynchronously, to avoid hogging the current thread when calling Solver.solve(), we would inject and use a SolverManager instead.

5.2. Test Data

Let’s generate a tiny dataset of five shifts and three employees as the input problem:

private ShiftSchedule loadProblem() {
    LocalDate monday = LocalDate.of(2030, 4, 1);
    LocalDate tuesday = LocalDate.of(2030, 4, 2);
    return new ShiftSchedule(List.of(
      new Employee("Ann", Set.of("Bartender")),
      new Employee("Beth", Set.of("Waiter", "Bartender")),
      new Employee("Carl", Set.of("Waiter"))
    ), List.of(
      new Shift(monday.atTime(6, 0), monday.atTime(14, 0), "Waiter"),
      new Shift(monday.atTime(9, 0), monday.atTime(17, 0), "Bartender"),
      new Shift(monday.atTime(14, 0), monday.atTime(22, 0), "Bartender"),
      new Shift(tuesday.atTime(6, 0), tuesday.atTime(14, 0), "Waiter"),
      new Shift(tuesday.atTime(14, 0), tuesday.atTime(22, 0), "Bartender")
    ));
}

5.3. Result

After we run the test data through our solver, we’ll print the output solution to System.out:

private void printSolution(ShiftSchedule solution) {
    logger.info("Shift assignments");
    for (Shift shift : solution.getShifts()) {
        logger.info("  " + shift.getStart().toLocalDate()
          + " " + shift.getStart().toLocalTime()
          + " - " + shift.getEnd().toLocalTime()
          + ": " + shift.getEmployee().getName());
    }
}

Here’s the result for our dataset:

Shift assignments
  2030-04-01 06:00 - 14:00: Carl
  2030-04-01 09:00 - 17:00: Ann
  2030-04-01 14:00 - 22:00: Beth
  2030-04-02 06:00 - 14:00: Beth
  2030-04-02 14:00 - 22:00: Ann

Ann wasn’t assigned to the first shift because she didn’t have the waiter skill. But why wasn’t Beth assigned to the first shift? She has the waiter skill.

If Beth had been assigned to the first shift, it would then be impossible to assign both the second and third shifts. Those both need a bartender, so Carl can’t do them. Only when Carl is assigned to the first shift is a feasible solution possible. In large, real-world datasets, these kinds of intricacies become a lot more complex. Let the Solver worry about them.

6. Conclusion

The Timefold Solver framework provides developers with a powerful tool to solve constraint satisfaction problems such as scheduling and resource allocation. It supports writing custom constraints in code (instead of mathematical equations), which makes it maintenance-friendly. Under the hood, it supports various Artificial Intelligence optimization algorithms that can be power-tweaked, but a typical user doesn’t need to do so.

For more information, see the Timefold Solver documentation. As always, the source code for this tutorial is over on GitHub.

       

Recursively Sum the Integers in an Array

$
0
0

1. Overview

When we work with numbers, summing all integers in an array is a common operation. Also, recursion often lends itself to elegant solutions.

In this tutorial, we’ll explore how to sum integers in an array using recursion.

2. Recursion With Array Copying

First, let’s initialize an array of integers:

private static final int[] INT_ARRAY = { 1, 2, 3, 4, 5 };

Obviously, the sum of the integers in the array above is 15.

A usual approach to sum numbers in an array is sum (array[0-n]) = array[0] + array[1] + array[2] + array[3] + … + array[n].

This method is straightforward. Alternatively, we can look at this problem from a different perspective: the sum of numbers in an array equals the first number plus the sum of a subarray consists of the rest numbers:

sumOf(array[0..n]) = array[0] + sumOf(subArray[1..n]).

Now, if we look at sumOf() as a function or method, we can see the sumOf()’s body calls sumOf() again. Therefore, sumOf() is a recursive method.

As Java doesn’t allow us to change the array’s length after the creation, removing an element from an array is technically impossible. But Java has offered various ways to copy an array. We can use these methods to create the subarray.

When we implement recursive methods, defining the base case is crucial. A base case is some point to exit the recursion. Otherwise, without a base case, the method endlessly calls itself recursively until StackOverflowError is thrown.

In our case, the base case is when the subarray has only one element. This is because the subarray is empty after taking out the only number.

So next, let’s implement the recursive method:

int sumIntArray1(int[] array) {
    if (array.length == 1) {
        return array[0];
    } else {
        return array[0] + sumIntArray1(Arrays.copyOfRange(array, 1, array.length));
    }
}

As we can see in the sumIntArray1() method, we use the Arrays.copyOfRange() method to create the subarray.

If we pass our example input to the method, the recursion steps look like the following:

sumIntarray1(array) = array[0] + sumOfArray1(arr1{2, 3, 4, 5})
                    = 1 + (arr1[0] + sumIntarray1(arr2{3, 4, 5}))
                    = 1 + (2 + (arr2[0] + sumIntarray1(arr3{4, 5})))
                    = 1 + (2 + (3 + (arr3[0] + sumIntarray1(arr4{5})))) <-- (arr4.length == 1) Base case reached
                    = 1 + (2 + (3 + (4 + (5))))
                    = 15

Next, let’s test the method with INT_ARRAY:

assertEquals(15, sumIntArray1(INT_ARRAY));

3. Recursion Without Creating Array Copies

In the sumIntArray1() method, we used the Arrays.copyOfRange() method to initialize the subarray. However, a new array will be created every time we call this method. If we face an enormous integer array, this approach creates many array objects.

We know we should avoid creating unnecessary objects to gain better performance. So, next, let’s see if we can improve the sumIntArray1() method.

The idea is to pass the required index to the next recursion step. Then, we can reuse the same array object:

int sumIntArray2(int[] array, int lastIdx) {
    if (lastIdx == 0) {
        return array[lastIdx];
    } else {
        return array[lastIdx] + sumIntArray2(array, lastIdx - 1);
    }
}

If we test it with our INT_ARRAY input, the test passes:

assertEquals(15, sumIntArray2(INT_ARRAY, INT_ARRAY.length - 1))

Next, let’s understand how the sumIntArray2() method works.

The method takes two parameters: the integer array (array) and the last index up to which we intend to calculate the sum (lastIdx). This time, the recursion follows this rule:

sumOf(array[0..n], n) = array[n] + sumOf(array[0..n], n-1).

As we can see, we reuse the original array in each recursion step. The base case of this approach is when lastIdx is zero, which means we’ve reversely (from n ->0) walked through the entire array:

sumIntArray2(array, 4) = array[4] + sumOfArray2(array, 3)
                       = 5 + (array[3] + sumIntArray2(array, 2))
                       = 5 + (4 + (array[2] + sumIntArray2(array, 1)))
                       = 5 + (4 + (3 + (array[1] + sumIntArray2(array, 0))))
                       = 5 + (4 + (3 + (2 + (array[0])))) <-- (idx == 0) Base case reached
                       = 5 + (4 + (3 + (2 + (1))))
                       = 15

Finally, let’s apply a performance comparison to see, given the same input, whether sumIntArray2() is faster than sumIntArray1().

4. Benchmarking the Two Recursive Solutions

We’ll use JMH (Java Microbenchmark Harness) to benchmark the two recursive solutions. So, let’s first create a benchmark class:

@BenchmarkMode(Mode.AverageTime)
@State(Scope.Thread)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 2)
@Fork(1)
@Measurement(iterations = 5)
public class SumArrayBenchmark {
    public static void main(String[] args) throws Exception {
        Options options = new OptionsBuilder()
          .include(SumArrayBenchmark.class.getSimpleName())
          .build();
        new Runner(options).run();
    }
    @Param({ "10", "10000" })
    public int size;
    int[] array;
    @Setup
    public void setup() {
        var r = new Random();
        array = new int[size];
        for (int i = 0; i < size; i++) {
            array[i] = r.nextInt();
        }
    }
    @Benchmark
    public int withArrayCopy() {
        return sumIntArray1(array);
    }
    @Benchmark
    public int withoutArrayCopy() {
        return sumIntArray2(array, array.length - 1);
    }
}

Our objective is to benchmark the two solutions. So, we won’t discuss each JMH configuration or annotation for brevity. However, it’s crucial to understand that SumArrayBenchmark performs each solution with two distinct input arrays:

  • An array with 10 random numbers
  • An array consists of 10000 random integers

Additionally, JMH conducts five iterations for each input array on each solution, ensuring a thorough evaluation of their performance.

Next, let’s look at the output SumArrayBenchmark produced:

Benchmark                           (size)  Mode  Cnt        Score       Error  Units
SumArrayBenchmark.withArrayCopy         10  avgt    5       30,576 ±     0,584  ns/op
SumArrayBenchmark.withArrayCopy      10000  avgt    5  7314150,000 ± 82516,421  ns/op
SumArrayBenchmark.withoutArrayCopy      10  avgt    5        6,764 ±     0,032  ns/op
SumArrayBenchmark.withoutArrayCopy   10000  avgt    5    30140,685 ±    91,804  ns/op

As the report shows, the withoutArrayCopy() solution is much faster than the withArrayCopy() approach:

  • Array[10] ~ 5 times faster (30576/6764)
  • Array[10000] ~ 242 times faster (7314150/30140)

5. Conclusion

In this article, we’ve explored two approaches to recursively summing integers in an array. Also, we analyzed their performance using the JMH tool. The “withoutArrayCopy” solution is much faster than the “withArrayCopy” approach.

As always, the complete source code for the examples is available over on GitHub.

       

BigDecimal.ZERO vs. new BigDecimal(0)

$
0
0

1. Overview

When we work with BigDecimal, representing the numerical value zero using BigDecimal is a common task. However, we’re often faced with a choice between two similar approaches: using BigDecimal.ZERO or creating a new BigDecimal object with the constructor new BigDecimal(0).

In this tutorial, we’ll explore the subtle yet significant distinctions between these two methods and discuss when to choose one over the other.

Also, for simplicity, we’ll use unit test assertions to verify results.

2. Comparing BigDecimal Objects

Before we compare BigDecimal.ZERO and new BigDecimal(0), let’s quickly see how to compare two BigDecimal objects.

Given that the BigDecimal class implements the Comparable interface, it provides us with the flexibility to compare two BigDecimals using either the equals() method or the compareTo() method. However, it’s crucial to recognize that these two methods conduct distinct comparisons between two BigDecimal instances.

Let’s say we have two BigDecimal objects, bd1 and bd2. If bd1.compareTo(bd2) == 0, it only indicates the two BigDecimals are equal in value. For example, BigDecimal 42.00 and 42.0000 are equal in value but different in scale:

BigDecimal bd1 = new BigDecimal("42.00");
BigDecimal bd2 = new BigDecimal("42.0000");
assertEquals(0, bd1.compareTo(bd2));

However, it’s important to note that the equals() method in BigDecimal evaluates equality based on both value and scale. Therefore, comparing BigDecimal 42.00 with 42.0000 using the equals() method would result in them being considered unequal:

BigDecimal bd1 = new BigDecimal("42.00");
BigDecimal bd2 = new BigDecimal("42.0000");
assertNotEquals(bd1, bd2);

So, next, let’s compare BigDecimal.ZERO and new BigDecimal(0) using the equals() method:

BigDecimal zero = BigDecimal.ZERO;
BigDecimal zero0 = new BigDecimal(0);
assertEquals(zero, zero0);

As demonstrated by the test above, BigDecimal.ZERO and new BigDecimal(0) exhibit equality in both value and scale. Consequently, they’re mathematically the same. In practical terms, this implies that there is no perceptible difference when employing them in calculations.

Next, let’s have a look at how these two objects get instantiated.

3. How Does BigDecimal.ZERO Work Internally?

BigDecimal.ZERO is a constant field in the BigDecimal class:

public static final BigDecimal ZERO = ZERO_THROUGH_TEN[0];

As we can see, it takes the first element from an array called ZERO_THROUGH_TEN:

private static final BigDecimal[] ZERO_THROUGH_TEN = {
    new BigDecimal(BigInteger.ZERO, 0,  0, 1),
    new BigDecimal(BigInteger.ONE, 1,  0, 1),
    new BigDecimal(BigInteger.TWO, 2,  0, 1),
    ...
    new BigDecimal(BigInteger.TEN, 10, 0, 2),
};

BigDecimal pre-instantiated eleven objects (0 to 10). So, BigDecimal.ZERO and other instances in the array are readily available for use without the need for additional object creation.

Therefore, whenever we use BigDecimal.ZERO, we’re referencing the same object:

BigDecimal z1 = BigDecimal.ZERO;
BigDecimal z2 = BigDecimal.ZERO;
assertSame(z1, z2);

4. How Does new BigDecimal(0) Work Internally?

On the other hand, new BigDecimal(0) creates a new BigDecimal object with the constructor by specifying the value 0:

public BigDecimal(int val) {
    this.intCompact = val;
    this.scale = 0;
    this.intVal = null;
}

As it invokes the constructor, every time we call new BigDecimal(0), a new object is created:

BigDecimal z1 = new BigDecimal(0);
BigDecimal z2 = new BigDecimal(0);
assertNotSame(z1, z2);

5. Which Approach Should We Take?

Both BigDecimal.ZERO and new BigDecimal(0) approaches create immutable BigDecimal objects, ensuring thread safety and consistency. However, as we discussed earlier, BigDecimal.ZERO has the additional advantage of reusing a shared constant object. When BigDecimal.ZERO is used across multiple parts of the codebase, the same object reference is employed, avoiding unnecessary object creation.

Additionally, one of the primary considerations when choosing between BigDecimal.ZERO and new BigDecimal(0) is the clarity and intent the code conveys. BigDecimal.ZERO is widely favored for its readability and conciseness. Its self-explanatory nature makes the code more expressive and aligns with the clear intent of representing 0.

Hence, opting for BigDecimal.ZERO is advisable when our intent is to have a BigDecimal object representing the value 0.

6. Conclusion

In this article, we first learned how BigDecimal.ZERO and new BigDecimal(0) approaches instantiate a BigDecimal instance. Then, we discussed which approach we should take from the readability and object reuse perspectives.

BigDecimal.ZERO stands out for its concise syntax, clear intent, and the potential for shared object references. So, the BigDecimal.ZERO approach should be our first choice if we want a BigDecimal object to represent the value 0.

As always, the complete source code for the examples is available over on GitHub.

       

Convert an OutputStream to a Byte Array in Java

$
0
0

1. Introduction

Dealing with strеams is a common task, еspеcially when working with input and output opеrations. Occasionally, the need arises to convеrt an OutputStrеam into a bytе array. This can be useful in various scеnarios such as nеtwork programming, filе handling, or data manipulation.

In this tutorial, we’ll еxplorе two mеthods to achiеvе this conversion.

2. Using FileUtils From Apache Commons IO Library

The Apachе Commons IO library providеs thе FileUtils class, which includes the readFileToByteArray() mеthod that can indirectly convert a FileOutputStrеam to a bytе array. This is achieved by first writing the file and then reading back the resulting bytes from the filesystem.

To use this library, we first need to include the following Commons IO dеpеndеncy in our project:

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.11.0</version>
</dependency>

Let’s take a simple example to achiеvе this:

@Test
public void givenFileOutputStream_whenUsingFileUtilsToReadTheFile_thenReturnByteArray(@TempDir Path tempDir) throws IOException {
    String data = "Welcome to Baeldung!";
    String fileName = "file.txt";
    Path filePath = tempDir.resolve(fileName);
    try (FileOutputStream outputStream = new FileOutputStream(filePath.toFile())) {
        outputStream.write(data.getBytes(StandardCharsets.UTF_8));
    }
    byte[] writtenData = FileUtils.readFileToByteArray(filePath.toFile());
    String result = new String(writtenData, StandardCharsets.UTF_8);
    assertEquals(data, result);
}

In the above tеst method, we initializе a string data and a filеPath. Furthermore, we utilize the FilеOutputStrеam to writе thе bytе rеprеsеntation of thе string to a filе. Subsеquеntly, we employ the FileUtils.readFileToByteArray() mеthod to еfficiеntly convеrt thе contеnt of thе filе into a bytе array.

Finally, thе bytе array is convеrtеd back to a string, and an assеrtion confirms thе еquality of thе original data and thе rеsult.

It’s crucial to note that this approach only works with a FilеOutputStrеam since we can inspect the written file after the stream is closed. For a morе gеnеral solution that works for different typеs of OutputStrеam, thе nеxt sеction will introduce an altеrnativе mеthod that providеs broadеr applicability.

3. Using a Custom DrainablеOutputStrеam

Anothеr approach involvеs crеating a custom DrainablеOutputStrеam class that еxtеnds FiltеrOutputStrеam. This class intеrcеpts thе bytеs bеing writtеn to thе undеrlying OutputStrеam and kееps a copy in mеmory, allowing for latеr convеrsion to a bytе array.

Lеt’s first crеatе a custom class DrainablеOutputStrеam that еxtеnds FiltеrOutputStrеam:

public class DrainableOutputStream extends FilterOutputStream {
    private final ByteArrayOutputStream buffer;
    public DrainableOutputStream(OutputStream out) {
        super(out);
        this.buffer = new ByteArrayOutputStream();
    }
    @Override
    public void write(byte b[]) throws IOException {
        buffer.write(b);
        super.write(b);
    }
    public byte[] toByteArray() {
        return buffer.toByteArray();
    }
}

In the code above, we first declare a class DrainablеOutputStrеam that will wrap a given OutputStrеam. We include a BytеArrayOutputStrеam buffеr to accumulatе a copy of thе bytеs written and an ovеrriddеn writе() mеthod to capturе thе bytеs. We also implement the toBytеArray() mеthod to provide access to the bytes captured.

Now, let’s utilize the DrainablеOutputStrеam:

@Test
public void givenSystemOut_whenUsingDrainableOutputStream_thenReturnByteArray() throws IOException {
    String data = "Welcome to Baeldung!\n";
    DrainableOutputStream drainableOutputStream = new DrainableOutputStream(System.out);
    try (drainableOutputStream) {
        drainableOutputStream.write(data.getBytes(StandardCharsets.UTF_8));
    }
    byte[] writtenData = drainableOutputStream.toByteArray();
    String result = new String(writtenData, StandardCharsets.UTF_8);
    assertEquals(data, result);
}

In the above test method, we start by initializing a string of data that we want to write to an OutputStrеam. We then utilize the DrainablеOutputStrеam to intеrcеpt this procеss by capturing thе bytеs bеforе thеy arе writtеn to thе actual OutputStrеam. Thе accumulatеd bytеs arе thеn convеrtеd into a bytе array using thе toBytеArray() mеthod.

Subsequently, we crеate a nеw string rеsult from thе intеrcеptеd bytе array and assert its еquality with thе original data.

Note that a comprеhеnsivе implеmеntation of DrainablеOutputStrеam would nееd to providе similar ovеrridеs for othеr writе mеthods bеyond thе еxamplе shown.

4. Considеrations and Limitations

Whilе thе mеthods prеsеntеd in the previous sections providе practical approachеs to convеrt an OutputStrеam to a bytе array, it’s еssеntial to acknowlеdgе cеrtain considеrations and limitations associatеd with this task.

Convеrting an arbitrary OutputStrеam to a bytе array is generally not a straightforward opеration since it may not be possible or practical to retrieve bytes after they have been written.

Cеrtain subclassеs, likе FilеOutputStrеam or BytеArrayOutputStrеam, have built-in mеchanisms that allow us to retrieve the output bytes, such as in-mеmory buffеrs or a written file. On the other hand, if there is no such copy of the output bytes available, we might instead consider using a technique like the DrainablеOutputStrеam to take a copy of the bytes as we are writing them.

5. Conclusion

In conclusion, there are scenarios in programming where transforming an OutputStrеam to a bytе array in Java can be a useful operation. We saw how to read the file resulting from a FilеOutputStrеam using FileUtils.readFileToByteArray() from the Apache Commons IO library, and a more general approach using a custom DrainablеOutputStrеam to take a copy of the written bytes for a given OutputStrеam.

As usual, the accompanying source code can be found over on GitHub.

       

Callbacks in ListenableFuture and CompletableFuture

$
0
0

1. Overview

ListenableFuture and CompletableFuture are built on top of Java’s Future interface. The former is part of Google’s Guava library, whereas the latter is part of Java 8.

As we know, the Future interface doesn’t provide callback functionality. ListenableFuture and CompletableFuture both fill this gap. In this tutorial, we’ll learn callback mechanisms using both of them.

2. Callback in Async Task

Let’s define a use case where we need to download image files from a remote server and persist the names of the image files in a database. Since this task consists of operations over the network and consumes time, it’s a perfect case of using Java async capability.

We can create a function that downloads files from a remote server and attaches a listener that then pushes data to a database when the download completes.

Let’s see how to achieve this task using both ListenableFuture and CompletableFuture.

3. Callback in ListenableFuture

Let’s start by adding Google’s Guava library dependency in the pom.xml:

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>32.1.3-jre</version>
</dependency>

Now, let’s mimic a Future that downloads files from a remote web server:

ExecutorService executorService = Executors.newFixedThreadPool(1);
ListeningExecutorService pool = MoreExecutors.listeningDecorator(executorService);
ListenableFuture<String> listenableFuture = pool.submit(downloadFile());
private static Callable<String> downloadFile() {
  return () -> {
    // Mimicking the downloading of a file by adding a sleep call
    Thread.sleep(5000);
    return "pic.jpg";
  };
}

The above code creates an ExecutorService wrapped inside MoreExecutors to create a thread pool. Inside the submit method of ListenableFutureService, we pass a Callable<String> that downloads a file and returns the name of the file that returns a ListenableFuture.

To attach a callback on the ListenableFuture instance, Guava provides a utility method in Future:

Futures.addCallback(
    listenableFuture,
    new FutureCallback<>() {
        @Override
        public void onSuccess(String fileName) {
            // code to push fileName to DB
        }
        @Override
        public void onFailure(Throwable throwable) {
            // code to take appropriate action when there is an error
        }
    },
    executorService);
}

So, in this callback, both success and exception scenarios are handled. This way of using a callback is quite natural.

We can also add a listener by adding it directly to the ListenableFuture:

listenableFuture.addListener(
    new Runnable() {
        @Override
        public void run() {
            // logic to download file
        }
    },
    executorService
);

This callback doesn’t have access to the result of Future as its input is Runnable. This callback method executes whether Future completes successfully or not.

After going through the callbacks in ListenableFuture, the next section explores the ways in CompletableFuture to achieve the same.

4. Callback in CompletableFuture

In CompletableFuture, there are many ways to attach a callback. The most popular ways are by using chaining methods such as thenApply(), thenAccept(), thenCompose(), exceptionally(), etc., that execute normally or exceptionally.

In this section, we’ll learn about a method whenComplete(). The best thing about this method is that it can be completed from any thread that wants it to be completed. Using the above file downloading example, let’s see how to use whenComplete():

CompletableFuture<String> completableFuture = new CompletableFuture<>();
Runnable runnable = downloadFile(completableFuture);
completableFuture.whenComplete(
  (res, error) -> {
      if (error != null) {
          // handle the exception scenario
      } else if (res != null) {
          // send data to DB
      }
  });
new Thread(runnable).start();
private static Runnable downloadFile(CompletableFuture<String> completableFuture) {
  return () -> {
      try {
          // logic to download to file;
      } catch (InterruptedException e) {
          log.error("exception is {} "+e);
      }
      completableFuture.complete("pic.jpg");
  };
}

When downloading the file completes, the whenComplete() method executes either the success or failure conditions.

5. Conclusion

In this article, we learned about the callback mechanism in ListenableFuture and CompletableFuture.

We saw that ListenableFuture is a more natural and fluid callback API when compared to CompletableFuture.

We’re free to choose what fits best to our use case as CompletableFuture is part of core Java, and ListenableFuture is part of the very popular Guava library.

All the code examples used in this article are available over on GitHub.

       

Creating a Kafka Listener Using the Consumer API

$
0
0

1. Overview

In this tutorial, we’ll learn how to create a Kafka listener and consume messages from a topic using Kafka’s Consumer API. After that, we’ll test our implementation using the Producer API and Testcontainers.

We’ll be focusing on setting up a KafkaConsumer without relying on Spring Boot modules.

2. Create a Custom Kafka Listener

Our custom listener will internally use the Producer and Consumer APIs from the kafka-clients library. Let’s start by adding this dependency to our pom.xml file:

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>3.6.1</version>
</dependency>

For the code examples in this article, we’ll create a CustomKafkaListener class that will listen to the topic named “baeldung.articles.published“. Internally, our class will wrap around a KafkaConsumer and leverage it to subscribe to the topic:

class CustomKafkaListener {
    private final String topic;
    private final KafkaConsumer<String, String> consumer;
    // ...
}

2.1. Create a KafkaConsumer

To create a KafkaConsumer, we need to supply a valid configuration via a Properties object. Let’s create a simple consumer that can be used as a default when creating our CustomKafkaListener instance:

public CustomKafkaListener(String topic, String bootstrapServers) {
    this(topic, defaultKafkaConsumer(bootstrapServers));
}
static KafkaConsumer<String, String> defaultKafkaConsumer(String boostrapServers) {
    Properties props = new Properties();
    props.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, boostrapServers);
    props.setProperty(ConsumerConfig.GROUP_ID_CONFIG, "test_group_id");
    props.setProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
    props.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
    props.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
    return new KafkaConsumer<>(props);
}

For this example, we hardcoded most of these properties, but, ideally, they should be loaded from a configuration file. Let’s quickly see what each of the properties means:

  • Boostrap Servers: a list of host and port pairs used for establishing the initial connection to the Kafka cluster
  • Group ID: an ID that allows a group of consumers to jointly consume a set of topic partitions
  • Auto Offset Reset: the position in the Kafka log to start reading data when there is no initial offset
  • Key/Value Deserializers: the deserializer classes for the keys and values. For our example, we’ll use String keys and values and the following deserializer: org.apache.kafka.common.serialization.StringDeserializer

With this minimal configuration, we’ll be able to subscribe to the topic and easily test out implementation. For a complete list of the available properties, consult the official documentation.

2.2. Subscribe to a Topic

Now, we need to provide a way to subscribe to the topic and start polling for messages. This can be achieved using KafkaConsumer‘s subscribe() method, followed by an infinite loop calling to the poll() method. Moreover, since this method will block the thread, we can implement the Runnable interface to provide a nice integration with CompletableFuture:

class CustomKafkaListener implements Runnable {
    private final String topic;
    private final KafkaConsumer<String, String> consumer;
    // constructors
    @Override
    void run() {
        consumer.subscribe(Arrays.asList(topic));
        while (true) {
            consumer.poll(Duration.ofMillis(100))
              .forEach(record -> log.info("received: " + record)); 
        } 
    }
}

Now, our CustomKafkaListener can be started  like this without blocking the main thread:

String topic = "baeldung.articles.published";
String bootstrapServers = "localhost:9092";
var listener = new CustomKafkaListener(topic, bootstrapServers)
CompletableFuture.runAsync(listener);

2.3. Consume Activity

Currently, our application only listens to the topic and logs all incoming messages. Let’s further improve it to allow more complex scenarios and make it more testable. For example, we can allow defining a Consumer<String> that will accept each new event from the topic:

class CustomKafkaListener implements Runnable {
    private final String topic;
    private final KafkaConsumer<String, String> consumer;
    private Consumer<String> recordConsumer;
    CustomKafkaListener(String topic, KafkaConsumer<String, String> consumer) {
        this.topic = topic;
        this.consumer = consumer;
        this.recordConsumer = record -> log.info("received: " + record);
    }
   // ...
    @Override
    public void run() {
        consumer.subscribe(Arrays.asList(topic));
        while (true) {
            consumer.poll(Duration.ofMillis(100))
              .forEach(record -> recordConsumer.accept(record.value()));
        }
    }
    CustomKafkaListener onEach(Consumer newConsumer) {
        recordConsumer = recordConsumer.andThen(newConsumer);
        return this;
    }
}

Declaring the recordConsumer as a Consumer<String> allows us to chain multiple functions using the default method andThen(). These functions will be called, one by one, for each incoming message.

3. Testing

To test our implementation, we’ll create a KafkaProducer and use it to publish some messages to the “baeldung.articles.published” topic. Then, we’ll start our CustomKafkaListener and verify it accurately processes all the activity.

3.1. Setup Kafka Testcontainer

We can utilize the Testcontainers library to spin up a Kafka container within our test environment. Firstly, we’ll need to add Testcontainer dependencies for the JUnit5 extension and the Kafka module:

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>kafka</artifactId>
    <version>1.19.3</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>1.19.3</version>
    <scope>test</scope>
</dependency>

We can now create a KafkaContainer with a specific Docker image name. Then, we’ll add the @Container and @Testcontainers annotations to allow the Testcontainers JUnit5 extension to manage the container’s lifecycle:

@Testcontainers
class CustomKafkaListenerLiveTest {
    @Container
    private static final KafkaContainer KAFKA_CONTAINER = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:latest"));
    // ...
}

3.2. Create and Start the Listener

Firstly, we’ll define the topic name as a hardcoded String and extract the bootstrapServers from the KAFKA_CONTAINER. Additionally, we’ll create an ArrayList<String> that will be used for collecting the messages:

String topic = "baeldung.articles.published";
String bootstrapServers = KAFKA_CONTAINER.getBootstrapServers();
List<String> consumedMessages = new ArrayList<>();

We’ll create an instance of a CustomKafkaListener using these properties and instruct it to capture new messages and add them to the consumedMessages list:

CustomKafkaListener listener = new CustomKafkaListener(topic, bootstrapServers).onEach(consumedMessages::add);
listener.run();

However, it’s crucial to note that running it as is might block the thread and freeze the test. To prevent this, we’ll execute it asynchronously using a CompletableFuture:

CompletableFuture.runAsync(listener);

While not critical for testing, we can also instantiate the listener within a try-with-resources block in the first place:

var listener = new CustomKafkaListener(topic, bootstrapServers).onEach(consumedMessages::add);
CompletableFuture.runAsync(listener);

3.3. Publish Messages

To send article names to the “baeldung.articles.published” topic, we’ll set up a KafkaProducer using a Properties object, following a similar approach to what we did for the consumer.

static KafkaProducer<String, String> testKafkaProducer() {
    Properties props = new Properties();
    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers());
    props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
    props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
    return new KafkaProducer<>(props);
}

This method will allow us to publish messages to test our implementation. Let’s create another test helper that will send a message for each article received as a parameter:

private void publishArticles(String topic, String... articles) {
    try (KafkaProducer<String, String> producer = testKafkaProducer()) {
        Arrays.stream(articles)
          .map(article -> new ProducerRecord<String,String>(topic, article))
          .forEach(producer::send);
    }
}

3.4. Verify

Let’s put all the pieces together and run our test. We’ve already discussed how to create a CustomKafkaListener and start publishing data:

@Test
void givenANewCustomKafkaListener_thenConsumesAllMessages() {
    // given
    String topic = "baeldung.articles.published";
    String bootstrapServers = KAFKA_CONTAINER.getBootstrapServers();
    List<String> consumedMessages = new ArrayList<>();
    // when
    CustomKafkaListener listener = new CustomKafkaListener(topic, bootstrapServers).onEach(consumedMessages::add);
    CompletableFuture.runAsync(listener);
    
    // and
    publishArticles(topic,
      "Introduction to Kafka",
      "Kotlin for Java Developers",
      "Reactive Spring Boot",
      "Deploying Spring Boot Applications",
      "Spring Security"
    );
    // then
    // ...
}

Our final task involves waiting for the asynchronous code to finish and confirming that the consumedMessages list contains the expected content. To achieve this, we’ll employ the Awaitility library, utilizing its await().untilAsserted():

// then
await().untilAsserted(() -> 
  assertThat(consumedMessages).containsExactlyInAnyOrder(
    "Introduction to Kafka",
    "Kotlin for Java Developers",
    "Reactive Spring Boot",
    "Deploying Spring Boot Applications",
    "Spring Security"
  ));

4. Conclusion

In this tutorial, we learned how to use Kafka’s Consumer and Producer API without relying on higher-level Spring modules. First, we created a consumer using CustomKafkaListener that encapsulates a KafkaConsumer. For testing, we implemented a KafkaProducer and verified our setup using Testcontainers and Awaitility.

As always, the source for the examples is available over on GitHub.

       
Viewing all 4469 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>