1. Introduction

Sorting an array is a common operation in Java, but sometimes, we also need to know the original indices of the elements after sorting. This information can be crucial for certain algorithms and applications.

In this tutorial, we’ll illustrate different approaches to achieve this in Java.

2. Problem Description

Sorting an array is a fundamental operation, but in some scenarios, it’s not just about arranging values in order; we also need to retain the original positions of these values. This becomes especially crucial when we want to know how the order of elements changes after sorting. Let’s consider the following array:

int[] array = {40, 10, 20, 30};

Before sorting, the indices (positions) of elements in this array are:

• Index 0: 40
• Index 1: 10
• Index 2: 20
• Index 3: 30

After sorting this array, we obtain the new indices of the elements:

• Index 0: 10 (originally at index 1)
• Index 1: 20 (originally at index 2)
• Index 2: 30 (originally at index 3)
• Index 3: 40 (originally at index 0)

We aim to sort this array in ascending order while also tracking how the indices change based on the sorted values.

3. Using a Custom Comparator with Indices

One approach to obtaining the indices after sorting is to use a custom comparator that maintains the index alongside the array elements. Furthermore, this method allows us to sort the array based on element values while keeping track of their original positions.

Now, let’s demonstrate this with some code:

int[] array = {40, 10, 20, 30};
@Test
void givenArray_whenUsingCustomComparator_thenSortedIndicesMatchExpected() {
    Integer[] indices = new Integer[array.length];
    for (int i = 0; i < array.length; i++) {
        indices[i] = i;
    }
    Arrays.sort(indices, Comparator.comparingInt(i -> array[i]));
    assertArrayEquals(new Integer[]{1, 2, 3, 0}, indices);
}

In this example, we initialize an indices array to hold the indices of the original array array. Each element of the indices array represents the index of the corresponding element in the array.

By using a custom comparator with the Arrays.sort() method, we specify that the indices array should be sorted based on the values in the array. Moreover, the Comparator.comparingInt(i -> array[i]) compares elements in the indices array based on the values of the array at those indices.

After sorting, we use assertArrayEquals() to verify that the sorted indices array matches the expected order [1, 2, 3, 0].

4. Using Java 8 Stream API

One of the significant new features in Java 8 was the introduction of the stream functionality – java.util.stream – which contains classes for processing sequences of elements.

Here’s how we can leverage the Java 8 Stream API to obtain and sort indices based on the values in the original array:

@Test
void givenArray_whenUsingStreamAPI_thenSortedIndicesMatchExpected() {
    List<Integer> indices = IntStream.range(0, array.length)
      .boxed().sorted(Comparator.comparingInt(i -> array[i])).collect(Collectors.toList());
    assertIterableEquals(Arrays.asList(1, 2, 3, 0), indices);
}

Here, we utilize the IntStream.range() method to generate a stream of integer indices from 0 to array.length – 1. This stream is then boxed into a Stream<Integer> using the boxed() method.

Then, we use the sorted() operation, using a comparator defined by Comparator.comparingInt(i -> array[i]). Here, each index i in the stream is mapped to the corresponding value in the array, and the comparator is based on these values. Finally, we use the collect(Collectors.toList()) method to collect the sorted indices into a List<Integer>.

5. Conclusion

In conclusion, we explored effective methods in Java for sorting arrays while preserving the original elements indices. This information is crucial for various algorithms and applications where maintaining the positional relationship of elements is important.

As usual, the accompanying source code can be found over on GitHub.

       

1. Overview

Java provides several libraries and APIs for working with XML and PDF documents. Converting XML to PDF in Java involves parsing the XML data, applying styles and formatting, and generating the PDF output.

This article explores different methods and libraries to convert XML to PDF in Java.

2. Understanding the Conversion Process

Before discussing implementation details, let’s highlight the essential steps to convert XML to PDF. This process typically entails two primary steps:

1. The first step is XML parsing, where the XML content is analyzed, and its structure and textual data are extracted. In Java, developers have access to various XML parsing libraries such as DOM (Document Object Model), SAX (Simple API for XML), and StAX (Streaming API for XML).
2. The second step involves PDF generation. This step includes creating PDF components such as paragraphs, tables, images, and other elements. These components are then organized and formatted according to the structure defined within the XML document.

3. Using Apache FOP (Formatting Objects Processor)

Apache FOP is a robust open-source library for converting XML data into various output formats, including PDF. Furthermore, FOP transforms XML content according to XSL-FO stylesheets, ultimately generating high-quality PDF documents.

3.1. How Apache FOP Works

Apache FOP works through the following key stages:

• XML Parsing: Apache FOP begins by parsing the input XML data. This process involves extracting the structure and content of the XML document, which typically represents the data to be presented in the final PDF output.
• XSL-FO Transformation: FOP applies an XSL-FO stylesheet to format XML elements into corresponding PDF elements like paragraphs, tables, and images, ensuring adherence to specified styles and layout rules.
• PDF Rendering: After transforming the content into XSL-FO format, Apache FOP renders it into a visually appealing PDF document that accurately reflects the original XML content.
• Output Generation: Finally, FOP generates a standalone PDF file encapsulating the formatted content, ready for saving, display, or distribution, suitable for various printing and viewing purposes.

3.2. Example: Converting XML to PDF using Apache FOP

To use the Apache FOP library and its features for converting XML to PDF, it is necessary to integrate the Apache FOP dependency into our project’s build configuration.

If we’re using Maven, we can achieve this by including the FOP dependency in our pom.xml file:

<dependency>
    <groupId>org.apache.xmlgraphics</groupId>
    <artifactId>fop</artifactId>
    <version>2.9</version>
</dependency>

Now, let’s create a method to convert XML to PDF using Apache FOP in Java:

void convertXMLtoPDFUsingFop(String xmlFilePath, String xsltFilePath, String pdfFilePath) throws Exception {
    FopFactory fopFactory = FopFactory.newInstance(new File(".").toURI());
    FOUserAgent foUserAgent = fopFactory.newFOUserAgent();
    try (OutputStream out = new BufferedOutputStream(Files.newOutputStream(new File(pdfFilePath).toPath()))) {
        Fop fop = fopFactory.newFop(MimeConstants.MIME_PDF, foUserAgent, out);
        TransformerFactory factory = TransformerFactory.newInstance();
        Transformer transformer = factory.newTransformer(new StreamSource(new File(xsltFilePath)));
        Source src= new StreamSource(new File(xmlFilePath));
        Result res = new SAXResult(fop.getDefaultHandler());
        transformer.transform(src, res);
    }
}

The above example highlights the key steps involved in the conversion process, which include:

• Initialization: we first initialize Apache FOP by creating instances of FopFactory and FOUserAgent.
• Output Stream: we specify the output stream for the resulting PDF file.
• FOP Instance Creation: a new Fop instance is created using the FopFactory, specifying the PDF output format.
• XSLT Transformation: we create a Transformer instance from the XSLT stylesheet specified in the xsltFilePath parameter.
• Transformation Application: the XML data defined in the xmlFilePath parameter is transformed using the XSLT stylesheet, and the resulting FO (Formatting Object) is sent to the FOP instance for rendering.
• Output Generation: finally, the method generates the PDF output and saves it to the specified file path provided in the pdfFilePath parameter.

4. Using IText Library

The iText library is a robust and flexible solution for generating and managing PDF files. Its comprehensive capabilities enable seamless conversion of XML content into PDF documents, offering tailored customization and adaptability.

4.1. How IText Works

IText works through the following key stages:

• HTML to PDF Conversion: iText converts XML data to PDF using HTML as an intermediate format. XML is transformed into HTML, leveraging iText’s HTML parsing capabilities for seamless integration into PDF documents.
• XML Parsing and Rendering: iText parses XML content and renders it directly into PDF. It supports various XML formats like XHTML, SVG, and MathML and can apply CSS styles for precise control over layout and appearance.
• PDF Generation: After parsing, iText generates PDF elements such as text, images, and tables. Developers can customize the output with headers, footers, and other elements, ensuring compliance with PDF standards for printing and viewing.

4.2. Converting XML to PDF using iText in Java

To use the iText library for PDF generation in Java, We must incorporate the iTextPDF dependency in our project configuration. For Maven, we can add the iText dependency to our pom.xml file:

<dependency>
    <groupId>com.itextpdf</groupId>
    <artifactId>itextpdf</artifactId>
    <version>5.5.13.3</version>
</dependency>

Here’s a simple example demonstrating how to convert XML to PDF using iText in Java:

public static void convertXMLtoPDFUsingIText(String xmlFilePath, String pdfFilePath) throws Exception {
    try (FileOutputStream outputStream = new FileOutputStream(pdfFilePath)) {
        Document document = new Document();
        PdfWriter.getInstance(document, outputStream);
        document.open();
        String xmlContent = new String(Files.readAllBytes(Paths.get(xmlFilePath)));
        document.add(new Paragraph(xmlContent));
        document.close();
    }
}

The above example illustrates a straightforward method for converting XML to PDF using iText in Java. First, we create a new PDF document object. Next, we open the document to write content. Following this, we read the XML content from the specified file path and embed it into the PDF document.

Finally, we close the document and the output stream, ensuring the saved PDF file contains the XML content in a structured format.

5. Conclusion

Exploring XML to PDF conversion with FOP and iText in this article has provided us with valuable knowledge and practical skills. Mastery of these techniques enables us to efficiently convert XML data into refined PDF documents, enhancing the functionality of our Java applications.

As always, the source code is available over on GitHub.

       

1. Introduction

When working with databases in Java, handling large text data is a common task. Besides, the Character Large Object (CLOB) type allows databases to store extensive text data. Moreover, converting between the CLOB and String objects is often necessary when reading from or writing to databases.

In this tutorial, we’ll explore how to perform this conversion efficiently in Java.

2. Converting CLOB to String

In this approach, we’ll leverage standard Java I/O (Reader and Writer) operations to efficiently handle character data from a SQL Clob object. The Reader reads data from the Clob, which is then processed and written to a StringWriter for conversion to a String object.

Here’s how we can achieve this:

@Test
public void givenCLOB_whenConvertToString_thenCorrect() throws SQLException, IOException {
    Clob clob = new javax.sql.rowset.serial.SerialClob("This is a sample CLOB content.".toCharArray());
    String clobAsString;
    try (Reader reader = clob.getCharacterStream();
         StringWriter w = new StringWriter()) {
        char[] buffer = new char[4096];
        int charsRead;
        while ((charsRead = reader.read(buffer)) != -1) {
            w.write(buffer, 0, charsRead);
        }
        clobAsString = w.toString();
    }
    assertEquals("This is a sample CLOB content.", clobAsString);
}

Here, we first create a Clob object with sample content using SerialClob. Next, we obtain a Reader from the Clob using the getCharacterStream() method, which allows us to read the character data from the Clob. We use a StringWriter named w to capture the character data read from the Reader.

Inside the try block with resources, we define a buffer (char[] buffer) to read characters from the Reader. We then read characters from the Reader into the buffer and write them to the StringWriter using the write() method.

After reading all characters from the Clob into the StringWriter, we convert the content of the StringWriter to a String object using the toString() method, which gives us the content of the Clob as a String object. Finally, we use the assertEquals() method to verify that the colbAsString matches the expected content of the original Clob object.

3. Converting CLOB to String

Let’s dive into the implementation of how to convert a String object into a Clob object:

@Test
public void givenString_whenConvertToCLOB_thenCorrect() throws SQLException {
    String sampleText = "This is a sample text to be stored as CLOB.";
    char[] charArray = sampleText.toCharArray();
    Clob clob = new javax.sql.rowset.serial.SerialClob(charArray);
    assertEquals(sampleText, clob.getSubString(1, (int) clob.length()));
}

Here, we define a String object named sampleText containing the text we want to store as a Clob object. Next, we convert the String into a character array (charArray) using the toCharArray() method. This step prepares our text for storage in the Clob object.

Afterward, we create a Clob object using its constructor, SerialClob(charArray), where charArray represents the character data to be stored.

4. Conclusion

In this tutorial, we’ve explored how to convert between Clob objects and String representations in Java, which is essential for managing large text data when interacting with databases.

As always, the complete code samples for this article can be found over on GitHub.

       

1. Overview

In this tutorial, we’ll review how entities are managed in JPA and then explore a scenario where the persistence context may not return fresh data due to external changes.

2. Persistence Context

Each EntityManager is associated with a persistence context that stores managed entities in memory. Whenever we perform any data operations on an entity via the EntityManager, the entity becomes managed by the persistence context.

When we retrieve the entity again, JPA returns the managed entity from the persistence context instead of fetching it from the database. This caching mechanism helps improve the performance without fetching the same data repetitively from the database.

The persistence context is also known as the first-level (L1) cache in JPA. 

3. Scenario Setup

First, let’s create a simple entity class:

@Entity
@Table(name = "person")
public class Person {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String name;
    // constructors, getters and setters
}

Next, we’ll create a Person entity and persist it to the database:

EntityTransaction transaction = entityManager.getTransaction();
transaction.begin();
Person person = new Person();
person.setName("David Jones");
entityManager.persist(person);
transaction.commit();

4. Without Clearing Managed Entities

Our sample data is ready. Let’s mask the Person‘s name field using an SQL update query. We obtain a JDBC connection from the EntityManager and execute the update query:

Session session = entityManager.unwrap(Session.class);
session.doWork(connection -> {
    try (PreparedStatement pStmt = connection.prepareStatement("UPDATE person SET name=? WHERE id=?")) {
        pStmt.setString(1, "*****");
        pStmt.setLong(2, 1);
        pStmt.executeUpdate();
    }
});

Now, we can retrieve the same entity again by the EntityManager:

Person updatedPerson = entityManager.find(Person.class, 1);

Intuitively, we’d assume the retrieved Person entity’s name has been masked already due to the update query. However, the name remains “David Jones” when we verify the name of the entity:

assertThat(updatedPerson.getName()).isNotEqualTo("*****");

This occurs because JPA retrieves the managed entity from the persistence context which hasn’t been updated accordingly since the JDBC data update. When data changes occur outside of the EntityManager, it’s  unaware of them and is unable to update the corresponding managed entities.

5. With Clearing Managed Entities

Consequently, whenever we make any changes to the database data without using EntityManager, we must force JPA to clear all managed entities from the persistence context such that entities can be fetched again from the database.

To do so, we invoke the clear() method of the EntityManager:

entityManager.clear();

This action detaches all managed entities from the persistence context ensuring JPA no longer keeps track of those entities.

Subsequently, we can retrieve the same entity using the EntityManager again. We’ll see the following SQL being executed from the console log, if we enable hibernate.show_sql option in the persistence unit configuration:

Hibernate: 
    select
        p1_0.id,
        p1_0.name 
    from
        person p1_0 
    where
        p1_0.id=?

This SQL statement indicates that the EntityManager executes a select query to retrieve fresh data from the database. We can expect the Person entity’s name has been masked this time:

assertThat(updatedPerson.getName()).isEqualTo("*****");

6. Conclusion

In this article, we’ve learned the role of persistence context in JPA. The persistence context won’t reload the entities if we perform data changes without involving EntityManager. We need to invoke the clear() method of EntityManager to remove all manage entities and allow reloading entities from the database again.

As always, the full source code of the examples can be found over on GitHub.

       

1. Overview

In the demanding world of software development, ensuring that applications perform optimally and reliably once deployed in production environments is not just preferable — it’s essential. Using Spring Boot, developers can effortlessly set up standalone, high-grade applications. However, to truly enhance performance, availability, and reliability, integrating a sophisticated monitoring tool like Prometheus is key.

This tutorial aims to provide a detailed walkthrough on connecting Prometheus with a Spring Boot application, enriching our monitoring strategy with both fundamental and intricate configurations.

2. What Is Prometheus?

Prometheus is an open source project designed to dive deep into our application data, creating the filtering layers to collect and analyze everything from the simplest to the most complex. It’s not just about numbers and charts, either: It’s about understanding the heartbeat of our application through its advanced query language and time-series data capabilities.

Integrating Prometheus gives us the ability to identify problems before they occur, fine-tune our systems, ensure our application runs at peak performance, and ultimately, means a better experience for our users – convenient, fast, and reliable.

3. Getting Started With Prometheus in Spring Boot

Integrating Prometheus with our Spring Boot application allows us to effectively monitor by exposing application metrics in a format that Prometheus can understand and scrape. This process involves two main steps: adding the necessary dependencies to our project and configuring our application to expose metrics.

3.1. Adding Dependencies

To get started, we add the Spring Boot Actuator and the Micrometer Prometheus registry to the dependencies of our project. The actuator provides a series of built-in endpoints to display performance information about the running application, such as health, metrics, and more. The Micrometer Prometheus registry then formats these metrics into a Prometheus-readable format.

Let’s add the dependencies to our pom.xml file for a Maven project:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

If we’re using Gradle, we include these in our build.gradle file:

implementation 'org.springframework.boot:spring-boot-starter-actuator'
implementation 'io.micrometer:micrometer-registry-prometheus'

3.2. Configuring the Application

After adding the dependencies, the next step is to configure our Spring Boot application to expose the Prometheus metrics endpoint. This is done by updating the application.properties or application.yml file in our project.

For application.properties, we add:

management.endpoints.web.exposure.include=*
management.endpoint.health.show.details=always

This configuration ensures the Actuator’s /actuator/prometheus endpoint is exposed, providing a wealth of application metrics in a format that Prometheus can scrape.

It’s important to note that exposing all Actuator endpoints (management.endpoints.web.exposure.include=*) can provide useful insights during development but may expose sensitive operational data. For production environments, it’s best practice for us to carefully select which endpoints to expose based on our monitoring and operational needs.

By following these steps, our Spring Boot application will now expose valuable metrics that Prometheus can collect and store. This foundational setup is crucial for the next stages of our monitoring strategy, including scraping these metrics with Prometheus and visualizing them using tools like Grafana.

4. Setting up Prometheus to Scrape Metrics

With our application now configured to expose metrics, the next step involves setting up Prometheus to collect these metrics. This part of the process requires us to download and configure Prometheus — detailing a step-by-step guide based on the operating system — and to adjust the prometheus.yml file to target our Spring Boot application.

4.1. Installation

Installing Prometheus is straightforward. Let’s review the steps for the two most common operating systems and Docker:

For Windows:

1. Download the latest Prometheus version from the official Prometheus website, selecting the Windows version.
2. Extract the downloaded .zip file to our desired location, which will serve as the Prometheus home directory.
3. Run Prometheus by opening a command prompt, navigating to the Prometheus directory, and executing prometheus.exe. This action starts the Prometheus server.

For Linux/Mac:

1. Download the latest version for Linux or Mac from the Prometheus downloads page.
2. Extract the tarball using the command tar xvfz prometheus-*.tar.gz.
3. Navigate into the extracted directory, making it the Prometheus home directory.
4. Start Prometheus by executing ./prometheus in the terminal, which launches the Prometheus server.

Using Docker:

Opting for Docker to deploy Prometheus facilitates the setup process, offering a consistent and simplified installation method regardless of the operating system. Here’s how we can achieve this:

1. Pull the Prometheus Image: Initially, we pull the official Prometheus image from Docker Hub to ensure we have the latest version:

   docker pull prom/prometheus

2. Run the Prometheus Container: Following the image pull, we can launch a Prometheus container. This command forwards the Prometheus default port (9090) to the same port on our host, enabling access to the Prometheus web UI via http://localhost:9090:

   docker run --name prometheus -d -p 9090:9090 prom/prometheus

   For deployments requiring a custom configuration, a custom prometheus.yml file located on our host can be mounted into the container:

   docker run --name prometheus -d -p 9090:9090 -v /our/path/to/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

   We replace /our/path/to/prometheus.yml with the actual path to our configuration file.

3. Access Prometheus Web UI: With Prometheus actively running in a Docker container, the UI will become accessible through http://localhost:9090. This interface is instrumental for executing queries, gazing at gathered metrics, and verifying the status of scrape goals.

4.2. Configuring Prometheus to Scrape Metrics

Once installed, we need to configure Prometheus to scrape metrics from our Spring Boot application. This calls for editing the prometheus.yml record located within the Prometheus home listing.

We add our Spring Boot application as a target under the scrape_configs section:

scrape_configs:
  - job_name: 'spring-boot-application'
    metrics_path: '/actuator/prometheus'
    scrape_interval: 15s # This can be adjusted based on our needs
    static_configs:
      - targets: ['localhost:8080']

Here, localhost:8080 should be replaced with the appropriate host and port where our Spring Boot application is running. The scrape_interval specifies how often Prometheus should scrape metrics from our application.

4.3. Visualizing Metrics With Grafana

While Prometheus excels at collecting and storing metrics, Grafana is our tool of choice for visualizing these metrics through insightful dashboards.

Integrating Grafana with Prometheus:

1. Install Grafana by visiting the official Grafana download page.
2. Launch Grafana and access its web interface by going to http://localhost:3000 on a web browser.
3. Add Prometheus as a data source by navigating to Configuration > Data Sources > Add data source in Grafana’s UI. We select Prometheus as the type and specify the URL where Prometheus is running, usually http://localhost:9090.
4. Save & Test to confirm Grafana can successfully connect to Prometheus.

Creating Dashboards in Grafana:

1. Create a brand new dashboard by clicking on the icon at the left sidebar and deciding on the Dashboard.
2. Add a brand new panel to the dashboard. Here, we pick the metric to show, decide on the visualization kind (Graph, Gauge, Table, and so on.), and customize the panel’s appearance.
3. Select our Prometheus records source and utilize the Prometheus Query Language (PromQL) to pick the metrics we wish to visualize. For example, to reveal the charge of HTTP requests, we would use a query like price(http_requests_total[5m]).
4. Save our panel and dashboard. We can create as many panels as possible to visualize exceptional metrics from our Spring Boot utility.

Following these steps enables us to set up Prometheus to scrape metrics from our Spring Boot application and leverage Grafana to visualize them. This setup provides us with crucial insights into our application’s health and overall performance.

5. Advanced Configurations and Best Practices

This section addresses advanced configurations that can enhance our Spring Boot application’s observability and security. We’ll explore setting up alerting rules in Prometheus and creating custom metrics in Spring Boot using Micrometer.

Following these best practices ensures that our application remains robust, secure, and highly available.

5.1. Custom Metrics in Spring Boot

Spring Boot’s integration with Micrometer provides a seamless approach to adding custom metrics to our application, enabling us to monitor application-specific behaviors and operations. Here’s how we can create and register custom metrics:

@Component
public class CustomMetricsService {
    private final Counter customMetricCounter;
    public CustomMetricsService(MeterRegistry meterRegistry) {
        customMetricCounter = Counter.builder("custom_metric_name")
          .description("Description of custom metric")
          .tags("environment", "development")
          .register(meterRegistry);
    }
    public void incrementCustomMetric() {
        customMetricCounter.increment();
    }
}

In this example, we define a custom counter metric named custom_metric_name. This counter can be incremented to track a specific event within our application, such as user registrations or login attempts.

By injecting the MeterRegistry, we register our custom metric, making it available for scraping by Prometheus.

5.2. Best Practices for Monitoring and Alerting

Let’s review a few best practices for this endeavor:

• Set up Alerting Rules in Prometheus: Define alerting rules based on application-specific metrics and thresholds. This proactive approach helps in identifying and addressing issues before they impact users.
• Monitor Business-Critical Transactions: Beyond system health, track metrics that represent critical business functionality, such as order completions or payment transactions.
• Secure Access to Metrics: Ensure that metrics endpoints are protected to prevent unauthorized access. Utilize Spring Security to configure access controls as needed.
• Regularly Review Metrics and Alerts: Periodically review configured metrics and alerts to ensure they remain relevant to operational and business needs. Adjust thresholds and metrics as the application evolves.

By implementing these advanced configurations and adhering to best practices, we can achieve a robust monitoring and alerting setup that not only safeguards our application but also provides deep insights into its performance and usage patterns.

6. Conclusion

Utilizing Prometheus and Grafana to monitor our Spring Boot applications provides a robust approach to understanding application behavior and preempting potential problems. This article has guided us through putting in place an effective monitoring solution, improving our packages’ observability.

By integrating that equipment into our improvement and deployment workflows, we extensively bolster the reliability and performance of our packages. This not only ensures they meet user and business requirements but also fosters a culture of continuous improvement.

Adopting Prometheus and Grafana for tracking is more than a strategy for maintaining utility health. It’s a step toward proactive management and optimization of our packages, setting the stage for sustained success and growth.

       

1. Overview

In this tutorial, we’ll explore the usage of Spring Boot’s @Autowired and Mockito’s @InjectMocks while injecting dependencies in Spring Boot tests. We’ll go over the use cases that require us to use them and look at examples for the same.

2. Understanding Test Annotations

Before starting with the code example, let’s quickly look at the basics of some test annotations.

First, the most commonly used @Mock annotation of Mockito creates a mock instance of a dependency for testing. It’s often used in conjunction with @InjectMocks which injects the mocks marked with @Mock into the target object being tested.

In addition to Mockito’s annotations, Spring Boot’s annotation @MockBean can help create a mocked Spring bean. The mocked bean can then be used by other beans in the context. Moreover, if a Spring context creates beans on its own that can be utilized without mocking, we can use the @Autowired annotation to inject them.

3. Example Setup

In our code example, we’ll create a service having two dependencies. We’ll then explore using the above annotations to test the service.

3.1. Dependencies

Let’s start by adding the required dependencies. We’ll include the Spring Boot Starter Web and Spring Boot Starter Test dependencies:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>3.2.5</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <version>3.2.5</version>
    <scope>test</scope>
</dependency>

In addition to this, we’ll add the Mockito Core dependency that we’ll need to mock our services:

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>5.11.0</version>
</dependency>

3.2. DTO

Next, let’s create a DTO that we’ll use in our services:

public class Book {
    private String id;
    private String name;
    private String author;
    
    // constructor, setters/getters
}

3.3. Services

Next, let’s look at our services. First, let’s define a service that is responsible for database interactions:

@Service
public class DatabaseService {
    public Book findById(String id) {
        // querying a Database and getting a book
        return new Book("id","Name", "Author");
    }
}

We’ll not go into the database interactions as they are irrelevant to the example. We use the @Service annotation to declare the class a Spring bean of Service stereotype.

Next, let’s introduce a service that is dependent on the above service:

@Service
public class BookService {
    private DatabaseService databaseService;
    private ObjectMapper objectMapper;
    BookService(DatabaseService databaseService, ObjectMapper objectMapper) {
        this.databaseService = databaseService;
        this.objectMapper = objectMapper;
    }
    String getBook(String id) throws JsonProcessingException {
        Book book = databaseService.findById(id);
        return objectMapper.writeValueAsString(book);
    }
}

Here we have a small service that has a getBook() method. The method utilizes the DatabaseService to get a book from the database. It then uses the ObjectMapper API of Jackson to convert and return the Book object into a JSON string.

Therefore, this service has two dependencies: DatabaseService and ObjectMapper.

4. Testing

Now that our services are set up, let’s look at ways to test BookService using the annotations we defined earlier.

4.1. Using @Mock and @InjectMocks

The first option is to mock both dependencies of the service using @Mock and inject them into the service using @InjectMocks. Let’s create a test class for the same:

@SpringBootTest
class BookServiceMockAndInjectMocksUnitTest {
    @Mock
    private DatabaseService databaseService;
    @Mock
    private ObjectMapper objectMapper;
    @InjectMocks
    private BookService bookService;
    @Test
    void givenBookService_whenGettingBook_thenBookIsCorrect() throws JsonProcessingException {
        Book book1 = new Book("1234", "Inferno", "Dan Brown");
        when(databaseService.findById(eq("1234"))).thenReturn(book1);
        when(objectMapper.writeValueAsString(any())).thenReturn(new ObjectMapper().writeValueAsString(book1));
        String bookString1 = bookService.getBook("1234");
        Assertions.assertTrue(bookString1.contains("Dan Brown"));
    }
}

First, we annotate the test class with @SpringBootTest. This signifies that the application context will be loaded before running the test. This is required when replacing Spring bean dependencies using @InjectMocks.

Next, we declare the DatabaseService and ObjectMapper fields and annotate them with @Mock. This creates mocked objects for both of them. We add the @InjectMocks annotation when declaring our BookService instance that we’ll test. This injects any dependencies that the service requires and have been declared earlier with @Mocks.

Finally, in our test, we mock the behavior of our mocked objects and test the getBook() method of our service.

It’s mandatory to mock all the dependencies of the service when using this method. For example, if we don’t mock ObjectMapper, it leads to a NullPointerException when it’s called in the tested method.

4.2. Using @Autowired With @MockBean

In the above method, we mocked both the dependencies. However, it may be required to mock some of the dependencies and not mock the others. Let’s assume we don’t need to mock the behavior of ObjectMapper and mock only DatabaseService.

Since we’re loading the Spring context in our test, we can use the combination of @Autowired and @MockBean annotations to do so:

@MockBean
private DatabaseService databaseService;
@Autowired
private BookService bookService;
@Test
void givenBookService_whenGettingBook_thenBookIsCorrect() throws JsonProcessingException {
    Book book1 = new Book("1234", "Inferno", "Dan Brown");
    when(databaseService.findById(eq("1234"))).thenReturn(book1);
    String bookString1 = bookService.getBook("1234");
    Assertions.assertTrue(bookString1.contains("Dan Brown"));
}

We annotate DatabaseService with @MockBean. Then we get the BookService instance from the application context using @Autowired.

When the BookService bean is injected, the actual DatabaseService bean will be replaced by the mocked bean. Conversely, the ObjectMapper bean remains the same as originally created by the application.

When we test this instance now, we don’t need to mock any behavior for ObjectMapper.

This method is useful when we need to test the behavior of nested beans and don’t want to mock every dependency.

4.3. Using @Autowired and @InjectMocks Together

We could also use @InjectMocks instead of @MockBean for the above use case.

Let’s look at the code to see the difference between the two methods:

@Mock
private DatabaseService databaseService;
@Autowired
@InjectMocks
private BookService bookService;
@Test
void givenBookService_whenGettingBook_thenBookIsCorrect() throws JsonProcessingException {
    Book book1 = new Book("1234", "Inferno", "Dan Brown");
    MockitoAnnotations.openMocks(this);
    when(databaseService.findById(eq("1234"))).thenReturn(book1);
    String bookString1 = bookService.getBook("1234");
    Assertions.assertTrue(bookString1.contains("Dan Brown"));
}

Here, we mock the DatabaseService using @Mock instead of @MockBean. In addition to @Autowired, we add the @InjectMocks annotation to the BookService instance.

When both annotations are used together, @InjectMocks doesn’t inject the mocked dependency automatically and the auto-wired BookService object is injected when the test starts.

However, we can inject the mocked instance of DatabaseService later in our test by calling the MockitoAnnotations.openMocks() method. This method looks for fields marked with @InjectMocks and injects mocked objects into it.

We call it in our test just before we need to mock the behavior of DatabaseService. This method is useful when we want to dynamically decide when to use mocks and when to use the actual bean for the dependency.

5. Comparison of Approaches

Now that we’ve looked at multiple approaches, let’s summarise a comparison between them:

6. Conclusion

In this article, we looked at different use cases of Mockito and Spring Boot annotations – @Mock, @InjectMocks, @Autowired and @MockBean. We explored when to use different combinations of the annotations as per our testing needs.

As usual, the code examples for this tutorial are available over on GitHub.

       

1. Overview

In the Reactor library, the Flux.map() and Flux.doOnNext() operators play different roles in working with stream data elements.

The Flux.map() operator helps to transform each element emitted by the Flux. The Flux.doOnNext() operator is a lifecycle hook that allows us to perform side effects on each element as it’s emitted.

In this tutorial, we’ll dive deep into the details of these operators, exploring their internal implementations and practical use cases. Also, we’ll see how to use the two operators together.

2. Maven Dependencies

To use the Flux publisher and other reactive operators, let’s add the reactor-core dependency to the pom.xml:

<dependency>
    <groupId>io.projectreactor</groupId>
    <artifactId>reactor-core</artifactId>
    <version>3.6.5</version>
</dependency>

This dependency provides the core classes like Flux, Mono, etc.

Also, let’s add the reactor-test dependency to help with our unit tests:

<dependency>
    <groupId>io.projectreactor</groupId>
    <artifactId>reactor-test</artifactId>
    <version>3.6.5</version>
    <scope>test</scope>
</dependency>

The dependency above provides classes like StepVerifier which allow us to create test scenarios and assert the expected behavior of a reactive pipeline.

3. Understanding Flux.map() Operator

The Flux.map() operator works similarly to Java’s built-in Stream.map(), but it operates on reactive streams.

3.1. The Marble Diagram

Let’s understand the internals of Flux.map() operator through a marble diagram:

In the diagram above, we have a Flux publisher that emits a stream of data without any errors. Also, it shows the effect of the map() operator on the emitted data. The operator transforms the data from circles to squares and returns the transformed data. Upon subscription, the transform data will be emitted, and not the original data.

3.2. The Method Definition

The Flux.map() operator takes a Function as an argument and returns a new Flux with the transformed elements.

Here’s the method signature:

public final <V> Flux<V> map(Function<? super T,? extends V> mapper)

In this case, the input is the data stream from the Flux publisher.  The mapper function is applied synchronously to each element emitted by the Flux. The output is a new Flux containing the transformed elements based on the provided mapper function.

3.3. Example Code

Let’s transform some data into a new sequence by multiplying each value by 10:

Flux<Integer> numbersFlux = Flux.just(50, 51, 52, 53, 54, 55, 56, 57, 58, 59)
  .map(i -> i * 10)
  .onErrorResume(Flux::error);

Then, let’s assert that the emitted new sequence of data is equal to the expected numbers:

StepVerifier.create(numbersFlux)
  .expectNext(500, 510, 520, 530, 540, 550, 560, 570, 580, 590)
  .verifyComplete();

The map() operator acted on the data as described by the marble diagram and the function definition, producing a new output with each value multiplied by 10.

4. Understanding doOnNext() Operator

The Flux.doOnNext() operator is a lifecycle hook that helps to peek into an emitted stream of data. It’s similar to Stream.peek(). It provides a way to perform side effects on each element as emitted without altering the original stream of data.

4.1. The Marble Diagram

Let’s understand the internals of Flux.doOnNext() method through a marble diagram:

The diagram above shows the emitted stream of data from a Flux and the action of the doOnNext() operator on that data.

4.2. The Method Definition

Let’s look at the method definition of the doOnNext() operator:

public final Flux<T> doOnNext(Consumer<? super T> onNext)

The method accepts a Consumer<T> as an argument. The Consumer is a functional interface that represents a side-effect operation. It consumes the input but doesn’t produce any output, making it suitable for performing side effects operations.

4.3. Example Code

Let’s apply the doOnNext() operator to log items in a data stream to the console on subscription:

Flux<Integer> numberFlux = Flux.just(1, 2, 3, 4, 5)
  .doOnNext(number -> {
      LOGGER.info(String.valueOf(number));
  })
  .onErrorResume(Flux::error);

In the code above, the doOnNext() operator logs each number as it’s emitted by the Flux, without modifying the actual number.

5. Using Both Operators Together

Since Flux.map() and Flux.doOnNext() serve different purposes, they can be combined in a reactive pipeline to achieve data transformation and side effects.

Let’s peek into items of an emitted data stream by logging the items to the console and transforming the original data into a new one:

Flux numbersFlux = Flux.just(10, 11, 12, 13, 14)
  .doOnNext(number -> {
      LOGGER.info("Number: " + number);
  })
  .map(i -> i * 5)
  .doOnNext(number -> {
      LOGGER.info("Transformed Number: " + number);
  })
  .onErrorResume(Flux::error);

In the code above, we first use the doOnNext() operator to log each original number emitted by the Flux. Next, we apply the map() operator to transform each number by multiplying it by 5. Then, we use another doOnNext() operator to log the transformed numbers.

Finally, let’s assert that the emitted data is the expected data:

StepVerifier.create(numbersFlux)
  .expectNext(50, 55, 60, 65, 70)
  .verifyComplete();

This combined usage helps us to transform the data stream while also providing visibility into the original and transformed elements through logging.

6. Key Differences

As we know, the two operators act on emitted data. However, the Flux.map() operator is a transformative operator that alters the original emitted stream of data by applying a provided function to each element. This operator is useful in cases where we want to perform computations, data conversions, or manipulations on the elements of the stream.

On the other hand, the Flux.doOnNext() operator is a lifecycle hook that allows us to inspect and perform operations on each emitted element. It cannot modify the data. This operator is useful in the case of logging, debugging, etc.

7. Conclusion

In this article, we look into the details of the Flux.map() and Flux.doOnNext() operators in the Project Reactor library. We delved into their internal workings by examining marble diagrams, type definitions, and practical examples.

The two operators serve different use cases and can be used together to build powerful and robust reactive systems.

As always, the full source code for the examples is available over on GitHub.

       

1. Introduction

In the modern business and document management workflow, the ability to merge multiple PDF files into a single PDF document is a common requirement. Common use cases include presentations, consolidating reports, or compiling packages into a single package.

In Java, multiple libraries exist that provide out-of-the-box features to handle PDFs and merge them into a single PDF. Apache PDFBox and iText are among the popular ones.

In this tutorial, we’ll implement the PDF merge functionality using Apache PDFBox and iText.

2. Setup

Before diving into the implementation, let’s go through the necessary setup steps. We’ll add the required dependencies for the project, additionally, we’ll create helper methods for our tests.

2.1. Dependencies

We’ll use Apache PDFBox and iText for merging the PDF files. To use Apache PDFBox, We  need to add the below dependency in the pom.xml file:

<dependency> 
    <groupId>org.apache.pdfbox</groupId>
    <artifactId>pdfbox</artifactId> 
    <version>2.0.31</version> 
</dependency>

To use the iText, we need to add the below dependency in the pom.xml file:

<dependency>
    <groupId>com.itextpdf</groupId>
    <artifactId>itextpdf</artifactId>
    <version>5.5.13.3</version>
</dependency>

2.2. Test Setup

Let’s create a sample PDF file that we’ll use to test our logic. We can create a utility method to create PDF so that we can use it across different tests:

static void createPDFDoc(String content, String filePath) throws IOException {
    PDDocument document = new PDDocument();
    for (int i = 0; i < 3; i++) {
        PDPage page = new PDPage();
        document.addPage(page);
        try (PDPageContentStream contentStream = new PDPageContentStream(document, page)) {
            contentStream.beginText();
            contentStream.setFont(PDType1Font.HELVETICA_BOLD, 14);
            contentStream.showText(content + ", page:" + i);
            contentStream.endText();
        }
    }
    document.save("src/test/resources/temp/" + filePath);
    document.close();
}

In the above logic, we create a PDF document and add three pages to it using a custom font. Now that we have the createPDFDoc() method, let’s call it before each test and delete the file after finishing the test :

@BeforeEach
public void create() throws IOException {
    File tempDirectory = new File("src/test/resources/temp");
    tempDirectory.mkdirs();
    List.of(List.of("hello_world1", "file1.pdf"), List.of("hello_world2", "file2.pdf"))
        .forEach(pair -> {
            try {
                createPDFDoc(pair.get(0), pair.get(1));
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        });
}

@AfterEach
public void destroy() throws IOException {
    Stream<Path> paths = Files.walk(Paths.get("src/test/resources/temp/"));
    paths.sorted((p1, p2) -> -p1.compareTo(p2))
         .forEach(path -> {
            try {
                Files.delete(path);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        });
}

3. Using Apache PDFBox

Apache PDFBox is an open-source Java library for working with PDF documents. It provides a range of functionalities to create, manipulate, and extract content from PDF files programmatically.

PDFBox provides a PDFMergerUtility helper class to merge multiple PDF documents. We can use the addSource() method to add PDF files. The mergeDocuments() method merges all the added sources, which results in the final merged PDF document:

void mergeUsingPDFBox(List<String> pdfFiles, String outputFile) throws IOException {
    PDFMergerUtility pdfMergerUtility = new PDFMergerUtility();
    pdfMergerUtility.setDestinationFileName(outputFile);
    pdfFiles.forEach(file -> {
        try {
            pdfMergerUtility.addSource(new File(file));
        } catch (FileNotFoundException e) {
            throw new RuntimeException(e);
        }
    });
    pdfMergerUtility.mergeDocuments(MemoryUsageSetting.setupMainMemoryOnly());
}

As demonstrated above, the mergeDocuments() method takes an argument to configure memory usage when merging documents. We’re defining exclusively to use only main memory, i.e., the RAM, for buffering during the merging of documents. We can choose from plenty of other options for buffering memory, including disk, a combination of RAM and disk, and so on.

We can write a unit test to verify if the merge logic works as expected:

@Test
void givenMultiplePdfs_whenMergeUsingPDFBoxExecuted_thenPdfsMerged() throws IOException {
    List<String> files = List.of("src/test/resources/temp/file1.pdf", "src/test/resources/temp/file2.pdf");
    PDFMerge pdfMerge = new PDFMerge();
    pdfMerge.mergeUsingPDFBox(files, "src/test/resources/temp/output.pdf");
    try (PDDocument document = PDDocument.load(new File("src/test/resources/temp/output.pdf"))) {
        PDFTextStripper pdfStripper = new PDFTextStripper();
        String actual = pdfStripper.getText(document);
        String expected = """
            hello_world1, page:0
            hello_world1, page:1
            hello_world1, page:2
            hello_world2, page:0
            hello_world2, page:1
            hello_world2, page:2
            """;
        assertEquals(expected, actual);
    }
}

In the test above, we merged two PDF files using PDFBox into an output file and validated the merged content.

4. Using iText

iText is another popular Java library for creating and manipulating PDF documents. It provides a wide range of features, such as generating PDF files while including text, images, tables, and other elements such as hyperlinks and form fields.

iText provides the PdfReader and PdfWriter classes that are very handy in reading input files and writing them to the output files:

void mergeUsingIText(List<String> pdfFiles, String outputFile) throws IOException, DocumentException {
    List<PdfReader> pdfReaders = List.of(new PdfReader(pdfFiles.get(0)), new PdfReader(pdfFiles.get(1)));
    Document document = new Document();
    FileOutputStream fos = new FileOutputStream(outputFile);
    PdfWriter writer = PdfWriter.getInstance(document, fos);
    document.open();
    PdfContentByte directContent = writer.getDirectContent();
    PdfImportedPage pdfImportedPage;
    for (PdfReader pdfReader : pdfReaders) {
        int currentPdfReaderPage = 1;
        while (currentPdfReaderPage <= pdfReader.getNumberOfPages()) {
            document.newPage();
            pdfImportedPage = writer.getImportedPage(pdfReader, currentPdfReaderPage);
            directContent.addTemplate(pdfImportedPage, 0, 0);
            currentPdfReaderPage++;
        }
    }
    fos.flush();
    document.close();
    fos.close();
}

In the above logic, we read and then import the pages of PdfReader to PdfWrite using the getImportedPage() method and then add that to the directContent object, which essentially stores the read buffer of contents. Once we finish the reading, we flush the output stream fos, which writes to the output file.

We can verify our logic by writing a unit test:

@Test
void givenMultiplePdfs_whenMergeUsingITextExecuted_thenPdfsMerged() throws IOException, DocumentException {
    List<String> files = List.of("src/test/resources/temp/file1.pdf", "src/test/resources/temp/file2.pdf");
    PDFMerge pdfMerge = new PDFMerge();
    pdfMerge.mergeUsingIText(files, "src/test/resources/temp/output1.pdf");
    try (PDDocument document = PDDocument.load(new File("src/test/resources/temp/output1.pdf"))) {
        PDFTextStripper pdfStripper = new PDFTextStripper();
        String actual = pdfStripper.getText(document);
        String expected = """
            hello_world1, page:0
            hello_world1, page:1
            hello_world1, page:2
            hello_world2, page:0
            hello_world2, page:1
            hello_world2, page:2
            """;
        assertEquals(expected, actual);
    }
}

Our test is almost the same as in the previous section. The only difference is we’re calling the mergeUsingIText() method, which uses iText for merging the PDF files.

5. Conclusion

In this article, we explored how we can merge PDF files using Apache PDFBox and iText. Both libraries are feature-rich and allow us to handle different types of content inside PDF files. We implemented merge functionality and also wrote tests to verify the results.

As usual, the complete source code for the examples is available over on GitHub.

       

1. Overview

When working with dates and times in Java, we often encounter different formats, such as LocalDateTime and Instant. While LocalDateTime represents a date-time without a time zone, Instant represents a specific point in time, usually with reference to the epoch (January 1, 1970, 00:00:00 UTC). In many scenarios, we need to map between these two types. Fortunately, MapStruct, a powerful Java mapping framework, allows us to do this easily.

In this tutorial, we’ll learn how to map LocalDateTime to Instant in MapStruct.

2. Understanding LocalDateTime and Instant

There are several reasons why we might need to map LocalDateTime to Instant.

LocalDateTime is useful for representing events that occur at a specific local time, without considering time zones. We commonly use it to store timestamps in databases and log files. LocalDateTime is a good choice for applications where all users operate in the same time zone.

Instant is great for tracking global events, ensuring time zone consistency and providing a reliable format for interacting with external systems or APIs. In addition, it’s also useful for storing timestamps in a database with a need for time zone consistency.

We’ll deal with LocalDateTime and Instant values and convert them frequently.

3. Mapping Scenario

Let’s suppose that we’re implementing an order processing service. We have two flavours of orders – order and local order. Order uses Instant to support global order processing whereas the local order uses LocalDateTime to represent the local time.

Here is the implementation of the order model:

public class Order {
    private Long id;
    private Instant created;
    // other fields
    // getters and setters
}

Then, we have the implementation of the local order:

public class LocalOrder {
    private Long id;
    private LocalDateTime created;
    // other fields
    // getters and setters
}

4. Mapping LocalDateTime to Instant

Let’s now learn how to implement the mapper to convert LocalDateTime to Instant.

Let’s start with the OrderMapper interface: :

@Mapper
public interface OrderMapper {
    OrderMapper INSTANCE = Mappers.getMapper(OrderMapper.class);
 
    ZoneOffset DEFAULT_ZONE = ZoneOffset.UTC;
    @Named("localDateTimeToInstant")
    default Instant localDateTimeToInstant(LocalDateTime localDateTime) {
        return localDateTime.toInstant(DEFAULT_ZONE);
    }
    @Mapping(target = "id", source = "id")
    @Mapping(target = "created", source = "created", qualifiedByName = "localDateTimeToInstant")
    Order toOrder(LocalOrder source);
}

This OrderMapper interface is a MapStruct mapper that converts a LocalOrder object to an Order object while handling a custom conversion for date-time fields. It declares a constant DEFAULT_ZONE with a value of ZoneOffset.UTC, which represents the UTC timezone.

The localDateTimeToInstant() method, annotated with @Named, converts a LocalDateTime into an Instant using the specified ZoneOffset.

The toOrder() method maps LocalOrder to Order. It uses @Mapping to define how fields are mapped. The id field is directly mapped from source to target. For the created field, it applies the custom localDateTimeToInstant() method by specifying qualifiedByName = “localDateTimeToInstant”. This ensures that the LocalDateTime in LocalOrder is properly converted to Instant in Order.

MapStruct uses conventions for mapping data types. Mapping complex objects with nested properties or converting between certain data types can raise errors. Default methods in a MapStruct interface can define explicit conversions between types that MapStruct doesn’t natively support. These methods resolve ambiguities and provide necessary instructions for conversion. That ensures accurate and reliable mappings. In addition, they’re also useful for complex or nested property mapping. In conclusion, they’re a best practice for maintaining clean, maintainable code in MapStruct mappings.

Let’s test our mapper:

class OrderMapperUnitTest {
    private OrderMapper mapper = OrderMapper.INSTANCE;
    
    @Test
    void whenLocalOrderIsMapped_thenGetsOrder() {
        LocalDateTime localDateTime = LocalDateTime.now();
        long sourceEpochSecond = localDateTime.toEpochSecond(OrderMapper.DEFAULT_ZONE);
        LocalOrder localOrder = new LocalOrder();
        localOrder.setCreated(localDateTime);
      
        Order target = mapper.toOrder(localOrder);
      
        Assertions.assertNotNull(target);
        long targetEpochSecond = target.getCreated().getEpochSecond();
        Assertions.assertEquals(sourceEpochSecond, targetEpochSecond);
    }
}

This test case checks if OrderMapper correctly converts a LocalOrder to an Order, focusing on the created field mapping from LocalDateTime to Instant. It creates a LocalDateTime, calculates its epoch second value, sets it to a new LocalOrder, maps it to an Order, and checks if the result isn’t null. Finally, it compares the epoch second values between the LocalDateTime of LocalOrder and the Instant of Order. If they match, the test passes.

5. Mapping Instant to LocalDateTime

Now we’ll see how to map the Instant back to LocalDateTime:

@Mapper
public interface OrderMapper {
    OrderMapper INSTANCE = Mappers.getMapper(OrderMapper.class);
    ZoneOffset DEFAULT_ZONE = ZoneOffset.UTC;
    @Named("instantToLocalDateTime")
    default LocalDateTime instantToLocalDateTime(Instant instant) {
        return LocalDateTime.ofInstant(instant, DEFAULT_ZONE);
    }
  
    @Mapping(target = "id", source = "id")
    @Mapping(target = "created", source = "created", qualifiedByName = "instantToLocalDateTime")
    LocalOrder toLocalOrder(Order source);
}

The OrderMapper now defines a mapping that converts Order objects to LocalOrder objects. It includes a custom mapping method, instantToLocalDateTime(), which converts Instant to LocalDateTime using a predefined ZoneOffset (UTC).

The @Mapping annotations in toLocalOrder() indicate that the id field is directly mapped from Order to LocalOrder. Then it uses the custom method (qualifiedByName = “instantToLocalDateTime”) for the created field and converts Instant to LocalDateTime.

Let’s verify our mapping:

@Test
void whenOrderIsMapped_thenGetsLocalOrder() {
    Instant source = Instant.now();
    long sourceEpochSecond = source.getEpochSecond();
    Order order = new Order();
    order.setCreated(source);
    
    LocalOrder target = mapper.toLocalOrder(order);
    
    Assertions.assertNotNull(target);
    long targetEpochSecond = target.getCreated().toEpochSecond(OrderMapper.DEFAULT_ZONE);
    Assertions.assertEquals(sourceEpochSecond, targetEpochSecond);
}

This test verifies that the OrderMapper correctly converts an Order object to a LocalOrder object, focusing on mapping Instant to LocalDateTime.

The test creates an Instant object with the current timestamp and calculates its epoch second. It then creates an Order object and sets the Instant value to its created field.

The test maps the Order object to LocalOrder using mapper.toLocalOrder(). It checks that the resulting LocalOrder isn’t null and verifies that the epoch second of the LocalDateTime in LocalOrder matches the epoch second of the Instant in Order, ensuring correct mapping with the specified ZoneOffset.

6. Conclusion

In this article, we learned how to map LocalDateTime to Instant and vice versa using MapStruct. We saw how to create custom mapping methods with @Named to convert between these types, along with the correct use of @Mapping and qualifiedByName. This approach ensures smooth data transformation and time zone consistency in Java applications.

As always, the complete code samples for this article can be found over on GitHub.

       

1. Overview

When using Spring Data JPA, we often leverage derived and custom queries that return the result in our preferred formats. A typical example is the DTO projection, which offers a great way to select only some specific columns and reduce the overhead of selecting unnecessary data.

However, the DTO projection isn’t always easy and may lead to ConverterNotFoundException when it’s not implemented properly. So, in this short tutorial, we’ll elucidate how to avoid the ConverterNotFoundException exception when working with Spring Data JPA.

2. Understanding the Exception in Practice

Before jumping to the solution, let’s try to understand what the exception and its stack trace mean through a practical example.

To keep things simple, we’ll use the H2 database. Let’s start by adding its dependency to the pom.xml file:

<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <version>2.2.224</version>
</dependency>

2.1. H2 Configuration

Spring Boot provides intrinsic support for the H2 embeddable database. By design, it configures the application to connect to H2 using the username sa and an empty password.

First, let’s add the database connection credentials to the application.properties file:

spring.datasource.url=jdbc:h2:mem:mydb
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=

That’s all we need to set up the H2 configuration with Spring Boot.

2.2. Entity Class

Now, let’s define a JPA entity. For instance, we’ll consider the Employee class:

@Entity
public class Employee {
    @Id
    private int id;
    @Column
    private String firstName;
    @Column
    private String lastName;
    @Column
    private double salary;
    // standards getters and setters
}

In this example, we define an employee by their identifier, first name, last name, and salary.

Typically, we use the @Entity annotation to denote that the Employee class is a JPA entity. Moreover, @Id marks the field that represents the primary key. Furthermore, we use @Column to bind each entity field to its respective table column.

2.3. JPA Repository

Next, we’re going to create a Spring Data JPA repository to handle the logic of storing and retrieving employees:

@Repository
public interface EmployeeRepository extends JpaRepository<Employee, Integer> {
}

Here, we’ll assume that we need to display the full name of an employee. So, we’ll rely on DTO projection to select only firstName and lastName.

Since the Employee class holds additional data, let’s create a new class named EmployeeFullName that contains only the first and the last names:

public class EmployeeFullName {
    private String firstName;
    private String lastName;
    // standards getters and setters
    public String fullName() {
        return getFirstName()
          .concat(" ")
          .concat(getLastName());
    }
}

Notably, we create a custom method fullName() to display the employee’s full name. Now, let’s add a derived query that returns the full name of an employee to EmployeeRepository:

EmployeeFullName findEmployeeFullNameById(int id);

Lastly, let’s create a test to make sure that everything works as expected:

@Test
void givenEmployee_whenGettingFullName_thenThrowException() {
    Employee emp = new Employee();
    emp.setId(1);
    emp.setFirstName("Adrien");
    emp.setLastName("Juguet");
    emp.setSalary(4000);
    employeeRepository.save(emp);
    assertThatThrownBy(() -> employeeRepository
      .findEmployeeFullNameById(1))
      .isInstanceOfAny(ConverterNotFoundException.class)
      .hasMessageContaining("No converter found capable of converting from type" 
        + "[com.baeldung.spring.data.noconverterfound.models.Employe");
}

As shown above, the test fails with ConverterNotFoundException.

The root cause of the exception is that JpaRepository expects that its derived queries return an instance of the Employee entity class. Since the method returns an EmployeeFullName object, Spring Data JPA fails to find a converter suitable to convert the expected Employee object to the new EmployeeFullName object.

3. Solutions

When using a class to implement the DTO projection, Spring Data JPA uses, by default, the constructor to determine the fields that are supposed to be retrieved. So, the basic solution here is to add a parameterized constructor to the EmployeeFullName class:

public EmployeeFullName(String firstName, String lastName) {
    this.firstName = firstName;
    this.lastName = lastName;
}

That way, we tell Spring Data JPA to select only firstName and lastName. Now, let’s add another test to test the solution:

@Test
void givenEmployee_whenGettingFullNameUsingClass_thenReturnFullName() {
    Employee emp = new Employee();
    emp.setId(2);
    emp.setFirstName("Azhrioun");
    emp.setLastName("Abderrahim");
    emp.setSalary(3500);
    employeeRepository.save(emp);
    assertThat(employeeRepository.findEmployeeFullNameById(2).fullName())
      .isEqualTo("Azhrioun Abderrahim");
}

Unsurprisingly, the test passes with success.

Another solution would be to use the interface-based projection. That way, we don’t have to worry about the constructor. So, instead of using a class, we can use an interface that exposes getters for the fields to be read:

public interface IEmployeeFullName {
    String getFirstName();
    String getLastName();
    default String fullName() {
        return getFirstName().concat(" ")
          .concat(getLastName());
    }
}

Here, we used a default method to display the full name. Next, let’s create another derived query that returns an instance of type IEmployeeFullName:

IEmployeeFullName findIEmployeeFullNameById(int id);

Finally, let’s add another test to verify this second solution:

@Test
void givenEmployee_whenGettingFullNameUsingInterface_thenReturnFullName() {
    Employee emp = new Employee();
    emp.setId(3);
    emp.setFirstName("Eva");
    emp.setLastName("Smith");
    emp.setSalary(6500);
    employeeRepository.save(emp);
    assertThat(employeeRepository.findIEmployeeFullNameById(3).fullName())
      .isEqualTo("Eva Smith");
}

As expected, the interface-based solution works.

4. Conclusion

In this article, we learned what causes Spring Data JPA to fail with ConverterNotFoundException. Along the way, we saw how to reproduce and fix the exception in practice.

As always, the full source code of the examples is available over on GitHub.

       

1. Overview

The Java Stream API introduced many features that significantly enhance the functionality and readability of our code. Among these, the map() method stands out as a powerful tool for transforming elements within a collection. A common requirement is ensuring that the results of these transformations don’t include null elements.

In this tutorial, we’ll explore how to effectively collect non-null elements from Stream‘s map() method.

2. Introduction to the Problem

The map() method provides a high-level abstraction for working with sequences of elements. It’s an intermediate operation that applies a mapper function to each element of the Stream, producing a new Stream of transformed elements.

Sometimes, the mapper function can return null. However, we want to exclude those null values from the transformed results. For example, let’s say we have a List of String values:

static final List<String> INPUT = List.of("f o o", "a,b,c,d", "b a r", "w,x,y,z", "w,o,w");

We’d like to transform the String elements in INPUT using the following mapper function:

String commaToSpace(String input) {
    if (input.contains(",")) {
        return input.replaceAll(",", " ");
    } else {
        return null;
    }
}

As we can see, the commaToSpace() method simply replaces all commas with spaces in the input String and returns the result. However, the method returns null if input doesn’t contain a comma.

Now, we want to use commaToSpace() to transform our INPUT and ensure that null values aren’t included in the result. So, here’s our expected outcome:

static final List<String> EXPECTED = List.of("a b c d", "w x y z", "w o w");

As we can see, the INPUT List has five elements, but the EXPECTED List has three.

It’s worth mentioning that, in practice, we might take a more straightforward approach to this task. For example, we could first filter out String elements that don’t contain any commas and then perform the comma-to-space substitutions. However, since we want to demonstrate how to collect non-null elements from a Stream‘s map() method call, we’ll use the commaToSpace() method as the mapper function and invoke it with Stream.map().

Next, let’s see how to achieve it using Stream API and the map() method.

3. Using the map() + filter() Approach

We’ve mentioned the map() method applies a mapper function, which in this case is commaToSpace(), to each element of the Stream to complete the transformation.

The mapper function takes one input and produces one transformed output, and the map() method doesn’t perform any filtering. Therefore, the Stream that map() produces is always the same size as the original Stream. In other words, if the mapper function returns null, those null values are in the transformed Stream. However, we can use the filter() method in conjunction with the map() method to remove null elements from the resulting Stream.

Next, let’s see how this is done through a test:

List<String> result = INPUT.stream()
  .map(str -> commaToSpace(str))
  .filter(Objects::nonNull)
  .collect(Collectors.toList());
 
assertEquals(EXPECTED, result);

In the above code, we use the filter() method with the Objects::nonNull method reference to remove all null elements from the resulting Stream.

4. How About Using Optional to Handle null Values?

When it comes to handling null values, some may consider leveraging the Optional class, which is designed to handle optional values without explicitly using null:

List<String> result = INPUT.stream()
  .map(str -> Optional.ofNullable(commaToSpace(str)))
  .filter(Optional::isPresent)
  .map(Optional::get)
  .collect(Collectors.toList());
 
assertEquals(EXPECTED, result);

As the above example shows, we first wrap nullable values in Optional objects, resulting in a Stream<Optional<String>>. Then, we remove all absent Optionals from the Stream using filter(). Finally, to obtain the String values held by Stream<Optional<String>>, we need an extra step to extract the value with map(Optional::get).

Therefore, as we can see, the Optional approach isn’t efficient for this problem due to the unnecessary wrapping and unwrapping of elements in the Streams.

5. What if the Mapper Function Returns Optional?

We’ve discussed that using Optional to handle null elements is inefficient for this problem. However, there are cases where the mapper function returns an Optional object instead of a nullable result, for instance:

Optional<String> commaToSpaceOptional(String input) {
    if (input.contains(",")) {
        return Optional.of(input.replaceAll(",", " "));
    } else {
        return Optional.empty();
    }
}

In such situations, we can use Optional.orElse(null) to extract the element value from the Optional returned by the mapper function. This allows us to convert absent Optionals to null elements within the map() method:

List<String> result = INPUT.stream()
  .map(str -> commaToSpaceOptional(str).orElse(null))
  .filter(Objects::nonNull)
  .collect(Collectors.toList());
 
assertEquals(EXPECTED, result);

As the code shows, the map() method performs two tasks:

• transform the Stream using the mapper function
• unwrap each transformed Optional object

The rest of the steps are the same as in the “map() + filter()” approach.

6. Conclusion

In this article, we’ve explored how to collect non-null elements from Stream‘s map() effectively. Additionally, we’ve discussed why wrapping the mapper function’s results in Optionals can lead to inefficiencies.

As always, the complete source code for the examples is available over on GitHub.

       

1. Introduction

When working with Java, enums serve as a convenient way to define a fixed set of constants. However, creating enums within inner classes can introduce some complexities and considerations in Java versions before Java 16.

In this tutorial, we’ll delve into historical constraints on static types within inner classes before Java 16 and then discuss the significant relaxation of these rules in Java 16 and later versions.

2. Historical Constraints on Static Types in Inner Classes

Before Java 16, the Java Language Specification (JLS) strictly enforced rules regarding static types within inner classes:

• Nested enum types were implicitly static, as stated in JLS §8.9
• It was prohibited to declare a static nested type (including enums) within a non-static nested type (inner class), as outlined in JLS §8.1.3

Let’s illustrate this constraint with an example in Java versions before Java 16:

public class Outer {
    public class Inner {
        public static enum MyEnum {
            VALUE1, VALUE2, VALUE3
        }
    }
    public static void main(String[] args) {
        Outer outer = new Outer();
        Outer.Inner.MyEnum value = Outer.Inner.MyEnum.VALUE1;
    }
}

Attempting to define the MyEnum enum as a static member within the non-static inner class results in a compilation error in Java versions before Java 16.

3. Relaxation of Rules in Java 16 and Later Versions

Java 16 introduced a significant change with the implementation of JEP 395, which relaxed certain rules concerning static members within inner classes.

This relaxation was reflected in the updated phrasing of §8.1.3 in the JLS, explicitly allowing inner classes to declare and inherit static members, including static enums, despite the inner class itself not being static.

Accordingly, the previous code snippet will run successfully, and we can access enum constants (VALUE1, VALUE2, and VALUE3) using the fully qualified name Outer.Inner.MyEnum.VALUE1.

4. Conclusion

In conclusion, the relaxation of rules in Java 16 and later versions regarding static types within inner classes, including enums, represents a significant evolution in Java’s language features. Moreover, this change enables developers to adopt more flexible and expressive coding patterns, enhancing the encapsulation and organization of code within inner classes.

As always, we can find the complete code samples for this article over on GitHub.

       

1. Introduction

A fraction is another way of representing a number, consisting of a numerator and a denominator. For example, the fraction 3/5 can be thought of as “3 out of 5,” representing the same value as the decimal number 0.6. In this tutorial, we’ll explore different approaches to convert decimal numbers to fractions in Java.

2. Using Multiplying With a Power of 10

One simple way to convert a decimal to a fraction is to multiply the decimal by a power of 10 and then use the resulting numerator and denominator as the fraction.

Here is the code snippet for this approach:

String convertDecimalToFractionUsingMultiplyingWithPowerOf10(double decimal) {
    String decimalStr = String.valueOf(decimal);
    int decimalPlaces = decimalStr.length() - decimalStr.indexOf('.') - 1;
    
    long denominator = (long) Math.pow(10, decimalPlaces);
    long numerator = (long) (decimal * denominator);
    return numerator + "/" + denominator;
}

First, we calculate the number of decimal places by finding the position of the decimal point in the string and subtracting it from the length of the string. Then we compute the denominator by raising 10 to the power of the number of decimal places.

Next, we multiply the original decimal number with the denominator to determine the numerator. The method returns a string representation of the fraction by concatenating the numerator and denominator with a slash (/) separator.

Let’s validate our solution using assertEquals():

assertEquals("5/10", convertDecimalToFractionUsingMultiplyingWithPowerOf10(0.5));
assertEquals("1/10", convertDecimalToFractionUsingMultiplyingWithPowerOf10(0.1));
assertEquals("6/10", convertDecimalToFractionUsingMultiplyingWithPowerOf10(0.6));
assertEquals("85/100", convertDecimalToFractionUsingMultiplyingWithPowerOf10(0.85));
assertEquals("125/100", convertDecimalToFractionUsingMultiplyingWithPowerOf10(1.25));
assertEquals("1333333333/1000000000", convertDecimalToFractionUsingMultiplyingWithPowerOf10(1.333333333));

This approach is simple and easy to implement, but it has some limitations. The simple conversion might not produce the most reduced form of the fraction. For instance, converting 0.5 using this method would result in 5/10, which can be simplified to 1/2.

3. Using the Greatest Common Divisor (GCD)

A more robust way to convert a decimal to a fraction is to use the Greatest Common Divisor (GCD) to simplify the fraction. The GCD is the largest number that divides both the numerator and denominator without leaving a remainder.

First, we create a gcd() method to calculate the greatest common divisor of two integers a and b using the Euclidean algorithm:

long gcd(long a, long b) {
    if (b == 0) {
        return a;
    } else {
        return gcd(b, a % b);
    }
}

Inside the method, we check:

• If b is 0, the GCD is a.
• Otherwise, the GCD is calculated recursively by finding the GCD of b and the remainder of a divided by b.

Next, we create a method to apply the gcd() method to simplify the fraction:

String convertDecimalToFractionUsingGCD(double decimal) {
    String decimalStr = String.valueOf(decimal);
    int decimalPlaces = decimalStr.length() - decimalStr.indexOf('.') - 1;
    long denominator = (long) Math.pow(10, decimalPlaces);
    long numerator = (long) (decimal * denominator);
    long gcd = gcd(numerator, denominator);
    numerator /= gcd;
    denominator /= gcd;
    return numerator + "/" + denominator;
}

Similar to the previous approach, we calculate the number of decimal places to determine the denominator. This is followed by multiplying the original decimal number with the denominator to determine the numerator.

Then we apply the gcd() method to calculate the GCD of the numerator and denominator. Afterward, we simplify the fraction by dividing both the numerator and denominator by the gcd. This ensures that the fraction is in its simplest form.

Finally, the method returns the simplified fraction as a string by concatenating the numerator and denominator with a slash (/) separator:

assertEquals("1/2", convertDecimalToFractionUsingGCD(0.5));
assertEquals("1/10", convertDecimalToFractionUsingGCD(0.1));
assertEquals("3/5", convertDecimalToFractionUsingGCD(0.6));
assertEquals("17/20", convertDecimalToFractionUsingGCD(0.85));
assertEquals("5/4", convertDecimalToFractionUsingGCD(1.25));
assertEquals("1333333333/1000000000", convertDecimalToFractionUsingGCD(1.333333333));

While this approach is efficient for finding GCD, it can become computationally expensive for very large numbers. This is because each recursive call involves calculating the modulo operation (%), which can be slow for large inputs.

4. Using Apache Commons Math

Lastly, we can also use a library like Apache Commons Math to convert a decimal to a fraction. In this case, we’re leveraging the Fraction class from Apache Commons Math to convert a decimal to a fraction:

String convertDecimalToFractionUsingApacheCommonsMath(double decimal) {
    Fraction fraction = new Fraction(decimal);
    return fraction.toString();
}

To convert a decimal to a fraction, we create a Fraction object by passing the decimal value to its constructor. Once we have the Fraction object representing the decimal value, we can call its toString() method to get the string representation of the fraction.

The toString() method returns the fraction in the form “numerator / denominator”:

assertEquals("1 / 2", convertDecimalToFractionUsingApacheCommonsMath(0.5));
assertEquals("1 / 10", convertDecimalToFractionUsingApacheCommonsMath(0.1));
assertEquals("3 / 5", convertDecimalToFractionUsingApacheCommonsMath(0.6));
assertEquals("17 / 20", convertDecimalToFractionUsingApacheCommonsMath(0.85));
assertEquals("5 / 4", convertDecimalToFractionUsingApacheCommonsMath(1.25));
assertEquals("4 / 3", convertDecimalToFractionUsingApacheCommonsMath(1.333333333));

5. Handle Repeating Decimal

We may observe that the results computed from applying the first two methods and the Apache Commons Math library to the decimal value 1.333333333 differ. This is because they handle repeating decimals differently.

Repeating decimals are decimal numbers that have a sequence of digits that repeat indefinitely after the decimal point. For example, the decimal number 1.333333333 has a repeating sequence of the digit 3 after the decimal point.

To convert a repeating decimal to a fraction, we first identify the repeating sequence of digits that appears indefinitely after the decimal point:

String extractRepeatingDecimal(String fractionalPart) {
    int length = fractionalPart.length();
    for (int i = 1; i <= length / 2; i++) {
        String sub = fractionalPart.substring(0, i);
        boolean repeating = true;
        for (int j = i; j + i <= length; j += i) {
            if (!fractionalPart.substring(j, j + i).equals(sub)) {
                repeating = false;
                break;
            }
        }
        if (repeating) {
            return sub;
        }
    }
    return "";
}

Next, we enhance the convertDecimalToFractionUsingGCD() to handle the repeating decimal:

String convertDecimalToFractionUsingGCDRepeating(double decimal) {
    String decimalStr = String.valueOf(decimal);
    int indexOfDot = decimalStr.indexOf('.');
    String afterDot = decimalStr.substring(indexOfDot + 1);
    String repeatingNumber = extractRepeatingDecimal(afterDot);
    if (repeatingNumber == "") {
        return convertDecimalToFractionUsingGCD(decimal);
    } else {
        //...
    }
}

Once a repeating decimal is detected, we proceed by determining several key attributes:

int n = repeatingNumber.length();
int repeatingValue = Integer.parseInt(repeatingNumber);
int integerPart = Integer.parseInt(decimalStr.substring(0, indexOfDot));
int denominator = (int) Math.pow(10, n) - 1;
int numerator = repeatingValue + (integerPart * denominator);

• n: The length of digits in the repeating sequence.
• repeatingValue: The numerical value of the repeating digits.
• integerPart: The whole number extracted from the decimal before the decimal point
• denominator: The denominator is derived by raising 10 to the power of n and subtracting 1
• numerator: The numerator is calculated by summing the repeatingValue with the multiplication of the integerPart and the denominator.

Next, we can apply the gcd() approach to calculate the GCD for the numerator and denominator:

int gcd = gcd(numerator, denominator);
numerator /= gcd;
denominator /= gcd;
return numerator + "/" + denominator;

Now, let’s verify our solution to handle repeating decimals:

assertEquals("1/2", convertDecimalToFractionUsingGCDRepeating(0.5));
assertEquals("17/20", convertDecimalToFractionUsingGCDRepeating(0.85));
assertEquals("5/4", convertDecimalToFractionUsingGCDRepeating(1.25));
assertEquals("4/3", convertDecimalToFractionUsingGCDRepeating(1.333333333));
assertEquals("7/9", convertDecimalToFractionUsingGCDRepeating(0.777777));

For the test case 1.333333333, we identify that the repeating digit repeatingValue is 3. This means that the digit “3” repeats indefinitely after the decimal point. The length of the repeating sequence is 1, which indicates that the repeating pattern consists of a single-digit repetition.

Next, we determine the denominator by raising 10 to the power of n and subtracting 1, which would be 10^1 – 1 = 9. The numerator is calculated by adding the repeatingValue to the multiplication of the integerPart with the denominator, which would be 3 + (1 * 9) = 12.

Up to this step, the fraction would be 12/9. After we applied the gcd() method to simplify the fraction, we got back the result of 4/3. 

Additionally, it’s important to note that this enhancement may not work as expected for decimals that contain both repeating and non-repeating parts, such as 0.1123123123.

6. Conclusion

In this article, we’ve explored several methods for converting decimal to fractions. For most cases, using the GCD approach provides a good balance between simplicity and ensuring a simplified fraction.

As always, the source code for the examples is available over on GitHub.

       

1. Overview

Sorting a collection that contains null values can lead to a NullPointerException if not handled properly. Java 8 has a convenient method Comparator.nullsLast() to address this issue. This method allows handling null values during sorting operations.

In this tutorial, we’ll learn how to use Comparator.nullsLast() to avoid NullPointerException when sorting in Java.

2. Understanding the Problem

Let’s create a simple scenario where we attempt to sort a list containing null values without proper exception handling:

List<String> strings = new ArrayList<>();
strings.add("BB");
strings.add("AA");
strings.add(null);
strings.add("EE");
strings.add("DD");
Collections.sort(strings);

In the above code, Collections.sort() encounters a null element during sorting. It throws a NullPointerException because null cannot be compared using the natural ordering of String.

Running this code results in the following exception:

Exception in thread "main" java.lang.NullPointerException
    at java.util.ComparableTimSort.countRunAndMakeAscending(ComparableTimSort.java:325)
    at java.util.ComparableTimSort.sort(ComparableTimSort.java:188)
    at java.util.Arrays.sort(Arrays.java:1312)
    at java.util.Arrays.sort(Arrays.java:1506)
    at java.util.ArrayList.sort(ArrayList.java:1464)
    at java.util.Collections.sort(Collections.java:143)
    at Main.main(Main.java:14)

This exception occurs because the default sorting behavior of Collections.sort() assumes that all sorted elements are Comparable, and null isn’t a valid Comparable object.

Now, let’s look at the solution using Comparator.nullsLast() to handle null values gracefully during sorting.

3. Using Comparator.nullsLast()

The Comparator.nullsLast() method is part of the Comparator interface introduced in Java 8. When sorting objects, it returns a comparator that considers null values as greater than non-null values. This is particularly useful when we want to sort a collection of objects based on a field that might be null, ensuring that null values are positioned at the end of the sorted list.

Let’s consider a practical example with a list of String strings that includes null values:

List<String> strings = new ArrayList<>();
strings.add("DD");
strings.add("BB");
strings.add(null);
strings.add("AA");
strings.add("EE");

We want to sort this list alphabetically while ensuring that null values are placed at the end of the sorted list.

So, after creating the list, we create a Comparator<String>, and using Comparator.nullsLast(Comparator.naturalOrder()), the string objects get stored in natural order while treating null values as greater than any non-null values:

Comparator<String> nullsLastComparator = Comparator.nullsLast(Comparator.naturalOrder());

Then, when we apply Collections.sort(), the list gets sorted with null values positioned at the end of the sorted list:

Collections.sort(strings, nullsLastComparator);

As a result, the sorting behavior becomes more predictable when dealing with collections that may contain null values, maintaining a consistent order based on our sorting criteria.

4. Conclusion

In this article, we explored the power of Comparator.nullsLast(). It allows us to sort data safely and predictably, enhancing the robustness and reliability of our sorting operations. Incorporating this method into our Java projects to handle null values effectively helps in maintaining code clarity and simplicity.

The source code of all these examples is available over on GitHub.

       

1. Overview

In this quick tutorial, we’ll learn the differences between CompletableFuture and Mono from Project Reactor in Java. We’ll focus on how they handle asynchronous tasks and the execution that occurs to accomplish those tasks.

Let’s start by looking at CompletableFuture.

2. Understanding CompletableFuture

Introduced in Java 8, CompletableFuture builds upon the previous Future class and provides a way to run code asynchronously. In short, it improves asynchronous programming and simplifies working with threads.

Moreover, we can create a chain of computations with methods like thenApply(), thenAccept(), and thenCompose() to coordinate our asynchronous tasks.

While CompletableFuture is asynchronous, meaning the main thread continues executing other tasks without waiting for the completion of the current operation, it isn’t fully non-blocking. A long-running operation can block the thread executing it:

CompletableFuture completableFuture = CompletableFuture.supplyAsync(() -> {
    try {
        Thread.sleep(1000);
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
    }
    return "Finished completableFuture";
});

Above, we’re simulating a long operation with the sleep() method from the Thread class. If not defined, supplyAsnc() will use ForkJoinPool and assign a thread to run the anonymous lambda function, and this thread gets blocked by the sleep() method.

Moreover, calling the get() method in the CompletableFuture instance before it completes the operation blocks the main thread:

try {
    completableFuture.get(); // This blocks the main thread
} catch (InterruptedException | ExecutionException e) {
    e.printStackTrace();
}

To avoid such occurrences, we can handle the CompletableFuture completion manually using the completeExceptionally() or complete() methods in the callback pattern. For example, suppose we have a function that we want to run without blocking the main thread, and then, we want to handle the future when it fails and when it completes successfully:

public void myAsyncCall(String param, BiConsumer<String, Throwable> callback) {
    new Thread(() -> {
        try {
            Thread.sleep(1000);
            callback.accept("Response from API with param: " + param, null);
        } catch (InterruptedException e) {
            callback.accept(null, e);
        }
    }).start();
}

Then, we use this function to create a CompletableFuture:

public CompletableFuture<String> nonBlockingApiCall(String param) {
    CompletableFuture<String> completableFuture = new CompletableFuture<>();
    myAsyncCall(param, (result, error) -> {
        if (error != null) {
            completableFuture.completeExceptionally(error);
        } else {
            completableFuture.complete(result);
        }
    });
    return completableFuture;
}

There’s an alternative and a more reactive way to handle the same operation, as we’ll see next.

3. Comparing Mono With CompletableFuture

The Mono class from Project Reactor uses reactive principles. Unlike CompletableFuture, Mono is designed to be non-blocking and to support concurrency with less overhead.

Additionally, Mono is lazy compared to the eager execution of the CompletableFuture, meaning that our application won’t consume resources unless we subscribe to Mono:

Mono<String> reactiveMono = Mono.fromCallable(() -> {
    Thread.sleep(1000); // Simulate some computation
    return "Reactive Data";
}).subscribeOn(Schedulers.boundedElastic());
reactiveMono.subscribe(System.out::println);

Above, we’re creating a Mono object using the fromCallable() method and providing the blocking operation as a supplier. Then, we delegate the operation, using the subscribeOn() method, to a separate thread.

Schedulers.boundedElastic() is similar to a cached thread pool but with a limit on the maximum number of threads. This ensures the main thread remains non-blocking. The only way to block the main thread is forcefully call the block() method. This method waits for the completion, successful or not, of the Mono instance.

Then, to run the reactive pipeline, we use subscribe() to subscribe the outcome of the Mono object to println() using method reference.

The Mono class is very flexible and provides a set of operators to transform and combine other Mono objects descriptively. It also supports backpressure to prevent the application from eating up all the resources.

4. Conclusion

In this quick article, we compared CompletableFuture with the Mono class from Project Reactor. First, we described how CompletableFuture can run an asynchronous task. Then, we showed that, if configured incorrectly, it can block the thread it’s working on as well as the main thread. Finally, we showed how to run an asynchronous operation in a reactive way using Mono.

As always, we can find the complete code over on GitHub.

       

1. Introduction

In the realm of software development, ensuring code quality is very important, especially for complex and large codebases. Static code analysis tools like Infer provide us with techniques to detect potential bugs and vulnerabilities in our codebase before they manifest into critical issues.

In this tutorial, we’ll explore the fundamentals of code analysis, explore Infer’s capabilities, and provide practical insights into incorporating it into our development workflow.

2. Static Code Analysis

Static analysis is a debugging method that automatically examines source code without executing the program. This process facilitates the identification of potential defects, security vulnerabilities, and maintainability issues. Typically conducted by third-party tools like the well-known SonarQube, static code analysis is straightforward when automated.

Generally, it occurs in early development phases. Once the code is written, a static code analyzer should be run to look over the code. It will check against defined coding rules from standards or custom predefined rules. Once the code is run through the static code analyzer, the analyzer will have identified whether or not the code complies with the set rules.

In a basic enterprise environment, it’s often part of the Continuous Integration (CI) process. With each commit, a job is triggered to build the application, run tests, and analyze the code, ensuring compliance, safety, and security before deployment.

3. Infer

Infer is a static code analysis tool for Java, C, C++, and Objective-C, written in OCaml. It was first developed at Facebook and open-sourced in 2015. Since then, it has gained popularity beyond Facebook, being adopted by other large companies.

For Java and Android code, it checks for issues like null pointer exceptions, resource leaks, and concurrency race conditions. In C, C++, and Objective-C, it detects problems such as null pointer issues, memory leaks, coding convention violations, and calls to unavailable APIs.

To efficiently handle large codebases, Infer utilizes techniques like separation logic and bi-abduction. Separation logic allows it to analyze small parts of the code storage independently, avoiding the need to process the entire memory space at once. Bi-abduction is a logical inference method that helps Infer discover properties about different parts of the code, allowing it to focus only on the modified sections during subsequent analyses.

By combining these approaches, this tool can find complex problems in modifications to an application built from millions of lines of code in a short time.

3.1. Infer Phases

Infer operates in two main phases regardless of the input language: the capture phase and the analysis phase.

During the capture phase, Infer captures compilation commands to translate the files for analysis into its internal intermediate language. This translation is similar to compilation, so Infer takes information from the compilation process to perform its own translation. Also, this is the reason why we call Infer with a compilation command such as:

infer run -- javac File.java

Therefore, the files get compiled as usual, and Infer also translates them to be analyzed in the second phase. Infer stores the intermediate files in the results directory, which is called infer-out/by default and is created in the folder where the infer command is invoked.

Also, we can invoke only the capture phase using the capture subcommand:

infer capture -- javac File.java

In the analysis phase, files in infer-out/ are analyzed individually by Infer. Each function and method is analyzed separately. If Infer encounters an error during the analysis of a method or function, it stops analysis for that specific entity but continues with others. Hence, a typical workflow involves running Infer on the code, addressing identified errors, and rerunning Infer to discover additional issues or confirm fixes.

Detected errors are reported in the standard output and in a file named infer-out/report.txt. Infer filters and highlight bugs that are most likely to be authentic.

We can invoke only the analysis phase using the analyze subcommand:

infer analyze

3.2. Infer Global and Differential Workflows

By default, Infer will delete the previous infer-out/ directory if existing. This leads to a global workflow, where the entire project is analyzed each time.

Passing –reactive or -r to Infer prevents in from deleting infer-out/ directory, leading to a differential workflow. For example, mobile apps use incremental build systems, where code evolves as a sequence of code changes. For these changes, it would make sense to analyze only the current changes in the project, instead of analyzing the whole project each time. Therefore, we can take advantage of Infer’s reactive mode and switch to a differential workflow.

3.3. Analyzing Projects

To analyze files with Infer we can use compilers such as javac and clang. Also, we can use gcc, although Infer will use clang internally. Moreover, we can use Infer with various build systems.

One popular build system for Java is Maven, and we can use together with Infer in the following way:

infer run -- mvn <maven target>

Also, we can use Infer together with Gradle:

infer run -- gradle <gradle task>

3.4. Recommended Flow for CI

Infer recommends using the differential workflow for Continuous Integration (CI). Therefore, the flow would be to determine the modified files and run the analysis in reactive mode. Also, if we would like to run more than one analyzer, it is more efficient to separate the capture phase, so that the result can be used by all the analyzers.

4. Running Infer

We have multiple options to use Infer: binary releases, build Infer from source, or Docker images. Getting started with the Infer page explains how we can get and run Infer.

Now, we can create some dummy Java code snippets and analyze them. Not that we will cover only a few issues Infer can identify, and a complete list of all types of issues that can be detected by Infer can be found here.

4.1. Null Dereference

public class NullPointerDereference {
    public static void main(String[] args) {
        NullPointerDereference.nullPointerDereference();
    }
    private static void nullPointerDereference() {
        String str = null;
        int length = str.length();
    }
}

If we Infer against this code, we’ll have the following output:

Analyzed 1 file
Found 1 issue
./NullPointerDereference.java:11: error: NULL_DEREFERENCE
  object str last assigned on line 10 could be null and is dereferenced at line 11
   9.       private static void nullPointerDereference() {
  10.           String str = null;
  11. >         int length = str.length();
  12.       }
  13.   }
  24.   
Summary of the reports
  NULL_DEREFERENCE: 1

4.2. Resource Leak

public class ResourceLeak {
    public static void main(String[] args) throws IOException {
        ResourceLeak.resourceLeak();
    }
    private static void resourceLeak() throws IOException {
        FileOutputStream stream;
        try {
            File file = new File("randomName.txt");
            stream = new FileOutputStream(file);
        } catch (IOException e) {
            return;
        }
        stream.write(0);
    }
}

Now, by running Infer, we can see that a resource leak was detected:

Analyzed 1 file
Found 1 issue
./ResourceLeak.java:21: error: RESOURCE_LEAK
   resource of type java.io.FileOutputStream acquired to stream by call to FileOutputStream(...) at line 17 is not released after line 21
  19.               return;
  20.           }
  21. >         stream.write(0);
  22.       }
  23.   }
  24.   
Summary of the reports
  RESOURCE_LEAK: 1

4.3. Division by Zero

public class DivideByZero {
    public static void main(String[] args) {
        DivideByZero.divideByZero();
    }
    private static void divideByZero() {
        int dividend = 5;
        int divisor = 0;
        int result = dividend / divisor;
    }
}

The following output is displayed for our code:

Analyzed 1 file
Found 1 issue
./DivideByZero.java:9: error: DIVIDE_BY_ZERO
  The denominator for division is zero, which triggers an Arithmetic exception.
  6. private static void divideByZero() {
  7.     int dividend = 5;
  8.     int divisor = 0;
  9. >    int result = dividend / divisor;
  10. }
Summary of the reports
DIVIDE_BY_ZERO: 1

5. Conclusion

       

1. Overview

In this tutorial, we’ll examine how to use the replaceAll() provided in the String class to replace text using regular expressions. Additionally, we’ll learn two methods, back reference and lookaround, to perform the same operation and then compare their performance.

Let’s begin by describing the first method.

2. Using Back Reference With replaceAll()

To understand back reference, we first need to learn about matching groups. In short, a group is nothing but multiple characters seen as a single unit. So, back-references is a feature in regular expressions that allows us to refer back to previously matched groups within the same regex. Typically, we denote them with numbers that refer to the capturing group in the pattern, like \1, \2, etc.

For example, the regex (a)(b)\1 uses \1 to refer back to the first captured group, which in our case is (a).

In string replacement operations, we use these references to replace the matching text with the one we want. When using the replaceAll() method, we refer to a capturing group in the replacement string as $1, $2, etc.

Now, to understand better, let’s consider the following use case. We want to remove all the asterisk symbols within a string. So, the task is to preserve asterisks only if they appear at the beginning or the end of a string while removing all others. For example, *text* remains unaltered while **te*x**t** becomes *text*.

2.1. Implement Back Reference

To complete our task, we’ll use the replaceAll() method with a regular expression and use the backreference in:

String str = "**te*xt**";
String replaced = str.replaceAll("(^\\*)|(\\*$)|\\*", "$1$2");
assertEquals("*text*", replaced);

Above, we are defining the regular expression “(^\\*)|(\\*$)|\\*” which is made of three parts. The first group (^\\*) captures the asterisk at the beginning of the string. The second group (\\*$) captures the asterisk at the end of the string. The third group \\* captures all the rest of the asterisks. So, the regex only selects certain parts of the string, and only those selected parts will be replaced. We highlight the different parts with different colors:

In short, the replacement string $1$2 returns all the selected characters in that group so they are kept in the final string.

Let’s look at a different approach to solving the same task.

3. Using Lookaround With replaceAll()

An alternative approach to back reference is using lookarounds, which allow us to ignore the surrounding characters when doing the match in the regular expression. In our example, we can remove the asterisks within the string in a more intuitive way:

String str = "**te*xt**";
String replacedUsingLookaround = str.replaceAll("(?<!^)\\*+(?!$)", "");
assertEquals("*text*", replacedUsingLookaround);

In this example, (?<!^)\\*+ captures one or more asterisks (\\*+) that don’t have the start of the string before them ((?<!^)). In short, we are doing a negative look behind. Next, the (?!$) part is a negative lookahead that we define to ignore asterisks that are followed by the end of the string. Finally, the empty replacement string here removes all the matching characters. Therefore, this method is more easy to reason about as we are selecting all the characters we want to remove:

Apart from readability, these two methods differ in performance. Let’s check them out next.

4. Performance Lookaround vs. Back Reference

To compare the performance of both of these methods, we’ll use the JMH library to benchmark and measure the average execution time required for each method to process a large number of string replacements.

For our performance test, we’ll use the same asterisk example from the previous task. In short, we’ll repeatedly use the replaceAll() function with the two regex methods 1000 times.

For this test, we’ll configure 2 warmup iterations and 5 measurement iterations. Additionally, we’ll measure the average time taken to complete the task:

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Fork(1)
@Warmup(iterations = 2)
@Measurement(iterations = 5)
public class RegexpBenchmark {
    private static final int ITERATIONS_COUNT = 1000;
    @State(Scope.Benchmark)
    public static class BenchmarkState {
        String testString = "*example*text**with*many*asterisks**".repeat(ITERATIONS_COUNT);
    }
    @Benchmark
    public void backReference(BenchmarkState state) {
        state.testString.replaceAll("(^\\*)|(\\*$)|\\*", "$1$2");
    }
    @Benchmark
    public void lookaround(BenchmarkState state) {
        state.testString.replaceAll("(?<!^)\\*+(?!$)", "");
    }
    public static void main(String[] args) throws Exception {
        Options opt = new OptionsBuilder().include(RegexpBenchmark.class.getSimpleName())
          .build();
        new Runner(opt).run();
    }
}

The resulting output of this example states clearly that the lookaround method is more performant:

Benchmark                      Mode  Cnt  Score   Error  Units
RegexpBenchmark.backReference  avgt    5  0.504 ± 0.011  ms/op
RegexpBenchmark.lookaround     avgt    5  0.315 ± 0.006  ms/op

So, back reference is slower because it requires an overhead to capture the groups individually and then replace those groups with the replacement string. While lookaround, as explained previously, selects the characters directly and removes them.

5. Conclusion

In this article, we saw how to use the replaceAll() method with back references and lookarounds in regular expressions. While back references are useful for reusing parts of the matched string, they can be slower due to the overhead of capturing groups. To demonstrate this, we performed a benchmark to compare the two methods.

As always, we can check the complete code over on GitHub.

       

1. Spring and Java

>> JEP 467: Java Enhances Documentation with Markdown Support [infoq.com]

Another look at the proposed feature for Java 23: enabling markdowns in JavaDocs, instead of a painful mix of different tags and HTML.

>> Persist LocalDateTime, ZonedDateTime & Co with Hibernate [thorben-janssen.com]

An in-depth article on how to store different types of JSR-310 date and time abstractions using Hibernate.

>> Java Runtimes: Insights From the Spring Boot Point of View [jetbrains.com]

Not all runtimes are equal! an insightful take on how different metrics can influence our decision when choosing a particular Java distribution.

Also worth reading:

• >> Oracle Alternatives Survey & Report [foojay.io]
• >> Calling Microservices in Java: Part 1 [foojay.io]

Webinars and presentations:

• >> Module Imports in Java 23 – Inside Java Newscast #69 [inside.java]
• >> A Bootiful Podcast: Oleg Šelajev, Docker and Testcontainers legend [spring.io]
• >> Introduction to JDK Mission Control [inside.java]
• >> 5 Ways to Debug Quarkus JVM Applications [sebastian-daschner.com]
• >> Foojay Podcast #49: JCON Report, Part 1 – JUGs, Communities, Open Source, Generative AI, LangChain4j, Machine Learning [foojay.io]

Time to upgrade:

• >> Spring Framework 6.1.7, 6.0.20 and 5.3.35 available now [spring.io]
• >> Hibernate Search 7.2.0.Alpha1 is out [in.relation.to]
• >> Spring Framework 6.2.0-M2 available now [spring.io]
• >> Java Visual Studio Code Extension Update [inside.java]
• >> Spring Data 2024.0 goes GA [spring.io]
• >> Spring Web Services 4.0.11 is released [spring.io]
• >> Spring AMQP 3.0.14 and 3.1.5 Available [spring.io]
• >> Spring for Apache Pulsar 1.1.0 available now [spring.io]

2. Technical & Musings

>> PostgreSQL plan_cache_mode [vladmihalcea.com]

Meet plan_cache_mode setting: for some prepared statements, it’s better to explicitly use a better plan cache mode!

Also worth reading:

• >> Data Fetching Patterns in Single-Page Applications [martinfowler.com]
• >> Workflow, from stateless to stateful [frankel.ch]
• >> Software Testing as a Debugging Tool [foojay.io]
• >> Parallel Data Fetching [martinfowler.com]
• >> Docker Compose Profiles, one the most useful and underrated features [event-driven.io]

3. Pick of the Week

>> You probably don’t need microservices [thrownewexception.com]

       

1. Overview

With the widespread use of generative AI and ChatGPT, in particular, many languages have started to provide libraries that interact with their OpenAI API. Java isn’t an exception.

In this tutorial, we’ll talk about openai-java. This is a client that allows more convenient communication with OpenAI API. However, reviewing the entire library in a single article is impossible. Thus, we’ll use a practical example and build a simple console tool connected to ChatGPT.

2. Dependencies

First, we must import the required dependencies for our project. We can find the libraries in the Maven repository. These three modules are dedicated to different aspects of the interaction:

<dependency>
    <groupId>com.theokanning.openai-gpt3-java</groupId>
    <artifactId>service</artifactId>
    <version>0.18.2</version>
</dependency>
<dependency>
    <groupId>com.theokanning.openai-gpt3-java</groupId>
    <artifactId>api</artifactId>
    <version>0.18.2</version>
</dependency>
<dependency>
    <groupId>com.theokanning.openai-gpt3-java</groupId>
    <artifactId>client</artifactId>
    <version>0.18.2</version>
</dependency>

Please note that the name explicitly mentions GPT3, but it works with GPT4 as well.

3. Baeldung Tutor

In this tutorial, we’ll build a tool that helps us to create our curriculum based on the articles and tutorials from our favorite learning platform, or at least try to do so. While the internet provides us with unlimited resources and we can find almost anything online, curating the information has become much harder.

Trying to learn new things is increasingly overwhelming as it’s hard to identify the best learning path and filter things that won’t benefit us. To resolve this problem, we’ll build a simple client to interact with ChatGPT and ask it to guide us in the vast ocean of Baeldung articles.

4. OpenAI API Token

The first step is to connect our application to the OpenAI API. To do so, we need to provide an OpenAI token, which can be generated on the website:

However, we should be careful about the token and avoid exposing it. The openai-java examples use the environment variables for this. This might not be the best solution for production, but it works well for small experiments.

During the runs, we don’t necessarily need to identify the environment variable for our entire machine; we can use configurations in our IDE. For example, IntelliJ IDEA provides a simple way to do so.

We can generate two types of tokens: personal and service accounts. The personal token is self-explanatory. The tokens for service accounts are used for bots or applications that can be connected to the OpenAI projects. While both would work, a personal token is good enough for our purpose.

5. OpenAiService

The entry point to OpenAI APIs is the class conveniently named OpenAiService. The instance of this class allows us to interact with the APIs and receive responses from the ChatGPT. To create it, we should pass the token we’ve generated in the previous step:

String token = System.getenv("OPENAI_TOKEN");
OpenAiService service = new OpenAiService(token);

This is the first step in our journey; we need to identify the information and populate the request.

5.1. ChatCompletionRequest

We create a request using ChatCompletionRequest. The minimal setup requires us to provide only messages and a model :

ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest
  .builder()
  .model(GPT_3_5_TURBO_0301.getName())
  .messages(messages)
  .build();

Let’s review these parameters step-by-step.

5.2. Model

It’s essential to pick the model that would fit our requirements, and also it affects the costs. Thus, we need to make a reasonable choice. For example, often, there’s no need to use the most advanced model to clean text or parse it based on some simple formats. At the same time, more complex or important tasks require more advanced models to reach our goals.

While we can pass the model name directly, it’s better to use the ModelEnum:

@Getter
@AllArgsConstructor
public enum ModelEnum {         
    GPT_3_5_TURBO("gpt-3.5-turbo"),
    GPT_3_5_TURBO_0301("gpt-3.5-turbo-0301"),
    GPT_4("gpt-4"),
    GPT_4_0314("gpt-4-0314"),
    GPT_4_32K("gpt-4-32k"),
    GPT_4_32K_0314("gpt-4-32k-0314"),
    GPT_4_1106_preview("gpt-4-1106-preview");
    private String name;
}

It doesn’t contain all the models, but in our case, it’s enough. If we want to use a different model, we can provide its name as a String.

5.3. Messages

The next thing is the messages we’ve created. We use the ChatMessage class for it. In our case, we pass only the role and the message itself:

List<ChatMessage> messages = new ArrayList<>();
ChatMessage systemMessage = new ChatMessage(ChatMessageRole.SYSTEM.value(), PROMPT);
messages.add(systemMessage);

The interesting part is that we send a collection of messages. Although in usual chats, we communicate by sending messages one by one, in this case, it’s more similar to email threads.

The system works on completion and appends the next message to the chain. This way, we can maintain the context of the conversation. We can think about this as a stateless service. However, this means we must pass the messages to keep the context.

At the same time, we can go another way and create an assistant. With this approach, we would store the messages in the threads, and it doesn’t require sending the entire history back and forth.

While passing, the content of the messages is reasonable, but the purpose of the roles isn’t. Because we send all the messages at once, we need to provide some way to identify the relationships between the messages and users based on their roles.

5.4. Roles

As was mentioned, roles are crucial for ChatGPT to understand the context of the conversation. We can use them to identify the actors behind the messages. This way, we can help ChatGPT interpret the messages correctly. ChatMessages support four roles: chat, system, assistant, and function:

public enum ChatMessageRole {
    SYSTEM("system"),
    USER("user"),
    ASSISTANT("assistant"),
    FUNCTION("function");
    private final String value;
    ChatMessageRole(final String value) {
        this.value = value;
    }
    public String value() {
        return value;
    }
}

Usually, the SYSTEM role refers to the initial context or prompt. The user represents the user of the ChatGPT, and the assistant is a ChatGPT itself. This means that, technically, we can also write the messages from the assistant’s standpoint. As its name suggests, the function role identifies the functions the assistant can use.

5.5. Tokens

While we previously talked about access tokens to the API, the meaning is different in the context of models and messages. We can think about tokens as the amount of information we can process and the amount we want to get in response.

We can restrict the model from generating huge responses by limiting the number of tokens in the response:

ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest
  .builder()
  .model(MODEL)
  .maxTokens(MAX_TOKENS)
  .messages(messages)
  .build();

There’s no direct mapping between the words and tokens because each model processes them slightly differently. This parameter restricts the answer to a specific number of tokens. Using the default values might allow excessive responses and increase the bills for usage. Thus, it’s a good practice to configure it explicitly.

We can add the information about used tokens after each response:

long usedTokens = result.getUsage().getTotalTokens();
System.out.println("Total tokens used: " + usedTokens);

5.6. Tokenization

In the previous example, we displayed the number of tokens used in the response. While this information is valuable, we often need to estimate the size of the request as well. To achieve this, we can use tokenizers provided by OpenAI.

To do this in a more automated way, openai-java provides us with TikTokensUtil, to which we can pass the name of the model and the messages and get the number of tokens as a result.

5.7. Options

An additional method we can use to configure our request is mysteriously named n(). It controls how many responses we want to get for each request. Simply put, we can have two different answers to the same request. By default, we’ll have only one.

Sometimes, it could be useful for bots and website assistants. However, the responses are billed based on the tokens across all the options.

5.8. Bias and Randomization

We can use a couple of additional options to control the randomness and biases of ChatGPT answers. For example, logitBias() can make it more probable to see or not to see specific tokens. Please note that we’re talking about tokens and not particular words here. However, it doesn’t mean this token won’t appear 100%.

Also, we can use topP() and temperature() to randomize the responses. While it’s useful for some cases, we won’t change the defaults for our learning tool.

6. Curriculum

Now, let’s check our tool in action. We’ll have the following overall code:

public static void main(String[] args) {
    String token = System.getenv("OPENAI_TOKEN");
    OpenAiService service = new OpenAiService(token);
    List<ChatMessage> messages = new ArrayList<>();
    ChatMessage systemMessage = new ChatMessage(ChatMessageRole.SYSTEM.value(), PROMPT);
    messages.add(systemMessage);
    System.out.print(GREETING);
    Scanner scanner = new Scanner(System.in);
    ChatMessage firstMsg = new ChatMessage(ChatMessageRole.USER.value(), scanner.nextLine());
    messages.add(firstMsg);
    while (true) {
        ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest
          .builder()
          .model(GPT_3_5_TURBO_0301.getName())
          .messages(messages)
          .build();
        ChatCompletionResult result = service.createChatCompletion(chatCompletionRequest);
        long usedTokens = result.getUsage().getTotalTokens();
        ChatMessage response = result.getChoices().get(0).getMessage();
        messages.add(response);
        System.out.println(response.getContent());
        System.out.println("Total tokens used: " + usedTokens);
        System.out.print("Anything else?\n");
        String nextLine = scanner.nextLine();
        if (nextLine.equalsIgnoreCase("exit")) {
            System.exit(0);
        }
        messages.add(new ChatMessage(ChatMessageRole.USER.value(), nextLine));
    }
}

If we run it, we can interact with it via the console:

Hello!
What do you want to learn?

In response, we can write the topics that we’re interested in:

$ I would like to learn about binary trees.

As expected, the tool would provide us with some articles we can use to learn about the topics:

Great! Here's a suggested order for Baeldung's articles on binary trees:
1. Introduction to Binary Trees: https://www.baeldung.com/java-binary-tree-intro
2. Implementing a Binary Tree in Java: https://www.baeldung.com/java-binary-tree
3. Depth First Traversal of Binary Tree: https://www.baeldung.com/java-depth-first-binary-tree-traversal
4. Breadth First Traversal of Binary Tree: https://www.baeldung.com/java-breadth-first-binary-tree-traversal
5. Finding the Maximum Element in a Binary Tree: https://www.baeldung.com/java-binary-tree-maximum-element
6. Binary Search Trees in Java: https://www.baeldung.com/java-binary-search-tree
7. Deleting from a Binary Search Tree: https://www.baeldung.com/java-binary-search-tree-delete
I hope this helps you out!
Total tokens used: 288
Anything else?

This way, we solved the problem by creating a curriculum and learning new things. However, not everything is so bright; the problem is that only one article is real. For the most part, ChatGPT listed non-existent articles with appropriate links. While the names and the links sound reasonable, they won’t lead us anywhere.

This is a crucial aspect of any AI tool. Generative models have a hard time checking the validity of the information. As they are based on predicting and picking the most appropriate next word, it might be hard for them to verify the information. We cannot rely 100% on the information from generative models.

7. Conclusion

AI tools are great and help us to improve applications and automate daily chores, from processing emails and creating shopping lists to optimizing education. Java provides a couple of ways to interact with OpenAI APIs, and openai-java is one such library.

However, it’s important to remember that generative models, despite being quite convincing, have trouble validating if the information is true. Thus, it’s our responsibility to either recheck crucial information or provide a model with enough information that it will be able to give us valid answers.

As usual, all the code from this tutorial is available over on GitHub.

       

1. Overview

Apache Kafka is an event streaming platform that collects, processes, stores, and integrates data at scale. Sometimes, we may want to delay the processing of messages from Kafka. An example is a customer order processing system designed to process orders after a delay of X seconds, accommodating cancellations within this timeframe.

In this article, we’ll explore consumer processing of Kafka messages with delay using Spring Kafka. Although Kafka doesn’t provide out-of-the-box support for the delayed consumption of messages, we’ll look at an alternative option for implementation.

2. Application Context

Kafka offers multiple ways to retry on errors. We’ll use this retry mechanism to delay the consumer processing of messages. Therefore, it’s worth understanding how Kafka retry works.

Let’s consider an order processing application where a customer can place an order on a UI. The user can cancel mistakenly placed orders within 10 seconds. These orders go to the Kafka topic web.orders, where our application processes them.

An external service exposes the latest order status (CREATED, ORDER_CONFIRMED, ORDER_PROCESSED, DELETED). Our application needs to receive the message, wait for 10 seconds, and check with the external service to process the order if it’s in CONFIRMED status, i.e., the user hasn’t canceled it within the 10 seconds.

For testing, the internal orders received from web.orders.internal shouldn’t be delayed.

Let’s add a simple Order model that has orderGeneratedDateTime populated by the producer and orderProcessedTime populated by the consumer after a delayed duration:

@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class Order {
    private UUID orderId;
    private LocalDateTime orderGeneratedDateTime;
    private LocalDateTime orderProcessedTime;
    private List<String> address;
    private double price;
}

3. Kafka Listener and External Service

Next, we’ll add a listener for topic consumption and a service that exposes the status of the orders.

Let’s add a KafkaListener which reads and processes messages from topics web.orders and web.internal.orders :

@RetryableTopic(attempts = "1", include = KafkaBackoffException.class, dltStrategy = DltStrategy.NO_DLT)
@KafkaListener(topics = { "web.orders", "web.internal.orders" }, groupId = "orders")
public void handleOrders(String order) throws JsonProcessingException {
    Order orderDetails = objectMapper.readValue(order, Order.class);
    OrderService.Status orderStatus = orderService.findStatusById(orderDetails.getOrderId());
    if (orderStatus.equals(OrderService.Status.ORDER_CONFIRMED)) {
        orderService.processOrder(orderDetails);
    }
}

It’s important to include KafkaBackoffException so that the listener allows retries. For simplicity, let’s consider that the external OrderService always returns the order status as CONFIRMED. Also, the processOrder() method sets the order processed time as the current time and saves the order into a HashMap:

@Service
public class OrderService {
    HashMap<UUID, Order> orders = new HashMap<>();
    public Status findStatusById(UUID orderId) {
        return Status.ORDER_CONFIRMED;
    }
    public void processOrder(Order order) {
        order.setOrderProcessedTime(LocalDateTime.now());
        orders.put(order.getOrderId(), order);
    }
}

4. Custom Delayed Message Listener

Spring-Kafka comes up with the KafkaBackoffAwareMessageListenerAdapter which extends AbstractAdaptableMessageListener and implements AcknowledgingConsumerAwareMessageListener. This adapter examines the backoff dueTimestamp header and either backs off the message by invoking KafkaConsumerBackoffManager or retries the processing.

Let’s now implement the DelayedMessageListenerAdapter similar to KafkaBackoffAwareMessageListenerAdapter. This adapter should provide flexibility to configure delays per topic along with a default delay of 0 seconds:

public class DelayedMessageListenerAdapter<K, V> extends AbstractDelegatingMessageListenerAdapter<MessageListener<K, V>> 
  implements AcknowledgingConsumerAwareMessageListener<K, V> {
    // Field declaration and constructor
    public void setDelayForTopic(String topic, Duration delay) {
        Objects.requireNonNull(topic, "Topic cannot be null");
        Objects.requireNonNull(delay, "Delay cannot be null");
        this.logger.debug(() -> String.format("Setting delay %s for listener id %s", delay, this.listenerId));
        this.delaysPerTopic.put(topic, delay);
    }
    public void setDefaultDelay(Duration delay) {
        Objects.requireNonNull(delay, "Delay cannot be null");
        this.logger.debug(() -> String.format("Setting delay %s for listener id %s", delay, this.listenerId));
        this.defaultDelay = delay;
    }
    @Override
    public void onMessage(ConsumerRecord<K, V> consumerRecord, Acknowledgment acknowledgment, Consumer<?, ?> consumer) throws KafkaBackoffException {
        this.kafkaConsumerBackoffManager.backOffIfNecessary(createContext(consumerRecord,
          consumerRecord.timestamp() + delaysPerTopic.getOrDefault(consumerRecord.topic(), this.defaultDelay)
          .toMillis(), consumer));
        invokeDelegateOnMessage(consumerRecord, acknowledgment, consumer);
    }
    private KafkaConsumerBackoffManager.Context createContext(ConsumerRecord<K, V> data, long nextExecutionTimestamp, Consumer<?, ?> consumer) {
        return this.kafkaConsumerBackoffManager.createContext(nextExecutionTimestamp, 
          this.listenerId, 
          new TopicPartition(data.topic(), data.partition()), consumer);
    }
}

For every incoming message, this adapter first receives the record and checks the delay set for the topic. This will be set in the configuration, and if not set, it uses the default delay.

The existing implementation of KafkaConsumerBackoffManager#backOffIfNecessary method checks the difference between the context record timestamp and the current timestamp. If the difference is positive, showing no consumption due, the partition pauses and raises a KafkaBackoffException. Otherwise, it sends the record to the KafkaListener method for consumption.

5. Listener Configuration

The ConcurrentKafkaListenerContainerFactory is the default implementation of Spring Kafka which is responsible for building containers for KafkaListener. It allows us to configure the number of concurrent KafkaListener instances. Each container can be considered a logical thread pool, where each thread is responsible for listening to messages from one or more Kafka topics.

The DelayedMessageListenerAdapter needs to be configured with the listener by declaring a custom ConcurrentKafkaListenerContainerFactory. We can set the delay for specific topics like web.orders and also set a default delay of 0 for any other topics:

@Bean
public ConcurrentKafkaListenerContainerFactory<Object, Object> kafkaListenerContainerFactory(ConsumerFactory<Object, Object> consumerFactory, 
  ListenerContainerRegistry registry, TaskScheduler scheduler) {
    ConcurrentKafkaListenerContainerFactory<Object, Object> factory = new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory);
    KafkaConsumerBackoffManager backOffManager = createBackOffManager(registry, scheduler);
    factory.getContainerProperties()
      .setAckMode(ContainerProperties.AckMode.RECORD);
    factory.setContainerCustomizer(container -> {
        DelayedMessageListenerAdapter<Object, Object> delayedAdapter = wrapWithDelayedMessageListenerAdapter(backOffManager, container);
        delayedAdapter.setDelayForTopic("web.orders", Duration.ofSeconds(10));
        delayedAdapter.setDefaultDelay(Duration.ZERO);
        container.setupMessageListener(delayedAdapter);
    });
    return factory;
}
@SuppressWarnings("unchecked")
private DelayedMessageListenerAdapter<Object, Object> wrapWithDelayedMessageListenerAdapter(KafkaConsumerBackoffManager backOffManager, 
  ConcurrentMessageListenerContainer<Object, Object> container) {
    return new DelayedMessageListenerAdapter<>((MessageListener<Object, Object>) container.getContainerProperties()
      .getMessageListener(), backOffManager, container.getListenerId());
}
private ContainerPartitionPausingBackOffManager createBackOffManager(ListenerContainerRegistry registry, TaskScheduler scheduler) {
    return new ContainerPartitionPausingBackOffManager(registry, 
      new ContainerPausingBackOffHandler(new ListenerContainerPauseService(registry, scheduler)));
}

Notably, setting the acknowledgment mode at the RECORD level is essential to ensure that the consumer redelivers messages if an error happens during processing.

Finally, we need to define a TaskScheduler bean to resume paused partitions after the delay duration and this scheduler needs to be injected into the BackOffManager which will be used by DelayedMessageListenerAdapter:

@Bean
public TaskScheduler taskScheduler() {
    return new ThreadPoolTaskScheduler();
}

6. Testing

Let’s ensure orders on the web.orders topic undergo a 10-second delay before processing through testing:

@Test
void givenKafkaBrokerExists_whenCreateOrderIsReceived_thenMessageShouldBeDelayed() throws Exception {
    // Given
    var orderId = UUID.randomUUID();
    Order order = Order.builder()
      .orderId(orderId)
      .price(1.0)
      .orderGeneratedDateTime(LocalDateTime.now())
      .address(List.of("41 Felix Avenue, Luton"))
      .build();
    String orderString = objectMapper.writeValueAsString(order);
    ProducerRecord<String, String> record = new ProducerRecord<>("web.orders", orderString);
    
    // When
    testKafkaProducer.send(record)
      .get();
    await().atMost(Duration.ofSeconds(1800))
      .until(() -> {
          // then
          Map<UUID, Order> orders = orderService.getOrders();
          return orders != null && orders.get(orderId) != null && Duration.between(orders.get(orderId)
              .getOrderGeneratedDateTime(), orders.get(orderId)
              .getOrderProcessedTime())
            .getSeconds() >= 10;
      });
}

Next, we’ll test any orders to web.internal.orders follow a default delay of 0 seconds:

@Test
void givenKafkaBrokerExists_whenCreateOrderIsReceivedForOtherTopics_thenMessageShouldNotBeDelayed() throws Exception {
    // Given
    var orderId = UUID.randomUUID();
    Order order = Order.builder()
      .orderId(orderId)
      .price(1.0)
      .orderGeneratedDateTime(LocalDateTime.now())
      .address(List.of("41 Felix Avenue, Luton"))
      .build();
    String orderString = objectMapper.writeValueAsString(order);
    ProducerRecord<String, String> record = new ProducerRecord<>("web.internal.orders", orderString);
    
    // When
    testKafkaProducer.send(record)
      .get();
    await().atMost(Duration.ofSeconds(1800))
      .until(() -> {
          // Then
          Map<UUID, Order> orders = orderService.getOrders();
          System.out.println("Time...." + Duration.between(orders.get(orderId)
              .getOrderGeneratedDateTime(), orders.get(orderId)
              .getOrderProcessedTime())
            .getSeconds());
          return orders != null && orders.get(orderId) != null && Duration.between(orders.get(orderId)
              .getOrderGeneratedDateTime(), orders.get(orderId)
              .getOrderProcessedTime())
            .getSeconds() <= 1;
      });
}

7. Conclusion

In this tutorial, we explored how a Kafka consumer can delay processing messages by fixed intervals.

We can modify the implementation to dynamically set processing delays by utilizing embedded message durations as part of the message.

As always, the source code for the examples is available over on GitHub.