Get List of JSON Objects with WebClient

1. Overview

Our services often communicate with other REST services to fetch information.

From Spring 5, we get to use WebClient to perform these requests in a reactive, non-blocking way. WebClient is part of the new WebFlux Framework, built on top of Project Reactor. It has a fluent, reactive API, and it uses HTTP protocol in its underlying implementation.

When we make a web request, the data is often returned as JSON. WebClient can convert this for us.

In this article, we'll find out how to convert a JSON Array into a Java Array of Object, Array of POJO, and a List of POJO using WebClient.

2. Dependencies

To use WebClient, we'll need to add a couple of dependencies to our pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
    <groupId>org.projectreactor</groupId>
    <artifactId>reactor-spring</artifactId>
    <version>1.0.1.RELEASE</version>
</dependency>

3. JSON, POJO, and Service

Let's start with an endpoint http://localhost:8080/readers that returns a list of readers with their favorite books as a JSON array:

[{
    "id": 1,
    "name": "reader1",
    "favouriteBook": { 
        "author": "Milan Kundera",
        "title": "The Unbearable Lightness of Being"
    }
}, {
    "id": 2,
    "name": "reader2"
    "favouriteBook": { 
        "author": "Douglas Adams",
        "title": "The Hitchhiker's Guide to the Galaxy"
    }
}]

We'll require the corresponding Reader and Book classes to process data:

public class Reader {
    private int id;
    private String name;
    private Book favouriteBook;
    // getters and setters..
}

public class Book {
    private final String author;
    private final String title;
   // getters and setters..
}

For our interface implementation, we write a ReaderConsumerServiceImpl with WebClient as its dependency:

public class ReaderConsumerServiceImpl implements ReaderConsumerService {
    private final WebClient webClient;
    public ReaderConsumerServiceImpl(WebClient webclient) {
        this.webclient = webclient;
    }
    // ...
}

4. Mapping a List of JSON Objects

When we receive a JSON array from a REST request, there are multiple ways to convert it to a Java collection. Let's look at the various options and see how easy it is to process the data returned. We'll look at extracting the readers' favorite books.

4.1. Mono vs. Flux

Project Reactor has introduced two implementations of Publisher: Mono and Flux.

Flux<T> is useful when we need to handle zero to many or potentially infinite results. We can think of a Twitter feed as an example.

When we know that the results are returned all at once – as in our use case – we can use Mono<T>.

4.2. WebClient with Object Array

First, let's make the GET call with WebClient.get and use a Mono of type Object[] to collect the response:

Mono<Object[]> response = webClient.get()
  .accept(MediaType.APPLICATION_JSON)
  .retrieve()
  .bodyToMono(Object[].class).log();

Next, let's extract the body into our array of Object:

Object[] objects = response.block();

The actual Object here is an arbitrary structure that contains our data. Let's convert it into an array of Reader objects.

For this, we'll need an ObjectMapper:

ObjectMapper mapper = new ObjectMapper();

Here, we declared it inline, though this is usually done as a private static final member of the class.

Lastly, we're ready to extract the readers' favorite books and collect them to a list:

return Arrays.stream(objects)
  .map(object -> mapper.convertValue(object, Reader.class))
  .map(Reader::getFavouriteBook)
  .collect(Collectors.toList());

When we ask the Jackson deserializer to produce Object as the target type, it actually deserializes JSON into a series of LinkedHashMap objects. Post-processing with convertValue is inefficient. We can avoid this if we provide our desired type to Jackson during deserialization.

4.3. WebClient with Reader Array

We can provide Reader[] instead of Object[] to our WebClient:

Mono<Reader[]> response = webClient.get()
  .accept(MediaType.APPLICATION_JSON)
  .retrieve()
  .bodyToMono(Reader[].class).log();
Reader[] readers = response.block();
return Arrays.stream(readers)
  .map(Reader:getFavouriteBook)
  .collect(Collectors.toList());

Here, we can observe that we no longer need the ObjectMapper.convertValue. However, we still need to do additional conversions to use the Java Stream API and for our code to work with a List.

4.4. WebClient with Reader List

If we want Jackson to produce a List of Readers instead of an array, we need to describe the List we want to create. To do this, we provide a ParameterizedTypeReference produced by an anonymous inner class to the method:

Mono<List> response = webClient.get()
  .accept(MediaType.APPLICATION_JSON)
  .retrieve()
  .bodyToMono(new ParameterizedTypeReference<List>() {});
List readers = response.block();
return readers.stream()
  .map(Reader::getFavouriteBook)
  .collect(Collectors.toList());

This gives us the List that we can work with.

Let's take a deeper dive into why we need to use the ParameterizedTypeReference.

Spring's WebClient can easily deserialize the JSON into a Reader.class when the type information is available at runtime.

With generics, however, type erasure occurs if we try to use List<Reader>.class. So, Jackson will not be able to determine the generic's type parameter.

By using ParameterizedTypeReference, we can overcome this problem. Instantiating it as an anonymous inner class exploits the fact that subclasses of generic classes contain compile-time type information that is not subject to type erasure and can be consumed through reflection.

5. Conclusion

In this tutorial, we saw three different ways of processing JSON objects using WebClient. We saw ways of specifying the types of arrays of Object and our own custom classes.

We then learned how to provide the type information to produce a List by using the ParameterizedTypeReference.

As always, the code for this article is available over on GitHub.