Long Polling in Spring MVC

1. Overview

Long polling is a method that server applications use to hold a client connection until information becomes available. This is often used when a server must call a downstream service to get information and await a result.

In this tutorial, we'll explore the concept of long polling in Spring MVC by using DeferredResult. We'll start by looking at a basic implementation using DeferredResult and then discuss how we can handle errors and timeouts. Finally, we'll look at how all this can be tested.

2. Long Polling Using DeferredResult

We can use DeferredResult in Spring MVC as a way to handle inbound HTTP requests asynchronously. It allows the HTTP worker thread to be freed up to handle other incoming requests and offloads the work to another worker thread. As such, it helps with service availability for requests that require long computations or arbitrary wait times.

Our previous article on Spring's DeferredResult class covers its capabilities and use cases in greater depth.

2.1. Publisher

Let's start our long polling example by creating a publishing application that uses DeferredResult. 

Initially, let's define a Spring @RestController that makes use of DeferredResult but does not offload its work to another worker thread:

@RestController
@RequestMapping("/api")
public class BakeryController { 
    @GetMapping("/bake/{bakedGood}")
    public DeferredResult<String> publisher(@PathVariable String bakedGood, @RequestParam Integer bakeTime) {
        DeferredResult<String> output = new DeferredResult<>();
        try {
            Thread.sleep(bakeTime);
            output.setResult(format("Bake for %s complete and order dispatched. Enjoy!", bakedGood));
        } catch (Exception e) {
            // ...
        }
        return output;
    }
}

This controller works synchronously in the same way that a regular blocking controller works. As such, our HTTP thread is completely blocked until bakeTime has passed. This is not ideal if our service has a lot of inbound traffic.

Let's now set the output asynchronously by offloading the work to a worker thread:

private ExecutorService bakers = Executors.newFixedThreadPool(5);
@GetMapping("/bake/{bakedGood}")
public DeferredResult<String> publisher(@PathVariable String bakedGood, @RequestParam Integer bakeTime) {
    DeferredResult<String> output = new DeferredResult<>();
    bakers.execute(() -> {
        try {
            Thread.sleep(bakeTime);
            output.setResult(format("Bake for %s complete and order dispatched. Enjoy!", bakedGood));
        } catch (Exception e) {
            // ...
        }
    });
    return output;
}

In this example, we're now able to free up the HTTP worker thread to handle other requests. A worker thread from our bakers pool is doing the work and will set the result upon completion. When the worker calls setResult, it will allow the container thread to respond to the calling client.

Our code is now a good candidate for long polling and will allow our service to be more available to inbound HTTP requests than with a traditional blocking controller. However, we also need to take care of edge cases such as error handling and timeout handling.

To handle checked errors thrown by our worker, we'll use the setErrorResult method provided by DeferredResult:

bakers.execute(() -> {
    try {
        Thread.sleep(bakeTime);
        output.setResult(format("Bake for %s complete and order dispatched. Enjoy!", bakedGood));
     } catch (Exception e) {
        output.setErrorResult("Something went wrong with your order!");
     }
});

The worker thread is now able to gracefully handle any exception thrown.

Since long polling is often implemented to handle responses from downstream systems both asynchronously and synchronously, we should add a mechanism to enforce a timeout in the case that we never receive a response from the downstream system. The DeferredResult API provides a mechanism for doing this. First, we pass in a timeout parameter in the constructor of our DeferredResult object:

DeferredResult<String> output = new DeferredResult<>(5000L);

Next, let's implement the timeout scenario. For this, we'll use onTimeout:

output.onTimeout(() -> output.setErrorResult("the bakery is not responding in allowed time"));

This takes in a Runnable as input — it's invoked by the container thread when the timeout threshold is reached. If the timeout is reached, then we handle this as an error and use setErrorResult accordingly.

2.2. Subscriber

Now that we have our publishing application set up, let's write a subscribing client application.

Writing a service that calls this long polling API is fairly straightforward, as it's essentially the same as writing a client for standard blocking REST calls. The only real difference is that we want to ensure we have a timeout mechanism in place due to the wait time of long polling. In Spring MVC, we can use RestTemplate or WebClient to achieve this, as both have built-in timeout handling.

First, let's start with an example using RestTemplate. Let's create an instance of RestTemplate using RestTemplateBuilder so that we can set the timeout duration:

public String callBakeWithRestTemplate(RestTemplateBuilder restTemplateBuilder) {
    RestTemplate restTemplate = restTemplateBuilder
      .setConnectTimeout(Duration.ofSeconds(10))
      .setReadTimeout(Duration.ofSeconds(10))
      .build();
    try {
        return restTemplate.getForObject("/api/bake/cookie?bakeTime=1000", String.class);
    } catch (ResourceAccessException e) {
        // handle timeout
    }
}

In this code, by catching the ResourceAccessException from our long polling call, we're able to handle the error upon timeout.

Next, let's create an example using WebClient to achieve the same result:

public String callBakeWithWebClient() {
    WebClient webClient = WebClient.create();
    try {
        return webClient.get()
          .uri("/api/bake/cookie?bakeTime=1000")
          .retrieve()
          .bodyToFlux(String.class)
          .timeout(Duration.ofSeconds(10))
          .blockFirst();
    } catch (ReadTimeoutException e) {
        // handle timeout
    }
}

Our previous article on setting Spring REST timeouts covers this topic in greater depth.

3. Testing Long Polling

Now that we have our application up and running, let's discuss how we can test it. We can start by using MockMvc to test calls to our controller class:

MvcResult asyncListener = mockMvc
  .perform(MockMvcRequestBuilders.get("/api/bake/cookie?bakeTime=1000"))
  .andExpect(request().asyncStarted())
  .andReturn();

Here, we're calling our DeferredResult endpoint and asserting that the request has started an asynchronous call. From here, the test will await the completion of the asynchronous result, meaning that we do not need to add any waiting logic in our test.

Next, we want to assert when the asynchronous call has returned and that it matches the value that we're expecting:

String response = mockMvc
  .perform(asyncDispatch(asyncListener))
  .andReturn()
  .getResponse()
  .getContentAsString();
assertThat(response)
  .isEqualTo("Bake for cookie complete and order dispatched. Enjoy!");

By using asyncDispatch(), we can get the response of the asynchronous call and assert its value.

To test the timeout mechanism of our DeferredResult, we need to alter the test code slightly by adding a timeout enabler between the asyncListener and the response calls:

((MockAsyncContext) asyncListener
  .getRequest()
  .getAsyncContext())
  .getListeners()
  .get(0)
  .onTimeout(null);

This code might look strange, but there's a specific reason we call onTimeout in this way. We do this to let the AsyncListener know that an operation has timed out. This will ensure that the Runnable class that we've implemented for our onTimeout method in our controller is called correctly.

4. Conclusion

In this article, we covered how to use DeferredResult in the context of long polling. We also discussed how we can write subscribing clients for long polling, and how it can be tested. The source code is available over on GitHub.