Advanced HttpClient Configuration

1. Overview

In this article, we will be looking at the advanced usage of the Apache HttpClient library.

We’ll look at the examples of adding custom headers to HTTP requests, and we’ll see how to configure the client to authorize and send requests through a proxy server.

We will be using Wiremock for stubbing the HTTP server. If you want to read more about Wiremock, check out this article.

2. HTTP Request With a Custom User-Agent Header

Let’s say that we want to add a custom User-Agent header to an HTTP GET request. The User-Agent header contains a characteristic string that allows the network protocol peers to identify the application type, operating system, and software vendor or software version of the requesting software user agent.

Before we start writing our HTTP client, we need to start our embedded mock server:

@Rule
public WireMockRule serviceMock = new WireMockRule(8089);

When we’re creating a HttpGet instance we can simply use a setHeader() method to pass a name of our header together with the value. That header will be added to an HTTP request:

String userAgent = "BaeldungAgent/1.0"; 
HttpClient httpClient = HttpClients.createDefault();

HttpGet httpGet = new HttpGet("http://localhost:8089/detail");
httpGet.setHeader(HttpHeaders.USER_AGENT, userAgent);

HttpResponse response = httpClient.execute(httpGet);

assertEquals(response.getStatusLine().getStatusCode(), 200);

We’re adding a User-Agent header and sending that request via an execute() method.

When GET request is sent for a URL /detail with header User-Agent that has a value equal to “BaeldungAgent/1.0” then serviceMock will return 200 HTTP response code:

serviceMock.stubFor(get(urlEqualTo("/detail"))
  .withHeader("User-Agent", equalTo(userAgent))
  .willReturn(aResponse().withStatus(200)));

3. Sending Data in the POST Request Body

Usually, when we are executing the HTTP POST method, we want to pass an entity as a request body. When creating an instance of a HttpPost object, we can add the body to that request using a setEntity() method:

String xmlBody = "<xml><id>1</id></xml>";
HttpClient httpClient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost("http://localhost:8089/person");
httpPost.setHeader("Content-Type", "application/xml");

StringEntity xmlEntity = new StringEntity(xmlBody);
httpPost.setEntity(xmlEntity);

HttpResponse response = httpClient.execute(httpPost);

assertEquals(response.getStatusLine().getStatusCode(), 200);

We are creating a StringEntity instance with a body that is in the XML format. It is important to set the Content-Type header to “application/xml” to pass information to the server about the type of content we’re sending. When the serviceMock receives the POST request with XML body, it responds with status code 200 OK:

serviceMock.stubFor(post(urlEqualTo("/person"))
  .withHeader("Content-Type", equalTo("application/xml"))
  .withRequestBody(equalTo(xmlBody))
  .willReturn(aResponse().withStatus(200)));

4. Sending Requests via a Proxy Server

Often, our web service can be behind a proxy server that performs some additional logic, caches static resources, etc. When we’re creating the HTTP Client and sending a request to an actual service, we don’t want to deal with that on each and every HTTP request.

To test this scenario, we’ll need to start up another embedded web server:

@Rule
public WireMockRule proxyMock = new WireMockRule(8090);

With two embedded servers, the first actual service is on the 8089 port and a proxy server is listening on the 8090 port.

We are configuring our HttpClient to send all requests via proxy by creating a DefaultProxyRoutePlanner that takes the HttpHost instance proxy as an argument:

HttpHost proxy = new HttpHost("localhost", 8090);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
HttpClient httpclient = HttpClients.custom()
  .setRoutePlanner(routePlanner)
  .build();

Our proxy server is redirecting all requests to the actual service that listens on the 8090 port. In the end of the test, we verify that request was sent to our actual service via a proxy:

proxyMock.stubFor(get(urlMatching(".*"))
  .willReturn(aResponse().proxiedFrom("http://localhost:8089/")));

serviceMock.stubFor(get(urlEqualTo("/private"))
  .willReturn(aResponse().withStatus(200)));

assertEquals(response.getStatusLine().getStatusCode(), 200);
proxyMock.verify(getRequestedFor(urlEqualTo("/private")));
serviceMock.verify(getRequestedFor(urlEqualTo("/private")));

5. Configuring the HTTP Client to Authorize via Proxy

Extending the previous example, there are some cases when the proxy server is used to perform authorization. In such configuration, a proxy can authorize all requests and pass them to the server that is hidden behind a proxy.

We can configure the HttpClient to send each request via proxy, together with the Authorization header that will be used to perform an authorization process.

Suppose that we have a proxy server that authorizes only one user – “username_admin“, with a password “secret_password“.

We need to create the BasicCredentialsProvider instance with credentials of the user that will be authorized via proxy. To make HttpClient automatically add the Authorization header with the proper value, we need to create a HttpClientContext with credentials provided and a BasicAuthCache that stores credentials:

HttpHost proxy = new HttpHost("localhost", 8090);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);

//Client credentials
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(proxy), 
  new UsernamePasswordCredentials("username_admin", "secret_password"));

// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();

BasicScheme basicAuth = new BasicScheme();
authCache.put(proxy, basicAuth);
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credentialsProvider);
context.setAuthCache(authCache);

HttpClient httpclient = HttpClients.custom()
  .setRoutePlanner(routePlanner)
  .setDefaultCredentialsProvider(credentialsProvider)
  .build();

When we set up our HttpClient, making requests to our service will result in sending a request via proxy with an Authorization header to perform authorization process. It will be set in each request automatically.

Let’s execute an actual request to the service:

HttpGet httpGet = new HttpGet("http://localhost:8089/private");
HttpResponse response = httpclient.execute(httpGet, context);

Verifying an execute() method on the httpClient with our configuration confirms that a request went through a proxy with an Authorization header:

proxyMock.stubFor(get(urlMatching("/private"))
  .willReturn(aResponse().proxiedFrom("http://localhost:8089/")));
serviceMock.stubFor(get(urlEqualTo("/private"))
  .willReturn(aResponse().withStatus(200)));

assertEquals(response.getStatusLine().getStatusCode(), 200);
proxyMock.verify(getRequestedFor(urlEqualTo("/private"))
  .withHeader("Authorization", containing("Basic")));
serviceMock.verify(getRequestedFor(urlEqualTo("/private")));

6. Conclusion

This article shows how to configure the Apache HttpClient to perform advanced HTTP calls. We saw how to send requests via a proxy server and how to authorize via proxy.

The implementation of all these examples and code snippets can be found in the GitHub project – this is a Maven project, so it should be easy to import and run as it is.