Enabling Logging for Apache HttpClient

1. Overview

In this tutorial, we'll show how to enable logging in Apache's HttpClient. Additionally, we'll explain how logging is implemented inside the library. Afterward, we'll show how to enable different levels of logging.

2. Logging Implementation

The HttpClient library provides efficient, up-to-date, and feature-rich implementation client site of the HTTP protocol.

Indeed as a library, HttpClient doesn't force logging implementation. With this intention, version 4.5, provides logs with Commons Logging. Similarly, the latest version, 5.1, uses a logging facade provided by SLF4J. Both versions use a hierarchy schema to match loggers with their configurations.

Thanks to that, it is possible to set up loggers for single classes or for all classes related to the same functionality.

3. Log Types

Let's have a look at log levels defined by the library. We can distinguish 3 types of logs:

• context logging – logs information about all internal operations of HttpClient. It contains wire and header logs as well.
• wire logging – logs only data transmitted to and from the server
• header logging – logs HTTP headers only

In version 4.5 the corresponding packages are org.apache.http.impl.client and org.apache.http.wire, org.apache.http.headers.

Acordingly in version 5.1 the are packages org.apache.hc.client5.http, org.apache.hc.client5.http.wire and org.apache.hc.client5.http.headers.

4. Log4j Configuration

Let's have a look at how to enable logging in both versions.  Our aim is to achieve the same flexibility in both versions. In version 4.1, we'll redirect logs to SLF4j. Thanks to that, different logging frameworks can be used.

4.1. Version 4.5 Configuration

Let's add the httpclient dependency:

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.8</version>
    <exclusions>
        <exclusion>
            <artifactId>commons-logging</artifactId>
            <groupId>commons-logging</groupId>
        </exclusion>
    </exclusions>
</dependency>

We'll use jul-to-slf4j to redirect logs to SLF4J. Therefore we excluded commons-logging. Let's then add a dependency to the bridge between JUL and SLF4J:

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jul-to-slf4j</artifactId>
    <version>1.7.26</version>
</dependency>

Because SLF4J is just a facade, we need a binding. In our example, we'll use logback:

<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.2.3</version>
</dependency>

Let's now create the ApacheHttpClientUnitTest class:

public class ApacheHttpClientUnitTest {
    private final Logger logger = LoggerFactory.getLogger(this.getClass());
    public static final String DUMMY_URL = "https://postman-echo.com/get";
    @Test
    public void whenUseApacheHttpClient_thenCorrect() throws IOException {
        HttpGet request = new HttpGet(DUMMY_URL);
        try (CloseableHttpClient client = HttpClients.createDefault(); CloseableHttpResponse response = client.execute(request)) {
            HttpEntity entity = response.getEntity();
            logger.debug("Response -> {}",  EntityUtils.toString(entity));
        }
    }
}

The test fetches a dummy web page and prints the contents to the log.

Let's now define a logger configuration with our logback.xml file:

<configuration debug="false">
    <appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%date [%level] %logger - %msg %n</pattern>
        </encoder>
    </appender>
    <logger name="com.baeldung.httpclient.readresponsebodystring" level="debug"/>
    <logger name="org.apache.http" level="debug"/>
    <root level="WARN">
        <appender-ref ref="stdout"/>
    </root>
</configuration>

After running our test, all HttpClient's logs can be found in the console:

...
2021-06-19 22:24:45,378 [DEBUG] org.apache.http.impl.execchain.MainClientExec - Executing request GET /get HTTP/1.1 
2021-06-19 22:24:45,378 [DEBUG] org.apache.http.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED 
2021-06-19 22:24:45,379 [DEBUG] org.apache.http.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED 
2021-06-19 22:24:45,382 [DEBUG] org.apache.http.headers - http-outgoing-0 >> GET /get HTTP/1.1 
...

4.2. Version 5.1 Configuration

Let's have a look now at the higher version. It contains redesigned logging. Therefore, instead of Commons Logging, it utilizes SLF4J. As a result, a binding for the logger facade is the only additional dependency. Therefore we'll use logback-classic as in the first example.

Let's add the httpclient5 dependency:

<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5</artifactId>
    <version>5.1</version>
</dependency>

Let's add a similar test as in the previous example:

public class ApacheHttpClient5UnitTest {
    private final Logger logger = LoggerFactory.getLogger(this.getClass());
    public static final String DUMMY_URL = "https://postman-echo.com/get";
    @Test
    public void whenUseApacheHttpClient_thenCorrect() throws IOException, ParseException {
        HttpGet request = new HttpGet(DUMMY_URL);
        try (CloseableHttpClient client = HttpClients.createDefault(); CloseableHttpResponse response = client.execute(request)) {
            HttpEntity entity = response.getEntity();
            logger.debug("Response -> {}", EntityUtils.toString(entity));
        }
    }
}

Next, we need to add a logger to the logback.xml file:

<configuration debug="false">
...
    <logger name="org.apache.hc.client5.http" level="debug"/>
...
</configuration>

Let's run the test class ApacheHttpClient5UnitTest and check the output. It is similar to the old version:

...
2021-06-19 22:27:16,944 [DEBUG] org.apache.hc.client5.http.impl.classic.InternalHttpClient - ep-0000000000 endpoint connected 
2021-06-19 22:27:16,944 [DEBUG] org.apache.hc.client5.http.impl.classic.MainClientExec - ex-0000000001 executing GET /get HTTP/1.1 
2021-06-19 22:27:16,944 [DEBUG] org.apache.hc.client5.http.impl.classic.InternalHttpClient - ep-0000000000 start execution ex-0000000001 
2021-06-19 22:27:16,944 [DEBUG] org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager - ep-0000000000 executing exchange ex-0000000001 over http-outgoing-0 
2021-06-19 22:27:16,960 [DEBUG] org.apache.hc.client5.http.headers - http-outgoing-0 >> GET /get HTTP/1.1 
...

5. Conclusion

That concludes this short tutorial on how to configure logging for Apache's HttpClient. Firstly, we explained how logging is implemented in the library. Secondly, we configured logging in two versions and executed simple test cases to show the output.

As always, the source code of the example is available over on GitHub.