Log4j 2 Plugins

1. Overview

Log4j 2 uses plugins like Appenders and Layouts to format and output logs. These are known as core plugins, and Log4j 2 provides a lot of options for us to choose from.

However, in some cases, we may also need to extend the existing plugin or even write custom ones.

In this tutorial, we'll use the Log4j 2 extension mechanism to implement custom plugins.

2. Extending Log4j 2 Plugins

Plugins in Log4j 2 are broadly divided into five categories:

1. Core Plugins
2. Convertors
3. Key Providers
4. Lookups
5. Type Converters

Log4j 2 allows us to implement custom plugins in all the above categories using a common mechanism. Moreover, it also allows us to extend existing plugins with the same approach.

In Log4j 1.x, the only way to extend an existing plugin is to override its implementation class. On the other hand, Log4j 2 makes it easier to extend existing plugins by annotating a class with @Plugin.

In the following sections, we'll implement a custom plugin in a few of these categories.

3. Core Plugin

3.1. Implementing a Custom Core Plugin

Key elements like Appenders, Layouts, and Filters are known as core plugins in Log4j 2. Although there is a diverse list of such plugins, in some cases, we may need to implement a custom core plugin. For example, consider a ListAppender that only writes log records into an in-memory List:

@Plugin(name = "ListAppender", 
  category = Core.CATEGORY_NAME, 
  elementType = Appender.ELEMENT_TYPE)
public class ListAppender extends AbstractAppender {

    private List<LogEvent> logList;

    protected ListAppender(String name, Filter filter) {
        super(name, filter, null);
        logList = Collections.synchronizedList(new ArrayList<>());
    }

    @PluginFactory
    public static ListAppender createAppender(
      @PluginAttribute("name") String name, @PluginElement("Filter") final Filter filter) {
        return new ListAppender(name, filter);
    }

    @Override
    public void append(LogEvent event) {
        if (event.getLevel().isLessSpecificThan(Level.WARN)) {
            error("Unable to log less than WARN level.");
            return;
        }
        logList.add(event);
    }
}

We have annotated the class with @Plugin that allows us to name our plugin. Also, the parameters are annotated with @PluginAttribute. The nested elements like filter or layout are passed as @PluginElement. Now we can refer this plugin in the configuration using the same name:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns:xi="http://www.w3.org/2001/XInclude"
    packages="com.baeldung" status="WARN">
    <Appenders>
        <ListAppender name="ListAppender">
            <BurstFilter level="INFO" rate="16" maxBurst="100"/>
        </MapAppender>
    </Appenders>
    <Loggers
        <Root level="DEBUG">
            <AppenderRef ref="ConsoleAppender" />
            <AppenderRef ref="ListAppender" />
        </Root>
    </Loggers>
</Configuration>

3.2. Plugin Builders

The example in the last section is rather simple and only accepts a single parameter name. Generally speaking, core plugins like appenders are much more complex and usually accepts several configurable parameters.

For example, consider an appender that writes logs into Kafka:

<Kafka2 name="KafkaLogger" ip ="127.0.0.1" port="9010" topic="log" partition="p-1">
    <PatternLayout pattern="%pid%style{%message}{red}%n" />
</Kafka2>

To implement such appenders, Log4j 2 provides a plugin builder implementation based on the Builder pattern:

@Plugin(name = "Kafka2", category = Core.CATEGORY_NAME)
public class KafkaAppender extends AbstractAppender {

    public static class Builder implements org.apache.logging.log4j.core.util.Builder<KafkaAppender> {

        @PluginBuilderAttribute("name")
        @Required
        private String name;

        @PluginBuilderAttribute("ip")
        private String ipAddress;

        // ... additional properties

        // ... getters and setters

        @Override
        public KafkaAppender build() {
            return new KafkaAppender(
              getName(), getFilter(), getLayout(), true, new KafkaBroker(ipAddress, port, topic, partition));
        }
    }

    private KafkaBroker broker;

    private KafkaAppender(String name, Filter filter, Layout<? extends Serializable> layout, 
      boolean ignoreExceptions, KafkaBroker broker) {
        super(name, filter, layout, ignoreExceptions);
        this.broker = broker;
    }

    @Override
    public void append(LogEvent event) {
        connectAndSendToKafka(broker, event);
    }
}

In short, we introduced a Builder class and annotated the parameters with @PluginBuilderAttribute. Because of this, KafkaAppender accepts the Kafka connection parameters from the config shown above.

3.3. Extending an Existing Plugin

We can also extend an existing core plugin in Log4j 2. We can achieve this by giving our plugin the same name as an existing plugin. For example, if we're extending the RollingFileAppender:

@Plugin(name = "RollingFile", category = Core.CATEGORY_NAME, elementType = Appender.ELEMENT_TYPE)
public class RollingFileAppender extends AbstractAppender {

    public RollingFileAppender(String name, Filter filter, Layout<? extends Serializable> layout) {
        super(name, filter, layout);
    }
    @Override
    public void append(LogEvent event) {
    }
}

Notably, we now have two appenders with the same name. In such a scenario, Log4j 2 will use the appender that is discovered first. We'll see more on plugin discovery in a later section.

Please note that Log4j 2 discourages multiple plugins with the same name. It's better to implement a custom plugin instead and use that in the logging configuration.

4. Converter Plugin

The layout is a powerful plugin in Log4j 2. It allows us to define the output structure for our logs. For instance, we can use JsonLayout for writing the logs in JSON format.

Another such plugin is the PatternLayout. In some cases, an application wants to publish information like thread id, thread name, or timestamp with each log statement. PatternLayout plugin allows us to embed such details through a conversion pattern string in the configuration:

<Configuration status="debug" name="baeldung" packages="">
    <Appenders>
        <Console name="stdout" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} %p %m%n"/>
        </Console>
    </Appenders>
</Configuration>

Here, %d is the conversion pattern. Log4j 2 converts this %d pattern through a DatePatternConverter that understands the conversion pattern and replaces it with the formatted date or timestamp.

Now suppose an application running inside a Docker container wants to print the container name with every log statement. To do this, we'll implement a DockerPatterConverter and change the above config to include the conversion string:

@Plugin(name = "DockerPatternConverter", category = PatternConverter.CATEGORY)
@ConverterKeys({"docker", "container"})
public class DockerPatternConverter extends LogEventPatternConverter {

    private DockerPatternConverter(String[] options) {
        super("Docker", "docker");
    }

    public static DockerPatternConverter newInstance(String[] options) {
        return new DockerPatternConverter(options);
    }

    @Override
    public void format(LogEvent event, StringBuilder toAppendTo) {
        toAppendTo.append(dockerContainer());
    }

    private String dockerContainer() {
        return "container-1";
    }
}

So we implemented a custom DockerPatternConverter similar to the date pattern. It will replace the conversion pattern with the name of the Docker container.

This plugin is similar to the core plugin we implemented earlier. Notably, there is just one annotation that is different from the last plugin. @ConverterKeys annotation accepts the conversion pattern for this plugin.

As a result, this plugin will convert %docker or %container pattern string into the container name in which the application is running:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns:xi="http://www.w3.org/2001/XInclude" packages="com.baeldung" status="WARN">
    <Appenders>
        <xi:include href="log4j2-includes/console-appender_pattern-layout_colored.xml" />
        <Console name="DockerConsoleLogger" target="SYSTEM_OUT">
            <PatternLayout pattern="%pid %docker %container" />
        </Console>
    </Appenders>
    <Loggers>
        <Logger name="com.baeldung.logging.log4j2.plugins" level="INFO">
            <AppenderRef ref="DockerConsoleLogger" />
        </Logger>
    </Loggers>
</Configuration>

5. Lookup Plugin

Lookup plugins are used to add dynamic values in the Log4j 2 configuration file. They allow applications to embed runtime values to some properties in the configuration file. The value is added through a key-based lookup in various sources like a file system, database, etc.

One such plugin is the DateLookupPlugin that allows replacing a date pattern with the current system date of the application:

<RollingFile name="Rolling-File" fileName="${filename}" 
  filePattern="target/rolling1/test1-$${date:MM-dd-yyyy}.%i.log.gz">
    <PatternLayout>
        <pattern>%d %p %c{1.} [%t] %m%n</pattern>
    </PatternLayout>
    <SizeBasedTriggeringPolicy size="500" />
</RollingFile>

In this sample configuration file, RollingFileAppender uses a date lookup where the output will be in MM-dd-yyyy format. As a result, Log4j 2 writes logs to an output file with a date suffix.

Similar to other plugins, Log4j 2 provides a lot of sources for lookups. Moreover, it makes it easy to implement custom lookups if a new source is required:

@Plugin(name = "kafka", category = StrLookup.CATEGORY)
public class KafkaLookup implements StrLookup {

    @Override
    public String lookup(String key) {
        return getFromKafka(key);
    }

    @Override
    public String lookup(LogEvent event, String key) {
        return getFromKafka(key);
    }

    private String getFromKafka(String topicName) {
        return "topic1-p1";
    }
}

So KafkaLookup will resolve the value by querying a Kafka topic. We'll now pass the topic name from the configuration:

<RollingFile name="Rolling-File" fileName="${filename}" 
  filePattern="target/rolling1/test1-$${kafka:topic-1}.%i.log.gz">
    <PatternLayout>
        <pattern>%d %p %c{1.} [%t] %m%n</pattern>
    </PatternLayout>
    <SizeBasedTriggeringPolicy size="500" />
</RollingFile>

We replaced the date lookup in our earlier example with Kafka lookup that will query topic-1.

Since Log4j 2 only calls the default constructor of a lookup plugin, we didn't implement the @PluginFactory as we did in earlier plugins.

6. Plugin Discovery

Finally, let's understand how Log4j 2 discovers the plugins in an application. As we saw in the examples above, we gave each plugin a unique name. This name acts as a key, which Log4j 2 resolves to a plugin class.

There's a specific order in which Log4j 2 performs a lookup to resolve a plugin class:

1. Serialized plugin listing file in the log4j2-core library. Specifically, a Log4j2Plugins.dat is packaged inside this jar to list the default Log4j 2 plugins
2. Similar Log4j2Plugins.dat file from the OSGi bundles
3. A comma-separated package list in the log4j.plugin.packages system property
4. In programmatic Log4j 2 configuration, we can call PluginManager.addPackages() method to add a list of package names
5. A comma-separated list of packages can be added in the Log4j 2 configuration file

As a prerequisite, annotation processing must be enabled to allow Log4j 2 to resolve plugin by the name given in the @Plugin annotation.

Since Log4j 2 uses names to look up the plugin, the above order becomes important. For example, if we have two plugins with the same name, Log4j 2 will discover the plugin that is resolved first. Therefore, if we need to extend an existing plugin in Log4j 2, we must package the plugin in a separate jar and place it before the log4j2-core.jar.

7. Conclusion

In this article, we looked at the broad categories of plugins in Log4j 2. We discussed that even though there is an exhaustive list of existing plugins, we may need to implement custom plugins for some use cases.

Later, we looked at the custom implementation of some useful plugins. Furthermore, we saw how Log4j 2 allows us to name these plugins and subsequently use this plugin name in the configuration file. Finally, we discussed how Log4j 2 resolves plugins based on this name.

As always, all examples are available over on GitHub.