Introducing KivaKit

1. Overview

KivaKit is a modular Java application framework designed to make developing microservices and applications quicker and easier. KivaKit has been developed at Telenav since 2011. It is now available as an Apache Licensed, Open Source project on GitHub.

In this article, we'll explore the design of KivaKit as a collection of “mini-frameworks” that work together. In addition, we'll take a look at the essential features of each mini-framework.

2. KivaKit Mini-frameworks

Looking in the kivakit and kivakit-extensions repositories, we can see that KivaKit 1.0 contains 54 modules. We could find this overwhelming. However, if we take things one step at a time, it's not so bad. For starters, we can pick and choose what we want to include in our projects. Each module in KivaKit is designed to be used on its own.

Some KivaKit modules contain mini-frameworks. A mini-framework is a simple, abstract design that addresses a common problem. If we examine KivaKit's mini-frameworks, we will find that they have straightforward, broadly applicable interfaces. As a result, they are a bit like Legos. That is to say, they are simple pieces that snap together.

Here, we can see KivaKit's mini-frameworks and how they relate to each other:

We can find these frameworks in the kivakit repository. On the other hand, we will find less important modules like service providers in kivakit-extensions.

2.1. Messaging

As the diagram above shows, messaging is the central point of integration. In KivaKit, messaging formalizes status reporting. As Java programmers, we're used to status information being logged. Sometimes, we also see this information returned to callers or thrown as exceptions. By contrast, status information in KivaKit is contained in Messages. We can write components that broadcast these messages. Also, we can write components that listen to them.

We can see that this design allows components to focus on reporting status consistently. To a component, where status messages go is unimportant. In one case, we might direct messages to a Logger. In another, we might include them in a statistic. We could even display them to an end-user. The component doesn't care. It just reports issues to whoever might be interested.

KivaKit components that broadcast messages can be connected to one or more message repeaters, forming a listener chain:

In KivaKit, an Application is usually the end of a listener chain. Therefore, the Application class logs any messages it receives. Along the way, components in the listener chain may have other uses for these messages.

2.2. Mixins

Another integration feature of KivaKit is the mixins mini-framework. KivaKit mixins allow base classes to be “mixed in” to types through interface inheritance.  Sometimes, mixins are known as “stateful traits“.

For example, the BaseComponent class in KivaKit provides a foundation for building components. BaseComponent provides convenience methods for sending messages. In addition, it offers easy access to resources, settings, and registered objects.

But we quickly run into a problem with this design. As we know, in Java, a class that already has a base class cannot also extend BaseComponent. KivaKit mixins allow BaseComponent functionality to be added to a component that already has a base class. For example:

public class MyComponent extends MyBaseClass implements ComponentMixin { [...] }

We can see here that the interface ComponentMixin extends both Mixin and Component:

The Mixin interface provides a state() method to ComponentMixin. First, this method is used to create and associate a BaseComponent with the object implementing ComponentMixin. Second, ComponentMixin implements each method of the Component interface as a Java default method. Third, each default method delegates to the associated BaseComponent. In this way, implementing ComponentMixin provides the same methods as extending BaseComponent.

2.3. Service Locator

The service locator class Registry allows us to wire components together. A Registry provides roughly the same functionality as dependency injection (DI). However, it differs from the typical use of DI in one important way. In the service locator pattern, components reach out for the interfaces that they need. On the other hand, DI pushes interfaces into components. As a result, the service locator approach improves encapsulation. It also reduces the scope of references. For example, Registry can be used neatly within a method:

class MyData extends BaseComponent {
    [...]
    public void save() {
        var database = require(Database.class);
        database.save(this);
    }
}

The BaseComponent.require(Class) method here looks up objects in a Registry. When our save() method returns, the database reference leaves scope. This ensures that no external code can obtain our reference.

When our application starts, we can register service objects with one of the BaseComponent.registerObject() methods. Later, code elsewhere in our application can look them up with require(Class).

2.4. Resources and Filesystems

The kivakit-resource module provides abstractions for reading and writing streamed resources and accessing filesystems. We can see here a few of the more important resource types that KivaKit includes:

• Files (local, Zip, S3, and HDFS)
• Package resources
• Network protocols (sockets, HTTP, HTTPS, and FTP)
• Input and output streams

We get two valuable benefits from this abstraction. We can:

• Access any streamed resource through a consistent, object-oriented API
• Use the Resource and WritableResource interfaces to allow the use of unknown resource types

2.5. Components

The kivakit-component module gives us ready access to common functionality. We can:

• Send and receive messages
• Access packages and packaged resources
• Register and lookup objects, components, and settings

The Component interface is implemented by both BaseComponent and ComponentMixin. As a result, we can add “component nature” to any object.

2.6. Logging

Listener chains formed by KivaKit components often terminate in a Logger. A Logger writes the messages it receives to one or more Logs. In addition, the kivakit-kernel module provides a service provider interface (SPI) for implementing Logs. We can see the full design of the logging mini-framework in UML here.

Using the logging SPI, the kivakit-extensions repository provides us with some Log implementations:

One or more logs can be selected and configured from the command line. This is done by defining the KIVAKIT_LOG system property.

2.7. Conversion and Validation

The kivakit-kernel module contains mini-frameworks for type conversion and object validation. These frameworks are integrated with KivaKit messaging. This allows them to report problems consistently. It also simplifies usage. To implement a type Converter like a StringConverter, we need to write the conversion code. We don't need to worry about exceptions, empty strings, or null values.

We can see converters in use in many places in KivaKit, including:

• Switch and argument parsing
• Loading settings objects from properties files
• Formatting objects as debug strings
• Reading objects from CSV files

Messages broadcast by Validatables are captured by the validation mini-framework. Subsequently, they are analyzed to provide us with easy access to error statistics and validation problems.

2.8. Applications, Command Lines, and Settings

The kivakit-application, kivakit-configuration, and kivakit-commandline modules provide a simple, consistent model for developing applications.

The kivakit-application project supplies the Application base class. An Application is a Component. It provides settings information using kivakit-configuration. In addition, it provides command-line parsing with kivakit-commandline.

The kivakit-configuration project uses the kivakit-resource module to load settings information from .properties resources (and other sources in the future). It converts the properties in these resources to objects using kivakit-kernel converters. The converted objects are then validated with the validation mini-framework.

Command-line arguments and switches for an application are parsed by the kivakit-commandline module using KivaKit converters and validators. We can see issues that arise in the application's ConsoleLog.

2.9. Microservices

So far, we've discussed KivaKit features that are generally useful to any application. In addition, KivaKit also provides functionality in kivakit-extensions that is explicitly targeted at microservices. Let's take a quick look at kivakit-web.

The kivakit-web project contains modules for rapidly developing a simple REST and web interface to a microservice. The JettyServer class provides us with a way to plug in servlets and filters with a minimum of hassle. Plugins that we can use with JettyServer include:

These plugins can be combined to provide a RESTful microservice with Swagger documentation and a web interface:

var application = new MyRestApplication();
listenTo(new JettyServer())
    .port(8080)
    .add("/*", new JettyWicket(MyWebApplication.class))
    .add("/open-api/*", new JettySwaggerOpenApi(application))
    .add("/docs/*", new JettySwaggerIndex(port))
    .add("/webapp/*", new JettySwaggerStaticResources())
    .add("/webjar/*", new JettySwaggerWebJar(application))
    .add("/*", new JettyJersey(application))
    .start();

KivaKit 1.1 will include a dedicated microservices mini-framework. This will make it even easier for us to build microservices.

3. Documentation and Lexakai

The documentation for KivaKit is generated by Lexakai. Lexakai creates UML diagrams (guided by annotations when desired) and updates README.md markdown files. In the readme file for each project, Lexakai updates a standard header and footer. In addition, it maintains indexes for the generated UML diagrams and Javadoc documentation. Lexakai is an open-source project distributed under Apache License.

4. Building KivaKit

KivaKit targets a Java 11 or higher virtual machine (but can be used from Java 8 source code). We can find all the artifacts for KivaKit modules on Maven Central. However, we might want to modify KivaKit or contribute to the open-source project. In this case, we'll need to build it.

To get started, let's set up Git, Git Flow, Java 16 JDK, and Maven 3.8.1 or higher.

First, we clone the kivakit repository into our workspace:

mkdir ~/Workspace
cd ~/Workspace
git clone --branch develop https://github.com/Telenav/kivakit.git

Next, we copy the sample bash profile to our home folder:

cp kivakit/setup/profile ~/.profile

Then we modify ~/.profile to point to our workspace, and our Java and Maven installations:

export KIVAKIT_WORKSPACE=$HOME/Workspace 
export JAVA_HOME=/Library/Java/JavaVirtualMachines/temurin-16.jdk/Contents/Home 
export M2_HOME=$HOME/Developer/apache-maven-3.8.2

After our profile is set up, we ensure that we are running bash (on macOS, zsh is the default now):

chsh -s /bin/bash

And finally, we restart our terminal program and execute the command:

$KIVAKIT_HOME/setup/setup.sh

The setup script will clone kivakit-extensions and some other related repositories. Afterward, it will initialize git-flow and build all our KivaKit projects.

5. Conclusion

In this article, we took a brief look at the design of KivaKit. We also toured some of the more important functions it provides. KivaKit is ideally suited for developing microservices. It has been designed to be learned and used in easy-to-digest, independent pieces.