Introduction to Spring AI

1. Overview

Spring Framework officially enabled the power of AI generative prompts with the Spring AI project. This article aims to provide a robust introduction to the generative AI integration in the Spring Boot applications. Within the tutorial, we’ll familiarize ourselves with the essential AI concepts.

Also, we will gain an understanding of how Spring AI interacts with the models and create an application to demonstrate its capabilities.

2. Spring AI Main Concepts

Before we start, let’s review some key domain terms and concepts.

Spring AI initially focused on models designed to handle language input and generate language output. The idea behind the project was to provide developers with an abstract interface, the foundation for enabling generative AI APIs into the application as an isolated component.

One such abstraction is the interface AiClient, which has two basic implementations — OpenAI and Azure OpenAI.

public interface AiClient {
    default String generate(String message);
    AiResponse generate(Prompt prompt);
}

AiClient provides two options for the generative function. The simplified one – generate(String message) – uses String as input and output, and it could be used to avoid the extra complexity of Promt and AiResponse classes.

Now, let’s take a closer look at their difference.

2.1. Advanced Prompt and AiResponse

In the AI domain, prompt refers to a text message provided to AI. It consists of the context and question, and that model is used for the answer generation.

From the Spring AI project perspective, the Prompt is a list of parametrized Messages. 

public class Prompt {
    private final List<Message> messages;
    // constructors and utility methods 
}
public interface Message {
    String getContent();
    Map<String, Object> getProperties();
    MessageType getMessageType();
}

Prompt enables developers to have more control over the text input. A good example is the prompt templates, constructed with a predefined text and set of placeholders. Then, we may populate them with the Map<String, Object> values passed to the Message constructor.

Tell me a {adjective} joke about {content}.

The Message interface also holds advanced information about the categories of messages that an AI model can process. For example, OpenAI implementation distinguishes between conversational roles, effectively mapped by the MessageType. In the case of other models, it could reflect the message format or some other custom properties. For more details, please refer to the official documentation.

public class AiResponse {
    private final List<Generation> generations;
    // getters and setters
}
public class Generation {
    private final String text;
    private Map<String, Object> info;
}

The AiResponse consists of the list of Generation objects, each holding output from the corresponding prompt. In addition, the Generation object provides metadata information of the AI response.

However, while the Spring AI project is still in beta, not all features are finished and documented. Please follow the progress with the issues on the GitHub repository.

3. Getting Started with the Spring AI Project

First of all, AiClient requires the API key for all communications with the OpenAI platform. For that, we will create a token on the API Keys page.

Spring AI project defines configuration property: spring.ai.openai.api-key. We may set it up in the application.yml file.

spring:
  ai:
    openai.api-key: ${OPEN_AI_KEY}

The next step would be configuring a dependency repository. The Spring AI project provides artifacts in the Spring Milestone Repository.

Therefore, we need to add the repository definition:

<repositories>
    <repository>
        <id>spring-snapshots</id>
        <name>Spring Snapshots</name>
        <url>https://repo.spring.io/snapshot</url>
        <releases>
            <enabled>false</enabled>
        </releases>
    </repository>
</repositories>

After that, we are ready to import open-ai-spring-boot-starter:

<dependency>
    <groupId>org.springframework.experimental.ai</groupId>
    <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
    <version>0.7.1-SNAPSHOT</version>
</dependency>

Please keep in mind that the Spring AI project is actively evolving, so check the official GitHub page for the latest version

That’s all! Now, let’s put the concept into practice.

4. Spring AI in Action

Now, we will write a simple REST API for demonstration purposes. It will consist of two endpoints that return poetry on whatever theme and genre we’d like:

• /ai/cathaiku — will implement the basic generate() method and return a plain string value with Haiku about cats;
• /ai/poetry?theme={{theme}}&genre={{genre}} — will demonstrate capabilities of PromtTemplate and AiResponse classes;

4.1. Injecting AiClient in Spring Boot Application

To keep things simple, let’s start with the cat haiku endpoint. With @RestController annotation, we will set up PoetryController and add GET method mapping:

@RestController
@RequestMapping("ai")
public class PoetryController {
    private final PoetryService poetryService;
    // constructor
    @GetMapping("/cathaiku")
    public ResponseEntity<String> generateHaiku(){
        return ResponseEntity.ok(poetryService.getCatHaiku());
    }
}

Next, following the DDD concept, the service layer would define all domain logic. All we need to do to call the generate() method is inject the AiClient into the PoetryService. Now, we may define the String prompt, where we will specify our request to generate the Haiku.

@Service
public class PoetryServiceImpl implements PoetryService {
    public static final String WRITE_ME_HAIKU_ABOUT_CAT = """
        Write me Haiku about cat,
        haiku should start with the word cat obligatory""";
    private final AiClient aiClient;
    // constructor
    @Override
    public String getCatHaiku() {
        return aiClient.generate(WRITE_ME_HAIKU_ABOUT_CAT);
    }
}

The endpoint is up and ready to receive the requests. The response will contain a plain string:

Cat prowls in the night,
Whiskers twitch with keen delight,
Silent hunter's might.

It looks good so far; however, the current solution has a few pitfalls. The response of plain string isn’t the best solution for REST contracts in the first place.

Furthermore, there is not this much value in querying ChatGPT with the same old prompt all the time. So, our next step would be to add the parametrized values: theme and genre. That’s when PromtTemplate could serve us the best!

4.2. Configurable Queries With PromptTemplate

In its nature, PromptTemplate works quite similarly to a combination of StringBuilder and dictionary. Similarly to /cathaiku endpoint, we will first define the base string for the prompt. Moreover, this time, we will define the placeholders populated with actual values by their names:

String promptString = """
    Write me {genre} poetry about {theme}
    """;
PromptTemplate promptTemplate = new PromptTemplate(promptString);
promptTemplate.add("genre", genre);
promptTemplate.add("theme", theme);

Next, what we may want to do is to standardize the endpoint output. For that, we will introduce the simple record class — PoetryDto, which will contain poetry title, name, and genre:

public record PoetryDto (String title, String poetry, String genre, String theme){}

A further step would be registering PoetryDto in the BeanOutputParser class; it provides functionality to serialize and deserialize OpenAI API output.

Then, we will provide this parser to the promtTemple, and from now on, our messages will be serialized into the DTO objects.

In the end, our generative function would look like this:

@Override
public PoetryDto getPoetryByGenreAndTheme(String genre, String theme) {
    BeanOutputParser<PoetryDto> poetryDtoBeanOutputParser = new BeanOutputParser<>(PoetryDto.class);
    String promptString = """
        Write me {genre} poetry about {theme}
        {format}
    """;
    PromptTemplate promptTemplate = new PromptTemplate(promptString);
    promptTemplate.add("genre", genre);
    promptTemplate.add("theme", theme);
    promptTemplate.add("format", poetryDtoBeanOutputParser.getFormat());
    promptTemplate.setOutputParser(poetryDtoBeanOutputParser);
    AiResponse response = aiClient.generate(promptTemplate.create());
    return poetryDtoBeanOutputParser.parse(response.getGeneration().getText());
}

The response our client would receive now looks much better, and more importantly, it fits into the REST API standards and best practices:

{
    "title": "Dancing Flames",
    "poetry": "In the depths of night, flames dance with grace,
       Their golden tongues lick the air with fiery embrace.
       A symphony of warmth, a mesmerizing sight,
       In their flickering glow, shadows take flight.
       Oh, flames so vibrant, so full of life,
       Burning with passion, banishing all strife.
       They consume with ardor, yet do not destroy,
       A paradox of power, a delicate ploy.
       They whisper secrets, untold and untamed,
       Their radiant hues, a kaleidoscope unnamed.
       In their gentle crackling, stories unfold,
       Of ancient tales and legends untold.
       Flames ignite the heart, awakening desire,
       They fuel the soul, setting it on fire.
       With every flicker, they kindle a spark,
       Guiding us through the darkness, lighting up the dark.
       So let us gather 'round, bask in their warm embrace,
       For in the realm of flames, magic finds its place.
       In their ethereal dance, we find solace and release,
       And in their eternal glow, our spirits find peace.",
    "genre": "Liric",
    "theme": "Flames"
}

5. Error Handling

Spring AI project provides an abstraction over OpenAPI errors with the OpenAiHttpException class. Unfortunately, it does not provide individual mapping of classes per error type. However, thanks to such abstraction, we may handle all exceptions with RestControllerAdvice in one handler.

The code below uses the ProblemDetail standard of the Spring 6 Framework. If you are unfamiliar with it, please check the official documentation.

@RestControllerAdvice
public class ExceptionTranslator extends ResponseEntityExceptionHandler {
    public static final String OPEN_AI_CLIENT_RAISED_EXCEPTION = "Open AI client raised exception";
    @ExceptionHandler(OpenAiHttpException.class)
    ProblemDetail handleOpenAiHttpException(OpenAiHttpException ex) {
        HttpStatus status = Optional
          .ofNullable(HttpStatus.resolve(ex.statusCode))
          .orElse(HttpStatus.BAD_REQUEST);
        ProblemDetail problemDetail = ProblemDetail.forStatusAndDetail(status, ex.getMessage());
        problemDetail.setTitle(OPEN_AI_CLIENT_RAISED_EXCEPTION);
        return problemDetail;
    }
}

Now, if the OpenAPI response contains errors, we will handle it. Here is an example of the response:

{
    "type": "about:blank",
    "title": "Open AI client raised exception",
    "status": 401,
    "detail": "Incorrect API key provided: sk-XG6GW***************************************wlmi. 
       You can find your API key at https://platform.openai.com/account/api-keys.",
    "instance": "/ai/cathaiku"
}

The complete list of possible exception statuses is on the official documentation page.

6. Conclusion

In this article, we familiarized ourselves with the Spring AI Project and its capabilities in the context of REST APIs. Despite the fact that at the time this article was written, spring-ai-starter remained in active development and was accessible in a snapshot version. It provided a reliable interface for generative AI integration into the Spring Boot application.

In the context of this article, we covered both basic and advanced integrations with Spring AI, including how the AiClient works under the hood. As the proof of concept, we implemented a basic REST application that generates poetry. Along with a basic example of a generative endpoint, we provided a sample using advanced Spring AI features: PromtTemplate, AiResponse, and BeanOutputParser. In addition, we implemented the error handling functionality.

The complete examples are available over on GitHub.