1. Introduction

The aim of this series is to explain the idea of genetic algorithms.

Genetic algorithms are designed to solve problems by using the same processes as in nature — they use a combination of selection, recombination, and mutation to evolve a solution to a problem.

Let’s start by explaining the concept of those algorithms using the simplest binary genetic algorithm example.

2. How Genetic Algorithms Work

Genetic algorithms are a part of evolutionary computing, which is a rapidly growing area of artificial intelligence.

An algorithm starts with a set of solutions (represented by individuals) called population. Solutions from one population are taken and used to form a new population, as there is a chance that the new population will be better than the old one.

Individuals that are chosen to form new solutions (offspring) are selected according to their fitness — the more suitable they are, the more chances they have to reproduce.

3. Binary Genetic Algorithm

Let’s take a look at the basic process for a simple genetic algorithm.

3.1. Initialization

In the initialization step, we generate a random Population that serves as a first solution. First, we need to decide how big the Population will be and what is the final solution that we expect:

SimpleGeneticAlgorithm.runAlgorithm(50,
  "1011000100000100010000100000100111001000000100000100000000001111");

In the above example, the Population size is 50, and the correct solution is represented by the binary bit string that we may change at any time.

In the next step we are going to save our desired solution and create a random Population:

setSolution(solution);
Population myPop = new Population(populationSize, true);

Now we are ready to run the main loop of the program.

3.2. Fitness Check

In the main loop of the program, we are going to evaluate each Individual by the fitness function (in simple words, the better the Individual is, the higher value of fitness function it gets):

while (myPop.getFittest().getFitness() < getMaxFitness()) {
    System.out.println(
      "Generation: " + generationCount
      + " Correct genes found: " + myPop.getFittest().getFitness());
    
    myPop = evolvePopulation(myPop);
    generationCount++;
}

Let’s start with explaining how we get the fittest Individual:

public int getFitness(Individual individual) {
    int fitness = 0;
    for (int i = 0; i < individual.getDefaultGeneLength()
      && i < solution.length; i++) {
        if (individual.getSingleGene(i) == solution[i]) {
            fitness++;
        }
    }
    return fitness;
}

As we can observe, we compare two Individual objects bit by bit. If we cannot find a perfect solution, we need to proceed to the next step, which is an evolution of the Population.

3.3. Offspring

In this step, we need to create a new Population. First, we need to Select two parent Individual objects from a Population, according to their fitness. Please note that it is beneficial to allow the best Individual from the current generation to carry over to the next, unaltered. This strategy is called an Elitism:

if (elitism) {
    newPopulation.getIndividuals().add(0, pop.getFittest());
    elitismOffset = 1;
} else {
    elitismOffset = 0;
}

In order to select two best Individual objects, we are going to apply tournament selection strategy:

private Individual tournamentSelection(Population pop) {
    Population tournament = new Population(tournamentSize, false);
    for (int i = 0; i < tournamentSize; i++) {
        int randomId = (int) (Math.random() * pop.getIndividuals().size());
        tournament.getIndividuals().add(i, pop.getIndividual(randomId));
    }
    Individual fittest = tournament.getFittest();
    return fittest;
}

The winner of each tournament (the one with the best fitness) is selected for the next stage, which is Crossover:

private Individual crossover(Individual indiv1, Individual indiv2) {
    Individual newSol = new Individual();
    for (int i = 0; i < newSol.getDefaultGeneLength(); i++) {
        if (Math.random() <= uniformRate) {
            newSol.setSingleGene(i, indiv1.getSingleGene(i));
        } else {
            newSol.setSingleGene(i, indiv2.getSingleGene(i));
        }
    }
    return newSol;
}

In the crossover, we swap bits from each chosen Individual at a randomly chosen spot. The whole process runs inside the following loop:

for (int i = elitismOffset; i < pop.getIndividuals().size(); i++) {
    Individual indiv1 = tournamentSelection(pop);
    Individual indiv2 = tournamentSelection(pop);
    Individual newIndiv = crossover(indiv1, indiv2);
    newPopulation.getIndividuals().add(i, newIndiv);
}

As we can see, after the crossover, we place new offspring in a new Population. This step is called the Acceptance.

Finally, we can perform a Mutation. Mutation is used to maintain genetic diversity from one generation of a Population to the next. We used the bit inversion type of mutation, where random bits are simply inverted:

private void mutate(Individual indiv) {
    for (int i = 0; i < indiv.getDefaultGeneLength(); i++) {
        if (Math.random() <= mutationRate) {
            byte gene = (byte) Math.round(Math.random());
            indiv.setSingleGene(i, gene);
        }
    }
}

All types of the Mutation and the Crossover are nicely described in this tutorial.

We then repeat steps from subsections 3.2 and 3.3, until we reach a termination condition, for example, the best solution.

4. Tips and Tricks

In order to implement an efficient genetic algorithm, we need to tune a set of parameters. This section should give you some basic recommendations how to start with the most importing parameters:

• Crossover rate – it should be high, about 80%-95%
• Mutation rate – it should be very low, around 0.5%-1%.
• Population size – good population size is about 20-30, however, for some problems sizes 50-100 are better
• Selection – basic roulette wheel selection can be used with the concept of elitism
• Crossover and mutation type – it depends on encoding and the problem

Please note that recommendations for tuning are often results of empiric studies on genetic algorithms, and they may vary, based on the proposed problems.

5. Conclusion

This tutorial introduces fundamentals of genetic algorithms. You can learn about genetic algorithms without any previous knowledge of this area, having only basic computer programming skills.

The complete source code for the code snippets in this tutorial is available in the GitHub project.

Please also note that we use Lombok to generate getters and setters. You can check how to configure it correctly in your IDE in this article.

For further examples of genetic algorithms, check out all the articles of our series:

• How to Design a Genetic Algorithm? (this one)
• The Traveling Salesman Problem in Java

1. Introduction

In this tutorial, we will see how we can configure Spring Security to work with two different login pages using two different Spring Security http elements in the configuration.

2. Configuring 2 Http Elements

One of the situations in which we may need two login pages is when we have one page for administrators of an application and a different page for normal users.

We will configure two http elements that will be differentiated by the URL pattern associated with each:

• /user* for pages that will need a normal user authentication to be accessed
• /admin* for pages that will be accessed by an administrator

Each http element will have a different login page and a different login processing URL.

In order to configure two different http elements, let’s create two static classes annotated with @Configuration that extend the WebSecurityConfigurerAdapter.

Both will be placed inside a regular @Configuration class:

@Configuration
@EnableWebSecurity
public class SecurityConfig {
    ...
}

Let’s define the WebSecurityConfigurerAdapter for the “ADMIN” users:

@Configuration
@Order(1)
public static class App1ConfigurationAdapter extends WebSecurityConfigurerAdapter {
    public App1ConfigurationAdapter() {
        super();
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.antMatcher("/admin*")
          .authorizeRequests()
          .anyRequest()
          .hasRole("ADMIN")
          
          .and()
          .formLogin()
          .loginPage("/loginAdmin")
          .loginProcessingUrl("/admin_login")
          .failureUrl("/loginAdmin?error=loginError")
          .defaultSuccessUrl("/adminPage")
          
          .and()
          .logout()
          .logoutUrl("/admin_logout")
          .logoutSuccessUrl("/protectedLinks")
          .deleteCookies("JSESSIONID")
          
          .and()
          .exceptionHandling()
          .accessDeniedPage("/403")
          
          .and()
          .csrf().disable();
    }
}

And now, let’s define the WebSecurityConfigurerAdapter for normal users:

@Configuration
@Order(2)
public static class App2ConfigurationAdapter extends WebSecurityConfigurerAdapter {

    public App2ConfigurationAdapter() {
        super();
    }

    protected void configure(HttpSecurity http) throws Exception {
        http.antMatcher("/user*")
          .authorizeRequests()
          .anyRequest()
          .hasRole("USER")
          
          .and()
          .formLogin()
          .loginPage("/loginUser")
          .loginProcessingUrl("/user_login")
          .failureUrl("/loginUser?error=loginError")
          .defaultSuccessUrl("/userPage")
          
          .and()
          .logout()
          .logoutUrl("/user_logout")
          .logoutSuccessUrl("/protectedLinks")
          .deleteCookies("JSESSIONID")
          
          .and()
          .exceptionHandling()
          .accessDeniedPage("/403")
          
          .and()
          .csrf().disable();
    }
}

Note that by placing the @Order annotation on each static class, we are specifying the order in which the two classes will be considered based on the pattern matching when a URL is requested.

Two configuration classes cannot have the same order.

3. Custom Login Pages

We will create our own custom login pages for each type of user. For the administrator user, the login form will have a “user_login” action, as defined in the configuration:

<p>User login page</p>
<form name="f" action="user_login" method="POST">
    <table>
        <tr>
            <td>User:</td>
            <td><input type="text" name="username" value=""></td>
        </tr>
        <tr>
            <td>Password:</td>
            <td><input type="password" name="password" /></td>
        </tr>
        <tr>
            <td><input name="submit" type="submit" value="submit" /></td>
        </tr>
    </table>
</form>

The administrator login page is similar, except the form will have an action of “admin_login” as per the java configuration.

4. Authentication Configuration

Now we need to configure authentication for our application. Let’s look at two ways to accomplish this — one using a common source for user authentication, and the other using two separate sources.

4.1. Using a Common User Authentication Source

If both login pages share a common source for authenticating users, you can create a single bean of type UserDetailsService that will handle the authentication.

Let’s demonstrate this scenario using an InMemoryUserDetailsManager that defines two users — one with a role of “USER” and one with a role of “ADMIN”:

@Bean
public UserDetailsService userDetailsService() throws Exception {
    InMemoryUserDetailsManager manager = new InMemoryUserDetailsManager();
    manager.createUser(User
      .withUsername("user")
      .password("userPass")
      .roles("USER")
      .build());
    
    manager.createUser(User
      .withUsername("admin")
      .password("adminPass")
      .roles("ADMIN")
      .build());
    
    return manager;
}

4.2. Using Two Different User Authentication Sources

If you have different sources for user authentication — one for administrators and one for normal users — you can configure an AuthenticationManagerBuilder inside each static @Configuration class. Let’s look at an example of an authentication manager for an “ADMIN” user:

@Configuration
@Order(1)
public static class App1ConfigurationAdapter extends WebSecurityConfigurerAdapter { 
    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.inMemoryAuthentication()
          .withUser("admin")
          .password("admin")
          .roles("ADMIN");
    }
}

In this case, the UserDetailsService bean from the previous section will no longer be used.

6. Conclusion

In this quick tutorial, we’ve shown how to implement two different login pages in the same Spring Security application.

The complete code for this article can be found in the GitHub project.

When you run the application, you can access the examples above on the /protectedLinks URI.

1. Overview

In this article, we will be looking at a java.util.Stream API and we’ll see how we can use that construct to operate on an infinite stream of data/elements.

The possibility of working on the infinite sequence of elements is predicated on the fact that streams are built to be lazy.

This laziness is achieved by a separation between two types of the operations that could be executed on streams: intermediate and terminal operations.

2. Intermediate and Terminal Operations

All Stream operations are divided into intermediate and terminal operations and are combined to form stream pipelines.

A stream pipeline consists of a source (such as a Collection, an array, a generator function, an I/O channel, or infinite sequence generator); followed by zero or more intermediate operations and a terminal operation.

2.1. Intermediate Operations

Intermediate operations are not executed unit some terminal operation is invoked.

They’re composed forming a pipeline of a Stream execution. The intermediate operation can be added to a Stream pipeline by methods:

• filter()
• map()
• flatMap()
• distinct()
• sorted()
• peek()
• limit()
• skip()

All Intermediate operations are lazy, so they’re not executed until a result of a processing is actually needed.

Basically, intermediate operations return a new stream. Executing an intermediate operation does not actually perform any operation, but instead creates a new stream that, when traversed, contains the elements of the initial stream that match the given predicate.

As such, traversal of the Stream doesn’t begin until the terminal operation of the pipeline is executed.

That is very important property, specifically important for infinite streams – because it allows us to create streams that will be actually invoked only when a Terminal operation is called.

2.2. Terminal Operations

Terminal operations may traverse the stream to produce a result or a side effect.

After the terminal operation is performed, the stream pipeline is considered consumed, and can no longer be used. In almost all cases, terminal operations are eager, completing their traversal of the data source and processing of the pipeline before returning.

The eagerness of a terminal operation is important concerning infinite streams because at the moment of processing we need to think carefully if our Stream is properly bounded by, for example, a limit() transformation. Terminal operations are:

• forEach()
• forEachOrdered()
• toArray()
• reduce()
• collect()
• min()
• max()
• count()
• anyMatch()
• allMatch()
• noneMatch()
• findFirst()
• findAny()

Each of these operations will trigger execution of all intermediate operations.

3. Infinite Streams

Now that we understand these two concepts – Intermediate and Terminal operations – we’re able to write an infinite stream that leverage laziness of Streams.

Let’s say that we want to create an infinite stream of elements from zero that will be incremented by two. Then we need to limit that sequence before calling terminal operation.

It is crucial to  use a limit() method before executing a collect() method that is a terminal operation, otherwise our program will run indefinitely:

// given
Stream<Integer> infiniteStream = Stream.iterate(0, i -> i + 2);

// when
List<Integer> collect = infiniteStream
  .limit(10)
  .collect(Collectors.toList());

// then
assertEquals(collect, Arrays.asList(0, 2, 4, 6, 8, 10, 12, 14, 16, 18));

We created an infinite stream using an iterate() method. Then we called a limit() transformation and a collect() terminal operation. Then in our resulting List, we will have first 10 elements of an infinite sequence due to a laziness of a Stream.

4. Infinite Stream of a Custom Type of Elements

Let’s say that we want to create an infinite stream of random UUIDs.

The first step to achieving this using Stream API is to create a Supplier of those random values:

Supplier<UUID> randomUUIDSupplier = UUID::randomUUID;

When we define a supplier we can create an infinite stream using a generate() method:

Stream<UUID> infiniteStreamOfRandomUUID = Stream.generate(randomUUIDSupplier);

Then we could take a couple of elements from that stream. We need to remember to use a limit() method if we want our program to finish in a finite time:

List<UUID> randomInts = infiniteStreamOfRandomUUID
  .skip(10)
  .limit(10)
  .collect(Collectors.toList());

We use a skip() transformation to discard first 10 results and take the next 10 elements. We can create an infinite stream of any custom type elements by passing a function of a Supplier interface to a generate() method on a Stream.

6. Do-While – the Stream Way

Let’s say that we have a simple do..while loop in our code:

int i = 0;
while (i < 10) {
    System.out.println(i);
    i++;
}

We are printing i counter ten times. We can expect that such construct can be easily written using Stream API and ideally, we would have a doWhile() method on a stream.

Unfortunately, there is no such method on a stream and when we want to achieve functionality similar to standard do-while loop we need to use a limit() method:

Stream<Integer> integers = Stream
  .iterate(0, i -> i + 1);
integers
  .limit(10)
  .forEach(System.out::println);

We achieved same functionality like an imperative while loop with less code, but call to the limit() function is not as descriptive as it would be if we had a doWhile() method on a Stream object.

5. Conclusion

This article explains how we can use the Stream API to create infinite streams. These, when used together with transformations such as limit() – can make some scenarios quite a bit easier to understand and implement.

The code supporting of all these examples can be found in the GitHub project – this is a Maven project, so it should be easy to import and run as it is.

1. Overview

In this article, we’ll be looking at ways of testing code written using RxJava.

The typical flow we are creating with RxJava consists of an Observable and an Observer. The observable is a source of a data that is a sequence of elements. One or more observers subscribe to it to receive emitted events.

Typically, the observer and observables are executed in separate threads in an asynchronous fashion – that makes the code hard to test in a traditional way.

Fortunately, RxJava provides a TestSubscriber class which gives us the ability to test asynchronous, event-driven flow.

2. Testing RxJava – The Traditional Way

Let’s start with an example – we have a sequence of letters that we want to zip with a sequence of integers from 1 inclusive.

Our test should assert that a subscriber that listens to events emitted by zipped observable receives letters zipped with integers.

Writing such a test in a traditional way means that we need to keep a list of results and update that list from an observer. Adding elements to a list of integers means that our observable and observers need to work in the same thread – they cannot work asynchronously.

And so we would be missing one of the biggest advantages of RxJava – processing of events in separate threads.

Here’s what that limited version of the test would look like:

List<String> letters = Arrays.asList("A", "B", "C", "D", "E");
List<String> results = new ArrayList<>();
Observable<String> observable = Observable
  .from(letters)
  .zipWith(
     Observable.range(1, Integer.MAX_VALUE), 
     (string, index) -> index + "-" + string);

observable.subscribe(results::add);

assertThat(results, notNullValue());
assertThat(results, hasSize(5));
assertThat(results, hasItems("1-A", "2-B", "3-C", "4-D", "5-E"));

We’re aggregating results from an observer by adding elements to a results list. The observer and the observable work in the same thread so our assertion blocks properly and waits for a subscribe() method to finish.

3. Testing RxJava Using a TestSubscriber

RxJava comes with a TestSubsriber class that allows us to write tests that work with an asynchronous processing of events. This is a normal observer that subscribes to the observable.

In a test, we can examine the state of a TestSubscriber and make assertions on that state:

List<String> letters = Arrays.asList("A", "B", "C", "D", "E");
TestSubscriber<String> subscriber = new TestSubscriber<>();

Observable<String> observable = Observable
  .from(letters)
  .zipWith(
    Observable.range(1, Integer.MAX_VALUE), 
    ((string, index) -> index + "-" + string));

observable.subscribe(subscriber);

subscriber.assertCompleted();
subscriber.assertNoErrors();
subscriber.assertValueCount(5);
assertThat(
  subscriber.getOnNextEvents(),
  hasItems("1-A", "2-B", "3-C", "4-D", "5-E"));

We are passing a TestSubscriber instance to a subscribe() method on the observable. Then we can examine the state of this subscriber.

TestSubscriber has some very useful assertion methods that we’ll use to validate our expectations. The subscriber should receive 5 emitted elements by an observer and we assert that by calling the assertValueCount() method.

We can examine all events that a subscriber received by calling the getOnNextEvents() method.

Calling the assertCompleted() method checks if a stream to which the observer is subscribed to is completed. The assertNoErrors() method asserts that there were no errors while subscribing to a stream.

4. Testing Expected Exceptions

Sometimes in our processing, when an observable is emitting events or an observer is processing events, an error occurs. The TestSubscriber has a special method for examining error state – the assertError() method that takes the type of an exception as an argument:

List<String> letters = Arrays.asList("A", "B", "C", "D", "E");
TestSubscriber<String> subscriber = new TestSubscriber<>();

Observable<String> observable = Observable
  .from(letters)
  .zipWith(Observable.range(1, Integer.MAX_VALUE), ((string, index) -> index + "-" + string))
  .concatWith(Observable.error(new RuntimeException("error in Observable")));

observable.subscribe(subscriber);

subscriber.assertError(RuntimeException.class);
subscriber.assertNotCompleted();

We are creating the observable that is joined with another observable using the concatWith() method. The second observable throws a RuntimeException while emitting next event. We can examine a type of that exception on a TestSubsciber by calling the assertError() method.

The observer that receives an error ceases processing and ends up in a non-completed state. That state can be checked by the assertNotCompleted() method.

5. Testing Time-Based Observable

Let’s say that we have an Observable that emits one event per second and we want to test that behavior with a TestSubsciber.

We can define a time-based Observable using the Observable.interval() method and pass a TimeUnit as an argument:

List<String> letters = Arrays.asList("A", "B", "C", "D", "E");
TestScheduler scheduler = new TestScheduler();
TestSubscriber<String> subscriber = new TestSubscriber<>();
Observable<Long> tick = Observable.interval(1, TimeUnit.SECONDS, scheduler);

Observable<String> observable = Observable.from(letters)
  .zipWith(tick, (string, index) -> index + "-" + string);

observable.subscribeOn(scheduler)
  .subscribe(subscriber);

The tick observable will emit a new value every one second.

At the beginning of a test we are at time zero, so our TestSubscriber will not be completed:

subscriber.assertNoValues();
subscriber.assertNotCompleted();

To emulate time passing in our test we need to use a TestScheduler class. We can simulate that one-second pass by calling the advanceTimeBy() method on a TestScheduler:

scheduler.advanceTimeBy(1, TimeUnit.SECONDS);

The advanceTimeBy() method will make an observable produce one event. We can assert that one event was produced by calling an assertValueCount() method:

subscriber.assertNoErrors();
subscriber.assertValueCount(1);
subscriber.assertValues("0-A");

Our list of letters has 5 elements in it so when we want to cause an observable to emit all events, 6 seconds of processing needs to pass. To emulate that 6 seconds, we use the advanceTimeTo() method:

scheduler.advanceTimeTo(6, TimeUnit.SECONDS);
 
subscriber.assertCompleted();
subscriber.assertNoErrors();
subscriber.assertValueCount(5);
assertThat(subscriber.getOnNextEvents(), hasItems("0-A", "1-B", "2-C", "3-D", "4-E"));

After emulating passed time, we can execute assertions on a TestSubscriber. We can assert that all events were produced by calling the assertValueCount() method.

6. Conclusion

In this article, we examined ways of testing observers and observables in RxJava. We looked at a way of testing emitted events, errors and time-based observables.

The implementation of all these examples and code snippets can be found in the GitHub project – this is a Maven project, so it should be easy to import and run as it is.

1. Introduction

In this article, we’ll give an overview of the Optional class, and then explain some problems that we might run into when using it with Jackson.

Following this, we’ll introduce a solution which will get Jackson to treat Optionals as if they were ordinary nullable objects.

2. Problem Overview

First, let’s take a look at what happens when we try to serialize and deserialize Optionals with Jackson.

2.1. Maven Dependency

In order to use Jackson, let’s make sure we’re using its latest version:

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-core</artifactId>
    <version>2.8.6</version>
</dependency>

2.2. Our Book Object

Then, let’s create a class Book, containing one ordinary and one Optional field:

public class Book {
   String title;
   Optional<String> subTitle;
   
   // getters and setters omitted
}

Keep in mind that Optionals should not be used as fields and we are doing this to illustrate the problem.

2.3. Serialization

Now, let’s instantiate a Book:

Book book = new Book();
book.setTitle("Oliver Twist");
book.setSubTitle(Optional.of("The Parish Boy's Progress"));

And finally, let’s try serializing it using a Jackson ObjectMapper:

String result = mapper.writeValueAsString(book);

We’ll see that the output of the Optional field, does not contain its value, but instead a nested JSON object with a field called present:

{"title":"Oliver Twist","subTitle":{"present":true}}

Although this may look strange, it’s actually what we should expect.

In this case, isPresent() is a public getter on the Optional class. This means it will be serialized with a value of true or false, depending on whether it is empty or not. This is Jackson’s default serialization behavior.

If we think about it, what we actually want is for actual the value of the subtitle field to be serialized.

2.4. Deserialization

Now, let’s reverse our previous example, this time trying to deserialize an object into an Optional. We’ll see that now we get a JsonMappingException:

@Test(expected = JsonMappingException.class)
public void givenFieldWithValue_whenDeserializing_thenThrowException
    String bookJson = "{ \"title\": \"Oliver Twist\", \"subTitle\": \"foo\" }";
    Book result = mapper.readValue(bookJson, Book.class);
}

Let’s view the stack trace:

com.fasterxml.jackson.databind.JsonMappingException:
  Can not construct instance of java.util.Optional:
  no String-argument constructor/factory method to deserialize from String value ('The Parish Boy's Progress')

This behavior again makes sense. Essentially, Jackson needs a constructor which can take the value of subtitle as an argument. This is not the case with our Optional field.

3. Solution

What we actually want, is for Jackson to treat an empty Optional as null, and to treat a present Optional as a field representing its value.

Fortunately, this problem has been solved for us. Jackson has a set of modules that deal with JDK 8 datatypes, including Optional.

3.1. Maven Dependency and Registration

First, let’s add the latest version as a Maven dependency:

<dependency>
   <groupId>com.fasterxml.jackson.datatype</groupId>
   <artifactId>jackson-datatype-jdk8</artifactId>
   <version>2.8.6</version>
</dependency>

Now, all we need to do is register the module with our ObjectMapper:

ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new Jdk8Module());

3.2. Serialization

Now, let’s test it. If we try and serialize our Book object again, we’ll see that there is now a subtitle, as opposed to a nested JSON:

Book book = new Book();
book.setTitle("Oliver Twist");
book.setSubTitle(Optional.of("The Parish Boy's Progress"));
String serializedBook = mapper.writeValueAsString(book);
 
assertThat(from(serializedBook).getString("subTitle"))
  .isEqualTo("The Parish Boy's Progress");

If we try serializing an empty book, it will be stored as null:

book.setSubTitle(Optional.empty());
String serializedBook = mapper.writeValueAsString(book);
 
assertThat(from(serializedBook).getString("subTitle")).isNull();

3.3. Deserialization

Now, let’s repeat our tests for deserialization. If we read our Book again, we’ll see that we no longer get a JsonMappingException:

Book newBook = mapper.readValue(result, Book.class);
 
assertThat(newBook.getSubTitle()).isEqualTo(Optional.of("The Parish Boy's Progress"));

Finally, let’s repeat the test again, this time with null. We’ll see that yet again we don’t get a JsonMappingException, and in fact, have an empty Optional:

assertThat(newBook.getSubTitle()).isEqualTo(Optional.empty());

4. Conclusion

We’ve showed how to get around this problem by leveraging the JDK 8 DataTypes module, demonstrating how it enables Jackson to treat an empty Optional as null, and a present Optional as an ordinary field.

The implementation of these examples can be found over on GitHub; this is a Maven-based project, so should be easy to run as is.

1. Overview

Java 8 introduced the concept of Streams as an efficient way of carrying out bulk operations on data. And parallel Streams can be obtained in environments that support concurrency.

These streams can come with improved performance – at the cost of multi-threading overhead.

In this quick tutorial, we’ll look at one of the biggest limitations of Stream API and see how to make a parallel stream work with a custom ThreadPool instance.

2. Parallel Stream

Let’s start with a simple example – calling the parallelStream method on any of the Collection types – which will return a possibly parallel Stream:

@Test
public void givenList_whenCallingParallelStream_shouldBeParallelStream(){
    List<Long> aList = new ArrayList<>();
    Stream<Long> parallelStream = aList.parallelStream();
        
    assertTrue(parallelStream.isParallel());
}

The default processing that occurs in such a Stream uses the ForkJoinPool.commonPool(), a Thread Pool shared by the entire application.

3. Custom Thread Pool

We can actually pass a custom ThreadPool when processing the stream.

The following example lets have a parallel Stream use a custom Thread Pool to calculate the sum of long values from 1 to 1,000,000, inclusive:

@Test
public void giveRangeOfLongs_whenSummedInParallel_shouldBeEqualToExpectedTotal() 
  throws InterruptedException, ExecutionException {
    
    long firstNum = 1;
    long lastNum = 1_000_000;

    List<Long> aList = LongStream.rangeClosed(firstNum, lastNum).boxed()
      .collect(Collectors.toList());

    ForkJoinPool customThreadPool = new ForkJoinPool(4);
    long actualTotal = customThreadPool.submit(
      () -> aList.parallelStream().reduce(0L, Long::sum)).get();
 
    assertEquals((lastNum + firstNum) * lastNum / 2, actualTotal);
}

We used the ForkJoinPool constructor with a parallelism level of 4. Some experimentation is required to determine the optimal value for different environments, but a good rule of thumb is simply choosing the number based on how many cores your CPU has.

Next, we processed the content of the parallel Stream, summing them up in the reduce call.

This simple example may not demonstrate the full usefulness of using a custom Thread Pool, but the benefits become obvious in situations where we do not want to tie-up the common Thread Pool with long-running tasks (e.g. processing data from a network source), or the common Thread Pool is being used by other components within the application.

4. Conclusion

We have briefly looked at how to run a parallel Stream using a custom Thread Pool. In the right environment and with the proper use of the parallelism level, performance gains can be had in certain situations.

The complete code samples referenced in this article can be found over on Github.

1. Introduction

In this article, we will show how to customize the access denied page in a Spring Security project.

This can be achieved either through the Spring Security configuration or web application configuration in the web.xml file.

In the remaining sections, we will take a more in-depth look at each of these options.

2. Custom JSP 

Whenever a user attempts to access a page that is restricted to roles they do not have, the application will return a status code of 403, which means Access Denied.

In order to replace the Spring 403 status response page with a custom one, let’s first create a JSP file called accessDenied.jsp:

<body>
<h2>Sorry, you do not have permission to view this page.</h2>

Click <a href="<c:url value="/homepage.html" /> ">here</a>
to go back to the Homepage.
</body>

3. Spring Security Configuration

By default, Spring Security has an ExceptionTranslationFilter defined which handles exceptions of type AuthenticationException and AccessDeniedException. The latter is done through a property called accessDeniedHandler, which uses the AccessDeniedHandlerImpl class.

In order to customize this behavior to use our own page that we created above, we need to override the properties of the ExceptionTranslationFilter class. This can be done through either Java configuration or XML configuration.

3.1. Access Denied Page

Using Java, we can customize the 403 error handling process by using the accessDeniedPage() or accessDeniedHandler() methods while configuring the HttpSecurity element.

Let’s create an authentication configuration that restricts the “/admin/**” URLs to the ADMIN role and sets the access denied page to our custom accessDenied.jsp page:

@Override
protected void configure(final HttpSecurity http) throws Exception {
    http
      // ...
      .and()
      .exceptionHandling().accessDeniedPage("/accessDenied.jsp");
}

Let’s take a look at the equivalent XML configuration for the access denied page:

<http use-expressions="true">
    <access-denied-handler error-page="/accessDenied"/>
 </http>

3.2. Access Denied Handler

Using an access denied handler instead of a page has the advantage that we can define custom logic to be executed before redirecting to the 403 page. For this, we need to create a class that implements the AccessDeniedHandler interface and overrides the handle() method.

Let’s create a custom AccessDeniedHandler class that logs a warning message for every access denied attempt containing the user that made the attempt and the protected URL they were trying to access:

public class CustomAccessDeniedHandler implements AccessDeniedHandler {

    public static final Logger LOG
      = Logger.getLogger(CustomAccessDeniedHandler.class);

    @Override
    public void handle(
      HttpServletRequest request,
      HttpServletResponse response, 
      AccessDeniedException exc) throws IOException, ServletException {
        
        Authentication auth 
          = SecurityContextHolder.getContext().getAuthentication();
        if (auth != null) {
            LOG.warn("User: " + auth.getName() 
              + " attempted to access the protected URL: "
              + request.getRequestURI());
        }

        response.sendRedirect(request.getContextPath() + "/accessDenied");
    }
}

In the security configuration, we will define the bean and set the custom AccessDeniedHandler:

@Bean
public AccessDeniedHandler accessDeniedHandler(){
    return new CustomAccessDeniedHandler();
}

//...
.exceptionHandling().accessDeniedHandler(accessDeniedHandler());

If we want to configure the CustomAccessDeniedHandler class defined above using XML, the configuration will look slightly different:

<bean name="customAccessDeniedHandler" 
  class="org.baeldung.security.CustomAccessDeniedHandler" />

<http use-expressions="true">
    <access-denied-handler ref="customAccessDeniedHandler"/>
</http>

4. Application Configuration

Handling the access denied error can be done through the web.xml file of a web application, by defining an error-page tag. This contains two subtags called error-code, which specifies the status code to be intercepted, and location, which signifies the URL to which the user will be redirected in case the error code is encountered:

<error-page>
    <error-code>403</error-code>
    <location>/accessDenied</location>
</error-page>

If an application does not have a web.xml file, as is the case with Spring Boot, the Spring annotations do not currently provide an exact alternative to the error-page tag. According to the Spring documentation, in this case, the recommended approach is to use the methods accessDeniedPage() and accessDeniedHandler() presented in section 3.

5. Conclusion

In this quick article, we have detailed the various ways that an access denied error can be handled using a custom 403 page.

The complete source code of the article can be found in the GitHub project.

1. Overview

In this article, we’ll go through the new @ServletComponentScan annotation in Spring Boot.

The aim is to support the following Servlet 3.0 annotations:

• javax.servlet.annotation.WebFilter
• javax.servlet.annotation.WebListener
• javax.servlet.annotation.WebServlet

@WebServlet, @WebFilter, and @WebListener annotated classes can be automatically registered with an embedded Servlet container by annotating @ServletComponentScan on a @Configuration class and specifying the packages.

We have introduced the basic usage of @WebServlet in Introduction to Java Servlets and @WebFilter in Introduction to Intercepting Filter Pattern in Java. For @WebListener, you can take a peek at this article which demonstrates a typical use case of web listeners.

2. Servlets, Filters, and Listeners

Before diving into @ServletComponentScan, let’s take a look at how the annotations: @WebServlet, @WebFilter and @WebListener were used before @ServletComponentScan came into play.

2.1. @WebServlet

Now we’ll first define a Servlet that serves GET requests and responds “hello”:

@WebServlet("/hello")
public class HelloServlet extends HttpServlet {

    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response) {
        try {
            response
              .getOutputStream()
              .write("hello");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

2.2. @WebFilter

Then a filter that filters requests to target “/hello”, and prepends “filtering “ to the output:

@WebFilter("/hello")
public class HelloFilter implements Filter {

    //...
    @Override
    public void doFilter(
      ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) 
      throws IOException, ServletException {
        servletResponse
          .getOutputStream()
          .print("filtering ");
        filterChain.doFilter(servletRequest, servletResponse);
    }
    //...

}

2.3. @WebListener

Finally, a listener that sets a custom attribute in ServletContext:

@WebListener
public class AttrListener implements ServletContextListener {

    @Override
    public void contextInitialized(ServletContextEvent servletContextEvent) {
        servletContextEvent
          .getServletContext()
          .setAttribute("servlet-context-attr", "test");
    }
    //...
}

2.4. Deploy to a Servlet Container

Now that we’ve built the basic components of a simple web application, we can package and deploy it into a Servlet container. Each component’s behavior can be readily verified by deploying the packaged war file into Jetty, Tomcat or any Servlet containers that support Servlet 3.0.

3. Using @ServletComponentScan in Spring Boot

You might wonder since we can use those annotations in most Servlet containers without any configuration, why do we need @ServletComponentScan?  The problem lies in embedded Servlet containers.

Due to the fact that embedded containers do not support @WebServlet, @WebFilter and @WebListener annotations, Spring Boot, relying greatly on embedded containers, introduced this new annotation @ServletComponentScan to support some dependent jars that use these 3 annotations.

The detailed discussion can be found in this issue on Github.

3.1. Maven Dependencies

To use @ServletComponentScan, we need Spring Boot with version 1.3.0 or above. Let’s add the latest version of spring-boot-starter-parent and spring-boot-starter-web to the pom:

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.5.1.RELEASE</version>
    <relativePath /> <!-- lookup parent from repository -->
</parent>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
        <version>1.5.1.RELEASE</version>
    </dependency>
</dependencies>

3.2. Using @ServletComponentScan

The Spring Boot app is pretty simple. We add @ServletComponentScan to enable scanning for @WebFilter, @WebListener and @WebServlet:

@ServletComponentScan
@SpringBootApplication
public class SpringBootAnnotatedApp {

    public static void main(String[] args) {
        SpringApplication.run(SpringBootAnnotatedApp.class, args);
    }

}

Without any change to the previous web application, it just works:

@Autowired private TestRestTemplate restTemplate;

@Test
public void givenServletFilter_whenGetHello_thenRequestFiltered() {
 
    ResponseEntity<String> responseEntity = 
      restTemplate.getForEntity("/hello", String.class);
 
    assertEquals(HttpStatus.OK, responseEntity.getStatusCode());
    assertEquals("filtering hello", responseEntity.getBody());
}

@Autowired private ServletContext servletContext;

@Test
public void givenServletContext_whenAccessAttrs_thenFoundAttrsPutInServletListner() {
 
    assertNotNull(servletContext);
    assertNotNull(servletContext.getAttribute("servlet-context-attr"));
    assertEquals("test", servletContext.getAttribute("servlet-context-attr"));
}

3.3. Specify Packages to Scan

By default, @ServletComponentScan will scan from the package of the annotated class. To specify which packages to scan, we can use its attributes:

• value
• basePackages
• basePackageClasses

The default value attribute is an alias for basePackages.

Say our SpringBootAnnotatedApp is under package com.baeldung.annotation, and we want to scan classes in package com.baeldung.annotation.components created in the web application above, the following configurations are equivalent:

@ServletComponentScan

@ServletComponentScan("com.baeldung.annotation.components")

@ServletComponentScan(basePackages = "com.baeldung.annotation.components")

@ServletComponentScan(
  basePackageClasses = 
    {AttrListener.class, HelloFilter.class, HelloServlet.class})

4. Under the Hood

The @ServletComponentScan annotation is processed by ServletComponentRegisteringPostProcessor. After scanning specified packages for @WebFilter, @WebListener and @WebServlet annotations, a list of ServletComponentHandlers will process their annotation attributes, and register scanned beans:

class ServletComponentRegisteringPostProcessor
  implements BeanFactoryPostProcessor, ApplicationContextAware {
  
    private static final List<ServletComponentHandler> HANDLERS;

    static {
        List<ServletComponentHandler> handlers = new ArrayList<>();
        handlers.add(new WebServletHandler());
        handlers.add(new WebFilterHandler());
        handlers.add(new WebListenerHandler());
        HANDLERS = Collections.unmodifiableList(handlers);
    }
    
    //...
    
    private void scanPackage(
      ClassPathScanningCandidateComponentProvider componentProvider, 
      String packageToScan){
        //...
        for (ServletComponentHandler handler : HANDLERS) {
            handler.handle(((ScannedGenericBeanDefinition) candidate),
              (BeanDefinitionRegistry) this.applicationContext);
        }
    }
}

As said in the official Javadoc, @ServletComponentScan annotation only works in embedded Servlet containers, which is what comes with Spring Boot by default.

5. Conclusion

In this article, we introduced @ServletComponentScan and how it can be used to support applications that depends on any of the annotations: @WebServlet, @WebFilter, @WebListener.

The implementation of the examples and code can be found in the GitHub project.

1. Overview

Java 8 has introduced a new Stream API that lets us process data in a declarative manner.

In this quick article, we would learn how to use the Stream API to split a comma-separated String into a list of Strings and how to join a String array into a comma-separated String.

Nearly all of the time we face situations, where we need to iterate some Java Collections and filter the Collection based on some filtering logic. In a traditional approach for this type of situation, we would use lots of loops and if-else operations to get the desired result.

If you want to read more about the Stream API, check this article.

2. Joining Strings with the Stream API

Let’s use the Stream API to create a function which would join a String array into a comma separated String:

public static String join(String[] arrayOfString){
    return Arrays.asList(arrayOfString)
      .stream()
      //.map(...)
      .collect(Collectors.joining(","));
}

Points to note here:

• The stream() function converts any Collection into a stream of data
• map() function is used to process the data
• There is also another function, named filter(), where we can include filtering criteria

There can be scenarios, where we may want to join a String with some fixed prefix and postfix. With the Stream API we can do that in the following way:

public static String joinWithPrefixPostfix(String[] arrayOfString){
    return Arrays.asList(arrayOfString)
      .stream()
      //.map(...)
      .collect(Collectors.joining(",","[","]"));
}

As we can see in the Collectors.joining() method, we are declaring our prefix as ‘[‘ and postfix as ‘]’; hence the generated String will be created with declared […..] format.

3. Splitting Strings with Stream API

Now, let’s create a function, which would split a comma separated String into a list of String using Stream API:

public static List<String> split(String str){
    return Stream.of(str.split(","))
      .map (elem -> new String(elem))
      .collect(Collectors.toList());
}

It’s also possible to directly convert a String to a Character list using the Stream API:

public static List<Character> splitToListOfChar(String str) {
    return str.chars()
      .mapToObj(item -> (char) item)
      .collect(Collectors.toList());
}

One interesting fact to note here it that the chars() method converts the String into a stream of Integer where each Integer value denotes the ASCII value of each and every Char sequence. That’s why we need to explicitly typecast the mapper object in the mapToObj() method.

Please remember that in order to avoid compilation error, we need to ensure that code is compiled using Java 1.8. To do this, we need to add the following plugin in the pom.xml:

<build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.3</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>
    </plugins>        
</build>

4. Testing

Since we are done creating the functions, let’s create test cases to verify the outcome.

First, let’s test our simple joining method:

@Test
public void givenArray_transformedToStream_convertToString() {
    String[] programmingLanguages = {"java", "python", "nodejs", "ruby"};
    String expectation = "java,python,nodejs,ruby";

    String result  = JoinerSplitter.join(programmingLanguages);
    assertEquals(result, expectation);
}

Next, let’s create another one to test our simple splitting functionality:

@Test
public void givenString_transformedToStream_convertToList() {
    String programmingLanguages = "java,python,nodejs,ruby";

    List<String> expectation = new ArrayList<>();
    expectation.add("java");
    expectation.add("python");
    expectation.add("nodejs");
    expectation.add("ruby");

    List<String> result  = JoinerSplitter.split(programmingLanguages);

    assertEquals(result, expectation);
}

In the same way, we need to create rest of the test cases.

5. Conclusion

Stream API provides us with sophisticated data processing techniques. This new way of writing code is very efficient in terms of heap memory management in a multi-threaded environment.

Like always, the full source code is available over on Github.

Lots of interesting writeups on Java 9 this week.

Here we go…

1. Spring and Java

>> Signing and verifying a standalone JAR [frankel.ch]

Signing a JAR file created by Spring Boot can be quite tricky.

>> Repackaging Exceptions In Streams [codefx.org]

Handling checked exception within lambdas can be painful. And there are no solutions for this – only workarounds.

>> RXJava2 by Example [infoq.com]

A deep dive into the basic principles of reactive programming with RxJava and loads of examples.

>> Building Multi-Release JARs with Maven [in.relation.to]

Exploring a new Java 9 feature – multi-release JAR files.

>> Integration testing strategies for Spring Boot microservices [codecentric.de]

Spring Boot and microservices solve some problems but create different challenges – testing is certainly one of them.

>> Beyond POJOs – Ten More Ways to Reduce Boilerplate with Lombok [sitepoint.com]

Most people use Lombok for POJOs or data classes but you can find other very useful uses in there.

>> Creating stubs using the Hoverfly Java DSL [ontestautomation.com]

A short tutorial on stubbing APIs using Hoverfly.

>> Setting up your logging with SLF4J – How to automate the prevention of mistakes and disappearing log statements [vanwilgenburg.com]

Properly setting up logging can actually be difficult to do well. There are a few things in here you might not have considered before.

Also worth reading:

• >> How to run integration tests at warp speed using Docker and tmpfs [vladmihalcea.com]

• >> The InfoQ eMag: Reactive Programming with Java  [infoq.com]

• >> How Does the Default hashCode() Work? [sitepoint.com]

• >> Bootstrapping an OAuth2 Authorization server using UAA [java-allandsundry.com]

• >> Hibernate Tips: How to call a user-defined function in a CriteriaQuery [thoughts-on-java.org]

• >> Date and Time Formatting in Java 9 Will Get Closer to Unicode Locale Standards [infoq.com]

Webinars and presentations:

• >> Java Puzzlers NG S02: down the Rabbit Hole [infoq.com]

• >> Java SE 9: Continuing to Thrive in the Cloud! [infoq.com]

• >> Spring Tips: JOOQ [spring.io]

Time to upgrade:

• >> Spring Cloud Pipelines 1.0.0.M3 Released [spring.io]

• >> Spring Cloud Task 1.2.0.M1 is now available [spring.io]

• >> Data Geekery Releases Version 3.9.0 of jOOQ, a Java ORM Tool for Building Type Safe Queries [infoq.com]

• >> IntellIJ IDEA 2017.1 EAP: Async Smart Step Into and Faster Git File History [jetbrains.com]

2.Technical

>> FunctionAsObject [martinfowler.com]

The base of functional programming – leveraging functions as first-class citizens. A quick and worthwhile little writeup.

>> 4 Challenges to Building Multi-factor Authentication [stormpath.com]

Multi-factor authentication solutions do significantly increase the security of a system, but have their own set of challenges.

>> How does a relational database work [vladmihalcea.com]

Refreshing the basics

Also worth reading:

• >> Getting Started with Oracle Container Cloud Service [couchbase.com]

• >> Crossing the Streams – Joins in Apache Kafka [codecentric.de]

• >> Cleaner Tests with Spock [tidyjava.com]

3. Musings

>> Computer Science Concepts That Non-Technical People Should Know [techblog.bozho.net]

Sometimes we don’t speak the same language and we might not be aware of it. There are few concepts that would make communication between technical and non-technical people much easier.

>> Logging for Fun: Things You’d Never Thought to Log [daedtech.com]

Logging and monitoring of everyday things can give us ideas for future improvements

>> Extracting Value as an Employee [zachholman.com]

Some really pragmatic, potentially uncomfortable advice.

>> A couple of thoughts on Clean Architecture [insaneprogramming.be]

A couple of fresh thoughts about applying the Clean Architecture principles.

>> Habits that Pay Off for Programmers [daedtech.com]

The Law of Diminishing Returns applies to technical skills as well. Developing critical complementary skills – outside of pure coding – pays off in the long run.

Also worth reading:

• >> Taking the Guild Metaphor Too Far [daedtech.com]

• >> How much does ‘crunch time’ hurt team morale? [mdswanson.com]

• >> Book review: Fifty quick ideas to improve your tests [giorgiosironi.com]

• >> Amazon Rekognition Update – Estimated Age Range for Faces [amazon.com]

• >> Software ate the world [mdswanson.com]

• >> Interview about the Agile Manifesto [stitcher.com]

• >> Back-to-Basic Weekend Reading: Monte-Carlo Methods [allthingsdistributed.com]

4. Comics

And my favorite Dilberts of the week:

>> The agile programmers can jump out of the way [dilbert.com]

>> I’ll be the judge of that [dilbert.com]

>> My part is mostly talk [dilbert.com]

5. Pick of the Week

>> The Builder’s High [randsinrepose.com]

1. Introduction

In this article, we’ll introduce the Guava CacheLoader. 

Before reading further, it’s recommended that there is a basic understanding of the LoadingCache class first. This is because the CacheLoader works with it specifically.

Essentially, the CacheLoader is a function used for computing a value in the event of it not being found in a Guava LoadingCache.

2. Using a CacheLoader with a LoadingCache

When there is a cache miss with a LoadingCache, or the cache needs to be refreshed, the CacheLoader will be used for calculating values. This helps to encapsulate our caching logic in one place, making our code more cohesive.

2.1. Maven Dependency

First, let’s add our Maven dependency:

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
  <version>21.0</version>
</dependency>

You can find the latest version in the Maven repository.

2.2. Computing and Caching Values

Now, let’s see how we can instantiate a LoadingCache with a CacheLoader:

LoadingCache<String, String> loadingCache = CacheBuilder.newBuilder()
  .build(new CacheLoader<String, String>() {
    @Override
    public String load(final String s) throws Exception {
      return slowMethod(s);
    }
});

Essentially, the LoadingCache will call our inline CacheLoader whenever it needs to compute a value which has not been cached. Let’s try counting how many times our slowMethod() is called when we retrieve something from cache multiple times:

String value = loadingCache.get("key");
value = loadingCache.get("key");

assertThat(callCount).isEqualTo(1);
assertThat(value).isEqualTo("expectedValue");

As we can see, it was only called once. The first time the value was not cached as it had yet to be computed. The second time, it was cached from the previous call, so we could avoid the overhead of calling our slowMethod() again.

2.3. Refreshing the Cache

Another common problem with caching is refreshing the cache. Although the most difficult aspect is knowing when to refresh the cache, another one is knowing how. 

Solving the how is simple when using the CacheLoader. The LoadingCache will simply invoke it for each value which needs to be refreshed. Let’s try this with a test:

String value = loadingCache.get("key");
loadingCache.refresh("key");

assertThat(callCount).isEqualTo(2);
assertThat(value).isEqualTo("key");

Unlike our subsequent calls to get(), refresh() will force the CacheLoader to be called again, making sure our values are up to date.

3. Conclusion

In this article, we’ve explained how a LoadingCache is used by a CacheLoader in order to calculate values on cache misses, and also on cache refreshes. It’s also worth checking out this more in-depth article on Guava Caching.

The implementation of these examples can be found over on GitHub. This is a Maven project, so should be easy to run as is.

1. Overview

In this article, we will be looking at the advanced usage of the Apache HttpClient library.

We’ll look at the examples of adding custom headers to HTTP requests, and we’ll see how to configure the client to authorize and send requests through a proxy server.

We will be using Wiremock for stubbing the HTTP server. If you want to read more about Wiremock, check out this article.

2. HTTP Request With a Custom User-Agent Header

Let’s say that we want to add a custom User-Agent header to an HTTP GET request. The User-Agent header contains a characteristic string that allows the network protocol peers to identify the application type, operating system, and software vendor or software version of the requesting software user agent.

Before we start writing our HTTP client, we need to start our embedded mock server:

@Rule
public WireMockRule serviceMock = new WireMockRule(8089);

When we’re creating a HttpGet instance we can simply use a setHeader() method to pass a name of our header together with the value. That header will be added to an HTTP request:

String userAgent = "BaeldungAgent/1.0"; 
HttpClient httpClient = HttpClients.createDefault();

HttpGet httpGet = new HttpGet("http://localhost:8089/detail");
httpGet.setHeader(HttpHeaders.USER_AGENT, userAgent);

HttpResponse response = httpClient.execute(httpGet);

assertEquals(response.getStatusLine().getStatusCode(), 200);

We’re adding a User-Agent header and sending that request via an execute() method.

When GET request is sent for a URL /detail with header User-Agent that has a value equal to “BaeldungAgent/1.0” then serviceMock will return 200 HTTP response code:

serviceMock.stubFor(get(urlEqualTo("/detail"))
  .withHeader("User-Agent", equalTo(userAgent))
  .willReturn(aResponse().withStatus(200)));

3. Sending Data in the POST Request Body

Usually, when we are executing the HTTP POST method, we want to pass an entity as a request body. When creating an instance of a HttpPost object, we can add the body to that request using a setEntity() method:

String xmlBody = "<xml><id>1</id></xml>";
HttpClient httpClient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost("http://localhost:8089/person");
httpPost.setHeader("Content-Type", "application/xml");

StringEntity xmlEntity = new StringEntity(xmlBody);
httpPost.setEntity(xmlEntity);

HttpResponse response = httpClient.execute(httpPost);

assertEquals(response.getStatusLine().getStatusCode(), 200);

We are creating a StringEntity instance with a body that is in the XML format. It is important to set the Content-Type header to “application/xml” to pass information to the server about the type of content we’re sending. When the serviceMock receives the POST request with XML body, it responds with status code 200 OK:

serviceMock.stubFor(post(urlEqualTo("/person"))
  .withHeader("Content-Type", equalTo("application/xml"))
  .withRequestBody(equalTo(xmlBody))
  .willReturn(aResponse().withStatus(200)));

4. Sending Requests via a Proxy Server

Often, our web service can be behind a proxy server that performs some additional logic, caches static resources, etc. When we’re creating the HTTP Client and sending a request to an actual service, we don’t want to deal with that on each and every HTTP request.

To test this scenario, we’ll need to start up another embedded web server:

@Rule
public WireMockRule proxyMock = new WireMockRule(8090);

With two embedded servers, the first actual service is on the 8089 port and a proxy server is listening on the 8090 port.

We are configuring our HttpClient to send all requests via proxy by creating a DefaultProxyRoutePlanner that takes the HttpHost instance proxy as an argument:

HttpHost proxy = new HttpHost("localhost", 8090);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
HttpClient httpclient = HttpClients.custom()
  .setRoutePlanner(routePlanner)
  .build();

Our proxy server is redirecting all requests to the actual service that listens on the 8090 port. In the end of the test, we verify that request was sent to our actual service via a proxy:

proxyMock.stubFor(get(urlMatching(".*"))
  .willReturn(aResponse().proxiedFrom("http://localhost:8089/")));

serviceMock.stubFor(get(urlEqualTo("/private"))
  .willReturn(aResponse().withStatus(200)));

assertEquals(response.getStatusLine().getStatusCode(), 200);
proxyMock.verify(getRequestedFor(urlEqualTo("/private")));
serviceMock.verify(getRequestedFor(urlEqualTo("/private")));

5. Configuring the HTTP Client to Authorize via Proxy

Extending the previous example, there are some cases when the proxy server is used to perform authorization. In such configuration, a proxy can authorize all requests and pass them to the server that is hidden behind a proxy.

We can configure the HttpClient to send each request via proxy, together with the Authorization header that will be used to perform an authorization process.

Suppose that we have a proxy server that authorizes only one user – “username_admin“, with a password “secret_password“.

We need to create the BasicCredentialsProvider instance with credentials of the user that will be authorized via proxy. To make HttpClient automatically add the Authorization header with the proper value, we need to create a HttpClientContext with credentials provided and a BasicAuthCache that stores credentials:

HttpHost proxy = new HttpHost("localhost", 8090);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);

//Client credentials
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(proxy), 
  new UsernamePasswordCredentials("username_admin", "secret_password"));

// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();

BasicScheme basicAuth = new BasicScheme();
authCache.put(proxy, basicAuth);
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(credentialsProvider);
context.setAuthCache(authCache);

HttpClient httpclient = HttpClients.custom()
  .setRoutePlanner(routePlanner)
  .setDefaultCredentialsProvider(credentialsProvider)
  .build();

When we set up our HttpClient, making requests to our service will result in sending a request via proxy with an Authorization header to perform authorization process. It will be set in each request automatically.

Let’s execute an actual request to the service:

HttpGet httpGet = new HttpGet("http://localhost:8089/private");
HttpResponse response = httpclient.execute(httpGet, context);

Verifying an execute() method on the httpClient with our configuration confirms that a request went through a proxy with an Authorization header:

proxyMock.stubFor(get(urlMatching("/private"))
  .willReturn(aResponse().proxiedFrom("http://localhost:8089/")));
serviceMock.stubFor(get(urlEqualTo("/private"))
  .willReturn(aResponse().withStatus(200)));

assertEquals(response.getStatusLine().getStatusCode(), 200);
proxyMock.verify(getRequestedFor(urlEqualTo("/private"))
  .withHeader("Authorization", containing("Basic")));
serviceMock.verify(getRequestedFor(urlEqualTo("/private")));

6. Conclusion

This article shows how to configure the Apache HttpClient to perform advanced HTTP calls. We saw how to send requests via a proxy server and how to authorize via proxy.

The implementation of all these examples and code snippets can be found in the GitHub project – this is a Maven project, so it should be easy to import and run as it is.

1. Introduction

This article is about Neo4j – one of the most mature and full-featured graph databases on the market today. Graph databases approach the task of data modeling with the view that many things in life lend themselves to being represented as a collection of nodes (V) and connections between them called edges (E).

2. Embedded Neo4j

The easiest way to get started with Neo4j is to use the embedded version in which Neo4j runs in the same JVM as your application.

First, we need to add a Maven dependency:

<dependency>
    <groupId>org.neo4j</groupId>
    <artifactId>neo4j</artifactId>
    <version>3.1.0</version>
</dependency>

You can check this link to download the latest version.

Next, let’s create a factory:

GraphDatabaseFactory graphDbFactory = new GraphDatabaseFactory();

Finally, we create an embedded database:

GraphDatabaseService graphDb = graphDbFactory.newEmbeddedDatabase(
  new File("data/cars"));

Now the real action can begin! First, we need to create some nodes in our graph and for that, we need to start a transaction since Neo4j will reject any destructive operation unless a transaction has been started:

graphDb.beginTx();

Once we have a transaction in progress, we can start adding nodes:

Node car = graphDb.createNode(Label.label("Car"));
car.setProperty("make", "tesla");
car.setProperty("model", "model3");

Node owner = graphDb.createNode(Label.label("Person"));
owner.setProperty("firstName", "baeldung");
owner.setProperty("lastName", "baeldung");

Here we added a node Car with properties make and model as well as node Person with properties firstName and lastName

Now we can add a relationship:

owner.createRelationshipTo(car, RelationshipType.withName("owner"));

The statement above added an edge joining the two nodes with an owner label. We can verify this relationship by running a query written in Neo4j’s powerful Cypher language:

Result result = graphDb.execute(
  "MATCH (c:Car) <-[owner]- (p:Person) " +
  "WHERE c.make = 'tesla'" +
  "RETURN p.firstName, p.lastName");

Here we ask to find a car owner for any car whose make is tesla and give us back his/her firstName and lastName. Unsurprisingly, this returns: {p.firstName=baeldung, p.lastName=baeldung}

3. Cypher Query Language

Neo4j provides a very powerful and pretty intuitive querying language which supports the full range of features one would expect from a database. Let us examine how we can accomplish that standard create, retrieve, update and delete tasks.

3.1. Create Node

Create keyword can be used to create both nodes and relationships.

CREATE (self:Company {name:"Baeldung"})
RETURN self

Here we’ve created a company with a single property name. A node definition is marked by parentheses and its properties are enclosed in curly braces. In this case, self is an alias for the node and Company is a node label.

3.2. Create Relationship

It is possible to create a node and a relationship to that node all in a single query:

Result result = graphDb.execute(
  "CREATE (baeldung:Company {name:\"Baeldung\"}) " +
  "-[:owns]-> (tesla:Car {make: 'tesla', model: 'modelX'})" +
  "RETURN baeldung, tesla");

Here we’ve created nodes baeldung and tesla and established an ownership relationship between them. Creating relationships to pre-existing nodes is, of course, also possible.

3.3. Retrieve Data

MATCH keyword is used to find data in combination with RETURN to control which data points are returned. The WHERE clause can be utilized to filter out only those nodes that have the properties we desire.

Let us figure out the name of the company that owns tesla modelX:

Result result = graphDb.execute(
  "MATCH (company:Company)-[:owns]-> (car:Car)" +
  "WHERE car.make='tesla' and car.model='modelX'" +
  "RETURN company.name");

3.4. Update Nodes

SET keyword can be used for updating node properties or labels. Let us add mileage to our tesla:

Result result = graphDb.execute("MATCH (car:Car)" +
  "WHERE car.make='tesla'" +
  " SET car.milage=120" +
  " SET car :Car:Electro" +
  " SET car.model=NULL" +
  " RETURN car");

Here we add a new property called milage, modify labels to be both Car and Electro and finally, we delete model property altogether.

3.5. Delete Nodes

DELETE keyword can be used for permanent removal of nodes or relationships from the graph:

graphDb.execute("MATCH (company:Company)" +
  " WHERE company.name='Baeldung'" +
  " DELETE company");

Here we deleted a company named Baeldung.

3.6. Parameter Binding

In the above examples, we have hard-coded parameter values which isn’t the best practice. Luckily, Neo4j provides a facility for binding variables to a query:

Map<String, Object> params = new HashMap<>();
params.put("name", "baeldung");
params.put("make", "tesla");
params.put("model", "modelS");

Result result = graphDb.execute("CREATE (baeldung:Company {name:$name}) " +
  "-[:owns]-> (tesla:Car {make: $make, model: $model})" +
  "RETURN baeldung, tesla", params);

4. Java Driver

So far we’ve been looking at interacting with an embedded Neo4j instance, however, in all probability for production, we’d want to run a stand-alone server and connect to it via a provided driver. First, we need to add another dependency in our maven pom.xml:

<dependency>
    <groupId>org.neo4j.driver</groupId>
    <artifactId>neo4j-java-driver</artifactId>
    <version>1.1.1</version>
</dependency>

You can follow this link to check for the latest version of this driver.

Now we can establish a connection:

Driver driver = GraphDatabase.driver(
  "bolt://localhost:7687", AuthTokens.basic("neo4j", "12345"));

Then, create a session:

Session session = driver.session();

Finally, we can run some queries:

session.run("CREATE (baeldung:Company {name:\"Baeldung\"}) " +
  "-[:owns]-> (tesla:Car {make: 'tesla', model: 'modelX'})" +
  "RETURN baeldung, tesla");

Once we are done with all our work we need to close both session and the driver:

session.close();
driver.close();

5. JDBC Driver

It is also possible to interact with Neo4j via a JDBC driver. Yet another dependency for our pom.xml:

<dependency>
    <groupId>org.neo4j</groupId>
    <artifactId>neo4j-jdbc-driver</artifactId>
    <version>3.0.1</version>
</dependency>

You can follow this link to download the latest version of this driver.

Next, let’s establish a JDBC connection:

Connection con = DriverManager.getConnection(
  "jdbc:neo4j:bolt://localhost/?user=neo4j,password=12345,scheme=basic");

Here con is a regular JDBC connection which can be used for creating and executing statements or prepared statements:

try (Statement stmt = con.
  stmt.execute("CREATE (baeldung:Company {name:\"Baeldung\"}) "
  + "-[:owns]-> (tesla:Car {make: 'tesla', model: 'modelX'})"
  + "RETURN baeldung, tesla")

    ResultSet rs = stmt.executeQuery(
      "MATCH (company:Company)-[:owns]-> (car:Car)" +
      "WHERE car.make='tesla' and car.model='modelX'" +
      "RETURN company.name");

    while (rs.next()) {
        rs.getString("company.name");
    }
}

6. Object-Graph-Mapping

Object-Graph-Mapping or OGM is a technique which enables us to use our domain POJOs as entities in the Neo4j database. Let us examine how this works. The first step, as usually, we add new dependencies to our pom.xml:

<dependency>
    <groupId>org.neo4j</groupId>
    <artifactId>neo4j-ogm-core</artifactId>
    <version>2.1.1</version>
</dependency>

<dependency> 
    <groupId>org.neo4j</groupId>
    <artifactId>neo4j-ogm-embedded-driver</artifactId>
    <version>2.1.1</version>
</dependency>

You can check the OGM Core Link and OGM Embedded Driver Link to check for the latest versions of these libraries.

Second, we annotate our POJO’s with OGM annotations:

@NodeEntity
public class Company {
    private Long id;

    private String name;

    @Relationship(type="owns")
    private Car car;
}

@NodeEntity
public class Car {
    private Long id;

    private String make;

    @Relationship(direction = "INCOMING")
    private Company company;
}

@NodeEntity informs Neo4j that this object will need to be represented by a node in the resulting graph. @Relationship communicates the need to create a relationship with a node representing the related type. In this case, a company owns a car.

Please note that Neo4j requires each entity to have a primary key, with a field named id being picked up by default. An alternatively named field could be used by annotating it with @GraphId.

Then, we need to create a configuration that will be used to bootstrap Neo4j‘s OGM. For simplicity, let us use an embedded in-memory only database:

Configuration conf = new Configuration();
conf.driverConfiguration().setDriverClassName(
  "org.neo4j.ogm.drivers.embedded.driver.EmbeddedDriver");

After that, we initialize SessionFactory with the configuration we created and a package name in which our annotated POJO’s reside:

SessionFactory factory = new SessionFactory(conf, "com.baeldung.graph");

Finally, we can create a Session and begin using it:

Session session = factory.openSession();
Car tesla = new Car("tesla", "modelS");
Company baeldung = new Company("baeldung");

baeldung.setCar(tesla);
session.save(baeldung);

Here we initiated a session, created our POJO’s and asked OGM session to persist them. Neo4j OGM runtime transparently converted objects to a set of Cypher queries which created appropriate nodes and edges in the database.

If this process seems familiar, that is because it is! That is precisely how JPA works, the only difference being whether object gets translated into rows that are persisted to an RDBMS, or a series of nodes and edges persisted to a graph database.

7. Conclusion

This article looked at some basics of a graph-oriented database Neo4j.

As always, the code in this write-up is all available over on Github.

1. Overview

Guava library provides the EventBus which allows publish-subscribe communication between components. In this tutorial, we will look at how to use some of the features of the EventBus.

2. Setup

To start we add the Google Guava library dependency in the pom.xml:

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>21.0</version>
</dependency>

3. Using the EventBus

Let’s start by using a simple example.

3.1. Setup

We start by looking at the EventBus object. It can register listeners and post events. Using it is as simple as instantiating the class:

EventBus eventBus = new EventBus();

Guava library gives you the freedom of using the EventBus in any way that best suits your development needs.

3.2. Creating Listeners

public class EventListener {

    private static int eventsHandled;

    @Subscribe
    public void stringEvent(String event) {
        eventsHandled++;
    }
}

3.3. Registering Listeners

EventListener listener = new EventListener();
eventBus.register(listener);

3.4. Unregistering Listeners

If for any reason we want to unregister a class from the EventBus, that can also be easily done:

eventBus.unregister(listener);

3.5. Posting Events

@Test
public void givenStringEvent_whenEventHandled_thenSuccess() {
    eventBus.post("String Event");
    assertEquals(1, listener.getEventsHandled());
}

3.6. Posting Custom Events

public class CustomEvent {
    private String action;

    // standard getters/setters and constructors
}

@Subscribe
public void someCustomEvent(CustomEvent customEvent) {
    eventsHandled++;
}

@Test
public void givenCustomEvent_whenEventHandled_thenSuccess() {
    CustomEvent customEvent = new CustomEvent("Custom Event");
    eventBus.post(customEvent);

    assertEquals(1, listener.getEventsHandled());
}

@Subscribe
public void handleDeadEvent(DeadEvent deadEvent) {
    eventsHandled++;
}

1. Overview

Velocity is a Java-based templating engine.

It’s an open source web framework designed to be used as a view component in the MVC architecture, and it provides an alternative to some existing technologies such as JSP.

Velocity can be used to generate XML files, SQL, PostScript and most other text-based formats.

In this article, we will explore how it can be used to create dynamic web pages.

2. How Velocity Works

The core class of Velocity is the VelocityEngine.

It orchestrates the whole process of reading, parsing and generating content using data model and velocity template.

Simply put, here are the steps we need to follow for any typical velocity application:

• Initialize the velocity engine
• Read the template
• Put the data model in context object
• Merge the template with context data and render the view

Let’s go through an example following these simple steps:

VelocityEngine velocityEngine = new VelocityEngine();
velocityEngine.init();
   
Template t = velocityEngine.getTemplate("index.vm");
    
VelocityContext context = new VelocityContext();
context.put("name", "World");
    
StringWriter writer = new StringWriter();
t.merge( context, writer );

3. Maven Dependencies

To work with Velocity, we need to add following dependencies to our Maven project:

<dependency>
    <groupId>org.apache.velocity</groupId>
    <artifactId>velocity</artifactId>
    <version>1.7</version>
    </dependency>
<dependency>
     <groupId>org.apache.velocity</groupId>
     <artifactId>velocity-tools</artifactId>
     <version>2.0</version>
</dependency>

The latest version of both these dependencies can be here: velocity and velocity-tools.

4. Velocity Template Language

Velocity Template Language (VTL) provides the simplest and cleanest way of incorporating the dynamic content in a web page by using VTL references.

VTL reference in velocity template starts with a $ and is used for getting the value associated with that reference. VTL provides also a set of directives which can be used for manipulating the output of the Java code. Those directives start with #. 

4.1. References

There are three types of references in Velocity, variables, properties and methods:

• variables – defined within the page using #set directive or value returned from Java object’s field:

  #set ($message="Hello World")

• properties – refer to fields within an object; they can also refer to a getter method of the property:

  $customer.name

• methods – refer to the method on Java object:

  $customer.getName()

The final value resulting from every reference is converted to a string when it is rendered into the final output.

4.2. Directives

VTL provides a rich set of directives:

• set – it can be used for setting the value of a reference; this value can be assigned to a variable or a property reference:

  #set ($message = "Hello World")
  #set ($customer.name = "Brian Mcdonald")

• conditionals – #if, #elseif and #else directives provide a way to generate the content based on conditional checks:

  #if($employee.designation == "Manager")
      <h3> Manager </h3>
  #elseif($employee.designation == "Senior Developer")
      <h3> Senior Software Engineer </h3>
  #else
      <h3> Trainee </h3>
  #end

• loops – #foreach directive allows looping over a collection of objects:

  <ul>
      #foreach($product in $productList)
          <li> $product </li>
      #end
  </ul>

• include – #include element provides the ability to import files into the template:

  #include("one.gif","two.txt","three.html"...)

• parse – #parse statement allows the template designer to import another local file which contains VTL; Velocity will then parse the content and render it:

  #parse (Template)

• evaluate – #evaluate directive can be used to evaluate VTL dynamically; this allows the template to evaluate a String at render time, for example to internationalise the template:

  #set($firstName = "David")
  #set($lastName = "Johnson")
  
  #set($dynamicsource = "$firstName$lastName")
  
  #evaluate($dynamicsource)

• break – #break directive stops any further rendering of current execution scope (i.e. #foreach, #parse)
• stop –  #stop directive stops any further rendering and execution of the template.
• velocimacros – #macro directive allows the template designer to define a repeated segment of VTL:

  #macro(tablerows)
      <tr>
          <td>
          </td>
      </tr>
  #end

  This macro now can be put in any place in the template as #tablerows():

  #macro(tablerows $color $productList)
      #foreach($product in $productList)
          <tr>
              <td bgcolor=$color>$product.name</td>
          </tr>
      #end
  #end

4.3. Other Features

• math – a handful built-in mathematical functions, which can be used in templates:

  #set($percent = $number / 100)
  #set($remainder = $dividend % $divisor)

• range operator – that can be used in conjunction with #set and #foreach:

  #set($array = [0..10])
  
  #foreach($elem in $arr)
      $elem
  #end

5. Velocity Servlet

The primary job of the Velocity Engine is to generate content based on a template.

The Engine doesn’t contain any web related functionalities in itself. To implement a web application, we need to use a servlet or servlet-based framework.

Velocity provides one out of the box implementation VelocityViewServlet, which is a part of the velocity-tools subproject.

To make use of the built-in functionality provided by VelocityViewServlet, we can extend our servlet from VelocityViewServlet and override the handleRequest() method:

public class ProductServlet extends VelocityViewServlet {

    ProductService service = new ProductService();

    @Override
    public Template handleRequest(
      HttpServletRequest request, 
      HttpServletResponse response,
      Context context) throws Exception {
      
        List<Product> products = service.getProducts();
        context.put("products", products);

        return getTemplate("index.vm");
    }
}

6. Configuration

6.1. Web Configuration

Let’s now see how to configure the VelocityViewServlet in the web.xml.

We need to specify the optional initialization parameters which include velocity.properties and toolbox.xml:

<web-app>
    <display-name>apache-velocity</display-name>
      //...
       
    <servlet>
        <servlet-name>velocity</servlet-name>
        <servlet-class>org.apache.velocity.tools.view.VelocityViewServlet</servlet-class>

        <init-param>
            <param-name>org.apache.velocity.properties</param-name>
            <param-value>/WEB-INF/velocity.properties</param-value>
        </init-param>
    </servlet>
        //...
</web-app>

We also need to specify the mapping for this servlet. All the requests for velocity templates (*.vm) need to be served by the velocity servlet:

<servlet-mapping>
    <servlet-name>velocityLayout</servlet-name>
    <url-pattern>*.vm</url-pattern>
</servlet-mapping>

6.2. Resource Loader

Velocity provides flexible resource loader system. It allows one or more resource loader to be in operation at the same time:

• FileResourceLoader
• JarResourceLoader
• ClassPathResourceLoader
• URLResourceLoader
• DataSourceResourceLoader
• WebappResourceLoader

These resource loaders are configured in velocity.properties:

resource.loader=webapp
webapp.resource.loader.class=org.apache.velocity.tools.view.WebappResourceLoader
webapp.resource.loader.path = .
webapp.resource.loader.cache = true

7. Velocity Template

Velocity template is the place where all the view generation logic is written. These pages are written using Velocity Template Language (VTL):

<html>
    ...
    <body>
        <center>
        ...
        <h2>$products.size() Products on Sale!</h2>
        <br/>
            We are proud to offer these fine products
            at these amazing prices.
        ...
        #set( $count = 1 )
        <table class="gridtable">
            <tr>
                <th>Serial #</th>
                <th>Product Name</th>
                <th>Price</th>
            </tr>
            #foreach( $product in $products )
            <tr>
                <td>$count)</td>
                <td>$product.getName()</td>
                <td>$product.getPrice()</td>
            </tr>
            #set( $count = $count + 1 )
            #end
        </table>
        <br/>
        </center>
    </body>
</html>

8. Managing the Page Layout

Velocity provides a simple layout control and customizable error screens for Velocity Tool based application.

VelocityLayoutServlet encapsulates this capability to render the specified layouts. VelocityLayoutServlet is an extension to VelocityViewServlet. 

8.1. Web Configuration

Let’s see how to configure the VelocityLayoutServlet. The servlet is defined for intercepting the requests for velocity template pages and the layout specific properties are defined in velocity.properties file:

<web-app>
    ...
    <servlet>
        <servlet-name>velocityLayout</servlet-name>
        <servlet-class>org.apache.velocity.tools.view.VelocityLayoutServlet</servlet-class>

        <init-param>
            <param-name>org.apache.velocity.properties</param-name>
            <param-value>/WEB-INF/velocity.properties</param-value>
        </init-param>
    </servlet>
    ....
    <servlet-mapping>
        <servlet-name>velocityLayout</servlet-name>
        <url-pattern>*.vm</url-pattern>
    </servlet-mapping>
    ...
</web-app>

8.2. Layout Templates

Layout template defines the typical structure of a velocity page. By default, the VelocityLayoutServlet searches for Default.vm under the layout folder. Overriding few properties can change this location:

tools.view.servlet.layout.directory = layout/
tools.view.servlet.layout.default.template = Default.vm

The layout file consists of header template, footer template and a velocity variable $screen_content which renders the contents of requested velocity page:

<html>
    <head>
        <title>Velocity</title>
    </head>
    <body>
        <div>
            #parse("/fragments/header.vm")
        </div>
        <div>
            <!-- View index.vm is inserted here -->
            $screen_content
        </div>
        <div>
            #parse("/fragments/footer.vm")
        </div>
    </body>
</html>

8.3. Layout Specification in the Requested Screen

Layout for a particular screen can be defined as a velocity variable at the beginning of a page. That is done by putting this line in the page:

#set($layout = "MyOtherLayout.vm")

8.4. Layout Specification in the Request Parameter

We can add a request parameter in the query string layout=MyOtherLayout.vm and VLS will find it and render the screen within that layout instead of searching for default layout.

8.5. Error Screens

Customized error screen can be implemented using velocity layout. VelocityLayoutServlet provides two variables $error_cause and $stack_trace to present the exception details.

Error page can be configured in velocity.properties file:

tools.view.servlet.error.template = Error.vm

9. Conclusion

In this article, we have learned how Velocity is a useful tool for rendering the dynamic web pages. Also, we have seen different ways of using velocity provided servlets.

We also have an article focused on a Velocity configuration with Spring MVC here at Baeldung.

The complete code for this tutorial is available over on GitHub.

1. Overview

JUnit and TestNG are undoubtedly the two most popular unit-testing frameworks in the Java ecosystem. While TestNG itself is inspired by JUnit, it provides its own distinctive features and unlike JUnit, it works for functional and higher levels of testing.

In this article, we will discuss and compare these frameworks by covering their features and common use cases.

2. Test Setup

While writing test cases, often we need to execute some configuration or initialization instructions before test executions, and also some cleanup after completion of tests. Let’s evaluate these in both frameworks.

JUnit offers initialization and cleanup at two levels, before and after each method and class. @Before and @After annotations at method level and @BeforeClass and @AfterClass at the class level:

public class SummationServiceTest {

    private static List<Integer> numbers;

    @BeforeClass
    public static void initialize() {
        numbers = new ArrayList<>();
    }

    @AfterClass
    public static void tearDown() {
        numbers = null;
    }

    @Before
    public void runBeforeEachTest() {
        numbers.add(1);
        numbers.add(2);
        numbers.add(3);
    }

    @After
    public void runAfterEachTest() {
        numbers.clear();
    }

    @Test
    public void givenNumbers_sumEquals_thenCorrect() {
        int sum = numbers.stream().reduce(0, Integer::sum);
        assertEquals(6, sum);
    }
}

Similar to JUnit, TestNG also provides initialization and cleanup at the method and class level. While @BeforeClass and @AfterClass remain the same at the class level, the method level annotations are @BeforeMethod and @AfterMethod:

@BeforeClass
public void initialize() {
    numbers = new ArrayList<>();
}

@AfterClass
public void tearDown() {
    numbers = null;
}

@BeforeMethod
public void runBeforeEachTest() {
    numbers.add(1);
    numbers.add(2);
    numbers.add(3);
}

@AfterMethod
public void runAfterEachTest() {
    numbers.clear();
}

TestNG also offers, @BeforeSuite, @AfterSuite, @BeforeGroup and @AfterGroup annotations, for configurations at suite and group levels:

@BeforeGroups("positive_tests")
public void runBeforeEachGroup() {
    numbers.add(1);
    numbers.add(2);
    numbers.add(3);
}

@AfterGroups("negative_tests")
public void runAfterEachGroup() {
    numbers.clear(); 
}

Also, we can use the @BeforeTest and @AfterTest if we need any configuration before or after test cases included in the <test> tag in TestNG XML configuration file:

<test name="test setup">
    <classes>
        <class name="SummationServiceTest">
            <methods>
                <include name="givenNumbers_sumEquals_thenCorrect" />
            </methods>
        </class>
    </classes>
</test>

Note that, the declaration of @BeforeClass and @AfterClass method has to be static in JUnit whereas, there is more flexibility in TestNG in the method declaration, it does not have these constraints.

3. Ignoring Tests

Both frameworks support ignoring test cases, though they do it quite differently. JUnit offers @Ignore annotation:

@Ignore
@Test
public void givenNumbers_sumEquals_thenCorrect() {
    int sum = numbers.stream().reduce(0, Integer::sum);
    Assert.assertEquals(6, sum);
}

while TestNG uses @Test with a parameter “enabled” with a boolean value true or false:

@Test(enabled=false)
public void givenNumbers_sumEquals_thenCorrect() {
    int sum = numbers.stream.reduce(0, Integer::sum);
    Assert.assertEquals(6, sum);
}

4. Running Tests Together

Running tests together as a collection is possible in both JUnit 4 and TestNG, but they do it in different ways.

The @RunWith and @Suite annotations are used to group test cases and run them as a suite. A suite is a collection of test cases that can be grouped together and run as a single test.

All the declarations are defined inside a single class, let’s look at the example:

@RunWith(Suite.class)
@Suite.SuiteClasses({ RegistrationTest.class, SignInTest.class })
public class SuiteTest {

}

In TestNG grouping tests in groups are done using XML file:

<suite name="suite">
    <test name="test suite">
        <classes>
            <class name="com.baeldung.RegistrationTest" />
            <class name="com.baeldung.SignInTest" />
        </classes>
    </test>
</suite>

This indicates RegistrationTest and SignInTest will run together.

Apart from grouping classes, TestNG can group methods as well. Methods are grouped with @Test(groups=”groupName”) annotation:

@Test(groups = "regression")
public void givenNegativeNumber_sumLessthanZero_thenCorrect() {
    int sum = numbers.stream().reduce(0, Integer::sum);
    Assert.assertTrue(sum < 0);
}

Let’s use an XML to execute the groups:

<test name="test groups">
    <groups>
        <run>
            <include name="regression" />
        </run>
    </groups>
    <classes>
        <class 
          name="com.baeldung.SummationServiceTest" />
    </classes>
</test>

This will execute the test method tagged with group regression.

5. Testing Exceptions

The feature for testing for exceptions using annotations is available in both JUnit 4:

@Test(expected = ArithmeticException.class) 
public void givenNumber_whenThrowsException_thenCorrect() { 
    int i = 1 / 0;
}

and TestNG:

@Test(expectedExceptions = ArithmeticException.class) 
public void givenNumber_whenThrowsException_thenCorrect() { 
    int i = 1 / 0;
}

This feature implies what exception is thrown from a piece of code, that is being unit tested.

6. Parameterized Tests

Parameterized unit tests are used for testing the same code under several conditions. With the help of parameterized unit tests, we can set up a test method that obtains data from some data source. The main idea is to make the unit test method reusable and to test with a different set of inputs.

In JUnit, the test class has to be annotated with @RunWith to make it a parameterized class and @Parameter to use the denote the parameter values for unit test. @Parameters will have to return a List[], and these parameters will be passed to the constructor of the test class as argument:

@RunWith(value = Parameterized.class)
public class ParametrizedTests {

    private int value;
    private boolean isEven;

    public ParametrizedTests(int value, boolean isEven) {
        this.value = value;
        this.isEven = isEven;
    }

    @Parameters
    public static Collection<Object[]> data() {
        Object[][] data = new Object[][]{{1, false}, {2, true}, {4, true}};
        return Arrays.asList(data);
    }

    @Test
    public void givenParametrizedNumber_ifEvenCheckOK_thenCorrect() {
        Assert.assertEquals(isEven, value % 2 == 0);
    }
}

Note that the declaration of the parameter has to be static and it has to be passed to the constructor in order to initialize the class member as value for testing. In addition to that, the return type of parameter class must be a List, and the values are limited to String or primitive data types.

In TestNG, we can parametrize tests using @Parameter or @DataProvider annotation. While using the XML file annotate the test method with @Parameter:

@Test
@Parameters({"value", "isEven"})
public void 
  givenNumberFromXML_ifEvenCheckOK_thenCorrect(int value, boolean isEven) {
    Assert.assertEquals(isEven, value % 2 == 0);
}

and provide the data in the XML file:

<suite name="My test suite">
    <test name="numbersXML">
        <parameter name="value" value="1"/>
        <parameter name="isEven" value="false"/>
        <classes>
            <class name="baeldung.com.ParametrizedTests"/>
        </classes>
    </test>
</suite>

While using data in the XML file is simple and useful, in some cases, you might need to provide more complex data.

@DataProvider annotation is used to handle these scenarios, which can be used to map complex parameter types for testing methods.

@DataProvider for primitive data types:

@DataProvider(name = "numbers")
public static Object[][] evenNumbers() {
    return new Object[][]{{1, false}, {2, true}, {4, true}};
}

@Test(dataProvider = "numbers")
public void givenNumberFromDataProvider_ifEvenCheckOK_thenCorrect
  (Integer number, boolean expected) {
    Assert.assertEquals(expected, number % 2 == 0);
}

@DataProviderfor objects:

@Test(dataProvider = "numbersObject")
public void givenNumberObjectFromDataProvider_ifEvenCheckOK_thenCorrect
  (EvenNumber number) {
    Assert.assertEquals(number.isEven(), number.getValue() % 2 == 0);
}

@DataProvider(name = "numbersObject")
public Object[][] parameterProvider() {
    return new Object[][]{{new EvenNumber(1, false)},
      {new EvenNumber(2, true)}, {new EvenNumber(4, true)}};
}

In the same way, any specific objects that are to be tested can be created and returned using data provider. It’s useful when integrating with frameworks like Spring.

7. Test Timeout

Timed out tests means, a test case should fail if the execution is not completed within certain specified time period. Both JUnit and TestNG support timed out tests, in the same way, annotating with @Test(timeout=1000):

@Test(timeOut = 1000)
public void givenExecution_takeMoreTime_thenFail() {
    while (true);
}

8. Dependent Tests

TestNG supports dependency testing. This means in a set of test methods, if the initial test fails, then all subsequent dependent tests will be skipped, not marked as failed as in the case for JUnit.

Let’s have a look at a scenario, where we need to validate email, and if it’s successful, will proceed to log in:

@Test
public void givenEmail_ifValid_thenTrue() {
    boolean valid = email.contains("@");
    Assert.assertEquals(valid, true);
}

@Test(dependsOnMethods = {"givenEmail_ifValid_thenTrue"})
public void givenValidEmail_whenLoggedIn_thenTrue() {
    LOGGER.info("Email {} valid >> logging in", email);
}

9. Conclusion

Both JUnit and TestNG are popular tools for testing in the Java ecosystem.

In this article we had a quick look at the various needs and ways of writing tests with each of these two test frameworks.

The implementation of all the code snippets can be found in TestNG and  core-java Github project.

1. Introduction

In this article, we will take a look at the Apache BVal library’s implementation of the Java Bean Validation specification (JSR 349).

2. Maven Dependencies

In order to use Apache BVal, we first need to add the following dependencies to our pom.xml file:

<dependency>
    <groupId>org.apache.bval</groupId>
    <artifactId>bval-jsr</artifactId>
    <version>1.1.2</version>
</dependency>
<dependency>
    <groupId>javax.validation</groupId>
    <artifactId>validation-api</artifactId>
    <version>1.1.0.Final</version>
</dependency>

Custom BVal constraints can be found in the optional bval-extras dependency:

<dependency>
    <groupId>org.apache.bval</groupId>
    <artifactId>bval-extras</artifactId>
    <version>1.1.2</version>
</dependency>

The latest versions of bval-jsr, bval-extras, and validation-api can be downloaded from Maven Central.

3. Applying Constraints

Apache BVal provides implementations for all the constraints defined in the javax.validation package. In order to apply a constraint to a property of a bean, we can add the constraint annotation to the property declaration.

Let’s create a User class which has four attributes, then apply the @NotNull, @Size, and @Min annotations:

public class User {
    
    @NotNull
    private String email;
    
    private String password;

    @Size(min=1, max=20)
    private String name;

    @Min(18)
    private int age;

    // standard constructor, getters, setters
}

4. Validating Beans

To validate the constraints applied on the User class, we need to obtain a ValidatorFactory instance and one or more Validator instances.

4.1. Obtaining a ValidatorFactory

It is recommended by the Apache BVal documentation to obtain a single instance of this class, as factory creation is a demanding process:

ValidatorFactory validatorFactory 
  = Validation.byProvider(ApacheValidationProvider.class)
  .configure().buildValidatorFactory();

4.2. Obtaining a Validator

Next, we need to get a Validator instance from the validatorFactory defined above:

Validator validator = validatorFactory.getValidator();

This is a thread-safe implementation, so we can safely reuse already created instances.

The Validator class offers three methods for determining the validity of a bean: validate(), validateProperty() and validateValue().

Each of these methods returns a set of ConstraintViolation objects that contain information about the constraint that was not obeyed.

4.3. validate() API

The validate() method checks the validity of the whole bean which means that it verifies all the constraints applied to properties of an object that is passed as a parameter.

Let’s create a JUnit test where we define a User object and use the validate() method to test its properties:

@Test
public void givenUser_whenValidate_thenValidationViolations() {
    User user
      = new User("ana@yahoo.com", "pass", "nameTooLong_______________", 15);

    Set<ConstraintViolation<User>> violations = validator.validate(user);
    assertTrue("no violations", violations.size() > 0);
}

4.4. validateProperty() API

The validateProperty() method can be used to validate a single property of a bean.

Let’s create a JUnit test in which we will define a User object with an age property less than the required minimum value of 18 and verify that validating this property results in one violation:

@Test
public void givenInvalidAge_whenValidateProperty_thenConstraintViolation() {
    User user = new User("ana@yahoo.com", "pass", "Ana", 12);

    Set<ConstraintViolation<User>> propertyViolations
      = validator.validateProperty(user, "age");
 
    assertEquals("size is not 1", 1, propertyViolations.size());
}

4.5. validateValue() API

The validateValue() method can be used to check if some value would be a valid value for a property of a bean before setting it on the bean.

Let’s create a JUnit test with a User object, then verify that the value 20 is a valid value for the age property:

@Test
public void givenValidAge_whenValidateValue_thenNoConstraintViolation() {
    User user = new User("ana@yahoo.com", "pass", "Ana", 18);
    
    Set<ConstraintViolation<User>> valueViolations
      = validator.validateValue(User.class, "age", 20);
 
    assertEquals("size is not 0", 0, valueViolations.size());
}

4.6. Closing the ValidatorFactory

After using the ValidatorFactory, we must remember to close it at the end:

if (validatorFactory != null) {
    validatorFactory.close();
}

5. Non-JSR Constraints

The Apache BVal library also provides a series of constraints that are not a part of the JSR specification and provide additional and more powerful validation capabilities.

The bval-jsr package contains two additional constraints: @Email for validating a valid email address, and @NotEmpty for ensuring that a value is not empty.

The rest of the custom BVal constraints are provided in the optional package bval-extras.

This package contains constraints for verifying various number formats such as the @IBAN annotation that ensures a number is a valid International Bank Account Number, the @Isbn annotation that verifies a valid Standard Book Number, and the @EAN13 annotation for verifying an International Article Number.

The library also provides annotations for ensuring the validity of various types of credit card numbers: @AmericanExpress, @Diners, @Discover, @Mastercard, and @Visa.

You can determine if a value contains a valid domain or Internet Address by using the @Domain and @InetAddress annotations.

And finally, the package contains the @Directory and @NotDirectory annotations for verifying whether a File object is a directory or not.

Let’s define additional properties on our User class and apply some of the non-JSR annotations to them:

public class User {
    
    @NotNull
    @Email
    private String email;
    
    @NotEmpty
    private String password;

    @Size(min=1, max=20)
    private String name;

    @Min(18)
    private int age;
    
    @Visa
    private String cardNumber = "";
    
    @IBAN
    private String iban = "";
    
    @InetAddress
    private String website = "";

    @Directory
    private File mainDirectory = new File(".");

    // standard constructor, getters, setters
}

The constraints can be tested in a similar manner to the JSR constraints:

@Test
public void whenValidateNonJSR_thenCorrect() {
    User user = new User("ana@yahoo.com", "pass", "Ana", 20);
    user.setCardNumber("1234");
    user.setIban("1234");
    user.setWebsite("10.0.2.50");
    user.setMainDirectory(new File("."));
    
    Set<ConstraintViolation<User>> violations 
      = validator.validateProperty(user,"iban");
 
    assertEquals("size is not 1", 1, violations.size());
 
    violations = validator.validateProperty(user,"website");
 
    assertEquals("size is not 0", 0, violations.size());

    violations = validator.validateProperty(user, "mainDirectory");
 
    assertEquals("size is not 0", 0, violations.size());
}

While these additional annotations are convenient for potential validation needs, the one disadvantage of using annotations which are not part of the JSR specification is that you cannot easily switch to a different JSR implementation should that become necessary later on.

6. Custom Constraints

In order to define our own constraints, we first need to create an annotation following the standard syntax.

Let’s create a Password annotation that will define the conditions that a user’s password must satisfy:

@Constraint(validatedBy = { PasswordValidator.class })
@Target({METHOD, FIELD, ANNOTATION_TYPE, CONSTRUCTOR, PARAMETER })
@Retention(RetentionPolicy.RUNTIME)
public @interface Password {
    String message() default "Invalid password";

    Class<?>[] groups() default {};

    Class<? extends Payload>[] payload() default {};

    int length() default 6;

    int nonAlpha() default 1;
}

The actual validation of the password value is done in a class that implements the ConstraintValidator interface — in our case, the PasswordValidator class. This class overrides the isValid() method and verifies if the length of the password is less than the length attribute, and if it contains fewer than the specified number of non-alphanumeric characters in the nonAlpha attribute:

public class PasswordValidator 
  implements ConstraintValidator<Password, String> {

    private int length;
    private int nonAlpha;

    @Override
    public void initialize(Password password) {
        this.length = password.length();
        this.nonAlpha = password.nonAlpha();
    }

    @Override
    public boolean isValid(String value, ConstraintValidatorContext ctx) {
        if (value.length() < length) {
            return false;
        }
        int nonAlphaNr = 0;
        for (int i = 0; i < value.length(); i++) {
            if (!Character.isLetterOrDigit(value.charAt(i))) {
                nonAlphaNr++;
            }
        }
        if (nonAlphaNr < nonAlpha) {
            return false;
        }
        return true;
    }
}

Let’s apply our custom constraint to the password property of User class:

@Password(length = 8)
private String password;

We can create a JUnit test to verify that an invalid password value results in a constraint violation:

@Test
public void givenValidPassword_whenValidatePassword_thenNoConstraintViolation() {
    User user = new User("ana@yahoo.com", "password", "Ana", 20);
    Set<ConstraintViolation<User>> violations 
      = validator.validateProperty(user, "password");
 
    assertEquals(
      "message incorrect",
      "Invalid password", 
      violations.iterator().next().getMessage());
}

Now let’s create a JUnit test in which we verify a valid password value:

@Test
public void givenValidPassword_whenValidatePassword_thenNoConstraintViolation() {
    User user = new User("ana@yahoo.com", "password#", "Ana", 20);
		
    Set<ConstraintViolation<User>> violations 
      = validator.validateProperty(user, "password");
    assertEquals("size is not 0", 0, violations.size());
}

7. Conclusion

In this article, we have exemplified the use of the Apache BVal bean validation implementation.

The complete source code for this article can be found over on GitHub.

1. Introduction

In this article, we will demonstrate how to upload Excel files and display their content in a web page using the Spring MVC framework.

2. Uploading Excel Files

In order to be able to upload files, we will first create a controller mapping that receives a MultipartFile and saves it in the current location:

private String fileLocation;

@PostMapping("/uploadExcelFile")
public String uploadFile(Model model, MultipartFile file) throws IOException {
    InputStream in = file.getInputStream();
    File currDir = new File(".");
    String path = currDir.getAbsolutePath();
    fileLocation = path.substring(0, path.length() - 1) + file.getOriginalFilename();
    FileOutputStream f = new FileOutputStream(fileLocation);
    int ch = 0;
    while ((ch = in.read()) != -1) {
        f.write(ch);
    }
    f.flush();
    f.close();
    model.addAttribute("message", "File: " + file.getOriginalFilename() 
      + " has been uploaded successfully!");
    return "excel";
}

Next, let’s create a JSP file with a form that contains an input of type file which will have the accept attribute set to only allow Excel files:

<c:url value="/uploadExcelFile" var="uploadFileUrl" />
<form method="post" enctype="multipart/form-data"
  action="${uploadFileUrl}">
    <input type="file" name="file" accept=".xls,.xlsx" /> <input
      type="submit" value="Upload file" />
</form>

3. Reading Excel Files

In order to parse the uploaded excel file, we will use the Apache POI library, which can work with both .xls and .xlsx files.

Let’s create a helper class called MyCell which will contain properties of an Excel cell related to content and formatting:

public class MyCell {
    private String content;
    private String textColor;
    private String bgColor;
    private String textSize;
    private String textWeight;

    public MyCell(String content) {
        this.content = content;
    }
    
    //standard constructor, getters, setters
}

We will read the content of the Excel file into a Map that contains lists of MyCell objects.

3.1. Parsing a .xls File

A .xls file is represented in the Apache POI library by an HSSFWorkbook class, which is made up of HSSFSheet objects. For opening and reading the content of a .xls file, you can view our article on Working with Microsoft Excel in Java.

For parsing the formatting of a cell, we will obtain the HSSFCellStyle object, which can help us determine properties like the background color and font. All the read properties will be set in the attributes of the MyCell object:

HSSFCellStyle cellStyle = cell.getCellStyle();

MyCell myCell = new MyCell();

HSSFColor bgColor = cellStyle.getFillForegroundColorColor();
if (bgColor != null) {
    short[] rgbColor = bgColor.getTriplet();
    myCell.setBgColor("rgb(" + rgbColor[0] + ","
      + rgbColor[1] + "," + rgbColor[2] + ")");
    }
HSSFFont font = cell.getCellStyle().getFont(workbook);

The colors are read in an rgb(rVal, gVal, bVal) format to make it easier to display them using CSS in a JSP page.

Let’s also obtain the font size, weight, and color:

myCell.setTextSize(font.getFontHeightInPoints() + "");
if (font.getBold()) {
    myCell.setTextWeight("bold");
}
HSSFColor textColor = font.getHSSFColor(workbook);
if (textColor != null) {
    short[] rgbColor = textColor.getTriplet();
    myCell.setTextColor("rgb(" + rgbColor[0] + ","
      + rgbColor[1] + "," + rgbColor[2] + ")");
}

3.2. Parsing a .xlsx File

For files in the newer .xlsx format, we can use the XSSFWorkbook class and similar ones for the contents of a workbook, also documented in the Working with Microsoft Excel in Java article.

Let’s take a closer look at reading the formatting of a cell in the .xlsx format. First, we will retrieve the XSSFCellStyle object associated with a cell and use it to determine the background color and font:

XSSFCellStyle cellStyle = cell.getCellStyle();

MyCell myCell = new MyCell();
XSSFColor bgColor = cellStyle.getFillForegroundColorColor();
if (bgColor != null) {
    byte[] rgbColor = bgColor.getRGB();
    myCell.setBgColor("rgb(" 
      + (rgbColor[0] < 0 ? (rgbColor[0] + 0xff) : rgbColor[0]) + ","
      + (rgbColor[1] < 0 ? (rgbColor[1] + 0xff) : rgbColor[1]) + ","
      + (rgbColor[2] < 0 ? (rgbColor[2] + 0xff) : rgbColor[2]) + ")");
}
XSSFFont font = cellStyle.getFont();

In this case, the RGB values of the color will be signed byte values, so we will obtain the unsigned values by adding 0xff to the negative values.

Let’s also determine the properties of the font:

myCell.setTextSize(font.getFontHeightInPoints() + "");
if (font.getBold()) {
    myCell.setTextWeight("bold");
}
XSSFColor textColor = font.getXSSFColor();
if (textColor != null) {
    byte[] rgbColor = textColor.getRGB();
    myCell.setTextColor("rgb("
      + (rgbColor[0] < 0 ? (rgbColor[0] + 0xff) : rgbColor[0]) + "," 
      + (rgbColor[1] < 0 ? (rgbColor[1] + 0xff) : rgbColor[1]) + "," 
      + (rgbColor[2] < 0 ? (rgbColor[2] + 0xff) : rgbColor[2]) + ")");
}

3.3. Handling Empty Rows 

The methods described above do not account for empty rows in an Excel file. If we want a faithful rendition of the file that displays the empty rows as well, we will need to simulate these in our resulting HashMap with an ArrayList of MyCell objects containing empty Strings as content.

Initially, after reading the Excel file, the empty rows in the file will be ArrayList objects with a size of 0.

In order to determine how many empty String objects we should add, we will first determine the longest row in the Excel file, using the maxNrCols variable. Then we will add that number of empty String objects to all the lists in our HashMap that have a size of 0:

int maxNrCols = data.values().stream()
  .mapToInt(List::size)
  .max()
  .orElse(0);

data.values().stream()
  .filter(ls -> ls.size() < maxNrCols)
  .forEach(ls -> {
      IntStream.range(ls.size(), maxNrCols)
        .forEach(i -> ls.add(new MyCell("")));
  });

4. Displaying Excel Files

For displaying the Excel files read using Spring MVC, we will need to define a controller mapping and JSP page.

4.1. Spring MVC Controller

Let’s create a @RequestMapping method that will call the code above to read the content of the uploaded file, then add the returned Map as a Model attribute:

@Resource(name = "excelPOIHelper")
private ExcelPOIHelper excelPOIHelper;

@RequestMapping(method = RequestMethod.GET, value = "/readPOI")
public String readPOI(Model model) throws IOException {

  if (fileLocation != null) {
      if (fileLocation.endsWith(".xlsx") || fileLocation.endsWith(".xls")) {
          Map<Integer, List<MyCell>> data
            = excelPOIHelper.readExcel(fileLocation);
          model.addAttribute("data", data);
      } else {
          model.addAttribute("message", "Not a valid excel file!");
      }
  } else {
      model.addAttribute("message", "File missing! Please upload an excel file.");
  }
  return "excel";
}

4.2. JSP 

For visually displaying the content of the file, we will create an HTML table and, in the style attribute of each table cell, add the formatting properties corresponding to each cell from the Excel file:

<c:if test="${not empty data}">
    <table style="border: 1px solid black; border-collapse: collapse;">
        <c:forEach items="${data}" var="row">
            <tr>
                <c:forEach items="${row.value}" var="cell">
                    <td style="border:1px solid black;height:20px;width:100px;
                      background-color:${cell.bgColor};color:${cell.textColor};
                      font-weight:${cell.textWeight};font-size:${cell.textSize}pt;">
                      ${cell.content}
                    </td>
                </c:forEach>
            </tr>
        </c:forEach>
    </table>
</c:if>

5. Conclusion

In this article, we have shown an example project for uploading Excel files and displaying them in a web page using the Spring MVC framework.

The full source code can be found in the GitHub project.

1. Overview

In this quick article, we’ll explore the build-in web request utils in Spring MVC – WebUtils, ServletRequestUtils.

2. WebUtils and ServletRequestUtils

In almost all applications, we face situations where we need to fetch some parameters from an incoming HTTP request.

To do that, we had to create some really hectic code segments like:

HttpSession session = request.getSession(false);
if (session != null) {
    String foo = session.getAttribute("parameter");
}

String name = request.getParameter("parameter");
if (name == null) {
    name = "DEFAULT";
}

Using WebUtils and ServletRequestUtils, we can do it with a just one line of code.

To see how these utilities work, let’s create a simple web application.

3. Sample Pages

We need to create sample pages in order to be able to link the URLs. We would use Spring Boot and Thymeleaf as our template engine. We need to add required dependencies for them.

Let’s create a page with a simple form:

<form action="setParam" method="POST">
    <h3>Set Parameter:  </h3>
    <p th:text="${parameter}" class="param"/>
    <input type="text" name="param" id="param"/>
    <input type="submit" value="SET"/>
</form>
<br/>
<a href="other">Another Page</a>

As we can see, we are creating a form for initiating the POST request.

There is also one link, which will forward users to the next page, where we will show the submitted parameter from the session attribute.

And let’s create a second page:

Parameter set by you: <p th:text="${parameter}" class="param"/>

4. Usage

Now that we’re done building the views, let’s create our controller and use ServletRequestUtils and fetch the request parameter:

@PostMapping("/setParam")
public String post(HttpServletRequest request, Model model) {
    String param 
      = ServletRequestUtils.getStringParameter(
        request, "param", "DEFAULT");

    WebUtils.setSessionAttribute(request, "parameter", param);

    model.addAttribute("parameter", "You set: " + (String) WebUtils
      .getSessionAttribute(request, "parameter"));

    return "utils";
}

Note how we’re using the getStringParameter API in ServletRequestUtils to fetch the request parameter name param; a default value will be assigned to the request parameter if no value is coming into the controller.

And of course notince the setSessionAttribute API out of WebUtils used to set a value in the session attribute. We don’t need to explicitly check if a session already exists nor link in vanilla servlet. Spring will configure it on the fly.

In the same way, let’s create the other handler which would show following session attribute:

@GetMapping("/other")
public String other(HttpServletRequest request, Model model) {
    
    String param = (String) WebUtils.getSessionAttribute(
      request, "parameter");
    
    model.addAttribute("parameter", param);
    
    return "other";
}

That’s all we need to create our application.

One quick point to note here is that ServletRequestUtils has some wonderful inbuilt features which will automatically typecast the request parameter based on our need.

Here’s how we can convert the request parameter to Long:

Long param = ServletRequestUtils.getLongParameter(request, "param", 1L);

Similarly, we can convert the request parameter to other types:

boolean param = ServletRequestUtils.getBooleanParameter(
  request, "param", true);

double param = ServletRequestUtils.getDoubleParameter(
  request, "param", 1000);

float param = ServletRequestUtils.getFloatParameter(
  request, "param", (float) 1.00);

int param = ServletRequestUtils.getIntParameter(
  request, "param", 100);

Another point to note is that ServletRequestUtils has another method getRequiredStringParameter(ServletRequest request, String name) for fetching the request parameter. The difference is that if the parameter is not found in the incoming request, it would throw ServletRequestBindingException. This might be useful when we need to play with critical data.

Below is a sample code snippet:

try {
    ServletRequestUtils.getRequiredStringParameter(request, "param");
} catch (ServletRequestBindingException e) {
    e.printStackTrace();
}

We could also create one simple JUnit test case to test the application:

@Test
public void givenParameter_setRequestParam_andSetSessionAttribute() 
  throws Exception {
      String param = "testparam";
 
      this.mockMvc.perform(
        post("/setParam")
          .param("param", param)
          .sessionAttr("parameter", param))
          .andExpect(status().isOk());
  }

5. Conclusion

In this article, we see that using WebUtils and ServletRequestUtils can greatly reduce plenty of boilerplate coding overhead. However, on the other hand, it certainly increases dependency on the Spring framework – which is something to keep in mind if that’s a concern.

As always, the source code is available over on GitHub.

1. Introduction

In this article, we’ll be implementing a simple Continuous Delivery pipeline with Jenkins, Marathon and Mesos.

First, we will give a high level overview of the technology stack and architecture, with an explanation of how everything fits together. Following that, we’ll move onto a practical, step by step example.

The outcome of this will be a fully automated Jenkins pipeline, deploying our application to our Mesos cluster using Marathon.

2. Overview of Technology Stack

When working with containers and microservice architectures, we face new operational problems that we wouldn’t have done with more traditional stacks.

For example, when deploying to a cluster, we have to deal with scaling, failover, networking and more. These difficult, distributed computing problems, can be solved with distributed kernels and schedulers, like Apache Mesos and Marathon.

2.1. Mesos

Mesos, in simplest terms, can be seen as the single server where our applications will be run. In reality we have a cluster, but it’s this abstraction that makes it so useful.

2.2. Marathon

Marathon, is the framework which is used to deploy our applications to Mesos, solving difficult problems for us (health checking, auto-scaling, failover, monitoring etc).

3. Setup and Installation

This article assumes you already have Jenkins, Mesos and Marathon running. If this is not the case, consult the official documentation for each of them to learn how to set them up. Without this, you won’t be able to implement any of the steps in the guide.

4. Our Delivery Pipeline

We shall be creating the following Jenkins pipeline:

There’s nothing particularly complex about this approach – it’s synonymous to the flow you of most modern CD pipelines. In our case, building will mean containerising the application, and deploying will mean using Marathon to schedule it on a Mesos cluster.

5. Testing and Building our Application

The first step is to build and test our application. To keep things simple, the application we are going to be working with is a Spring Boot application. Because of this, our resulting artifact will be an executable jar. It won’t have any external dependencies other than the JRE, making it very simple to execute.

5.1. Creating Our Job

The first thing we want to do is create our Jenkins job. Let’s select “New Item” in the left hand navigation bar, then select create a freestyle project, naming it “marathon-mesos-demo“:

5.2. Integrating with Git

Next, let’s configure it to clone the Github repository containing our application:

For the sake of simplicity, our repository is public, meaning that we are able to clone over https. If this wasn’t the case and we were cloning over SSH, there would be an additional step to set up an SSH user and private key, beyond the scope of this article.

5.3. Setting up Build Triggers

Next, let’s set up some build triggers so our job will poll git for new commits every minute:

5.4. Generating our Build Script

We can now tell our job to execute a shell script when it runs. As we are working with a simple Spring Boot Maven project, all we need to do is run the command “mvn clean install“. This will run all the tests, and build our executable jar:

5.5. Building our Project

Now we’ve set up the beginning of our pipeline, let’s trigger it manually by clicking “Build Now” on the job. Once the job is finished, we can confirm it has passed by it being marked as blue.

6. Containerizing our Application

Let us move onto the next stage of our pipeline, which is packaging and publishing our application with Docker. We need to use Docker as containers are specifically what Marathon manages. This isn’t unreasonable, as virtually anything can run in a container. It’s easier for a tool like Marathon to work with the abstraction granted by these.

6.1. Creating the Dockerfile

First, let’s create a Dockerfile in the project root. Essentially, a Dockerfile is a file containing instructions to the Docker Deamon on how to build an image:

FROM openjdk:8-jre-alpine
ADD target/mesos-marathon-0.0.1-SNAPSHOT.jar app.jar
EXPOSE 8082 
ENTRYPOINT ["java","-jar","/app.jar"]

The image we are building is simple – all it contains is an executable jar, and a shell command which will execute it when the container starts. We also have to make sure that we are exposing the port that our application will listen on, in this case ‘8080’.

6.2. Publishing the Image

Now that we are able build our image, let’s create a simple bash script which builds and publishes it to our private Docker Hub repository, and put it in our project root:

#!/usr/bin/env bash
set -e
docker login -u baeldung -p $DOCKER_PASSWORD
docker build -t baeldung/mesos-marathon-demo:$BUILD_NUMBER .
docker push baeldung/mesos-marathon-demo:$BUILD_NUMBER

You might need to push your image to the public docker registry or your private one.

The $BUILD_NUMBER environment variable is populated by Jenkins, incrementing with every build. Although slightly brittle, it is a quick way of getting each build to increase in version number. The $DOCKER_PASSWORD is also populated by Jenkins, and in this case we will make use of the EnvInject plugin in order to keep it secret.

Whilst we could store this script directly in Jenkins, it’s better practice for it to remain in version control, as it can then be versioned and audited alongside the rest of our project.

6.3. Building and Publishing on Jenkins

Now let’s modify our Jenkins job so it runs “Dockerise.sh” after building the jar:

And then, let’s run our job to confirm again, confirming everything is working by it going blue.

7. Deploying our Image

Our pipeline is nearly complete. There is only one more stage, which is to use Marathon to deploy our application to our Mesos cluster.

Jenkins comes with a “Deploy with Marathon” plugin. This acts as a wrapper around the Marathon API, making it easier than it would be when working with traditional shell scripting. You can install it via the plugin manager.

7.1. Creating our Marathon.json file

Before we can use the Marathon plugin, we need to create a “marathon.json” file, and store it in our project root. This is because the plugin is dependent on it.

This file: “marathon.json” contains a Mesos Application Definition. This is a description of a long running service (application) that we want to run. Ultimately, the Jenkins Marathon plugin will POST the contents of the file to the Marathon /v2/apps endpoint. Marathon will then in turn schedule the defined application to run on Mesos:

{
  "id": "mesos-marathon-demo",
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "",
      "network": "BRIDGE",
      "portMappings": [
        { "containerPort": 8082, "hostPort": 0 }
      ]
    }
  }
}

This is the simplest configuration we can give for a containerized application.

The property:  “portMappings” needs to be set correctly in order to make our application accessible from our Mesos slave. It basically means, map the container port 8082, to a random port on the host (mesos slave) so we can talk to our application from the outside world. After deploying our application, Marathon will tell us what that port used.

7.2. Adding a Marathon Deployment Build Step

Let’s add a Marathon Deployment post build action to our job:

Notice we’re telling the plugin where Marathon is running, in this case ‘localhost:8081’. We’re also telling it the image we want to deploy. This is what the empty “image” field in our file get’s replaced with.

Now we’ve created the final stage of our pipeline, let’s run our job one more time and confirm that it’s still passing, this time with the extra step where it sends our application to Marathon.

7.3. Verifying our Deployment in Marathon

Now it’s been deployed, let’s take a look in the Marathon UI:

As we can see, our application is now shown in the UI. In order to access it, we just need to check what host and port it has been assigned:

In this case, it’s been randomly allocated the port 31143 on localhost, which will internally map to port 8082 in our container as configured in the application definition. We can then visit this URL in our browser to confirm the application is being served correctly.

8. Conclusion

In this article, we’ve created a simple Continuous Delivery pipeline using Jenkins, Marathon and Mesos. Whenever we push a change to our code, it will be running in an environment a few minutes later.

Later articles in this series will cover more advanced Marathon topics, such as application health checking, scaling, failover. Other use cases for Mesos, such as batch processing may also be covered.

The source code for our application is available on over on GitHub; this is a Maven project which should be able to run as is.