Introduction to Serenity BDD

1. Introduction

In this tutorial, we’ll give an introduction to Serenity BDD – a great tool for Behaviour Driven Development (BDD). This is a solution for automated acceptance testing that generates well-illustrated testing reports.

2. Core Concepts

The concepts behind Serenity follow the concepts behind BDD. If you want to read more about it, check our article about Cucumber and JBehave.

2.1. Requirements

In Serenity, requirements are organized into three levels:

1. capabilities
2. features
3. stories

Typically, a project implements high-level capabilities, e.x. order management and membership management capabilities in an e-commerce project. Each capability is comprised of many features, and features are explained in detail by user stories.

2.2. Steps and Tests

Steps contain a group of resource manipulation operations. It can be an action, verification or a context related operation. The classic Given_When_Then format can be reflected in the steps.

And tests go hand in hand with Steps. Each test tells a simple user story, which is carried out using certain Step.

2.3. Reports

Serenity not only reports the test results but also uses them for producing living documentation describing the requirements and application behaviors.

3. Testing with SerenityBDD

To run our Serenity tests with JUnit, we need to @RunWith the SerenityRunner, test runner. SerenityRunner instruments the step libraries and ensures that the test results will be recorded and reported on by the Serenity reporters.

3.1. Maven Dependencies

To make use of Serenity with JUnit, we should include serenity-core and serenity-junit in the pom.xml:

<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-core</artifactId>
    <version>1.2.5-rc.11</version>
</dependency>
<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-junit</artifactId>
    <version>1.2.5-rc.11</version>
</dependency>

We also need serenity-maven-plugin to have reports aggregated from test results:

<plugin>
    <groupId>net.serenity-bdd.maven.plugins</groupId>
    <artifactId>serenity-maven-plugin</artifactId>
    <version>1.2.5-rc.6</version>
    <executions>
        <execution>
            <id>serenity-reports</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>aggregate</goal>
            </goals>
        </execution>
    </executions>
</plugin>

If we want Serenity to generate reports even if there’s a test failure, add the following to the pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.20</version>
    <configuration>
        <testFailureIgnore>true</testFailureIgnore>
    </configuration>
</plugin>

3.2. A Membership Points Example

Initially, our tests are based on the typical membership points feature in an e-commerce application. A customer can join the member program. As the customer purchases goods on the platform, the membership points will increase, and the customer’s membership grade would grow accordingly.

Now let’s write several tests against the scenarios described above and see how Serenity works.

First, let’s write the test for membership initialisation and see which steps do we need:

@RunWith(SerenityRunner.class)
public class MemberStatusLiveTest {

    @Steps 
    private MemberStatusSteps memberSteps;

    @Test
    public void membersShouldStartWithBronzeStatus() {
        memberSteps.aClientJoinsTheMemberProgram();
        memberSteps.theMemberShouldHaveAStatusOf(Bronze);
    }
}

Then we implement the two steps as follows:

public class MemberStatusSteps {

    private Member member;

    @Step("Given a member has {0} points")
    public void aMemberHasPointsOf(int points) {
        member = Member.withInitialPoints(points);
    }

    @Step("Then the member grade should be {0}")
    public void theMemberShouldHaveAStatusOf(MemberGrade grade) {
        assertThat(member.getGrade(), equalTo(grade));
    }
}

Now we are ready to run an integration test with mvn clean verify. The reports will be located at target/site/serenity/index.html:

From the report, we can see that we have only one acceptance test ‘Members should start with bronze status, has the ability to’ and is passing. By clicking on the test, the steps is illustrated:

As we can see, Serenity’s report gives us a thorough understanding of what our application is doing and if it aligns our requirements. If we have some steps to implement, we can mark them as @Pending:

@Pending
@Step("When the member exchange {}")
public void aMemberExchangeA(Commodity commodity){
    //TODO
}

The report would remind us what needs to be done next. And in case any test fails, it can be seen in the report as well:

Each failed, ignored or skipped step will be listed respectively:

4. Integration with JBehave

Serenity can also integrate with existing BDD frameworks such as JBehave.

4.1. Maven Dependencies

To integrate with JBehave, one more dependency serenity-jbehave is needed in the POM:

<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-jbehave</artifactId>
    <version>1.24.0</version>
</dependency>

4.2. JBehave Github REST API Test Continued

As we have introduced how to do REST API testing with JBehave, we can continue with our JBehave REST API test and see how it fits in Serenity.

Our story was:

Scenario: Github user's profile should have a login payload same as username
 
Given github user profile api
When I look for eugenp via the api
Then github's response contains a 'login' payload same as eugenp

The Given_When_Then steps can be migrated to as @Steps without any changes:

public class GithubRestUserAPISteps {

    private String api;
    private GitHubUser resource;

    @Step("Given the github REST API for user profile")
    public void withUserProfileAPIEndpoint() {
        api = "https://api.github.com/users/%s";
    }

    @Step("When looking for {0} via the api")
    public void getProfileOfUser(String username) throws IOException {
        HttpResponse httpResponse = getGithubUserProfile(api, username);
        resource = retrieveResourceFromResponse(httpResponse, GitHubUser.class);
    }

    @Step("Then there should be a login field with value {0} in payload of user {0}")
    public void profilePayloadShouldContainLoginValue(String username) {
        assertThat(username, Matchers.is(resource.getLogin()));
    }

}

To make JBehave’s story-to-code mapping work as expected, we need to implement JBehave’s step definition using @Steps:

public class GithubUserProfilePayloadStepDefinitions {

    @Steps
    GithubRestUserAPISteps userAPISteps;

    @Given("github user profile api")
    public void givenGithubUserProfileApi() {
        userAPISteps.withUserProfileAPIEndpoint();
    }

    @When("looking for $user via the api")
    public void whenLookingForProfileOf(String user) throws IOException {
        userAPISteps.getProfileOfUser(user);
    }

    @Then("github's response contains a 'login' payload same as $user")
    public void thenGithubsResponseContainsAloginPayloadSameAs(String user) {
        userAPISteps.profilePayloadShouldContainLoginValue(user);
    }
}

With SerenityStories, we can run JBehave tests both from within our IDE and in the build process:

import net.serenitybdd.jbehave.SerenityStory;

public class GithubUserProfilePayload extends SerenityStory {}

After the verify build finished, we can see our test report:

Compared to plain text report of JBehave, the rich report by Serenity gives us a more eye-pleasing and live overview of our story and the test result.

5. Integration with REST-assured

It is noteworthy that Serenity supports integration with REST-assured. To have a review of REST-assured, take a look at the guide to REST-assured.

5.1. Maven Dependencies

To make use of REST-assured with Serenity, the serenity-rest-assured dependency should be included:

<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-rest-assured</artifactId>
    <version>1.2.5-rc.11</version>
</dependency>

5.2. Use REST-assured in Github REST API Test

Now we can replace our web client with REST-assured utilities:

import static net.serenitybdd.rest.SerenityRest.rest;
import static net.serenitybdd.rest.SerenityRest.then;

public class GithubRestAssuredUserAPISteps {

    private String api;

    @Step("Given the github REST API for user profile")
    public void withUserProfileAPIEndpoint() {
        api = "https://api.github.com/users/{username}";
    }

    @Step("When looking for {0} via the api")
    public void getProfileOfUser(String username) throws IOException {
        rest().get(api, username);
    }

    @Step("Then there should be a login field with value {0} in payload of user {0}")
    public void profilePayloadShouldContainLoginValue(String username) {
        then().body("login", Matchers.equalTo(username));
    }

}

After replacing the implementation of userAPISteps in the StepDefition, we can re-run the verify build:

public class GithubUserProfilePayloadStepDefinitions {

    @Steps
    GithubRestAssuredUserAPISteps userAPISteps;

    //...

}

In the report, we can see the actual API invoked during the test, and by clicking on the REST Query button, the details of request and response will be presented:

6. Integration with JIRA

As of now, we already have a great test report describing details and status of our requirements with Serenity. But for an agile team, issue tracking systems such as JIRA are often used to keep track of requirements. It would be better if we could use them seamlessly.

Luckily, Serenity already supports integration with JIRA.

6.1. Maven Dependencies

To integrate with JIRA, we need another dependency: serenity-jira-requirements-provider.

<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-jira-requirements-provider</artifactId>
    <version>1.1.3-rc.5</version>
</dependency>

6.2. One-way Integration

To add JIRA links in the story, we can add the JIRA issue using story’s meta tag:

Meta:
@issue #BDDTEST-1

Besides, JIRA account and links should be specified in the file serenity.properties at the root of the project:

jira.url=<jira-url>
jira.project=<jira-project>
jira.username=<jira-username>
jira.password=<jira-password>

Then there would be a JIRA link appended in the report:

Serenity also supports two-way integration with JIRA, we can refer to the official documentation for more details.

7. Summary

In this article, we introduced Serenity BDD and multiple integrations with other test frameworks and requirement management systems.

Although we have covered most of what Serenity can do, it can certainly do more. In our next article, we’ll cover on how Serenity with WebDriver support can enable us to automate web application pages using screenplay.

As always, the full implementation code can be found over on the GitHub project.