1. Overview
In this tutorial, we’ll take a look at Spock extensions.
Sometimes, we might need to modify or enhance our spec’s lifecycle. For example, we’d like to add some conditional execution, retry on randomly failing integration test, and more. For this, we can use Spock’s extension mechanism.
Spock has a wide range of various extensions that we can hook onto a specification’s lifecycle.
Let’s discover how to use the most common extensions.
2. Maven Dependencies
Before we start, let’s set up our Maven dependencies:
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>z
<version>1.3-groovy-2.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.4.7</version>
<scope>test</scope>
</dependency>
3. Annotation-Based Extensions
Most of Spock‘s built-in extensions are based on annotations.
We can add annotations on a spec class or feature to trigger a specific behavior.
3.1. @Ignore
Sometimes we need to ignore some feature methods or spec classes. Like, we might need to merge our changes as soon as possible, but continuous integration still fails. We can ignore some specs and still make a successful merge.
We can use @Ignore on a method level to skip a single specification method:
@Ignore
def "I won't be executed"() {
expect:
true
}
Spock won’t execute this test method. And most IDEs will mark the test as skipped.
Additionally, we can use @Ignore on the class level:
@Ignore class IgnoreTest extends Specification
We can simply provide a reason why our test suite or method is ignored:
@Ignore("probably no longer needed")
3.2. @IgnoreRest
Likewise, we can ignore all specifications except one, which we can mark with a @IgnoreRest annotation:
def "I won't run"() { }
@IgnoreRest
def 'I will run'() { }
def "I won't run too"() { }
3.3. @IgnoreIf
Sometimes, we’d like to conditionally ignore a test or two. In that case, we can use @IgnoreIf, which accepts a predicate as an argument:
@IgnoreIf({System.getProperty("os.name").contains("windows")})
def "I won't run on windows"() { }
Spock provides the set of properties and helper classes, to make our predicates easier to read and write:
- os – Information about the operating system (see spock.util.environment.OperatingSystem).
- jvm – the JVM’s information (see spock.util.environment.Jvm).
- sys – System’s properties in a map.
- env – Environment variables in a map.
We can re-write the previous example throughout the use of os property. Actually, it’s the spock.util.environment.OperatingSystem class with some useful methods, like for example isWindows():
@IgnoreIf({ os.isWindows() })
def "I'm using Spock helper classes to run only on windows"() {}
Note, that Spock uses System.getProperty(…) underhood. The main goal is to provide a clear interface, rather than defining complicated rules and conditions.
Also, as in the previous examples, we can apply the @IgnoreIf annotation at the class level.
3.4. @Requires
Sometimes, it’s easier to invert our predicate logic from @IgnoreIf. In that case, we can use @Requires:
@Requires({ System.getProperty("os.name").contains("windows") })
def "I will run only on Windows"()
So, while the @Requires makes this test run only if the OS is Windows, the @IgnoreIf, using the same predicate, makes the test run only if the OS is not Windows.
In general, it’s much better to say under which condition the test will execute, rather than when it gets ignored.
3.5. @PendingFeature
In TDD, we write tests first. Then, we need to write a code to make these tests pass. In some cases, we will need to commit our tests before the feature is implemented.
This is a good use case for @PendingFeature:
@PendingFeature def 'test for not implemented yet feature. Maybe in the future it will pass'()
There is one main difference between @Ignore and @PendingFeature. In @PedingFeature, tests are executed, but any failures are ignored.
If a test marked with @PendingFeature ends without error, then it will be reported as a failure, to remind about removing annotation.
In this way, we can initially ignore fails of not implemented features, but in the future, these specs will become a part of normal tests, instead of being ignored forever.
3.6. @Stepwise
We can execute a spec’s methods in a given order with the @Stepwise annotation:
def 'I will run as first'() { }
def 'I will run as second'() { }
In general, tests should be deterministic. One should not depend on another. That’s why we should avoid using @Stepwise annotation.
But if we have to, we need to be aware that @Stepwise doesn’t override the behavior of @Ignore, @IgnoreRest, or @IgnoreIf. We should be careful with combining these annotations with @Stepwise.
3.7. @Timeout
We can limit the execution time of a spec’s single method and fail earlier:
@Timeout(1)
def 'I have one second to finish'() { }
Note, that this is the timeout for a single iteration, not counting the time spent in fixture methods.
By default, the spock.lang.Timeout uses seconds as a base time unit. But, we can specify other time units:
@Timeout(value = 200, unit = TimeUnit.SECONDS)
def 'I will fail after 200 millis'() { }
@Timeout on the class level has the same effect as applying it to every feature method separately:
@Timeout(5)
class ExampleTest extends Specification {
@Timeout(1)
def 'I have one second to finish'() {
}
def 'I will have 5 seconds timeout'() {}
}
Using @Timeout on a single spec method always overrides class level.
3.8. @Retry
Sometimes, we can have some non-deterministic integration tests. These may fail in some runs for reasons such as async processing or depending on other HTTP clients response. Moreover, the remote server with build and CI will fail and force us to run the tests and build again.
To avoid this situation, we can use @Retry annotation on a method or class level, to repeat failed tests:
@Retry
def 'I will retry three times'() { }
By default, it will retry three times.
It’s very useful to determine the conditions, under which we should retry our test. We can specify the list of exceptions:
@Retry(exceptions = [RuntimeException])
def 'I will retry only on RuntimeException'() { }
Or when there is a specific exception message:
@Retry(condition = { failure.message.contains('error') })
def 'I will retry with a specific message'() { }
Very useful is a retry with a delay:
@Retry(delay = 1000)
def 'I will retry after 1000 millis'() { }
And finally, like almost always, we can specify retry on the class level:
@Retry class RetryTest extends Specification
3.9. @RestoreSystemProperties
We can manipulate environment variables with @RestoreSystemProperties.
This annotation, when applied, saves the current state of variables and restores them afterward. It also includes setup or cleanup methods:
@RestoreSystemProperties
def 'all environment variables will be saved before execution and restored after tests'() {
given:
System.setProperty('os.name', 'Mac OS')
}
Please note that we shouldn’t run the tests concurrently when we’re manipulating the system properties. Our tests might be non-deterministic.
3.10. Human-Friendly Titles
We can add a human-friendly test title by using the @Title annotation:
@Title("This title is easy to read for humans")
class CustomTitleTest extends Specification
Similarly, we can add a description of the spec with @Narrative annotation and with a multi-line Groovy String:
@Narrative("""
as a user
i want to save favourite items
and then get the list of them
""")
class NarrativeDescriptionTest extends Specification
3.11. @See
To link one or more external references, we can use the @See annotation:
@See("https://example.org")
def 'Look at the reference'()
To pass more than one link, we can use the Groovy [] operand for creating a list:
@See(["https://example.org/first", "https://example.org/first"]) def 'Look at the references'()
3.12. @Issue
We can denote that a feature method refers to an issue or multiple issues:
@Issue("https://jira.org/issues/LO-531")
def 'single issue'() {
}
@Issue(["https://jira.org/issues/LO-531", "http://jira.org/issues/LO-123"])
def 'multiple issues'()
3.13. @Subject
And finally, we can indicate which class is the class under test with @Subject:
@Subject ItemService itemService // initialization here...
Right now, it’s only for informational purposes.
4. Configuring Extensions
We can configure some of the extensions in the Spock configuration file. This includes describing how each extension should behave.
Usually, we create a configuration file in Groovy, called, for example, SpockConfig.groovy.
Of course, Spock needs to find our config file. First of all, it reads a custom location from the spock.configuration system property and then tries to find the file in the classpath. When not found, it goes to a location in the file system. If it’s still not found, then it looks for SpockConfig.groovy in the test execution classpath.
Eventually, Spock goes to a Spock user home, which is just a directory .spock within our home directory. We can change this directory by setting system property called spock.user.home or by an environment variable SPOCK_USER_HOME.
For our examples, we’ll create a file SpockConfig.groovy and put it on the classpath (src/test/resources/SpockConfig.Groovy).
4.1. Filtering the Stack Trace
By using a configuration file, we can filter (or not) the stack traces:
runner {
filterStackTrace false
}
The default value is true.
To see how it works and practice, let’s create a simple test which throws a RuntimeException:
def 'stacktrace'() {
expect:
throw new RuntimeException("blabla")
}
When filterStackTrace is set to false, then we’ll see in the output:
java.lang.RuntimeException: blabla at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83) at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247) // 34 more lines in the stack trace...
By setting this property to true, we’ll get:
java.lang.RuntimeException: blabla at extensions.StackTraceTest.stacktrace(StackTraceTest.groovy:10)
Although keep in mind, sometimes it’s useful to see the full stack trace.
4.2. Conditional Features in Spock Configuration File
Sometimes, we might need to filter stack traces conditionally. For example, we’ll need to see full stack traces in a Continuous Integration tool, but this isn’t necessary on our local machine.
We can add a simple condition, based for example on the environment variables:
if (System.getenv("FILTER_STACKTRACE") == null) {
filterStackTrace false
}
The Spock configuration file is a Groovy file, so it can contain snippets of Groovycode.
4.3. Prefix and URL in @Issue
Previously, we talked about the @Issue annotation. We can also configure this using the configuration file, by defining a common URL part with issueUrlPrefix.
The other property is issueNamePrefix. Then, every @Issue value is preceded by the issueNamePrefix property.
We need to add these two properties in the report:
report {
issueNamePrefix 'Bug '
issueUrlPrefix 'https://jira.org/issues/'
}
4.4. Optimize Run Order
The other very helpful tool is optimizeRunOrder. Spock can remember which specs failed and how often and how much time it needs to execute a feature method.
Based on this knowledge, Spock will first run the features which failed in the last run. In the first place, it will execute the specs which failed more successively. Furthermore, the fastest specs will run first.
This behavior may be enabled in the configuration file. To enable optimizer, we use optimizeRunOrder property:
runner {
optimizeRunOrder true
}
By default, the optimizer for run order is disabled.
4.5. Including and Excluding Specifications
Spock can exclude or include certain specs. We can lean on classes, super-classes, interfaces or annotations, which are applied on specification classes. The library can be of capable excluding or including single features, based on the annotation on a feature level.
We can simply exclude a test suite from class TimeoutTest by using the exclude property:
import extensions.TimeoutTest
runner {
exclude TimeoutTest
}
TimeoutTest and all its subclasses will be excluded. If TimeoutTest was an annotation applied on a spec’s class, then this spec would be excluded.
We can specify annotations and base classes separately:
import extensions.TimeoutTest
import spock.lang.Issue
exclude {
baseClass TimeoutTest
annotation Issue
}
The above example will exclude test classes or methods with the @Issue annotation as well as TimeoutTest or any of its subclasses.
To include any spec, we simply use include property. We can define the rules of include in the same way as exclude.
4.6. Creating a Report
Based on the test results and previously known annotations, we can generate a report with Spock. Additionally, this report will contain things like @Title, @See, @Issue, and @Narrative values.
We can enable generating a report in the configuration file. By default, it won’t generate the report.
All we have to do is pass values for a few properties:
report {
enabled true
logFileDir '.'
logFileName 'report.json'
logFileSuffix new Date().format('yyyy-MM-dd')
}
The properties above are:
- enabled – should or not generate the report
- logFileDir – directory of report
- logFileName – the name of the report
- logFileSuffix – a suffix for every generated report basename separated with a dash
When we set enabled to true, then it’s mandatory to set logFileDir and logFileName properties. The logFileSuffix is optional.
We can also set all of them in system properties: enabled, spock.logFileDir, spock.logFileName and spock.logFileSuffix.
5. Conclusion
In this article, we described the most common Spock extensions.
We know that most of them are based on annotations. In addition, we learned how to create a Spock configuration file, and what the available configuration options are. In short, our newly acquired knowledge is very helpful for writing effective and easy to read tests.
The implementation of all our examples can be found in our Github project.