1. Introduction
In this tutorial, we’re going to look at how to query a relational database using Exposed.
Exposed is an open source library (Apache license) developed by JetBrains, which provides an idiomatic Kotlin API for some relational database implementations while smoothing out the differences among database vendors.
Exposed can be used both as a high-level DSL over SQL and as a lightweight ORM (Object-Relational Mapping). Thus, we’ll cover both usages over the course of this tutorial.
2. Exposed Framework Setup
Exposed isn’t yet on Maven Central, so we have to use a dedicated repository:
<repositories>
<repository>
<id>exposed</id>
<name>exposed</name>
<url>https://dl.bintray.com/kotlin/exposed</url>
</repository>
</repositories>
Then, we can include the library:
<dependency>
<groupId>org.jetbrains.exposed</groupId>
<artifactId>exposed</artifactId>
<version>0.10.4</version>
</dependency>
Also, in the following sections, we’ll show examples using the H2 database in memory:
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.4.197</version>
</dependency>
We can find the latest version of Exposed on Bintray and the latest version of H2 on Maven Central.
3. Connecting to the Database
We define database connections with the Database class:
Database.connect("jdbc:h2:mem:test", driver = "org.h2.Driver")
We can also specify a user and a password as named parameters:
Database.connect( "jdbc:h2:mem:test", driver = "org.h2.Driver", user = "myself", password = "secret")
Note that invoking connect doesn’t establish a connection to the DB right away. It just saves the connection parameters for later.
3.1. Additional Parameters
If we need to provide other connection parameters, we’ll use a different overload of the connect method that gives us full control over the acquisition of a database connection:
Database.connect({ DriverManager.getConnection("jdbc:h2:mem:test;MODE=MySQL") })
This version of connect requires a closure parameter. Exposed invokes the closure whenever it needs a new connection to the database.
3.2. Using a DataSource
If, instead, we connect to the database using a DataSource, as is usually the case in enterprise applications (e.g., to benefit from connection pooling), we can use the appropriate connect overload:
Database.connect(datasource)
4. Opening a Transaction
Every database operation in Exposed needs an active transaction.
The transaction method takes a closure and invokes it with an active transaction:
transaction {
//Do cool stuff
}
The transaction returns whatever the closure returns. Then, Exposed automatically closes the transaction when the execution of the block terminates.
4.1. Commit and Rollback
When the transaction block successfully returns, Exposed commits the transaction. When, instead, the closure exits by throwing an exception, the framework rolls back the transaction.
We can also manually commit or roll back a transaction. The closure that we provide to transaction is actually an instance of the Transaction class thanks to Kotlin magic.
Thus, we have a commit and a rollback method available:
transaction {
//Do some stuff
commit()
//Do other stuff
}
4.2. Logging Statements
When learning the framework or debugging, we might find it useful to inspect the SQL statements and queries that Exposed sends to the database.
We can easily add such a logger to the active transaction:
transaction {
addLogger(StdOutSqlLogger)
//Do stuff
}
5. Defining Tables
Usually, in Exposed we don’t work with raw SQL strings and names. Instead, we define tables, columns, keys, relationships, etc., using a high-level DSL.
We represent each table with an instance of the Table class:
object StarWarsFilms : Table()
Exposed automatically computes the name of the table from the class name, but we can also provide an explicit name:
object StarWarsFilms : Table("STAR_WARS_FILMS")
5.1. Columns
A table is meaningless without columns. We define columns as properties of our table class:
object StarWarsFilms : Table() {
val id = integer("id").autoIncrement().primaryKey()
val sequelId = integer("sequel_id").uniqueIndex()
val name = varchar("name", 50)
val director = varchar("director", 50)
}
We’ve omitted the types for brevity, as Kotlin can infer them for us. Anyway, each column is of type Column<T> and it has a name, a type and possibly type parameters.
5.2. Primary Keys
As we can see from the example in the previous section, we can easily define indexes and primary keys with a fluent API.
However, for the common case of a table with an integer primary key, Exposed provides classes IntIdTable and LongIdTable that define the key for us:
object StarWarsFilms : IntIdTable() {
val sequelId = integer("sequel_id").uniqueIndex()
val name = varchar("name", 50)
val director = varchar("director", 50)
}
There’s also a UUIDTable; furthermore, we can define our own variants by subclassing IdTable.
5.3. Foreign Keys
Foreign keys are easy to introduce. We also benefit from static typing because we always refer to properties known at compile time.
Suppose we want to track the names of the actors playing in each movie:
object Players : Table() {
val sequelId = integer("sequel_id")
.uniqueIndex()
.references(StarWarsFilms.sequelId)
val name = varchar("name", 50)
}
To avoid having to spell the type of the column (in this case, integer) when it can be derived from the referenced column, we can use the reference method as a shorthand:
val sequelId = reference("sequel_id", StarWarsFilms.sequelId).uniqueIndex()
If the reference is to the primary key, we can omit the column’s name:
val filmId = reference("film_id", StarWarsFilms)
5.4. Creating Tables
We can create the tables as defined above programmatically:
transaction {
SchemaUtils.create(StarWarsFilms, Players)
//Do stuff
}
The tables are only created if they don’t already exist. However, there’s no support for database migrations.
6. Queries
Once we’ve defined some table classes as we’ve shown in the previous sections, we can issue queries to the database by using the extension functions provided by the framework.
6.1. Select All
To extract data from the database, we use Query objects built from table classes. The simplest query is one that returns all the rows of a given table:
val query = StarWarsFilms.selectAll()
A query is an Iterable, so it supports forEach:
query.forEach {
assertTrue { it[StarWarsFilms.sequelId] >= 7 }
}
The closure parameter, implicitly called it in the example above, is an instance of the ResultRow class. We can see it as a map keyed by column.
6.2. Selecting a Subset of Columns
We can also select a subset of the table’s columns, i.e., perform a projection, using the slice method:
StarWarsFilms.slice(StarWarsFilms.name, StarWarsFilms.director).selectAll()
.forEach {
assertTrue { it[StarWarsFilms.name].startsWith("The") }
}
We use slice to apply a function to a column, too:
StarWarsFilms.slice(StarWarsFilms.name.countDistinct())
Often, when using aggregate functions such as count and avg, we’ll need a group by clause in the query. We’ll talk about the group by in section 6.5.
6.3. Filtering with Where Expressions
Exposed contains a dedicated DSL for where expressions, which are used to filter queries and other types of statements. This is a mini-language based on the column properties we’ve encountered earlier and a series of boolean operators.
This is a where expression:
{ (StarWarsFilms.director like "J.J.%") and (StarWarsFilms.sequelId eq 7) }
Its type is complex; it’s a subclass of SqlExpressionBuilder, which defines operators such as like, eq, and. As we can see, it is a sequence of comparisons combined together with and and or operators.
We can pass such an expression to the select method, which again returns a query:
val select = StarWarsFilms.select { ... }
assertEquals(1, select.count())
Thanks to type inference, we don’t need to spell out the complex type of the where expression when it’s directly passed to the select method as in the above example.
Since where expressions are Kotlin objects, there are no special provisions for query parameters. We simply use variables:
val sequelNo = 7
StarWarsFilms.select { StarWarsFilms.sequelId >= sequelNo }
6.4. Advanced Filtering
The Query objects returned by select and its variants have a number of methods that we can use to refine the query.
For example, we might want to exclude duplicate rows:
query.withDistinct(true).forEach { ... }
Or we might want to only return a subset of the rows, for example when paginating the results for the UI:
query.limit(20, offset = 40).forEach { ... }
These methods return a new Query, so we can easily chain them.
6.5. Order By and Group By
The Query.orderBy method accepts a list of columns mapped to a SortOrder value indicating if sorting should be ascending or descending:
query.orderBy(StarWarsFilms.name to SortOrder.ASC)
While the grouping by one or more columns, useful in particular when using aggregate functions (see section 6.2.), is achieved using the groupBy method:
StarWarsFilms .slice(StarWarsFilms.sequelId.count(), StarWarsFilms.director) .selectAll() .groupBy(StarWarsFilms.director)
6.6. Joins
Joins are arguably one of the selling points of relational databases. In the most simple of cases, when we have a foreign key and no join conditions, we can use one of the built-in join operators:
(StarWarsFilms innerJoin Players).selectAll()
Here we’ve shown innerJoin, but we also have left, right and cross join available with the same principle.
Then, we can add join conditions with a where expression; for example, if there isn’t a foreign key and we must perform the join explicitly:
(StarWarsFilms innerJoin Players)
.select { StarWarsFilms.sequelId eq Players.sequelId }
In the general case, the full form of a join is the following:
val complexJoin = Join(
StarWarsFilms, Players,
onColumn = StarWarsFilms.sequelId, otherColumn = Players.sequelId,
joinType = JoinType.INNER,
additionalConstraint = { StarWarsFilms.sequelId eq 8 })
complexJoin.selectAll()
6.7. Aliasing
Thanks to the mapping of column names to properties, we don’t need any aliasing in a typical join, even when the columns happen to have the same name:
(StarWarsFilms innerJoin Players)
.selectAll()
.forEach {
assertEquals(it[StarWarsFilms.sequelId], it[Players.sequelId])
}
In fact, in the above example, StarWarsFilms.sequelId and Players.sequelId are different columns.
However, when the same table appears more than once in a query, we might want to give it an alias. For that we use the alias function:
val sequel = StarWarsFilms.alias("sequel")
We can then use the alias a bit like a table:
Join(StarWarsFilms, sequel,
additionalConstraint = {
sequel[StarWarsFilms.sequelId] eq StarWarsFilms.sequelId + 1
}).selectAll().forEach {
assertEquals(
it[sequel[StarWarsFilms.sequelId]], it[StarWarsFilms.sequelId] + 1)
}
In the above example, we can see that the sequel alias is a table participating in a join. When we want to access one of its columns, we use the aliased table’s column as a key:
sequel[StarWarsFilms.sequelId]
7. Statements
Now that we’ve seen how to query the database, let’s look at how to perform DML statements.
7.1. Inserting Data
To insert data, we call one of the variants of the insert function. All variants take a closure:
StarWarsFilms.insert {
it[name] = "The Last Jedi"
it[sequelId] = 8
it[director] = "Rian Johnson"
}
There are two notable objects involved in the closure above:
- this (the closure itself) is an instance of the StarWarsFilms class; that’s why we can access the columns, which are properties, by their unqualified name
- it (the closure parameter) is an InsertStatement; it is a map-like structure with a slot for each column to insert
7.2. Extracting Auto-Increment Column Values
When we have an insert statement with auto-generated columns (typically auto-increment or sequences), we may want to obtain the generated values.
In the typical case, we only have one generated value and we call insertAndGetId:
val id = StarWarsFilms.insertAndGetId {
it[name] = "The Last Jedi"
it[sequelId] = 8
it[director] = "Rian Johnson"
}
assertEquals(1, id.value)
If we have more than one generated value, we can read them by name:
val insert = StarWarsFilms.insert {
it[name] = "The Force Awakens"
it[sequelId] = 7
it[director] = "J.J. Abrams"
}
assertEquals(2, insert[StarWarsFilms.id]?.value)
7.3. Updating Data
We can now use what we have learned about queries and insertions to update existing data in the database. Indeed, a simple update looks like a combination of a select with an insert:
StarWarsFilms.update ({ StarWarsFilms.sequelId eq 8 }) {
it[name] = "Episode VIII – The Last Jedi"
}
We can see the use of a where expression combined with an UpdateStatement closure. In fact, UpdateStatement and InsertStatement share most of the API and logic through a common superclass, UpdateBuilder, which provides the ability to set the value of a column using idiomatic square brackets.
When we need to update a column by computing a new value from the old value, we leverage the SqlExpressionBuilder:
StarWarsFilms.update ({ StarWarsFilms.sequelId eq 8 }) {
with(SqlExpressionBuilder) {
it.update(StarWarsFilms.sequelId, StarWarsFilms.sequelId + 1)
}
}
This is an object that provides infix operators (like plus, minus and so on) that we can use to build an update instruction.
7.4. Deleting Data
Finally, we can delete data with the deleteWhere method:
StarWarsFilms.deleteWhere ({ StarWarsFilms.sequelId eq 8 })
8. The DAO API, a Lightweight ORM
So far, we’ve used Exposed to directly map from operations on Kotlin objects to SQL queries and statements. Each method invocation like insert, update, select and so on results in a SQL string being immediately sent to the database.
However, Exposed also has a higher-level DAO API that constitutes a simple ORM. Let’s now dive into that.
8.1. Entities
In the previous sections, we’ve used classes to represent database tables and to express operations over them, using static methods.
Moving a step further, we can define entities based on those table classes, where each instance of an entity represents a database row:
class StarWarsFilm(id: EntityID<Int>) : Entity<Int>(id) {
companion object : EntityClass<Int, StarWarsFilm>(StarWarsFilms)
var sequelId by StarWarsFilms.sequelId
var name by StarWarsFilms.name
var director by StarWarsFilms.director
}
Let’s now analyze the above definition piece by piece.
In the first line, we can see that an entity is a class extending Entity. It has an ID with a specific type, in this case, Int.
class StarWarsFilm(id: EntityID<Int>) : Entity<Int>(id) {
Then, we encounter a companion object definition. The companion object represents the entity class, that is, the static metadata defining the entity and the operations we can perform on it.
Furthermore, in the declaration of the companion object, we connect the entity, StarWarsFilm – singular, as it represents a single row – to the table, StarWarsFilms – plural, because it represents the collection of all the rows.
companion object : EntityClass<Int, StarWarsFilm>(StarWarsFilms)
Finally, we have the properties, implemented as property delegates to the corresponding table columns.
var sequelId by StarWarsFilms.sequelId var name by StarWarsFilms.name var director by StarWarsFilms.director
Note that previously we declared the columns with val because they’re immutable metadata. Now, instead, we’re declaring the entity properties with var, because they’re mutable slots in a database row.
8.2. Inserting Data
To insert a row in a table, we simply create a new instance of our entity class using the static factory method new in a transaction:
val theLastJedi = StarWarsFilm.new {
name = "The Last Jedi"
sequelId = 8
director = "Rian Johnson"
}
Note that operations against the database are performed lazily; they’re only issued when the warm cache is flushed. For comparison, Hibernate calls the warm cache a session.
This happens automatically when required; e.g., the first time we read the generated identifier, Exposed silently executes the insert statement:
assertEquals(1, theLastJedi.id.value) //Reading the ID causes a flush
Compare this behavior with the insert method from section 7.1., which immediately issues a statement against the database. Here, we’re working at a higher level of abstraction.
8.3. Updating and Deleting Objects
To update a row, we simply assign to its properties:
theLastJedi.name = "Episode VIII – The Last Jedi"
While to delete an object we call delete on it:
theLastJedi.delete()
As with new, the update and operations are performed lazily.
Updates and deletions can only be performed on a previously loaded object. There is no API for massive updates and deletions. Instead, we have to use the lower-level API that we’ve seen in section 7. Still, the two APIs can be used together in the same transaction.
8.4. Querying
With the DAO API, we can perform three types of queries.
To load all the objects without conditions we use the static method all:
val movies = StarWarsFilm.all()
To load a single object by ID we call findById:
val theLastJedi = StarWarsFilm.findById(1)
If there’s no object with that ID, findById returns null.
Finally, in the general case, we use find with a where expression:
val movies = StarWarsFilm.find { StarWarsFilms.sequelId eq 8 }
8.5. Many-to-One Associations
Just as joins are an important feature of relational databases, the mapping of joins to references is an important aspect of an ORM. So, let’s see what Exposed has to offer.
Suppose we want to track the rating of each movie by users. First, we define two additional tables:
object Users: IntIdTable() {
val name = varchar("name", 50)
}
object UserRatings: IntIdTable() {
val value = long("value")
val film = reference("film", StarWarsFilms)
val user = reference("user", Users)
}
Then, we’ll write the corresponding entities. Let’s omit the User entity, which is trivial, and move straight to the UserRating class:
class UserRating(id: EntityID<Int>): IntEntity(id) {
companion object : IntEntityClass<UserRating>(UserRatings)
var value by UserRatings.value
var film by StarWarsFilm referencedOn UserRatings.film
var user by User referencedOn UserRatings.user
}
In particular, note the referencedOn infix method call on properties that represent associations. The pattern is the following: a var declaration, by the referenced entity, referencedOn the referencing column.
Properties declared this way behave like regular properties, but their value is the associated object:
val someUser = User.new {
name = "Some User"
}
val rating = UserRating.new {
value = 9
user = someUser
film = theLastJedi
}
assertEquals(theLastJedi, rating.film)
8.6. Optional Associations
The associations we’ve seen in the previous section are mandatory, that is, we must always specify a value.
If we want an optional association, we must first declare the column as nullable in the table:
val user = reference("user", Users).nullable()
Then, we’ll use optionalReferencedOn instead of referencedOn in the entity:
var user by User optionalReferencedOn UserRatings.user
That way, the user property will be nullable.
8.7. One-to-Many Associations
We might also want to map the opposite side of the association. A rating is about a film, that’s what we model in the database with a foreign key; consequently, a film has a number of ratings.
To map a film’s ratings, we simply add a property to the “one” side of the association, that is, the film entity in our example:
class StarWarsFilm(id: EntityID<Int>) : Entity<Int>(id) {
//Other properties elided
val ratings by UserRating referrersOn UserRatings.film
}
The pattern is similar to that of many-to-one relationships, but it uses referrersOn. The property thus defined is an Iterable, so we can traverse it with forEach:
theLastJedi.ratings.forEach { ... }
Note that, unlike regular properties, we’ve defined ratings with val. Indeed, the property is immutable, we can only read it.
The value of the property has no API for mutation as well. So, to add a new rating, we must create it with a reference to the film:
UserRating.new {
value = 8
user = someUser
film = theLastJedi
}
Then, the film’s ratings list will contain the newly added rating.
8.8. Many-to-Many Associations
In some cases, we might need a many-to-many association. Let’s say we want to add a reference an Actors table to the StarWarsFilm class:
object Actors: IntIdTable() {
val firstname = varchar("firstname", 50)
val lastname = varchar("lastname", 50)
}
class Actor(id: EntityID<Int>): IntEntity(id) {
companion object : IntEntityClass<Actor>(Actors)
var firstname by Actors.firstname
var lastname by Actors.lastname
}
Having defined the table and the entity, we need another table to represent the association:
object StarWarsFilmActors : Table() {
val starWarsFilm = reference("starWarsFilm", StarWarsFilms).primaryKey(0)
val actor = reference("actor", Actors).primaryKey(1)
}
The table has two columns that are both foreign keys and that also make up a composite primary key.
Finally, we can connect the association table with the StarWarsFilm entity:
class StarWarsFilm(id: EntityID<Int>) : IntEntity(id) {
companion object : IntEntityClass<StarWarsFilm>(StarWarsFilms)
//Other properties elided
var actors by Actor via StarWarsFilmActors
}
At the time of writing, it’s not possible to create an entity with a generated identifier and include it in a many-to-many association in the same transaction.
In fact, we have to use multiple transactions:
//First, create the film
val film = transaction {
StarWarsFilm.new {
name = "The Last Jedi"
sequelId = 8
director = "Rian Johnson"r
}
}
//Then, create the actor
val actor = transaction {
Actor.new {
firstname = "Daisy"
lastname = "Ridley"
}
}
//Finally, link the two together
transaction {
film.actors = SizedCollection(listOf(actor))
}
Here, we’ve used three different transactions for convenience. However, two would have been sufficient.
9. Conclusion
In this article, we’ve given a thorough overview of the Exposed framework for Kotlin. For additional information and examples, see the Exposed wiki.
The implementation of all these examples and code snippets can be found in the GitHub project as a Maven project, so it should be easy to import and run as is.