1. Introduction
Recently, we looked at an algorithm for solving the game 2048. We discussed this from a theoretical point of view, and not with any real code behind it.
Here we're going to write an implementation of this in Java. This will play as both the human and computer players, showing how well a more optimal game can be played.
2. Initial Setup
The first thing we need is a setup in which we can play the game and see how progress is going.
This will give us all of the constructs that we need to play the game, and fully implement the computer player – which only places random tiles anyway. This then gives us the scope to implement a “human” player to play the game.
2.1. Game Board
Before anything else, we need a game board. This is a grid of cells into which numbers can be placed.
To make some things a bit easier to work with, let's begin with a simple representation of a cell location. This is literally just a wrapper around a pair of coordinates:
public class Cell {
private final int x;
private final int y;
// constructor, getters, and toString
}
We can now write a class to represent the board itself. This is going to store the values in a simple two-dimensional array, but allow us to access them via the above Cell class:
public class Board {
private final int[][] board;
private final int score;
public Board(int size) {
this.board = new int[size][];
this.score = 0;
for (int x = 0; x < size; ++x) {
this.board[x] = new int[size];
for (int y = 0; y < size; ++y) {
board[x][y] = 0;
}
}
}
public int getSize() {
return board.length;
}
public int getScore() {
return score;
}
public int getCell(Cell cell) {
return board[cell.getX()][cell.getY()];
}
public boolean isEmpty(Cell cell) {
return getCell(cell) == 0;
}
public List<Cell> emptyCells() {
List<Cell> result = new ArrayList<>();
for (int x = 0; x < board.length; ++x) {
for (int y = 0; y < board[x].length; ++y) {
Cell cell = new Cell(x, y);
if (isEmpty(cell)) {
result.add(cell);
}
}
}
return result;
}
}
This is an immutable class that represents a board and lets us interrogate it to find out the current state. It also keeps track of a current score, which we will come to later.
2.2. A Computer Player and Placing Tiles
Now that we've got a game board, we want to be able to play with it. The first thing we want is the computer player because this is a purely random player and will be exactly as needed later on.
The computer player does nothing more than place a tile into a cell, so we need some way to achieve that on our board. We want to keep this as being immutable, so placing a tile will generate a brand new board in the new state.
First, we want a constructor that will take the actual board state, as opposed to our earlier one that just constructed a blank board:
private Board(int[][] board, int score) {
this.score = score;
this.board = new int[board.length][];
for (int x = 0; x < board.length; ++x) {
this.board[x] = Arrays.copyOf(board[x], board[x].length);
}
}
This is private so that it can only ever be used by other methods within the same class. This helps with our encapsulation of the board.
Next, we'll add a method to place a tile. This returns a brand new board that is identical to the current one except that it has the given number in the given cell:
public Board placeTile(Cell cell, int number) {
if (!isEmpty(cell)) {
throw new IllegalArgumentException("That cell is not empty");
}
Board result = new Board(this.board, this.score);
result.board[cell.getX()][cell.getY()] = number;
return result;
}
Finally, we'll write a new class representing a computer player. This will have a single method that will take the current board and return the new one:
public class Computer {
private final SecureRandom rng = new SecureRandom();
public Board makeMove(Board input) {
List<Cell> emptyCells = input.emptyCells();
double numberToPlace = rng.nextDouble();
int indexToPlace = rng.nextInt(emptyCells.size());
Cell cellToPlace = emptyCells.get(indexToPlace);
return input.placeTile(cellToPlace, numberToPlace >= 0.9 ? 4 : 2);
}
}
This gets the list of every empty cell from the board, picks a random one, and then puts a number in it. We'll randomly decide to put a “4” into the cell 10% of the time, and a “2” the other 90%.
2.2. A “Human” Player and Shifting Tiles
The next thing we need is a “human” player. This isn't going to be the end goal, but a purely random player that picks a random direction to shift the tiles every time it makes a move. This will then act as a place that we can build upon to make our optimal player.
Firstly, we need to define an enumeration of the possible moves that can be made:
public enum Move {
UP,
DOWN,
LEFT,
RIGHT
}
Next, we need to augment the Board class to support making moves by shifting tiles in one of these directions. To reduce the complexity here, we want to rotate the board such that we're always shifting tiles in the same direction.
This means that we need a means both to transpose and to reverse the board:
private static int[][] transpose(int[][] input) {
int[][] result = new int[input.length][];
for (int x = 0; x < input.length; ++x) {
result[x] = new int[input[0].length];
for (int y = 0; y < input[0].length; ++y) {
result[x][y] = input[y][x];
}
}
return result;
}
private static int[][] reverse(int[][] input) {
int[][] result = new int[input.length][];
for (int x = 0; x < input.length; ++x) {
result[x] = new int[input[0].length];
for (int y = 0; y < input[0].length; ++y) {
result[x][y] = input[x][input.length - y - 1];
}
}
return result;
}
Transposing the board will swap all rows and columns around, such that the top edge becomes the left edge. Reversing the board simply mirrors it such that the left edge becomes the right edge.
Next, we add a method to the Board to make a move in a given direction, and return a new Board in the new state.
We start by making a copy of the board state that we can then work with:
public Board move(Move move) {
int newScore = 0;
// Clone the board
int[][] tiles = new int[this.board.length][];
for (int x = 0; x < this.board.length; ++x) {
tiles[x] = Arrays.copyOf(this.board[x], this.board[x].length);
}
Next, we manipulate our copy so that we're always going to be shifting tiles up:
if (move == Move.LEFT || move == Move.RIGHT) {
tiles = transpose(tiles);
}
if (move == Move.DOWN || move == Move.RIGHT) {
tiles = reverse(tiles);
}
We need yet another array of tiles – this time the one that we will build the final result into – and a tracker for the new score gained for this move:
int[][] result = new int[tiles.length][]; int newScore = 0;
Now that we're ready to start shifting tiles, and we've manipulated things so that we're always working in the same direction, we can start.
We can shift each column independently of the others. We just need to iterate over the columns and repeat, starting with building yet another copy of the tiles we are shifting.
This time we build them into a LinkedList because we will want to be able to pop values off of it easily. We also only add the actual tiles that have numbers and skip over empty tiles.
This achieves our shifting but not yet the merging of tiles:
for (int x = 0; x < tiles.length; ++x) {
LinkedList<Integer> thisRow = new LinkedList<>();
for (int y = 0; y < tiles[0].length; ++y) {
if (tiles[x][y] > 0) {
thisRow.add(tiles[x][y]);
}
}
Next, we need to merge tiles. We need to do this separately from the above; otherwise, we risk merging the same tile multiple times.
This is achieved by building another LinkedList of the tiles from the above, but this time merging as we go:
LinkedList<Integer> newRow = new LinkedList<>();
while (thisRow.size() >= 2) {
int first = thisRow.pop();
int second = thisRow.peek();
if (second == first) {
int newNumber = first * 2;
newRow.add(newNumber);
newScore += newNumber;
thisRow.pop();
} else {
newRow.add(first);
}
}
newRow.addAll(thisRow);
Here we're also calculating the new score for this move. This is the sum of the tiles created as a result of merges.
We can now build this into the result array. Once we've run out of tiles from our list, the rest get populated with the value “0” to indicate that they are blank:
result[x] = new int[tiles[0].length];
for (int y = 0; y < tiles[0].length; ++y) {
if (newRow.isEmpty()) {
result[x][y] = 0;
} else {
result[x][y] = newRow.pop();
}
}
}
Once we've finished shifting tiles, we need to manipulate them again back to the correct rotation. This is the exact opposite that we did earlier:
if (move == Move.DOWN || move == Move.RIGHT) {
result = reverse(result);
}
if (move == Move.LEFT || move == Move.RIGHT) {
result = transpose(result);
}
And finally, we can build and return a new board with this new set of tiles and the newly calculated score:
return new Board(result, this.score + newScore); }
We're now in a position where we can write our random “human” player. This does nothing more than generate a random move and call the above method to play that move:
public class Human {
private SecureRandom rng = new SecureRandom();
public Board makeMove(Board input) {
Move move = Move.values()[rng.nextInt(4)];
return input.move(move);
}
}
2.3. Playing the Game
We have enough components to play the game, albeit not very successfully. However, soon we will be improving the way that the Human class plays, and this will allow us to see the differences easily.
First, we need a way to print out the game board.
For this example, we're just going to print to the console, so System.out.print is good enough. For a real game we would want to do better graphics:
private static void printBoard(Board board) {
StringBuilder topLines = new StringBuilder();
StringBuilder midLines = new StringBuilder();
for (int x = 0; x < board.getSize(); ++x) {
topLines.append("+--------");
midLines.append("| ");
}
topLines.append("+");
midLines.append("|");
for (int y = 0; y < board.getSize(); ++y) {
System.out.println(topLines);
System.out.println(midLines);
for (int x = 0; x < board.getSize(); ++x) {
Cell cell = new Cell(x, y);
System.out.print("|");
if (board.isEmpty(cell)) {
System.out.print(" ");
} else {
StringBuilder output = new StringBuilder(Integer.toString(board.getCell(cell)));
while (output.length() < 8) {
output.append(" ");
if (output.length() < 8) {
output.insert(0, " ");
}
}
System.out.print(output);
}
}
System.out.println("|");
System.out.println(midLines);
}
System.out.println(topLines);
System.out.println("Score: " + board.getScore());
}
We're nearly ready to go. We just need to set things up.
This means creating the board, the two players, and having the computer make two initial moves – that is, placing two random numbers on the board:
Board board = new Board(4);
Computer computer = new Computer();
Human human = new Human();
for (int i = 0; i < 2; ++i) {
board = computer.makeMove(board);
}
And now we have the actual game loop. This is going to be a repetition of the human and computer players taking turns, and stopping only when there are no empty cells left:
printBoard(board);
do {
System.out.println("Human move");
System.out.println("==========");
board = human.makeMove(board);
printBoard(board);
System.out.println("Computer move");
System.out.println("=============");
board = computer.makeMove(board);
printBoard(board);
} while (!board.emptyCells().isEmpty());
System.out.println("Final Score: " + board.getScore());
At this point, if we were to run the program, we would see a random game of 2048 being played.
3. Implementing the 2048 Player
Once we have a base from which to play the game, we can start implementing the “human” player and play a better game than just picking a random direction.
3.1. Simulating Moves
The algorithm we are implementing here is based on the Expectimax algorithm. As such, the core of the algorithm is to simulate every possible move, allocate a score to each one, and select the one that does best.
We'll be making heavy use of Java 8 Streams to help structure this code, for reasons we'll see later.
We'll start by re-writing the makeMove() method from inside our Human class:
public Board makeMove(Board input) {
return Arrays.stream(Move.values())
.map(input::move)
.max(Comparator.comparingInt(board -> generateScore(board, 0)))
.orElse(input);
}
For every possible direction we can move in, we generate the new board and then start the scoring algorithm – passing in this board and a depth of 0. We then select the move that has the best score.
Our generateScore() method then simulates every possible computer move – that is, placing either a “2” or a “4” into every empty cell – and then sees what could happen next:
private int generateScore(Board board, int depth) {
if (depth >= 3) {
return calculateFinalScore(board);
}
return board.emptyCells().stream()
.flatMap(cell -> Stream.of(new Pair<>(cell, 2), new Pair<>(cell, 4)))
.mapToInt(move -> {
Board newBoard = board.placeTile(move.getFirst(), move.getSecond());
int boardScore = calculateScore(newBoard, depth + 1);
return (int) (boardScore * (move.getSecond() == 2 ? 0.9 : 0.1));
})
.sum();
}
If we have reached our depth limit, then we'll immediately stop and calculate a final score for how good this board is; otherwise, we continue with our simulation.
Our calculateScore() method is then the continuation of our simulation, running the human move side of the equation.
This is very similar to the makeMove() method above, but we're returning the ongoing score instead of the actual board:
private int calculateScore(Board board, int depth) {
return Arrays.stream(Move.values())
.map(board::move)
.mapToInt(newBoard -> generateScore(newBoard, depth))
.max()
.orElse(0);
}
3.2. Scoring Final Boards
We're now in a situation where we can simulate moves back and forth by the human and computer players, stopping when we've simulated enough of them. We need to be able to generate a score for the final board in each simulation branch, so that we can see which branch is the one we want to pursue.
Our scoring is a combination of factors, each of which we are going to apply to every row and every column on the board. These all get summed together, and the total is returned.
As such, we need to generate a list of rows and columns to score against:
List<List<Integer>> rowsToScore = new ArrayList<>();
for (int i = 0; i < board.getSize(); ++i) {
List<Integer> row = new ArrayList<>();
List<Integer> col = new ArrayList<>();
for (int j = 0; j < board.getSize(); ++j) {
row.add(board.getCell(new Cell(i, j)));
col.add(board.getCell(new Cell(j, i)));
}
rowsToScore.add(row);
rowsToScore.add(col);
}
Then we take the list that we've built, score each of them, and sum the scores together. This is a placeholder that we're about to fill out:
return rowsToScore.stream()
.mapToInt(row -> {
int score = 0;
return score;
})
.sum();
Finally, we need actually to generate our scores. This goes inside the above lambda, and is several different factors that all contribute:
- A fixed score for every row
- The sum of every number in the row
- Every merge possible in the row
- Every empty cell in the row
- The monotonicity of the row. This represents the amount the row is organized in ascending numerical order.
Before we can calculate the scores, we need to build some extra data.
First, we want a list of the numbers with blank cells removed:
List<Integer> preMerged = row.stream() .filter(value -> value != 0) .collect(Collectors.toList());
We can then make some counts from this new list, giving the number of adjacent cells with the same number, with strictly ascending numbers and strictly descending numbers:
int numMerges = 0;
int monotonicityLeft = 0;
int monotonicityRight = 0;
for (int i = 0; i < preMerged.size() - 1; ++i) {
Integer first = preMerged.get(i);
Integer second = preMerged.get(i + 1);
if (first.equals(second)) {
++numMerges;
} else if (first > second) {
monotonicityLeft += first - second;
} else {
monotonicityRight += second - first;
}
}
Now we can calculate our score for this row:
int score = 1000; score += 250 * row.stream().filter(value -> value == 0).count(); score += 750 * numMerges; score -= 10 * row.stream().mapToInt(value -> value).sum(); score -= 50 * Math.min(monotonicityLeft, monotonicityRight); return score;
The numbers selected here are relatively arbitrary. Different numbers will have an impact on how well the game plays, prioritizing different factors in how we play.
4. Improvements to the Algorithm
What we have so far works, and we can see that it plays a good game, but it's slow. It takes around 1 minute per human move. We can do better than this.
4.1. Parallel Processing
The obvious thing that we can do is to do work in parallel. This is a huge benefit of working with Java Streams – we can make this work in parallel by just adding a single statement to each stream.
This change alone gets us down to around 20 seconds per move.
4.2. Pruning Unplayable Branches
The next thing we can do is to prune out any branches that are unplayable. That is, any time that a human move results in an unchanged board. These are almost certainly branches that are going to result in worse outcomes – they are effectively giving the computer a free move – but they cost us processing time to pursue them.
To do this, we need to implement an equals method on our Board so that we can compare them:
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
Board board1 = (Board) o;
return Arrays.deepEquals(board, board1.board);
}
We can then add some filters to our stream pipelines to stop processing anything that hasn't changed.
return Arrays.stream(Move.values())
.parallel()
.map(board::move)
.filter(moved -> !moved.equals(board))
........
This has minimal impact on the early parts of playing – when there are very few filled cells, there are very few moves that can be trimmed. However, later on, this starts to make a much bigger impact, reducing move times down to only a few seconds.
5. Summary
Here we built a framework for playing the game 2048. Then, we wrote a solver into this so that we can play a better game. All of the examples seen here can be found over on GitHub.
Why not try varying the rules to see how they impact the gameplay.