Get Number of Files in a Directory and Its Subdirectories in Java

Image may be NSFW.
Clik here to view.

1. Overview

Counting the number of files in a directory and its subdirectories is a common task in programming, whether you’re building a backup utility, monitoring disk usage, or synchronizing files across systems. For Java developers, this seemingly simple problem offers a chance to explore both traditional and modern approaches to file handling.

In this tutorial, we’ll dive into two primary methods: the recursive java.io.File approach, familiar to many, and the more efficient NIO-based solution introduced in Java 7.

2. Setup

The task of counting files in a directory and its subdirectories requires traversing a potentially complex tree structure, where each subdirectory might contain more files or additional subdirectories. This recursive nature poses technical challenges: how do we ensure accuracy across all levels, avoid infinite loops from symbolic links, and manage performance for large datasets?

For Java developers, the solution lies in choosing the right tools and designing a clear and flexible method.

Let’s define our method signature for the upcoming implementations:

public long numberOfFilesIn(String path) {
    // TODO: Implementation 
}

The method accepts a String path input parameter as the starting directory and returns the number of files found as long. The long type ensures we can handle counts beyond the 32-bit int limit, which is critical for enterprise-scale directories.

3. Using java.io.File

Our first implementations, which are to read files in Java and count them, will use the java.io.File library.

3.1. With Java 8+

Let’s start with implementations that use Java 8 and above, allowing us to write in a functional manner with Streams.

Our task to go through a directory, search for files, and continue on to a deeper nested directory is repetitive. One implementation style makes obvious sense here: recursion. The possibility to call oneself helps us to keep the complexity down, compared to an iterative approach:

File currentFile = new File(path);
File[] filesOrNull = currentFile.listFiles();
// Is this a file already?
long currentFileNumber = currentFile.isFile() ? 1 : 0;
if (filesOrNull == null) { // no sub directories found
    return currentFileNumber; // stop condition #1
}
return currentFileNumber + Arrays.stream(filesOrNull)
  .mapToLong(FindFolder::filesInside) // <-- recursion call here
  .sum();

Our implementation has basically three parts:

1. Check if the current path resolves to a file or directory
2. If the current directory can’t list any files, we return immediately
3. Otherwise we count the files in the subfolders recursively and return the sum, including the current file count

private static long filesInside(File it) {
    if (it.isFile()) {
        return 1; // stop condition #2
    } else if (it.isDirectory()) {
        return numberOfFilesIn(it.getAbsolutePath()); // <-- recursion to caller
    } else {
        return 0; // stop condition #3
    }
}

Be aware, that we include three stop conditions to the recursive implementation. Not paying enough attention to break out of the recursion cycle may lead to out-of-memory exceptions. 

3.2. Before Java 8

If we are working with Java below the version 8, we could simply refactor our method from the recursive stream implementation above.

This is how our second approach, the rewritten part without streams would look like:

for (File file : filesOrNull) {
    if (file.isDirectory()) {
        currentFileNumber += numberOfFilesIn(file.getAbsolutePath());
    } else if (file.isFile()) {
        currentFileNumber += 1; // add this file to count
    }
}
return currentFileNumber;

It looks very similar to the mapper function we used before, only this time we modify currentFileNumber variable by adding intermediate results. If we disregard the mutability issue, this solution glances with simplicity, little code and good readability.

4. Using NIO

Working with the file system in Java has another good alternative library, shipped since Java 7: NIO.

4.1. Files.find

Our third approach to the task – counting files in directory and sub directories – will make use of the Files.find functionality:

try (Stream<Path> stream = Files.find(
  Paths.get(path), 
  Integer.MAX_VALUE,
  (__, attr) -> attr.isRegularFile())) {
    return stream.count();
} catch (IOException e) {
    // or log here
    throw new RuntimeException(e);
}

We use try with resources to open a Path Stream inside our initial path directory.

4.2. Walkthrough

A second NIO possibility is to “walkthrough” the file system in an iterative manner.

This is how an implementation with Files.walk would look like:

Path dir = Path.of(path);
try (Stream<Path> stream = Files.walk(dir)) {
    return stream.parallel()
      .map(getFileOrEmpty())
      .flatMap(Optional::stream)
      .filter(it -> !it.isDirectory())
      .count();
} catch (IOException e) {
    throw new RuntimeException(e);
}

Just like before, we get a Stream of Path inside our initial path directory. Since the order in which we search through the directories does not matter and the amount of paths to cover can be great, we convert it to a parallel Stream. Then, we filter out Paths, which can’t be associated with the default provider and wrap them in Java Optional. At last, the count of all present elements, which are not a directory (i.e. file) is returned.

private static Function<Path, Optional<File>> getFileOrEmpty() {
    return it -> {
        try {
            return Optional.of(it.toFile());
        } catch (UnsupportedOperationException e) {
            // You may print or log the exception here;
            return Optional.empty();
        }
    };
}

The extracted method getFileOrEmpty returns a mapper function, to safely wrap a valid File in an Optional. It is useful in two ways: keeping the caller method small and handling the UnsupportedOperationException, which should not be left unhandled inside a Stream.

5. Overall Considerations

Several factors warrant attention when implementing file-counting solutions. Performance is critical – recursive java.io.File methods may struggle with deep directories due to stack overflow risks, while NIO’s stream-based approaches scale better for large datasets. For extensive structures, consider parallel streams with Files.walk, though this requires careful handling of concurrent file modifications.

Security also matters: if user input drives the path, validate it to prevent directory traversal attacks.

Additionally, ensure compatibility with international file names by leveraging NIO’s Unicode support, avoiding issues with non-ASCII characters.

Balancing efficiency, safety, and robustness ensures these methods meet real-world demands effectively.

6. Validation

Now that we’ve seen all our different implementations let’s design a test to run against.

Since the validation must go through real directories, let’s create some files and folders to search through:

filesToBeFound
|-- file1.txt
|-- subEmptyFolder
|-- subFolder1
    |-- file2.txt
    |-- file3.txt
|-- subFolder2 
    |-- file4.txt
    |-- subSubFolder
        |-- subSubSubFolder
            |-- file5.txt

The only remaining part is the test itself:

private final String resourcePath = this.getClass().getResource("/filesToBeFound").getPath();
@Test
void shouldReturnNumberOfAllFilesInsidePath() {
    assertThat(FindFolder.numberOfFilesIn(resourcePath)).isEqualTo(5);
}

Each of our implementations should find the same five files in the setup directory.

7. Conclusion

In this tutorial, we’ve explored two powerful approaches to counting files in a directory and its subdirectories in Java. The java.io.File method, with its recursive simplicity, suits smaller, shallow directory structures and offers a clear entry point for developers. However, its reliance on recursion can falter with deep hierarchies, risking stack overflow errors. In contrast, the NIO-based solutions – using Files.find and Files.walk – provide efficiency and scalability, leveraging streams to handle large datasets. The parallel Files.walk option further optimizes performance for extensive directories, though it requires careful exception handling.

For most modern applications, the NIO approach stands out as the better choice due to its robustness and performance. Choose java.io.File for quick, simple tasks, but opt for NIO when scalability matters.

As always, the entire code used in this article can be found over on GitHub.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view. Image may be NSFW.
Clik here to view. Image may be NSFW.
Clik here to view. Image may be NSFW.
Clik here to view. Image may be NSFW.
Clik here to view. Image may be NSFW.
Clik here to view. Image may be NSFW.
Clik here to view.