Directory listing with the NIO.2 API

The NIO.2 (“New Input/Output 2”) is a new API introduced in JDK 7 to operate more efficiently and uniformly on file systems. This new API covers many file and directory-specific aspects, such as handling attributes and symbolic links. This article discusses one of the most straightforward aspects of NIO.2, listing directory entries using the new DirectoryStream interface.

• Directory listing before JDK 7
• Directory listing since JDK 7
• Usage examples
• Is globbing case-sensitive?
• Some words about SecureDirectoryStream

Directory listing before JDK 7

Before JDK 7, the only way to list directory entries was through the list and listFiles methods of the well-known java.io.File class.

public String[] list()
public String[] list(FilenameFilter filter)
public File[] listFiles()
public File[] listFiles(FileFilter filter)
public File[] listFiles(FilenameFilter filter)

These five methods aren’t evil by themselves. However, they have some drawbacks. First of all, they are not particularly efficient when a directory contains thousands and thousands of entries. Indeed, these methods must scan the entire directory before returning the entries array. Another drawback is that any globbing logic (filtering entries matching a pattern like A*.jpeg) is not easy to implement and requires at least a custom implementation of either FileFilter or FilenameFilter interface.

Directory listing since JDK 7

The NIO.2 API provides a new interface called DirectoryStream. It represents a new way to enumerate directory entries efficiently and practically. DirectoryStream has some notable features:

1) It is a generic interface. However, the concrete parameterization used in the standard framework is always <Path> (java.nio.file.Path), not something else.

2) It implements the Iterable<T> interface. A DirectoryStream object can be iterated using the enhanced-for (“for-each”) loop available since Java 5. Note that DirectoryStream is not a general-purpose implementation of Iterable. You can perform only one iteration on a DirectoryStream object. Any further implicit or explicit invocation of the iterator() method throws IllegalStateException.

3) It implements both the Closeable and the AutoCloseable interface. A DirectoryStream must be closed to release all the resources associated with the stream. The best way to ensure proper closure of the stream is with the try-with-resource statement, which was also introduced in Java 7.

Obtaining a DirectoryStream

Since DirectoryStream is an interface, you need to obtain a concrete implementation to do something useful. The java.nio.file.Files class contains 3 static “factory” methods to open a directory stream:

public static DirectoryStream<Path> newDirectoryStream(Path dir)
                                                throws IOException

public static DirectoryStream<Path> newDirectoryStream(Path dir, DirectoryStream.Filter<? super Path> filter)
                                                throws IOException

public static DirectoryStream<Path> newDirectoryStream(Path dir, String glob)
                                                throws IOException

The first version is the simplest since it iterates over all directory entries without filtering. The second version receives a DirectoryStream.Filter. This functional interface works as a specialized “predicate” to decide whether an entry should be accepted.

The third version with the String glob parameter is the most interesting. A glob pattern can contain special characters like “?” and “*” to match more directory entries. Examples of glob patterns are A*.jpeg or config.* or report-0??.doc. Note that all major Operating Systems (Windows/Unix/Mac) support some form of globbing.

The glob syntax of the NIO.2 API is specified and documented by the getPathMatcher method in the FileSystem class. If you look at the linked Javadoc, you can notice that the glob syntax supports some unusual forms. In particular, a form like *.{java,class} is not natively available on all Operating Systems. But fortunately, the NIO.2 API manages it uniformly on all systems, even Windows systems (where this form is not natively available).

Usage examples

The following are just two simple examples of the use of DirectoryStream. The first example uses a custom filter, while the second uses a glob pattern.

import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

public class NioDirList1 {
    public static void main(String[] args) {
        Path dirPath = Paths.get(".");    // current directory as example

        try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(dirPath,
                path -> isWritableRegularFile(path))) {
            for (Path entry : dirStream) {
                System.out.println("Found: " + entry);
            }
        } catch (IOException e) {
            System.err.println(e);
        }
    }

    private static boolean isWritableRegularFile(Path path) {
        return Files.isRegularFile(path) && Files.isWritable(path);
    }
}

import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

public class NioDirList2 {
    public static void main(String[] args) {
        Path dirPath = Paths.get(".");    // current directory as example

        try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(dirPath, "*.java")) {
            for (Path entry : dirStream) {
                System.out.println("Found: " + entry);
            }
        } catch (IOException e) {
            System.err.println(e);
        }
    }
}

Is globbing case-sensitive?

This is a good question. Unfortunately, the Javadoc documentation of getPathMatcher is a bit vague about this aspect of globbing:

[…] For both the glob and regex syntaxes, the matching details, such as whether the matching is case sensitive, are implementation-dependent and therefore not specified.

So I decided to do a test on two machines I own, one with Microsoft Windows (Windows 10) and the other with a Linux distribution (Xubuntu 22.04). On both systems, I have created the following 3 (empty) files in a test directory:

• one.txt
• two.TXT
• three.TxT

And then I tried the following code in that directory:

import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.SecureDirectoryStream;

public class GlobCaseTest {
    public static void main(String[] args) throws IOException {
        try (DirectoryStream<Path> dirStream = Files.newDirectoryStream(Paths.get("."), "*.txt")) {
            dirStream.forEach(System.out::println);
        }
    }
}

The following is the output on Windows 10:

.\one.txt
.\three.TxT
.\two.TXT

The following is the output on Xubuntu 22.04:

./one.txt

So the bottom line about globbing is:

• It is case-insensitive on Windows machines.
• It is case-sensitive on Linux machines.

Some words about SecureDirectoryStream

If you look at the Javadoc documentation of the java.nio.file package, you should notice another interface called SecureDirectoryStream, a sub-interface of DirectoryStream.

This SecureDirectoryStream interface performs the operations safely to avoid race conditions. For example, imagine that you want to delete all the files in a directory and that, while iterating on the DirectoryStream, “someone” moves that directory elsewhere. What happens? If you use the deleteFile method of SecureDirectoryStream, you can ensure that the delete operation works correctly, even in this particular scenario.

Note that you cannot explicitly request a SecureDirectoryStream. If the underlying Operating System supports “secure” operations, then the Files.newDirectoryStream methods return an object that implements SecureDirectoryStream, not just only DirectoryStream. So you have to do something like this:

try (DirectoryStream<Path> dirStream = Files.newDirectoryStream( /*.....*/ )) {
    if (dirStream instanceof SecureDirectoryStream) {
        SecureDirectoryStream<Path> secDirStream = (SecureDirectoryStream<Path>) dirStream;

        // Ok, we can perform “secure” operations ...
    }
}

From what I have tested so far, I found that SecureDirectoryStream is supported on Xubuntu but not on Windows 10.