org.apache.flink.core.fs

Class FileSystemSafetyNet

java.lang.Object

org.apache.flink.core.fs.FileSystemSafetyNet

@Internal
public class FileSystemSafetyNet
extends Object

The FileSystemSafetyNet can be used to guard a thread against FileSystem stream resource leaks. When activated for a thread, it tracks all streams that are opened by FileSystems that the thread obtains. The safety net has a global cleanup hook that will close all streams that were not properly closed.

The main thread of each Flink task, as well as the checkpointing thread are automatically guarded by this safety net.

Important: This safety net works only for streams created by Flink's FileSystem abstraction, i.e., for FileSystem instances obtained via FileSystem.get(URI) or through Path.getFileSystem().

Important: When a guarded thread obtains a FileSystem or a stream and passes them to another thread, the safety net will close those resources once the former thread finishes.

The safety net can be used as follows:

 class GuardedThread extends Thread {

     public void run() {
         FileSystemSafetyNet.initializeSafetyNetForThread();
         try {
             // do some heavy stuff where you are unsure whether it closes all streams
             // like some untrusted user code or library code
         }
         finally {
             FileSystemSafetyNet.closeSafetyNetAndGuardedResourcesForThread();
         }
     }
 }
 

Constructor Summary

Constructors

Constructor and Description
FileSystemSafetyNet()

Method Summary

Modifier and Type	Method and Description
static void	closeSafetyNetAndGuardedResourcesForThread() Closes the safety net for a thread.
static void	initializeSafetyNetForThread() Activates the safety net for a thread.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileSystemSafetyNet

public FileSystemSafetyNet()

Method Detail

initializeSafetyNetForThread

@Internal
public static void initializeSafetyNetForThread()

Activates the safety net for a thread. FileSystem instances obtained by the thread that called this method will be guarded, meaning that their created streams are tracked and can be closed via the safety net closing hook.

This method should be called at the beginning of a thread that should be guarded.

Throws:

IllegalStateException - Thrown, if a safety net was already registered for the thread.

closeSafetyNetAndGuardedResourcesForThread

@Internal
public static void closeSafetyNetAndGuardedResourcesForThread()

Closes the safety net for a thread. This closes all remaining unclosed streams that were opened by safety-net-guarded file systems. After this method was called, no streams can be opened any more from any FileSystem instance that was obtained while the thread was guarded by the safety net.

This method should be called at the very end of a guarded thread.