org.apache.flink.core.fs

Interface EntropyInjectingFileSystem

All Known Implementing Classes:

FlinkS3FileSystem, FlinkS3PrestoFileSystem

@PublicEvolving
public interface EntropyInjectingFileSystem

An interface to be implemented by a FileSystem that is aware of entropy injection.

Entropy injection is a technique to spread files/objects across more parallel shards of a distributed storage (typically object store) by adding random characters to the beginning of the path/key and hence spearing the keys across a wider domain of prefixes.

Entropy injection typically works by having a recognized marker string in paths and replacing that marker with random characters.

This interface is used in conjunction with the EntropyInjector (as a poor man's way to build a mix-in in Java).

Method Summary

Modifier and Type	Method and Description
String	generateEntropy() Creates a string with random entropy to be injected into a path.
String	getEntropyInjectionKey() Gets the marker string that represents the substring of a path to be replaced by the entropy characters.

Method Detail

getEntropyInjectionKey

@Nullable
String getEntropyInjectionKey()

Gets the marker string that represents the substring of a path to be replaced by the entropy characters.

You can disable entropy injection if you return null here.

generateEntropy

String generateEntropy()

Creates a string with random entropy to be injected into a path.