Package org.apache.flink.state.forst

Class ForStStateDataTransfer

java.lang.Object

org.apache.flink.state.forst.ForStStateDataTransfer

All Implemented Interfaces:

Closeable, AutoCloseable

public class ForStStateDataTransfer
extends Object
implements Closeable

Data transfer util class for ForStKeyedStateBackend.

Field Summary

Fields

Modifier and Type	Field	Description
static int	DEFAULT_THREAD_NUM
protected ExecutorService	executorService

Constructor Summary

Constructors

Constructor	Description
ForStStateDataTransfer(int threadNum)
ForStStateDataTransfer(int threadNum, FileSystem forStFs)

Method Summary

Modifier and Type	Method	Description
void	close()
void	transferAllStateDataToDirectory(Collection<StateHandleTransferSpec> transferSpecs, CloseableRegistry closeableRegistry)	Transfer all state data to the target directory, as specified in the transfer requests.
List<IncrementalKeyedStateHandle.HandleAndLocalPath>	transferFilesToCheckpointFs(List<Path> files, CheckpointStreamFactory checkpointStreamFactory, CheckpointedStateScope stateScope, CloseableRegistry closeableRegistry, CloseableRegistry tmpResourcesRegistry)	Transfer a batch of files to checkpoint filesystem.
IncrementalKeyedStateHandle.HandleAndLocalPath	transferFileToCheckpointFs(Path file, long transferBytes, CheckpointStreamFactory checkpointStreamFactory, CheckpointedStateScope stateScope, CloseableRegistry snapshotCloseableRegistry, CloseableRegistry tmpResourcesRegistry)	Transfer a single file to checkpoint filesystem.
IncrementalKeyedStateHandle.HandleAndLocalPath	writeFileToCheckpointFs(String filename, String fileContent, CheckpointStreamFactory checkpointStreamFactory, CheckpointedStateScope stateScope, CloseableRegistry closeableRegistry, CloseableRegistry tmpResourcesRegistry)	Write a file to checkpoint filesystem.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_THREAD_NUM

public static final int DEFAULT_THREAD_NUM

See Also:

Constant Field Values

executorService

protected final ExecutorService executorService

Constructor Detail

ForStStateDataTransfer

public ForStStateDataTransfer(int threadNum)

ForStStateDataTransfer

public ForStStateDataTransfer(int threadNum,
                              FileSystem forStFs)

Method Detail

transferFileToCheckpointFs

public IncrementalKeyedStateHandle.HandleAndLocalPath transferFileToCheckpointFs(Path file,
                                                                                 long transferBytes,
                                                                                 CheckpointStreamFactory checkpointStreamFactory,
                                                                                 CheckpointedStateScope stateScope,
                                                                                 CloseableRegistry snapshotCloseableRegistry,
                                                                                 CloseableRegistry tmpResourcesRegistry)
                                                                          throws Exception

Transfer a single file to checkpoint filesystem.

Parameters:

transferBytes - Bytes will be transfer from the head of the file. If < 0, the whole file will be transferred.

Throws:

Exception

transferFilesToCheckpointFs

public List<IncrementalKeyedStateHandle.HandleAndLocalPath> transferFilesToCheckpointFs(List<Path> files,
                                                                                        CheckpointStreamFactory checkpointStreamFactory,
                                                                                        CheckpointedStateScope stateScope,
                                                                                        CloseableRegistry closeableRegistry,
                                                                                        CloseableRegistry tmpResourcesRegistry)
                                                                                 throws Exception

Transfer a batch of files to checkpoint filesystem.

Throws:

Exception

writeFileToCheckpointFs

public IncrementalKeyedStateHandle.HandleAndLocalPath writeFileToCheckpointFs(String filename,
                                                                              String fileContent,
                                                                              CheckpointStreamFactory checkpointStreamFactory,
                                                                              CheckpointedStateScope stateScope,
                                                                              CloseableRegistry closeableRegistry,
                                                                              CloseableRegistry tmpResourcesRegistry)
                                                                       throws IOException

Write a file to checkpoint filesystem.

Throws:

IOException

transferAllStateDataToDirectory

public void transferAllStateDataToDirectory(Collection<StateHandleTransferSpec> transferSpecs,
                                            CloseableRegistry closeableRegistry)
                                     throws Exception

Transfer all state data to the target directory, as specified in the transfer requests.

Parameters:

transferSpecs - the list of transfers.

Throws:

Exception - If anything about the transfer goes wrong.

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable