org.apache.flink.runtime.blob

Interface BlobWriter

All Known Implementing Classes:

BlobServer, VoidBlobWriter

public interface BlobWriter

BlobWriter is used to upload data to the BLOB store.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Method Summary

Modifier and Type	Method and Description
boolean	deletePermanent(JobID jobId, PermanentBlobKey permanentBlobKey) Delete the uploaded data with the given JobID and PermanentBlobKey.
int	getMinOffloadingSize() Returns the min size before data will be offloaded to the BLOB store.
static <T> Either<SerializedValue<T>,PermanentBlobKey>	offloadWithException(SerializedValue<T> serializedValue, JobID jobId, BlobWriter blobWriter)
PermanentBlobKey	putPermanent(JobID jobId, byte[] value) Uploads the data of the given byte array for the given job to the BLOB server and makes it a permanent BLOB.
PermanentBlobKey	putPermanent(JobID jobId, InputStream inputStream) Uploads the data from the given input stream for the given job to the BLOB server and makes it a permanent BLOB.
static <T> Either<SerializedValue<T>,PermanentBlobKey>	serializeAndTryOffload(T value, JobID jobId, BlobWriter blobWriter) Serializes the given value and offloads it to the BlobServer if its size exceeds the minimum offloading size of the BlobServer.
static <T> Either<SerializedValue<T>,PermanentBlobKey>	tryOffload(SerializedValue<T> serializedValue, JobID jobId, BlobWriter blobWriter)

Field Detail

LOG

static final org.slf4j.Logger LOG

Method Detail

putPermanent

PermanentBlobKey putPermanent(JobID jobId,
                              byte[] value)
                       throws IOException

Uploads the data of the given byte array for the given job to the BLOB server and makes it a permanent BLOB.

Parameters:

jobId - the ID of the job the BLOB belongs to

value - the buffer to upload

Returns:

the computed BLOB key identifying the BLOB on the server

Throws:

IOException - thrown if an I/O error occurs while writing it to a local file, or uploading it to the HA store

putPermanent

PermanentBlobKey putPermanent(JobID jobId,
                              InputStream inputStream)
                       throws IOException

Uploads the data from the given input stream for the given job to the BLOB server and makes it a permanent BLOB.

Parameters:

jobId - ID of the job this blob belongs to

inputStream - the input stream to read the data from

Returns:

the computed BLOB key identifying the BLOB on the server

Throws:

IOException - thrown if an I/O error occurs while reading the data from the input stream, writing it to a local file, or uploading it to the HA store

deletePermanent

boolean deletePermanent(JobID jobId,
                        PermanentBlobKey permanentBlobKey)

Delete the uploaded data with the given JobID and PermanentBlobKey.

Parameters:

jobId - ID of the job this blob belongs to

permanentBlobKey - the key of this blob

getMinOffloadingSize

int getMinOffloadingSize()

Returns the min size before data will be offloaded to the BLOB store.

Returns:

minimum offloading size

serializeAndTryOffload

static <T> Either<SerializedValue<T>,PermanentBlobKey> serializeAndTryOffload(T value,
                                                                              JobID jobId,
                                                                              BlobWriter blobWriter)
                                                                       throws IOException

Serializes the given value and offloads it to the BlobServer if its size exceeds the minimum offloading size of the BlobServer.

Type Parameters:

T - type of the value to serialize

Parameters:

value - to serialize

jobId - to which the value belongs.

blobWriter - to use to offload the serialized value

Returns:

Either the serialized value or the stored blob key

Throws:

IOException - if the data cannot be serialized

tryOffload

static <T> Either<SerializedValue<T>,PermanentBlobKey> tryOffload(SerializedValue<T> serializedValue,
                                                                  JobID jobId,
                                                                  BlobWriter blobWriter)

offloadWithException

static <T> Either<SerializedValue<T>,PermanentBlobKey> offloadWithException(SerializedValue<T> serializedValue,
                                                                            JobID jobId,
                                                                            BlobWriter blobWriter)