Modifier and Type | Method and Description |
---|---|
static DataType |
appendRowFields(DataType dataType,
List<DataTypes.Field> fields)
Appends the given list of fields to an existing row data type.
|
static DataType |
createProctimeDataType()
Returns a PROCTIME data type.
|
static ResolvedSchema |
expandCompositeTypeToSchema(DataType dataType)
Expands a composite
DataType to a corresponding ResolvedSchema . |
static List<DataType> |
flattenToDataTypes(DataType dataType)
Returns the data types of the flat representation in the first level of the given data type.
|
static List<String> |
flattenToNames(DataType dataType)
Returns the names of the flat representation in the first level of the given data type.
|
static List<String> |
flattenToNames(DataType dataType,
List<String> existingNames)
Returns the names of the flat representation in the first level of the given data type.
|
static Optional<DataType> |
getField(DataType compositeType,
int index)
Retrieves a nested field from a composite type at given position.
|
static Optional<DataType> |
getField(DataType compositeType,
String name)
Retrieves a nested field from a composite type with given name.
|
static boolean |
isInternal(DataType dataType)
Checks whether a given data type is an internal data structure.
|
static DataType |
projectRow(DataType dataType,
int[] indices)
Projects a (possibly nested) row data type by returning a new data type that only includes
fields of the given indices.
|
static DataType |
projectRow(DataType dataType,
int[][] indexPaths)
Projects a (possibly nested) row data type by returning a new data type that only includes
fields of the given index paths.
|
static DataType |
removeTimeAttribute(DataType dataType)
Removes time attributes from the
DataType . |
static DataType |
replaceLogicalType(DataType dataType,
LogicalType replacement)
Replaces the
LogicalType of a DataType , i.e., it keeps the bridging class. |
static DataType |
stripRowPrefix(DataType dataType,
String prefix)
Removes a string prefix from the fields of the given row data type.
|
static DataType |
toInternalDataType(DataType dataType)
|
static DataType |
toInternalDataType(LogicalType logicalType)
Creates a
DataType from the given LogicalType with internal data structures. |
static DataType |
transform(DataTypeFactory factory,
DataType typeToTransform,
TypeTransformation... transformations)
Transforms the given data type to a different data type using the given transformations.
|
static DataType |
transform(DataType typeToTransform,
TypeTransformation... transformations)
Transforms the given data type to a different data type using the given transformations.
|
static void |
validateInputDataType(DataType dataType)
The
DataType class can only partially verify the conversion class. |
static void |
validateOutputDataType(DataType dataType)
The
DataType class can only partially verify the conversion class. |
public static DataType projectRow(DataType dataType, int[][] indexPaths)
Note: Index paths allow for arbitrary deep nesting. For example, [[0, 2, 1], ...]
specifies to include the 2nd field of the 3rd field of the 1st field in the top-level row.
Sometimes, it may get name conflicts when extract fields from the row field. Considering the
the path is unique to extract fields, it makes sense to use the path to the fields with
delimiter `_` as the new name of the field. For example, the new name of the field `b` in the
row `a` is `a_b` rather than `b`. But it may still gets name conflicts in some situation,
such as the field `a_b` in the top level schema. In such situation, it will use the postfix
in the format '_$%d' to resolve the name conflicts.
public static DataType projectRow(DataType dataType, int[] indices)
Note: This method only projects (possibly nested) fields in the top-level row.
public static DataType stripRowPrefix(DataType dataType, String prefix)
public static DataType appendRowFields(DataType dataType, List<DataTypes.Field> fields)
public static DataType toInternalDataType(LogicalType logicalType)
DataType
from the given LogicalType
with internal data structures.public static boolean isInternal(DataType dataType)
public static DataType replaceLogicalType(DataType dataType, LogicalType replacement)
LogicalType
of a DataType
, i.e., it keeps the bridging class.public static DataType removeTimeAttribute(DataType dataType)
DataType
. As everywhere else in the code base, this
method does not support nested time attributes for now.public static DataType transform(DataType typeToTransform, TypeTransformation... transformations)
public static DataType transform(@Nullable DataTypeFactory factory, DataType typeToTransform, TypeTransformation... transformations)
The transformations will be called in the given order. In case of constructed or composite types, a transformation will be applied transitively to children first.
Both the DataType.getLogicalType()
and DataType.getConversionClass()
can
be transformed.
factory
- DataTypeFactory
if availabletypeToTransform
- data type to be transformed.transformations
- the transformations to transform data type to another type.public static ResolvedSchema expandCompositeTypeToSchema(DataType dataType)
DataType
to a corresponding ResolvedSchema
. Useful for
flattening a column or mapping a physical to logical type of a table source
Throws an exception for a non composite type. You can use LogicalTypeChecks.isCompositeType(LogicalType)
to check that.
It does not expand an atomic type on purpose, because that operation depends on the
context. E.g. in case of a FLATTEN
function such operation is not allowed, whereas
when mapping a physical type to logical the field name should be derived from the logical
schema.
dataType
- Data type to expand. Must be a composite type.public static Optional<DataType> getField(DataType compositeType, int index)
Throws an exception for a non composite type. You can use LogicalTypeChecks.isCompositeType(LogicalType)
to check that.
compositeType
- Data type to expand. Must be a composite type.index
- Index of the field to retrieve.public static Optional<DataType> getField(DataType compositeType, String name)
Throws an exception for a non composite type. You can use LogicalTypeChecks.isCompositeType(LogicalType)
to check that.
compositeType
- Data type to expand. Must be a composite type.name
- Name of the field to retrieve.public static List<DataType> flattenToDataTypes(DataType dataType)
public static List<String> flattenToNames(DataType dataType)
public static List<String> flattenToNames(DataType dataType, List<String> existingNames)
public static void validateInputDataType(DataType dataType)
DataType
class can only partially verify the conversion class. This method can
perform the final check when we know if the data type should be used for input.public static void validateOutputDataType(DataType dataType)
DataType
class can only partially verify the conversion class. This method can
perform the final check when we know if the data type should be used for output.public static DataType createProctimeDataType()
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.