InType
- The accepted type of input expressions, it is Expression
for Scala and
Object
for Java. Generally the expression DSL works on expressions, the reason why
Java accepts Object is to remove cumbersome call to lit()
for literals. Scala
alleviates this problem via implicit conversions.OutType
- The produced type of the DSL. It is ApiExpression
for Java and Expression
for Scala. In Scala the infix operations are included via implicit conversions.
In Java we introduced a wrapper that enables the operations without pulling them through the
whole stack.@PublicEvolving public abstract class BaseExpressions<InType,OutType> extends Object
Expression
AST for expression operations.Constructor and Description |
---|
BaseExpressions() |
Modifier and Type | Method and Description |
---|---|
OutType |
abs()
Calculates the absolute value of given value.
|
OutType |
acos()
Calculates the arc cosine of a given number.
|
OutType |
and(InType other)
Boolean AND in three-valued logic.
|
OutType |
arrayAgg()
Returns array aggregate of a given expression.
|
OutType |
arrayAppend(InType element)
Appends an element to the end of the array and returns the result.
|
OutType |
arrayConcat(InType... arrays)
Returns an array that is the result of concatenating at least one array.
|
OutType |
arrayContains(InType needle)
Returns whether the given element exists in an array.
|
OutType |
arrayDistinct()
Returns an array with unique elements.
|
OutType |
arrayExcept(InType array)
Returns an ARRAY that contains the elements from array1 that are not in array2.
|
OutType |
arrayIntersect(InType array)
Returns an ARRAY that contains the elements from array1 that are also in array2, without
duplicates.
|
OutType |
arrayMax()
Returns the maximum value from the array.
|
OutType |
arrayMin()
Returns the minimum value from the array.
|
OutType |
arrayPosition(InType needle)
Returns the position of the first occurrence of element in the given array as int.
|
OutType |
arrayPrepend(InType element)
Appends an element to the beginning of the array and returns the result.
|
OutType |
arrayRemove(InType needle)
Removes all elements that equal to element from array.
|
OutType |
arrayReverse()
Returns an array in reverse order.
|
OutType |
arraySlice(InType startOffset) |
OutType |
arraySlice(InType startOffset,
InType endOffset)
Returns a subarray of the input array between 'start_offset' and 'end_offset' inclusive.
|
OutType |
arrayUnion(InType array)
Returns an array of the elements in the union of array1 and array2, without duplicates.
|
OutType |
as(String name,
String... extraNames)
Specifies a name for an expression i.e. a field.
|
OutType |
asc()
Specifies ascending order of an expression i.e. a field for orderBy unresolvedCall.
|
OutType |
ascii()
Returns the numeric value of the first character of the input string.
|
OutType |
asin()
Calculates the arc sine of a given number.
|
OutType |
at(InType index)
Accesses the element of an array or map based on a key or an index (starting at 1).
|
OutType |
atan()
Calculates the arc tangent of a given number.
|
OutType |
avg()
Returns the average (arithmetic mean) of the numeric field across all input values.
|
OutType |
between(InType lowerBound,
InType upperBound)
Returns true if the given expression is between lowerBound and upperBound (both inclusive).
|
OutType |
bin()
Returns a string representation of an integer numeric value in binary format.
|
OutType |
cardinality()
Returns the number of elements of an array or number of entries of a map.
|
OutType |
cast(DataType toType)
Returns a new value being cast to
toType . |
OutType |
cast(TypeInformation<?> toType)
Deprecated.
This method will be removed in future versions as it uses the old type system. It
is recommended to use
cast(DataType) instead which uses the new type system
based on DataTypes . Please make sure to use either the
old or the new type system consistently to avoid unintended behavior. See the website
documentation for more information. |
OutType |
ceil()
Calculates the smallest integer greater than or equal to a given number.
|
OutType |
ceil(TimeIntervalUnit timeIntervalUnit)
Rounds up a time point to the given unit.
|
OutType |
charLength()
Returns the length of a string.
|
OutType |
chr()
Returns the ASCII character result of the input integer.
|
OutType |
collect()
Returns multiset aggregate of a given expression.
|
OutType |
cos()
Calculates the cosine of a given number.
|
OutType |
cosh()
Calculates the hyperbolic cosine of a given value.
|
OutType |
cot()
Calculates the cotangent of a given number.
|
OutType |
count()
Returns the number of input rows for which the field is not null.
|
OutType |
day()
Creates an interval of the given number of days.
|
OutType |
days()
Creates an interval of the given number of days.
|
OutType |
decode(InType charset)
Decodes the first argument into a String using the provided character set.
|
OutType |
degrees()
Converts numeric from radians to degrees.
|
OutType |
desc()
Specifies descending order of an expression i.e. a field for orderBy unresolvedCall.
|
OutType |
distinct()
Similar to a SQL distinct aggregation clause such as COUNT(DISTINCT a), declares that an
aggregation function is only applied on distinct input values.
|
OutType |
dividedBy(InType other)
Returns left divided by right.
|
OutType |
element()
Returns the sole element of an array with a single element.
|
OutType |
encode(InType charset)
Encodes the string into a BINARY using the provided character set.
|
OutType |
end()
Returns the end time (exclusive) of a window when applied on a window reference.
|
OutType |
exp()
Calculates the Euler's number raised to the given power.
|
OutType |
extract(TimeIntervalUnit timeIntervalUnit)
Extracts parts of a time point or time interval.
|
OutType |
firstValue()
Returns the first value of field across all input values.
|
OutType |
flatten()
Converts a Flink composite type (such as Tuple, POJO, etc.) and all of its direct subtypes
into a flat representation where every subtype is a separate field.
|
OutType |
floor()
Calculates the largest integer less than or equal to a given number.
|
OutType |
floor(TimeIntervalUnit timeIntervalUnit)
Rounds down a time point to the given unit.
|
OutType |
fromBase64()
Returns the base string decoded with base64.
|
OutType |
get(int index)
Accesses the field of a Flink composite type (such as Tuple, POJO, etc.) by index and returns
it's value.
|
OutType |
get(String name)
Accesses the field of a Flink composite type (such as Tuple, POJO, etc.) by name and returns
it's value.
|
OutType |
hex()
Returns a string representation of an integer numeric value or a string in hex format.
|
OutType |
hour()
Creates an interval of the given number of hours.
|
OutType |
hours()
Creates an interval of the given number of hours.
|
OutType |
ifNull(InType nullReplacement)
Returns
nullReplacement if the given expression is NULL; otherwise the expression is
returned. |
OutType |
in(InType... elements)
Returns true if an expression exists in a given list of expressions.
|
OutType |
in(Table table)
Returns true if an expression exists in a given table sub-query.
|
OutType |
initCap()
Converts the initial letter of each word in a string to uppercase.
|
OutType |
instr(InType str)
Returns the position of the first occurrence of the input string.
|
OutType |
isEqual(InType other)
Equals.
|
OutType |
isFalse()
Returns true if given boolean expression is false.
|
OutType |
isGreater(InType other)
Greater than.
|
OutType |
isGreaterOrEqual(InType other)
Greater than or equal.
|
OutType |
isJson()
Determine whether a given string is valid JSON.
|
OutType |
isJson(JsonType type)
Determine whether a given string is valid JSON.
|
OutType |
isLess(InType other)
Less than.
|
OutType |
isLessOrEqual(InType other)
Less than or equal.
|
OutType |
isNotEqual(InType other)
Not equal.
|
OutType |
isNotFalse()
Returns true if given boolean expression is not false (for null and true).
|
OutType |
isNotNull()
Returns true if the given expression is not null.
|
OutType |
isNotTrue()
Returns true if given boolean expression is not true (for null and false).
|
OutType |
isNull()
Returns true if the given expression is null.
|
OutType |
isTrue()
Returns true if given boolean expression is true.
|
OutType |
jsonExists(String path)
Determines whether a JSON string satisfies a given search criterion.
|
OutType |
jsonExists(String path,
JsonExistsOnError onError)
Returns whether a JSON string satisfies a given search criterion.
|
OutType |
jsonQuery(String path)
Extracts JSON values from a JSON string.
|
OutType |
jsonQuery(String path,
DataType returnType)
Extracts JSON values from a JSON string.
|
OutType |
jsonQuery(String path,
DataType returnType,
JsonQueryWrapper wrappingBehavior)
Extracts JSON values from a JSON string.
|
OutType |
jsonQuery(String path,
DataType returnType,
JsonQueryWrapper wrappingBehavior,
JsonQueryOnEmptyOrError onEmpty,
JsonQueryOnEmptyOrError onError)
Extracts JSON values from a JSON string.
|
OutType |
jsonQuery(String path,
JsonQueryWrapper wrappingBehavior)
Extracts JSON values from a JSON string.
|
OutType |
jsonQuery(String path,
JsonQueryWrapper wrappingBehavior,
JsonQueryOnEmptyOrError onEmpty,
JsonQueryOnEmptyOrError onError)
Extracts JSON values from a JSON string.
|
OutType |
jsonValue(String path)
Extracts a scalar from a JSON string.
|
OutType |
jsonValue(String path,
DataType returningType)
Extracts a scalar from a JSON string.
|
OutType |
jsonValue(String path,
DataType returningType,
InType defaultOnEmptyOrError)
Extracts a scalar from a JSON string.
|
OutType |
jsonValue(String path,
DataType returningType,
JsonValueOnEmptyOrError onEmpty,
InType defaultOnEmpty,
JsonValueOnEmptyOrError onError,
InType defaultOnError)
Extracts a scalar from a JSON string.
|
OutType |
lastValue()
Returns the last value of field across all input values.
|
OutType |
left(InType len)
Returns the leftmost integer characters from the input string.
|
OutType |
like(InType pattern)
Returns true, if a string matches the specified LIKE pattern.
|
OutType |
listAgg()
Concatenates the values of string expressions and places separator(,) values between them.
|
OutType |
listAgg(String separator)
Concatenates the values of string expressions and places separator values between them.
|
OutType |
ln()
Calculates the natural logarithm of the given value.
|
OutType |
locate(InType str)
Returns the position of the first occurrence in the input string.
|
OutType |
locate(InType str,
InType pos)
Returns the position of the first occurrence in the input string after position integer.
|
OutType |
log()
Calculates the natural logarithm of the given value.
|
OutType |
log(InType base)
Calculates the logarithm of the given value to the given base.
|
OutType |
log10()
Calculates the base 10 logarithm of the given value.
|
OutType |
log2()
Calculates the base 2 logarithm of the given value.
|
OutType |
lowerCase()
Returns all of the characters in a string in lower case using the rules of the default
locale.
|
OutType |
lpad(InType len,
InType pad)
Returns a string left-padded with the given pad string to a length of len characters.
|
OutType |
ltrim()
Returns a string that removes the left whitespaces from the given string.
|
OutType |
mapEntries()
Returns an array of all entries in the given map.
|
OutType |
mapKeys()
Returns the keys of the map as an array.
|
OutType |
mapUnion(InType... inputs)
Returns a map created by merging at least one map.
|
OutType |
mapValues()
Returns the values of the map as an array.
|
OutType |
max()
Returns the maximum value of field across all input values.
|
OutType |
md5()
Returns the MD5 hash of the string argument; null if string is null.
|
OutType |
milli()
Creates an interval of the given number of milliseconds.
|
OutType |
millis()
Creates an interval of the given number of milliseconds.
|
OutType |
min()
Returns the minimum value of field across all input values.
|
OutType |
minus(InType other)
Returns left minus right.
|
OutType |
minute()
Creates an interval of the given number of minutes.
|
OutType |
minutes()
Creates an interval of the given number of minutes.
|
OutType |
mod(InType other)
Calculates the remainder of division the given number by another one.
|
OutType |
month()
Creates an interval of the given number of months.
|
OutType |
months()
Creates an interval of the given number of months.
|
OutType |
not()
Inverts a given boolean expression.
|
OutType |
notBetween(InType lowerBound,
InType upperBound)
Returns true if the given expression is not between lowerBound and upperBound (both
inclusive).
|
OutType |
or(InType other)
Boolean OR in three-valued logic.
|
OutType |
over(InType alias)
Defines an aggregation to be used for a previously specified over window.
|
OutType |
overlay(InType newString,
InType starting)
Replaces a substring of string with a string starting at a position (starting at 1).
|
OutType |
overlay(InType newString,
InType starting,
InType length)
Replaces a substring of string with a string starting at a position (starting at 1).
|
OutType |
parseUrl(InType partToExtract)
Parse url and return various parameter of the URL.
|
OutType |
parseUrl(InType partToExtract,
InType key)
Parse url and return various parameter of the URL.
|
OutType |
plus(InType other)
Returns left plus right.
|
OutType |
position(InType haystack)
Returns the position of string in an other string starting at 1.
|
OutType |
power(InType other)
Calculates the given number raised to the power of the other value.
|
OutType |
proctime()
Declares a field as the proctime attribute for indicating, accessing, and working in Flink's
processing time.
|
OutType |
quarter()
Creates an interval of the given number of quarters.
|
OutType |
quarters()
Creates an interval of the given number of quarters.
|
OutType |
radians()
Converts numeric from degrees to radians.
|
OutType |
regexp(InType regex)
Returns TRUE if any (possibly empty) substring matches the Java regular expression, otherwise
FALSE.
|
OutType |
regexpExtract(InType regex)
Returns a string extracted with a specified regular expression.
|
OutType |
regexpExtract(InType regex,
InType extractIndex)
Returns a string extracted with a specified regular expression and a regex match group index.
|
OutType |
regexpReplace(InType regex,
InType replacement)
Returns a string with all substrings that match the regular expression consecutively being
replaced.
|
OutType |
repeat(InType n)
Returns a string that repeats the base string n times.
|
OutType |
replace(InType search,
InType replacement)
Returns a new string which replaces all the occurrences of the search target with the
replacement string (non-overlapping).
|
OutType |
reverse()
Reverse each character in current string.
|
OutType |
right(InType len)
Returns the rightmost integer characters from the input string.
|
OutType |
round(InType places)
Rounds the given number to integer places right to the decimal point.
|
OutType |
rowtime()
Declares a field as the rowtime attribute for indicating, accessing, and working in Flink's
event time.
|
OutType |
rpad(InType len,
InType pad)
Returns a string right-padded with the given pad string to a length of len characters.
|
OutType |
rtrim()
Returns a string that removes the right whitespaces from the given string.
|
OutType |
second()
Creates an interval of the given number of seconds.
|
OutType |
seconds()
Creates an interval of the given number of seconds.
|
OutType |
sha1()
Returns the SHA-1 hash of the string argument; null if string is null.
|
OutType |
sha2(InType hashLength)
Returns the hash for the given string expression using the SHA-2 family of hash functions
(SHA-224, SHA-256, SHA-384, or SHA-512).
|
OutType |
sha224()
Returns the SHA-224 hash of the string argument; null if string is null.
|
OutType |
sha256()
Returns the SHA-256 hash of the string argument; null if string is null.
|
OutType |
sha384()
Returns the SHA-384 hash of the string argument; null if string is null.
|
OutType |
sha512()
Returns the SHA-512 hash of the string argument; null if string is null.
|
OutType |
sign()
Calculates the signum of a given number.
|
OutType |
similar(InType pattern)
Returns true, if a string matches the specified SQL regex pattern.
|
OutType |
sin()
Calculates the sine of a given number.
|
OutType |
sinh()
Calculates the hyperbolic sine of a given value.
|
OutType |
split(InType delimiter)
Returns an array of substrings by splitting the input string based on a given delimiter.
|
OutType |
splitIndex(InType separator,
InType index)
Split target string with custom separator and pick the index-th(start with 0) result.
|
OutType |
sqrt()
Calculates the square root of a given value.
|
OutType |
start()
Returns the start time (inclusive) of a window when applied on a window reference.
|
OutType |
stddevPop()
Returns the population standard deviation of an expression (the square root of varPop()).
|
OutType |
stddevSamp()
Returns the sample standard deviation of an expression (the square root of varSamp()).
|
OutType |
strToMap()
Creates a map by parsing text.
|
OutType |
strToMap(InType listDelimiter,
InType keyValueDelimiter)
Creates a map by parsing text.
|
OutType |
substr(InType beginIndex)
Creates a substring of the given string beginning at the given index to the end.
|
OutType |
substr(InType beginIndex,
InType length)
Creates a substring of the given string at given index for a given length.
|
OutType |
substring(InType beginIndex)
Creates a substring of the given string beginning at the given index to the end.
|
OutType |
substring(InType beginIndex,
InType length)
Creates a substring of the given string at given index for a given length.
|
OutType |
sum()
Returns the sum of the numeric field across all input values.
|
OutType |
sum0()
Returns the sum of the numeric field across all input values.
|
OutType |
tan()
Calculates the tangent of a given number.
|
OutType |
tanh()
Calculates the hyperbolic tangent of a given number.
|
OutType |
then(InType ifTrue,
InType ifFalse)
Ternary conditional operator that decides which of two other expressions should be evaluated
based on a evaluated boolean condition.
|
OutType |
times(InType other)
Returns left multiplied by right.
|
protected abstract OutType |
toApiSpecificExpression(Expression expression) |
OutType |
toBase64()
Returns the base64-encoded result of the input string.
|
OutType |
toDate()
Parses a date string in the form "yyyy-MM-dd" to a SQL Date.
|
protected abstract Expression |
toExpr() |
OutType |
toTime()
Parses a time string in the form "HH:mm:ss" to a SQL Time.
|
OutType |
toTimestamp()
Parses a timestamp string in the form "yyyy-MM-dd HH:mm:ss[.SSS]" to a SQL Timestamp.
|
OutType |
trim()
Removes leading and trailing space characters from the given string.
|
OutType |
trim(InType character)
Removes leading and trailing characters from the given string.
|
OutType |
trimLeading()
Removes leading space characters from the given string.
|
OutType |
trimLeading(InType character)
Removes leading characters from the given string.
|
OutType |
trimTrailing()
Removes trailing space characters from the given string.
|
OutType |
trimTrailing(InType character)
Removes trailing characters from the given string.
|
OutType |
truncate()
Returns a number of truncated to 0 decimal places.
|
OutType |
truncate(InType n)
Returns a number of truncated to n decimal places.
|
OutType |
tryCast(DataType toType)
Like
cast(DataType) , but in case of error, returns null rather than failing
the job. |
OutType |
upperCase()
Returns all of the characters in a string in upper case using the rules of the default
locale.
|
OutType |
varPop()
Returns the population standard variance of an expression.
|
OutType |
varSamp()
Returns the sample variance of a given expression.
|
OutType |
week()
Creates an interval of the given number of weeks.
|
OutType |
weeks()
Creates an interval of the given number of weeks.
|
OutType |
year()
Creates an interval of the given number of years.
|
OutType |
years()
Creates an interval of the given number of years.
|
protected abstract Expression toExpr()
protected abstract OutType toApiSpecificExpression(Expression expression)
public OutType as(String name, String... extraNames)
name
- name for one fieldextraNames
- additional names if the expression expands to multiple fieldspublic OutType arrayExcept(InType array)
If one or both arguments are NULL, the function returns NULL. The order of the elements from array1 is kept.
public OutType arrayIntersect(InType array)
If one or both arguments are NULL, the function returns NULL. The order of the elements from array1 is kept.
public OutType and(InType other)
Expressions.and(Object, Object, Object...)
for prefix notation with multiple arguments.public OutType or(InType other)
Expressions.or(Object, Object, Object...)
for prefix notation with multiple arguments.public OutType not()
This method supports a three-valued logic by preserving NULL
. This means if the
input expression is NULL
, the result will also be NULL
.
The resulting type is nullable if and only if the input type is nullable.
Examples:
lit(true).not() // false
lit(false).not() // true
lit(null, DataTypes.BOOLEAN()).not() // null
public OutType between(InType lowerBound, InType upperBound)
lowerBound
- numeric or comparable expressionupperBound
- numeric or comparable expressionpublic OutType notBetween(InType lowerBound, InType upperBound)
lowerBound
- numeric or comparable expressionupperBound
- numeric or comparable expressionpublic OutType then(InType ifTrue, InType ifFalse)
e.g. lit(42).isGreater(5).then("A", "B") leads to "A"
ifTrue
- expression to be evaluated if condition holdsifFalse
- expression to be evaluated if condition does not holdpublic OutType ifNull(InType nullReplacement)
nullReplacement
if the given expression is NULL; otherwise the expression is
returned.
This function returns a data type that is very specific in terms of nullability. The
returned type is the common type of both arguments but only nullable if the nullReplacement
is nullable.
The function allows to pass nullable columns into a function or table that is declared with a NOT NULL constraint.
E.g., $('nullable_column').ifNull(5)
returns never NULL.
public OutType isNull()
public OutType isNotNull()
public OutType isTrue()
public OutType isFalse()
public OutType isNotTrue()
public OutType isNotFalse()
public OutType distinct()
For example:
orders
.groupBy($("a"))
.select($("a"), $("b").sum().distinct().as("d"))
public OutType sum()
public OutType sum0()
public OutType min()
public OutType max()
public OutType count()
public OutType avg()
public OutType firstValue()
public OutType lastValue()
public OutType listAgg()
public OutType listAgg(String separator)
separator
- string containing the characterpublic OutType stddevPop()
public OutType stddevSamp()
public OutType varPop()
public OutType varSamp()
public OutType collect()
public OutType arrayAgg()
public OutType cast(DataType toType)
toType
. A cast error throws an exception and fails
the job. When performing a cast operation that may fail, like DataTypes.STRING()
to
DataTypes.INT()
, one should rather use tryCast(DataType)
, in order to handle
errors. If ExecutionConfigOptions.TABLE_EXEC_LEGACY_CAST_BEHAVIOUR
is enabled, this
function behaves like tryCast(DataType)
.
E.g. "42".cast(DataTypes.INT())
returns 42
; null.cast(DataTypes.STRING())
returns null
of type DataTypes.STRING()
;
"non-number".cast(DataTypes.INT())
throws an exception and fails the job.
public OutType tryCast(DataType toType)
cast(DataType)
, but in case of error, returns null
rather than failing
the job.
E.g. "42".tryCast(DataTypes.INT())
returns 42
; null.tryCast(DataTypes.STRING())
returns null
of type DataTypes.STRING()
;
"non-number".tryCast(DataTypes.INT())
returns null
of type DataTypes.INT()
; coalesce("non-number".tryCast(DataTypes.INT()), 0)
returns 0
of type DataTypes.INT()
.
@Deprecated public OutType cast(TypeInformation<?> toType)
cast(DataType)
instead which uses the new type system
based on DataTypes
. Please make sure to use either the
old or the new type system consistently to avoid unintended behavior. See the website
documentation for more information.public OutType asc()
public OutType desc()
@SafeVarargs public final OutType in(InType... elements)
If the testing set contains null, the result will be null if the element can not be found and true if it can be found. If the element is null, the result is always null.
e.g. lit("42").in(1, 2, 3) leads to false.
public OutType in(Table table)
Note: This operation is not supported in a streaming environment yet.
public OutType start()
public OutType end()
e.g. if a window ends at 10:59:59.999 this property will return 11:00:00.000.
public OutType mod(InType other)
public OutType exp()
public OutType log10()
public OutType log2()
public OutType ln()
public OutType log()
public OutType power(InType other)
public OutType cosh()
public OutType sqrt()
public OutType abs()
public OutType floor()
public OutType sinh()
public OutType ceil()
public OutType sin()
public OutType cos()
public OutType tan()
public OutType cot()
public OutType asin()
public OutType acos()
public OutType atan()
public OutType tanh()
public OutType degrees()
public OutType radians()
public OutType sign()
public OutType round(InType places)
public OutType bin()
public OutType hex()
E.g. a numeric 20 leads to "14", a numeric 100 leads to "64", and a string "hello,world" leads to "68656c6c6f2c776f726c64".
public OutType truncate(InType n)
public OutType truncate()
public OutType substring(InType beginIndex, InType length)
beginIndex
- first character of the substring (starting at 1, inclusive)length
- number of characters of the substringpublic OutType substring(InType beginIndex)
beginIndex
- first character of the substring (starting at 1, inclusive)public OutType substr(InType beginIndex, InType length)
beginIndex
- first character of the substring (starting at 1, inclusive)length
- number of characters of the substringpublic OutType substr(InType beginIndex)
beginIndex
- first character of the substring (starting at 1, inclusive)public OutType trimLeading()
public OutType trimLeading(InType character)
character
- string containing the characterpublic OutType trimTrailing()
public OutType trimTrailing(InType character)
character
- string containing the characterpublic OutType trim()
public OutType trim(InType character)
character
- string containing the characterpublic OutType replace(InType search, InType replacement)
public OutType charLength()
public OutType upperCase()
public OutType lowerCase()
public OutType initCap()
public OutType like(InType pattern)
e.g. "Jo_n%" matches all strings that start with "Jo(arbitrary letter)n"
public OutType similar(InType pattern)
e.g. "A+" matches all strings that consist of at least one A
public OutType position(InType haystack)
e.g. lit("a").position("bbbbba") leads to 6
public OutType lpad(InType len, InType pad)
e.g. lit("hi").lpad(4, "??") returns "??hi", lit("hi").lpad(1, '??') returns "h"
public OutType rpad(InType len, InType pad)
e.g. lit("hi").rpad(4, "??") returns "hi??", lit("hi").rpad(1, '??') returns "h"
public OutType over(InType alias)
For example:
table
.window(Over partitionBy 'c orderBy 'rowtime preceding 2.rows following CURRENT_ROW as 'w)
.select('c, 'a, 'a.count over 'w, 'a.sum over 'w)
public OutType overlay(InType newString, InType starting)
e.g. lit("xxxxxtest").overlay("xxxx", 6) leads to "xxxxxxxxx"
public OutType overlay(InType newString, InType starting, InType length)
e.g. lit("xxxxxtest").overlay("xxxx", 6, 2) leads to "xxxxxxxxxst"
public OutType regexp(InType regex)
public OutType regexpReplace(InType regex, InType replacement)
public OutType regexpExtract(InType regex, InType extractIndex)
public OutType regexpExtract(InType regex)
public OutType fromBase64()
public OutType toBase64()
public OutType ascii()
public OutType chr()
public OutType decode(InType charset)
public OutType encode(InType charset)
public OutType right(InType len)
public OutType instr(InType str)
public OutType locate(InType str)
public OutType locate(InType str, InType pos)
public OutType parseUrl(InType partToExtract)
public OutType parseUrl(InType partToExtract, InType key)
public OutType ltrim()
public OutType rtrim()
public OutType reverse()
public OutType splitIndex(InType separator, InType index)
separator
- custom separator.index
- index of the result which you want.public OutType strToMap()
public OutType strToMap(InType listDelimiter, InType keyValueDelimiter)
listDelimiter
and keyValueDelimiter
are treated as regular expressions.listDelimiter
- the delimiter to separates pairskeyValueDelimiter
- the delimiter to separates key and valuepublic OutType toDate()
public OutType toTime()
public OutType toTimestamp()
public OutType extract(TimeIntervalUnit timeIntervalUnit)
e.g. lit("2006-06-05").toDate().extract(DAY) leads to 5
public OutType floor(TimeIntervalUnit timeIntervalUnit)
e.g. lit("12:44:31").toDate().floor(MINUTE) leads to 12:44:00
public OutType ceil(TimeIntervalUnit timeIntervalUnit)
e.g. lit("12:44:31").toDate().ceil(MINUTE) leads to 12:45:00
public OutType get(String name)
name
- name of the field (similar to Flink's field expressions)public OutType get(int index)
index
- position of the fieldpublic OutType flatten()
public OutType at(InType index)
index
- key or position of the element (array index starting at 1)public OutType cardinality()
public OutType element()
public OutType arrayAppend(InType element)
If the array itself is null, the function will return null. If an element to add is null, the null element will be added to the end of the array. The given element is cast implicitly to the array's element type if necessary.
public OutType arrayContains(InType needle)
Checking for null elements in the array is supported. If the array itself is null, the function will return null. The given element is cast implicitly to the array's element type if necessary.
public OutType arrayDistinct()
If the array itself is null, the function will return null. Keeps ordering of elements.
public OutType arrayPosition(InType needle)
NOTE: that this is not zero based, but 1-based index. The first element in the array has index 1.
public OutType arrayPrepend(InType element)
If the array itself is null, the function will return null. If an element to add is null, the null element will be added to the beginning of the array. The given element is cast implicitly to the array's element type if necessary.
public OutType arrayRemove(InType needle)
If the array itself is null, the function will return null. Keeps ordering of elements.
public OutType arrayReverse()
If the array itself is null, the function will return null.
public OutType arraySlice(InType startOffset, InType endOffset)
Returns null if any input is null.
public OutType arrayUnion(InType array)
If any of the array is null, the function will return null.
public OutType arrayConcat(InType... arrays)
If any input array is NULL, the function returns NULL.
public OutType arrayMax()
if array itself is null, the function returns null.
public OutType arrayMin()
if array itself is null, the function returns null.
public OutType split(InType delimiter)
If the delimiter is not found in the string, the original string is returned as the only element in the array. If the delimiter is empty, every character in the string is split. If the string or delimiter is null, a null value is returned. If the delimiter is found at the beginning or end of the string, or there are contiguous delimiters, then an empty string is added to the array.
public OutType mapKeys()
public OutType mapValues()
public OutType mapEntries()
public OutType mapUnion(InType... inputs)
public OutType rowtime()
public OutType proctime()
public OutType year()
The produced expression is of type DataTypes.INTERVAL
public OutType years()
public OutType quarter()
public OutType quarters()
public OutType month()
public OutType months()
public OutType week()
public OutType weeks()
public OutType day()
public OutType days()
public OutType hour()
public OutType hours()
public OutType minute()
public OutType minutes()
public OutType second()
public OutType seconds()
public OutType milli()
public OutType millis()
public OutType md5()
public OutType sha1()
public OutType sha224()
public OutType sha256()
public OutType sha384()
public OutType sha512()
public OutType sha2(InType hashLength)
hashLength
- bit length of the result (either 224, 256, 384, or 512)public OutType isJson(JsonType type)
Specifying the optional argument puts a constraint on which type of JSON
object is allowed. If the string is valid JSON, but not that type, false
is returned.
The default is JsonType.VALUE
.
Examples:
lit("1").isJson() // true
lit("[]").isJson() // true
lit("{}").isJson() // true
lit("\"abc\"").isJson() // true
lit("abc").isJson() // false
nullOf(DataTypes.STRING()).isJson() // false
lit("1").isJson(JsonType.SCALAR) // true
lit("1").isJson(JsonType.ARRAY) // false
lit("1").isJson(JsonType.OBJECT) // false
lit("{}").isJson(JsonType.SCALAR) // false
lit("{}").isJson(JsonType.ARRAY) // false
lit("{}").isJson(JsonType.OBJECT) // true
type
- The type of JSON object to validate against.true
if the string is a valid JSON of the given , false
otherwise.public OutType isJson()
This is a shortcut for isJson(JsonType.VALUE)
. See isJson(JsonType)
.
true
if the string is a valid JSON value, false
otherwise.public OutType jsonExists(String path, JsonExistsOnError onError)
This follows the ISO/IEC TR 19075-6 specification for JSON support in SQL.
Examples:
// true
lit("{\"a\": true}").jsonExists("$.a")
// false
lit("{\"a\": true}").jsonExists("$.b")
// true
lit("{\"a\": [{ \"b\": 1 }]}").jsonExists("$.a[0].b")
// true
lit("{\"a\": true}").jsonExists("strict $.b", JsonExistsOnError.TRUE)
// false
lit("{\"a\": true}").jsonExists("strict $.b", JsonExistsOnError.FALSE)
path
- JSON path to search for.onError
- Behavior in case of an error.true
if the JSON string satisfies the search criterion.public OutType jsonExists(String path)
This follows the ISO/IEC TR 19075-6 specification for JSON support in SQL.
Examples:
// true
lit("{\"a\": true}").jsonExists("$.a")
// false
lit("{\"a\": true}").jsonExists("$.b")
// true
lit("{\"a\": [{ \"b\": 1 }]}").jsonExists("$.a[0].b")
// true
lit("{\"a\": true}").jsonExists("strict $.b", JsonExistsOnError.TRUE)
// false
lit("{\"a\": true}").jsonExists("strict $.b", JsonExistsOnError.FALSE)
path
- JSON path to search for.true
if the JSON string satisfies the search criterion.public OutType jsonValue(String path, DataType returningType, JsonValueOnEmptyOrError onEmpty, InType defaultOnEmpty, JsonValueOnEmptyOrError onError, InType defaultOnError)
This method searches a JSON string for a given path expression and returns the value if
the value at that path is scalar. Non-scalar values cannot be returned. By default, the value
is returned as DataTypes.STRING()
. Using a different type can
be chosen, with the following types being supported:
For empty path expressions or errors a behavior can be defined to either return null
, raise an error or return a defined default value instead.
See jsonQuery(String, JsonQueryWrapper, JsonQueryOnEmptyOrError,
JsonQueryOnEmptyOrError)
for extracting non-scalar values from a JSON string.
Examples:
// STRING: "true"
lit("{\"a\": true}").jsonValue("$.a")
// DOUBLE: 0.998
lit("{\"a.b\": [0.998,0.996]}").jsonValue("$.['a.b'][0]", DataTypes.DOUBLE())
// BOOLEAN: true
lit("{\"a\": true}").jsonValue("$.a", DataTypes.BOOLEAN())
// BOOLEAN: "false"
lit("{\"a\": true}").jsonValue("lax $.b",
JsonValueOnEmptyOrError.DEFAULT, false, JsonValueOnEmptyOrError.NULL, null)
// BOOLEAN: "false"
lit("{\"a\": true}").jsonValue("strict $.b",
JsonValueOnEmptyOrError.NULL, null, JsonValueOnEmptyOrError.DEFAULT, false)
path
- JSON path to extract.returningType
- Type to convert the extracted scalar to, otherwise defaults to DataTypes.STRING()
.onEmpty
- Behavior in case the path expression is empty.defaultOnEmpty
- Default value to return if the path expression is empty and is set to JsonValueOnEmptyOrError.DEFAULT
.onError
- Behavior in case of an error.defaultOnError
- Default value to return if there is an error and is
set to JsonValueOnEmptyOrError.DEFAULT
.public OutType jsonValue(String path, DataType returningType)
This method searches a JSON string for a given path expression and returns the value if
the value at that path is scalar. Non-scalar values cannot be returned. By default, the value
is returned as DataTypes.STRING()
.
See also jsonValue(String, DataType, JsonValueOnEmptyOrError, Object,
JsonValueOnEmptyOrError, Object)
.
path
- JSON path to extract.returningType
- Type to convert the extracted scalar to, otherwise defaults to DataTypes.STRING()
.public OutType jsonValue(String path, DataType returningType, InType defaultOnEmptyOrError)
This method searches a JSON string for a given path expression and returns the value if
the value at that path is scalar. Non-scalar values cannot be returned. By default, the value
is returned as DataTypes.STRING()
.
See also jsonValue(String, DataType, JsonValueOnEmptyOrError, Object,
JsonValueOnEmptyOrError, Object)
.
This is a convenience method using JsonValueOnEmptyOrError.DEFAULT
for both empty
and error cases with the same default value.
path
- JSON path to extract.returningType
- Type to convert the extracted scalar to, otherwise defaults to DataTypes.STRING()
.public OutType jsonValue(String path)
This method searches a JSON string for a given path expression and returns the value if
the value at that path is scalar. Non-scalar values cannot be returned. By default, the value
is returned as DataTypes.STRING()
.
See also jsonValue(String, DataType, JsonValueOnEmptyOrError, Object,
JsonValueOnEmptyOrError, Object)
.
path
- JSON path to extract.public OutType jsonQuery(String path, DataType returnType, JsonQueryWrapper wrappingBehavior, JsonQueryOnEmptyOrError onEmpty, JsonQueryOnEmptyOrError onError)
This follows the ISO/IEC TR 19075-6 specification for JSON support in SQL. The result is
always returned as a DataTypes.STRING()
.
The determines whether the extracted value should be wrapped into an array, and whether to do so unconditionally or only if the value itself isn't an array already.
and determine the behavior in case the path expression
is empty, or in case an error was raised, respectively. By default, in both cases null
is returned. Other choices are to use an empty array, an empty object, or to raise an
error.
See jsonValue(String, DataType, JsonValueOnEmptyOrError, Object,
JsonValueOnEmptyOrError, Object)
for extracting scalars from a JSON string.
Examples:
lit("{ \"a\": { \"b\": 1 } }").jsonQuery("$.a") // "{ \"b\": 1 }"
lit("[1, 2]").jsonQuery("$") // "[1, 2]"
nullOf(DataTypes.STRING()).jsonQuery("$") // null
// Wrap result into an array
lit("{}").jsonQuery("$", JsonQueryWrapper.CONDITIONAL_ARRAY) // "[{}]"
lit("[1, 2]").jsonQuery("$", JsonQueryWrapper.CONDITIONAL_ARRAY) // "[1, 2]"
lit("[1, 2]").jsonQuery("$", JsonQueryWrapper.UNCONDITIONAL_ARRAY) // "[[1, 2]]"
// Scalars must be wrapped to be returned
lit(1).jsonQuery("$") // null
lit(1).jsonQuery("$", JsonQueryWrapper.CONDITIONAL_ARRAY) // "[1]"
// Behavior if path expression is empty / there is an error
// "{}"
lit("{}").jsonQuery("lax $.invalid", JsonQueryWrapper.WITHOUT_ARRAY,
JsonQueryOnEmptyOrError.EMPTY_OBJECT, JsonQueryOnEmptyOrError.NULL)
// "[]"
lit("{}").jsonQuery("strict $.invalid", JsonQueryWrapper.WITHOUT_ARRAY,
JsonQueryOnEmptyOrError.NULL, JsonQueryOnEmptyOrError.EMPTY_ARRAY)
// Return results as an array instead of a string
lit("[1, 2]").jsonQuery("$", DataTypes.ARRAY(DataTypes.STRING()),
JsonQueryWrapper.CONDITIONAL_ARRAY) // ["1", "2"]
lit("[1, 2]").jsonQuery("$", DataTypes.ARRAY(DataTypes.STRING()),
JsonQueryWrapper.UNCONDITIONAL_ARRAY) // ["[1, 2]"]
path
- JSON path to search for.returnType
- Type to convert the extracted array to, otherwise defaults to DataTypes.STRING()
.wrappingBehavior
- Determine if and when to wrap the resulting value into an array.onEmpty
- Behavior in case the path expression is empty.onError
- Behavior in case of an error.public OutType jsonQuery(String path, JsonQueryWrapper wrappingBehavior, JsonQueryOnEmptyOrError onEmpty, JsonQueryOnEmptyOrError onError)
The result is returned as a DataTypes.STRING()
.
See also jsonQuery(String, DataType, JsonQueryWrapper, JsonQueryOnEmptyOrError,
JsonQueryOnEmptyOrError)
.
path
- JSON path to search for.wrappingBehavior
- Determine if and when to wrap the resulting value into an array.onEmpty
- Behavior in case the path expression is empty.onError
- Behavior in case of an error.public OutType jsonQuery(String path, JsonQueryWrapper wrappingBehavior)
The determines whether the extracted value should be wrapped into an array, and whether to do so unconditionally or only if the value itself isn't an array already.
See also jsonQuery(String, JsonQueryWrapper, JsonQueryOnEmptyOrError,
JsonQueryOnEmptyOrError)
.
path
- JSON path to search for.wrappingBehavior
- Determine if and when to wrap the resulting value into an array.public OutType jsonQuery(String path, DataType returnType, JsonQueryWrapper wrappingBehavior)
The determines whether the extracted value should be wrapped into an array, and whether to do so unconditionally or only if the value itself isn't an array already.
See also jsonQuery(String, JsonQueryWrapper, JsonQueryOnEmptyOrError,
JsonQueryOnEmptyOrError)
.
path
- JSON path to search for.returnType
- Type to convert the extracted array to, otherwise defaults to DataTypes.STRING()
.wrappingBehavior
- Determine if and when to wrap the resulting value into an array.public OutType jsonQuery(String path)
See also jsonQuery(String, JsonQueryWrapper, JsonQueryOnEmptyOrError,
JsonQueryOnEmptyOrError)
.
path
- JSON path to search for.public OutType jsonQuery(String path, DataType returnType)
See also jsonQuery(String, JsonQueryWrapper, JsonQueryOnEmptyOrError,
JsonQueryOnEmptyOrError)
.
path
- JSON path to search for.returnType
- Type to convert the extracted array to, otherwise defaults to DataTypes.STRING()
.Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.