public class StopWordsRemover extends Object implements Transformer<StopWordsRemover>, StopWordsRemoverParams<StopWordsRemover>
Note: null values from input array are preserved unless adding null to stopWords explicitly.
Modifier and Type | Class and Description |
---|---|
static class |
StopWordsRemover.RemoveStopWordsFunction
A Scalar Function that removes stop words from input string array.
|
CASE_SENSITIVE, LOCALE, STOP_WORDS
INPUT_COLS
OUTPUT_COLS
Constructor and Description |
---|
StopWordsRemover() |
Modifier and Type | Method and Description |
---|---|
static Set<String> |
getAvailableLocales()
Returns a set of all installed locales.
|
static String |
getDefaultOrUS()
Returns system default locale, or
Locale.US if the default locale is not in available
locales in JVM. |
Map<Param<?>,Object> |
getParamMap()
Returns a map which should contain value for every parameter that meets one of the following
conditions.
|
static StopWordsRemover |
load(org.apache.flink.table.api.bridge.java.StreamTableEnvironment env,
String path) |
static String[] |
loadDefaultStopWords(String language)
Loads the default stop words for the given language.
|
void |
save(String path)
Saves the metadata and bounded data of this stage to the given path.
|
org.apache.flink.table.api.Table[] |
transform(org.apache.flink.table.api.Table... inputs)
Applies the AlgoOperator on the given input tables and returns the result tables.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
get, getParam, set
getCaseSensitive, getLocale, getStopWords, setCaseSensitive, setLocale, setStopWords
getInputCols, setInputCols
getOutputCols, setOutputCols
public org.apache.flink.table.api.Table[] transform(org.apache.flink.table.api.Table... inputs)
AlgoOperator
transform
in interface AlgoOperator<StopWordsRemover>
inputs
- a list of tablespublic void save(String path) throws IOException
Stage
save
in interface Stage<StopWordsRemover>
IOException
public static StopWordsRemover load(org.apache.flink.table.api.bridge.java.StreamTableEnvironment env, String path) throws IOException
IOException
public Map<Param<?>,Object> getParamMap()
WithParams
1) set(...) has been called to set value for this parameter.
2) The parameter is a public final field of this WithParams instance. This includes fields inherited from its interfaces and super-classes.
The subclass which implements this interface could meet this requirement by returning a
member field of the given map type, after having initialized this member field using the
ParamUtils.initializeMapWithDefaultValues(Map, WithParams)
method.
getParamMap
in interface WithParams<StopWordsRemover>
public static String[] loadDefaultStopWords(String language)
Supported languages: danish, dutch, english, finnish, french, german, hungarian, italian, norwegian, portuguese, russian, spanish, swedish, turkish
public static String getDefaultOrUS()
Locale.US
if the default locale is not in available
locales in JVM. The locale is returned as a String.Copyright © 2019–2023 The Apache Software Foundation. All rights reserved.