org.apache.flink.examples.java.distcp

Class DistCp

java.lang.Object

org.apache.flink.examples.java.distcp.DistCp

public class DistCp
extends Object

A main class of the Flink distcp utility. It's a simple reimplementation of Hadoop distcp (see http://hadoop.apache.org/docs/r1.2.1/distcp.html) with a dynamic input format Note that this tool does not deal with retries. Additionally, empty directories are not copied over.

When running locally, local file systems paths can be used. However, in a distributed environment HDFS paths must be provided both as input and output.

Note: All Flink DataSet APIs are deprecated since Flink 1.18 and will be removed in a future Flink major version. You can still build your application in DataSet, but you should move to either the DataStream and/or Table API. This class is retained for testing purposes.

Field Summary

Fields

Modifier and Type	Field and Description
static String	BYTES_COPIED_CNT_NAME
static String	FILES_COPIED_CNT_NAME

Constructor Summary

Constructors

Constructor and Description
DistCp()

Method Summary

Modifier and Type	Method and Description
static void	main(String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BYTES_COPIED_CNT_NAME

public static final String BYTES_COPIED_CNT_NAME

See Also:

Constant Field Values

FILES_COPIED_CNT_NAME

public static final String FILES_COPIED_CNT_NAME

See Also:

Constant Field Values

Constructor Detail

DistCp

public DistCp()

Method Detail

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception