org.apache.hadoop.tools
Enum DistCpOptionSwitch

java.lang.Object
  java.lang.Enum<DistCpOptionSwitch>
      org.apache.hadoop.tools.DistCpOptionSwitch

All Implemented Interfaces:

Serializable, Comparable<DistCpOptionSwitch>

public enum DistCpOptionSwitch
extends Enum<DistCpOptionSwitch>

Enumeration mapping configuration keys to distcp command line options.

Enum Constant Summary

ATOMIC_COMMIT
Copy all the source files and commit them atomically to the target This is typically useful in cases where there is a process polling for availability of a file/dir.

BANDWIDTH
Specify bandwidth per map in MB

BANDWIDTH_KB
Specify bandwidth per map in MB

BLOCKING
Should DisctpExecution be blocking

COPY_STRATEGY
Copy strategy is use.

DELETE_MISSING
Deletes missing files in target that are missing from source This allows the target to be in sync with the source contents Typically used in conjunction with SYNC_FOLDERS Incompatible with ATOMIC_COMMIT

FILE_LIMIT

IGNORE_FAILURES
Ignores any failures during copy, and continues with rest.

LOG_PATH
Log path where distcp output logs are written to

MAX_MAPS
Max number of maps to use during copy.

OVERWRITE
Overwrite target-files unconditionally.

PRESERVE_SRC_PATH
Preserve source path

PRESERVE_STATUS
Preserves status of file/path in the target.

SIZE_LIMIT

SKIP_CRC
Skip CRC checks between source and target, when determining what files need to be copied.

SOURCE_FILE_LISTING
Source file listing can be provided to DistCp in a file.

SSL_CONF
Configuration file to use with hftps:// for securely copying files across clusters.

SYNC_FOLDERS
Update target location by copying only files that are missing in the target.

WORK_PATH
Work path to be used only in conjunction in Atomic commit

Method Summary

static void	addToConf(org.apache.hadoop.conf.Configuration conf, DistCpOptionSwitch option) Helper function to set an option to hadoop configuration object
static void	addToConf(org.apache.hadoop.conf.Configuration conf, DistCpOptionSwitch option, String value) Helper function to add an option to hadoop configuration object
String	getConfigLabel() Get Configuration label for the option
org.apache.commons.cli.Option	getOption() Get CLI Option corresponding to the distcp option
String	getSwitch() Get Switch symbol
String	toString()
static DistCpOptionSwitch	valueOf(String name) Returns the enum constant of this type with the specified name.
static DistCpOptionSwitch[]	values() Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class java.lang.Enum

clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, valueOf

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Enum Constant Detail

IGNORE_FAILURES

public static final DistCpOptionSwitch IGNORE_FAILURES

Ignores any failures during copy, and continues with rest. Logs failures in a file

PRESERVE_SRC_PATH

public static final DistCpOptionSwitch PRESERVE_SRC_PATH

Preserve source path

PRESERVE_STATUS

public static final DistCpOptionSwitch PRESERVE_STATUS

Preserves status of file/path in the target. Default behavior with -p, is to preserve replication, block size, user, group and permission on the target file If any of the optional switches are present among rbugp, then only the corresponding file attribute is preserved

SYNC_FOLDERS

public static final DistCpOptionSwitch SYNC_FOLDERS

Update target location by copying only files that are missing in the target. This can be used to periodically sync two folders across source and target. Typically used with DELETE_MISSING Incompatible with ATOMIC_COMMIT

DELETE_MISSING

public static final DistCpOptionSwitch DELETE_MISSING

Deletes missing files in target that are missing from source This allows the target to be in sync with the source contents Typically used in conjunction with SYNC_FOLDERS Incompatible with ATOMIC_COMMIT

SSL_CONF

public static final DistCpOptionSwitch SSL_CONF

Configuration file to use with hftps:// for securely copying files across clusters. Typically the configuration file contains truststore/keystore information such as location, password and type

MAX_MAPS

public static final DistCpOptionSwitch MAX_MAPS

Max number of maps to use during copy. DistCp will split work as equally as possible among these maps

SOURCE_FILE_LISTING

public static final DistCpOptionSwitch SOURCE_FILE_LISTING

Source file listing can be provided to DistCp in a file. This allows DistCp to copy random list of files from source and copy them to target

ATOMIC_COMMIT

public static final DistCpOptionSwitch ATOMIC_COMMIT

Copy all the source files and commit them atomically to the target This is typically useful in cases where there is a process polling for availability of a file/dir. This option is incompatible with SYNC_FOLDERS & DELETE_MISSING

WORK_PATH

public static final DistCpOptionSwitch WORK_PATH

Work path to be used only in conjunction in Atomic commit

LOG_PATH

public static final DistCpOptionSwitch LOG_PATH

Log path where distcp output logs are written to

COPY_STRATEGY

public static final DistCpOptionSwitch COPY_STRATEGY

Copy strategy is use. This could be dynamic or uniform size etc. DistCp would use an appropriate input format based on this.

SKIP_CRC

public static final DistCpOptionSwitch SKIP_CRC

Skip CRC checks between source and target, when determining what files need to be copied.

OVERWRITE

public static final DistCpOptionSwitch OVERWRITE

Overwrite target-files unconditionally.

BLOCKING

public static final DistCpOptionSwitch BLOCKING

Should DisctpExecution be blocking

FILE_LIMIT

public static final DistCpOptionSwitch FILE_LIMIT

SIZE_LIMIT

public static final DistCpOptionSwitch SIZE_LIMIT

BANDWIDTH

public static final DistCpOptionSwitch BANDWIDTH

Specify bandwidth per map in MB

BANDWIDTH_KB

public static final DistCpOptionSwitch BANDWIDTH_KB

Specify bandwidth per map in MB

Method Detail

values

public static DistCpOptionSwitch[] values()

Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:

for (DistCpOptionSwitch c : DistCpOptionSwitch.values())
    System.out.println(c);

Returns:

an array containing the constants of this enum type, in the order they are declared

valueOf

public static DistCpOptionSwitch valueOf(String name)

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

name - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum type has no constant with the specified name

NullPointerException - if the argument is null

getConfigLabel

public String getConfigLabel()

Get Configuration label for the option

Returns:

configuration label name

getOption

public org.apache.commons.cli.Option getOption()

Get CLI Option corresponding to the distcp option

Returns:

option

getSwitch

public String getSwitch()

Get Switch symbol

Returns:

switch symbol char

toString

public String toString()

Overrides:

toString in class Enum<DistCpOptionSwitch>

addToConf

public static void addToConf(org.apache.hadoop.conf.Configuration conf,
                             DistCpOptionSwitch option,
                             String value)

Helper function to add an option to hadoop configuration object

Parameters:

conf - - Configuration object to include the option

option - - Option to add

value - - Value

addToConf

public static void addToConf(org.apache.hadoop.conf.Configuration conf,
                             DistCpOptionSwitch option)

Helper function to set an option to hadoop configuration object

Parameters:

conf - - Configuration object to include the option

option - - Option to add