Class InternalUtilities
- java.lang.Object
-
- com.marklogic.mapreduce.utilities.InternalUtilities
-
- All Implemented Interfaces:
MarkLogicConstants
public class InternalUtilities extends Object implements MarkLogicConstants
Internal utilities shared by mapreduce package. No need to document.
-
-
Field Summary
Fields Modifier and Type Field Description static org.apache.commons.logging.LogLOG-
Fields inherited from interface com.marklogic.mapreduce.MarkLogicConstants
ADVANCED_MODE, ASSIGNMENT_POLICY, BASIC_MODE, BATCH_SIZE, BIND_SPLIT_RANGE, COLLECTION_FILTER, CONTENT_TYPE, COPY_COLLECTIONS, COPY_METADATA, COPY_QUALITY, DEFAULT_BATCH_SIZE, DEFAULT_CONTENT_TYPE, DEFAULT_LOCAL_MAX_SPLIT_SIZE, DEFAULT_MAX_SPLIT_SIZE, DEFAULT_OUTPUT_CONTENT_ENCODING, DEFAULT_OUTPUT_XML_REPAIR_LEVEL, DEFAULT_PROPERTY_OPERATION_TYPE, DEFAULT_TXN_SIZE, DIRECTORY_FILTER, DOCUMENT_SELECTOR, EXECUTION_MODE, EXTRACT_URI, INDENTED, INPUT_API_KEY, INPUT_BASE_PATH, INPUT_DATABASE_NAME, INPUT_GRANT_TYPE, INPUT_HOST, INPUT_KEY_CLASS, INPUT_KEYSTORE_PASSWD, INPUT_KEYSTORE_PATH, INPUT_LEXICON_FUNCTION_CLASS, INPUT_MODE, INPUT_PASSWORD, INPUT_PORT, INPUT_QUERY, INPUT_QUERY_LANGUAGE, INPUT_QUERY_TIMESTAMP, INPUT_RESTRICT_HOSTS, INPUT_SSL_OPTIONS_CLASS, INPUT_SSL_PROTOCOL, INPUT_TOKEN_DURATION, INPUT_TOKEN_ENDPOINT, INPUT_TRUSTSTORE_PASSWD, INPUT_TRUSTSTORE_PATH, INPUT_USE_SSL, INPUT_USERNAME, INPUT_VALUE_CLASS, MAX_SPLIT_SIZE, MIN_NODEUPDATE_VERSION, MODE_DISTRIBUTED, MODE_LOCAL, MR_NAMESPACE, NODE_OPERATION_TYPE, OUTPUT_API_KEY, OUTPUT_BASE_PATH, OUTPUT_CLEAN_DIR, OUTPUT_COLLECTION, OUTPUT_CONTENT_ENCODING, OUTPUT_CONTENT_LANGUAGE, OUTPUT_CONTENT_NAMESPACE, OUTPUT_DATABASE_NAME, OUTPUT_DIRECTORY, OUTPUT_FAST_LOAD, OUTPUT_FOREST_HOST, OUTPUT_GRANT_TYPE, OUTPUT_GRAPH, OUTPUT_HOST, OUTPUT_KEY_TYPE, OUTPUT_KEY_VARNAME, OUTPUT_KEYSTORE_PASSWD, OUTPUT_KEYSTORE_PATH, OUTPUT_NAMESPACE, OUTPUT_OVERRIDE_GRAPH, OUTPUT_PARTITION, OUTPUT_PASSWORD, OUTPUT_PERMISSION, OUTPUT_PORT, OUTPUT_PROPERTY_ALWAYS_CREATE, OUTPUT_QUALITY, OUTPUT_QUERY, OUTPUT_QUERY_LANGUAGE, OUTPUT_RESTRICT_HOSTS, OUTPUT_SSL_OPTIONS_CLASS, OUTPUT_SSL_PROTOCOL, OUTPUT_STREAMING, OUTPUT_TOKEN_DURATION, OUTPUT_TOKEN_ENDPOINT, OUTPUT_TRUSTSTORE_PASSWD, OUTPUT_TRUSTSTORE_PATH, OUTPUT_URI_PREFIX, OUTPUT_URI_REPLACE, OUTPUT_URI_SUFFIX, OUTPUT_USE_SSL, OUTPUT_USERNAME, OUTPUT_VALUE_TYPE, OUTPUT_VALUE_VARNAME, OUTPUT_XML_REPAIR_LEVEL, PATH_NAMESPACE, PROPERTY_OPERATION_TYPE, QUERY_FILTER, RECORD_TO_FRAGMENT_RATIO, REDACTION_RULE_COLLECTION, SPLIT_END_VARNAME, SPLIT_QUERY, SPLIT_START_VARNAME, SUBDOCUMENT_EXPRESSION, TEMPORAL_COLLECTION, TXN_SIZE, TYPE_FILTER
-
-
Constructor Summary
Constructors Constructor Description InternalUtilities()
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static <VALUEIN> voidassignResultValue(Class<? extends org.apache.hadoop.io.Writable> valueClass, com.marklogic.xcc.ResultItem result, VALUEIN value)Assign value in Writable type from XCC result item.static voidcheckQueryLanguage(String s)static intcompareUnsignedLong(long x, long y)static StringgetHost(TextArrayWritable hosts)Return the host from the host array based on a random fashionstatic com.marklogic.xcc.ContentSourcegetInputContentSource(org.apache.hadoop.conf.Configuration conf)Get content source for input server.static com.marklogic.xcc.ContentSourcegetInputContentSource(org.apache.hadoop.conf.Configuration conf, String host)Get input content source.static com.marklogic.xcc.ContentSourcegetOutputContentSource(org.apache.hadoop.conf.Configuration conf, String hostName)Get output content source.static StringgetUriWithOutputDir(DocumentURI key, String outputDir)If outputDir is available and valid, modify DocumentURI, and return uri in stringstatic com.marklogic.xcc.types.XdmValuenewValue(com.marklogic.xcc.types.ValueType valueType, Object value)Create new XdmValue from value type and Writables.static voidsleep(long millis)Wake up every 1 second to check whether to abortstatic Stringunparse(String s)static voidverifyHosts(String hostList, String portStr)
-
-
-
Method Detail
-
getInputContentSource
public static com.marklogic.xcc.ContentSource getInputContentSource(org.apache.hadoop.conf.Configuration conf) throws URISyntaxException, com.marklogic.xcc.exceptions.XccConfigException, IOExceptionGet content source for input server.- Parameters:
conf- job configuration.- Returns:
- ContentSource for input server.
- Throws:
URISyntaxExceptioncom.marklogic.xcc.exceptions.XccConfigExceptionIOException
-
getInputContentSource
public static com.marklogic.xcc.ContentSource getInputContentSource(org.apache.hadoop.conf.Configuration conf, String host) throws com.marklogic.xcc.exceptions.XccConfigException, IOExceptionGet input content source.- Parameters:
conf- job configurationhost- host to connect to- Returns:
- content source
- Throws:
IOExceptioncom.marklogic.xcc.exceptions.XccConfigException
-
assignResultValue
public static <VALUEIN> void assignResultValue(Class<? extends org.apache.hadoop.io.Writable> valueClass, com.marklogic.xcc.ResultItem result, VALUEIN value)
Assign value in Writable type from XCC result item.- Type Parameters:
VALUEIN-- Parameters:
valueClass-result-value-
-
getOutputContentSource
public static com.marklogic.xcc.ContentSource getOutputContentSource(org.apache.hadoop.conf.Configuration conf, String hostName) throws com.marklogic.xcc.exceptions.XccConfigException, IOExceptionGet output content source.- Parameters:
conf- job configurationhostName- host name- Returns:
- content source
- Throws:
IOExceptioncom.marklogic.xcc.exceptions.XccConfigExceptionIOException
-
getHost
public static String getHost(TextArrayWritable hosts) throws IOException
Return the host from the host array based on a random fashion- Parameters:
hosts- a WritableArray of host names- Returns:
- the host name
- Throws:
IOException
-
newValue
public static com.marklogic.xcc.types.XdmValue newValue(com.marklogic.xcc.types.ValueType valueType, Object value)Create new XdmValue from value type and Writables.
-
getUriWithOutputDir
public static String getUriWithOutputDir(DocumentURI key, String outputDir)
If outputDir is available and valid, modify DocumentURI, and return uri in string- Parameters:
key-outputDir-- Returns:
- URI
-
compareUnsignedLong
public static int compareUnsignedLong(long x, long y)
-
checkQueryLanguage
public static void checkQueryLanguage(String s)
-
sleep
public static void sleep(long millis) throws InterruptedExceptionWake up every 1 second to check whether to abort- Parameters:
millis-- Throws:
InterruptedException
-
-