Package com.marklogic.mapreduce.examples
Examples of using MarkLogic Server in MapReduce jobs.
For details on setting up and running the examples in this package, see "Using the Sample Applications" in the MarkLogic Connector for Hadoop Developer's Guide.
Each sample class has an associated configuration property file in CONNECTOR_HOME/conf. See the javadoc for each example class for the configuration file name.
The LinkCount family of examples require a specific database configuration and data set. For set up information, see the MarkLogic Connector for Hadoop Developer's Guide.
-
Class Summary Class Description BinaryReader Read binary documents from MarkLogic Server and write the content out to HDFS.BinaryReader.DocMapper ContentLoader Load documents from HDFS into MarkLogicServer.ContentLoader.ContentMapper ContentReader Read documents from MarkLogic Server using an SSL-enabled connection and write them out to HDFS.ContentReader.DocMapper HelloWorld Read the first word from each input document, then produce a single output document containing the words, sorted, and concatenated into a single string.HelloWorld.MyMapper HelloWorld.MyReducer LinkCount Count the frequency of occurrences of link titles in documents in MarkLogic Server, and write a link count summary to HDFS.LinkCount.IntSumReducer LinkCount.RefMapper LinkCountCooccurrences Count the occurrences of each link title in documents stored in MarkLogic Server, using the lexicon function cts:element-attribute-co-occurrences, then write the link count summary to HDFS.LinkCountCooccurrences.IntSumReducer LinkCountCooccurrences.RefMapper LinkCountHDFS LinkCountHDFS.IntSumReducer LinkCountHDFS.RefMapper LinkCountInDoc Count the number of occurrences of each link title in documents in MarkLogic Server, and save the link count as a child node of each referenced document.LinkCountInDoc.IntSumReducer LinkCountInDoc.RefMapper LinkCountInProperty Read title attributes in links as text and write summary as property to referenced documents.LinkCountInProperty.IntSumReducer LinkCountInProperty.RefMapper LinkCountValue Count the occurrences of each link title in documents stored in MarkLogic Server, then write the link count summary to HDFS.LinkCountValue.IntSumReducer LinkCountValue.RefMapper RevisionGrouper Extract the revision year from the document and add that as a collection to the document.RevisionGrouper.RevisionMapper ZipContentLoader Read zip file stored in HDFS and load their contents into MarkLogic Server.ZipContentLoader.ZipContentMapper