[MarkLogic Dev General] Error while creating MarkLogic Forest with HDFS as data directory

Sudheer Yalaverthi sudheer.yalaverthi at db.com
Mon Mar 23 07:06:51 PDT 2015


The unknown error was due to Java I had. I had the JAVA_HOME pointing to IBM version of java and while trying to connect with HDFS, MarkLogic keeps giving errors in the logs about missing .io files in the ibm java installation directory. We found those missing io files and placed them in the appropriate directory which finally resulted to an unknown error.
After we installed the oracle Java 7 and made the JAVA_HOME to point to that location, MarkLogic worked with CDH 4.3.1 version which is the certified version of Hadoop by MarkLogic.
When I tried with CDH5.3.1 version of hadoop with latest jars, MarkLogic keeps giving below error though I had the jar containing this file in the HDFS client.
2015-03-19 15:53:44.516 Alert: XDMP-FORESTERR: Error in initialization of forest Foresthadoop2: SVC-NOJCLASS: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.util.VersionInfo
When I approached MarkLogic support team, they confirmed that CDH5.3.1 version of Hadoop is not yet certified by MarkLogic and they have it as part of their product map.
The conclusion for now is CDH 5.3.1 HDFS will not work with MarkLogic.

-Sudheer

From: general-bounces at developer.marklogic.com [mailto:general-bounces at developer.marklogic.com] On Behalf Of Sudheer Yalaverthi
Sent: Friday, March 13, 2015 11:20 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Error while creating MarkLogic Forest with HDFS as data directory

I am using MarkLogic 7.0-4.3 version and Cloudera Hadoop Distrbution 5.3.1 for HDFS.


Thanks,
Sudheer

From: general-bounces at developer.marklogic.com<mailto:general-bounces at developer.marklogic.com> [mailto:general-bounces at developer.marklogic.com] On Behalf Of Justin Makeig
Sent: Friday, March 13, 2015 11:16 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Error while creating MarkLogic Forest with HDFS as data directory

Which version of MarkLogic and which vendor and version of Hadoop are you using?

Justin


--
Justin Makeig
Director, Product Management
MarkLogic

On Mar 13, 2015, at 7:48 AM, Sudheer Yalaverthi <sudheer.yalaverthi at db.com<mailto:sudheer.yalaverthi at db.com>> wrote:


Hi,

I have been trying a MarkLogic tiered storage POC with HDFS as storage layer for one of the tier. I have been trying to create a forest with data directory as hadoop file system directory.
I have one hadoop cluster and one Marklogic cluster. I downloaded the configuration files from Hadoop and copied them to /usr/hadoop directory on the MarkLogic machine and I have also downloaded the required jar files based on the documentation here. https://docs.marklogic.com/guide/performance/disk-storage#id_27091
I have placed these as well in the /usr/hadoop directory with proper lib structure.
I am getting the below error when I try creating the forest.
2015-03-12 19:17:20.087 Error: Automount Foresthadoop: SVC-HDFSNOT: HDFS not available for 'hdfs://{namdenode_hostname}:8020/tmp': unknown error
I tried changing the log level to finest in the group configurations and I have also added trace events for forest. But I am not able to get any additional details that could point me to what the error is about.
Any help in this regard would be appreciated. Please let me know if there are any other ways to connect to HDFS as a forest directory.



Thanks,
Sudheer


---
This communication may contain confidential and/or privileged information. If you are not the intended recipient (or have received this communication in error) please notify the sender immediately and destroy this communication. Any unauthorized copying, disclosure or distribution of the material in this communication is strictly forbidden.

Deutsche Bank does not render legal or tax advice, and the information contained in this communication should not be regarded as such.
_______________________________________________
General mailing list
General at developer.marklogic.com<mailto:General at developer.marklogic.com>
http://developer.marklogic.com/mailman/listinfo/general



---
This communication may contain confidential and/or privileged information. If you are not the intended recipient (or have received this communication in error) please notify the sender immediately and destroy this communication. Any unauthorized copying, disclosure or distribution of the material in this communication is strictly forbidden.

Deutsche Bank does not render legal or tax advice, and the information contained in this communication should not be regarded as such.


---
This communication may contain confidential and/or privileged information. If you are not the intended recipient (or have received this communication in error) please notify the sender immediately and destroy this communication. Any unauthorized copying, disclosure or distribution of the material in this communication is strictly forbidden.

Deutsche Bank does not render legal or tax advice, and the information contained in this communication should not be regarded as such.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://developer.marklogic.com/pipermail/general/attachments/20150323/4035d815/attachment-0001.html 


More information about the General mailing list