marklogic:使用mlcp导入文件时出错

3htmauhk 于 2021-05-30 发布在 Hadoop

关注(0)|答案(3)|浏览(314)

我尝试使用以下命令在生产环境中运行的marklogic服务器中使用marklogic content pump导入文件。

C:\Users\Admin\Desktop\mlcp-1.3-3\bin>mlcp.bat import -host localhost -port 8891 -username admin -password admin -mode local -input_file_type archive -input_file_path /d:/NewFolder/

输入文件路径包含二进制文件和xml文件。

"D:\NewFolder\20150626200126+0800-000000-BINARY.zip"
"D:\NewFolder\20150626200126+0800-000001-XML.zip"

当我试图在命令提示符下运行导入命令时。我得到了以下我不熟悉的回答。

15/06/29 16:53:11 INFO contentpump.ContentPump: Hadoop library version: 2.6.0
15/06/29 16:53:11 INFO contentpump.LocalJobRunner: Content type: XML
15/06/29 16:53:11 ERROR contentpump.ContentPump: Error running a ContentPump job

java.lang.RuntimeException: Error while running command to get file permissions
: ExitCodeException exitCode=-1073741515:
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:
715)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:808)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
    at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
loadPermissionInfo(RawLocalFileSystem.java:582)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
getPermission(RawLocalFileSystem.java:557)
    at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:
42)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1699)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1681)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:303)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:264)
    at com.marklogic.contentpump.FileAndDirectoryInputFormat.getSplits(FileA
ndDirectoryInputFormat.java:80)
    at com.marklogic.contentpump.ArchiveInputFormat.getSplits(ArchiveInputFo
rmat.java:56)
    at com.marklogic.contentpump.LocalJobRunner.run(LocalJobRunner.java:128)

    at com.marklogic.contentpump.ContentPump.runJobLocally(ContentPump.java:
307)
    at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:204
)
    at com.marklogic.contentpump.ContentPump.main(ContentPump.java:67)

    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
loadPermissionInfo(RawLocalFileSystem.java:620)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
getPermission(RawLocalFileSystem.java:557)
    at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:
42)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1699)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1681)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:303)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:264)
    at com.marklogic.contentpump.FileAndDirectoryInputFormat.getSplits(FileA
ndDirectoryInputFormat.java:80)
    at com.marklogic.contentpump.ArchiveInputFormat.getSplits(ArchiveInputFo
rmat.java:56)
    at com.marklogic.contentpump.LocalJobRunner.run(LocalJobRunner.java:128)

    at com.marklogic.contentpump.ContentPump.runJobLocally(ContentPump.java:
307)
    at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:204
)
    at com.marklogic.contentpump.ContentPump.main(ContentPump.java:67)
java.lang.RuntimeException: Error while running command to get file        permissions
: ExitCodeException exitCode=-1073741515:
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:
715)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:808)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
    at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
loadPermissionInfo(RawLocalFileSystem.java:582)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
getPermission(RawLocalFileSystem.java:557)
    at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:
42)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1699)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1681)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:303)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:264)
    at com.marklogic.contentpump.FileAndDirectoryInputFormat.getSplits(FileA
ndDirectoryInputFormat.java:80)
    at com.marklogic.contentpump.ArchiveInputFormat.getSplits(ArchiveInputFo
rmat.java:56)
    at com.marklogic.contentpump.LocalJobRunner.run(LocalJobRunner.java:128)

    at com.marklogic.contentpump.ContentPump.runJobLocally(ContentPump.java:
307)
    at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:204
)
    at com.marklogic.contentpump.ContentPump.main(ContentPump.java:67)

    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
loadPermissionInfo(RawLocalFileSystem.java:620)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
getPermission(RawLocalFileSystem.java:557)
    at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:
42)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1699)
    at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1681)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:303)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:264)
    at com.marklogic.contentpump.FileAndDirectoryInputFormat.getSplits(FileA
ndDirectoryInputFormat.java:80)
    at com.marklogic.contentpump.ArchiveInputFormat.getSplits(ArchiveInputFo
rmat.java:56)
    at com.marklogic.contentpump.LocalJobRunner.run(LocalJobRunner.java:128)

    at com.marklogic.contentpump.ContentPump.runJobLocally(ContentPump.java:
307)
    at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:204
)
    at com.marklogic.contentpump.ContentPump.main(ContentPump.java:67)

在这个问题上有人能帮我吗？
谢谢。

hadoop marklogic mlcp

来源：https://stackoverflow.com/questions/31113529/marklogic-error-while-importing-files-using-mlcp

3条答案

按热度按时间

gwo2fgha1#

我在做marklogic大学课程xqueryii时遇到了同样的错误。他们为课程提供的虚拟机速度非常慢，所以我并行安装了自己的Windows7虚拟机。
当我在第六单元写这个的时候： mlcp.bat import -host localhost -port 8012 -username admin -password admin -input_file_path C:\mls-developer-2\socialmedia\content\enriched -mode local -input_file_pattern "disqus.*\.xml" -output_uri_replace "C:/mls-developer-2/socialmedia/content/enriched, 'socialmedia/disqus'" 我也犯了同样的错误。。。我试着设置java选项来增加jvm可用的内存，就像一些帖子建议的那样，但是没有效果。
我最终将命令改为： mlcp.bat import -host localhost -port 8012 -username admin -password admin -input_file_path "C:\mls-developer-2\socialmedia\content\enriched\*.xml" -mode local -output_uri_replace "C:/mls-developer-2/socialmedia/content/enriched, 'socialmedia/disqus'" 它不喜欢输入文件模式“discus.*.xml”。看起来和你的情况不一样，但我想我会把我的问题贴在这里给其他人看。

赞(0）回复(0）举报 2021-05-30

n7taea2i2#

错误消息类似于这里列出的：hadoop error stalling job reduce process
解决方案是增加堆大小。添加 -Xmx512m 一个名为 JVM_OPTS ..
嗯！

赞(0）回复(0）举报 2021-05-30

r9f1avp53#

输入文档是使用mlcp导出内容的结果吗？这似乎就是 -input_file_type 档案文件。如果那不是文件的来源，试试看 -input_file_type documents -input_compressed true .

赞(0）回复(0）举报 2021-05-30