我是hadoop/java世界的新手，所以请温柔一点，随时纠正异常错误。我正在尝试使用在本地运行hadoop（独立模式）的ubuntu机器上编译的本机库。除了编译的.jar之外，我还尝试使用外部的.jar。我尝试制作一个fatjar失败了，我决定尝试通过命令行将外部jar和本机库传递给hadoop。这些库在我创建的自定义记录读取器中使用。我可以通过hadoop命令在没有外部库的情况下运行mapreduce作业。当我设置ld\u library\u path class变量时，我也可以在eclipse中运行这个程序。我不确定需要设置哪些变量才能在hadoop中成功运行此作业，因此请告诉我是否需要一些变量，尽管我已尝试设置$hadoop\u classpath。
即。

./bin/hadoop jar ~/myjar/cdf-11-16.jar CdfInputDriver -libjars cdfjava.jar -files libcdf.so,libcdfNativeLibrary.so input output

我尝试从本地访问jar和so文件，并将它们复制到hdfs。
我从作业中得到以下错误：

Exception in thread "main" java.lang.NoClassDefFoundError: gsfc/nssdc/cdf/CDFConstants
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:274)
    at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1844)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1809)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1903)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobContextImpl.java:174)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:490)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
    at CdfInputDriver.run(CdfInputDriver.java:45)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at CdfInputDriver.main(CdfInputDriver.java:50)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: gsfc.nssdc.cdf.CDFConstants
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 36 more

我试过查看文件是否用以下代码加载到缓存中，它“cache files:”打印为空：

public class CdfInputDriver extends Configured implements Tool{

    @Override
    public int run(String[] args) throws Exception {

        Job job = Job.getInstance(getConf());

        System.out.println("cache files:" + getConf().get("mapreduce.job.cache.files"));
        Path[] uris = job.getLocalCacheFiles();
        for(Path uri: uris){

              System.out.println(uri.toString());
              System.out.println(uri.getName());            

        } 
        job.setJarByClass(getClass());

        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(LongWritable.class);

        job.setInputFormatClass(CdfInputFormat.class);
        job.setOutputFormatClass(TextOutputFormat.class);

        FileInputFormat.setInputPaths(job, new Path(args[0]));
        FileOutputFormat.setOutputPath(job, new Path(args[1]));

        job.setMapperClass(CdfMapper.class);
        //job.setReducerClass(WordCount.IntSumReducer.class);

        return job.waitForCompletion(true) ? 0 : 1;
    }

    public static void main(String[] args)  throws Exception,
        InterruptedException, ClassNotFoundException {
        int exitCode = ToolRunner.run(new CdfInputDriver(), args);
        System.exit(exitCode);
    }
}

另外，我只是测试这个，以不可避免地在amazonemr上运行这个任务。在s3上存储.so和.jar并使用类似的方法理论上可行吗？
谢谢你的帮助！