addmount时未找到org.apache.hadoop.fs.azurebfs.azureblobfilesystem类

ldxq2e6h  于 2021-05-27  发布在  Hadoop
关注(0)|答案(1)|浏览(423)

我有以下设置:

$ export |grep HADOOP
declare -x HADOOP_HOME="/home/jesaremi/hadoop-3.1.3-bin"
declare -x HADOOP_OPTIONAL_TOOLS="hadoop-azure"

$ echo $PATH
/home/jesaremi/hadoop-3.1.3-bin/bin:/home/jesaremi/hadoop-3.1.3-bin/sbin:/opt/protobuf/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/jesaremi/.dotnet/tools:/opt/HP_Fortify/HP_Fortify_SCA_and_Apps_4.30/bin:/opt/HP_Fortify/HP_Fortify_SCA_and_Apps_4.30/bin

$ ls -l /home/jesaremi/hadoop-3.1.3-bin/share/hadoop/common/
total 7452
-rw-rw-r-- 1 jesaremi jesaremi  303043 Dec 19 17:30 hadoop-azure-3.1.3.jar
-rw-rw-r-- 1 jesaremi jesaremi 4092533 Dec 17 09:42 hadoop-common-3.1.3-SNAPSHOT.jar
-rw-rw-r-- 1 jesaremi jesaremi 2877294 Dec 17 09:42 hadoop-common-3.1.3-SNAPSHOT-tests.jar
-rw-rw-r-- 1 jesaremi jesaremi  130001 Dec 17 09:42 hadoop-kms-3.1.3-SNAPSHOT.jar
-rw-rw-r-- 1 jesaremi jesaremi  201637 Dec 17 09:42 hadoop-nfs-3.1.3-SNAPSHOT.jar
drwxrwxr-x 2 jesaremi jesaremi    4096 Dec 17 09:42 jdiff
drwxrwxr-x 2 jesaremi jesaremi    4096 Dec 17 09:42 lib
drwxrwxr-x 2 jesaremi jesaremi    4096 Dec 17 09:42 sources
drwxrwxr-x 3 jesaremi jesaremi    4096 Dec 17 09:42 webapps

然而,当我尝试计算abfs文件系统的数量时,我得到了以下结果:

$ hdfs dfsadmin -addMount abfs://main@test1.dfs.core.windows.net/ /mnt/abfs/main "fs.azure.abfs.account.name=test1.dfs.core.windows.net,fs.azure.account.key.test1.dfs.core.windows.net=<key removed>"
addMount: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem not found
f0ofjuux

f0ofjuux1#

担心hadoop\u可选的\u工具env var不够;你需要把hadoopazurejar和其他一些东西放到common/lib中
从share/hadoop/tools/lib将hadoop azure jar、azure-*和wildfly-openssl.jar复制到share/hadoop/common/lib中
cloudstorejar带有诊断功能,因为它告诉您缺少哪个jar。

bin/hadoop jar cloudstore-0.1-SNAPSHOT.jar storediag abfs://c@account.dfs.core.windows.net/

相关问题