EclipseIDE上带有配置单元的ApacheSpark抛出特权错误—只读数据库问题

tyg4sfes  于 2021-07-06  发布在  Mysql
关注(0)|答案(0)|浏览(177)

我尝试在eclipseide上测试apachespark与hive集成。以下是每个项目的版本—Hadoop2.7.4、Spark2.2和hive-2.3.2以及EclipseMarsIDE上的MySQL5.7。hive-site.xml的内容如下:,

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value> 
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>password</value>
</property>
<property>
  <name>hive.metastore.port</name>
  <value>33306</value>
</property>
<property>
    <name>hive.server2.enable.doAs</name>
    <value>false</value>
</property>

eclipse ide上的spark hive java api代码是

public static void main(String[] args) {
        Logger.getLogger("org").setLevel(Level.OFF);
        Logger.getLogger("akka").setLevel(Level.OFF);

        // TODO Auto-generated method stub
        SparkSession spark = SparkSession.builder()
                .appName("Spark Hive Test")
                .master("local[*]")
                .config("spark.sql.warehouse.dir","/user/hive/warehouse")
                .enableHiveSupport()
                .getOrCreate();

        System.out.println(spark.version());

        String tableName = "testHiveDriverTable";
      spark.sql("drop table if exists " + tableName);
      String createTable = "create table " + tableName + " (key int, value string) "
                + "ROW FORMAT DELIMITED "
                + "FIELDS TERMINATED BY ',' "
                + "LINES TERMINATED BY '\n' "
                + "STORED AS TEXTFILE";

        spark.sql(createTable);
      spark.sql("LOAD DATA LOCAL INPATH '/home/m_usr/temp/hiveTest.txt' INTO TABLE " + tableName);

      // Queries are expressed in HiveQL
      spark.sql("SELECT * FROM " + tableName).show();

      // Aggregation queries are also supported.
      spark.sql("SELECT COUNT(*) FROM " + tableName).show();

        spark.stop();
        spark.close();
    }

此代码引发异常:

java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
    at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown Source)
    at com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324)
    at com.jolbox.bonecp.ConnectionHandle.<init>(ConnectionHandle.java:262)
    at com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:115)
    at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Caused by: ERROR 25505: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
    at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
    at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
    at org.apache.derby.impl.sql.conn.GenericAuthorizer.setReadOnlyConnection(Unknown Source)
    at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.setReadOnly(Unknown Source)
    ... 8 more

但是在根模式下执行eclipseide时,不会抛出异常。我认为这个问题与特权问题有关。但是我不知道是哪个过程导致了这个问题。还有一个问题是apachederby抛出的异常,而不是mysql。我在hive-site.xml上的mysql配置似乎是错误的。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题