使用sqoop将表从rdbms导入hive之后,约束是否仍然存在?

qcbq4gxm  于 2021-06-03  发布在  Sqoop
关注(0)|答案(1)|浏览(322)

当我们使用 sqoop import 要将rdbms表传输到hive,表的约束(如主键)是否仍然存在?
i、 e.表中作为主键的列是否会保留为当前的主键 HIVE . 这些信息会在hive元存储中吗?
谢谢。

w51jfk4q

w51jfk4q1#

正如您在下面的hiveql官方文档的链接中所看到的,自hiveversion2.1.0以来已经添加了主约束和外部约束。Hiveql

因此,我假设在使用sqoop将表导入配置单元时,主键和外键约束将保持不变。
我测试了mysql数据库的sqoop导入,可以看到在导入过程中没有维护主键约束。
mysql表格式:

mysql> show create table employees;
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table     | Create Table                                                                                                                                                                                                                                                                                   |
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| employees | CREATE TABLE `employees` (
  `emp_no` int(11) NOT NULL,
  `birth_date` date NOT NULL,
  `first_name` varchar(14) NOT NULL,
  `last_name` varchar(16) NOT NULL,
  `gender` enum('M','F') NOT NULL,
  `hire_date` date NOT NULL,
  PRIMARY KEY (`emp_no`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0,00 sec)

已使用以下命令将数据从mysql导入配置单元:

sqoop import --connect jdbc:mysql://localhost/employees --username root --password password --table employees --hive-import --create-hive-table --hive-table employees

当我在配置单元中描述表时,我看不到主键约束

hive> show create table employees;
OK
CREATE TABLE `employees`(
  `emp_no` int, 
  `birth_date` string, 
  `first_name` string, 
  `last_name` string, 
  `gender` string, 
  `hire_date` string)
COMMENT 'Imported by sqoop on 2019/03/18 00:24:11'
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'field.delim'='', 
  'line.delim'='\n', 
  'serialization.format'='') 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://localhost:9000/user/hive/warehouse/employees'
TBLPROPERTIES (
  'transient_lastDdlTime'='1552865076')
Time taken: 1.304 seconds, Fetched: 22 row(s)

我插入了一个具有相同雇员编号的新行,以检查hive manage pk约束。新行已添加,如您所见:

hive> insert into employees values (10001, "1986-04-17", "Hichem", 
"BOUSSETTA", "M", "2014-09-91");
Moving data to directory hdfs://localhost:9000/user/hive/warehouse/employees/.hive-staging_hive_2019-03-18_00-32-16_851_8569619447966100947-1/-ext-10000
Loading data to table default.employees
MapReduce Jobs Launched: 
Stage-Stage-1: Map: 1   Cumulative CPU: 5.79 sec   HDFS Read: 5080 HDFS Write: 120 SUCCESS
Total MapReduce CPU Time Spent: 5 seconds 790 msec
OK
Time taken: 42.422 seconds
hive> select * from employees;
OK
10001   1986-04-17  Hichem  BOUSSETTA   M   2014-09-91
10001   1953-09-02  Georgi  Facello M   1986-06-26
10002   1964-06-02  Bezalel Simmel  F   1985-11-21

因此可以得出结论:sqoop在将rdbms数据导入hive时不保留pk约束

相关问题