加速mysql insert存储过程

eivgtgni  于 2021-06-20  发布在  Mysql
关注(0)|答案(1)|浏览(408)

我试图加快从csv插入1000条记录的过程。我有一个连接到联系人电话表的联系人表。
这是我的相关sql结构:

Contact Table
+----+-----------+----------+------------------+------------+----------------+
| id | firstName | lastName |     primaryEmail | locationId | organizationId |
+----+-----------+----------+------------------+------------+----------------+
|  1 |      John |      Doe | jdoe@noemail.com |          1 |              1 |
+----+-----------+----------+------------------+------------+----------------+

Contact Phone Table
+----+-----------+--------------+---------+----------------+
| id | contactId |       number | primary | organizationId |
+----+-----------+--------------+---------+----------------+
|  1 |         1 | +15555555555 |       1 |              1 |
+----+-----------+--------------+---------+----------------+
|  2 |         1 | +11231231234 |       0 |              1 |
+----+-----------+--------------+---------+----------------+

如果手机和/或电子邮件不存在,我需要插入一个新联系人,该联系人的手机设置为primary。联系人在csv中不能有多个电话号码,但添加后可以手动更新。
这是我提出的mysql存储过程

DELIMITER $$

CREATE PROCEDURE `save_bulk_contact`(IN last_name VARCHAR(128), IN first_name VARCHAR(128), IN email VARCHAR(320), IN location_id BIGINT, IN organization_id BIGINT, IN phone_number VARCHAR(15))
BEGIN

    DECLARE CheckExists INT;
    DECLARE insert_id BIGINT;

    SELECT COUNT(*) INTO CheckExists FROM contact
    LEFT JOIN contact_phone ON contact.id = contact_phone.contactId
    WHERE contact.organizationId = organization_id 
        AND contact.locationId = location_id
        AND ((`primaryEmail` <> '' AND `primaryEmail` = email) OR `number` = phone_number);

    IF (CheckExists = 0) THEN
        INSERT INTO contact
            (`lastName`, `firstName`, `primaryEmail`, `locationId`, `organizationId`)
        VALUE (last_name, first_name, email, location_id, organization_id);
        SET insert_id = LAST_INSERT_ID();

        INSERT INTO contact_phone
            (`contactId`, `number`, `type`, `primary`, `organizationId`)
        VALUE (insert_id, phone_number, 'CELL', 1, organization_id);
    END IF;

END$$

DELIMITER ;

我将这个存储过程用于springjdbc模板批更新。联系人CSV可以包含50000+个联系人。我试过很多方法来解决这个问题,但没有一个是好的。这是另一种尝试:使用jdbc和mysql插入1000条有关系的记录并忽略重复的记录,但我没有得到任何答案。我用一个包含100000个联系人的csv文件运行了这个重java的方法,我的数据库中已经有大约5000个联系人了,花了将近3个小时。
大约30分钟前,我使用上面的存储过程从我的web应用程序开始了50000个联系人的csv上传。到目前为止,已经增加了大约23000个。
我能做些什么来提高这个过程的效率和更快地完成?
更新:我刚完成了50000个插入,花了1.7个小时。

0yycz8jy

0yycz8jy1#

第一。如果两个表中都没有索引,请向其中的组织id和位置id添加索引。把你的检查分为两个语句,使用内部连接并去掉“or”

SELECT COUNT(*) INTO CheckExists FROM contact
INNER JOIN contact_phone ON contact.id = 
contact_phone.contactId
WHERE contact.organizationId = organization_id 
    AND contact.locationId = location_id
    AND ((`primaryEmail` <> '' AND `primaryEmail` = email);

SELECT COUNT(*) INTO CheckExistsTwo FROM contact
INNER JOIN contact_phone ON contact.id = 
contact_phone.contactId
WHERE contact.organizationId = organization_id 
    AND contact.locationId = location_id
    AND  `number` = phone_number;

IF (CheckExists = 0 OR CheckExistsTwo = 0)

相关问题