MariaDB:如何使用“INSERT ... SELECT”和WITH语句

oymdgrw7  于 2022-11-23  发布在  其他
关注(0)|答案(2)|浏览(213)

注意:这涉及到ColumnStore。
在工作中,我们有一个很大的SQL语句,它占用了太多的内存来在prod上执行。我目前正在努力减少查询所占用的内存。我尝试过使用不同的方法,但由于某种原因,到目前为止,除了WITH ... AS (...)之外,没有任何方法能解决这个问题。但是,我需要将它与INSERT INTO ...结合起来。
这是我正在尝试的代码

TRUNCATE db1.myTable;

INSERT INTO db1.myTable(`all`, `needed`, `columns`)
(WITH everything AS (
  SELECT all, needed, columns
   FROM db1.mainTable T1
   JOIN db1.secondTable T2
     ON (T1.someCol = T2.someCol)
   JOIN db2.thirdTable T3
     ON (T1.anotherCol = T3.anotherCol)
   LEFT JOIN db1.fourthTable T4
     ON (T4.anotherCol = T1.anotherCol)
   WHERE T2.yetAnotherCol >= (some_SELECT_subquery)
     AND T1.valid = 1
) SELECT * FROM everything);

EXPLAIN (WITH everything AS ...返回

+------+-------------+-----------------------+------+---------------+------+---------+------+------+-------------------------------------------------+
| id   | select_type | table                 | type | possible_keys | key  | key_len | ref  | rows | Extra                                           |
+------+-------------+-----------------------+------+---------------+------+---------+------+------+-------------------------------------------------+
|    1 | PRIMARY     | <derived2>            | ALL  | NULL          | NULL | NULL    | NULL | 16000000000000 |                                                 |
|    2 | PRIMARY     | T1                    | ALL  | NULL          | NULL | NULL    | NULL | 2000 | Using where with pushed condition               |
|    2 | PRIMARY     | T2                    | ALL  | NULL          | NULL | NULL    | NULL | 2000 | Using where; Using join buffer (flat, BNL join) |
|    2 | PRIMARY     | T3                    | ALL  | NULL          | NULL | NULL    | NULL | 2000 | Using where; Using join buffer (flat, BNL join) |
|    2 | PRIMARY     | T4                    | ALL  | NULL          | NULL | NULL    | NULL | 2000 | Using where                                     |
|    3 | SUBQUERY    | some_SELECT_subquery  | ALL  | NULL          | NULL | NULL    | NULL | 2000 | Using where with pushed condition               |
+------+-------------+-----------------------+------+---------------+------+---------+------+------+-------------------------------------------------+
5 rows in set (0,21 sec)

如果我只使用WITH-语句,我可以让它工作。就像在,我不使用INSERT INTO。没有任何问题,而且查询甚至更快的方式。我也做了我的快速测试,试图将查询分为几个WITH,但放弃了,因为我认为我搞砸了语法。我不太擅长SQL,而对于JOINs(初级开发人员)则更是如此。
当我把WITH-语句和INSER INTO ...结合在一起时,MariaDB的响应是ERROR 1064 (42000) at line 3: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') SELECT * FROM everything)' at line 1。我也尝试过在... valid = 1后面添加一个分号,合并最后两行,在新行中定位... AS后面的左括号,以及其他一些我能想到的可能与语法相关的问题。没有成功。
我目前的想法是,不能将INSERT INTO ... SELECT ...WITH ...组合在一起。至少在开头不要有WITH,SELECT应该在那里。这是我从docs中收集到的。
所以,简而言之,我的问题是:我能把INSERT INTO ... SELECTWITH-语句结合起来吗?如果不能,我能用另一种技术实现类似的功能吗?
有没有其他方法可以提高查询的内存利用率?我不想为MariaDB或Docker配置选项,但如果这是唯一的可能性,我会考虑它。

xbp102n0

xbp102n01#

你试过这个吗?

TRUNCATE db1.myTable;

WITH everything AS (
  SELECT all, needed, columns
   FROM db1.mainTable T1
   JOIN db1.secondTable T2
     ON (T1.someCol = T2.someCol)
   JOIN db2.thirdTable T3
     ON (T1.anotherCol = T3.anotherCol)
   LEFT JOIN db1.fourthTable T4
     ON (T4.anotherCol = T1.anotherCol)
   WHERE T2.yetAnotherCol >= (some_SELECT_subquery)
     AND T1.valid = 1
) INSERT INTO db1.myTable SELECT * FROM everything;
nzkunb0c

nzkunb0c2#

虽然我没有找到我最初问题的答案,但我们决定通过减少子查询中收集的数据量来解决这个问题。我没有在最初的问题中透露这一点,因为我在发布问题时并不知道这是一个解决方案。我们将从Python脚本中调用SQL,在那里我们可以循环我们想要获取的周数。

WHERE T2.ID >= (SELECT ID - {week_number} FROM db1.secondTable WHERE NOW() BETWEEN monday AND sunday) AND T1.valid = 1);

相关问题