使用'mysqldump'转储CSV格式的所有表

lx0bsm1f  于 12个月前  发布在  Mysql
关注(0)|答案(8)|浏览(100)

我需要以CSV格式转储MySQL中的 * 所有 * 表。
是否有一个使用mysqldump的命令可以 * 只 * 以CSV格式输出每个表的每一行?

hiz5n14c

hiz5n14c1#

首先,我可以给予你一个表的答案:
所有这些INTO OUTFILE--tab=tmpfile(和-T/path/to/directory)答案的问题在于,它需要在与MySQL服务器相同的服务器上运行**mysqldump*,并具有这些访问权限。
我的解决方案是简单地使用mysqlnotmysqldump)和-B参数,用-e内联SELECT语句,然后用sed处理ASCII输出,最后得到包含标题字段行的CSV:
范例:

mysql -B -u username -p password database -h dbhost -e "SELECT * FROM accounts;" \
 | sed "s/\"/\"\"/g;s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g"

“id”,“login”,“password”,“folder”,“email”“8”,“mariana”,“xxxxxxxxxx”,“mariana”,”3”,“squaredesign”,“xxxxxxxxxxxxx”,“squaredesign”,”email protected(https://stackoverflow.com/cdn-cgi/l/email-protection)““4”,“miedziak”,“xxxxxxxx”,“miedziak”,”email protected(https://stackoverflow.com/cdn-cgi/l/email-protection)““5”,“Sarko”,“xxxxxxx”,“Sarko”,“6”,“Logitrans波兰”,“xxxxxxxxxx”,“LogitransPoland ",“7”,“Amos”,“xxxxxxxxxxxxxxxx”,“Amos”,”9”,“Annabelle”,“xxxxxxxxxxxxxx”,“Annabelle”,”11”,“Brandfathers and Sons”,“xxxxxxxxxxxxxxx”,“BrandfathersAndSons”,”“12”,“Imagine Group”,“xxxxxxxxxxxx”,“ImagineGroup”,“13”,”EduSquare.pl“,“xxxxxxxxxxxxx”,“EduSquare.pl”,”101”,“tmp”,“xxxxxxxxxxxxxxxxx”,"_",”email protected(https://stackoverflow.com/cdn-cgi/l/email-protection)“
在这个一行程序的末尾添加一个> outfile.csv,以获取该表的CSV文件。
接下来,获取一个包含所有表的列表

mysql -u username -ppassword dbname -sN -e "SHOW TABLES;"

从那里,只需再做一个循环,例如,在Bash shell中迭代这些表:

for tb in $(mysql -u username -ppassword dbname -sN -e "SHOW TABLES;"); do
     echo .....;
 done

do; done之间插入我在上面的第1部分中编写的长命令,但将表名替换为$tb

nhjlsmyf

nhjlsmyf2#

此命令将在 /path/to/directory table_name.sqltable_name.txt 中创建两个文件。
SQL文件将包含表创建模式,txt文件将包含mytable表的记录,字段由逗号分隔。

mysqldump -u username -p -t  -T/path/to/directory dbname table_name --fields-terminated-by=','
gkl3eglg

gkl3eglg3#

mysqldump有CSV格式选项:

--fields-terminated-by=name
                  Fields in the output file are terminated by the given
--lines-terminated-by=name
                  Lines in the output file are terminated by the given

name应包含以下内容之一:

`--fields-terminated-by`

\t"\""

`--fields-enclosed-by=name`
   Fields in the output file are enclosed by the given


--lines-terminated-by

  • \r
  • \n
  • \r\n

当然,你应该mysqldump每个表单独。
我建议您将所有表名收集到一个文本文件中。然后,遍历运行mysqldump的所有表。下面是一个脚本,它将一次转储和gzip 10个表:

MYSQL_USER=root
MYSQL_PASS=rootpassword
MYSQL_CONN="-u${MYSQL_USER} -p${MYSQL_PASS}"
SQLSTMT="SELECT CONCAT(table_schema,'.',table_name)"
SQLSTMT="${SQLSTMT} FROM information_schema.tables WHERE table_schema NOT IN "
SQLSTMT="${SQLSTMT} ('information_schema','performance_schema','mysql')"
mysql ${MYSQL_CONN} -ANe"${SQLSTMT}" > /tmp/DBTB.txt
COMMIT_COUNT=0
COMMIT_LIMIT=10
TARGET_FOLDER=/path/to/csv/files
for DBTB in `cat /tmp/DBTB.txt`
do
    DB=`echo "${DBTB}" | sed 's/\./ /g' | awk '{print $1}'`
    TB=`echo "${DBTB}" | sed 's/\./ /g' | awk '{print $2}'`
    DUMPFILE=${DB}-${TB}.csv.gz
    mysqldump ${MYSQL_CONN} -T ${TARGET_FOLDER} --fields-terminated-by="," --fields-enclosed-by="\"" --lines-terminated-by="\r\n" ${DB} ${TB} | gzip > ${DUMPFILE}
    (( COMMIT_COUNT++ ))
    if [ ${COMMIT_COUNT} -eq ${COMMIT_LIMIT} ]
    then
        COMMIT_COUNT=0
        wait
    fi
done
if [ ${COMMIT_COUNT} -gt 0 ]
then
    wait
fi
8xiog9wr

8xiog9wr4#

如果你使用的是MySQL或MariaDB,最简单和高性能的方式转储单表的CSV是-

SELECT customer_id, firstname, surname INTO OUTFILE '/exportdata/customers.txt'
  FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
  LINES TERMINATED BY '\n'
  FROM customers;

现在可以使用其他技术对多个表重复此命令。请在此处查看更多详细信息:

j2qf4p5b

j2qf4p5b5#

这对我来说很有效:

mysqldump <DBNAME> --fields-terminated-by ',' \
--fields-enclosed-by '"' --fields-escaped-by '\' \
--no-create-info --tab /var/lib/mysql-files/

或者,如果只想转储特定的表:

mysqldump <DBNAME> <TABLENAME> --fields-terminated-by ',' \
--fields-enclosed-by '"' --fields-escaped-by '\' \
--no-create-info --tab /var/lib/mysql-files/

我转储到/var/lib/mysql-files/以避免此错误:
mysqldump:Got error:1290:MySQL服务器正在使用--secure-file-priv选项运行,因此在执行“SELECT INTO OUTFILE”时无法执行此语句

lfapxunr

lfapxunr6#

看起来其他人也有这个问题,现在是there is a simple Python script,用于将mysqldump的输出转换为CSV文件。

wget https://raw.githubusercontent.com/jamesmishra/mysqldump-to-csv/master/mysqldump_to_csv.py
mysqldump -u username -p --host=rdshostname database table | python mysqldump_to_csv.py > table.csv
dgenwo3n

dgenwo3n7#

您也可以使用dbForge Studio for MySQL中的数据导出工具来执行此操作。
它将允许您选择部分或所有表并将其导出为CSV格式。

tzcvj98z

tzcvj98z8#

下面是一个Python解决方案:

import os
import subprocess

import pymysql.cursors

def get_table_names(cursor: pymysql.cursors.Cursor, database_name) -> list[str]:
    """Returns a list of all the table names in the database"""
    with cursor:
        cursor.execute(f"SHOW TABLES FROM {database_name};")
        tables = cursor.fetchall()
    tables = [table[0] for table in tables]
    return tables

def save_clean_data(traget_directory: str, mysql_user: str, my_sql_password: str, database_name: str, host: str, cursor: pymysql.cursors.Cursor) -> None:
    """Saves the each table in the database to a csv file"""
    os.makedirs(traget_directory, exist_ok=True)
    expresion = r"s/\"/\"\"/g;s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g"
    for table_name in get_table_names(cursor, database_name):
        file_path = os.path.join(traget_directory, f'{table_name}.csv')
        if not os.path.exists(file_path):
            dump_command = (
                f'mysql -B -u {mysql_user} -p{my_sql_password} {database_name} -h {host}'
                f' -e "SELECT * FROM {table_name};" | sed "{expresion}" > {file_path}'
            )
            subprocess.call(dump_command, shell=True)

相关问题