自动将csv文件插入mysql表

sbdsn5lh  于 2021-06-20  发布在  Mysql
关注(0)|答案(1)|浏览(284)

我正在尝试将大约2000个csv文件中的每一行插入mysql表。在下面的代码中,我只插入了一个文件中的一行。如何使代码自动化,以便为每个文件插入所有行?插入只需要做一次。

import pymysql.cursors
connection = pymysql.connect(host='localhost',
                         user='s',
                         password='n9',
                         db='si',
                         charset='utf8mb4',
                         cursorclass=pymysql.cursors.DictCursor)

try:
with connection.cursor() as cursor:
    sql = "INSERT INTO `TrainsS` (`No.`, `Name`,`Zone`,`From`,`Delay`,`ETA`,`Location`,`To`) VALUES (%s,%s,%s,%s,%s,%s,%s, %s)"
    cursor.execute(sql, ('03', 'P Exp','SF','HWH', 'none','no arr today','n/a','ND'))

connection.commit()
finally:
connection.close()
qfe3c7zg

qfe3c7zg1#

检查一下这个代码怎么样?
要运行这个程序,你可以把所有的.csv文件放在一个文件夹中,os.walk(文件夹位置)这个文件夹可以得到所有的.csv文件的位置,然后我把它们一个接一个地打开,插入到这里需要的db(mysql)中。

import pandas as pd
import os
import subprocess
import warnings
warnings.simplefilter("ignore")

cwd = os.getcwd()

connection = pymysql.connect(host='localhost',
                     user='s',
                     password='n9',
                     db='si',
                     charset='utf8mb4',
                     cursorclass=pymysql.cursors.DictCursor)
files_csv = []
for subdir, dir, file in os.walk(cwd):
    files_csv += [ fi for fi in file if fi.endswith(".csv") ]

print(files_csv)

for i in range(len(files_csv)):
    with open(os.path.join(cwd, files_csv[i])) as f:
        lis=[line.split() for line in f]
        for i,x in enumerate(lis):
            #print("line{0} = {1}".format(i,x))
            #HERE x contains the row data and you can access it individualy using x[0], x[1], etc
            #USE YOUR MySQL INSERTION commands here and insert the x row here.
            with connection.cursor() as cursor:
                sql = "INSERT INTO `TrainsS` (`No.`, `Name`,`Zone`,`From`,`Delay`,`ETA`,`Location`,`To`) VALUES (%s,%s,%s,%s,%s,%s,%s, %s)"
                cursor.execute(sql, (#CONVERTED VALUES FROM x))

connection.commit()

更新-获取的值(从x转换的值)

values = ""
for i in range(len(columns)):
    values = values + x[i] + "," # Here x[i] gives a record data in ith row. Here i'm just appending the all values to be inserted in the sql table.
values = values[:-1] # Removing the last extra comma.
command = "INSERT INTO `TrainsS` (`No.`, `Name`,`Zone`,`From`,`Delay`,`ETA`,`Location`,`To`) VALUES (" + str(values) + ")"
cursor.execute(command)

# Then commit using connection.commit()

相关问题