从sqlalchemy转储csv

t1rydlwq  于 12个月前  发布在  其他
关注(0)|答案(9)|浏览(72)

出于某种原因,我想以csv文件的形式从数据库(sqlite3)转储一个表。我使用python脚本和elixir(基于sqlalchemy)来修改数据库。我想知道是否有任何方法可以将我使用的表转储到csv。
我看过sqlalchemy serializer,但它似乎不是我想要的。我做错了吗?我应该在关闭我的sqlalchemy会话后调用sqlite3 python module来转储到一个文件吗?或者我应该用一些自制的东西?

knpiaxh1

knpiaxh11#

这里稍微修改一下汉森的答案,使用SQLAlchemy而不是原始数据库访问

import csv
outfile = open('mydump.csv', 'wb')
outcsv = csv.writer(outfile)
records = session.query(MyModel).all()
[outcsv.writerow([getattr(curr, column.name) for column in MyTable.__mapper__.columns]) for curr in records]
# or maybe use outcsv.writerows(records)

outfile.close()
izj3ouym

izj3ouym2#

有很多方法可以实现这一点,包括一个简单的os.system()调用sqlite3实用程序,如果你已经安装了,但这里大致是我在Python中所做的:

import sqlite3
import csv

con = sqlite3.connect('mydatabase.db')
outfile = open('mydump.csv', 'wb')
outcsv = csv.writer(outfile)

cursor = con.execute('select * from mytable')

# dump column titles (optional)
outcsv.writerow(x[0] for x in cursor.description)
# dump rows
outcsv.writerows(cursor.fetchall())

outfile.close()
eoigrqb6

eoigrqb63#

我把上面的例子改编成我的基于sqlalchemy的代码,如下所示:

import csv
import sqlalchemy as sqAl

metadata = sqAl.MetaData()
engine = sqAl.create_engine('sqlite:///%s' % 'data.db')
metadata.bind = engine

mytable = sqAl.Table('sometable', metadata, autoload=True)
db_connection = engine.connect()

select = sqAl.sql.select([mytable])
result = db_connection.execute(select)

fh = open('data.csv', 'wb')
outcsv = csv.writer(fh)

outcsv.writerow(result.keys())
outcsv.writerows(result)

fh.close

我用sqlalchemy 0.7.9就可以了。我认为这将适用于所有的sqlalchemy表和结果对象。

bbuxkriu

bbuxkriu4#

我知道这是旧的,但我只是有这个问题,这是我如何解决它

import pandas as pd
from sqlalchemy import create_engine

basedir = os.path.abspath(os.path.dirname(__file__))
sql_engine = create_engine(os.path.join('sqlite:///' + os.path.join(basedir, 'single_file_app.db')), echo=False)
results = pd.read_sql_query('select * from users',sql_engine)
results.to_csv(os.path.join(basedir, 'mydump2.csv'),index=False,sep=";")
qcuzuvrc

qcuzuvrc5#

with open('dump.csv', 'wb') as f:
    out = csv.writer(f)
    out.writerow(['id', 'description'])

    for item in session.query(Queue).all():
        out.writerow([item.id, item.description])

如果您不介意手工制作列标签,我发现这很有用。

jaql4c8m

jaql4c8m6#

import csv

f = open('ratings.csv', 'w')
out = csv.writer(f)
out.writerow(['id', 'user_id', 'movie_id', 'rating'])

for item in db.query.all():
    out.writerow([item.username, item.username, item.movie_name, item.rating])
f.close()
8ftvxx2r

8ftvxx2r7#

我花了很多时间寻找这个问题的解决方案,最终创建了这样的东西:

from sqlalchemy import inspect

with open(file_to_write, 'w') as file:
    out_csv = csv.writer(file, lineterminator='\n')

    columns = [column.name for column in inspect(Movies).columns][1:]
    out_csv.writerow(columns)

    session_3 = session_maker()

    extract_query = [getattr(Movies, col) for col in columns]
    for mov in session_3.query(*extract_query):
        out_csv.writerow(mov)

    session_3.close()

它创建了一个CSV文件,其中包含列名和整个“movies”表的转储,而不包含“id”主列。

mo49yndu

mo49yndu8#

In a modular way: an example using slqalchemy with automap and mysql.
database.py:

from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
from sqlalchemy import create_engine

Base = automap_base()

engine = create_engine('mysql://user:pass@localhost:3306/database_name', echo=True)

Base.prepare(engine, reflect=True)

# Map the tables
State = Base.classes.states

session = Session(engine, autoflush=False)

export_to_csv.py:

from databases import *
import csv

def export():

    q = session.query(State)

    file = './data/states.csv'

    with open(file, 'w') as csvfile:
        outcsv = csv.writer(csvfile, delimiter=',',quotechar='"', quoting = csv.QUOTE_MINIMAL)

        header = State.__table__.columns.keys()

        outcsv.writerow(header)     

        for record in q.all():
            outcsv.writerow([getattr(record, c) for c in header ])

if __name__ == "__main__":
    export()

Results:
name,abv,country,is_state,is_lower48,slug,latitude,longitude,population,area Alaska,AK,US,y,n,alaska,61.370716,-152.404419,710231,571951.25 Alabama,AL,US,y,y,alabama,32.806671,-86.79113,4779736,50744.0 Arkansas,AR,US,y,y,arkansas,34.969704,-92.373123,2915918,52068.17 Arizona,AZ,US,y,y,arizona,33.729759,-111.431221,6392017,113634.57 California,CA,US,y,y,california,36.116203,-119.681564,37253956,155939.52 Colorado,CO,US,y,y,colorado,39.059811,-105.311104,5029196,103717.53 Connecticut,CT,US,y,y,connecticut,41.597782,-72.755371,3574097,4844.8 District of Columbia,DC,US,n,n,district-of-columbia,38.897438,-77.026817,601723,68.34 Delaware,DE,US,y,y,delaware,39.318523,-75.507141,897934,1953.56 Florida,FL,US,y,y,florida,27.766279,-81.686783,18801310,53926.82 Georgia,GA,US,y,y,georgia,33.040619,-83.643074,9687653,57906.14

i2loujxw

i2loujxw9#

一个简单的方法来做它使用pandas + sqlalchemy

import os
import pandas as pd
from sqlalchemy import create_engine, select
from sqlalchemy import MetaData, Table
from pathlib import Path   

def convert_to_csv(tablename, filename):
    engine = create_engine('sqlite:///your-file.sqlite')
    connection = engine.connect()
    
    metadata = MetaData()
    table = Table(tablename, metadata, autoload_with=engine)
    stmt = select(table)
    results = connection.execute(stmt).fetchall() # .fetchmany(size=10)

    filepath = Path(filename)  
    filepath.parent.mkdir(parents=True, exist_ok=True)  

    df = pd.DataFrame(results)
    df.to_csv(filepath, index=False)

    print(f'\n🆗 data has exported successfully into {os.getcwd()}/{filepath}\n')

convert_to_csv('your-table-name', '../your-path/your-new-file.csv')

相关问题