python-3.x 已成功转换为CSV文件的JSON文件,而不是将生成的文件保存到本地文件夹并直接保存到S3存储桶

s5a0g9ez  于 2023-06-25  发布在  Python
关注(0)|答案(1)|浏览(93)

我有一些json然后转换为csv文件,现在相同的文件应该保存到s3而不是我的本地文件夹。

logs = {
"testing1": "testing1_value", 
"testing2": "testing2_value",
"testing3": {"testing3a": "testing1_value3a"}, 
"testing4": {"testing4a": {"testing4a1": "testing_value4a1"}}
}
file_name = "testing_file.csv"
bucket_name = "testing_bucket"
file_to_save_in_path = "path_in_s3/testing_file.csv"
client = boto3.client("s3")

from fastapi.responses import StreamingResponse
stream = await create_csv_for_download(logs, file_name)
    
response = StreamingResponse(iter([stream]), media_type="text/csv")
response.headers["Content-Disposition"] = f"attachment; filename={file_name}"

client.put_object(Bucket=bucket_name, Key=file_to_save_in_path, Body=response)

client.upload_file(response, bucket_name, file_to_save_in_path)

现在响应像一些东西=><starlette.responses.StreamingResponse object at 0x7fe084e75fd0>
如何将s3中响应保存在适当的csv文件中。

使用client.put_object时出错:就像下面
**

Parameter validation failed:
Invalid type for parameter Body, value: <starlette.responses.StreamingResponse object at 0x7fe084e75fd0>, type: <class 'starlette.responses.StreamingResponse'>, valid types: <class 'bytes'>, <class 'bytearray'>, file-like object

**

juzqafwq

juzqafwq1#

错误信息非常清楚:

Invalid type for parameter Body, value: <starlette.responses.StreamingResponse object at 0x7fe084e75fd0>, 
  type: <class 'starlette.responses.StreamingResponse'>, 
  valid types: <class 'bytes'>, <class 'bytearray'>, file-like object

它告诉你不能将StreamingResponse对象传递给put_object(),它必须是一个字节数组或文件对象。假设你的create_csv_for_download()函数返回一个流对象,你应该只读取它的字节,然后把它发送给put_object()
此外,您在StreamingResponse中设置的HTTP头应该直接传递给put_object()

import boto3
import csv
import io

def create_csv_for_download(logs, filename):
    # Just a stub so this is a self-contained example
    ret = io.StringIO()
    cw = csv.writer(ret)
    for key, value in logs.items():
        cw.writerow([key, str(value)])
    return ret

logs = {
    "testing1": "testing1_value", 
    "testing2": "testing2_value",
    "testing3": {"testing3a": "testing1_value3a"}, 
    "testing4": {"testing4a": {"testing4a1": "testing_value4a1"}}
}

file_name = "testing_file.csv"
bucket_name = "testing_bucket"
file_to_save_in_path = "path_in_s3/testing_file.csv"
client = boto3.client("s3")

stream = create_csv_for_download(logs, file_name)

# Ready out the body from the stream returned
body = stream.read()
if isinstance(body, str):
    # If this stream returns a string, encode it to a byte array
    body = body.encode("utf-8")

client.put_object(
    Bucket=bucket_name, 
    Key=file_to_save_in_path, 
    Body=body, 
    ContentDisposition=f"attachment; filename={file_name}", 
    ContentType="test/csv",
    # Uncomment the following line if you want the link to be 
    # publicly  downloadable from S3 without credentials:
    # ACL="public-read",
)

相关问题