跳过CSV文件中的标题

mnowg1ta  于 2022-12-30  发布在  其他
关注(0)|答案(4)|浏览(191)

我添加数据从csv文件使用lambda函数的数据被添加,但有一个错误,在我的表中dynamodb我看到我的标题也在表中的一行这里是我的代码:

import boto3
s3=boto3.client("s3")

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('maysales')

def lambda_handler(event, context):
    bucketna=event['Records'][0]['s3']['bucket']['name']
    s3_name=event['Records'][0]['s3']['object']['key']
    response=s3.get_object(Bucket=bucketn,Key=s3_name)
    data=response['Body'].read().decode("utf-8")
    salesnbs=data.split("\n")
    for ko in salesnbs:
        kos=ko.split(",")
        table.put_item(
            Item = { 
            "Date": kos[0],
            "name": kos[1],
            "fam": kos[2],
            "locati": kos[3],
            "adress": kos[4],
            "country": kos[5],
            "city": kos[6]
        })

我的表格已经包含标题:

shstlldc

shstlldc1#

大多数CSV文件的第一行包含标题标签,如果您不想将该行添加到dynamodb表中,您需要跳过第一行,然后再开始插入,即:

row = 0
for ko in salesnbs:
    if row == 0:
       continue # don't process this line

    row = row + 1
    kos=ko.split(",")
    table.put_item(
        Item = { 
        "Date": kos[0],
        "name": kos[1],
        "fam": kos[2],
        "locati": kos[3],
        "adress": kos[4],
        "country": kos[5],
        "city": kos[6]
    })

(语法可能不是100%正确,但这就是思想)

zujrkrfu

zujrkrfu2#

从描述中还不完全清楚问题出在哪里,但是我建议使用Python内置的csv模块来处理CSV数据,这样你就不用担心标头和将文件拆分成列,因为模块提供了相关的工具。

import csv
...

# Here you can also specify delimiter if need be
reader = csv.DictReader(response['Body'])
for row in reader:
    table.put_item(
            Item = { 
            "Date": row["Date"],
            "name": row["name"],
            "fam": row["fam"],
            ...
        })

模块将文件的第一行用作列名。

af7jpaap

af7jpaap3#

现在从下面修改的@ E. J. Brennan代码,我们可以跳过头,同时把csv文件从s3到dynamodb.下面的代码替换到您的lambda函数.

import boto3
s3_client = boto3.client('s3')
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('yourdynamodbtablename')
def lambda_handler(event, context):
    bucket = event['Records'][0]['s3']['bucket']['name']
    s3_file_name = event['Records'][0]['s3']['object']['key']
    response = s3_client.get_object(Bucket=bucket,Key=s3_file_name)
    fileData = response['Body'].read().decode("utf-8")
    print(fileData)
    modelData = fileData.split("\n")
    header = 0
    for row in modelData:
        print(row)
        if header == 0:
            header = header+1
            continue
        row_data = row.split(",")
        try:
            table.put_item(
                Item = {
                    'ID': row_data[0],
                    'NAME': row_data[1],
                    'SUBJECT': row_data[2]
                }
            )
        except Exceptions as e:
            print('End of File')
    return 'news rows were inserted successful without header into db'
8fq7wneg

8fq7wneg4#

使用boto3客户端而不是资源。安装dynamodb-json

from dynamodb_json import json_util as dynamo_json
import json
import boto3
s3=boto3.client("s3")

dynamodb = boto3.client('dynamodb')

def lambda_handler(event, context):
    bucketna=event['Records'][0]['s3']['bucket']['name']
    s3_name=event['Records'][0]['s3']['object']['key']
    response=s3.get_object(Bucket=bucketn,Key=s3_name)
    data=response['Body'].read().decode("utf-8")
    salesnbs=data.split("\n")
    for ko in salesnbs:
        kos=ko.split(",")
        data = { 
            "Date": kos[0],
            "name": kos[1],
            "fam": kos[2],
            "locati": kos[3],
            "adress": kos[4],
            "country": kos[5],
            "city": kos[6]
        }
       client.put_item(TableName='maysales',Item=json.loads(dynamo_json.dumps(data)))

相关问题