我试图使用Bigquery Python API从CSV文件导入数据到BigQuery表中。由于它包含一些ASCII控制字符,加载作业失败,错误如下。
第一个月
据观察,我们可以通过在bq命令行(documentation)中设置***--preserve_ascii_control_characters=true***来允许使用ascii控制字符。但通过python API无法找到相同的功能。是否有解决方案来避免此问题?
样本代码:
import six
from google.cloud import bigquery
# Construct a BigQuery client object.
client = bigquery.Client()
# TODO(developer): Set table_id to the ID of the table to create.
# table_id = "your-project.your_dataset.your_table_name
job_config = bigquery.LoadJobConfig(
schema=[
bigquery.SchemaField("name", "STRING"),
bigquery.SchemaField("post_abbr", "STRING"),
],
)
body = six.BytesIO(b"Washington,WA")
client.load_table_from_file(body, table_id, job_config=job_config).result()
previous_rows = client.get_table(table_id).num_rows
assert previous_rows > 0
job_config = bigquery.LoadJobConfig(
write_disposition=bigquery.WriteDisposition.WRITE_TRUNCATE,
source_format=bigquery.SourceFormat.CSV,
skip_leading_rows=1,
)
uri = "gs://cloud-samples-data/bigquery/us-states/us-states.csv"
load_job = client.load_table_from_uri(
uri, table_id, job_config=job_config
) # Make an API request.
load_job.result() # Waits for the job to complete.
destination_table = client.get_table(table_id)
print("Loaded {} rows.".format(destination_table.num_rows))
2条答案
按热度按时间56lgkhnf1#
正如@Ricco D在评论中提到的,
preserve_Ascii_ControlCharacters
还没有在BQ客户端库中实现,你可以关注这个Feature request来获得更新。同时,您可以遵循下列解决方法:
preserveAsciiControlCharacters
设置为true
--preserve_ascii_control_characters=true
的bq命令行3pvhb19x2#
它还没有被添加到图书馆从谷歌的结束,但在那之前,我们可以使用这个