pandas bigquery客户端查询run to_dataframe()在GCP中的airflow composer中不起作用,并引发“bigquery.readsessions.create”权限错误

ulmd4ohb  于 2023-04-28  发布在  其他
关注(0)|答案(1)|浏览(78)

我尝试使用GCP中的airflow composer使bigquery客户端对dataframe的查询运行。这在没有airflow的本地系统中运行良好。即使在airflow中,client.query(query)也运行良好,但client.query(query).to_dataframe()不行,我不明白为什么,如果这是一个权限问题,那么client.query(query)这也不应该起作用。我已经尝试了给予所有可能的权限,但我没有。我不认为这是权限问题。
下面的代码在本地工作,但不是在gcp气流

from google.cloud import bigquery

client = bigquery.Client(project="GCP PROJECT NAME")

#get Brand image score for all interview IDs
query="""SELECT * from Table LIMIT 10"""

df_bi=client.query(query).to_dataframe()

print("Success")

**预期结果:**成功
实际结果

[2023-04-20, 12:47:25 UTC] {taskinstance.py:1853} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/opt/python3.8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/opt/python3.8/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/opt/python3.8/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.PERMISSION_DENIED
    details = "request failed: the user does not have 'bigquery.readsessions.create' permission for 'projects/projectname'"
x8diyxa7

x8diyxa71#

您是否可以尝试首先使用result()方法从客户端获取结果,然后将其转换为dataframe?

from google.cloud import bigquery

client = bigquery.Client(project="GCP PROJECT NAME")

query = """
    SELECT * from Table LIMIT 10
"""

table = client.query(query).result()
df = table.to_dataframe()
print(df)

相关问题