python-3.x Vertex AI Workbench执行器导入Aipplatform包时的上下文版本冲突

oxf4rvwz  于 2023-01-14  发布在  Python
关注(0)|答案(1)|浏览(122)

我在Vertex AI workbench的executor中导入aipplatform包时遇到了以下问题,在Vertex AI workbench中手动运行代码时没有出现这个问题,但是当我设置一个executor按计划运行代码时出现了错误,错误信息如下:上下文版本冲突:(谷歌云大查询3.4.1(/opt/conda/lib/python3.7/站点包),需求.解析(“谷歌云大查询〈3.0.0dev,〉= 1.15.0 '),{”谷歌云应用平台“})
我试过升级aiplatform包(!pip install google-cloud-aiplatform --upgrade),但还是有同样的问题。看起来即使我们把google-cloud-bigquery包(!pip install google-cloud-bigquery==2.34.2 --user)降级到3.0.0以下的版本,执行器容器也会恢复到3.4.1。这又导致了同样的问题。
下面是顶点AI工作台中的脚本:

!pip install google-cloud-aiplatform --upgrade
!pip install google-cloud-bigquery==2.34.2 --user

from google.cloud import bigquery
print('bigquery version:', bigquery.__version__)

from google.cloud import aiplatform
print('aiplatform version:',aiplatform.__version__)

下面是错误消息:

Collecting google-cloud-aiplatform
  Downloading google_cloud_aiplatform-1.20.0-py2.py3-none-any.whl (2.3 MB)
[2K     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 9.6 MB/s eta 0:00:00
[?25hRequirement already satisfied: proto-plus<2.0.0dev,>=1.22.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-aiplatform) (1.22.1)
Collecting google-cloud-bigquery<3.0.0dev,>=1.15.0
  Downloading google_cloud_bigquery-2.34.4-py2.py3-none-any.whl (206 kB)
[2K     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 206.6/206.6 kB 16.6 MB/s eta 0:00:00
[?25hCollecting packaging<22.0.0dev,>=14.3
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
[2K     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 kB 6.2 MB/s eta 0:00:00
[?25hRequirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 in /opt/conda/lib/python3.7/site-packages (from google-cloud-aiplatform) (3.19.6)
Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3
  Downloading google_cloud_resource_manager-1.7.0-py2.py3-none-any.whl (235 kB)
[2K     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 235.3/235.3 kB 13.1 MB/s eta 0:00:00
[?25hRequirement already satisfied: google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-aiplatform) (1.34.0)
Requirement already satisfied: google-cloud-storage<3.0.0dev,>=1.32.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-aiplatform) (2.7.0)
Requirement already satisfied: google-auth<3.0dev,>=1.25.0 in /opt/conda/lib/python3.7/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (2.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (1.57.0)
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /opt/conda/lib/python3.7/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (2.28.1)
Requirement already satisfied: grpcio<2.0dev,>=1.33.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (1.51.1)
Requirement already satisfied: grpcio-status<2.0dev,>=1.33.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (1.48.2)
Requirement already satisfied: python-dateutil<3.0dev,>=2.7.2 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery<3.0.0dev,>=1.15.0->google-cloud-aiplatform) (2.8.2)
Requirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery<3.0.0dev,>=1.15.0->google-cloud-aiplatform) (2.4.0)
Requirement already satisfied: google-cloud-core<3.0.0dev,>=1.4.1 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery<3.0.0dev,>=1.15.0->google-cloud-aiplatform) (2.3.2)
Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4
  Downloading grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl (26 kB)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging<22.0.0dev,>=14.3->google-cloud-aiplatform) (3.0.9)
Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.7/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (4.9)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (0.2.8)
Requirement already satisfied: six>=1.9.0 in /opt/conda/lib/python3.7/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (1.16.0)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (5.2.0)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /opt/conda/lib/python3.7/site-packages (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery<3.0.0dev,>=1.15.0->google-cloud-aiplatform) (1.5.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (1.26.13)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (2022.12.7)
Requirement already satisfied: charset-normalizer<3,>=2 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (3.4)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.0.0dev,>=1.32.0->google-cloud-aiplatform) (0.4.8)
Installing collected packages: packaging, grpc-google-iam-v1, google-cloud-resource-manager, google-cloud-bigquery, google-cloud-aiplatform
  Attempting uninstall: packaging
    Found existing installation: packaging 22.0
    Uninstalling packaging-22.0:
      Successfully uninstalled packaging-22.0
  Attempting uninstall: google-cloud-bigquery
    Found existing installation: google-cloud-bigquery 3.4.1
    Uninstalling google-cloud-bigquery-3.4.1:
      Successfully uninstalled google-cloud-bigquery-3.4.1
Successfully installed google-cloud-aiplatform-1.20.0 google-cloud-bigquery-2.34.4 google-cloud-resource-manager-1.7.0 grpc-google-iam-v1-0.12.4 packaging-21.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Collecting google-cloud-bigquery==2.34.2
  Downloading google_cloud_bigquery-2.34.2-py2.py3-none-any.whl (206 kB)
[2K     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 206.1/206.1 kB 4.7 MB/s eta 0:00:00
[?25hRequirement already satisfied: google-resumable-media<3.0dev,>=0.6.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (2.4.0)
Requirement already satisfied: google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (1.34.0)
Requirement already satisfied: grpcio<2.0dev,>=1.38.1 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (1.51.1)
Requirement already satisfied: python-dateutil<3.0dev,>=2.7.2 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (2.8.2)
Requirement already satisfied: packaging>=14.3 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (21.3)
Requirement already satisfied: protobuf>=3.12.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (3.19.6)
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (2.28.1)
Requirement already satisfied: google-cloud-core<3.0.0dev,>=1.4.1 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (2.3.2)
Requirement already satisfied: proto-plus>=1.15.0 in /opt/conda/lib/python3.7/site-packages (from google-cloud-bigquery==2.34.2) (1.22.1)
Requirement already satisfied: google-auth<3.0dev,>=1.25.0 in /opt/conda/lib/python3.7/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery==2.34.2) (2.15.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery==2.34.2) (1.57.0)
Requirement already satisfied: grpcio-status<2.0dev,>=1.33.2 in /opt/conda/lib/python3.7/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery==2.34.2) (1.48.2)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /opt/conda/lib/python3.7/site-packages (from google-resumable-media<3.0dev,>=0.6.0->google-cloud-bigquery==2.34.2) (1.5.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging>=14.3->google-cloud-bigquery==2.34.2) (3.0.9)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.7/site-packages (from python-dateutil<3.0dev,>=2.7.2->google-cloud-bigquery==2.34.2) (1.16.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery==2.34.2) (2022.12.7)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery==2.34.2) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery==2.34.2) (1.26.13)
Requirement already satisfied: charset-normalizer<3,>=2 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery==2.34.2) (2.1.1)
Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.7/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery==2.34.2) (4.9)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery==2.34.2) (0.2.8)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery==2.34.2) (5.2.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-bigquery==2.34.2) (0.4.8)
Installing collected packages: google-cloud-bigquery
Successfully installed google-cloud-bigquery-2.34.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
bigquery version: 3.4.1
---------------------------------------------------------------------------
ContextualVersionConflict                 Traceback (most recent call last)
/tmp/ipykernel_296/4222064428.py in <module>
      8 print('bigquery version:', bigquery.__version__)
      9 
---> 10 from google.cloud import aiplatform
     11 print('aiplatform version:',aiplatform.__version__)

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform/__init__.py in <module>
     22 
     23 
---> 24 from google.cloud.aiplatform import initializer
     25 
     26 from google.cloud.aiplatform.datasets import (

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform/initializer.py in <module>
     29 from google.auth.exceptions import GoogleAuthError
     30 
---> 31 from google.cloud.aiplatform import compat
     32 from google.cloud.aiplatform.constants import base as constants
     33 from google.cloud.aiplatform import utils

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform/compat/__init__.py in <module>
     16 #
     17 
---> 18 from google.cloud.aiplatform.compat import services
     19 from google.cloud.aiplatform.compat import types
     20 

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform/compat/services/__init__.py in <module>
     16 #
     17 
---> 18 from google.cloud.aiplatform_v1beta1.services.dataset_service import (
     19     client as dataset_service_client_v1beta1,
     20 )

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform_v1beta1/__init__.py in <module>
     15 #
     16 
---> 17 from .services.dataset_service import DatasetServiceClient
     18 from .services.dataset_service import DatasetServiceAsyncClient
     19 from .services.deployment_resource_pool_service import (

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform_v1beta1/services/dataset_service/__init__.py in <module>
     14 # limitations under the License.
     15 #
---> 16 from .client import DatasetServiceClient
     17 from .async_client import DatasetServiceAsyncClient
     18 

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform_v1beta1/services/dataset_service/client.py in <module>
     55 from google.protobuf import struct_pb2  # type: ignore
     56 from google.protobuf import timestamp_pb2  # type: ignore
---> 57 from .transports.base import DatasetServiceTransport, DEFAULT_CLIENT_INFO
     58 from .transports.grpc import DatasetServiceGrpcTransport
     59 from .transports.grpc_asyncio import DatasetServiceGrpcAsyncIOTransport

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform_v1beta1/services/dataset_service/transports/__init__.py in <module>
     17 from typing import Dict, Type
     18 
---> 19 from .base import DatasetServiceTransport
     20 from .grpc import DatasetServiceGrpcTransport
     21 from .grpc_asyncio import DatasetServiceGrpcAsyncIOTransport

/opt/conda/lib/python3.7/site-packages/google/cloud/aiplatform_v1beta1/services/dataset_service/transports/base.py in <module>
     40     DEFAULT_CLIENT_INFO = gapic_v1.client_info.ClientInfo(
     41         gapic_version=pkg_resources.get_distribution(
---> 42             "google-cloud-aiplatform",
     43         ).version,
     44     )

/opt/conda/lib/python3.7/site-packages/pkg_resources/__init__.py in get_distribution(dist)
    476         dist = Requirement.parse(dist)
    477     if isinstance(dist, Requirement):
--> 478         dist = get_provider(dist)
    479     if not isinstance(dist, Distribution):
    480         raise TypeError("Expected string, Requirement, or Distribution", dist)

/opt/conda/lib/python3.7/site-packages/pkg_resources/__init__.py in get_provider(moduleOrReq)
    352     """Return an IResourceProvider for the named module or requirement"""
    353     if isinstance(moduleOrReq, Requirement):
--> 354         return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
    355     try:
    356         module = sys.modules[moduleOrReq]

/opt/conda/lib/python3.7/site-packages/pkg_resources/__init__.py in require(self, *requirements)
    907         included, even if they were already activated in this working set.
    908         """
--> 909         needed = self.resolve(parse_requirements(requirements))
    910 
    911         for dist in needed:

/opt/conda/lib/python3.7/site-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras)
    798                 # Oops, the "best" so far conflicts with a dependency
    799                 dependent_req = required_by[req]
--> 800                 raise VersionConflict(dist, req).with_context(dependent_req)
    801 
    802             # push the new requirements onto the stack

ContextualVersionConflict: (google-cloud-bigquery 3.4.1 (/opt/conda/lib/python3.7/site-packages), Requirement.parse('google-cloud-bigquery<3.0.0dev,>=1.15.0'), {'google-cloud-aiplatform'})
11dmarpk

11dmarpk1#

笔记本执行器将使用您在创建执行时指定的容器。我将创建一个自定义容器,安装您需要的版本,并在创建执行时指定此容器。

相关问题