tensorflow 在苹果芯片(M1 Max)上使用Python时,有没有办法防止ray.init()挂起?

czq61nw1  于 2022-11-25  发布在  Python
关注(0)|答案(1)|浏览(196)

因此,我尝试在Apple Silicon(M1 Max)上的Jupyter笔记本电脑(在Miniforge虚拟环境中)中运行ray[rllib]。虽然我可以正常地将import ray导入笔记本电脑,但下一个步骤(运行ray.init())会导致笔记本电脑挂起。没有返回错误--ray.init()永远不会完成。是否有解决此问题的方法?
这是我第一次使用Ray。我不认为笔记本电脑或我输入的命令是问题所在,因为笔记本电脑是由教师预先制作的,我已经设法让一个相同的笔记本电脑在Windows 10的Miniforge环境中正常运行。
我听从了Ray M1 Mac(Apple Silicon)Support开发人员的建议,安装了Miniforge for the M1并创建了一个虚拟环境。我还利用What is the proper way to install TensorFlow on Apple M1 in 2022线程设计了一个策略,用于安装强化学习应用程序所需的应用程序。以下是我用来设置Miniforge虚拟环境的environment.yml文件的内容:

name: tf-metal
channels:
  - apple
  - conda-forge
dependencies:
  - python=3.9
  - gym-all=0.21.0
  - pip
  - tensorflow-deps

  ## uncommented for use with Jupyter
  - ipykernel

  ## PyPI packages
  - pip:
    - jupyterlab
    - ray[rllib]==1.11
    - tensorflow-macos
    - tensorflow-metal

我在“终端”中创建虚拟环境的步骤如下:

# Download Miniforge3-MacOSX-arm64.sh and make it executable:
chmod u+x ./Miniforge3-MacOSX-arm64.sh

# run Miniforge
./Miniforge3-MacOSX-arm64.sh
# (or update it) ./Miniforge3-MacOSX-arm64.sh -u

# accept terms and conditions...
# run 'conda init' by entering 'yes'
# configure conda (then close and reopen Terminal):
conda config --set auto_activate_base false
# confirm '~/.bash_profile' reflects miniforge settings
# good-to-go...

# set up virtual environment
conda create --name rl_course2  # (choose any name you want)
# confirm acceptability of location (enter 'yes')
# activate env:
conda activate rl_course2
# configure channels (settings recommended by an instructor)
conda config --env --add channels conda-forge
conda config --env --set channel_priority strict
# install dependencies using environment.yml file shown above:
conda env update --name rl_course2 --file '/Users/.../environment.yml'
# check output for errors...(none found via text search)

因此,我创建了虚拟环境,并安装了所有依赖项,没有出现任何错误,就我所知:

Successfully installed MarkupSafe-2.1.1 PyWavelets-1.4.1 Send2Trash-1.8.0 absl-py-1.3.0 anyio-3.6.2 argon2-cffi-21.3.0 argon2-cffi-bindings-21.2.0 astunparse-1.6.3 async-timeout-4.0.2 attrs-22.1.0 babel-2.11.0 beautifulsoup4-4.11.1 bleach-5.0.1 cachetools-5.2.0 certifi-2022.9.24 cffi-1.15.1 charset-normalizer-2.1.1 click-8.1.3 contourpy-1.0.6 cycler-0.11.0 defusedxml-0.7.1 dm-tree-0.1.7 fastjsonschema-2.16.2 filelock-3.8.0 flatbuffers-22.10.26 fonttools-4.38.0 gast-0.4.0 google-auth-2.14.1 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.43.0 idna-3.4 imageio-2.22.4 importlib-metadata-5.0.0 ipython-genutils-0.2.0 jinja2-3.1.2 json5-0.9.10 jsonschema-4.17.1 jupyter-server-1.23.3 jupyterlab-3.5.0 jupyterlab-pygments-0.2.2 jupyterlab-server-2.16.3 keras-2.10.0 keras-preprocessing-1.1.2 kiwisolver-1.4.4 libclang-14.0.6 markdown-3.4.1 matplotlib-3.6.2 mistune-2.0.4 msgpack-1.0.4 nbclassic-0.4.8 nbclient-0.7.0 nbconvert-7.2.5 nbformat-5.7.0 networkx-2.8.8 notebook-6.5.2 notebook-shim-0.2.2 oauthlib-3.2.2 opt-einsum-3.3.0 pandas-1.5.1 pandocfilters-1.5.0 pillow-9.3.0 prometheus-client-0.15.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pyrsistent-0.19.2 pytz-2022.6 pyyaml-6.0 ray-1.11.0 redis-4.3.5 requests-2.28.1 requests-oauthlib-1.3.1 rsa-4.9 scikit-image-0.19.3 sniffio-1.3.0 soupsieve-2.3.2.post1 tabulate-0.9.0 tensorboard-2.10.1 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorboardX-2.5.1 tensorflow-estimator-2.10.0 tensorflow-macos-2.10.0 tensorflow-metal-0.6.0 termcolor-2.1.1 terminado-0.17.0 tifffile-2022.10.10 tinycss2-1.2.1 tomli-2.0.1 typing-extensions-4.4.0 urllib3-1.26.12 webencodings-0.5.1 websocket-client-1.4.2 werkzeug-2.2.2 wrapt-1.14.1 zipp-3.10.0

使用“终端”的最后一步(在rl_course2环境中工作时):发射木星号...

(rl_course2) MacBook-Pro ~$ jupyter notebook

现在,在Jupyter/Python笔记本(Chrome浏览器)中:

import ray   # works!
ray.init()   # never completes (no errors)!

因此,我在相同的环境中使用终端(无笔记本电脑)尝试了类似的步骤:

(rl_course2) MacBook-Pro ~$ python3
Python 3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:48:25) 
[Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
>>> import ray
>>> ray.init()
[no errors, but never completes]

有没有办法解决这个问题,并在我的Jupyter环境中正常运行Ray?
更新1:刚才,我能够使用上面讨论的虚拟环境运行Apple推荐的简单TensorFlow测试脚本(参见Get started with tensorflow-metal),并且在64 GB内存的M1 Max上在大约两分钟内完成了五个阶段的训练,没有任何错误,因此环境看起来工作正常。

ha5z0ras

ha5z0ras1#

我已经找到了我的问题的几个答案之一。更改environment.yml文件(如上所述)导入ray[rllib]而不是ray[rllib]==1.11,使Jupyter笔记本能够正常运行ray.init()并执行笔记本中的剩余代码。()在某些情况下从M1 Max上运行。
总结一下:为了解决Apple Silicon(M1Max)上的ray.init()挂起问题,我将environment.yml文件修改为:

name: tf-metal
channels:
  - apple
  - conda-forge
dependencies:
  - python=3.9
  - gym-all=0.21.0
  - pip
  - tensorflow-deps

  ## uncommented for use with Jupyter
  - ipykernel

  ## PyPI packages
  - pip:
    - jupyterlab
    - ray[rllib]
    - tensorflow-macos
    - tensorflow-metal

我随后使用上面描述的过程创建了一个Miniforge环境,Python版本3. 9. 15和Ray版本2. 1. 0自动安装在笔记本中,笔记本在M1 Max上正常运行。

相关问题