python 使用asyncio.runJupyter Notebook时,“无法从正在运行的事件循环中调用www.example.com()”

z6psavjg  于 2022-12-10  发布在  Python
关注(0)|答案(6)|浏览(216)

我想使用asyncio获得网页html。
我在jupyter notebook中运行以下代码:

import aiofiles
import aiohttp
from aiohttp import ClientSession

async def get_info(url, session):
    resp = await session.request(method="GET", url=url)
    resp.raise_for_status()
    html = await resp.text(encoding='GB18030')
    with open('test_asyncio.html', 'w', encoding='utf-8-sig') as f:
        f.write(html)
    return html
    
async def main(urls):
    async with ClientSession() as session:
        tasks = [get_info(url, session) for url in urls]
        return await asyncio.gather(*tasks)

if __name__ == "__main__":
    url = ['http://huanyuntianxiazh.fang.com/house/1010123799/housedetail.htm', 'http://zhaoshangyonghefu010.fang.com/house/1010126863/housedetail.htm']
    result = asyncio.run(main(url))

但是,它返回RuntimeError: asyncio.run() cannot be called from a running event loop
有什么问题吗?
怎么解决呢?

yzxexxkh

yzxexxkh1#

asyncio.run()文档说明:
当另一个asyncio事件循环正在同一线程中运行时,不能调用此函数。
在本例中,jupyter(IPython ≥ 7.0)已经在运行一个事件循环:
现在,您可以在IPython终端和笔记本电脑的顶层使用async/await,  在大多数情况下 ,它应该 “正常工作”。将IPython更新到版本7+,将IPykernel更新到版本5+,您就可以开始比赛了。
因此,您不需要自己启动事件循环,而是可以直接调用await main(url),即使您的代码位于任何异步函数之外。

Jupyter(IPython ≥ 7.0)

async def main():
    print(1)
    
await main()

Python ≥ 3.7且IPython〈7.0

import asyncio

async def main():
    print(1)
    
asyncio.run(main())

在您的代码中,将给予:

url = ['url1', 'url2']
result = await main(url)

for text in result:
    pass # text contains your html (text) response

注意事项

与IPython相比,Jupyter使用循环的方式略有不同。

uyto3xhc

uyto3xhc2#

为了补充cglacet的答案-如果你想检测一个循环是否正在运行并自动调整(即在现有循环上运行main(),否则运行asyncio.run()),这里有一个片段可能会被证明是有用的:

# async def main():
#     ...

try:
    loop = asyncio.get_running_loop()
except RuntimeError:  # 'RuntimeError: There is no current event loop...'
    loop = None

if loop and loop.is_running():
    print('Async event loop already running. Adding coroutine to the event loop.')
    tsk = loop.create_task(main())
    # ^-- https://docs.python.org/3/library/asyncio-task.html#task-object
    # Optionally, a callback function can be executed when the coroutine completes
    tsk.add_done_callback(
        lambda t: print(f'Task done with result={t.result()}  << return val of main()'))
else:
    print('Starting new event loop')
    result = asyncio.run(main())
dwbf0jvd

dwbf0jvd3#

就用这个:
https://github.com/erdewit/nest_asyncio

import nest_asyncio
nest_asyncio.apply()
biswetbf

biswetbf4#

结合Pankaj Sharma和Jean Monet的方法,我编写了以下代码片段,它充当asyncio.run(语法略有不同),但也适用于Jupyter笔记本。

class RunThread(threading.Thread):
    def __init__(self, func, args, kwargs):
        self.func = func
        self.args = args
        self.kwargs = kwargs
        self.result = None
        super().__init__()

    def run(self):
        self.result = asyncio.run(self.func(*self.args, **self.kwargs))

def run_async(func, *args, **kwargs):
    try:
        loop = asyncio.get_running_loop()
    except RuntimeError:
        loop = None
    if loop and loop.is_running():
        thread = RunThread(func, args, kwargs)
        thread.start()
        thread.join()
        return thread.result
    else:
        return asyncio.run(func(*args, **kwargs))

用法:

async def test(name):
    await asyncio.sleep(5)
    return f"hello {name}"

run_async(test, "user")  # blocks for 5 seconds and returns "hello user"
b4lqfgs4

b4lqfgs45#

我发现unsync包对于编写在Python脚本和Jupyter REPL中表现相同的代码很有用。

import asyncio
from unsync import unsync

@unsync
async def demo_async_fn():
    await asyncio.sleep(0.1)
    return "done!"

print(demo_async_fn().result())
0aydgbwb

0aydgbwb6#

正如cglacet提到的,文档中说
当另一个asyncio事件循环正在同一线程中运行时,不能调用此函数。
您可以使用其他线程,即-

class ResolveThread(threading.Thread):
            def __init__(self,result1,fun,url):
                self.result1= result1
                self.fun = fun
                self.url = url
                threading.Thread.__init__(self)
            def run(self):
                result1[0] = asyncio.run(self.fun(self.url))

result1 = [None]
sp = ResolveThread(result1)
sp.start()
sp.join() # connect main thread
result = result1[0]

相关问题