NodeJS 我如何拦截所有的页面流量在一个可滚动的网站上剧作家?

q8l4jmvw  于 2023-05-17  发布在  Node.js
关注(0)|答案(2)|浏览(82)

所以我使用以下代码:

import { firefox } from 'playwright';
// domain
let domain = 'https://www.reddit.com/'

// create a new page
const page = await browser.newPage();
// set routes
await page.route('**', async route => {
  const response = await route.fetch();
  await route.fulfill({ response });
});
// go to the domain
await page.goto(domain);

当运行它时,我可以看到流量通过我的路线,但过了一段时间,当我开始向下滚动reddit页面时,我就看不到它了。我可以在开发工具上看到流量,但它不会被我的路由所捕获。我认为它可能只是为了加载页面而拦截了所有的流量,而不是之后的任何其他请求。我怎么能拦截所有的交通coming从和到该网页。感谢您的评分
我尝试了其他可滚动的网络应用程序,如tiktok,结果也是一样。我希望能够路由所有的交通从我的网页。

rt4zxlrg

rt4zxlrg1#

你需要处理请求事件而不是获取事件:
import { firefox } from 'playwright';

const browser = await firefox.launch();
  const context = await browser.newContext();
  const page = await context.newPage();

  // Set up the request interception
  await page.route('**', async (route) => {
    const request = route.request();
    console.log('Intercepted request:', request.url());
    await route.continue();
  });

  // Navigate to the website
  await page.goto('https://www.reddit.com/');

  // Scroll down to trigger additional requests
  await page.evaluate(() => {
    window.scrollTo(0, document.body.scrollHeight);
  });

  // Wait for a while to allow more requests to be intercepted
  await page.waitForTimeout(3000);

  // Close the browser
  await browser.close();
snvhrwxg

snvhrwxg2#

监控网络
使用playwright,您可以监视页面上的所有RequestsResponses

// Subscribe to 'request' and 'response' events.
page.on('request', request => console.log('>>', request.method(), request.url()));
page.on('response', response => console.log('<<', response.status(), response.url()));

// Navigate to the website
  await page.goto('https://www.reddit.com/');


// Scroll down to trigger additional requests
    await page.evaluate(() => {
        window.scrollTo(0, document.body.scrollHeight);
    });

相关问题