我试图从蒸汽刮游戏评论。当运行蜘蛛以上，我得到了10评论的第一页。然后与10评论的第二页三次

class MySpider(scrapy.Spider):
    name = "MySpider"
    download_delay = 6
    page_number = 1
    start_urls = (
    'https://steamcommunity.com/app/1794680/reviews/', 
    )

    custom_settings = {
        'LOG_LEVEL': logging.WARNING,
        'LOG_ENABLED': False,
        'LOG_FILE': 'logging.txt',
        'LOG_FILE_APPEND': False,
        'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7',
        'FEEDS': {"items.json": {"format": "json", 'overwrite': True},},
    }

    def parse(self, response):
        soup = BeautifulSoup(response.text, 'lxml')
        for review in soup.find_all('div', class_="apphub_UserReviewCardContent"):
           {...}
            
        if(self.page_number<4):
            self.page_number +=1
            yield scrapy.Request('https://steamcommunity.com/app/1794680/homecontent/?userreviewscursor=AoIIPwYYanu12fcD&userreviewsoffset={offset}&p={p}&workshopitemspage={p}&readytouseitemspage={p}&mtxitemspage={p}&itemspage={p}&screenshotspage={p}&videospage={p}&artpage={p}&allguidepage={p}&webguidepage={p}&integratedguidepage={p}&discussionspage={p}&numperpage=10&browsefilter=trendweek&browsefilter=trendweek&l=english&appHubSubSection=10&filterLanguage=default&searchText=&maxInappropriateScore=100'.format(offset=10*(self.page_number-1) ,p=self.page_number),method='GET', callback=self.parse)

json output
我在滚动评论时收到了一些请求。我更改了所有看起来像页码的值，并将其替换为{p}，还尝试更改“userreviewsoffset”以适应请求格式
我注意到'userreviewscursor'有一个不断变化的值每个请求，但我不知道它是从哪里来的.