使用 Scrapy 绕过 Python-telegraph - bot 泛 洪 和 429 错误

yzckvree  于 2022-11-09  发布在  Python
关注(0)|答案(1)|浏览(134)

我跟踪目标站点上的价格下降。如果有价格下降符合我设置的规则,它会被记录在通知表中。从那里,电报通知是通过我在pipelines.py文件中创建的代码发送的。有时目标站点折扣太多的产品,可以从通知表中获得200个产品。我被电报卡住了。在发送这些邮件的同时,还可以防止洪水。
我尝试过的事情:
1-Rotating multiple tokens
2-add sleep time
但是,我不能说我成功了。我仍然被困在防洪堤上。
我想做的是无论来自Notificate表的通知有多少,都要将它们排队,并以每秒不超过20条消息的方式发送这些通知。
我怎么能那样做。
我的pipeline.py代码:

def sendnotifications(self, token):
        cursor = self.cnx.cursor()
        req = requests
        cursor.execute("SELECT * FROM notificate WHERE token= '"+token+"'")
        notifications = cursor.fetchall()
        for notification in notifications:
            print(notification)
            productid = notification[1]
            url = notification[3]
            name = notification[2]
            old = notification[4]
            new = notification[5]
            price_difference = old - new
            percentage = price_difference / old
            percentage_str = str("%.2f" % (percentage * 100))

            message = "<b>" + name + "</b>" + "\n\n" + \
                str(old) + " TL >>>> " + \
                str(new) + f" TL - {percentage_str}%" + "\n\n" + \
                url + "\n\n" + \

            if str(old) == "1.00" or str(old) == "2.00":
                message = "<b>" + name + "</b>" + "\n\n" + \
                    "<b>" + str(new) + " TL\n\n" + "</b>" + \
                    url + "\n\n" + \

            token_list = [
                "xxxxxxxxxxxxxxxxxxxxxxx",
                "yyyyyyyyyyyyyyyyyyyyyyyy",
                "zzzzzzzzzzzzzzzzzzzzzzzzzzz",
                "aaaaaaaaaaaaaaaaaaaaaaaaaaaa",
                "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
                "ccccccccccccccccccccccccccccccccc",

            ]

            TOKEN = token_list[random.randint(0, len(token_list)-1)]

            chat_id = "-100xxxxxxxxxxxxx"
            bot = telegram.Bot(token=TOKEN)
            # tel_url = bot.sendMessage(chat_id = chat_id, text = message, parse_mode=ParseMode.HTML)

            try:
                bot.sendMessage(chat_id = chat_id, text = message, parse_mode=ParseMode.HTML)
                sleep(0.05)

            except Exception:

                return False
        cursor.close()
        return True
wgxvkvu9

wgxvkvu91#

最明显的方法似乎是将更新分成20个批次,并在更新之间休眠超过1秒:


# ...

notifications = cursor.fetchall()
cursor.close()

for i in range(0, len(notifications), 20):
    chunk = notifications[i:i+20]
    for notification in chunk:
        print(notification)
        # ...
    time.sleep(1.5)

相关问题