我有socks 5代理-〉socks 5://登录名:密码@ip:端口
我累坏了:
class ProxyMiddleware:
def process_request(self, request, spider):
# Set the location of the proxy
request.meta['proxy'] = "http://ip:port"
proxy_user_pass = "login:password"
encoded_user_pass = base64.encodestring(proxy_user_pass)
request.headers['Proxy-Authorization'] = 'Basic ' + encoded_user_pass
中间件:
DOWNLOADER_MIDDLEWARES = {
'bog_name.middlewares.ProxyMiddleware': 100,
}
Scrapy不使用代理
1条答案
按热度按时间nzk0hqpo1#
看起来scrappy不支持socks 5代理,下面是关于它的issue on Github,以及用户在这个主题中提出的两个解决方案-一,二