postman 解决响应测试期间的Forbidden 403错误

liwlm1x9  于 2023-11-18  发布在  Postman
关注(0)|答案(1)|浏览(432)

我试图得到网站https://clutch.co/sitemap.xml的响应
我在Python和Postman中测试过,它显示了一个禁止的403状态码。我复制了它的所有头部,但它仍然没有得到成功的响应。
下面是我尝试的一段代码片段。

import requests

headers = {
  'authority': 'clutch.co',
  'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8',
  'accept-language': 'en-US,en;q=0.6',
  'cache-control': 'max-age=0',
  'cookie': 'cf_clearance=mvAUZW1nJug5PB0M8rf93YBwyn4qOdDcU5gy.vVZ6rc-1698494124-0-1-1c669b3c.347dd517.85f5c18c-0.2.1698494124; _ga=GA1.1.1809619058.1698494124; FPID=FPID2.2.pSduR8wDRmNkf3s5Z4HEhBZ18leJVTEtvWiCgKds9tw%3D.1698494124; FPLC=uF%2BpXHT1%2BvYReCd4BPS47IQeM5yO9mB8uwXh95ALGacLDIy4dW%2BByDOe2DeSGiYTq43YZp4EfAPJiXyUnKMJBD52uvoiJDauKzt3M4%2FKxXBfzLunSznieaoZ68xCiA%3D%3D; __cf_bm=tv3SekyYvPpYLykgxBnrp2LtXvkG5Wvh4LFQVrV2_Ks-1698497124-0-AQFsc9n3lMN5iKh+93PrwD1KgyBDNpofby6pZlu3n+02f1WPA84GsHf2Ym6wJARjQ5kokSaz9d9TNsIbP2qMkXQ=; _ga_D0WFGX8X3V=GS1.1.1698494124.1.1.1698497403.56.0.0',
  'if-modified-since': 'Sat, 28 Oct 2023 09:03:15 GMT',
  'sec-ch-ua': '"Brave";v="117", "Not;A=Brand";v="8", "Chromium";v="117"',
  'sec-ch-ua-mobile': '?0',
  'sec-ch-ua-platform': '"Windows"',
  'sec-fetch-dest': 'document',
  'sec-fetch-mode': 'navigate',
  'sec-fetch-site': 'none',
  'sec-fetch-user': '?1',
  'sec-gpc': '1',
  'upgrade-insecure-requests': '1',
  'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36'
}

response = requests.get('https://clutch.co/sitemap.xml', headers=headers)

字符串
我该怎么解决呢?提前谢谢大家。

bgtovc5b

bgtovc5b1#

该网站正在使用cloudflare作为防止抓取。一个解决方案是复制您的User-Agent HTTP头和__cf_bm cookie来获取结果。但要注意,cookie经常更改!

import requests

url = "https://clutch.co/sitemap.xml"

headers = {
    "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0"
}

cookies = {
    "__cf_bm": "ap_wV732pImoyM_3yvD6TJ7YngLHt6L5xQ0Dm8Qqi50-1698525930-0-ASLsB7CJ9rzp67z8CTqKm6fnmvExRKdmwdWt12XrWcc/J7vfy7GKNvVAP+b0G4Fnxqfcluj1VAHhMVZbeo1KSnU="
}

t = requests.get(url, headers=headers, cookies=cookies).text
print(t)

字符串
打印:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd"><sitemap><loc>https://clutch.co/sitemap-directory-1.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-directory-2.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-directory-3.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-directory-4.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-directory-5.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-directory-6.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-directory-7.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-directory-8.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-directory-9.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-1.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-2.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-3.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-4.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-5.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-6.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-7.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-8.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-9.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-10.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-11.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-12.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-13.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-14.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-15.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-16.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-17.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-18.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-19.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-20.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-21.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-22.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-23.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-24.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-25.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-26.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-27.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-28.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-29.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-30.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-profile-31.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-standalone.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-articles.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-packages.xml</loc></sitemap><sitemap><loc>https://clutch.co/sitemap-pricing.xml</loc></sitemap></sitemapindex>

相关问题