尝试使用Selenium和Python获取物品价格

zujrkrfu  于 2023-05-08  发布在  Python
关注(0)|答案(1)|浏览(167)

我想在jiomart卖一公斤洋葱,
然而,我不断得到一系列的错误,最后如下:
警告(来自警告模块):文件“C:\Users\Chandor\Desktop\Quotation.py”,line 9 options.headless = True #将headless模式设置为True以抑制浏览器窗口DeprecationWarning:headless属性已被弃用,请改用add_argument('--headless')或add_argument('--headless= new')Traceback(最后一次调用最近一次):文件“C:\Users\Chandor\Desktop\Quotation.py”,line 17,in pincode_box = driver.find_element_by_xpath(“//input[@id='areaPinCode']”)AttributeError:“WebDriver”对象没有属性“find_element_by_xpath”

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
import time

# set the driver path and start the browser
driver_path = "C:/chromedriver/chromedriver"
options = Options()
options.headless = True  # set headless mode to True to suppress the browser window
service = Service(executable_path=driver_path)
driver = webdriver.Chrome(service=service, options=options)

# navigate to jiomart homepage
driver.get("https://www.jiomart.com/")

# locate the search bar and search for onion
pincode_box = driver.find_element_by_xpath("//input[@id='areaPinCode']")
pincode_box.clear()
pincode_box.send_keys("RANDOM AREA PIN")
pincode_box.submit()
time.sleep(3) # wait for page to load

# locate the search bar and search for onion
search_bar = driver.find_element_by_xpath("//input[@id='searchbar']")
search_bar.clear()
search_bar.send_keys("onion 1 kg")
search_bar.submit()
time.sleep(3) # wait for page to load

try:
# locate the product div and get the price
product = driver.find_element_by_xpath("//div[@data-name='Onion 1 Kg']")
price = product.find_element_by_class_name("sp__dp").text
print(f"The price of Onion 1 Kg is {price}")
except:
print("The product was not found on JioMart")

# close the browser
driver.quit()

下面是我使用Beautifulsoup + requests的代码

import requests
from bs4 import BeautifulSoup

# Set the URL for the search query
pincode = "110006" # Replace with your desired pincode
url = f"https://www.jiomart.com/category/fruits-vegetables/onion-garlic-ginger/onion/490?location= 
{pincode}"

# Set the user agent header to avoid 403 Forbidden error
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like 
Gecko) Chrome/58.0.3029.110 Safari/537.36"}

# Send a GET request to the URL and retrieve the HTML response
response = requests.get(url, headers=headers)

# Parse the HTML response using BeautifulSoup
soup = BeautifulSoup(response.content, 'html.parser')

# Find the product div for 1 kg onion on the page
product = soup.find('div', {'data-name': 'Onion 1 Kg'})

# Check if the product div exists and extract the price if it does
if product:
# Extract the price of the product
price = product.find('div', {'class': 'final-price'}).text.strip()

# Print the price of 1 kg onion
print("The price of 1 kg onion on JioMart for pincode", pincode, "is Rs.", price)
else:
# Handle the case where the product is not found
print("The product was not found on JioMart for pincode", pincode)
print(response.content)
uplii1fm

uplii1fm1#

虽然这是可能的,你不应该使用 selenium 这样的东西。尝试使用beautifulsoup + requests之类的工具,或者使用scrapy之类的一体化工具。
你本质上是在进行网页抓取(从网站获取数据),你不需要为此打开整个浏览器示例。使用web抓取会更容易,更快,更有效,更酷。您还将学习更多关于解析数据和HTML结构的知识。
Selenium是一个浏览器自动化工具。例如,拍摄网页的图片,或对物品下订单。虽然这些事情也可以通过Webscraping来完成,但大多数时候使用浏览器自动化更容易。
现在,如果您对项目使用Selenium的问题束手无策,请先添加from selenium.webdriver.common.by import By,您将需要这个。然后加上这个elem = WebDriverWait(driver, 30).until( EC.presence_of_element_located((By.XPATH, "XPATH TO ELEMENT")) ),这是为了确保输入框被加载。然后执行search_bar = driver.find_by(By.XPATH, '//input[@id="areaPinCode")]'

相关问题