使用mysql连接器python将值插入数据库,web抓取

dm7nw8vv  于 2021-07-26  发布在  Java
关注(0)|答案(2)|浏览(349)

这个代码应该从网站上找到前20个耳机,然后得到产品的名称和价格。然后将它们插入数据库(到“project”表中)。

import requests
from bs4 import BeautifulSoup
import mysql.connector

cnx=mysql.connector.connect(user="root",password='test',host='127.0.0.1',database='amirdb')
cursor=cnx.cursor()
response=requests.get(f"https://www.digikala.com/search/?q=headphone")
soup=BeautifulSoup(response.text,"html.parser")

container=soup.find("ul",{'class':'c-listing__items'})
item=container.find_all("div",{'class':'c-product-box__title'})
price=container.find_all("div",{'class':'c-price__value-wrapper'})

for num in range(20):
    cursor.execute('INSERT INTO project VALUES(%s,%s)',(item[num].text,price[num].text))
cnx.commit()
cnx.close()

当我想将它们插入到我的数据库(到“project”表中)时,它就不起作用了,我知道问题出在代码的这一部分:

for num in range(20):
        cursor.execute('INSERT INTO project VALUES(%s,%s)',(item[num].text,price[num].text))

上面这部分代码有什么问题?

表的结构:

+----------+--------------+------+-----+---------+-------+
| Field    | Type         | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
| filename | varchar(100) | YES  |     | NULL    |       |
| price    | varchar(100) | YES  |     | NULL    |       |
+----------+--------------+------+-----+---------+-------+

我有以下错误:

Traceback (most recent call last):
  File "C:/Users/Administratör/PycharmProjects/untitled/venv/seventh.py", line 13, in <module>
    cursor.execute('INSERT INTO project VALUES(%s,%s)',(item[num].text,price[num].text))
  File "C:\Users\Administratör\PycharmProjects\untitled\venv\lib\site-packages\mysql\connector\cursor.py", line 551, in execute
    self._handle_result(self._connection.cmd_query(stmt))
  File "C:\Users\Administratör\PycharmProjects\untitled\venv\lib\site-packages\mysql\connector\connection.py", line 490, in cmd_query
    result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
  File "C:\Users\Administratör\PycharmProjects\untitled\venv\lib\site-packages\mysql\connector\connection.py", line 395, in _handle_result
    raise errors.get_exception(packet)
mysql.connector.errors.DatabaseError: 1366 (HY000): Incorrect string value: '\xD9\x87\xD8\xAF\xD9\x81...' for column 'filename' at row 1
6ovsh4lw

6ovsh4lw1#

是数据库编码引起的,修改数据库编码。

ALTER TABLE project CONVERT TO CHARACTER SET utf8mb4;

试试这个。

cursor.execute('INSERT INTO project VALUES(%s,%s)',(item[num].text,price[num].text))

如果不是,则发布表项目的结构。

xytpbqjk

xytpbqjk2#

试试下面这样的。我想会有用的。

for container in soup.find_all(class_='c-product-box')[:20]:
    item_title = container.find(class_='c-product-box__title').get_text(strip=True)
    item_price = container.find(class_='c-price__value-wrapper').get_text(strip=True)
    cursor.execute('INSERT INTO project (item_header,price_header) VALUES (%s,%s)',(item_title,item_price))

cnx.commit()
cnx.close()

相关问题