python 对表进行Web擦除

vs3odd8k 于 2023-03-21 发布在 Python

关注(0)|答案(2)|浏览(107)

我试图在此链接https://www.cellmapper.net/arfcn?net=LTE&ARFCN=78&MCC=0中获取表值。每次我需要更改网络类型（LTE或3G或2G）时，我都会更改ARFCN值，并且我想从表结果中获取“Band Number”。我无法通过此代码读取结果表的所有数据。
我的代码：

from lxml.html import parse
from urllib.request import urlopen
import requests
import pandas as pd
from bs4 import BeautifulSoup

def htmltodf(url):
    page = requests.get(url)
    soup = BeautifulSoup(page.text,features="lxml")
    tables = soup.findAll('table')
    test = pd.io.html.read_html(str(tables))
    print(test)

htmltodf("https://www.cellmapper.net/arfcn?net=LTE&ARFCN=78&MCC=0")

我得到的结果是：都是NAN

[                                       Result  Result.1
0                                Network Type       NaN 
1                                   E/U/ARFCN       NaN 
2                                   Band Name       NaN 
3    Uplink Frequency (phone to base station)       NaN 
4  Downlink Frequency (base station to phone)       NaN 
5                                 **Band Number       NaN** 
6                         Possible Bandwidths       NaN 
7                                Sector Color       NaN]

python

来源：https://stackoverflow.com/questions/75371378/web-scraping-a-table

2条答案

按热度按时间

eulz3vhy1#

尝试此版本：

import pandas as pd
import requests
from itertools import product

url = 'https://api.cellmapper.net/v6/getFrequency'
channel = [70, 71]
rat = ['LTE', 'UMTS', 'GSM']

data = {}
for vals in product(channel, rat):
    payload = dict(zip(['Channel', 'RAT'], vals))
    r = requests.get(url, params=payload).json()
    if r['statusCode'] == 'OKAY':
        sr = pd.Series(r['responseData'])
        data[vals] = sr
df = pd.concat(data, names=['Channel', 'RAT'], axis=1).T.reset_index()

输出：

>>> df
   Channel  RAT netType Frequency startFrequencyDLNumber endFrequencyDLNumber bandName bandNumber rxFrequency txFrequency modulation region MCC possibleBandwidth
0       70  LTE     LTE        70                      1                  599      IMT          1      1927.0      2117.0        FDD  WORLD   0   [5, 10, 15, 20]
1       70  GSM     GSM        70                      0                  124  GSM 900        900       904.0       949.0       TDMA  WORLD   0             [0.2]
2       71  LTE     LTE        71                      1                  599      IMT          1      1927.1      2117.1        FDD  WORLD   0   [5, 10, 15, 20]
3       71  GSM     GSM        71                      0                  124  GSM 900        900       904.2       949.2       TDMA  WORLD   0             [0.2]

赞(0）回复(0）举报 2023-03-21

vh0rcniy2#

考虑看看这个。
https://www.cellmapper.net/TOS
Cellmapper是一个社区驱动的项目，请不要滥用它。

赞(0）回复(0）举报 2023-03-21