我是新手,我想从url中提取html。我决定尝试使用Selenium,并绘制了一个服务,看起来像这样:
package dataox.keithgellmanrestaurantmenuscraping;
import lombok.RequiredArgsConstructor;
import lombok.extern.log4j.Log4j2;
import org.apache.commons.lang3.RandomUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.stereotype.Service;
import java.util.List;
@Log4j2
@Service
@RequiredArgsConstructor
public class ParserService {
private final ChromeOptions options = new ChromeOptions();
@EventListener(ApplicationReadyEvent.class)
public void a() {
String url = "some url";
WebDriver webDriver = getWebDriver(url);
log.info(webDriver.getPageSource());
quitWebdriver(webDriver);
}
private WebDriver getWebDriver(String url) {
WebDriver webDriver = new ChromeDriver(options);
webDriver.get(url);
return webDriver;
}
private void quitWebdriver(WebDriver webDriver) {
webDriver.close();
webDriver.quit();
}
}
当我启动它,我得到了以下错误的时刻“WebDriver webDriver =新的ChromeDriver(选项)":
java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method
HandlerMethod details:
Bean [dataox.keithgellmanrestaurantmenuscraping.ParserService]
Method [public void dataox.keithgellmanrestaurantmenuscraping.ParserService.a()]
Resolved arguments:
Caused by: java.lang.NoClassDefFoundError: org/openqa/selenium/virtualauthenticator/HasVirtualAuthenticator
我在Gradle工作,我的gradle.build看起来像这样:
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
implementation group: 'org.apache.commons', name: 'commons-lang3', version: '3.12.0'
implementation group: 'org.seleniumhq.selenium', name: 'selenium-api', version: '3.141.59'
implementation group: 'org.seleniumhq.selenium', name: 'selenium-chrome-driver', version: '3.141.59'
implementation group: 'org.jsoup', name: 'jsoup', version: '1.15.3'
compileOnly 'org.projectlombok:lombok'
developmentOnly 'org.springframework.boot:spring-boot-devtools'
annotationProcessor 'org.projectlombok:lombok'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
在这种情况下,最好的做法是什么?
1条答案
按热度按时间aij0ehis1#
我自己找到了答案。事实证明,Gradle完全删除版本就足够了。Spring拉取那些与工作相关的版本。