我正在尝试分析此网站：

https://www.monster.com/jobs/search/?q=java&where=usa&stpage=1

在本质上，它并不复杂：它是一个单页应用程序，你给它关键字，单击搜索，然后它显示结果-它开始时只显示大约29个结果。向下滚动时，会加载新结果。
在加载新结果之前，它会向

https://www.monster.com/jobs/search/pagination/?q=java&where=usa&isDynamicPage=true&isMKPagination=true&page=2&total=26

会导致 JSON 回复，这是一个作业列表，看起来有点像这样：

{"Title":"Java Developer","TitleLink":"https://job-openings.monster.com/java-developer-orlando-fl-us-summit-technologies/215193478","DatePostedText":"6 days ago","DatePosted":"2020-01-18T12:00","LocationText":"Orlando, FL, 32801","JobViewUrl":"https://job-openings.monster.com/java-developer-orlando-fl-us-summit-technologies/215193478","ImpressionTracking":"data-m_impr_uuid=\"a7320356-70db-46ca-908e-e540f0e74cec\" data-m_impr_a_placement_id=\"JSR2CW\" data-m_impr_s_t=\"t\" data-m_impr_j_p=\"27\" data-m_impr_j_jpm=\"1\" data-m_impr_j_lat=\"28.5418\" data-m_impr_j_long=\"-81.3736\" data-m_impr_j_jawsid=\"418397617\" data-m_impr_j_postingid=\"b55f4409-3858-483a-a2e9-65e254ec1cd2\" data-m_impr_j_jobid=\"215193478\" data-m_impr_j_cid=\"660\" data-m_impr_j_occid=\"11970\" data-m_impr_j_lid=\"385\" data-m_impr_j_jpt=\"1\" data-m_impr_j_pvc=\"monster\" data-m_impr_j_coc=\"xsummittechx\" ","Company":{"Name":"Summit Technologies","HasCompanyAddress":true,"LogoLink":""},"Text":"Java Developer","ApplyType":"ApplyOnline","IsAggregated":"false","JobViewUrlMeta":"https://job-openings.monster.com/java-developer-orlando-fl-us-summit-technologies/215193478","MusangKingId":"215193478","CompanyLogoUrl":"","PrivateBoardIconImageUrl":"","FitIcon":"","FitIconType":""}

另一个post请求被发送到

https://ib.adnxs.com/ut/v3

（v3请求）：

价值何在 14162549 的 tag_id: 14162549 似乎是从上面的get请求中提取的。
所以当你向下滚动时，它会发送1个get和1个post请求，直到它没有发送为止-滚动结束，请求也会结束：

我不明白它是如何决定何时停止的。
我想把那些工作凑齐，然后我可以做一些事情，比如送礼物给你 https://www.monster.com/jobs/search/pagination/?q=java&where=usa&isDynamicPage=true&isMKPagination=true&page=N 但我不知道什么时候停止，因为如果说，它停止滚动时 &page=12 ，如果我向 &page=13 ，它不会返回空的json，相反，它会显示一些其他作业（可能相关性较低，因此在滚动到底部时不可见）。
我用 okHttp 发送请求，如下所示：

HttpUrl.Builder urlBuilder = HttpUrl.parse(getUrl()).newBuilder();
urlBuilder.addQueryParameter("page", "1");
String url = urlBuilder.build().toString();

Request request = new Request.Builder()
        .url(url)
        .addHeader("Content-Type", "application/json; charset=utf-8")
        .addHeader("Accept-Language", Locale.US.getLanguage())
        .build();

OkHttpClient client = new OkHttpClient();
Call call = client.newCall(request);
Response response = call.execute();
String responseBody = response.body().string();
System.out.println(responseBody);

Gson gson = new Gson();
List<MonsterJobJson> resultMonster = gson.fromJson(
        responseBody, new TypeToken<List<MonsterJobJson>>() {
        }.getType());