winforms WebClient挂起直到超时

yhuiod9q  于 2022-11-16  发布在  其他
关注(0)|答案(2)|浏览(293)

我尝试使用WebClient下载网页,但它挂起,直到达到WebClient中的超时,然后失败并出现异常。
以下代码将不起作用

WebClient client = new WebClient();
string url = "https://www.nasdaq.com/de/symbol/aapl/dividend-history";
string page = client.DownloadString(url);

如果使用不同的URL,则传输工作正常。

WebClient client = new WebClient();
string url = "https://www.ariva.de/apple-aktie";
string page = client.DownloadString(url);

完成得非常快,并在页面变量中包含整个html.
使用HttpClient或WebRequest/WebResponse在第一个URL上得到相同的结果:阻塞,直到出现超时异常。
这两个URL在浏览器中都能很好地加载,大约需要2-5秒钟。知道问题出在哪里吗,有什么解决方案吗?
我注意到,在Windows窗体对话框中使用WebBrowser控件时,第一个URL加载时会出现20多个javascript错误,需要单击确认。当在浏览器中打开开发人员工具访问第一个URL时,也会出现同样的情况。
然而,WebClient不会对它得到的返回进行操作。它不运行JavaScript,也不加载引用的图片、CSS或其他脚本,所以这应该不是问题。
谢谢你!
拉尔夫

fjaof16o

fjaof16o1#

第一个站点"https://www.nasdaq.com/de/symbol/aapl/dividend-history";需要:

此处的User-agent很重要。如果在WebRequest.UserAgent中指定了最新的User-agent,网站可能会激活Http 2.0协议和HSTSHTTP Strict Transport Security)。只有最新的浏览器(作为参考,FireFox 56或更高版本)才支持/理解这些协议。
必须使用较旧的浏览器作为User-agent,否则网站将期待(并等待)* 动态 * 响应。使用 * 较旧的 * User-agent,网站将激活Http 1.1协议,而不会激活HSTS。
第二个站点"https://www.ariva.de/apple-aktie";需要:

  • ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12
  • 不需要服务器证书验证
  • 不需要特定的用户代理

我建议这样设置WebRequest(或相应的HttpClient设置):
(WebClient * 可以 * 工作,但可能需要派生的自定义控件)

private async void button1_Click(object sender, EventArgs e)
{
    button1.Enabled = false;
    Uri uri = new Uri("https://www.nasdaq.com/de/symbol/aapl/dividend-history");
    string destinationFile = "[Some Local File]";
    await HTTPDownload(uri, destinationFile);
    button1.Enabled = true;
}

CookieContainer httpCookieJar = new CookieContainer();

//The 32bit IE11 header is the User-agent used here
public async Task HTTPDownload(Uri resourceURI, string filePath)
{
    // Windows 7 may require to explicitly set the Protocol
    ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
    // Only blindly accept the Server certificates if you know and trust the source
    ServicePointManager.ServerCertificateValidationCallback += (s, cert, ch, sec) => { return true; };
    ServicePointManager.DefaultConnectionLimit = 50;

    var httpRequest = WebRequest.CreateHttp(resourceURI);

    try
    {
        httpRequest.CookieContainer = httpCookieJar;
        httpRequest.Timeout = (int)TimeSpan.FromSeconds(15).TotalMilliseconds;
        httpRequest.AllowAutoRedirect = true;
        httpRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
        httpRequest.ServicePoint.Expect100Continue = false;
        httpRequest.UserAgent = "Mozilla / 5.0(Windows NT 6.1; WOW32; Trident / 7.0; rv: 11.0) like Gecko";
        httpRequest.Accept = "ext/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
        httpRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate;q=0.8");
        httpRequest.Headers.Add(HttpRequestHeader.CacheControl, "no-cache");

        using (var httpResponse = (HttpWebResponse)await httpRequest.GetResponseAsync())
        using (var responseStream = httpResponse.GetResponseStream())
        {
            if (httpResponse.StatusCode == HttpStatusCode.OK) {
                try {
                    int buffersize = 132072;
                    using (var fileStream = File.Create(filePath, buffersize, FileOptions.Asynchronous)) {
                        int read;
                        byte[] buffer = new byte[buffersize];
                        while ((read = await responseStream.ReadAsync(buffer, 0, buffer.Length)) > 0)
                        {
                            await fileStream.WriteAsync(buffer, 0, read);
                        }
                    };
                }
                catch (DirectoryNotFoundException) { /* Log or throw */}
                catch (PathTooLongException) { /* Log or throw */}
                catch (IOException) { /* Log or throw */}
            }
        };
    }
    catch (WebException) { /* Log and message */} 
    catch (Exception) { /* Log and message */}
}

第一个WebSite(nasdaq.com)返回的负载长度为101.562字节
第二个WebSite(www.ariva.de)返回的负载长度为56.919字节

7fhtutme

7fhtutme2#

很明显,下载该链接有问题(不正确的url,unothorized访问,...),但是您可以使用异步方法来解决socking部分:

WebClient client = new WebClient();
  client.DownloadStringCompleted += (s, e) =>
  {
       //here deal with downloaded file
  };
  client.DownloadStringAsync(url);

相关问题