需要帮助决定什么时候限制“线程池线程数.net应用程序消费”是个好主意?

j13ufse2  于 2023-11-20  发布在  .NET
关注(0)|答案(2)|浏览(151)

我有一个HTTP客户端,它基本上是对HTTP服务器调用多个Web请求。我在一个线程池线程中执行每个HTTP请求(同步调用),默认情况下使用30个TCP(使用httpwebrequest.servicepoint -http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.servicepoint.aspx)。根据我管理的系统,可以有~500/1000个线程池线程等待I/O(http响应)。
现在,我想知道我是否也需要限制我使用的线程数?(例如,http://msdn.microsoft.com/en-us/library/ee789351(v=vs.110).aspxSystem.Threading.Tasks - Limit the number of concurrent Tasks

编辑

是的,我想我需要限制我使用的线程数,因为即使这些线程处于等待状态,它们也会占用资源。这样我就可以控制我使用的资源/线程数,这使得我的组件更容易与其他组件集成,而不会导致它们资源不足/争用资源/线程。

编辑二

我已经决定完全采用异步模型,这样我就不用使用线程池线程来执行http请求,相反,我可以简单地依赖于“OS内核和I/O完成端口线程的协作”,这将确保在完成时将在回调中发送响应(这样我就可以最好地利用CPU以及资源)。我目前正在考虑使用(webclient. uploaddataskasync)http://msdn.microsoft.com/en-us/library/system.net.webclient.uploaddatataskasync(v=vs.110).aspx,并相应地更新代码。(一些参考资料:HttpWebRequest and I/O completion portsHow does .NET make use of IO Threads or IO Completion Ports?

编辑3

基本上,我使用了“上面提到的异步网络I/O .net API”,这基本上消除了对我的并行库的使用。有关详细信息,请参阅下面的答案(为了方便起见,我添加了它,以防万一有人感兴趣!)。

  • 给予如何使用Web客户端调用Web请求的概念的伪代码 *
//psudeo code to represents there can be varibale number of requests
//these can be ~500 to ~1000
foreach(var request in requests)
{
    //psudeo code which basically executes webrequest in threadpool thread
    //MY QUESTION: Is it OK to create as many worker threads as number rrequests
    //and simply let them wait on a semaphore, on should i limit the concurrency?
    MyThreadPoolConcurrentLibrary.ExedcuteAction(() =>
        {            
            var sem = new Semaphore(initialCount: 50, maximumCount: 50.Value);
            try
            {
                //using semaphore as the HTTP Server which i am taking to recommend 
                //to send '50' parallel requests in '30' TCP Connections
                sem.WaitOne();
                //using my custom webclient, so that i can configure 'tcp' connections 
                //(servicepoint connection limit) and ssl validation etc.
                using (MyCustomWebClient client = new MyCustomWebClient())
                {
                    //http://msdn.microsoft.com/en-us/library/tdbbwh0a(v=vs.110).aspx
                    //basically the worker thread simply waits here
                    client.UploadData(address: "urladdress", data: bytesdata);
                }
            }
            finally
            {
                sem.Release(1);
            }
        });
}
MyThreadPoolConcurrentLibrary.WaitAll(/*...*/);

字符串
基本上,我应该做些什么来限制我消耗的线程数量,还是让线程池来处理它(即,如果我的应用达到线程池的最大线程限制,它会以任何方式将请求排队--这样我就可以完全依赖它了)

  • 伪代码,它应该显示我的自定义Web客户端,我在其中配置了tcp连接、ssl验证等**
class MyCustomWebClient : WebClient
{
    protected override WebRequest GetWebRequest(Uri address)
    {
        HttpWebRequest request = (HttpWebRequest)base.GetWebRequest(address);            
        request.KeepAlive = true;
        request.Timeout = 300;
        request.ServicePoint.ConnectionLimit = TCPConnectionsLimit;
        request.ServerCertificateValidationCallback = this.ServerCertificateValidationCallback;
        return request;

    }
    private bool ServerCertificateValidationCallback(object sender, System.Security.Cryptography.X509Certificates.X509Certificate certificate, System.Security.Cryptography.X509Certificates.X509Chain chain, System.Net.Security.SslPolicyErrors sslPolicyErrors)
    {
        throw new NotImplementedException();
    }
}


致以最诚挚的问候。

h7appiyu

h7appiyu1#

由于我正在执行网络I/O(http web requests),使用'synchronous' httpwebrequests并让线程池线程阻塞同步调用并不是一个好主意。(网络客户端的task方法)根据评论中的建议,在上面的问题中提到。它自动删除了我的组件中线程数量的使用-详细信息,请参阅下面的伪代码片段。

这里有一些有用的链接,可以帮助我轻松地适应C# 5.0的一些概念(cnc/await):

http://blog.stephencleary.com/2013/11/there-is-no-thread.html

class Program
{        
    static SemaphoreSlim s_sem = new SemaphoreSlim(90, 90);
    static List<Task> s_tasks = new List<Task>();   
    public static void Main()
    {                     
        for (int request = 1; request <= 1000; request++)
        {                             
            var task = FetchData();
            s_tasks.Add(task);                
        }
        Task.WaitAll(s_tasks.ToArray());
    }        
    private static async Task<string> FetchData()
    {
        try
        {
            s_sem.Wait();
            using (var wc = new MyCustomWebClient())
            {
                string content = await wc.DownloadStringTaskAsync(
                    new Uri("http://www.interact-sw.co.uk/oops/")).ConfigureAwait(continueOnCapturedContext: false);
                return content;
            }
        }
        finally
        {
            s_sem.Release(1);
        }
    }
    private class MyCustomWebClient : WebClient
    {
        protected override WebRequest GetWebRequest(Uri address)
        {
            var req = (HttpWebRequest)base.GetWebRequest(address);
            req.ServicePoint.ConnectionLimit = 30;
            return req;
        }
    }
}

字符串

uklbhaso

uklbhaso2#

你可以总是简单地把目标定在浏览器运行时的限制上,这样服务器管理员就不会太讨厌你了。
现在,RFC说,你应该限制连接到2 pr域,但根据http://www.stevesouders.com/blog/2008/03/20/roundup-on-parallel-connections/
许多浏览器达到6或8个并行连接(这是在2008年)。

Browser HTTP/1.1    HTTP/1.0
IE 6,7         2    4
IE 8           6    6
Firefox 2      2    8
Firefox 3      6    6
Safari 3,4     4    4
Chrome 1,2     6    ?
Chrome 3       4    4
Chrome 4+      6    ?
iPhone 2       4    ?
iPhone 3       6    ?
iPhone 4        4   ?
Opera 9.63,     4   4
Opera 10.51+    8   ?

字符串

相关问题