.net ConcurrentBag与List的性能比较

olqngx59  于 2023-02-06  发布在  .NET
关注(0)|答案(3)|浏览(488)

前言:我问这个问题只是因为我没有一个环境(足够大的数据集+计算能力)来以可靠的方式测试它。
问题:给定一个ConcurrentBag<T>,加载了数十亿个项目,由单个线程访问/使用,它的性能与List<T>相似吗?换句话说,ConcurrentBag<T>上的枚举比List<T>上的枚举性能更好还是更差?

pw9qyyiw

pw9qyyiw1#

ConcurrentBag<T>不可避免地会比List<T>性能差。虽然您将只从单个线程访问它,但该结构仍然需要适当的机制来防止出现并发访问时可能出现的争用危险。
如果您将在开始枚举之前 * 从单个线程加载集合,则可以使用ConcurrentBag(IEnumerable<T>)构造函数来避免性能开销,而不是通过其Add方法单独添加每个项。
ConcurrentBag<T>为枚举提供“即时快照”语义;请参阅GetEnumerator方法的备注。当您从foreach循环访问ConcurrentBag<T>时,它会首先将其全部内容复制到一个普通的List<T>中,然后在其上枚举。每次在循环中使用它时,这将导致大量的性能开销(计算和内存方面)。
如果您的场景是列表将由多个线程填充,但只由一个线程读取,那么您应该在编写器完成后立即将其转换为List<T>

yacmzcpb

yacmzcpb2#

数十亿的项目和列表或并发包?这是一个“不去”。
就性能而言,请尝试以下测试添加:(您可以随意修改以测试其他操作)

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace ConcurrentBagTest
{
    // You must compile this for x64 or you will get OutOfMemory exception
    class Program
    {
        static void Main(string[] args)
        {
            ListTest(10000000);
            ListTest(100000000);
            ListTest(1000000000);
            ConcurrentBagTest(10000000);
            ConcurrentBagTest(100000000);

            Console.ReadKey();

        }

        static void ConcurrentBagTest(long count)
        {
            try
            {
                var bag = new ConcurrentBag<long>();
                Console.WriteLine($"--- ConcurrentBagTest count = {count}");
                Console.WriteLine($"I will use {(count * sizeof(long)) / Math.Pow(1024, 2)} MiB of RAM");
                Stopwatch stopwatch = new Stopwatch();
                stopwatch.Start();
                for (long i = 0; i < count; i++)
                {
                    bag.Add(i);
                }
                stopwatch.Stop();
                Console.WriteLine($"Inserted {bag.LongCount()} items in {stopwatch.Elapsed.TotalSeconds} s");
                Console.WriteLine();
                Console.WriteLine();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
            GC.Collect();
            GC.WaitForPendingFinalizers();
        }

        static void ListTest(long count)
        {
            try
            {
                var list = new List<long>();
                Console.WriteLine($"--- ListTest count = {count}");
                Console.WriteLine($"I will use {(count * sizeof(long)) / Math.Pow(1024, 2)} MiB of RAM");
                Stopwatch stopwatch = new Stopwatch();
                stopwatch.Start();
                for (long i = 0; i < count; i++)
                {
                    list.Add(i);
                }
                stopwatch.Stop();
                Console.WriteLine($"Inserted {list.LongCount()} items in {stopwatch.Elapsed.TotalSeconds} s");
                Console.WriteLine();
                Console.WriteLine();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
            GC.Collect();
            GC.WaitForPendingFinalizers();
        }
    }
}

我的输出:

--- ListTest count = 10000000
I will use 76,2939453125 MiB of RAM
Inserted 10000000 items in 0,0807315 s

--- ListTest count = 100000000
I will use 762,939453125 MiB of RAM
Inserted 100000000 items in 0,7741546 s

--- ListTest count = 1000000000
I will use 7629,39453125 MiB of RAM
System.OutOfMemoryException: Array dimensions exceeded supported range.

--- ConcurrentBagTest count = 10000000
I will use 76,2939453125 MiB of RAM
Inserted 10000000 items in 1,0744069 s

--- ConcurrentBagTest count = 100000000
I will use 762,939453125 MiB of RAM
Inserted 100000000 items in 11,3976436 s

使用的CPU:英特尔酷睿i7- 2600@3.4 GHz,
使用内存:16 GB
还可以查看this answer的限制。

xzv2uavs

xzv2uavs3#

但是,如果您需要删除项目,ConcurrentBag比List要快得多

void Main()
{
    ConcurrentBag<int> bag = new ConcurrentBag<int>();
    ConcurrentStack<int> stack = new ConcurrentStack<int>();
    ConcurrentQueue<int> q = new ConcurrentQueue<int>();
    List<int> list = new List<int>();

    Stopwatch sw = new Stopwatch();
    int count = 100000;
    sw.Start();
    for (int i = 0; i < count; i++)
    {
        bag.Add(i);
    }
    for (int i = 0; i< count; i++)
    {
        bag.TryTake(out _);
    }
    sw.Elapsed.Dump("BAG");
    sw.Start();
    for (int i = 0; i < count; i++)
    {
        stack.Push(i);
    }
    for (int i = 0; i < count; i++)
    {
        stack.TryPop(out _);
    }
    sw.Elapsed.Dump("Stack");
    sw.Start();
    for (int i = 0; i < count; i++)
    {
        q.Enqueue(i);
    }
    for (int i = 0; i < count; i++)
    {
        q.TryDequeue(out _);
    }
    sw.Elapsed.Dump("Q");
    sw.Start();
    for (int i = 0; i < count; i++)
    {
        list.Add(i);
    }
    for (int i = 0; i < count; i++)
    {
        list.RemoveAt(0);
    }
    sw.Elapsed.Dump("list remove at 0");
    sw.Start();
    for (int i = 0; i < count; i++)
    {
        list.Add(i);
    }
    for (int i = 0; i < count; i++)
    {
        list.RemoveAt(list.Count -1);
    }
    sw.Elapsed.Dump("list remove at end");
}
    • 结果:**
    • 袋子**00:00:00.0144421
    • 堆栈**00:00:00.0341379
    • Q**00时00分00秒0400114
    • 列表删除时间为0**00:00:00.6188329
    • 在结束时删除列表**00:00:00.6202170

相关问题