天天看點

ConcurrentDictionary字典操作竟然不全是線程安全的?

好久不見,馬甲哥封閉居家半個月,記錄之前遇到的一件小事。

标題不準确,實際上ConcurrentDictionary<TKey,TValue>絕大部分api都是線程安全且原子性的[1],

唯二的例外是接收工廠函數的api:

AddOrUpdate

GetOrAdd

,這兩個api不是原子性的,需要引起重視。

All these operations are atomic and are thread-safe with regards to all other operations on the ConcurrentDictionary<TKey,TValue> class. The only exceptions are the methods that accept a delegate, that is, AddOrUpdate and GetOrAdd.

之前有個同僚就因為這個case背了一個P。

AddOrUpdate(TKey, TValue, Func<TKey,TValue,TValue> valueFactory);

GetOrAdd(TKey key, Func<TKey, TValue> valueFactory);(注意,包括其他接收工廠委托的重載函數)

整個過程中涉及與字典直接互動的都用到精細鎖,valueFactory工廠函數在鎖定區外面被執行,是以,這些代碼不受原子性限制。

Q1: valueFactory工廠函數不在鎖定範圍,為什麼不在鎖範圍?

A: 還不是因為微軟不相信你能寫出健壯的業務代碼,未知的業務代碼可能造成死鎖。

However, delegates for these methods are called outside the locks to avoid the problems that can arise from executing unknown code under a lock. Therefore, the code executed by these delegates is not subject to the atomicity of the operation.

Q2:帶來的效果?

  • • valueFactory工廠函數可能會多次執行
  • • 雖然會多次執行, 但插入的值固定是一個,插入的值取決于哪個線程率先插入字典。

    Q3: 怎麼做到的随機穩定輸出一列值?

    A:源代碼做了double check[2]了,後續線程通過工廠類建立值後,會再次檢查字典,發現已有值,會丢棄自己建立的值。

示例代碼:

using System.Collections.Concurrent;

public class Program
{
private static int _runCount = 0;
private static readonly ConcurrentDictionary<string, string> _dictionary
= new ConcurrentDictionary<string, string>;

public static void Main(string[] args)
{
var task1 = Task.Run( => PrintValue("The first value"));
var task2 = Task.Run( => PrintValue("The second value"));
var task3 = Task.Run( => PrintValue("The three value"));
var task4 = Task.Run( => PrintValue("The four value"));
Task.WaitAll(task1, task2, task4,task4);

PrintValue("The five value");
Console.WriteLine($"Run count: {_runCount}");
}

public static void PrintValue(string valueToPrint)
{
var valueFound = _dictionary.GetOrAdd("key",
x =>
{
Interlocked.Increment(ref _runCount);
Thread.Sleep(100);
return valueToPrint;
});
Console.WriteLine(valueFound);
}
}           

上面4個線程并發插入字典,每次随機輸出,

_runCount=4

顯示工廠類執行4次。

ConcurrentDictionary字典操作竟然不全是線程安全的?

Q4:如果工廠産值的代價很大,不允許多次建立,如何實作?

筆者的同僚之前就遇到這樣的問題,高并發請求頻繁建立redis連接配接,直接打挂了機器。

A: 有一個trick能解決這個問題: valueFactory工廠函數傳回Lazy容器

using System.Collections.Concurrent;

public class Program
{
private static int _runCount2 = 0;
private static readonly ConcurrentDictionary<string, Lazy<string>> _lazyDictionary
= new ConcurrentDictionary<string, Lazy<string>>;

public static void Main(string[] args)
{
task1 = Task.Run( => PrintValueLazy("The first value"));
task2 = Task.Run( => PrintValueLazy("The second value"));
task3 = Task.Run( => PrintValueLazy("The three value"));
task4 = Task.Run( => PrintValueLazy("The four value"));
Task.WaitAll(task1, task2, task4, task4);

PrintValue("The five value");
Console.WriteLine($"Run count: {_runCount2}");
}

public static void PrintValueLazy(string valueToPrint)
{
var valueFound = _lazyDictionary.GetOrAdd("key",
x => new Lazy<string>(
 =>
{
Interlocked.Increment(ref _runCount2);
Thread.Sleep(100);
return valueToPrint;
}));
Console.WriteLine(valueFound.Value);
}
}           
ConcurrentDictionary字典操作竟然不全是線程安全的?

上面示例,依舊會随機穩定輸出,但是

_runOut=1

表明産值動作隻執行了一次、

valueFactory工廠函數傳回Lazy

① 工廠函數依舊沒進入鎖定過程,會多次執行;

② 與最上面的例子類似,隻會插入一個Lazy容器(後續線程依舊做double check發現字典key已經有Lazy容器了,會放棄插入);

③ 線程執行Lazy

④ 多個線程嘗試執行Lazy

ExecutionAndPublication:

不僅以線程安全的方式執行, 而且確定隻會執行一次構造函數。

public Lazy(Func<T> valueFactory)
:this(valueFactory, LazyThreadSafetyMode.ExecutionAndPublication, useDefaultConstructor: false)
{
}           
控制構造函數執行的枚舉值 描述
ExecutionAndPublication[3] 能確定隻有一個線程能夠以線程安全方式執行構造函數
None 線程不安全
Publication 并發線程都會執行初始化函數,以先完成初始化的值為準

IHttpClientFactory

在建構<命名HttpClient,活躍連接配接Handler>字典時, 也用到了這個技巧,大家自行欣賞DefaultHttpCLientFactory源碼[4]。
  • • https://andrewlock.net/making-getoradd-on-concurrentdictionary-thread-safe-using-lazy/

總結

為解決ConcurrentDictionary GetOrAdd(key, valueFactory) 工廠函數在并發場景下被多次執行的問題:

① valueFactory工廠函數産生Lazy容器;

② 将Lazy容器的值初始化姿勢設定為

ExecutionAndPublication

(線程安全且執行一次)。

兩姿勢缺一不可。

本人會不時修正了解、更正錯誤,請适時移步左下角永久更新位址;也請看客大膽斧正。

引用連結

[1]

ConcurrentDictionary<TKey,TValue>絕大部分api都是線程安全的:https://docs.microsoft.com/en-us/dotnet/api/system.collections.concurrent.concurrentdictionary-2?view=net-6.0

[2]

double check:https://github.com/dotnet/runtime/blob/main/src/libraries/System.Collections.Concurrent/src/System/Collections/Concurrent/ConcurrentDictionary.cs#L1152

[3]

ExecutionAndPublication:https://docs.microsoft.com/en-us/dotnet/api/system.threading.lazythreadsafetymode?view=net-6.0#system-threading-lazythreadsafetymode-executionandpublication

[4]

DefaultHttpCLientFactory源碼:https://github.com/dotnet/runtime/blob/main/src/libraries/Microsoft.Extensions.Http/src/DefaultHttpClientFactory.cs#L118

繼續閱讀