Select Page

A ConcurrentDictionary can safely store values while several workers use it, yet a counter built from a separate read and write can still lose increments. This tutorial forces that failure, replaces the counter operation with AddOrUpdate, and demonstrates why its update delegate must tolerate being called again.

The examples were compiled and run as .NET 10 console programs in an isolated Linux container. They use integer values and one key. The counter test deliberately excludes removal, reset, and overflow; it makes no throughput claim.

Identify the operation you need to protect

Thread safety of individual accesses does not make a sequence of accesses one indivisible action. Microsoft documents the indexer setter as an unconditional overwrite and provides dedicated methods for conditional changes. Its ConcurrentDictionary remarks are a useful method-selection reference.

Intent Operation to consider
Insert only if the key is absent TryAdd; inspect its Boolean result.
Replace only if the current value equals your expected value TryUpdate; handle a false result.
Add a counter or calculate its next value AddOrUpdate with a side-effect-free update delegate.

These choices express a single-key condition. A business rule spanning several keys needs a shared synchronization or transaction design covering all participating accesses. Replacing the collection alone does not supply that design.

Reproduce the lost update, then verify the fix

Imagine two requests recording a hit. Both read zero, both compute one, and both store one. Each access succeeds, but the stored total is short by one. The first half of this program makes both reads finish before either write, so the failure does not depend on a lucky scheduling race or a sleep.

Create a .NET 10 console project, then replace its entire Program.cs with the following program. The same setup works for the second example in a separate project.

dotnet new console --name CounterDemo --framework net10.0
using System.Collections.Concurrent;

using var deadline = new CancellationTokenSource(TimeSpan.FromSeconds(20));
CancellationToken token = deadline.Token;

for (int round = 0; round < 50; round++)
{
    var counts = new ConcurrentDictionary<string, int>();
    counts["hits"] = 0;
    var bothRead = new TaskCompletionSource(
        TaskCreationOptions.RunContinuationsAsynchronously);
    int readers = 0;

    async Task BrokenIncrement()
    {
        int before = counts["hits"];
        if (Interlocked.Increment(ref readers) == 2)
            bothRead.TrySetResult();
        await bothRead.Task.WaitAsync(token);
        counts["hits"] = before + 1;
    }

    Task[] broken = [BrokenIncrement(), BrokenIncrement()];
    try
    {
        await Task.WhenAll(broken);
    }
    finally
    {
        bothRead.TrySetCanceled();
        await Task.WhenAll(broken);
    }
    if (counts["hits"] != 1)
        throw new InvalidOperationException("Lost-update reproduction changed.");

    // Start with an absent key to exercise both addition and updates.
    counts.Clear();
    var start = new TaskCompletionSource(
        TaskCreationOptions.RunContinuationsAsynchronously);
    Task[] fixedWorkers = Enumerable.Range(0, 8).Select(_ => Task.Run(async () =>
    {
        await start.Task.WaitAsync(token);
        for (int i = 0; i < 100; i++)
        {
            token.ThrowIfCancellationRequested();
            counts.AddOrUpdate("hits", 1, static (_, old) => checked(old + 1));
        }
    })).ToArray();
    try
    {
        start.TrySetResult();
        await Task.WhenAll(fixedWorkers);
    }
    finally
    {
        start.TrySetCanceled();
        await Task.WhenAll(fixedWorkers);
    }
    if (counts["hits"] != 800)
        throw new InvalidOperationException("Counter lost an update.");
}

using var canceled = new CancellationTokenSource();
canceled.Cancel();
var untouched = new ConcurrentDictionary<string, int>();
try
{
    canceled.Token.ThrowIfCancellationRequested();
    untouched.AddOrUpdate("hits", 1, static (_, old) => checked(old + 1));
    throw new InvalidOperationException("Cancellation was ignored.");
}
catch (OperationCanceledException) when (canceled.IsCancellationRequested)
{
    if (!untouched.IsEmpty)
        throw new InvalidOperationException("Canceled work changed the counter.");
}

Console.WriteLine("50 rounds: two split increments stored 1, not 2");
Console.WriteLine("50 rounds: 800 AddOrUpdate increments stored 800");
Console.WriteLine("Pre-canceled work made no update; all workers drained");

Run it after replacing Program.cs:

dotnet run --project CounterDemo

Expected output, asserted exactly in the isolated test:

50 rounds: two split increments stored 1, not 2
50 rounds: 800 AddOrUpdate increments stored 800
Pre-canceled work made no update; all workers drained

The broken phase saves the old value before its gate opens. After the second reader arrives, both workers write that old value plus one. Repeating this interleaving proves the bug without trying to measure its frequency in production.

The corrected phase starts with an absent key. Eight workers each record 100 hits. The add value records the first hit; the update delegate calculates the next count from the value it receives. The AddOrUpdate documentation describes these two paths and warns that delegates can execute repeatedly. Returning a number is safe to repeat here; sending an invoice from the delegate would require a different design.

The test runs 50 corrected rounds and asserts 800 after every round. The main cancellation token bounds gate waits and is checked before each corrected increment. A separate pre-canceled case verifies that the caller’s cancellation check prevents an update. The AddOrUpdate call does not accept the token: cancellation after the check can still leave that increment committed. Both phases await every worker before moving on, including on failure. The finally blocks repeat that wait defensively; WhenAll already waits for all workers.

Demonstrate a repeated update delegate

Counting successful method calls and counting delegate invocations are different measurements. The next program seeds the key at zero, starts two dedicated workers, and holds their first update callbacks at a barrier. Both callbacks receive zero before either can return. Once released, one proposal becomes stale and the update delegate runs again in the tested runtime.

The callback counter below is deliberate test instrumentation, not a pattern for updating business state. Only each worker’s first callback waits at the barrier, so a retry does not wait for a partner that has already finished.

using System.Collections.Concurrent;

using var deadline = new CancellationTokenSource(TimeSpan.FromSeconds(20));
CancellationToken token = deadline.Token;

for (int round = 0; round < 25; round++)
{
    var counts = new ConcurrentDictionary<string, int>();
    counts["hits"] = 0;
    using var firstCalls = new Barrier(2);
    int callbackCalls = 0;

    Task Worker() => Task.Factory.StartNew(() =>
    {
        bool first = true;
        counts.AddOrUpdate("hits", 1, (_, old) =>
        {
            // Instrumentation only: this counts attempts, not committed updates.
            Interlocked.Increment(ref callbackCalls);
            if (first)
            {
                first = false;
                if (!firstCalls.SignalAndWait(TimeSpan.FromSeconds(5), token))
                    throw new TimeoutException("Second callback did not arrive.");
            }
            return checked(old + 1);
        });
    }, token, TaskCreationOptions.LongRunning, TaskScheduler.Default);

    // Dedicated workers avoid blocking thread-pool threads on the test barrier.
    Task[] workers = [Worker(), Worker()];
    await Task.WhenAll(workers);
    if (workers.Any(task => !task.IsCompletedSuccessfully))
        throw new InvalidOperationException("Worker cleanup failed.");
    if (counts["hits"] != 2 || callbackCalls != 3)
        throw new InvalidOperationException("Expected two updates and three attempts.");
}

Console.WriteLine("25 rounds: two updates stored 2; update delegate ran 3 times");
Console.WriteLine("All workers completed before each barrier was disposed");

Expected output:

25 rounds: two updates stored 2; update delegate ran 3 times
All workers completed before each barrier was disposed

The barrier wait has a five-second timeout plus a shared 20-second cancellation deadline. Dedicated workers keep this intentionally blocking test off the thread pool. Each round waits for both workers before disposing the barrier. The assertions check the final value, callback count, and successful worker completion.

Three callbacks is the observation asserted for this controlled .NET 10 run, not an API promise about every workload. Microsoft’s AddOrUpdate remarks state that delegates run outside dictionary locks and can execute multiple times. A successful dictionary update does not make a delegate’s external side effects part of the same atomic action.

Apply the result to your code

  • For a numeric counter like this one, calculate the replacement from the delegate’s supplied value and keep the delegate free of business side effects.
  • Do not copy an old value into a closure and increment that instead; the retry needs to calculate from its new input.
  • Do not count factory calls as committed changes. The class remarks also exclude GetOrAdd’s factory delegate from the dictionary operation’s atomicity guarantee.
  • If other code removes or resets a counter, define what a hit means across that change before adopting the example’s total-count assertion.

For background on typed collections, read Introduction to Generic Collections. For a batch where workers can fail independently, see Task.WhenAll: retain successful results and report every failure.

Share This