Note: phiên bản Tiếng Việt của bài này ở link dưới.

https://duongnt.com/autofixture-create-vie

When writing unit tests in C#, I frequently use AutoFixture to generate test data. However, we need to keep in mind that the data it creates is not truly random. Today, we will see how the default settings of AutoFixture generate data with bool, enum, and int type.

AutoFixture generated data is not really random

We will use AutoFixture to generate three lists of type bool, enum, and int.

public enum Season
{
    Spring,
    Summer,
    Fall,
    Winter
}

var fixture = new Fixture();
var bools = fixture.CreateMany<bool>(5)
var seasons = fixture.CreateMany<Season>(4);
var numbers = fixture.CreateMany<int>(255);

Below are the results.

bools.ToList();
# true
# false
# true
# false
# true

seasons.ToList();
# Spring
# Summer
# Fall
# Winter

It’s easy to see that the elements in bools just alternate between true and false. And the elements in seasons cycle through all values in Season.

It’s harder to see a pattern in numbers. At first glance, the generated numbers seem to be random. But the code below might surprise you.

var numberSet = numbers.ToHashSet();
Console.WriteLine($"Count: {numberSet.Count}. Max: {numberSet.Max()}. Min:{numberSet.Min()}");
# Count: 255. Max: 255. Min: 1

Again, it’s pretty unlikely that 255 randomly generated integers would fall into the [1, 255] range without any duplicates.

At its core, the CreateMany method simply calls the Create method of the generic type in a loop. We would still get the same result above if we repeatedly called Create instead of using CreateMany.

The BooleanSwitch class

With the default settings, AutoFixture uses the BooleanSwitch class to generate bool data. This is its Create method.

lock (this.syncRoot)
{
    this.b = !this.b;
    return this.b;
}

And this.b is simply a private field.

private bool b;

Every time we call the Create method, the BooleanSwitch class negates and returns the value of b. It even has a lock to make sure that two consecutive values are alternating, even if they are generated on different threads.

The EnumGenerator class

For the enum type, we need to check the EnumGenerator class. This is its Create method. After performing some type checks, it calls the private method CreateValue.

var generator = this.EnsureGenerator(t);
generator.MoveNext();
return generator.Current;

And how does EnsureGenerator create a generator?

IEnumerator enumerator = null;
if (!this.enumerators.TryGetValue(t, out enumerator))
{
    enumerator = new RoundRobinEnumEnumerable(t).GetEnumerator();
    this.enumerators.Add(t, enumerator);
}
return enumerator;

We can see that EnsureGenerator initializes an enumerator called RoundRobinEnumEnumerable. And everytime that enumerator is exhausted, it is recreated. The implementation of RoundRobinEnumEnumerable is straightforward.

this.values = Enum.GetValues(enumType).Cast<object>();

// ... omitted

public IEnumerator GetEnumerator()
{
    while (true)
    {
        foreach (var obj in this.values)
        {
            yield return obj;
        }
    }
}

We can verify that the generated enum values will cycle from the first value to the last value, then back to the first one.

The RandomNumericSequenceGenerator class

Use the Create method to generate an integer

For the int type, we need to check this Create method. If you follow its execution chain, you can see that it calls the GetNextRandom method and converts the result back to int.

GetNextRandom uses System.Random to generate a random number. And that’s not all. This method actually checks the generated number against a HashSet called numbers to make sure that it doesn’t return duplicated numbers.

do
{
    if (this.lower >= int.MinValue &&
        this.upper <= int.MaxValue)
    {
        result = this.random.Next((int)this.lower, (int)this.upper);
    }
    else
    {
        result = this.GetNextInt64InRange();
    }
}
while (this.numbers.Contains(result));

this.numbers.Add(result);

If result was not generated before, GetNextRandom adds it to the HashSet then returns it.

this.numbers.Add(result);
return result;

How does AutoFixture determine the lower and upper bounds?

The range of generated numbers is set inside the EvaluateRange method.

if (this.count == (this.upper - this.lower))
{
    this.count = 0;
    this.CreateRange();
}

this.count++;

We can see that AutoFixture will generate all numbers within a range before it tries the next range. For example, it will generate all 255 numbers between [1, 255] before moving on. And when it needs to create a new range, it calls the CreateRange method.

By default, limits array is [1, 255, 32767, 2147483647 ]. When first initialized, this code path will be executed.

this.lower = this.limits[0];
this.upper = this.GetUpperRangeFromLimits();

private long GetUpperRangeFromLimits()
{
    return this.limits[1] >= int.MaxValue
            ? this.limits[1]
            : this.limits[1] + 1;
}

This means initially lower == 1 and upper == 256. Remember that the upper range of System.Random.Next is exclusive. At first, AutoFixture generates each number in the [1, 255] range once. This is consistent with the result of our test.

What if we generate more than 255 integer numbers?

In this case, we will enter this code path.

this.lower = this.upper;
this.upper = remaining.Min() + 1;

Consider that remaining == this.limits.Where(x => x > this.upper - 1).ToArray(), we have the result below.

lower == 256
upper == 32768

This time, AutoFixture returns each number in the [256, 32767] once.

Similarly, after that AutoFixture will return each number in the [32768, 2147483647] range once.

What if AutoFixture exhausts all positive integer numbers?

Now, the remaining array will be empty. We return to this code path.

if (remaining.Any() && this.numbers.Any())
{
    this.lower = this.upper;
    this.upper = remaining.Min() + 1;
}

Clearly, AutoFixture will cycle back to the [1, 255] range. This is possible because at the end of the CreateRange method, it clears all values inside the HashSet.

this.numbers.Clear();

Recap

• First, AutoFixture generates all numbers in the [1, 255] range. Each number is returned just once.
• Then it moves to the [256, 32767] range, then the [32768, 2147483647] range. Each number is still only returned once.
• Then it cycles back to the [1, 255] range.

This algorithm has an unwanted effect when we generate a huge number of integers. Let’s say we want to generate exactly 2,147,483,647 numbers. When we create the last one, we have to repeatedly call System.Random.Next many, many times until we receive the last non-duplicated number. Each call has a 1/(2147483647 - 32768) == 4.65*10e-10 chance of success.

Conclusion

We can configure AutoFixture to generate random data. But with the default settings, the generated data for some types is not random at all. Personally, I have written bad unit tests in the past because I didn’t understand this quirk.