loading...
Cover image for So random!

So random!

fluffynuts profile image Davyd McColl ・7 min read

Random values can be really useful for testing and can be really simple to generate with the help of PeanutButter.RandomGenerators. But before I start showing off what you can do with this library, let's ask the pivotal question:

Why generate random values?

I'm a fan of using random values in testing because:

  • it adds a bit of a "fuzzing" component to my testing
    • no two test runs are exactly the same, so sometimes I get a failure that I didn't expect and it shows me a boundary condition that I hadn't thought of
  • it communicates when the actual values being used aren't that important - rather the reader's focus should go to the underlying logic
  • I spend less time trying to come up with values for properties and more time on my test and production logic
  • with a helpful library, I have to write less code like this:
[Test]
public void MyFancyTest()
{
  var person = new Person()
  {
    FirstName = "Carl",
    LastName = "Sagan",
    BirthDate = new DateTime(1934, 11, 9),
    FavoriteColor = "Space Grey",
    Gender = Genders.Male,
    HeightInMeters = 1.80M,
    ShoeSize = 9.5,
    Subject = "Astrophysics",
    Transport = "Bicycle",
    // and so on...
  };

  // ...
}

and, when it really doesn't matter exactly what attributes my testing person has, I can rather write this:

[Test]
public void AnotherTest()
{
  var person = GetRandom<Person>();

  // ... 
}

I especially write this kind of code when the actual values don't matter as much as the fact that I have a fully filled-in object for my order, customer, person, or whatever. It's especially useful for writing simple tests around storing and retrieving data. Often I have a test where only one field on an object really matters for the test, eg:

[Test]
public void ShouldDoSomethingWhenOrderIsCollect()
{
  var order = GetRandom<Order>();
  order.IsCollect = true;
}

When there are a few things I intend to change on an object for the purposes of testing, I might rather fall back on the builder pattern:

[Test]
public void ShouldDoSomethingWhenOrderIsForPizzaDeliveryBefore10am()
{
  var order = OrderBuilder.Create()
                // set up all props randomly
                .WithRandomProps()
                // now override with the specifics for this test
                .WithOrderItem(GetRandom<Pizza>())
                .AsDeliveryOrder()
                .WithDeliveryAt(
                  DateTime.Now.Date.AddHours(9)
                ).Build();
}

Enter PeanutButter

PeanutButter is a collection of useful libraries that I started working on a few years ago. It gets updated whenever I have a new itch to scratch (happens often enough), especially when I figure out a useful pattern for testing such as temporary databases for LocalDb or MySql, or temporary HTTP Servers for testing code making WebRequest calls.

The component I'm introducing to you today is RandomGenerators

Usage

Typically, I use a static import for RandomValueGen:

using static PeanutButter.RandomGenerators.RandomValueGen;

Now, from within test code, I can easily get random primitive values:

// either use specifically-named variants:
var intVal = GetRandomInt();
var stringVal = GetRandomString();
var dateVal = GetRandomDate();
var flag = GetRandomBoolean();
// and many others...

// or generic variants
var intVal2 = GetRandom<int>();
var longVal = GetRandom<long>();
var stringVal2 = GetRandom<string>()
var enumValue = GetRandomEnum<MyEnum>();

// all generators can take optional parameters
// eg if I want a random integer between 18 and 80 inclusive:
var age = GetRandomInt(18, 80);
// if I need a really long string:
var longString = GetRandomString(1024, 2048);
// perhaps I need a large faked data buffer:
var buffer = GetRandomBytes(4096, 8192);
// get a collection of strings, with between 4 and 12 items inclusive:
var strings = GetRandomCollection<string>(4, 12);
var jumbled = strings.Randomize(); // gets a new collection in random order

For more examples, check out the test code on GitHub:

I use these a lot, but, when I first wrote these methods, I found myself doing this kind of thing a lot:

var person = new Person()
{
  Id = 42,
  FirstName = GetRandomString(),
  LastName = GetRandomString(),
  BirthDate = GetRandomDate(),
  FavoriteColor = GetRandomFrom(new[] { "Red", "Blue", "Green" }),
  Gender = GetRandom<Genders>(),
  HeightInMeters = GetRandomDecimal(1.5M, 2,1M),
  ShoeSize = GetRandomFloat(6, 12),
  Subject = GetRandomString(),
  Transport = GetRandomString(),
  // and so on...
};

So I was thinking less about unimportant setup, but I was still writing a lot of it.

Enter GenericBuilder

I came up with a generic base builder that could be used to build practically anything:

public class PersonBuilder: GenericBuilder<PersonBuilder, Person>
{
}

[TestFixture]
public class TestPersonRepository
{
  [Test]
  public void ShouldBeAbleToPersistAndRecall()
  {
    var person = PersonBuilder.Create()
                   .WithRandomProps()
                   .WithProp(o => o.FavoriteColor =
                     GetRandomFrom(new[] { "Red", "Blue", "Green" })
                   ).WithProp(o => o.Height = 1.8M)
                   .Build()
  }
}

Note that a derivative of GenericBuilder requires two type arguments:

  • the type of the derivative builder class
    • this is used to be able to make fluent methods like WithProp return the proper builder type
  • the type of the entity being built
    • this is required to:
      • set up randomising code
      • instantiate a new instance of the object
        • types without a parameterless constructor can often be built by generating random values for constructor parameters, but you are also free to override ConstructEntity and build however you like
        • interfaces are also supported to some degree:
          • if a type implementing that interface is found in loaded assemblies, you'll get one of those
          • if NSubstitute is found to be loaded you'll get a substitute
          • if PeanutButter.DuckTyping is loaded, the type generator from that library will be used to generate a new type implementing that interface
          • if none of these conditions is met, you'll get a message that you need to override ConstructEntity() in your builder to proceed.

This builder saves transforms to be applied to the object it will be asked to build with each invocation of .Build(). Transforms are run in order, so the last one to run wins. This is how I can ask for .WithRandomProps(), then proceed to override the props that I'm interestg in for this specific test.

Of course, I encourage well-named methods, so this would invariably be refactored to something like:

public class PersonBuilder: GenericBuilder<PersonBuilder, Person>
{
  public PersonBuilder WithRandomFavoriteColor()
  {
    return WithProp(o => 
             o.FavoriteColor =
                GetRandomFrom(new[] { "Red", "Blue", "Green" })
             );
  }

  public PersonBuilder WithHeight(decimal height)
  {
    return WithProp(o => o.Height = 1.8M)
  }
}

[TestFixture]
public class TestPersonRepository
{
  [Test]
  public void ShouldBeAbleToPersistAndRecall()
  {
    var person = PersonBuilder.Create()
                   .WithRandomProps()
                   .WithRandomFavoriteColor()
                   .WithHeight(1.8M)
                   .Build()
  }
}

Which was better to read in my tests and easier to write against. However, I really wanted to be able to do something like:

var person = GetRandom<Person>();

So I learned about type generation

For quite some time now, anyone has been able to do code like the last line above, because of some smarts in RandomValueGen.GetRandom<T>():

  • if T is a primitive (or primitive-like, eg DateTime), you get a random value
  • if T is more complex (struct or class), then we need a GenericBuilder<TBuilder, Person> to do the hard work for us, so:
    • GetRandom first searches the current calling assembly for a builder -- if it finds one, it uses that one
    • then the search is expanded to all loaded assemblies. If a matching builder is found, that's used
    • finally, if no existing builder is found one will be generated for you and cached for further use.

The last strategy was an interesting journey in discovering how to create new types based on existing generic ones. If you're interested in that kind of stuff, go have a look at the source

Whilst it's really convenient that there's a path which will generate a builder for you, I find that I often want to guide random generation just a little, which is why the first two strategies are of interest.

Guiding random generation

Let's imagine that we have a Person class as above, and we'd like to generate a random one, within some constraints. We can:

  • create our own builder
  • override WithRandomProps to tack on alterations that make our end result better suited to our needs:
public class PersonBuilder: GenericBuilder<PersonBuilder, Person>
{
  public override PersonBuilder WithRandomProps()
  {
    return base.WithRandomProps()
             .WithRandomFavoriteColor()
             .WithHeight(GetRandomFloat(1.5, 2.1));
  }

  public PersonBuilder WithRandomFavoriteColor()
  {
    return WithProp(o => 
             o.FavoriteColor =
                GetRandomFrom(new[] { "Red", "Blue", "Green" })
             );
  }

  public PersonBuilder WithHeight(decimal height)
  {
    return WithProp(o => o.Height = 1.8M)
  }
}

Now we can just do this in our tests:

var person = GetRandom<Person>();

and that person object will:

  • have a favorite color that is one of "Red", "Green" or "Blue"
  • have a height between 1.5 and 2.1 meters.

Obviously, you can get as specific as you need to.

There are also attributes to guide with common requirements. For example, I might want to enforce that an Id property never be zero:

[RequireNonZero(nameof(Person.Id))]
public class PersonBuilder: GenericBuilder<PersonBuilder, Person>
{
}

Or guide the randomizer not to randomize a specific property (this can help with generated / scaffolding code, eg if you're using a library like LLBLGEN or some T4 generator which has it's own specially-generated properties):

[NoRandomize(nameof(Person.FavoriteColor))]
public class PersonBuilder: GenericBuilder<PersonBuilder, Person>
{
}

or perhaps you just need to be 100% sure that the same value won't appear for the same property on two different objects:

[RequireUnique(nameof(Person.Age))]
public class PersonBuilder: GenericBuilder<PersonBuilder, Person>
{
}

But wait, there's more!

GetRandom<T> not only knows how to make random primitives, objects and collections

  • it will also fill in child properties which are complex types (eg child nodes in a tree) up to a max depth of 10 by default (to prevent stack-overflow), though you can change this depth to suit your needs.
  • it can also fill in collection properties, though that isn't enabled by default as determining proper parent-child relationships can be tough. However, if you override WithRandomProps to include a call to WithFilledCollections, you'll get filled, randomized collection properties.

I hope this is useful for someone out there. I also hope to introduce a few of the other components of the PeanutButter suite at some point -- probably PeanutButter.DuckTyping, since I started a series on duck-typing which I need to round out a bit.

Thanks for your time. Please feel free to ask any questions (:

Posted on by:

Discussion

pic
Editor guide
 

If you test with random values, wouldn't that make your tests flaky?
They might pass with one value and fail with an other one. And in my experience if the developer sees this kind of random failures she/he would just rerun the test to make CI green. And then when that happens again and again the risk is that code with failed tests will be merged because "we know this tests sometimes fail, so we can merge it"

 

That's a good question, similar to concerns I've heard before!

If the values that are important for your test are randomised, then of course your test may flip (unless your test dies some kind of "double accounting, calculating a result via a different method).

The point here is that in a test like "when a customer places a delivery order before a certain time, she should get a discount", then the other data points shouldn't matter. If a test flops because of a random data value that shouldn't have mattered, then it helps to find a problen with the production code. I've experienced this exact scenario - of course debugging it requires running the test until failure with lots of logging, but it's saved me in the long run.