Verifying Sets Edit on GitHub

One of the most powerful features of Storyteller is the ability to declaratively compare a set of expected results against the system state and report back:

1. Which expected items were matched to the actual results
2. Which expected items were missing from the actual results
3. Items in the actual results that were unexpected
4. Optionally, verify an exact order of the items between the expected items and the actual values

Arrays

The tabular forms of set verifications shown below are more powerful and detailed in how they display data, but a much more terse form is to simply assert against a .Net Array type just like you'd assert against other values.

Below is an example Fixture that contains a pair of Sentence grammars to verify a string array and an integer array:

public class NameArrayFixture : Fixture
{
    public NameArrayFixture()
    {
        Title = "Comparing arrays";
    }

    [FormatAs("The array of names should be {names}")]
    public string[] TheNameArrayShouldBe()
    {
        return new string[]{"Han", "Luke", "Chewie"};
    }

    [FormatAs("The first 4 numbers in the Fibonacci sequence should be {numbers}")]
    public int[] FibonacciSeries()
    {
        return new int[] {1, 1, 2, 3};
    }
}

In usage, the results of a specification using the fixture above will look like this:

Some things to note:

1. The equivalence check is order specific. The two arrays have to be the same length and contain the same elements in the same order
2. Only Array's are supported at this time
3. Storyteller knows how to convert a string into an array of some type by treating the raw string as a comma delimited string, then converting the raw string values into the element type.
4. You can verify an array of any .Net type that Storyteller knows how to convert or parse from a string -- including your own application types
5. The results are shown using the ToString() method of the inner element type. Do consider using custom ToString() methods to provide more information in the case of specification failures.

See Data Conversion within Specifications for more information on how the conversion process works.

String Lists

If you'd rather express an expected set in rows, you can use the VerifyStringList() mechanism in Fixture classes to expose a special Set Verification for sets of strings.

The code below shows how to use this syntax to create a grammar that will verify a set of string data:

public class NameListFixture : Fixture
{
    private readonly List<string> _names = new List<string>();

    public NameListFixture()
    {
        Title = "Comparing a set of names";
    }

    public IGrammar TheNamesShouldBe()
    {
        return VerifyStringList(() => _names)
            .Titled("The names in order should be", "Name")
            .Ordered();
    }

    // The two grammars below are just to set up the name list
    [Hidden]
    public void AddName(string name)
    {
        _names.Add(name);
    }

    public IGrammar TheNamesAre()
    {
        return this["AddName"]
            .AsTable("The data is")
            .Before(() => _names.Clear());
    }


}

Note that set verifications can explicitly choose to enforce the exact order of the results, or do an unordered comparison by default.

In usage, the string list verification behaves like this sample specification below:

Object Sets

The VerifySetOf() mechanism on Fixture classes enables you to write grammars to express the qualities of an expected set of data by testing individual values of the data items in the actual result.

Let's say that you're building an invoicing system of some kind and you have a simple class to represent details like this (the very earliest version of Storyteller was originally conceived during a project on legal invoicing software that used the older FitNesse tool):

public class InvoiceDetail
{
    public double Amount { get; set; }
    public DateTime Date { get; set; }
    public string Name { get; set; }
    public string Part { get; set; }
}

You could write some simple set comparisons with against that InvoiceDetail object with grammars like this code sample below:

public class SetsFixture : Fixture
{
    public IGrammar UnorderedDetailsAre()
    {
        return VerifySetOf(() => _details)
            .Titled("The unordered details should be")

            // Use this shorthand syntax if you don't care about
            // cell defaults or overriding the table headers
            // in the html
            .MatchOn(o => o.Amount, o => o.Date, o => o.Name);
    }

    public IGrammar OrderedDetailsAre()
    {
        return VerifySetOf(() => _details)
            .Titled("The Ordered details should be")
            .Ordered()

            // Use this syntax if you want to customize
            // the cells in this SetVerification grammar
            .Comparisons(_ =>
            {
                _.Compare(o => o.Amount).DefaultValue("100").Header("The Amount");
                _.Compare(o => o.Date);
                _.Compare(o => o.Name).Header("The Name");
            });
    }

    private readonly List<InvoiceDetail> _details = new List<InvoiceDetail>();

    // This grammar is just used to set up the actual results
    // in the specification
    public IGrammar InvoiceDetailsAre()
    {
        return CreateNewObject<InvoiceDetail>("", x =>
        {
            x.SetProperty(o => o.Amount);
            x.SetProperty(o => o.Date);
            x.SetProperty(o => o.Name);
            x.Do(d => _details.Add(d));
        })
        .AsTable("The InvoiceDetails are")
        .Before(() => _details.Clear());
    }
}

The grammars above in action will render html results like so:

Tips for using Set Verifications Successfully

1. Since the world doesn't always conform to simple object structures, consider using intermediate types in your Set Verification grammars just to translate the actual system data to something that's easier to consume inside of Storyteller.
2. Do take advantage of Storyteller's ability to customize string conversions so that you can specify more complex types in Set Verification columns.
3. The Compare(x => [expression]) and MatchOn(x => [expression], x => [expression]) syntax can support following multiple properties and methods. SoCompare(x => x.Detail.Name)` is valid syntax that can be used inside of Storyteller.

Data Tables and Relational Databases

Set verification can also be done against a .Net DataTable so that you can easily and declaratively verify the expected results of a database table, view, or query. This feature was built specifically to ease testing against relational databases.

To set up a set verification against a DataTable, use the Fixture.VerifyDataTable() method as shown below: