Connecting Storyteller to your System Edit on GitHub

Storyteller needs to be able to interract with your actual system under test. While Storyteller can be used to test against a running application that is initialized separately, most of the time you'll opt to have Storyteller be able to spin up your system on demand as part of running or editing specifications. Storyteller also needs to be able to shut your system down cleanly so that you can swiftly make code changes and rerun specifications.

Storyteller can bootstrap a system with one of two mechanisms:

1. In a separate AppDomain.
2. In a completely separate process launched by using the dotnet run command against the specification project

Storyteller 3.0 can only support the separate AppDomain mechanism. Storyteller 4.0 by default uses the separate process approach, but you can opt into using the AppDomain approach against a .Net 4.6 application with the --app-domain switch from the command line if you are running dotnet storyteller run or dotnet storyteller open.

In both cases, Storyteller uses the ISystem interface described in the next section to control the system under test's lifecycle.

Understanding the ISystem Interface

The hook in your code that Storyteller uses to govern this lifecycle is the ISystem interface shown below."

public interface ISystem : IDisposable
{
    CellHandling Start();

    IExecutionContext CreateContext();

    Task Warmup();
}

The ISystem interface only has four methods:

1. Start() : CellHandling -- this is a hook to do any kind of system bootstrapping or activation and an opportunity to put together any custom cell conversions and system wide selection lists. This method is only called once each time the testing AppDomain is loaded.
2. CreateContext() : IExecutionContext -- this method is executed as the first step in executing a specification. Use this method to perform any globally applicable state setup actions that will apply to all specifications.
3. Dispose() -- Clean up after yourself! This method is called when the Storyteller application is shut down and anytime Storyteller tries to recycle the system under test.
4. Warmup() : Task -- For systems that might need a little head start in bootstrapping, this method gives Storyteller a chance to "warm up" the system under test before you run any actual specifications

The IExecutionContext only lives for the lifetime of a specification execution and is disposed immediately after the specification runs. The Dispose() method is called even if the specification times out or aborts early with a critical exception.

public interface IExecutionContext : IDisposable
{
    // BeforeExecution() is a hook to potentially
    // set up state or do any kind of custom logging
    void BeforeExecution(ISpecContext context);
    
    // AfterExecution is a hook to gather up custom logging
    // or to make custom assertions against the specification
    // execution
    void AfterExecution(ISpecContext context);

    T GetService<T>();
}

Use the Dispose() method to do any kind of global state cleanup between specification runs. The BeforeExecution(ISpecContext) and AfterExecution(ISpecContext) methods can be used to do any kind of per specification set up, clean up, or even to log additional errors and failures for non-functional concerns like performance.

Writing a Custom ISystem for your Application

You can just implement the ISystem and IExecutionContext interfaces shown above, but as of Storyteller 4.1 you can also subclass the SimpleSystem class and override any of the methods shown below:

public class MySystem : SimpleSystem
{
    protected override void configureCellHandling(CellHandling handling)
    {
        // Allows you to apply customizations to CellHandling
    }

    public override void BeforeEach(SimpleExecutionContext execution, ISpecContext context)
    {
        // Executes before each specification run
    }

    public override void AfterEach(ISpecContext context)
    {
        // Executes after each specification run
        // Handy for state cleanup
    }

    public override Task Warmup()
    {
        // do any necessary bootstrapping work to get
        // your system ready to use
        return Task.CompletedTask;
    }

    public override void Dispose()
    {
        // any kind of system cleanup when Storyteller is shutting down
    }
}

Using the Separate Process Approach

AppDomain's were dropped from the CoreCLR and won't make a reappearance until Netstandard 2.0. Rather than wait for that release, Storyteller 4.0 adopted a new default strategy to run the system under test in a separate process triggered from the dotnet run command. To use this approach, you need to make your Storyteller project a console application, and in your Program.Main() entry point, delegate to the new StorytellerAgent class like this:

public static class Program
{
    public static int Main(string[] args)
    {
        // If you do not need a custom ISystem
        return StorytellerAgent.Run(args);
    }
}

If you had a custom ISystem for your application (named GrammarSystem in this case), bootstrapping your application would look like this:

public static class Program
{
    public static void Main(string[] args)
    {
        DateTime time = DateTime.MinValue;
        DateTime.TryParse("anything", out time);
        Console.WriteLine(time);

        DependencyContext.Default.RuntimeLibraries.Each(x =>
        {
            Console.WriteLine($"{x.Name}: {x.Version}");
        });


        // GrammarSystem is a custom ISystem
        // that "knows" how to bootstrap and
        // gracefully shut down the system under test
        StorytellerAgent.Run(args, new GrammarSystem());
    }
}

Storyteller is no longer able to "auto restart" the running system under test when binaries change. If you are relying on the separate process approach, your workflow is to just recycle or restart the system under test through the Storyteller explorer. There are keyboard shortcuts available to restart the application before rerunning a specification. Do remember that dotnet run will trigger a compilation if necessary, so you can make your changes to code files and immediately recycle Storyteller's system under test.

See The Application Shell for more information on how to trigger the system recycling.

Using the AppDomain Approach

Like most testing tools in .Net, the Storyteller application has to open a second .Net AppDomain in order to load the application assemblies where they are deployed. This second AppDomain uses shadow copying so that Storyteller can remain open as you recompile new changes in the system under test. Storyteller uses a file system watcher to watch for all changes to files with the .dll, .exe, or .config file extensions. When Storyteller detects changes to these files, Storyteller attempts to:

1. Gracefully shutdown the system under test
2. Unload the running AppDomain for the system under test
3. Spin up a new AppDomain to load the new system binaries
4. Start the system under test again

You can always explicitly tell Storyteller which ISystem class to use as a flag to the st run or st open commands. Otherwise, Storyteller uses these rules to determine the ISytem -- or punt.

1. If there are no concrete ISystem classes in any assembly in the AppDomain, Storyteller uses a default, nullo system and happily loads all of your Fixture classes.
2. If there is exactly one concrete ISystem class in the AppDomain, Storyteller uses that one.
3. If there is more than one concrete ISystem class in the AppDomain, Storyteller will throw an exception asking you to explicitly specify which one you want to use.

Custom Conversion Providers, Selection Lists, and Extensions

Custom conversion providers and system wide selection lists can be added in the ISystem.Start() method. The CellHandling class below exposes methods to add and configure lists and conversions:

public class CellHandling
{
    private readonly IList<IRuntimeConverter> _runtimeConvertors = new List<IRuntimeConverter>();

    public CellHandling(EquivalenceChecker equivalence, Conversions conversions)
    {
        Equivalence = equivalence;
        Conversions = conversions;
    }

    public IEnumerable<IRuntimeConverter> RuntimeConvertors => _runtimeConvertors;

    public void RegisterRuntimeConversion<T>() where T : IRuntimeConverter, new()
    {
        _runtimeConvertors.Add(new T());
    }

    public EquivalenceChecker Equivalence { get; private set; }

    public Conversions Conversions { get; private set; }

    public IList<IExtension> Extensions { get; } = new List<IExtension>();

    internal readonly LightweightCache<string, OptionList> Lists = new LightweightCache<string, OptionList>(key => new OptionList(key));

    public void AddSystemLevelList(string name, IEnumerable<string> values)
    {
        var list = new OptionList(name);
        list.AddValues(values.ToArray());

        Lists[name] = list;
    }

    public void AddSystemLevelList(string name, IEnumerable<Option> options)
    {
        var list = new OptionList(name);
        list.AddOptions(options.ToArray());

        Lists[name] = list;
    }

    public static CellHandling Basic()
    {
        return new CellHandling(new EquivalenceChecker(), new Conversions());
    }
}

To add extensions to your specification system, just add them to the CellHandling.Extensions list like so:

public class SeleniumSystem : SimpleSystem
{
    protected override void configureCellHandling(CellHandling handling)
    {
        handling.Extensions.Add(new SeleniumExtension(() => new ChromeDriver()));
    }
}