Experiments

In the previous section, we took a look at a complex experiment. Here it is again for convenience.

$experiment = (new Scientist\\Laboratory)
  ->experiment('experiment title')
  ->control($controlCallback)
  ->trial('first trial',  $firstTrialCallback)
  ->trial('second trial', $secondTrialCallback)
  ->trial('third trial',  $thirdTrialCallback)
  ->matcher(new Scientist\\Matcher\\StandardMatcher)
  ->chance(50);

$value = $experiment->run(3, 'apple', $panda, 42);

Let's step through this process method by method. That way, we can learn all about the methods that are used to define an experiment. We'll break the method chain apart for convenience.

Create Laboratory

$laboratory = new Scientist\Laboratory;

All experiments are created within a Laboratory. You can use the same Laboratory instance to define multiple experiments if you like. Just be sure to start with the ->experiment() method.

Experiment

$experiment = $laboratory->experiment('experiment title')

The experiment() method can be used on a Laboratory instance to create a new experiment. Its only parameter is a string, containing the name of the experiment we are about to define. The name serves as a helpful identifier when viewing an experiment results. It's a unique key for our experiment, and shouldn't be used more than once.

Control

$experiment->control(function () { ... });

The control() method is used to provide our control callable. It's a callback value that contains the original code. The code that we wish to improve upon. The return value from this callable will be the 'control value' for our experiment. Every trial will be compared to it.

Callables can be Closures, or an array representation of a callback. The following is equally acceptable.

$experiment->control([new MyClass, 'someMethod']);

For more information about valid callable references, please see the PHP documentation for callable.

Trial

$experiment->trial('trial name', function () { ... });

Trials are our experimental code. We can define as many trials as we like, by continuing to chain the trial() method.

The first parameter to the trial, is a string identifier for the trial, followed by a callable containing the trial code.

The signature of the trial callback must be compatible with that of the control. We are improving on the existing functionality, after all. This means that both your control and trial callbacks must accept the same parameters.

Matcher (optional)

$experiment->matcher(new Scientist\Matcher\StandardMatcher)

Matchers are used to define the way that a trial must match a control. By default, the experiment will use the StandardMatcher which performs a strict === comparison on the output of the trial and control callable. The result of the match will be send to all registered journals.

If the returned value of our control and trials is a little more complex, we can easily define our own matcher class, and provide an instance to the matcher() method to allow for more complex evaluation. You'll find more on this technique in the next chapter.

Chance (optional)

$experiment->chance(50);

The chance() method allows you to specify a % chance of experiment execution. Each time your experiment runs, it will execute the control, and every trial callback. These callbacks, while valuable, might be a little heavy on resources.

If you're running a busy web application, you may want to use the chance() method to execute the experiment only 10% (or any defined value) of the time.

Please note that the % value is an influenced random chance. There's a small chance you could get two executions, back to back, using a chance of 1%. It's a dice roll.

Ru

Run

$value = $experiment->run(3, 'apple', $panda, 42);

The run() method will execute our experiment. The value returned from run() will always be the value that is returned from the control callback. Without fail. This means, that the result of trials will never be used in actual application. It's purely there for matching.

You can optionally supply a number of parameters of any type to the run() method. These parameters will be supplied to the callables for both control and trials when the experiment is executed. This saves you having to 'pass them in' manually. Neat, right?!

Report

$report = $experiment->report(3, 'apple', $panda, 42);

The report() method can optionally be used in place of the run() method. Instead of returning the value of the control callable, it will instead return the Report instance, containing the results of your experiment. In the Reports chapter, you'll find information about how to interrogate a report.

Tip

If you've retrieved a report, but still want to use the result of the control callback, then you can retrieve it using $report->getControl()->getValue();