Should FAKE have a Parallel target?

[MorganPersson]

After a discussion on twitter with @forki about parallel targets it may be worth discussing it a bit more.

• Where would it help / make sense to use parallel targets?
• Logging
  • If each task just logs stuff it will probably be (very) confusing reading the log
  • Buffer all logs until all parallel targets finished
  • other options?
• What happens when one target fails?

[mausch]

+1 for parallel targets. For example you could build docs and run tests at the same time. I also have a project with multiple test assemblies, I could run those in parallel as well (provided the test runner supports it, e.g. Gallio does not).

Some build systems with parallel execution: sbt, make -j. There's also an extension for MSBuild to do this. I don't have any first-hand experience with them but I think the questions about logging, failures, etc will be answered by taking a good look at these.

[forki]

Regarding the tests: Something like http://fsharp.github.io/FAKE/apidocs/fake-nunitparallel.html exists and it is probably much easier to handle this on task level.

[mausch]

Cool, didn't know about NUnitParallel! However it seems a little ad-hoc... if I wanted the same thing for xUnit, MbUnit, etc, I'd have to create specific tasks for each one. I agree it's simpler to implement this at the task level, but ultimately, I'd love to see FAKE run targets in parallel by default unless there are dependencies or the user says otherwise explicitly, as in Shake or sbt.

[t0yv0]

Just posted some thoughts on how I am planning to do this in FShake here. It should be easy/natural given the proper abstraction. I specifically discuss error handling and logging.

[vasily-kirichenko]

I think we need parallel both targets and tasks. For the former it may be declared as special dependencies like "this three targets can run in parallel" ("t1" <=> "t2" <=> "t3" or something like that). For the latter it may be like this:

Target "NUnit tests" {
    !+ @".\**\bin\**\*TestLib*.dll"
    |> parallel
    |> NUnit (fun p ->
          { p with ErrorLevel =  DontFailBuild })
}

In this case we'll have to change signatures of the tasks such as they get a string (file path) as the last argument instead of seq<string>.

[blacktaxi]

Any updates on this? +1 to parallel targets.

[krauthaufen]

i implemented a global option to specify the number of threads used for building (when possible according to the dependencies)
see pull request: #676

In general this option may cause problems and obviously obfuscates the log somehow but we were able to reduce build times for an internal project (about 150 MSBuild calls) by a factor of about 4

[dsyme]

Address sufficiently by #676