18: Upgrade runner to meet version 2 spec

[cdimitroulas]

Closes #18

The solution here relies on a custom hspec formatter which deals with producing the results.json file. In order to make this work, it uses code injection to modify the exercise test code to use this custom formatter.

Outstanding items

• At the moment, the new bin/run.sh executes this code injection by using stack runghc. We may wish to avoid this by building an executable and executing that directly.

[github-actions]

Hello. Thanks for opening a PR on Exercism 🙂

We ask that all changes to Exercism are discussed on our Community Forum before being opened on GitHub. To enforce this, we automatically close all PRs that are submitted. That doesn't mean your PR is rejected but that we want the initial discussion about it to happen on our forum where a wide range of key contributors across the Exercism ecosystem can weigh in.

You can use this link to copy this into a new topic on the forum. If we decide the PR is appropriate, we'll reopen it and continue with it, so please don't delete your local branch.

If you're interested in learning more about this auto-responder, please read this blog post.

---

Note: If this PR has been pre-approved, please link back to this PR on the forum thread and a maintainer or staff member will reopen it.

[ErikSchierboom]

Great work! I've left some comments.

[ErikSchierboom]

@cdimitroulas You'll also need to fix the tests. You can run them locally by running: ./bin/run-tests-in-docker.sh

[ErikSchierboom]

I'm doing some preliminary performance testing locally, and it does look like this new version is a bit slower. Hopefully pre-compiling will help.

[ErikSchierboom]

Whilst testing things locally, I'm getting this error:

Error: [S-305]
Failed to generate a Cabal file using the Hpack library on file:
/mnt/exercism-iteration/package.yaml

The error encountered was:

/mnt/exercism-iteration/package.yaml: Error while parsing $.tests.test.dependencies[1] - invalid dependency "hspec      - aeson"

I submitted these files:

src/LeapYear.hs:

module LeapYear (isLeapYear) where
 
isLeapYear :: Int -> Bool
isLeapYear year = hasFactor 4 && (not (hasFactor 100) || hasFactor 400)
  where hasFactor n = year `rem` n == 0

and

package.yaml:

name: leap

dependencies:
  - base

library:
  exposed-modules: LeapYear
  source-dirs: src

tests:
  test:
    main: Tests.hs
    source-dirs: test
    dependencies:
      - leap
      - hspec

One other request: could you make it such that after running ./bin/run-tests-in-docker.sh, the original files are restored? Now I'm getting changed package.yaml, HspecFormatter.hs and Tests.hs files.

[cdimitroulas]

@ErikSchierboom I've added some automatic cleanup when running bin/run-tests-in-docker.sh or even bin/run-tests.sh.

Regarding the error you encountered, I can't reproduce that. I believe that this used to occur when you ran the test-runner multiple times without cleanup in-between because the lines appended to package.yaml would only work correctly once. I think adding this newline char at the end of the code injection should have fixed the problem though.
Could you please try cleaning up the submitted input directory's package.yaml before running again?

Note that I presume this issue would not have occurred in a real run as the input directory would contain a clean copy of the files each time.

[ErikSchierboom]

edit: see comments

[ErikSchierboom]

Let's go!