adding CI support

[mwotton]

simple config for travisci.
you can see the build records at https://travis-ci.org/mwotton/HLearn/ now.

[mikeizbicki]

Sorry for waiting on this. I definitely plan to get travisci support now, I'm just doing some reorganization of the repo into a cleaner state first.

[mwotton]

No worries. My only interest is getting it consistently buildable so I can junk my crappy solution.

Mike Izbicki [email protected] wrote:

Sorryforwaitingonthis.Idefinitelyplantogettraviscisupportnow,I'mjustdoingsomereorganizationoftherepointoacleanerstatefirst.ReplytothisemaildirectlyorviewitonGitHub.

[mwotton]

Hi Mike - any movement on this? Happy to put some work into getting a consistent CI system set up, but only if you're at a stage where it would actually be useful.

[mikeizbicki]

Hi Mark. Thanks for following up!

Yes, I'm still interested in this. But I don't think it quite makes sense for this repo yet. There's a lot of changes still underway over the next month or two that are likely to entail some major restructuring still.

The subhask repo is probably at a place though where it would make sense. I'd be interested in gettng the CI support integrated into the ghc7.10 branch.

[mikeizbicki]

I've finally gotten around to getting Travis working. But it turns out they don't yet support GHC 7.10, so we can't use it.

[mwotton]

Hvr has a method for doing it that lets you choose 7.10 - I'll have a look
in a week, travelling atm.

On Thu, 18 Jun 2015 5:16 am Mike Izbicki [email protected] wrote:

I've finally gotten around to getting Travis working. But it turns out
they don't yet support GHC 7.10, so we can't use it.

—
Reply to this email directly or view it on GitHub
#45 (comment).

[mikeizbicki]

I think I've figured it out. I should be pushing out the changes in a bit.

[mikeizbicki]

I've finally got Travis working on the subhask repo. (See https://travis-ci.org/mikeizbicki/subhask) But there's two things I haven't figured out to my satisfaction:

1. GHC 7.10 requires llvm 3.5, which isn't available on Travis. My current solution is to download the source and compile it. This isn't ideal because of the huge build time overhead. But at least it works.
2. One of the errors I'm most scared of is performance regressions. I'm not sure the best way to catch these errors. Criterion gives us a good way to measure performance by hand, but not a good way to automate testing to see if performance has worsened. On my local machine, what I've been doing is outputting all the criterion benchmarks to a csv file, then checking if any value in the csv file has changed by more than 20%. I'm not sure how to do something similar on Travis, however, because I don't know what their architecture is like.

[mikeizbicki]

Once I've figured out if there's solutions to these problems I'll add travis to hlearn as well.

[mwotton]

I think you'd want to use travis artifacts

http://blog.travis-ci.com/2012-12-18-travis-artifacts/

mark

On Mon, Jun 22, 2015 at 6:15 PM Mike Izbicki [email protected]
wrote:

I've finally got Travis working on the subhask repo. (See
https://travis-ci.org/mikeizbicki/subhask) But there's two things I
haven't figured out to my satisfaction:

GHC 7.10 requires llvm 3.5, which isn't available on Travis. My
current solution is to download the source and compile it. This isn't ideal
because of the huge build time overhead. But at least it works.
2.

One of the errors I'm most scared of is performance regressions. I'm
not sure the best way to catch these errors. Criterion gives us a good way
to measure performance by hand, but not a good way to automate testing to
see if performance has worsened. On my local machine, what I've been doing
is outputting all the criterion benchmarks to a csv file, then checking if
any value in the csv file has changed by more than 20%. I'm not sure how to
do something similar on Travis, however, because I don't know what their
architecture is like.

—
Reply to this email directly or view it on GitHub
#45 (comment).

[tonyday567]

Criterion gives us a good way to measure performance by hand, but not a good way to automate testing to see if performance has worsened.

Assuming you're most worried mostly about gross degradations - O(n) -> O(n^2) say - you can always wrap the benchmark in a test and log an error on unacceptable performance. One trick is to measure performance on n=100 and n=1000 say, then have the test guess the order.

[mikeizbicki]

I'm really interested inconstant factor performance. I want to catch the scenario where a function that used to take 300ns goes up to 350ns on the same input.

[mikeizbicki]

I've completely redone the testing interface, so it's much improved. Thanks for the push in this direction :)

Unfortunately, Travis builds sometimes fail due to GHC using too much memory. I think the best way to get around this is to disable optimizations on the travis build, but I haven't figured out how to do this. So basically everything is working, but I can't but the build-passing button on the README file yet due to this problem.

[mwotton]

Great to hear it, looking forward to having the chance to play with HLearn a bit more seriously:)

[mwotton]

(would cabal configure --disable-optimization be what you're looking for?)

[mikeizbicki]

I thought so too, but it doesn't seem to actually do anything for me.

[thomasdziedzic]

Is this PR still relevant? It seems that master has a .travis.yml file already