Pick schemas up directly from repo

[macieksmuga]

Per @sarahhunt's suggestion in PR #27, re-architect so that the schemas are freshly loaded from the official GitHub repo every time the compliance suite is built. This would greatly aid in automating a continuous integration workflow between all three repos.

[macieksmuga]

Created matching issue ga4gh/ga4gh-server#660

[hjellinek]

I think it would offer a better user experience if we automatically copy the schemas to some well-known place when they're built (or tagged, maybe), and let developers pull them when needed.

I've created havoc for myself a couple of times by copying the schema files into my local compliance or server branches, only to discover some conflict (in the test data, say) that stopped me. If the copy could happen automatically/behind my back, I'd be even more confused, more often.

I have some thoughts about where we could push the schema files.

That said, we may want to wait to tackle this until the whole Avro/Protobuf conversion is out of the way.

[diekhans]

the well know place is github. Ideally it would happen given a repo+ tag/branch. That would make if far easier for developers to work in their own branches.

A 4th repo, containing the build system that pulls and build the whole system is something to consider. There are multiple systems built on git like this, for instance, Android has one.

[jeromekelleher]

I'm not sure how we'd do this --- have we a rough outline of how this would work? Would this pull down the schemas fresh on mvn install or whatever?

[diekhans]

I would have a build repo that has a mvn that does it all.

I am not sure if submodules would work, because the submodule is tied
to a particular commit. One would probably want a branch.

I believe the Android repo facility address just this problem:

 https://source.android.com/source/developing.html

Jerome Kelleher [email protected] writes:

I'm not sure how we'd do this --- have we a rough outline of how this would
work? Would this pull down the schemas fresh on mvn install or whatever?

—
Reply to this email directly or view it on GitHub.*

[diekhans]

Herb and Maciek will be proposing a system for building all three repos that handles the needs of developer working on topic branches as well as building official release.
(as soon as they survive the baseline release)

[macieksmuga]

At the very least we should have a single place in the compliance suite where the version number of the schemas is sourced. As currently stands, I had to grep some 20 files to update this in PR #138