Add method to directly load triples into BasicHDT

[rubensworks]

This allows triples to be directly loaded into a BasicHDT without having to read them from a file.

[RubenVerborgh]

@mielvds @joachimvh @laurensdv This is just what we needed for the management server. Now we can set up a pipeline unzip / clean / etc. without having to materialize.

[RubenVerborgh]

@LaurensRietveld @wouterbeek You will also like this; support for piping will likely simplify the LOD Laundromat internal pipeline.

[LaurensRietveld]

Cool stuff Ruben, thanks. This will indeed simplify some parts in our pipeline.
To verify: this PR means piped inputs, but not streamed HDT generation right? I.e., this does not affect the hardware footprint of the HDT generation process?

[rubensworks]

@LaurensRietveld Correct, it works the same way as generating HDT's from an input file, with the difference that this new method accepts a triple iterator instead of a file.

[mielvds]

Before I merge, what is the overlap between the loadTriples and loadDictionary methods? Could this be written shorter and cleaner?

[rubensworks]

@mielvds You're right, I could abstract some things. The only real difference is the ProgressListener parameter. I'll look into it.

[rubensworks]

@mielvds All overlapping code between the loadTriples and loadDictionary methods has been abstracted.

[mielvds]

@rubensworks I did an attempt at merging this with the current develop branch, but I can't make it compile. There seems to be an issue with the inline functions you declared:

 [HDT] Compiling src/hdt/BasicHDT.cpp
src/hdt/BasicHDT.cpp:273:9: error: expected expression
    return [fileName, baseUri, notation](RDFCallback& loader) {
           ^
src/hdt/BasicHDT.cpp:282:9: error: expected expression
    return [triples](RDFCallback& loader) {
           ^
2 errors generated.

Could you have a look? Result is in https://github.com/rdfhdt/hdt-cpp/tree/rubensworks-feature-loadtriples-single

[rubensworks]

@mielvds Apparently the Makefile isn't using c++11, in which lambda support was introduced.

So either we add -stdlib=libc++ -std=c++11 to the default Makefile flags, which should be supported on most machines nowadays. Or I rewrite this PR so it doesn't use lambda's.

[RubenVerborgh]

I'd say; try C++11, and if Travis takes it, it's good.

[mielvds]

If it doesn't break backwards compatibility, fine by me.
If you make changes to this PR, do it directly in the https://github.com/rdfhdt/hdt-cpp/tree/rubensworks-feature-loadtriples-single branch please.

[RubenVerborgh]

We can probably remove the gzip dependency after this PR?

[RubenVerborgh]

Could you please rebase this on the latest develop?

[RubenVerborgh]

How does this compare to #47? (CC: @drobilla)

[RubenVerborgh]

@rubensworks Based on your comment #77 (comment), how do we proceed with this one?

[rubensworks]

@RubenVerborgh As this PR only adds a new method for streaming triples into BasicHDT, STDIN streaming would have to be implemented on top of that.
Piping can however already be done on Mac and Linux.

So I suggest closing this PR, unless someone would require this functionality for something else.

[RubenVerborgh]

Agree on closing, thanks!

[mielvds]

@rubensworks can I delete the branch?

[LaurensRietveld]

Or, should we rebase this branch on dev and reopen this pr? I've yet to see anybody to convert a gzipped ntriples file to hdt, despite what the Readme says. And, got the impression @rubensworks that this pr can be used to convert gzipped files to hdt, is that correct?

[LaurensRietveld]

Sorry, ignore my last message (context: #47 (comment))