Bibliography API and listing functionality

[stschiff]

This PR finally adds a new Server API /bibliography as well as a new command-line option trident list --bibliography to gather bibliography information.

The feature is straight-forward in terms of usability.

In terms of implementation, I have made some additional changes to the Aeson interface specifically for the Server Communication. Specifically I looked a bit closer how Aeson encodes for Nothings in Maybe types, and found out that by default these get encoded as null in objects. This is slightly wasteful, as null-fields can also simply be omitted in a given object. When decoding into Haskell types again, we anyway use the operator .:?, which takes care of interpreting missing keys in objects as Nothings. I have therefore added a small function called removeNulls in ServerClient.hs to remove null-fields from objects. Not sure how big the effect is, but it was simple enough and I'm glad I now understand Maybe-encoding in Aeson better. The newest version of Aeson (>2.2.2.0) actually adds an encoding operator to remove null-values, which will eventually make my function redundant, but that latest version isn't even in the latest LTS, so it'll be some time.

Running the new bibliography function on the Community-Archive reveals some gaps in our bibliography records, which we can easily fix with a bit of curation work.

[codecov]

Codecov Report

Attention: Patch coverage is 42.94479% with 93 lines in your changes missing coverage. Please review.

Project coverage is 59.66%. Comparing base (d918c11) to head (2ee8f69).
Report is 14 commits behind head on master.

Files with missing lines	Patch %	Lines
src/Poseidon/CLI/List.hs	37.50%	26 Missing and 4 partials ⚠️
src/Poseidon/ServerClient.hs	48.07%	27 Missing ⚠️
src/Poseidon/CLI/OptparseApplicativeParsers.hs	0.00%	14 Missing ⚠️
src/Poseidon/CLI/Serve.hs	21.42%	10 Missing and 1 partial ⚠️
src/Poseidon/BibFile.hs	64.28%	3 Missing and 2 partials ⚠️
src/Poseidon/Package.hs	75.00%	2 Missing and 3 partials ⚠️
src/Poseidon/Utils.hs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #320      +/-   ##
==========================================
- Coverage   60.43%   59.66%   -0.77%     
==========================================
  Files          29       29              
  Lines        4218     4349     +131     
  Branches      489      498       +9     
==========================================
+ Hits         2549     2595      +46     
- Misses       1180     1256      +76     
- Partials      489      498       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[stschiff]

I bumped the version to 1.5.9.0, so I expect this to be merged in after #318.

[nevrome]

The code looks great and trident list --bibliography behaves as expected. I think, though, that the default table output is too wide. Maybe some columns can be omitted here (Journal? Author?).

I'll later test the server-client interaction with this new API feature.

[nevrome]

I played a bit more with this new feature. Everything works as expected.

The only thing I wonder is whether the server should immediatelly resolve the markup in the bibtex entries (e.g. \"{u} inestead of ü, or {DNA} for capitalisation). Maybe it's not feasible to do this for everything one may encounter in a bibtex file, if we don't want to depend on pandoc again.

[stschiff]

Thanks for the comments on the DOI and the table layout. I'll fix that!

Regarding the LaTeX stuff in BibTex: I find that super annoying, and I wonder whether we should just try and fix all the bibtex-entries manually (or with some script) in the PCA?

On the other hand, if we have a function to remove tags from BibTeX, why just using it for the API? Shouldn't we then also make that feature available to rectify or something? I don't know. I am a bit tempted to not act on this at all and simply put the blame on the Community-Archive, and make sure we fix things there?

[nevrome]

Regarding the LaTeX stuff in BibTex: I find that super annoying, and I wonder whether we should just try and fix all the bibtex-entries manually (or with some script) in the PCA?

In my opinion the bibtex entries are not broken. At least the capitalisation annotation with {} is a feature I frequently use myself. I don't think this is data problem, but something the parser/renderer should handle -- just as pandoc does with its massive machinery.

But I understand that writing the necessary code would be painful. The latex symbol list is extremely extensive (https://mirror.dogado.de/tex-archive/info/symbols/comprehensive/symbols-a4.pdf). Maybe there is some shortcut that covers the most relevant cases?

[stschiff]

OK, so I've implemented some Author rendering and removed the journal from the table. It's still a large table, but I think it's workable. Take a look.

[nevrome]

Yes - works 👍