Error messages needed for failed record imports

[mietcls]

Enhancement

TO BE REVISITED

Make errors more clear for DOI imports.
Basic errors already implemented, need more granular feedback.

Success criteria

This is a non-technical error for end-users that do not use technical terms.

When the import fails for a DOI:

• The errors appear for publications and datasets.
• The toaster does not disappear. _We give the user the time to read it and decide what to do.
• The toaster errors use this component https://biblio-backend-ugent-prototype.vercel.app/styleguide/b-toasts.html#component-02-b-toast-2-lines
• I get information about what went wrong, for publications and datasets
  • Provide error: DOI not found by trusted service #1138
  • Provide error: The import service for DOIs cannot be reached (Crossref or DataCite) #1137
  • Provide error: The API of DataCite or Crossref says there is a problem with accessing the record #1136
  • Provide error: JSON not recognised #1135

Out of scope: Information missing. We will let people complete the required fields and publish or save as draft.

Record Validation errors => the CrossRef JSON is valid; but fields are empty, or data is missing)

Automatic testing scenario

• When a DOI fails to import
• We check why it does not import
• And give the correct error message and solution

Additional context

Based on #1088 and #1035.

Suggestions:
Originally posted by @netsensei in #1088 (comment)

I think we have these classes of problems

• Record Validation errors => the CrossRef JSON is valid; but fields are empty, or data is missing)
• JSON Validation errors => The CrossRef JSON itself is invalid (it's not a JSON string; but garbled data)
• API errors => The API returns 4xx or 5xx errors. The former: a record could not be found, or can't be accessed; the latter: the API returns some kind of general system error.
• Connection errors => CrossRef is unreachable (The wire between the Backoffice & Crossref is blocked for whatever technical reason; or the CrossRef API is offline)

I've implemented validation errors in #1035 but the other types of errors will yield a more generic "something went wrong" error pop-up. We could be more specific; but I don't think we want to burden researchers with the minutiae of how an API works, no? We do want some human-friendly message that gives at least a general idea of what went wrong (e.g. "CrossRef could not be reached at this time, try again later", "The record could not be found with CrossRef", "The record could not be retrieved from CrossRef")

So, if we know how to word the errors, it shouldn't be that hard to make nicer error messages.

Screenshots

Current alert:
Try to import a DOI for a publication or dataset.

1. Go to https://backoffice.bibliotest.ugent.be/publication
2. Add a publication via DOI: 10.2143/TVF.84.3.3291503
3. See error

[nicolasfranck]

@netsensei indeed, we cannot burden reviewers with api problems, and if we are too vague in our error messaging and report something went wrong, end users will also report us vague problems.

Anyway, apart from the error messaging itself, weren't we also thinking about generating an identifier, report that to the end user, and note that in the log message next to the error for faster tracking?

[netsensei]

Anyway, apart from the error messaging itself, weren't we also thinking about generating an identifier, report that to the end user, and note that in the log message next to the error for faster tracking?

1/

All incoming requests get a request ID by default. We can display this ID when the server hits a fault condition and returns an error to the end-user. We currently do this in the context of adding or creating new contributors => when an error happens, you see a modal which includes a request ID and mention to contact the review team.

2/

Tagging a request with an ID and showing an error w/ ID can only be fully achieved when a HTTP request was received successfully, and a HTTP response was successfully delivered to the browser. That doesn't happen here: transmission of the HTTP request itself - the upload of a file - can't be completed e.g. because the connection was lost, or because the server itself became unavailable.

So, in that context, there's no use for generating a request ID and including it into an error message which could be pushed to the end user.

The above was related to issues with uploading files where request ID's wouldn't be useful at all.

[mietcls]

All incoming requests get a request ID by default. We can display this ID when the server hits a fault condition and returns an error to the end-user. We currently do this in the context of adding or creating new contributors => when an error happens, you see a modal which includes a request ID and mention to contact the review team.

Is there a way where we can avoid burdening the user with any type of message, and automate this?

[netsensei]

We've already added this feature in production to help debug contributors. In practice, I haven't received an issue yet where a request ID was mentioned. So, we might need to re-assess the usefulness here.

Right now; debugging DOI import errors boils down to us to taking the DOI as it trickles through the tickets towards us, and then replicating the import on our own laptops while looking into the logs and figuring out what went wrong. That's "good enough" for now, as the frequency with which these issues pop up isn't all that high.

Request ID's really become useful in a context where there's a high volume of operations, a significant amount of data problems occur, a commitment to return a swift response to the user and you need a way to delve through large amounts of log data to figure out what went wrong in a particular case.

[nicolasfranck]

And we also need to acknowledge that we allow reviewers to manually add/update a DOI,
while one cannot create such a publication with that DOI using the wizard.

[mietcls]

Can we split off side notes like this to keep this issue clear and concise? Reading this a month later is very confusing :D Thanks! You can create a new issue and remove any superfluous comments.

[mietcls]

@netsensei I have a suggestion:

• If there's an issue with a service (like CrossRef) we tell them: try again later, find a different import option or get in touch with the service. This covers:
  • Provide error: The import service for DOIs cannot be reached (Crossref or DataCite) #1137
  • Provide error: The API of DataCite or Crossref says there is a problem with accessing the record #1136
  • Provide error: JSON not recognised #1135
• If the DOI/other identifier is not found by the service; we tell them: check whether the DOI is already active, get in touch with your publisher. This covers:
  • Provide error: DOI not found by trusted service #1138
  • Question: is it also possible the DOI/other identifier is ill-formatted here?

Questions:

• Is it realistic to do validation on the DOI/other identifier?
• Does it cover however if something goes wrong on our side? What can go wrong on our side?