Discussion: NVD API client should not retry/continue on JsonProcessingExceptions?

[chadlwilson]

Currently whenever there is a JsonProcessingException (specific cause JsonMappingException the library current gets stuck in a loop retrying to retrieve and parse data that is never going to work.

This leads to confusion like mentioned in jeremylong/DependencyCheck#6747 and jeremylong/DependencyCheck#6746 since the errors are hidden and it looks like NVD API problems. This is exacerbated on Gradle, because enabling debug logging to see the problem is non-trivial and in bigger projects will cause Gradle to freak out as it produces too many logs (e.g when using dependencyCheckAggregate). Additionally, I believe it creates a storm of unnecessary extra load on the NVD API?

This was changed in the below, but no real reason shared as to why.

825977e#diff-13b186ee3d6fa6c024eff401d1250ba1e32d00a02464e586aba44bd84015ff66L311-R315

Can we re-evaluate why this was changed, and consider reverting? Generally I'd have thought a JsonProcessingException is a fatal, unretryable error, and the library should fast fail with a clear error.

Since there are multiple reasons for JsonProcessingException perhaps the concern was retrying on temporary stream read issues. If that is the case, an alternative is to fail fast on JsonMappingException but to retry on others, e.g StreamReadException. Not convinced that is the concern though, as the code seems to parse from a string in memory, fully extracted from the body text, not stream it into the parser:

Open-Vulnerability-Project/open-vulnerability-clients/src/main/java/io/github/jeremylong/openvulnerability/client/nvd/NvdCveClient.java

Lines 330 to 344 in 94b7996

	json = response.getBodyText();
	// resolve issue #20
	if (json == null && response.getBody().isBytes()) {
	json = new String(response.getBodyBytes(), StandardCharsets.UTF_8);
	}
	CveApiJson20 current;
	try {
	current = objectMapper.readValue(json, CveApiJson20.class);
	this.indexesToRetrieve.remove(call.getStartIndex());
	} catch (JsonProcessingException e) {
	LOG.debug("Error processing NVD data", e);
	// really re-fetch the same data?
	return next();
	}