Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consumeRecords does not provide enough context for errors #5967

Open
cpwright opened this issue Aug 22, 2024 · 0 comments
Open

consumeRecords does not provide enough context for errors #5967

cpwright opened this issue Aug 22, 2024 · 0 comments
Assignees
Labels
core Core development tasks feature request New feature or request kafka
Milestone

Comments

@cpwright
Copy link
Contributor

As a user, I want the KafkaTools.consumeToTable method to throw an error that includes the offset and partition that failed to parse so that I can effectively debug the problem with my topic.

I received the following error, which tells me that consumption failed, and my result table did correctly fail; but I do not have enough context to understand which message is actually broken.

08:30:18.728 ERROR aph-updateExecutor-5 | i.d.s.s.SessionService    | Internal Error '3041479e-d802-4973-9f3a-c489e9bb7a07' io.deephaven.UncheckedDeephavenException: Failed to parse JSON string.
	at io.deephaven.kafka.ingest.JsonNodeUtil.makeJsonNode(JsonNodeUtil.java:32)
	at io.deephaven.kafka.JsonImpl.lambda$jsonToObjectChunkMapper$0(JsonImpl.java:233)
	at io.deephaven.kafka.ingest.KafkaStreamPublisher.consumeRecords(KafkaStreamPublisher.java:321)
	at io.deephaven.kafka.KafkaTools$SimpleKafkaRecordConsumer.consume(KafkaTools.java:1770)
	at io.deephaven.kafka.ingest.KafkaIngester.pollOnce(KafkaIngester.java:386)
	at io.deephaven.kafka.ingest.KafkaIngester.consumerLoop(KafkaIngester.java:302)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Array (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 2]
	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportInvalidEOF(ParserMinimalBase.java:585)
	at com.fasterxml.jackson.core.base.ParserBase._handleEOF(ParserBase.java:535)
	at com.fasterxml.jackson.core.base.ParserBase._eofAsNextChar(ParserBase.java:552)
	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipWSOrEnd(ReaderBasedJsonParser.java:2491)
	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:673)
	at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:609)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:104)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:25)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4947)
	at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3280)
	at io.deephaven.kafka.ingest.JsonNodeUtil.makeJsonNode(JsonNodeUtil.java:30)
	... 6 more
@cpwright cpwright added feature request New feature or request triage labels Aug 22, 2024
@devinrsmith devinrsmith self-assigned this Aug 22, 2024
@devinrsmith devinrsmith added this to the 3. Triage milestone Aug 22, 2024
@rcaudy rcaudy added core Core development tasks kafka and removed triage labels Aug 28, 2024
@rcaudy rcaudy modified the milestones: Triage, 0.37.0 Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core development tasks feature request New feature or request kafka
Projects
None yet
Development

No branches or pull requests

3 participants