Skip to content

consumer.py: mark task_done for discarded items #309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

masenf
Copy link

@masenf masenf commented Aug 18, 2025

Fixes #308

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Summary

This PR fixes a critical queue synchronization bug in the Consumer class's next() method. The issue occurred when items exceeded the 900KiB size limit (MAX_MSG_SIZE) - these oversized items were being dropped from the queue without calling queue.task_done(), which left the queue's internal task counter unbalanced.

The Consumer class uses Python's Queue with task tracking, where every item retrieved with queue.get() must have a corresponding queue.task_done() call. When the consumer encountered an oversized item, it would log an error and continue processing, but never marked that task as complete. This caused the queue.join() method (used by the flush() operation) to hang indefinitely, waiting for tasks that had already been discarded.

The fix adds a single line queue.task_done() after the error logging for oversized items. This ensures that even discarded items are properly acknowledged in the queue's task counting system, allowing the flush operation to complete normally. The change maintains the existing behavior of dropping oversized items while fixing the synchronization issue that could cause the application to hang.

Confidence score: 5/5

  • This PR is extremely safe to merge with virtually no risk of causing issues
  • Score reflects a minimal, well-targeted fix for a clearly identified synchronization bug with no side effects
  • No files require special attention as this is a single-line fix in a well-understood code path

1 file reviewed, no comments

Edit Code Review Bot Settings | Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flush never completes if too large of item is submitted
1 participant