Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch can't be used to drain a queue which has workers #545

Open
chrisDeFouRire opened this issue Feb 19, 2025 · 1 comment
Open

Fetch can't be used to drain a queue which has workers #545

chrisDeFouRire opened this issue Feb 19, 2025 · 1 comment

Comments

@chrisDeFouRire
Copy link

chrisDeFouRire commented Feb 19, 2025

Hi! Thanks for your great job with pg-boss!

I've encountered a limitation:

  • I have workers working on a queue which are stuck, for instance because an FTP (don't blame me 😸 ) upload breaks
  • the queue is configured to retry jobs later, after start_after
  • I know the FTP server will remain unavailable for a while
  • so I'd like to drain the queue with a fetch, and collect all the jobs into an archive to send by email (again, don't blame me 😸)
  • fetch doesn't return anything, because of the line below, which states you can't return jobs before their start_after... except in this case, that's exactly what I would want
  • If I stop the workers, and wait until the start_after date, I can fetch the jobs... but it's not convenient at all
  • if I let the workers work, they'll pick the job, retry it and fail it, preventing the fetch.

The line in question:
https://github.com/timgit/pg-boss/blob/97604249556276fdbe36d5d665f53e7644ed81ae/src/plans.js#L512C1-L513C1

What's your take on this case? I see why waiting until start_after makes sense, but also why not have an option to skip waiting?

@timgit
Copy link
Owner

timgit commented Feb 25, 2025

Adding an option to fetch to ignore start_after looks like a good idea to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants