Make sure bodhi_update job runs don't get stuck in queued state (#…

…2609)

Make sure `bodhi_update` job runs don't get stuck in `queued` state

When an unhandled exception occurs, a Sentry issue is created but the job run stays in queued state, which breaks the condition that is supposed to prevent submitting the same update more than once, as it considers that job run to be in progress. Make sure the state is updated when that happens.
Question is, in such case should the exception and traceback be stored and issue with it created in the issue repository or do we consider that internal?
Related to #2596.

Reviewed-by: Matej Focko
Reviewed-by: Laura Barcziová
Reviewed-by: Nikola Forró

packit_service/worker/handlers/bodhi.py

@@ -192,6 +192,18 @@ def run(self) -> TaskResults:
                 target_model.set_status("error")
                 target_model.set_data({"error": error})
 
+            except Exception as ex:
+                if self.celery_task and not self.celery_task.is_last_try():
+                    target_model.set_status("retry")
+                    raise
+
+                error = f"Internal error, please contact us: {ex}"
+                errors[target_model.target] = error
+
+                target_model.set_status("error")
+                target_model.set_data({"error": error})
+                raise
+
         if errors:
             self.report_in_issue_repository(errors=errors)