JobManagement/DeleteJob does not completely delete the job

[lupreCSC]

Deleting a job using the heappe/JobManagement/DeleteJob endpoint appears to delete the job-related files on the cluster but not the actual job information in the database.

Subsequent requests to heappe/JobManagement/CurrentInfoForJob with the same job id still responds with all the information about the job and it is also still included in heappe/JobManagement/ListJobsForCurrentUser with no indication that it was deleted.

This is an issue as we now longer can tell which jobs still exist and which require cleanup, as well as the data transferred in responses to ListJobsForCurrentUser growing ever larger.

[jkonvicka]

Dear @lupreCSC,
in the next release there will be added propagation of JobState Deleted into the JobSpecification in the next release.

[lupreCSC]

Thanks. That doesn't quite solve the problem for me, though. I would prefer if deleted things are truly deleted and not included in the listing returned by heappe/JobManagement/ListJobsForCurrentUser. The problem is that this is becoming a rather large list of completed jobs for us, leading to long response times of the HEAppE API. We are mostly interested in monitoring the currently running ones, so we would like to be able to remove the completed ones. Either by completely deleting them from the system or maybe via a filtering options for the heappe/JobManagement/ListJobsForCurrentUser endpoint.

[jkonvicka]

I'm going to observe possibilities but filter option seems to be great idea.