[community] Updating the issue / pull requests management process

[fniephaus]

This issue documents recent and planned updates with regard to the issue / PR management process and allows further discussion within the community if required.

As discussed in the last advisory board meeting, the GraalVM team is further improving the way they handle issues and pull requests on GitHub. One example is that they are setting up GitHub reports internally to ensure that all open issues and pull requests are assigned to someone, highlighting the ones without any comments from the team. In case you're interested, I ran their reporting script and here are the two summaries (full results are here):

Getting issues....................
====================================================================================================
issues opened per year
====================================================================================================
2016: oracle=26	other=29
2017: oracle=2	other=30
2018: oracle=16	other=463
2019: oracle=22	other=877
2020: oracle=25	other=457

Getting pullRequests........
====================================================================================================
pullRequests opened per year
====================================================================================================
2016: oracle=59	other=97
2017: oracle=8	other=19
2018: oracle=25	other=101
2019: oracle=28	other=226
2020: oracle=57	other=166

Since we can finally use GitHub Actions, I had a quick chat with @dougxc about introducing something like a stalebot in the near future to deal with the growing number of open issues. Of course, closing issues automatically after some time does not actually fix any of them. However, there seem to be numerous outdated or even addressed issues that are still open and only distract from the important ones. The effectiveness of a stalebot mostly depends on the policy it uses. A first idea for such a policy could be: "close all issues that have at least one comment from a GraalVM contributor, is older than 12 months, and is not labeled as feature".

A stalebot might not be needed if the GraalVM team is willing to spend more time managing issues/PRs on GitHub and decides to close issues a bit more often. Maybe they can even find a few trusted members of the community willing to help as moderators (see "Triage" permissions).

[dougxc]

The first step before automatically closing stale issues is to have all (or most) open issues and pull requests assigned to someone. Otherwise I fear that stalebot notifications will go into the void. We are currently working on reducing the unassigned items.
However, we can in the meantime enable stalebot to label stale issues without closing them. I think a reasonable policy is to label any item without a comment from either oracle org member or the item's assignee (who may not be in the oracle org) in the last 2 months as stale. How does that sound? @chumer @lukasstadler @wirthi @thomaswue

[jaikiran]

On a related note - would it be possible to mark the resolved issues and/or merged PRs against specific community (and if needed enterprise) releases? Right now it isn't straightforward to figure out which merged PR made it to which particular release.

[dougxc]

That would be nice but I don't know how to automate that (and without automation, it will never be reliable). Any suggestions?

[jaikiran]

I don't have much experience with GitHub APIs and don't know of a way to automate this. However, given that we have some labeling automation being already used in Quarkus, I opened a enhancement request here maxandersen/jbang-issuelabeler#2 to see if something can be done with that tool.

[wirthi]

However, we can in the meantime enable stalebot to label stale issues without closing them. I think a reasonable policy is to label any item without a comment from either oracle org member or the item's assignee (who may not be in the oracle org) in the last 2 months as stale. How does that sound?

I wouldn't automatically close them; they might just be feature requests that we will eventually do, but not within 2 months. But labeling such tickets seems fine to me.

I went through all JavaScript and Ruby tickets I could find and assigned them to someone. Didn't touch Compiler/SVM tickets (that use JavaScript or Ruby) though.

-- Christian

[dougxc]

I wouldn't automatically close them;

I agree. We would need a very conservative policy if we decide to automate closing of items.

[RoxaBee]

Although @fniephaus proposed policy is reasonable, agree that we should be conservative in automatically closing issues. Some other options include using a no-response bot that only closes issues when the author hasn’t responded to a request for more information (per most recent GitHub best practices) and/or promoting issues to GitHub Discussions (now enabled for Graal repo!!).

[RoxaBee]

In the spirit of trying out Discussions, I am going to convert this Issue into a Discussion. Everyone buckle up!