From fe61ddc544e616b136271a2181c20dce267b2c87 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Sat, 20 May 2023 12:36:37 -0400 Subject: [PATCH 1/7] Design of issue tracking/reporting via GitHub --- doc/design/dandiset-issues-tracking.md | 54 ++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 doc/design/dandiset-issues-tracking.md diff --git a/doc/design/dandiset-issues-tracking.md b/doc/design/dandiset-issues-tracking.md new file mode 100644 index 000000000..1e35e82af --- /dev/null +++ b/doc/design/dandiset-issues-tracking.md @@ -0,0 +1,54 @@ +# Dandiset issues reporting/tracking + +Dandisets could have issues related to problems with + +- uploaded data files +- metadata records +- missing files +- validation errors +- fix/enhancement proposals +- etc. + +We need to enable an easy and consistent interface to allow archive users to search, report, followup, signup etc to them. +Overall, we need an issue tracking system, while avoiding creating one from scratch. + +## Current Process + +Largely not formalized: + +- DLP does not have any pointer to report an issue for dandiset + - footer has overall link to report issues for the archive (and incorrect: [PR to fix](https://github.com/dandi/dandi-archive/pull/1594)) +- We generally refer people to [helpdesk discussions](https://github.com/dandi/helpdesk/discussions) as common place to report issues +- While preapring DataLad versions of the dandisets and posting them on GitHub we do not disable `Issues` feature: + - for some dandisets, e.g. https://github.com/dandisets/000108/issues we already used GitHub issues + - we do not anyhow automatically add/invite original uploaders to those dandisets. + - note: ATM we do not collect audit information on who specifically uploaded any asset to reliably deduce github users to assign. +- The issue discussing this: https://github.com/dandi/dandi-archive/issues/863 + + +## Proposed Process + +**Overall**: Reuse existing Dandisets on GitHub (https://github.com/dandisets), and available for them GitHub's issue tracker. + +- [x] Enable Issue tracking per dandiset. + - In the process of creating DataLad dandisets on GitHub we already enable `Issues` functionality for all dandisets. +- [ ] Upon creation of a DataLad dandiset on GitHub, email dandiset owner inviting to subscribe to the issue tracker on GitHub +- [ ] Setup https://github.com/dandisets [organization-wide](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file) [`.github/ISSUE_TEMPLATE`](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms) with different templates/types of issues to report +- [ ] DLP: add a button "View/Report Issues" or "Issues" leading to issue Tracker (e.g. [Issues](https://github.com/dandisets/000108/issues) for 000108) + - Complimentary/alternative: Could be "File an issue" button which would prefeed some issue body with `@dandiset-owner` handles etc. But we might then need to introduce our own UI to choose among different types of issues + - Optionally: dandi-archive could query/cache number of known open issues and include within that button, e.g. have it [(1) Issues](https://github.com/dandisets/000108/issues) + - Complimentary: asset-level issues filing in UI context menu per file, would pre-fill in the asset path +- [ ] Optional: provide consistent collection of labels to outline typical problems which might occur + - e.g., `validation-error`, `unable-to-load`, `io-error`, `missing-file`, `permission`, etc. + +### Benefits + +- Users already must have a GitHub account to + - register/login to dandiarchive: we will reuse that same mechanism + - submit support requests in helpdesk +- GitHub provides a versatile and customizable issue tracking system many of the users might already be familiar with + +### Disadvantages + +- not "integrated" within dandiarchive.org + - A possible mitigation: I guess we could collect mirror issues/comments etc from GitHub internally in the archive. There are tools which could even be used to help. E.g. @yarikoptic has experience with using https://github.com/MichaelMure/git-bug to sync all issues from GitHub locally to collect all contributors to the project. E.g. [this script](https://github.com/nipy/heudiconv-joss-paper/blob/main/authors/tools/make-summaries#L92) processes a JSON dump of all issues from `git bug` mirror. From b72efb7342d93ec1f34e26317f8d85d60ebb4212 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Mon, 17 Jul 2023 13:45:18 -0400 Subject: [PATCH 2/7] Update dandiset-issues-tracking.md --- doc/design/dandiset-issues-tracking.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/design/dandiset-issues-tracking.md b/doc/design/dandiset-issues-tracking.md index 1e35e82af..f716b0e15 100644 --- a/doc/design/dandiset-issues-tracking.md +++ b/doc/design/dandiset-issues-tracking.md @@ -16,11 +16,11 @@ Overall, we need an issue tracking system, while avoiding creating one from scra Largely not formalized: -- DLP does not have any pointer to report an issue for dandiset +- DLP does not have any pointer to report an issue for Dandisets - footer has overall link to report issues for the archive (and incorrect: [PR to fix](https://github.com/dandi/dandi-archive/pull/1594)) - We generally refer people to [helpdesk discussions](https://github.com/dandi/helpdesk/discussions) as common place to report issues -- While preapring DataLad versions of the dandisets and posting them on GitHub we do not disable `Issues` feature: - - for some dandisets, e.g. https://github.com/dandisets/000108/issues we already used GitHub issues +- While preparing DataLad versions of the Dandisets and posting them on GitHub we do not disable `Issues` feature: + - for some Dandisets, e.g. https://github.com/dandisets/000108/issues we already used GitHub issues - we do not anyhow automatically add/invite original uploaders to those dandisets. - note: ATM we do not collect audit information on who specifically uploaded any asset to reliably deduce github users to assign. - The issue discussing this: https://github.com/dandi/dandi-archive/issues/863 @@ -30,9 +30,9 @@ Largely not formalized: **Overall**: Reuse existing Dandisets on GitHub (https://github.com/dandisets), and available for them GitHub's issue tracker. -- [x] Enable Issue tracking per dandiset. - - In the process of creating DataLad dandisets on GitHub we already enable `Issues` functionality for all dandisets. -- [ ] Upon creation of a DataLad dandiset on GitHub, email dandiset owner inviting to subscribe to the issue tracker on GitHub +- [x] Enable Issue tracking per Dandiset. + - In the process of creating DataLad Dandisets on GitHub we already enable `Issues` functionality for all dandisets. +- [ ] Upon creation of a DataLad Dandiset on GitHub, email the Dandiset owner inviting them to subscribe to the issue tracker on GitHub - [ ] Setup https://github.com/dandisets [organization-wide](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file) [`.github/ISSUE_TEMPLATE`](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms) with different templates/types of issues to report - [ ] DLP: add a button "View/Report Issues" or "Issues" leading to issue Tracker (e.g. [Issues](https://github.com/dandisets/000108/issues) for 000108) - Complimentary/alternative: Could be "File an issue" button which would prefeed some issue body with `@dandiset-owner` handles etc. But we might then need to introduce our own UI to choose among different types of issues From f1385b70e3086664b3e0c8c0d2b775b04a667791 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 17 Jul 2023 22:58:02 -0400 Subject: [PATCH 3/7] Add section on Changes to dandi-archive web ui Co-authored-by: Cody Baker <51133164+CodyCBakerPhD@users.noreply.github.com> --- doc/design/dandiset-issues-tracking.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/design/dandiset-issues-tracking.md b/doc/design/dandiset-issues-tracking.md index f716b0e15..6b4a6acf4 100644 --- a/doc/design/dandiset-issues-tracking.md +++ b/doc/design/dandiset-issues-tracking.md @@ -40,7 +40,10 @@ Largely not formalized: - Complimentary: asset-level issues filing in UI context menu per file, would pre-fill in the asset path - [ ] Optional: provide consistent collection of labels to outline typical problems which might occur - e.g., `validation-error`, `unable-to-load`, `io-error`, `missing-file`, `permission`, etc. +### Changes on dandi-archive Web UI +- Add a section to the web UI for each DANDI set for buttons, such as "See issues", "File an issue", "Ask a question" which routes to the corresponding Issues page on the GitHub repo for that DANDI set +- Buttons or side component could also report the number of questions asked about that DANDI set ### Benefits - Users already must have a GitHub account to From a16e5bc33403635bc3a0e3031283a3b8b6116355 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Fri, 13 Oct 2023 14:37:07 -0400 Subject: [PATCH 4/7] Modify the approach -- we will invite owners to Triage issues and use bot --- doc/design/dandiset-issues-tracking.md | 27 ++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/doc/design/dandiset-issues-tracking.md b/doc/design/dandiset-issues-tracking.md index 6b4a6acf4..82babcddd 100644 --- a/doc/design/dandiset-issues-tracking.md +++ b/doc/design/dandiset-issues-tracking.md @@ -32,26 +32,45 @@ Largely not formalized: - [x] Enable Issue tracking per Dandiset. - In the process of creating DataLad Dandisets on GitHub we already enable `Issues` functionality for all dandisets. -- [ ] Upon creation of a DataLad Dandiset on GitHub, email the Dandiset owner inviting them to subscribe to the issue tracker on GitHub +- [ ] Initiate documentation in handbook on managing issues of the dandisets +- [ ] Upon creation of a DataLad Dandiset on GitHub: + - Invite all Dandiset owners in Triage role to the dandiset. In Triage scope they can close, label, assign issues. The other actions (e.g. user management) would need to be implemented using [bot]. + - Email them pointing to documentation (item above) - [ ] Setup https://github.com/dandisets [organization-wide](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file) [`.github/ISSUE_TEMPLATE`](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms) with different templates/types of issues to report - [ ] DLP: add a button "View/Report Issues" or "Issues" leading to issue Tracker (e.g. [Issues](https://github.com/dandisets/000108/issues) for 000108) - - Complimentary/alternative: Could be "File an issue" button which would prefeed some issue body with `@dandiset-owner` handles etc. But we might then need to introduce our own UI to choose among different types of issues - - Optionally: dandi-archive could query/cache number of known open issues and include within that button, e.g. have it [(1) Issues](https://github.com/dandisets/000108/issues) + - Complimentary: dandi-archive could query/cache number of known open issues and include within that button, e.g. have it [(1) Issues](https://github.com/dandisets/000108/issues) - Complimentary: asset-level issues filing in UI context menu per file, would pre-fill in the asset path - [ ] Optional: provide consistent collection of labels to outline typical problems which might occur - e.g., `validation-error`, `unable-to-load`, `io-error`, `missing-file`, `permission`, etc. + - ref: https://github.com/dandi/dandisets/issues/361 + +### Alternatives considered + +#### GitHub, but not inviting users and operating using `@handles` + +Although sounded like a good idea at first, cumbersome to implement with consistent and/or desired behavior: + +- Dandiset owners would really have no power to do anything, like close the issue etc, unless we implement all such actions (assignment, labeling, etc) via a [bot]. +- To ensure that clicking on "New issue" would add needed `@handles` we would need to either + - add action to monitor new issues to enhance description with the `@handles`, or + - generate/update issue template per each dandiset with its own list of owners. + ### Changes on dandi-archive Web UI - Add a section to the web UI for each DANDI set for buttons, such as "See issues", "File an issue", "Ask a question" which routes to the corresponding Issues page on the GitHub repo for that DANDI set - Buttons or side component could also report the number of questions asked about that DANDI set + ### Benefits - Users already must have a GitHub account to - register/login to dandiarchive: we will reuse that same mechanism - submit support requests in helpdesk - GitHub provides a versatile and customizable issue tracking system many of the users might already be familiar with +- In Triage mode dandisets owners would have powers to assign/close issues. ### Disadvantages -- not "integrated" within dandiarchive.org +- not "integrated" within dandiarchive.org as issues (meta)data would not be contained within DANDI. - A possible mitigation: I guess we could collect mirror issues/comments etc from GitHub internally in the archive. There are tools which could even be used to help. E.g. @yarikoptic has experience with using https://github.com/MichaelMure/git-bug to sync all issues from GitHub locally to collect all contributors to the project. E.g. [this script](https://github.com/nipy/heudiconv-joss-paper/blob/main/authors/tools/make-summaries#L92) processes a JSON dump of all issues from `git bug` mirror. + +[bot]: https://github.com/dandi/dandisets/issues/360 From 467fcc3d4f258ece29a064ebcbc743e586225229 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 22 Nov 2023 18:16:04 -0500 Subject: [PATCH 5/7] Add needed changes for dandi-archive backend which would take care about creating repos if org is provisioned --- doc/design/dandiset-issues-tracking.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/doc/design/dandiset-issues-tracking.md b/doc/design/dandiset-issues-tracking.md index 82babcddd..09d41c1ad 100644 --- a/doc/design/dandiset-issues-tracking.md +++ b/doc/design/dandiset-issues-tracking.md @@ -55,10 +55,18 @@ Although sounded like a good idea at first, cumbersome to implement with consist - add action to monitor new issues to enhance description with the `@handles`, or - generate/update issue template per each dandiset with its own list of owners. +### Changes on dandi-archive Backend + +- Provision configuration settings for + - GitHub organization where to create repository for a new dandiset + - GitHub token with sufficient permissions to create repository in that organization +- If there is provisioned GitHub organization: Job creating a new dandiset entry would trigger a job to create a new repository. + ### Changes on dandi-archive Web UI -- Add a section to the web UI for each DANDI set for buttons, such as "See issues", "File an issue", "Ask a question" which routes to the corresponding Issues page on the GitHub repo for that DANDI set -- Buttons or side component could also report the number of questions asked about that DANDI set +- If there is provisioned GitHub organization: + - Add a section to the web UI for each DANDI set for buttons, such as "See issues", "File an issue", "Ask a question" which routes to the corresponding Issues page on the GitHub repo for that DANDI set + - Buttons or side component could also report the number of questions asked about that DANDI set ### Benefits @@ -66,7 +74,8 @@ Although sounded like a good idea at first, cumbersome to implement with consist - register/login to dandiarchive: we will reuse that same mechanism - submit support requests in helpdesk - GitHub provides a versatile and customizable issue tracking system many of the users might already be familiar with -- In Triage mode dandisets owners would have powers to assign/close issues. +- In Triage mode dandisets owners would have powers to assign/close issues +- We will make it optional, so if no GitHub organization provisioned, no backend/UI actions/elements. ### Disadvantages From d8adab53e4f9c7875eeb3c1a8225dcb3fcd4d2ef Mon Sep 17 00:00:00 2001 From: Roni Choudhury <2903332+waxlamp@users.noreply.github.com> Date: Fri, 8 Dec 2023 11:39:34 -0500 Subject: [PATCH 6/7] Use a specific organization to store issues That is, each instance will specify which org to use, rather than relying on out-of-band use of DataLad to create the repos. Co-authored-by: Yaroslav Halchenko --- doc/design/dandiset-issues-tracking.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/design/dandiset-issues-tracking.md b/doc/design/dandiset-issues-tracking.md index 09d41c1ad..e8a24832f 100644 --- a/doc/design/dandiset-issues-tracking.md +++ b/doc/design/dandiset-issues-tracking.md @@ -33,10 +33,11 @@ Largely not formalized: - [x] Enable Issue tracking per Dandiset. - In the process of creating DataLad Dandisets on GitHub we already enable `Issues` functionality for all dandisets. - [ ] Initiate documentation in handbook on managing issues of the dandisets -- [ ] Upon creation of a DataLad Dandiset on GitHub: +- [ ] Upon creation of a Dandiset on dandi-archive: + - Create (via Github REST API) a git repository on GitHub in a configured (per instance) organization (referring to as `{organization}` below) - Invite all Dandiset owners in Triage role to the dandiset. In Triage scope they can close, label, assign issues. The other actions (e.g. user management) would need to be implemented using [bot]. - Email them pointing to documentation (item above) -- [ ] Setup https://github.com/dandisets [organization-wide](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file) [`.github/ISSUE_TEMPLATE`](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms) with different templates/types of issues to report +- [ ] Setup https://github.com/{organization} [organization-wide](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file) [`.github/ISSUE_TEMPLATE`](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms) with different templates/types of issues to report - [ ] DLP: add a button "View/Report Issues" or "Issues" leading to issue Tracker (e.g. [Issues](https://github.com/dandisets/000108/issues) for 000108) - Complimentary: dandi-archive could query/cache number of known open issues and include within that button, e.g. have it [(1) Issues](https://github.com/dandisets/000108/issues) - Complimentary: asset-level issues filing in UI context menu per file, would pre-fill in the asset path From 9e250927e17af386f82e9aad5fe917c1887e20cc Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 18 Nov 2024 13:22:23 -0500 Subject: [PATCH 7/7] Add more of potential future integrations --- doc/design/dandiset-issues-tracking.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/doc/design/dandiset-issues-tracking.md b/doc/design/dandiset-issues-tracking.md index e8a24832f..78cdce967 100644 --- a/doc/design/dandiset-issues-tracking.md +++ b/doc/design/dandiset-issues-tracking.md @@ -84,3 +84,15 @@ Although sounded like a good idea at first, cumbersome to implement with consist - A possible mitigation: I guess we could collect mirror issues/comments etc from GitHub internally in the archive. There are tools which could even be used to help. E.g. @yarikoptic has experience with using https://github.com/MichaelMure/git-bug to sync all issues from GitHub locally to collect all contributors to the project. E.g. [this script](https://github.com/nipy/heudiconv-joss-paper/blob/main/authors/tools/make-summaries#L92) processes a JSON dump of all issues from `git bug` mirror. [bot]: https://github.com/dandi/dandisets/issues/360 + +### Miscellaneous other possible future integrations with GitHub + +- 2i2c uses github teams for resources management on the hub (ref: 1.1 within https://github.com/dandi/dandi-hub/pull/90/files#diff-82655098d9fb488babf6a5ce10d3d5f6a98d17b2f69de5ca28315e54e020bdf9R29) + +- Provide facility to propose changes to dandiset metadata: + - meditor would allow "Edit" operation on draft for any registered user + - upon edition, + - fork would be created (if does not exist), + - PR would be submitted with changes + - author notified/review and potentially merge + - CI or some bot should fold those changes into actual metadata and remove the merge commit?