feat: expose plugins and changelog in the API and app #744

kabaros · 2024-12-03T07:39:01Z

This adds functionality to show the changelog in the UI and visually differentiate plugins in the list of apps. Implements HUB-156 and HUB-144.

From a UX perspective, both these functionalities will be improved in upcoming tickets, but this PR sets the ground for how we do it, while exposing a very basic UI to test the functionality.

After a first discussion, we decided to move d2config and changelog columns to the app table instead of appeversion - in addition, we added a change_summary column to versions table that would allow users to highlight what changed in a specific version. The d2config is taken from the uploaded zip file - while we attempt to get the change log from the zip file or the GitHub source url. Once we have the full change log - if we have it - we can display it in the UI in numerous ways, for now, we just implemented the simplest solution to show the full change log

The workflow is as follows:

When uploading a new version or a new app, we save the d2.config.js (as a JSON type) and CHANGELOG.md to the app table.
- If we're uploading a new version, we patch the app entry with the d2config and changelog from the last version.
- This means we only have the last version of these files - this is ok because this is the information we'd be interested in in this context, i.e. an app will be shown as a "plugin" if the latest d2config has a plugin entry, it doesn't matter if it did or did not have such an entry in the past. The same for the changelog - with the addition that we can infer all history from one (good) changelog and we also didn't want to have a massive text file saved with each version unnecessarily
An indicator has_plugin is added to the app_view view. has_plugin inspects the JSON type and returns a flag whether a plugin entry point is found or not. This is returned to /apps endpoint and /apps/:id.
- the whole d2config - rather than just the plugin info - is saved as it could be used to infer other information (i.e offline support, RTL support etc..).
- if / when the entryPoints logic change (for example, to have multiple plugins), then the logic in the view can be updated.
Both d2.config and changelog.md are not returned fully as part of apps or appversions endpoints - only an indicator that they exist, or inferred info (i.e. has_plugins)
A separate endpoint to get the changelog is added /apps/:id/changelog. It is separate to avoid returning a potentially large text file in the apps or apps/:id response. We only return an indicator, which the UI can use to show a "view changes" button, and when that is clicked, it calls the API and returns the changelog.
- The changelog returned is for the app - that is all the app versions for that app. The UI, then, can make decisions based on that - for example, one changelog from a different version likely includes the changes from previous versions too. This minimises the need to have a script to backfill old versions change logs, as once a new version is added - it will contain all the changes for previous versions too (at least for apps that follow conventional change logs, like our internal apps), but it needs to be limited as well potentially as more versions with changelogs are added
We look for the changelog in the uploaded version zip file - in most cases (in our apps), it's not part of the bundle though. So we also try and get the changelog from the app github url if that exists.
- This is pretty crude at the moment, for example, it works with changelogs in master branch only (might add a second check for main branch)
- There is also a new field change_summary added to the version table where users can just highlight what changed in a specific version manually - this is not exposed yet in the UI.

ToDO :

I have a script to backfill previously added apps (by reading d2config and changelog) from GitHub - still undecided if we need to do that or not. With the current approach, we will get the changelog once a new version is uploaded, so it doesn't seem like this
[future] add ability to manually input a "what changed section" to the version info. This is the `change_summary_ field in the versions table, and can be used along the change log or instead of it.

UX

The UX, especially for the Changelog, is a placeholder - once we have the changelog info, we can parse and display as we want. I want a simple version in first, and that we agree on the database and API structure.

But here are some UX questions that came to mind while working on this:

now that there is a "plugin" tag, it makes the "Standard" tag less obvious - shall we change that to a Tag too? what does "standard" even mean?
how to filter plugins? right now, I added a "backdoor" where the user can search the word "plugin" and that will show all plugins
how to word the fact that an app is a a plugin - technically it has a plugin (or multiple plugins), but it is can also be an app
....

Screenshots

kabaros · 2024-12-03T07:41:45Z

@cooper-joe this is a very early version of the changelog (+ highlighting plugins) in the App Hub. My plan is to follow up with UX improvements once the basics are working, but wanted to keep you in the loop (and also, maybe - even in this basic version - there are some UX wins that can be gained)

kabaros · 2024-12-10T07:11:32Z

server/src/data/getApps.js

@@ -50,6 +50,9 @@ const getApps = (
                builder.where((builder) => {
                    builder.where('name', 'ilike', `%${query}%`)
                    builder.orWhere('organisation', 'ilike', `%${query}%`)
+                    if (query.match(/plugin/i)) {


an obscure way to find plugins - by searching for the exact word "plugin". A backdoor until we decide how/if we expose finding plugins in the UI.

kabaros · 2024-12-10T07:13:09Z

server/src/data/patchApp.js

@@ -0,0 +1,43 @@
+const joi = require('joi')
+
+const paramsSchema = joi


the existing updateApp did too many things and I didn't want to use it - although this in theory can be used to update any properties in the app object, for now, I am restricting, through the joi validation, it to only the d2config and changelog.

kabaros · 2024-12-10T07:21:24Z

@amcgee @Birkbjo this is ready for review now

@Birkbjo is it possible to deploy this PR to staging?

kabaros · 2024-12-10T10:17:17Z

created a next branch and changed the base branch for this PR - PR will get deployed to staging once merged to next

amcgee

Added a few comments. In particular curious about these two points which I may be misremembering where we landed in earlier discussions:

Exposing not just hasPlugin but also pluginType in view and API
Storing d2config per version (so that we can track changes to app d2config over time)

client/src/components/Versions/VersionsTable/VersionsTable.js

amcgee · 2024-12-16T12:28:18Z

server/src/routes/v1/apps/formatting/convertAppsToApiV1Format.js

@@ -15,6 +15,9 @@ const convertDbAppViewRowToAppApiV1Object = (app) => ({

    name: app.name,
    description: app.description || '',
+
+    hasChangelog: app.has_changelog,
+    hasPlugin: app.has_plugin,


Should hasPlugin also expose the pluginType from d2.config.js? That's an immediate win since we have both dashboard and capture plugin types available to day.

exposed plugin_type, and now the Plugin tag shows the plugin type if it exists

server/migrations/20241118143001_d2config_changelog_columns.js

dhis2-bot · 2024-12-16T20:31:01Z

🎉 This PR is included in version 2.35.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

kabaros requested review from Birkbjo and tomzemp December 3, 2024 07:39

dhis2-bot added client server labels Dec 3, 2024

kabaros requested review from amcgee, d-rita and KaiVandivier December 3, 2024 07:39

feat: expose plugins and changelog in the API and app

366653d

kabaros force-pushed the feat/show-changelog-and-plugins branch 2 times, most recently from ac4b7e7 to 9955193 Compare December 10, 2024 06:41

kabaros requested review from a team and removed request for d-rita, KaiVandivier and amcgee December 10, 2024 06:41

kabaros force-pushed the feat/show-changelog-and-plugins branch 2 times, most recently from bd14cde to 6cf7371 Compare December 10, 2024 07:09

kabaros commented Dec 10, 2024

View reviewed changes

kabaros force-pushed the feat/show-changelog-and-plugins branch 2 times, most recently from 51c2b02 to 1444a09 Compare December 10, 2024 07:18

refactor: move changelog to app rather than appversion

2e517e3

kabaros force-pushed the feat/show-changelog-and-plugins branch from 1444a09 to 2e517e3 Compare December 10, 2024 07:19

kabaros marked this pull request as ready for review December 10, 2024 07:21

fix: bump node version in docker image

e966a6c

kabaros changed the base branch from master to next December 10, 2024 10:16

refactor(changelog): check both main and master branches for changelog

cf2b128

amcgee reviewed Dec 16, 2024

View reviewed changes

kabaros added 4 commits December 16, 2024 19:05

refactor: move d2config to appversion

f1b94f9

refactor: expose pluginType for plugins

abd64a4

refactor: remove unnecessary modal property

973ddbf

feat: add organisation overview page

8bc5c3f

kabaros merged commit 94b5dfb into next Dec 16, 2024
5 checks passed

kabaros deleted the feat/show-changelog-and-plugins branch December 16, 2024 20:29

dhis2-bot added the released on @next label Dec 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: expose plugins and changelog in the API and app #744

feat: expose plugins and changelog in the API and app #744

kabaros commented Dec 3, 2024 •

edited

Loading

kabaros commented Dec 3, 2024

kabaros Dec 10, 2024

kabaros Dec 10, 2024

kabaros commented Dec 10, 2024

kabaros commented Dec 10, 2024

amcgee left a comment

amcgee Dec 16, 2024

kabaros Dec 16, 2024

dhis2-bot commented Dec 16, 2024

		@@ -0,0 +1,43 @@
		const joi = require('joi')

		const paramsSchema = joi

feat: expose plugins and changelog in the API and app #744

feat: expose plugins and changelog in the API and app #744

Conversation

kabaros commented Dec 3, 2024 • edited Loading

ToDO :

UX

Screenshots

kabaros commented Dec 3, 2024

kabaros Dec 10, 2024

Choose a reason for hiding this comment

kabaros Dec 10, 2024

Choose a reason for hiding this comment

kabaros commented Dec 10, 2024

kabaros commented Dec 10, 2024

amcgee left a comment

Choose a reason for hiding this comment

amcgee Dec 16, 2024

Choose a reason for hiding this comment

kabaros Dec 16, 2024

Choose a reason for hiding this comment

dhis2-bot commented Dec 16, 2024

kabaros commented Dec 3, 2024 •

edited

Loading