User extension dependencies - a packet manager?

[D8H]

Extension dependencies

User extension dependencies could allow to build complex extensions over existing ones.

Extension maintenance issues

Extensions interface may change from one version to another and this could break extensions relying on them. This problematic can be answered by a packet manager. But all extensions will be seen and used by the user. If several extensions versions are installed on the same time, I guess it could get really messy. In case of behaviors, functions are scoped to an object so if the 2 versions are on a different object it should be fine though.

• Do you think GDevelop will need a versioned extensions dependencies manager?
• Should this wait some other features to have a better view on the extension system?

A related discussion on backward compatibility: #2187

[arthuro555]

I was thinking about this a few weeks ago as well. I think a versioned package manager would definitely be the way to go.

[arthuro555]

I thought a bit more about this, and I can think of 3 main solutions for dependency management:

1. "Automatic" downloading of dependencies: When an extension is installed, run through the events sheets and download any missing extension (the extension can be detected through a naming convention ExtensionName::InstructionName for the internal name).
2. "Manual" downloading of dependencies: Each extension has to list explicitly any extension it depends on (and maybe its version if we add versioning to the extensions repository), which will be downloaded on installation.
3. Bundling of dependencies: At the time of exporting the extension, all required events functions would be copied into private member functions of the extension (or the whole other extensions can be embedded in the JSON file of the extension).

Here is a little comparison table (1 and 2 are similar enough to be grouped as the same):

Challenge	Dependency downloading	Bundled dependencies
Offline support	➖ No direct offline support, as the extension by itself is incomplete and requires the download of further extensions. Technically, those could be downloaded while online alongside the extension and installed before the other but we cannot really expect users to think about this and it would be very cumbersome especially if some extension have many dependencies that also have dependencies that also have....	✔️ Extension can be installed offline, as all necessary code is present in the bundled extension.
Correcting a bug in a dependency as a user	✔️ The extension is present normally and can be patched easily.	➖ The dependency is bundled and therefore "obscured", making it harder to patch a bugged function of a dependency.
Patching a dependency for supporting the current extension as an extension dev	❌ A fresh install of dependencies is performed every time the extension is installed, changes on the dev side to a dependency are not taken into account.	✔️ The modified extension is bundled into the extension on export, so the patches will be present for the user. The changes would also not break or impact other functions depending on the modified extension as each have their own unique copy of it.
Having multiple extension that depend on different versions of the same dependency	➖ It is possible to have multiple versions of a dependency, but that would require downloading it multiple times, renaming it to - and updating the extension depending on it to use the renamed version of it. It is rather cumbersome to implement and would add a lot of duplication bloating the extensions list though.	✔️ As all extensions contain their own copy of their dependencies, there is no need to manually handle multiple parallel versions of extensions.
Having an extension that interacts with an events based behavior	✔️ Works without problems.	❌ This is more problematic, as unlike functions behaviors cannot be made into a private copy. A copy would not reference the correct behavior and therefore simply not work without major changes.
Exported code size	✔️ Generates the least amount of code needed.	➖ If two extensions use the same dependency, that same dependency will be generated twice increasing the code size.
Project size	➖ Doesn't take too much space as sharing some functions definitions.	➖ Can take less space as a whole extension does not need to be downloaded to use one function of it, but can cause duplicate functions to be present.
Releasing a patch for an extension	✔️ A patch for an extension bug or vulnerability once published can be easily installed and effect everything.	❌ Once a patch is released, all dependent extension need to be updated too to use the new version of the dependency and users need to update all those extensions that depended on in the patched dependency.
Stability	❌ An update to an extension can break other extensions if they relied on it.	✔️ The extensions will always use the exact code they were developed with, assuring that it will work as intended.

[arthuro555]

We could also offer some kind of compromise between both options.

For example, when exporting an extension one could chose between an "export for the extensions library" option (which would export it without bundled dependencies, as we can expect a user to be online when downloading an extension through the extensions library) or an "export bundled extension" option (to export an extension with all dependencies bundled).

We could also bundle the full extension of all dependencies, and install the dependencies alongside the dependent extension when installing it.

[4ian]

I would tend to keep a flat structure of extensions:

• Way simpler to handle and avoid huge problems like the diamond problem or duplication of a different dependencies with different versions that might fight over global variables or worse (if A depends on C v1.0 and B depends on C v1.1, maybe C is not made to have multiple versions of it living in the game. Probably not at all).
• When you install an extension from the store, you most probably have internet. The extension store will install whatever extensions are necessary for the one you're installing.
• When you export an extension, we can as you suggest give an option to export an extension and its dependencies otherwise. So that you could install the extensions one by one if you don't have internet (or we bundle everything in a zip). And we warn you if you install an extension from a file with a missing dependency (either install it or risk your game being broken. Or we just don't allow the installation of the extension).