Skript Package Manager

[Moderocky]

Summary

Create a remote package-manager system that can be implemented by third-parties to deploy managed dependencies (addons, scripts) with compatibility and version checking, amongst other tools. Skript itself would provide no implementation for this and it would be an 'optional extra' for users that regularly use third-party resource sites like Spigot's resource section or skUnity resources.

Goals

• To create a package manager API for remote third-party resource managers to implement.
  • Provide an API data schema for a manager to be made in any language
  • Provide specific rules these managers have to follow in order to reduce safety risks
• To make using third-party tools as safe and easy as possible.
• To decrease the risks of unsafe or broken dependencies.
• To provide clear information about compatible resource versions and their dependencies, etc.

Motivation

The current Skript ecosystem is relatively unmanaged and quite toxic given the lack of metadata and tools: a user doesn't know whether a downloaded script is going to work until its run, and has to find all the required dependencies (and specific versions! themselves.)

Java has the advantage of package managers like maven, combined with insights from the compiler to check that a dependency is actually available/valid/usable.
Skript does not.

Bukkit plugins have the advantage of a plugin.yml with dependency requirements, specific API versions, etc.
Skript does not.

Description

A new config file resources.sk would contain resource information. By default it would be empty/unused (to make it an opt-in feature.)

This would have two main sections: package managers and dependencies.

Package managers would have an id: url format. They would be searched for dependencies in the order provided.
Multiple managers could be specified (e.g. spigot, skunity, etc.)

package managers:
    example manager: http://example.org/resources/
    # 'example manager' is the ID used for error messages, etc.

Dependencies can have multiple entries, though most are optional. Script dependencies are loaded in the order provided (to prevent conflicts) and before user-created scripts in the scripts/ folder, so that their functions/custom syntax/whatever can be used.

dependencies:
    my-cool-script:
    # the script id, searched for in the package managers
        type: script
        # the type of this entry (addon/script)
        source: example manager
        # optional, whether to search only a specific manager
        version: 1.0.0
        # optional, whether to get a specific version, or find the latest compatible
    skript-gui:
        type: addon
       # an example addon dependency

Dependencies would be downloaded and stored internally (to avoid conflicting with scripts in the scripts folder) with their hashes/information cached.
If a dependency uses a static version the local copy can be used indefinitely. Skript can check package managers for updates for dependencies and prompt users when dependencies need updating. This should not be done automatically, since it could pose a security risk. Instead, server admins should be given a prompt on start-up to check the new version before clicking a chat-command to update.

The package manager will provide metadata such as details for required transitive dependencies (e.g. my-script requires the addon skquery.) The user will be prompted to add these.
The package manager will also provide compatibility data: i.e. is this addon version actually compatible with the user's Minecraft version, Skript version, etc. Users can be warned about incompatibility and given a hint of how to fix this.

On the provider end, resource data would be accessible via a common HTTP API and script/addon data transmitted directly as a binary when requested. Users would not need to interact with this API at all, other than providing the base URL for the package manager.

Caveat: I believe Spigot does not like plugins downloading other plugins automatically (as addon dependencies would do) however since Skript is not on Spigot, this shouldn't be much of a concern for us.

Risks

1. Obviously, any third-party code comes at the risk of malicious code and attacks.

This has been the reason given for SkriptLang not providing any sort of script download system before: if we are distributing it, it's our responsibility to make sure it's safe.
Instead, my suggestion puts the responsibility on the third-party package manager (distributor) and the user to determine whether things are safe or reliable.

Solutions: If security is a concern, we could licence only specific vendors to be package managers if they comply to safety standards (e.g. reviewing resources - which Spigot doesn't) and following other safety regulations. This would make the system safer but put more burden on us to authorise these.
That said, I honestly don't believe it is any riskier than the current free-for-all ecosystem (and likely less so.)

2. Remote package managers may be unavailable (downtime, no internet connection, etc.)

If a package manager isn't available (or a server can't connect) then obviously it cannot download remote dependencies, check for updates, etc.

Solutions: Downloading/caching resources to some internal storage is probably the best option for this. The user can also be warned on start-up or prompted to download a manual version for the time being.

3. A large number of dependencies could increase load times.

If a user has a lot of remote dependencies, this could slow down load times while things are checked, updated, downloaded, etc.

Solutions: Caching dependencies/hashes will make sure dependencies don't need to be re-downloaded on every load.
They also do not need to be downloaded/updated in-sync: the server could load as normal, while downloading them in the background, and once complete the user could be prompted to reload.

[AyhamAl-Ali]

I really like this idea, it takes Skript into another level of being a reliable programming language used in Minecraft server.