Prepare binary plugins text for AGM

[mbernasocchi]

Import Initial points from various discussion:

Dear community voting members, last year discussion on allowing platform specific plugins in the official QGIS’ plugin repository created an interesting discussion with both pro and contra arguments [1].

The PSC agreed that due to the very split nature of the positions a community vote is needed.

We hereby prompt you to vote on the following items before February 7th 2020 at
https://forms.gle/AAhG1hGQuU5u2WLW7

• Plugins should not be platform specific
• Plugins should not be allowed to include any binaries
• Plugins should not be allowed to include proprietary binaries
• Plugins should not be allowed to fetch binaries post installation

[1] https://lists.osgeo.org/pipermail/qgis-developer/2019-February/056114.html

[jef-n]

Message (still) looks truncated…

[mbernasocchi]

yep, because I haven't finished writing it yet :)

[pcav]

Previous discussion: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-Windows-only-version-of-a-QGIS-2-18x-plugin-tt5394159.html#none
Issue summary:

• Binaries in plugin - majority in community don’t approve
• Platform specific plugins - most in community don’t seem to mind (Paolo strongly objects to “proprietary OS only” plugins e.g. ones that run only on Windows or Mac)
• Plugins requiring proprietary binaries - Paolo strongly objects
• Plugins fetching binaries post installation - Paolo OK with cases where binary being fetched is built from FOSS and the plugin will work on all platforms

[mbernasocchi]

@qgis/psc please proofread and comment

[wonder-sk]

What is not really clear from the question is how we want to deal with binary dependencies which are FOSS. For example many third party python modules (that may be used by plugins) use compiled binaries. It is common that they are distributed with plugins and this would essentially prohibit their use. Not sure if that is intended or if only non-FOSS binaries are meant to be excluded?

[mbernasocchi]

@wonder-sk thanks a lot, indeed it would kill wheels as well. I updated the formulation.

[wonder-sk]

For the voting I would suggest to add some extra context for each question, so that voting members better understand what is being voted - many of them are not developers. Maybe it would be good to name some existing QGIS plugins that would be affected.

Also it is worth mentioning the consequences. If any of the questions gets voted YES by the majority, we will have have to remove some plugins and make the life more complicated for users and developers. For example, if we would prohibit Windows-only plugins. Who wins? Not sure really...

[mbernasocchi]

thanks @wonder-sk I'm completely with you. We want to spec out much more the text to make sure that anyone has the full context and implications for each vote. @timlinux, myself and ideally @pcav (as the champion of this vote) will work on it and we'd really like your feedback.

[pcav]

Thanks for commenting, I'll add some context. Of course, help is appreciated.

[pcav]

Proposed text with context; very biased currently, please add different views:

• Plugins should not be platform specific
  The aim of QGIS.org is to freely empower all users; platform specific plugins limit functionalities to a subset of users. Furthermore, clean coding is usually multiplatform.

• Plugins should not be allowed to include any binaries
  Distributing binaries along with plugins does not allow us to examine it, and is a serious security risk.

• Plugins should not be allowed to include proprietary binaries
  Same as above, plus the issue of doubtful licencing.

• Plugins should not be allowed to fetch binaries post installation
  This could raise the same security issues as above. The main difference is that the user can be warned and decide whether the download is acceptable in his context.

[mbernasocchi]

Hey @pcav

I think we need to vote on unbiased texts.

• The text should be neutral or at least bring both sides of the argument.
• what does "should not" mean? is it a hard rule or we suggest not to do it?
• the test should be turned around, it is much harder to vote on negations. I suggest: the plugins are allowed to be ...

[m-kuhn]

I think we need to separate:

• Text we vote on
• Pro arguments
• Counter arguments

[mbernasocchi]

like the swiss votes, I like it.

For example:

Vote 1

Plugins are allowed to be platform-specific

Arguments in favor:

Arguments against:

The aim of QGIS.org is to freely empower all users; platform-specific plugins limit functionalities to a subset of users. Furthermore, clean coding is usually multiplatform.

Vote 2

Plugins are allowed to include FOSS binaries

Arguments in favor:

Arguments against:

Distributing binaries along with plugins does not allow us to examine it, and is a serious security risk.

Vote 3

Plugins are allowed to include proprietary binaries

Arguments in favor:

Arguments against:

Same as above, plus the issue of doubtful licencing.

Vote 4

Plugins are allowed fetch binaries post installation

Arguments in favor:

Arguments against:

This could raise the same security issues as above. The main difference is that the user can be warned and decide whether the download is acceptable in his context.

[mbernasocchi]

@wonder-sk, @m-kuhn something like this?
can you help out speccing out the in favour arguments?

[mbernasocchi]

should we change vote 2 to "Plugins are allowed to include FOSS binaries? @pcav what do you think?

[pcav]

should we change vote 2 to "Plugins are allowed to include FOSS binaries? @pcav what do you think?

+1

[m-kuhn]

Is someone able to provide a list of plugins affected by these decisions as requested by @wonder-sk ? Or a list of plugins that have not been allowed in the past because of these rules?

Once we have this list, I can add a couple of lines of advantages that a loose policy will offer for end users, expected effects of a hard policy and a comparison with the situation of some drivers in the osgeo4w installer (oracle, MrSID, ECW, ...).

[pcav]

like the swiss votes, I like it.

I like this approach, it goes exactly in the direction I was suggesting.

[pcav]

I do not have an easy way to find them. However, these are very exceptional cases. This is one of my arguments against relaxing these rules: adding risks for extremely occasional advantages, if any.

[m-kuhn]

Even if they are exceptional, it would be good looking at them to know what we discuss. I think politics and rule making should be different to shadow boxing.

[pcav]

fro windows-only plugins I can only recall one including one specific krigiging alg, implemented as a windows binary.

[m-kuhn]

What was so bad about this one that we need to ask 4 questions about it?

[pcav]

IMHO, breaking a quite reasonable rule just for one (possibly) missing option in a well known alg is not justified.

[m-kuhn]

Draft proposal, updates welcome

Plugins are allowed to be platform specific

QGIS.org wants to give users the freedom to use QGIS the way they want and to share what they did. By allowing users to share plugins even if they are platform specific, these can be used by others on the same platform. Everyone else has the possibility to take these plugins and adapt them for more platforms. By allowing plugins to be platform specific we also open up the possibility to ship plugins which give access to specific tools like the touch bar, a hardware piece that is only available on mac.

Plugins are allowed to include FOSS binaries

Many of the tools we use in everyday life are delivered in binary format. QGIS, Python, the operating system and much more are installed from binary packages on most systems. This allows everyone out there to benefit from these applications and libraries without using a compiler.

"Plugins are allowed to include proprietary binaries" and "Plugins should not be allowed to fetch (proprietary?) binaries post installation"

Plugins are a way to integrate QGIS with custom business logic, external applications and hardware. Some tools are not open source. We want to empower QGIS users to integrate QGIS in the most flexible way possible with the best tools for their workflow. This already applies for data providers, where we integrate with MrSID, Oracle, ECW, Excel files and other proprietary formats and databases.
Every plugin is required to comply with licensing requirements and inform the user about their rights and duties, this applies for proprietary as well as open source plugins.

General

The pro committee wants to give users the freedom to use QGIS, their information and their computers the way they want. We are convinced that open source has a very strong backup in the QGIS community and that we can better encourage developers and users to foster open source by leaving them the freedom rather than putting restrictions in place.
We are convinced that it is the right thing to give developers the possibility to share their work, to transparently communicate to a user what's inside their plugins and to give each individual user the power to decide what's appropriate for him.

Note: the above statements do not necessarily reflect my personal opinion!

[andreasneumann]

I also gave this some thoughts. Clearly we don't want to

• encourage plugins that only work in one OS
• encourage plugins containing binary software

On the other hand we don't want to completely forbid them, if there are really good reasons.

What if: in general we don't allow both, but if we really need to do it, we organize a voting on the individual exception. The plugin author would have to explain clearly, why it is a good idea to have such a plugin that works only in one OS (like the Mac touchbar thing), or why it is impossible to distribute the plugin without a binary. And such plugins, esp. the ones containing binary software would require extra reviews by more than just one person. I don't know what would be the right group to vote on the exception? Core devs? PSC? Voting members?

That way we send out a message that we want the plugin authors to do everything they can to support all platforms and to make all source code available, while still allowing exceptions.

Otherwise I fear a bit, like Paolo, that we get flooded by Windows only plugins. We need to have some barrier that prevents plugin authors from thinking - ok, now I can easily ship Windows only plugins. Since I don't care about Mac or Linux I can now save time and do everything Windows only.

[m-kuhn]

Please let me add some personal thoughts, next to the pro arguments above:

I am clearly in favor of allowing platform specific plugins -- with a recommendation to make them cross platform. Who gives us the right to request something else? I think we should be grateful for all the work which is shared. If someone wants more he is free to contribute. And it's not an issue anyway with the current case count of 1. Open source spirit.

I am also clearly in favor of plugins containing pre compiled floss binaries. Or we should stop using numpy wheels right now. (I am biased, we develop QGIS Model Baker which post loads a LGPL licensed library called ili2db, shipping this directly in the plugin has been discussed several times.)

On the other questions about regulating proprietary binaries for plugins I am undecided. I can live with a regulation to not distribute them through our plugin infrastructure. But I would like to have guidelines on what to do instead. What I don't want is uncertainty with developers "I can't use QGIS for this project because I need to use a proprietary hydrological calculation engine within it". And I don't want people having to use google to find funkylib.dll on spurious websites.

[jef-n]

• I'd also be in favor of allowing plugins that only work on one platform. Code working on one platform is better than no code at all. And plugins are GPL. Other interested parties can still pickup and add support for other platforms. There may also might be plugins, that only make sense on a single platform.
• Shipping binaries of free software shouldn't be a problem.
• Proprietary binaries are in a grey area. If something is publicly available anyway and a widely used solution (eg. because it's the recommended tool for a certain job by a local authority), then providing a convenient way to use it in QGIS is a good thing. On the other hand binaries could also be used to conceal how a plugin works. But if it's an integral part of the plugin, this should be covered by the GPL already.

[timlinux]

Can I suggest to pose questions rather than statements? I also tweaked the wording from others above in this thread to make the questions more natural and removed spurious or emotive statements.

Vote 1: Should platform-specific plugins be hosted in the QGIS plugin repository?

Arguments in favor:

QGIS.org wants to give users the freedom to use QGIS the way they want and to share what they did. By allowing users to share plugins even if they are operating system specific, these can be used by others on the same operating system. Everyone else has the possibility to take these plugins and adapt them for more platforms. By allowing plugins to be platform-specific we also open up the possibility to ship plugins that give access to specific tools like the touch bar, a hardware piece that is only available on mac.

Arguments against:

QGIS.org would like to provide as similar as possible experience across all operating systems. Platform-specific plugins limit functionality to a subset of users.

Vote 2: Should plugins hosted in the QGIS plugin repository be allowed to include FOSS binaries?

Arguments in favor:

Some python packages are actually written in C and compiled as python modules. Shipping these as pre-built binaries will allow users of a plugin to use it without using a compiler.

Arguments against:

Distributing binaries along with a plugin prevents our understanding of what that binary will do on systems where it is deployed, which poses a potential security risk.

Vote 3: Should plugins hosted in the QGIS plugin repository be allowed to include proprietary binaries?

Arguments in favor:

Plugins provide a way to integrate QGIS with custom business logic, external applications and hardware. Some third party tools may not be open source. We want to empower QGIS users to integrate QGIS in the most flexible way possible with the best tools for their workflow. There is already precedence in QGIS core with our data providers, where we integrate with MrSID, Oracle, MSSQL, ECW, Excel files, and other proprietary formats and databases.

Agreement with this proposal should not be seen as an end run around the fact that every plugin is required to comply with licensing requirements and inform the user about their rights and duties, this applies for proprietary as well as open-source plugins.

Arguments against:

Distributing binaries along with a plugin prevents our understanding of what that binary will do on systems where it is deployed, which poses a potential security risk. Proprietary binaries are even more opaque than open-source ones (which at least allow back-referencing the source project to view the code that was ostensibly used to compile the binary). Understanding the licensing implications of proprietary binaries used by plugins requires more work for our plugin reviewers to investigate whether the binary can legally be used in this manner.

Vote 4: Should plugins hosted in the QGIS plugin repository be allowed to download binaries onto your computer post-installation?

Arguments in favor:

For resource management reasons and concerns about providing a good user experience, we limit the size of plugins that can be hosted in the QGIS plugin repository. Allowing post-installation fetching of binaries allows larger resources to be included in a plugin but not uploaded to the plugin store.

Allowing post-installation fetching of binaries means that a plugin using binary components does not need to ship binaries for all supported operating systems within the plugin bundle. Rather the operating system specific binary can be retrieved post-installation.

Arguments against:

There are security concerns about plugins fetching arbitrary code from the internet and running it on our user's computers. Users should be warned before anything downloads to their computer so they can decide whether the download is acceptable in his context or not.

[3nids]

May I suggest to reformulate:

we also open up the possibility to ship plugins that give access to specific tools like the touch bar, a hardware piece that is only available on mac.

by

we also open up the possibility to ship plugins that give access to specific hardware or libraries which are only available on some platform (e.g. the touch bar is only available on mac; EPANET an hydraulic simulation tool is only available on Windows)

I know we're mixing 2 concepts here: platform dependent and binary linking but I think it helps understanding the topic and avoid people think it's going to be for mac users only

[3nids]

If we choose to shoot a bullet in our foot (by enforcing strict(er) rules), could we consider:

• we extend metadata to hold this information
• by default, we filter out the ones we don't like similarly to what is done for experimental right now
• we let power users enable it

I think the vote is already complex enough, but I would see it as an additional global question:

If one of the above questions is answered by "NO", would you be willing that the plugins repo still offer the hosting to these plugins, but flag them properly and still let users downloading them by explicitly requiring access to these plugins in the settings?

[vpicavet]

I think we miss the security aspects and the legal one here.

As for security and related responsibility : if QGIS.org hosts binaries and distribute them, the end-user should be able to trust them not to have any security problems. For this we need at least : 

• A/ access to the source code for the distributed binaries
• B/ a reproducible build process to generate said binaries
  As for B, we should in fact generate the binaries ourselves, and not allow upload of binaries from contributors to the plugin repository. This would really be risky. We have a large user base, and they are not necessarily aware of security risks generally. We therefore have to be very cautious here.
  For software deployment systems, the norm is generally now to rely on verified builds ( Public key signatures) from verified contributors. Maybe we do not want to go this far, but allowing for binaries allows for a whole new range of risks for QGIS.org and QGIS users alike.
  The security problem is true for opensource software ( if we don't have B ) and proprietary software ( A and B not possible).
  Post-plugin installation download and install is also a risk, but no more than other softwares, and but not endorsed directly by QGIS.org, but more by plugins contributors.

Distributing binaries also may have legal consequences : without A and B above, how do you certify that licences are respected ? Who will be in charge to assess licence policies for binary distribution ( opensource & proprietary) ? That person will represent QGIS.org and be responsible for all related risks, we should have a proposal before any decision.

As for legal, there is also another aspect, which is a grey area : our plugins are all GPL, as an import QGIS is considered a link in the meaning of the GPL licence. There have been some court decisions stating that if a GPL software cannot run without a specific proprietary module, then the software does not respect the GPL clause. This is a complex part of OpenSource IP, but it has to be taken into account. There is a risk here, and it could be taken by individual plugin contributors, but should not be taken by QGIS.org without analyzing all legal implications.

That said, did we study alternatives ? As soon as we ship binaries, we are starting building a software deployment system. Software deployment is really complex, and there are already multiple existing solutions for that : Python pip/wheels/pypi, OSGeo4W are among the tools we should probably favor for complex installation and deployment, particularly when there are binaries and dependencies. I have the impression that we are adding features leading to re-inventing the wheel ( no pun here) in a very low-quality way, while adding a lot of risks and complexity in our core system.

I also fail to see the touch bar argument : there is a real difference between having a plugin with some code specific to one platform ( like touch bar for quick access to plugin features), and a plugin which is platform-dependent and available only for one specific platform.

We should also identify for each issue mentioned, the impact on our plugin management system, how much development that would require and who would take care of the changes.

For what it's worth, my opinions here :

• no binary at all in QGIS.org plugin repository ( prorietary or opensource)
• out of question to host any proprietary software in QGIS.org
• allow for platform-specific plugins as an exception to the rule : platform-specific plugins can be integrated into QGIS.org respository if they provide a rational argument for the limitation to a specific platform (e.g. a feature impossible to implement on other platforms)
• use the right tool for the right use case : investigate into software deployment solutions and use a proven solution instead of trying to re-invent one for QGIS
• [Edited] I do not think the context and global prehension of the issue is advanced enough to allow for a vote