-
Notifications
You must be signed in to change notification settings - Fork 123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updating the PowerShellGet 3.0 RFC #237
Conversation
updating master
@SydneyhSmith please add a change log to your comment for what has changed from the original. |
Hi @SydneyhSmith Is this one already available for testing? I have a strong business case for this one, and I Couldn't find anything on the gallery. |
Not yet, we are currently implementing it and targeting a public preview on the gallery (not feature complete) for the end of this month (Jan 2020)--thanks for the patience! |
Great, looking forward to it @SydneyhSmith There has been quite some disucssions on this topic here -> https://github.com/PowerShell/PowerShellGet/issues/214 and I had the impression this was taken into consideration for version 3.0 of the module. |
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
This module will depend on https://www.nuget.org/packages/NuGet.Client. | ||
This module would be shipped in the PSGallery supporting older | ||
versions of PowerShell. | ||
This module will ship in PowerShell 7.1. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please clarify: Will it ship in addition to the exiting PowerShellGet 2.x or be a replacement? I understand they can work side-by-side but knowing what the 'out of the box' modules are is important for CI scenarios where installation of certain modules is assumed. Since the cmdlet names are different, removing PowerShellGet 2.x would be a big breaking change
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The plan is to ship in addition to PowerShellGet 2.x for the 7.1 timeframe, and before that we will be shipping it in the Gallery as a standalone module to work out issues before we ship in PowerShell
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
|
||
### Local cache | ||
|
||
Instead of always connecting to a repository to perform an online search, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whilst this would be nice I do not see it being required for the first minimum, viable version.
@Stephanevg thanks for the question, unfortunately this is not something that will ship in the first version of PowerShellGet 3.0, but is something we will highly consider for a vNext...I will update the RFC to clarify this point--sorry to disappoint! |
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
On any operation that reaches out to a repository, a REST API will be called to | ||
see if the hash of the cache matches the current cache and if not, a new one | ||
is downloaded. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure that this is the best idea....we might want to do something similar to how the update notification feature works (check a file to see if it's already been done in the last 24 hours)
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
`-Trusted` is a different parameter set from `-Priority`. | ||
|
||
By default, if this switch is not specified, the repository is untrusted. | ||
A `-Repositories` parameter will accept an array of hashtables equivalent to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-Repositories
should be Repository
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
### Dependencies and version management | ||
|
||
`Install-PSResource` can accept a path to a psd1 or json file (using `-RequiredResourcesFile`), | ||
or a hashtable or json (using `-RequiredResources`) where the key is the module name and the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
RequiredResource
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added comments on system wide repository registration - which is crucial for us
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
Proposal is to write PSResource in C# to reduce complexity and make easier to maintain. | ||
In addition, remove dependency on PackageManagement completely as well as dependency on | ||
nuget.exe. | ||
This module will depend on https://www.nuget.org/packages/NuGet.Client. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would this mean that users need to go and download nuget.exe first like we have to do today, or is it planned to add the dependency directly in the module?
If not, it would be nice to have that download part automated as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This reads to me that the Nuget client dll's needed will be shipped with the module
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
|
||
- A way to filter resources to those allowed to be installed on the system | ||
|
||
- Enterprise management of local cache/the ability to configure a PSRepository on a system-wide level |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this list set in a priority order? Because having the possibility to set a repository system-wide is really some strongly needed for us.
We would benfit way more from having a repository system wide, than from having a message saying we have don't have the latest module installed, or from a cleanup of old modules (which we intentionnally dont remove for backwards compatability / fallback scenarios).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, and for v3 GA this is a critical need to address the long standing issue that v2 has had requiring per user setup which doesn't work for systems that require shared management at all
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
`Update-PSResourceCache` first and then attempt a search. | ||
However, this cmdlet does not update the cache if one exists. | ||
|
||
A `-Version` paramter will accept a string of a [Nuget version range syntax](https://docs.microsoft.com/en-us/nuget/reference/package-versioning#version-ranges-and-wildcards). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Parameter has a typo.
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
|
||
A `-Prerelease` switch will allow include prerelease resources in the results. | ||
|
||
A `-ModuleName` paramter will accept a string, and will return commands, DSC resources, and Role Capabilities found in modules with that name. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Parameter has a typo.
1-Draft/RFCxxxx-PowerShellGet-3.0.md
Outdated
|
||
A `-ProxyCredential` parameter will accept a PSCredential | ||
|
||
A `-Credential` paramter will accept a PSCredential. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Parameter has a typo.
…xx-PowerShellGet-3.0.md Moving to Draft-Accepted folder
Since PSGet v3 dropped the
This works pre-v3, but if I have v3 and v2 installed, it no longer works because v3 does not have this cmdlet. I think having |
on this module to produce one. | ||
|
||
A `-SkipDependenciesCheck` switch can be used to bypass the default check that | ||
all dependencies are present. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
... all dependencies are present where? on the local system or in the target repository?
If the resource requires dependencies that are not already installed, then | ||
a prompt will appear _before anything is installed_, listing all the resources | ||
including module name, version, size, and repository to be installed unless | ||
`-IncludeDependencies` is specified which will install without prompting. | ||
Rejecting the prompt will result in nothing being installed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doc talks about resources that have dependencies but does not mention how will we handle modules that depend on a nuget library.
If we can define the nuget dependency in the published module, what happens when the user installs the module with out a -DestinationPath
? Are they prevented from installing it because they can't save the nuget package? How would the user know how to handle that situation or should they be responsible to solve it? Or if they do specify a -DestinationPath
that is their module folder and end up with libraries all over the root of the folder?
If they do install with the module, where do they go? What would module authors have to do to use those libraries in that module? Will they drop into a standard shared location that's in the $env:path
or a module's lib folder that can be referenced with the psd1? Or would they have to hand craft the loading logic in the psm1 or the scripts to process using?
Or is this out of scope and modules authors will continue to embed the libraries inside their module like they do today and distribute it as one (large) module? Or download them on demand from the psm1 with Install-PSResource
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some built in handling of nuget dependencies would be a huge benefit, would love to see that.
In order to open source PowerShellGet 3.0 we will fork the current PowerShellGet repo, rename it as PowerShellGetv2, | ||
and use the existing PowerShellGet repo as the primary support and development contact for PowerShellGet 3+. | ||
|
||
## Alternate Proposals and Considerations |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we leave the door open for custom repository providers or other sources? I am really interested in using blob storage as a module repository source. I'm seeing more and more multi account cloud configurations where network connectivity to a central nuget server doesn't work out very well.
@TylerLeonhardt with this
if this Issue PowerShell/PowerShell#8949 can get resolved in v7 this will not be that much of an issue but until then the syntax in this comment PowerShell/PowerShell#8949 (comment) would provide a possible middle ground (& works on all PSversions too) |
Just keep in mind... We do use the module qualified syntax in the PowerShell extension and it still has to support 5.1 for the foreseeable future. |
Pester also makes heavy use of module-qualified commands, from what I recall of how it builds its safe commands table, as well. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added my comments to this version of the RFC
nuget.exe. | ||
This module will depend on Nuget Server and Client packages including https://www.nuget.org/packages/NuGet.Client. | ||
This module would be shipped in the PSGallery supporting PowerShell 5.1+. | ||
This module will likely ship in PowerShell 7.1 alongside PowerShellGet 2.x. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer we didn't ship 2.x and 3 in 7.1 but know this will be needed for back compat until PowerShellGet v3 is the dominant version in use.
Unless
You bundle into PowerShellGetv3 a Install-Module wrapper that takes in all the same Params as Install-module in v2 and then spins them off to Install-PSResource - which would be the better User experience and would not break many thousands (perhaps millions ?) of build scripts or at least initiations of those scripts & IMO this needs to be baked in to the GA of PowerShellGet v3.
This will also enable changes in PowerShell to use `Find-PSResource -Type Command` to look | ||
in this cache when it can't find a command and suggest to the user the module to install to | ||
get that command. | ||
This will be a local json file (one per repository) containing sufficient metadata |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Depending on the size of the reposistory this likely could become a large file. do you have an estimate of how big this file would be for the PowerShell Gallery as that's the most common repository in use in the wild.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I note this was mentioned further down in the RFC
The cache will be stored in the user path. | ||
There is no system cache that is shared. | ||
A system wide cache would require elevation or sudo to create and update preventing | ||
it from being useful. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Considering lots of organisations running PowerShell install Modules to the machine level & not the user level we need the option of having this at both levels and with a preference for a location like ProgramData in Windows where it can be stored once and shared between user level and machine level, which I don't think should require admin rights on Windows, I can't speak for the linux side
An example cache with 5000 resources (approximately the number of unique packages | ||
currently published to PowerShellGallery.com) is ~700KB in compressed json form. | ||
|
||
The cache would have both latest stable and latest prerelease versions of resources. | ||
|
||
>[!NOTE] | ||
>Need to experiment if it makes sense to have a single cache file for all repositories | ||
>or a different cache per repository for size and perf reasons. | ||
>Perf tests will determine if the cache needs to be in binary form. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are currently over 47k packages (when you consider versions) & many orgs aren't & shouldn't be blindly updating to the latest stable version without being sure this doesn't break them, which is why I think that we need to at least have the latest minor versions for each major version
So that this would show both of the following examples.
PowerShellGet v2.2.4
PowerShellGet v3.1.0
but should also show up
PowerShellGet v2.2.5-preview2
PowerShellGet v3.1.1-preview5
I think that makes most sense for most orgs.
If the file ends up being Binary, instead of JSON then this would stop other ways of inspection of that file via scripted methods that would include anyone wanting to look at deltas between last cache file on disk & any subsequent updates - something for the likes of DSC Resources this is something I have a need in future to do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also & I just finished testing this, to get a single output from the Gallery now of all the Modules & versions took over 2hours and at well over 5.5k individual calls of Find-Module to the gallery to get all the info required for all modules in the Gallery, so therefore it would be good if we could get the option for both the slim file with the suggestions above, and the full fat file with everything as a choice.
Example cache entry: | ||
|
||
```json | ||
{ | ||
"name": "This is my module", | ||
"exportedFunctions": [ | ||
"Get-One", | ||
"Set-Two" | ||
], | ||
"version": "1.0.0.0", | ||
"type": "module", | ||
"tags": [ | ||
"Linux", | ||
"PSEdition_Core", | ||
"PSEdition_Desktop", | ||
"AzureAutomation" | ||
] | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
v3 will continue to use the versioning folder format as v2 in that it will be | ||
Major.Minor.Patch and not contain the semver prerelease label. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This I think should allow use of the pre-release label to enable side by side comparisons of pre-releases like we can for general releases, especially during the developing of these pre-release.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It could add the pre-release label in 7.1 and continue to use the old format for everything previous. At least get support for the pre-release label in 7.1 so we have the option to adopt this later.
>[!NOTE] | ||
>Credentials management will be using [Secrets Management RFC](https://github.com/PowerShell/PowerShell-RFC/pull/208) | ||
> for generalized credential management. | ||
The `repository` property can be the name of the registered repository or the URL | ||
to a repository. | ||
If no `repository` is specified, it will use the `default` repository. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean that SecretsManagement will be a dependency for PowerShellGet v3?
Also what other dependencies are expected?
Will these be correctly listed in the off case that a component you intend to use now is in a module that in future is not bundled in PowerShell directly as per this possible change that may come in a future PowerShell Release PowerShell/PowerShell#5681
>[!NOTE] | ||
>Installing dependencies is following the `apt` experience in that it prompts | ||
>by default instead of automatically installing dependencies. Since dependencies | ||
>can be quite large, this would be equivalent to `ConfirmImpact=High` which | ||
>would prompt by default. | ||
>Dependency installation works with `-DestinationPath` parameter. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to have the ability to push though for CI via use of either the force parameter or -Confirm:$false but should not need both
We will also introduce a `New-RequiredResourceFile` cmdlet which will create a template file. | ||
If the switch `-AsPSD1` is used it will create a psd1 file, otherwise it will default to json. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why the default to json, psd1 as a powershell data file makes more sense
In order to open source PowerShellGet 3.0 we will fork the current PowerShellGet repo, rename it as PowerShellGetv2, | ||
and use the existing PowerShellGet repo as the primary support and development contact for PowerShellGet 3+. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I so wish this had not been conducted this way as it is a terrible user experience for current users of PowerShellGetv2 including those like me that had it cloned already
Forking and calling it PowerShellGetv3 would have been a much better option, but generally a v3 branch & the branches off of this within the original repository generally makes the most sense for development & should be adhered to in future splits of modules where possible.
Hi, @SydneyhSmith now that winget got announced , how will powershellget integrate / influence / evolve with winget? |
@Stephanevg hard to say tbh. The WinGet team have more or less said they're not interested in integrating with PowerShell directly at the present time in this issue (despite quite a few folks asking for some kind of support): microsoft/winget-cli#221 Probably the PS team and the WinGet team will need to have a chat about that before they can make anything clear here. |
Updated based on current scope for PowershellGet 3.0 release. Includes the following updates: - Plans to ship in PowerShell 7.2 - CompatPowerShellGet compatibility module - No `-Filter` parameter for find and install - Integration with SecretManagement moved to post-GA - Install-PSResource installs dependencies by default - New-RequiredResourceFile cmdlet moved to post-GA - Save-PSResource `-Type Library` and `-IncludeAllRuntimes` moved to post-GA - Update-PSResoure `-UpdateTo` parameter moved to post-GA - Local Cache support moved to post-GA
As part of the plan for post GA of v3 can I ask for this RFC to also include details about plans to get v3 (& the compat module) into Windows 10 & Server 2019. I say this because its 2020, and Windows 10 version 2004 still ships with the awful 1.0.0.1 version by default. I know that @SteveL-MSFT @joeyaiello amd team will say "it's hard getting things into Windows" but this needs to be part of the plan (even if that takes 6-12 months after PowerShellGet v3 is GA) |
@PowerShell/powershell-committee |
Publishing an updated version of the PowerShellGet 3.0 RFC based on #185
We are currently targeting a preview of module (not feature complete) for end of year 2019.