From 484339f3694ca3821df2d488fe9c41675724be6f Mon Sep 17 00:00:00 2001 From: Santiago Squarzon Date: Tue, 18 Jun 2024 00:51:51 -0300 Subject: [PATCH 1/4] updating docs --- README.md | 116 +++++++- docs/Invoke-Parallel.md | 266 ------------------ docs/en-US/Invoke-Parallel.md | 136 +++++---- .../Commands/InvokeParallelCommand.cs | 1 + 4 files changed, 180 insertions(+), 339 deletions(-) delete mode 100644 docs/Invoke-Parallel.md diff --git a/README.md b/README.md index 9b3059f..0127a9e 100644 --- a/README.md +++ b/README.md @@ -10,13 +10,123 @@ -PSParallelPipeline is a PowerShell Module that includes the `Invoke-Parallel` function, a function that allows parallel processing of input objects with similar capabilities as [`ForEach-Object`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-7.3) with its `-Parallel` parameter, introduced in PowerShell 7.0. +PSParallelPipeline is a PowerShell Module that includes the `Invoke-Parallel` cmdlet that allows parallel processing of input objects sharing capabilities as +[`ForEach-Object -Parallel`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object) introduced in PowerShell v7.0. -This project is greatly inspired by RamblingCookieMonster's [`Invoke-Parallel`](https://github.com/RamblingCookieMonster/Invoke-Parallel) and Boe Prox's [`PoshRSJob`](https://github.com/proxb/PoshRSJob). +This project was inspired by RamblingCookieMonster's [`Invoke-Parallel`](https://github.com/RamblingCookieMonster/Invoke-Parallel) and is developed with Windows PowerShell 5.1 users in mind where the closest there is to parallel pipeline processing is [`Start-ThreadJob`](https://learn.microsoft.com/en-us/powershell/module/threadjob/start-threadjob?view=powershell-7.4). + +## What does this Module have to offer? + +Excepting [`-AsJob`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-7.4#-asjob), this module offers the same capabilities as `ForEach-Object -Parallel` in addition to supporting [Common Parameters](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_commonparameters), missing feature in the _built-in_ cmdlet. + +### Truly pipeline streaming capabilities + +```powershell +Measure-Command { + $null | Invoke-Parallel { 0..10 | ForEach-Object { Start-Sleep 1; $_ } } | + Select-Object -First 1 +} | Select-Object TotalSeconds + +# TotalSeconds +# ------------ +# 1.06 +``` + +### Support for [CommonParameters](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_commonparameters?view=powershell-7.4) + +#### ForEach-Object -Parallel + +This is something missing on `ForEach-Object -Parallel` as of `v7.5.0.3`. + +```powershell +PS \> 0..5 | ForEach-Object -Parallel { Write-Error $_ } -ErrorAction Stop +# ForEach-Object: The following common parameters are not currently supported in the Parallel parameter set: +# ErrorAction, WarningAction, InformationAction, PipelineVariable +``` + +#### Invoke-Parallel + +A few examples, they should all work properly, please submit an issue if not 😅. + +```powershell +PS \> 0..5 | Invoke-Parallel { Write-Error $_ } -ErrorAction Stop +# Invoke-Parallel: 0 + +PS \> 0..5 | Invoke-Parallel { Write-Warning $_ } -WarningAction Stop +# WARNING: 1 +# Invoke-Parallel: The running command stopped because the preference variable "WarningPreference" or common parameter is set to Stop: 1 + +PS \> 0..5 | Invoke-Parallel { $_ } -PipelineVariable pipe | ForEach-Object { "[$pipe]" } +# [0] +# [1] +# [5] +# [2] +# [3] +# [4] +``` + +## Improved `-TimeOutSeconds` error message + +### ForEach-Object -Parallel + +```powershell +PS \> 0..10 | ForEach-Object -Parallel { $_; Start-Sleep 5 } -TimeoutSeconds 2 +# 0 +# 1 +# 2 +# 3 +# 4 +# InvalidOperation: The pipeline has been stopped. +# InvalidOperation: The pipeline has been stopped. +# InvalidOperation: The pipeline has been stopped. +# InvalidOperation: The pipeline has been stopped. +# InvalidOperation: The pipeline has been stopped. +``` + +### Invoke-Parallel + +```powershell +PS \> 0..10 | Invoke-Parallel { $_; Start-Sleep 5 } -TimeoutSeconds 2 +# 0 +# 1 +# 2 +# 3 +# 4 +# Invoke-Parallel: Timeout has been reached. +``` + +### `$using:` Support + +Same as `ForEach-Object -Parallel` you can use the `$using:` keyword to pass-in variables to the parallel invocations. + +```powershell +$message = 'world!' +'hello ' | Invoke-Parallel { $_ + $using:message } +# hello world! +``` + +### `-Functions` and `-Variables` Parameters + +Both parameters are a quality of life addition, specially `-Functions`, which adds the locally defined functions to the runspaces [Initial Session State](https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate), a missing feature on `ForEach-Object -Parallel`. This is a much better alternative to passing-in the function definition to the parallel scope. + +#### [`-Variables` Parameter](./docs/en-US/Invoke-Parallel.md#-variables) + +```powershell +'hello ' | Invoke-Parallel { $_ + $msg } -Variables @{ msg = 'world!' } +# hello world! +``` + +#### [`-Functions` Parameter](./docs/en-US/Invoke-Parallel.md#-functions) + +```powershell +function Get-Message {param($MyParam) $MyParam + 'world!' } +'hello ' | Invoke-Parallel { Get-Message $_ } -Functions Get-Message +# hello world! +``` ## Documentation -Check out [__the docs__](./docs/en-US/) for information about how to use this Module. +Check out [__the docs__](./docs/en-US/Invoke-Parallel.md) for information about how to use this Module. ## Installation diff --git a/docs/Invoke-Parallel.md b/docs/Invoke-Parallel.md deleted file mode 100644 index 62e92d2..0000000 --- a/docs/Invoke-Parallel.md +++ /dev/null @@ -1,266 +0,0 @@ ---- -external help file: PSParallelPipeline-help.xml -Module Name: PSParallelPipeline -online version: https://github.com/santisq/PSParallelPipeline -schema: 2.0.0 ---- - -# Invoke-Parallel - -## SYNOPSIS -Enables parallel processing of pipeline input objects. - -## SYNTAX - -``` -Invoke-Parallel [-ScriptBlock] [-InputObject ] [-ThrottleLimit ] - [-TimeOutSeconds ] [-Variables ] [-Functions ] [-UseNewRunspace] - [-ProgressAction ] [] -``` - -## DESCRIPTION -\`Invoke-Parallel\` is a PowerShell function that allows parallel processing of input objects with similar capabilities as \`ForEach-Object\` (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object)with its \`-Parallel\` parameter. - -This function is mostly intended for users of Windows PowerShell 5.1 though fully compatible with newer versions of PowerShell. - -## EXAMPLES - -### Example 1: Run slow script in parallel batches -``` -$message = 'Hello world from {0}' - -0..10 | Invoke-Parallel { - $using:message -f [runspace]::DefaultRunspace.InstanceId - Start-Sleep 3 -} -ThrottleLimit 3 -``` - -### Example 2: Demonstrates how `-Variables` parameter works -``` -$message = 'Hello world from {0}' - -0..10 | Invoke-Parallel { - $message -f [runspace]::DefaultRunspace.InstanceId - Start-Sleep 3 -} -Variables @{ message = $message } -ThrottleLimit 3 -``` - -The \`-Variables\` parameter allows to pass variables to the parallel runspaces. -The hash table \`Keys\` become the Variable Name inside the Script Block. - -### Example 3: Adding to a single thread safe instance -``` -$threadSafeDictionary = [System.Collections.Concurrent.ConcurrentDictionary[string,object]]::new() - -Get-Process | Invoke-Parallel { - $dict = $using:threadSafeDictionary - $dict.TryAdd($_.ProcessName, $_) -} - -$threadSafeDictionary["pwsh"] -``` - -### Example 4: Adding to a single thread safe instance using `-Variables` parameter -``` -$threadSafeDictionary = [System.Collections.Concurrent.ConcurrentDictionary[string,object]]::new() - -Get-Process | Invoke-Parallel { - $dict.TryAdd($_.ProcessName, $_) -} -Variables @{ dict = $threadSafeDictionary } - -$threadSafeDictionary["pwsh"] -``` - -### Example 5: Passing a locally defined Function to the parallel scope -``` -function Greet { param($s) "$s hey there!" } - -0..10 | Invoke-Parallel { Greet $_ } -Functions Greet -``` - -This example demonstrates how to pass a locally defined Function to the Runspaces scope. - -### Example 6: Setting a timeout for parallel tasks -``` -Get-Process | Invoke-Parallel { - Get-Random -Maximum 4 | Start-Sleep - $_ -} -ThrottleLimit 10 -TimeoutSeconds 4 -``` - -If the timeout in seconds is reached all parallel invocations are stopped. - -### Example 7: Using a new runspace for each invocation -``` -0..5 | Invoke-Parallel { [runspace]::DefaultRunspace.InstanceId } - -Guid ----- -ca9e3ff2-1eb0-4911-a288-838574fc7cb2 -775c65bd-5267-4ecb-943c-a1a1788d1116 -0cffb831-8e41-44b6-9ad8-5c9acfca64ce -e5bc6cce-6cab-4d44-83e5-d947ab56ca15 -b7a9ba07-ad6d-4097-9224-3d87c10c01d7 -ca9e3ff2-1eb0-4911-a288-838574fc7cb2 - -0..5 | Invoke-Parallel { [runspace]::DefaultRunspace.InstanceId } -UseNewRunspace - -Guid ----- -e4047803-0ee7-43e3-b195-c5a456db0cee -3344f9f5-7b02-4926-b69e-313830cf4ee2 -ac22866a-7a41-4c24-b31c-47155054022f -d5be0085-6f80-49c6-9a31-e50f1960329d -80405d89-87fb-47f0-b6ba-a59392a99b6f -3b78d3de-5759-4364-85df-dc72427e6af8 -``` - -By default the runspaces are reused. -When the \`-UseNewRunspace\` parameter is used each parallel invocation will create a new runspace. - -## PARAMETERS - -### -Functions -Existing functions in the Local Session to have available in the Script Block (Runspaces). - -```yaml -Type: String[] -Parameter Sets: (All) -Aliases: funcs - -Required: False -Position: Named -Default value: None -Accept pipeline input: False -Accept wildcard characters: False -``` - -### -InputObject -Specifies the input objects to be processed in the ScriptBlock. - -\> \[!NOTE\] \> This parameter is intended to be bound from pipeline. - -```yaml -Type: Object -Parameter Sets: (All) -Aliases: - -Required: False -Position: Named -Default value: None -Accept pipeline input: True (ByValue) -Accept wildcard characters: False -``` - -### -ScriptBlock -Specifies the operation that is performed on each input object. -This script block is run for every object in the pipeline. - -```yaml -Type: ScriptBlock -Parameter Sets: (All) -Aliases: - -Required: True -Position: 1 -Default value: None -Accept pipeline input: False -Accept wildcard characters: False -``` - -### -ThrottleLimit -Specifies the number of script blocks that are invoked in parallel. -Input objects are blocked until the running script block count falls below the ThrottleLimit. - -\> \[!NOTE\] \> \`-ThrottleLimit\` default value is \`5\`. - -```yaml -Type: Int32 -Parameter Sets: (All) -Aliases: tl - -Required: False -Position: Named -Default value: 5 -Accept pipeline input: False -Accept wildcard characters: False -``` - -### -UseNewRunspace -Uses a new runspace for each parallel invocation instead of reusing them. - -```yaml -Type: SwitchParameter -Parameter Sets: (All) -Aliases: unr - -Required: False -Position: Named -Default value: False -Accept pipeline input: False -Accept wildcard characters: False -``` - -### -Variables -Specifies a hash table of variables to have available in the Script Block (Runspaces). -The hash table \`Keys\` become the Variable Name inside the Script Block. - -```yaml -Type: Hashtable -Parameter Sets: (All) -Aliases: vars - -Required: False -Position: Named -Default value: None -Accept pipeline input: False -Accept wildcard characters: False -``` - -### -ProgressAction -{{ Fill ProgressAction Description }} - -```yaml -Type: ActionPreference -Parameter Sets: (All) -Aliases: proga - -Required: False -Position: Named -Default value: None -Accept pipeline input: False -Accept wildcard characters: False -``` - -### -TimeOutSeconds -Specifies the number of seconds to wait for all input to be processed in parallel. -After the specified timeout time, all running scripts are stopped and any remaining input objects to be processed are ignored. - -```yaml -Type: Int32 -Parameter Sets: (All) -Aliases: to - -Required: False -Position: Named -Default value: 0 -Accept pipeline input: False -Accept wildcard characters: False -``` - -### CommonParameters -This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutVariable, -OutBuffer, -PipelineVariable, -Verbose, -WarningAction, and -WarningVariable. For more information, see [about_CommonParameters](http://go.microsoft.com/fwlink/?LinkID=113216). - -## INPUTS - -### Object -You can pipe any object to this cmdlet. - -## OUTPUTS - -### Object -This cmdlet returns objects that are determined by the input. - -## NOTES - -## RELATED LINKS diff --git a/docs/en-US/Invoke-Parallel.md b/docs/en-US/Invoke-Parallel.md index 6bc25be..681b445 100644 --- a/docs/en-US/Invoke-Parallel.md +++ b/docs/en-US/Invoke-Parallel.md @@ -28,62 +28,51 @@ Invoke-Parallel ## DESCRIPTION `Invoke-Parallel` is a PowerShell cmdlet that allows parallel processing of input objects with similar capabilities as -[`ForEach-Object -Parallel`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object). - -This function is mostly intended for users of Windows PowerShell 5.1 though fully compatible with newer versions of PowerShell. +[`ForEach-Object -Parallel`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object) introduced in PowerShell v7.0. ## EXAMPLES ### Example 1: Run slow script in parallel batches ```powershell -$message = 'Hello world from {0}' +$message = 'Hello world from ' 0..10 | Invoke-Parallel { - $using:message -f [runspace]::DefaultRunspace.InstanceId + $using:message + [runspace]::DefaultRunspace.InstanceId Start-Sleep 3 -} -ThrottleLimit 3 +} ``` -### Example 2: Demonstrates how `-Variables` parameter works +### Example 2: Demonstrates `-Variables` Parameter ```powershell -$message = 'Hello world from {0}' +$message = 'Hello world from ' 0..10 | Invoke-Parallel { - $message -f [runspace]::DefaultRunspace.InstanceId + $message + [runspace]::DefaultRunspace.InstanceId Start-Sleep 3 -} -Variables @{ message = $message } -ThrottleLimit 3 +} -Variables @{ message = $message } ``` -The `-Variables` parameter allows to pass variables to the parallel runspaces. The hash table `Keys` become the Variable Name inside the Script Block. +[`-Variables`](#-variables) specifies a hashtable with key / value pairs of variables to pass-in to the parallel scope. The hashtable keys defines the name for passed-in variables. This parameter is an alternative for the `$using:` keyword. -### Example 3: Adding to a single thread safe instance +### Example 3: Adding to a single thread safe instance with `$using:` keyword ```powershell -$threadSafeDictionary = [System.Collections.Concurrent.ConcurrentDictionary[string,object]]::new() - -Get-Process | Invoke-Parallel { - $dict = $using:threadSafeDictionary - $dict.TryAdd($_.ProcessName, $_) -} - -$threadSafeDictionary["pwsh"] +$dict = [System.Collections.Concurrent.ConcurrentDictionary[int, object]]::new() +Get-Process | Invoke-Parallel { ($using:dict)[$_.Id] = $_ } +$dict[$PID] ``` -### Example 4: Adding to a single thread safe instance using `-Variables` parameter +### Example 4: Adding to a single thread safe instance using `-Variables` ```powershell -$threadSafeDictionary = [System.Collections.Concurrent.ConcurrentDictionary[string,object]]::new() - -Get-Process | Invoke-Parallel { - $dict.TryAdd($_.ProcessName, $_) -} -Variables @{ dict = $threadSafeDictionary } - -$threadSafeDictionary["pwsh"] +$dict = [System.Collections.Concurrent.ConcurrentDictionary[int, object]]::new() +Get-Process | Invoke-Parallel { $dict[$_.Id] = $_ } -Variables @{ dict = $dict } +$dict[$PID] ``` -### Example 5: Passing a locally defined Function to the parallel scope +### Example 5: Demonstrates `-Functions` Parameter ```powershell function Greet { param($s) "$s hey there!" } @@ -91,57 +80,54 @@ function Greet { param($s) "$s hey there!" } 0..10 | Invoke-Parallel { Greet $_ } -Functions Greet ``` -This example demonstrates how to pass a locally defined Function to the Runspaces scope. +[`-Functions`](#-functions) adds locally defined functions to the runspaces [Initial Session State](https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate) allowing you to use them in the parallel scope. -### Example 6: Setting a timeout for parallel tasks +### Example 6: Demonstrates `-TimeoutSeconds` Parameter ```powershell -Get-Process | Invoke-Parallel { - Get-Random -Maximum 4 | Start-Sleep - $_ -} -ThrottleLimit 10 -TimeoutSeconds 4 +0..10 | Invoke-Parallel { Start-Sleep 1 } -TimeoutSeconds 3 ``` -If the timeout in seconds is reached all parallel invocations are stopped. +All parallel invocations are stopped when the timeout is reached and any remaining input objects to be processed are ignored. -### Example 7: Using a new runspace for each invocation +### Example 7: Demonstrates `-UseNewRunspace` Parameter ```powershell -0..5 | Invoke-Parallel { [runspace]::DefaultRunspace.InstanceId } - -Guid ----- -ca9e3ff2-1eb0-4911-a288-838574fc7cb2 -775c65bd-5267-4ecb-943c-a1a1788d1116 -0cffb831-8e41-44b6-9ad8-5c9acfca64ce -e5bc6cce-6cab-4d44-83e5-d947ab56ca15 -b7a9ba07-ad6d-4097-9224-3d87c10c01d7 -ca9e3ff2-1eb0-4911-a288-838574fc7cb2 - -0..5 | Invoke-Parallel { [runspace]::DefaultRunspace.InstanceId } -UseNewRunspace - -Guid ----- -e4047803-0ee7-43e3-b195-c5a456db0cee -3344f9f5-7b02-4926-b69e-313830cf4ee2 -ac22866a-7a41-4c24-b31c-47155054022f -d5be0085-6f80-49c6-9a31-e50f1960329d -80405d89-87fb-47f0-b6ba-a59392a99b6f -3b78d3de-5759-4364-85df-dc72427e6af8 +0..3 | Invoke-Parallel { [runspace]::DefaultRunspace.InstanceId } -ThrottleLimit 2 + +# Guid +# ---- +# c945ae1f-4e66-4312-b23c-f3994965308e +# 1c6af45c-8727-4488-937a-4dfc1d259e9e +# c945ae1f-4e66-4312-b23c-f3994965308e +# 1c6af45c-8727-4488-937a-4dfc1d259e9e + +0..3 | Invoke-Parallel { [runspace]::DefaultRunspace.InstanceId } -ThrottleLimit 2 -UseNewRunspace + +# Guid +# ---- +# 7a1c3871-6ce2-4b7f-ae90-fb1e92cd9678 +# 2488be9e-15fe-4be2-882d-7d98b068c913 +# d3dd7b5d-e7e3-457f-b6fb-def35fe837d7 +# 9af7c222-061d-4c89-b073-375ee925e538 ``` -By default the runspaces are reused. When the `-UseNewRunspace` parameter is used each parallel invocation will create a new runspace. +By default the runspaces are reused. With `-UseNewRunspace` a new runspace is created per input object. ## PARAMETERS ### -Functions -Existing functions in the Local Session to have available in the Script Block (Runspaces). +Specifies existing functions in the Local Session to have added to the runspaces [Initial Session State](https://learn.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.initialsessionstate). + +> [!TIP] +> +> This method is the recommended way of passing-in local functions to the parallel scope. The alternative to this method is passing-in the function definition (as a string) to the parallel scope and define the function in it. ```yaml Type: String[] Parameter Sets: (All) -Aliases: +Aliases: funcs Required: False Position: Named @@ -155,6 +141,7 @@ Accept wildcard characters: False Specifies the input objects to be processed in the ScriptBlock. > [!NOTE] +> > This parameter is intended to be bound from pipeline. ```yaml @@ -180,7 +167,7 @@ Parameter Sets: (All) Aliases: Required: True -Position: 1 +Position: 0 Default value: None Accept pipeline input: False Accept wildcard characters: False @@ -188,16 +175,17 @@ Accept wildcard characters: False ### -ThrottleLimit -Specifies the number of script blocks that are invoked in parallel. +Specifies the number of script blocks that are invoked in parallel (Degree of Parallelism). Input objects are blocked until the running script block count falls below the ThrottleLimit. > [!NOTE] +> > `-ThrottleLimit` default value is `5`. ```yaml Type: Int32 Parameter Sets: (All) -Aliases: +Aliases: tl Required: False Position: Named @@ -211,10 +199,14 @@ Accept wildcard characters: False Specifies the number of seconds to wait for all input to be processed in parallel. After the specified timeout time, all running scripts are stopped and any remaining input objects to be processed are ignored. +> [!NOTE] +> +> Default value of `0` disables the timeout and the cmdlet runs until all pipeline input is processed. + ```yaml Type: Int32 Parameter Sets: (All) -Aliases: +Aliases: to Required: False Position: Named @@ -230,7 +222,7 @@ Uses a new runspace for each parallel invocation instead of reusing them. ```yaml Type: SwitchParameter Parameter Sets: (All) -Aliases: +Aliases: unr Required: False Position: Named @@ -241,13 +233,17 @@ Accept wildcard characters: False ### -Variables -Specifies a hash table of variables to have available in the Script Block (Runspaces). -The hash table `Keys` become the Variable Name inside the Script Block. +Specifies a hashtable of variables to have available in the parallel scope. +The hashtable keys defines the name for passed-in variables. + +> [!TIP] +> +> This parameter is an alternative for the `$using:` keyword. ```yaml Type: Hashtable Parameter Sets: (All) -Aliases: +Aliases: vars Required: False Position: Named @@ -270,4 +266,4 @@ You can pipe any object to this cmdlet. ### Object -This cmdlet returns objects that are determined by the input. +This cmdlet returns objects that are determined by the script block. diff --git a/src/PSParallelPipeline/Commands/InvokeParallelCommand.cs b/src/PSParallelPipeline/Commands/InvokeParallelCommand.cs index 0dde3eb..6a4b06b 100644 --- a/src/PSParallelPipeline/Commands/InvokeParallelCommand.cs +++ b/src/PSParallelPipeline/Commands/InvokeParallelCommand.cs @@ -7,6 +7,7 @@ namespace PSParallelPipeline; [Cmdlet(VerbsLifecycle.Invoke, "Parallel")] [Alias("parallel")] +[OutputType(typeof(PSObject))] public sealed class InvokeParallelCommand : PSCmdlet, IDisposable { [Parameter(Position = 0, Mandatory = true)] From 1ee62971c042256fa7c33db81b133a6dd4c3ee31 Mon Sep 17 00:00:00 2001 From: Santiago Squarzon Date: Tue, 18 Jun 2024 00:55:22 -0300 Subject: [PATCH 2/4] updating docs --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 0127a9e..7bd933c 100644 --- a/README.md +++ b/README.md @@ -10,14 +10,14 @@ -PSParallelPipeline is a PowerShell Module that includes the `Invoke-Parallel` cmdlet that allows parallel processing of input objects sharing capabilities as +PSParallelPipeline is a PowerShell Module that includes `Invoke-Parallel`, a cmdlet that allows for parallel processing of input objects, sharing similar capabilities as [`ForEach-Object -Parallel`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object) introduced in PowerShell v7.0. This project was inspired by RamblingCookieMonster's [`Invoke-Parallel`](https://github.com/RamblingCookieMonster/Invoke-Parallel) and is developed with Windows PowerShell 5.1 users in mind where the closest there is to parallel pipeline processing is [`Start-ThreadJob`](https://learn.microsoft.com/en-us/powershell/module/threadjob/start-threadjob?view=powershell-7.4). ## What does this Module have to offer? -Excepting [`-AsJob`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-7.4#-asjob), this module offers the same capabilities as `ForEach-Object -Parallel` in addition to supporting [Common Parameters](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_commonparameters), missing feature in the _built-in_ cmdlet. +Except for [`-AsJob`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/foreach-object?view=powershell-7.4#-asjob), this module offers the same capabilities as `ForEach-Object -Parallel` in addition to supporting [Common Parameters](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_commonparameters), missing feature in the _built-in_ cmdlet. ### Truly pipeline streaming capabilities From 32302b3681e161593fc3c921492f6fd15429f5e2 Mon Sep 17 00:00:00 2001 From: Santiago Squarzon Date: Tue, 18 Jun 2024 00:59:42 -0300 Subject: [PATCH 3/4] updating docs --- README.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 7bd933c..5a4c492 100644 --- a/README.md +++ b/README.md @@ -34,8 +34,6 @@ Measure-Command { ### Support for [CommonParameters](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_commonparameters?view=powershell-7.4) -#### ForEach-Object -Parallel - This is something missing on `ForEach-Object -Parallel` as of `v7.5.0.3`. ```powershell @@ -44,8 +42,6 @@ PS \> 0..5 | ForEach-Object -Parallel { Write-Error $_ } -ErrorAction Stop # ErrorAction, WarningAction, InformationAction, PipelineVariable ``` -#### Invoke-Parallel - A few examples, they should all work properly, please submit an issue if not 😅. ```powershell @@ -67,7 +63,7 @@ PS \> 0..5 | Invoke-Parallel { $_ } -PipelineVariable pipe | ForEach-Object { "[ ## Improved `-TimeOutSeconds` error message -### ForEach-Object -Parallel +In `ForEach-Object -Parallel` we get an error message per stopped parallel invocation instead of a single one. ```powershell PS \> 0..10 | ForEach-Object -Parallel { $_; Start-Sleep 5 } -TimeoutSeconds 2 @@ -83,7 +79,7 @@ PS \> 0..10 | ForEach-Object -Parallel { $_; Start-Sleep 5 } -TimeoutSeconds 2 # InvalidOperation: The pipeline has been stopped. ``` -### Invoke-Parallel +With `Invoke-Parallel` you get a single, _friendlier_, error message. ```powershell PS \> 0..10 | Invoke-Parallel { $_; Start-Sleep 5 } -TimeoutSeconds 2 From 9eeaf9fda30e6c15666442698c6ef8a2e957ad69 Mon Sep 17 00:00:00 2001 From: Santiago Squarzon Date: Tue, 18 Jun 2024 01:06:10 -0300 Subject: [PATCH 4/4] updating docs --- README.md | 4 ++-- docs/en-US/Invoke-Parallel.md | 6 +++--- tests/PSParallelPipeline.tests.ps1 | 6 +++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 5a4c492..8d818c0 100644 --- a/README.md +++ b/README.md @@ -91,9 +91,9 @@ PS \> 0..10 | Invoke-Parallel { $_; Start-Sleep 5 } -TimeoutSeconds 2 # Invoke-Parallel: Timeout has been reached. ``` -### `$using:` Support +### [`$using:`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_language_keywords?view=powershell-7.4) Support -Same as `ForEach-Object -Parallel` you can use the `$using:` keyword to pass-in variables to the parallel invocations. +Same as `ForEach-Object -Parallel` you can use the `$using:` scope modifier to pass-in variables to the parallel invocations. ```powershell $message = 'world!' diff --git a/docs/en-US/Invoke-Parallel.md b/docs/en-US/Invoke-Parallel.md index 681b445..255d5a1 100644 --- a/docs/en-US/Invoke-Parallel.md +++ b/docs/en-US/Invoke-Parallel.md @@ -54,9 +54,9 @@ $message = 'Hello world from ' } -Variables @{ message = $message } ``` -[`-Variables`](#-variables) specifies a hashtable with key / value pairs of variables to pass-in to the parallel scope. The hashtable keys defines the name for passed-in variables. This parameter is an alternative for the `$using:` keyword. +[`-Variables`](#-variables) specifies a hashtable with key / value pairs of variables to pass-in to the parallel scope. The hashtable keys defines the name for passed-in variables. This parameter is an alternative for the `$using:` scope modifier. -### Example 3: Adding to a single thread safe instance with `$using:` keyword +### Example 3: Adding to a single thread safe instance with `$using:` scope modifier ```powershell $dict = [System.Collections.Concurrent.ConcurrentDictionary[int, object]]::new() @@ -238,7 +238,7 @@ The hashtable keys defines the name for passed-in variables. > [!TIP] > -> This parameter is an alternative for the `$using:` keyword. +> This parameter is an alternative for the [`$using:` scope modifier](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_scopes?view=powershell-7.4#scope-modifiers). ```yaml Type: Hashtable diff --git a/tests/PSParallelPipeline.tests.ps1 b/tests/PSParallelPipeline.tests.ps1 index 2394d96..8b1a3c8 100644 --- a/tests/PSParallelPipeline.tests.ps1 +++ b/tests/PSParallelPipeline.tests.ps1 @@ -156,8 +156,8 @@ Describe PSParallelPipeline { } } - Context '$using: keyword Support' { - It 'Allows passed-in variables through $using: keyword' { + Context '$using: scope modifier Support' { + It 'Allows passed-in variables through the $using: scope modifier' { $message = 'Hello world from {0:D2}' $items = 0..10 | Invoke-Parallel { $using:message -f $_ } | Sort-Object @@ -185,7 +185,7 @@ Describe PSParallelPipeline { } Context 'Script Block Assertions' { - It 'Should throw on passed-in Script Block via $using: keyword' { + It 'Should throw on passed-in Script Block via $using: scope modifier' { { $sb = { }; 1..1 | Invoke-Parallel { $using:sb } } | Should -Throw -ExceptionType ([PSArgumentException]) }