Why JSON instead of psd1?

[jcotton42]

While I applaud the PowerShell team's efforts to make it easier to bring native commands into the object pipeline it seems an odd choice to use JSON instead of the PowerShell-native psd1 format.

An issue I can see right away is that authoring the handlers is a lot less awkward in psd1, because you get syntax highlighting and IntelliSense. And you won't have to fight with quoting.

[aaronsb]

I think because JSON is agnostic. For example, I'm full-on interested in writing native commands from Linux, and IntelliSense doesn't make sense when I'm trying to write wrappers for commands like ip, apt, sysctl, service, uname, etc.

[jcotton42]

I think because JSON is agnostic. For example, I'm full-on interested in writing native commands from Linux, and IntelliSense doesn't make sense when I'm trying to write wrappers for commands like ip, apt, sysctl, service, uname, etc.

Not entirely sure what you mean by that. Crescendo files are meant for use by PowerShell, so being agnostic isn't relevant.

[jcotton42]

#28 this discussion is probably a better place to continue this.

[keithbabinec]

I do think the .psd1 idea makes sense.

Another possible compromise/solution idea— split the “code” parts of the json config out into dedicated files.

Example json:

{
    "$schema": "./Microsoft.PowerShell.Crescendo.Schema.json",
    "Verb": "Get",
    "Noun":"InstalledPackage",
    "OriginalName": "apt",
    "OriginalCommandElements": ["-q","list","--installed"],
    "OutputHandlers": [
        {
            "ParameterSetName":"Default",
            "Handler": "invoke-customhandler.ps1”
        }
    ]
}

Then the ps1 file has the actual PowerShell handler code in it.

You get the benefits of syntax highlighting and static analysis for the ps1 files. You also get a nice clean json file that just has config options in it that fit with your schema (in case the config itself needs to be deserialized to a specific type)

[SeeminglyScience]

Config files as psd1 is great when you can give the user a default file with commented values. If you can't do that, the UX kinda sucks without schema support.

I'm not saying json is the right choice either, but I don't think psd1 is as obvious as an answer as it may seem.

An easy-ish solution imo is a DSL to generate the json (or to generate the final product).

[poke]

Looking in particular at the second example in the announcement I also think that this choice is really odd. While the PowerShell code was limited to a single (albeit long) line in the first example, the second includes a fair amount of code. And having that within JSON is really troublesome for various reasons:

• The JSON string value is quoted with " so now we cannot directly use that within the PowerShell code. That means no interpolated strings. Or, we escape these as \" within the PowerShell code which will make this only more confusing.

• As others mentioned, there is no syntax highlighting or tooling support for code within a string.

• JSON actually does not allow multi-line strings as per the specification: “All Unicode characters may be placed within the quotation marks, except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)” (emphasis mine).

  So the example on the announcement page – while more readable of course – is actually invalid. It is also marked as such in editors that follow this properly. E.g. in Sublime Text this is all red, and VS Code complains about “Unexpected end of string” and “End of file expected”.

I feel like this could have been solved in a nicer way directly within PowerShell, utilizing existing PowerShell tooling instead of relying on impractical a data format. There could have been a separate DSL, or just a chain of PowerShell commands within a ps1-file that configure the module. Something like this maybe:

$module = New-CrescendoModule -Verb Get -Noun IpConfig `
    -OriginalName "c:/windows/system32/ipconfig.exe" `
    -Description "This will display the current IP configuration information on Windows"

Add-CrescendoParameter $module -Name All -OriginalName "/all" -ParameterType Switch `
    -Description "This switch provides all ip configuration details"

Add-CrescendoParameter $module -Name AllCompartments -OriginalName "/allcompartments" -ParameterType Switch `
        -Description "This switch provides compartment configuration details"

Add-CrescendoOutputHandler $module -ParameterSetName Default -Handler {
    param ( $lines )
    $post = $false;
    foreach($line in $lines | ?{$_.trim()}) {
        $LineToCheck = $line | select-string '^[a-z]';
        if ( $LineToCheck ) {
            if ( $post ) { [pscustomobject]$ht |add-member -pass -typename $oName }
            $oName = ($LineToCheck -match 'Configuration') ? 'IpConfiguration' : 'EthernetAdapter';
            $ht = @{};
            $post = $true
        }
        else {
            if ( $line -match '^   [a-z]' ) {
                $prop,$value = $line.split(' :',2);
                $pName = $prop -replace '[ .-]';
                $ht[$pName] = $value.Trim()
            }
            else {
                $ht[$pName] = .{$ht[$pName];$line.trim()}
            }
        }
    }
    [pscustomobject]$ht | add-member -pass -typename $oName
}

Export-CrescendoModule $module -ModuleName Ipconfig.psm1

As you can already see here, this already would have working syntax highlighting and with appropriately defined cmdlets, all those commands would offer full code completion for the various parameters – in all environments that support PowerShell without having to rely on a JSON schema to enable tooling.

Since Crescendo is still in early preview, maybe changing the format could be considered.

[jschpp]

This is a solution I would prefer very much. This will be especially helpful when using more complex [regex] constructs as to avoid escaping stuff multiple times (once for json once for regex)

This also has the added benefit of making the "handler compression" (compressing the handler down to one line to fit it in the JSON format) a thing which can be handled by this module instead of needing to be handled by user side tooling.

[theJasonHelmick]

JSON was chosen because we needed to schematize the file to assist authors in writing them. PSD1 don't support this at this time. We continue to be open to ideas, including revisiting PSD1. Can we agree that XML can be left of the list?

[Halkcyon]

@theJasonHelmick Why JSON specifically, though? As the above comment points out, even your examples are invalid JSON. YAML or TOML are more appropriate configuration languages (and YAML supports JSON schema as a plus).

Can we agree that XML can be left of the list?

Yes, please.

[jcotton42]

JSON was chosen because we needed to schematize the file to assist authors in writing them. PSD1 don't support this at this time. We continue to be open to ideas, including revisiting PSD1. Can we agree that XML can be left of the list?

While a schema certainly is a benefit, as a Crescendo file author I feel I would get much more out of not having to fight with quoting, lack of syntax highlighting, lack of IntelliSense, etc. than having a schema.

[JoseRa-KnowIT]

JSON was chosen because we needed to schematize the file to assist authors in writing them. PSD1 don't support this at this time. We continue to be open to ideas, including revisiting PSD1. Can we agree that XML can be left of the list?

While a schema certainly is a benefit, as a Crescendo file author I feel I would get much more out of not having to fight with quoting, lack of syntax highlighting, lack of IntelliSense, etc. than having a schema.

I couldn't agree more with @jcotton42 in every word!

Besides, as I can understand from @theJasonHelmick announcement, the very purpose and origin of this framework is to make any Powershell user/fanatic feel more comfortable using the tools, syntax and format we are used to.

IMHO, @poke 's approach (#27 (comment)) is much more PowerShell-ish and respectful of the "sacred vow".
Also JEA Endpoints already use PSD1 as a natural and perfectly valid format in role and sessions configuration.

Thinking about JEA, it came to my mind the gap still there in JEA for native and third-party commands and how this Cresendo Framework could fill-it that gap!

That last paragraph are just still-blurred ideas I'm hoping people a lot smarter than me could catch and bring to the reality.

- A PowerShell User / Fanatic

[robharman]

There's also no native way to sign the JSON files using Set-AuthenticodeSignature, which for me means dumping json to psd1, signing, then re/de-serializing the JSON when I need to use it again.

[hanneshayashi]

Just chipping in for team JSON here. I have already started experimenting with automatically creating Crescendo JSON definition files from cobra (https://github.com/spf13/cobra) commands. I would have no idea how to do that if I had to create psd1 files, because no library exists for those to my knowledge.
I think JSON is a great choice in that it is a platform agnostic format and pretty much every dev can work with it. If you want to encourage devs to provide PS modules for their native commands, I think JSON definitely is a lower barrier to cross.

[jantari]

JSON is objectively a bad choice:

• doesn't support comments
• doesn't support multiline strings
• requires backslashes to be escaped
• requires quotes to be escaped
• doesn't support powershell syntax highlighting
• picky and inflexible syntax (no trailing commas, lots of brackets etc.)

The most obvious alternative is YAML. It solves all of these problems except powershell syntax highlighting/intellisense, so with that said I cannot fathom how YAML wasn't chosen over JSON - it's much better in every way, has no disadvantage and is already used to great success by GitHub Actions to define multi-line PowerShell scripts inside of properties/nested structures

I am also open to psd1 of course - but I und erstand that doesn't support schemas yet. Either way it should be clear JSON was objectively the wrong choice and needs to go.

[aaronsb]

I've been watching comments trickle in here, and I have to admit I have tipped in favor of abstracting any powershell code out of the json object.

I like the suggestion @keithbabinec put forth, splitting the contents between the json scaffolding, and the code references. Static analysis, tests, signatures, revision control granularity, all the benefits of native intellisense and all that hoohah - all good things.

[hanneshayashi]

You make some good points and I agree that YAML would indeed be a workable format with many advantages and it does have a broad library support (funnily enough, there are still no ConvertFrom/To-YAML commandlets available, however: PowerShell/PowerShell#3607)

[Halkcyon]

I am also open to psd1 of course - but I und erstand that doesn't support schemas yet. Either way it should be clear JSON was objectively the wrong choice and needs to go.

@jantari I agree, but after more thought, I don't think YAML is an appropriate solution either due to the lack of support in the standard libraries, but also for the complex parsing and vulnerabilities that have come up in the spec. TOML took a while for me to grow to, but it also supports JSON schemas, multiline values, and all that jazz, without the complexity of parsing from YAML.

[kvprasoon]

@theJasonHelmick Are we really validating the schema as of now ? I cant see any schema validation.
We can still use psd1 and convert it to json then validate using Test-Json, ryt ?

[FH-Inway]

@theJasonHelmick Are we really validating the schema as of now ? I cant see any schema validation. We can still use psd1 and convert it to json then validate using Test-Json, ryt ?

@kvprasoon - yes - I'm not sure what preview you were on, but give preview.3 a try

Is that included in the GA version? I was running through the tutorial from the documentation and ran into a weird Test-Handler: Cannot bind argument to parameter 'script' because it is an empty string. error message. Turns out I misspelled Handler in the OutputHandlers of one command with a lower case h :)

Afterwards, I ran the configuration file through Test-Json with the Crescendo schema. That indicated an issue with the Handler property along some other stuff. But it doesn't seem like the Crescendo GA cmdlets do this.
Test-Json output:

Test-Json: ArrayItemNotValid: #/Commands[0]
{
  ObjectExpected: #/Commands[0].Elevation
  PropertyRequired: #/Commands[0].Elevation.Command
  ArrayItemNotValid: #/Commands[0].OutputHandlers[0]
  {
    PropertyRequired: #/Commands[0].OutputHandlers[0].Handler
  }

  StringExpected: #/Commands[0].DefaultParameterSetName
  StringExpected: #/Commands[0].ConfirmImpact
  NotInEnumeration: #/Commands[0].ConfirmImpact
  ObjectExpected: #/Commands[0].Usage
  PropertyRequired: #/Commands[0].Usage.Synopsis
  StringExpected: #/Commands[0].OriginalText
  ArrayExpected: #/Commands[0].HelpLinks
}

[theJasonHelmick]

@kvprasoon - yes - I'm not sure what preview you were on, but give preview.3 a try

[theJasonHelmick]

@Halkcyon - I would add to me previous comment that JSON is a simple intermediate that we are also using moving to for things like DSC. I'm not arguing that it is better/worse than YAML or , but it is a simple manner to convey the configuration that is more consistent with our other technologies. There are interesting discussions on creating the JSON from Powershell script in the Discussions tab.

[theJasonHelmick]

To further this discussion and keep everyone's valuable comments - I'm transferring this issue to the Crescendo discussion tab.