DYN-5944 Removed NDesk.Options Library #14106

JimmySanchezGlobant · 2023-06-22T23:00:02Z

Please Note:

Before submitting the PR, please review How to Contribute to Dynamo
Dynamo Team will meet 1x a month to review PRs found on Github (Issues will be handled separately)
PRs will be reviewed from oldest to newest
If a reviewed PR requires changes by the owner, the owner of the PR has 30 days to respond. If the PR has seen no activity by the next session, it will be either closed by the team or depending on its utility will be taken over by someone on the team
PRs should use either Dynamo's default PR template or one of these other template options in order to be considered for review.
PRs that do not have one of the Dynamo PR templates completely filled out with all declarations satisfied will not be reviewed by the Dynamo team.
PRs made to the DynamoRevit repo will need to be cherry-picked into all the DynamoRevit Release branches that Dynamo supports. Contributors will be responsible for cherry-picking their reviewed commits to the other branches after a LGTM label is added to the PR.

Purpose

Remove NDesk.Options library and replace it with CommandLineParser library to unify all CLI programs.

Declarations

Check these if you believe they are true

The codebase is in a better state after this PR
Is documented according to the standards
The level of testing this PR includes is appropriate
User facing strings, if any, are extracted into *.resx files
All tests pass using the self-service CI.
Snapshot of UI changes, if any.
Changes to the API follow Semantic Versioning and are documented in the API Changes document.
This PR modifies some build requirements and the readme is updated
This PR contains no files larger than 50 MB

Release Notes

(FILL ME IN) Brief description of the fix / enhancement. Mandatory section

Reviewers

(FILL ME IN) Reviewer 1 (If possible, assign the Reviewer for the PR)

(FILL ME IN, optional) Any additional notes to reviewers or testers.

FYIs

(FILL ME IN, Optional) Names of anyone else you wish to be notified of

src/Tools/Md2Html/Md2Html.csproj

src/Tools/SignDynamo/SignDynamo.csproj

src/Tools/Md2Html/Md2Html.csproj

mjkkirschner · 2023-06-27T16:35:59Z

@pinzart90 I've responded to your questions, let me know if you want to discuss further.

mjkkirschner · 2023-06-27T16:39:48Z

ILMerge failure:

The command "C:\Users\svc_p_dynamobuild\.nuget\packages\ilmerge\3.0.41\build\..\tools\net452\ILMerge.exe bin\AnyCPU\Release\Md2Html.exe ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		/allowDup ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		/lib:bin\AnyCPU\Release\ ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		/out:E:\Builds\Dynamo_master\Dynamo\src\..\bin\AnyCPU\Release\Md2Html\Md2Html.exe ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		AngleSharp.Css.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		AngleSharp.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073:         CommandLine.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		HtmlSanitizer.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		Markdig.dll ^		
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Buffers.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Memory.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Numerics.Vectors.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Runtime.CompilerServices.Unsafe.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Text.Encoding.CodePages.dll " exited with code 9009.

    392 Warning(s)
    1 Error(s)

potentially we remove this step for now and resolve in the other ILMerge update task.

@JimmySanchezGlobant do local builds work for net48 and net6? Do you see the same errors?

I expect there to be test failures on this PR once parallel tests run.
at least temporarily (2.19) I believe we'll need to add a copy of ndesk.options to the extern folder and copy it to the bin folder using the legacy_remove_me infrastructure.

aparajit-pratap · 2023-06-27T19:45:53Z

ILMerge failure:

The command "C:\Users\svc_p_dynamobuild\.nuget\packages\ilmerge\3.0.41\build\..\tools\net452\ILMerge.exe bin\AnyCPU\Release\Md2Html.exe ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		/allowDup ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		/lib:bin\AnyCPU\Release\ ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		/out:E:\Builds\Dynamo_master\Dynamo\src\..\bin\AnyCPU\Release\Md2Html\Md2Html.exe ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		AngleSharp.Css.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		AngleSharp.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073:         CommandLine.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		HtmlSanitizer.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		Markdig.dll ^		
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Buffers.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Memory.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Numerics.Vectors.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Runtime.CompilerServices.Unsafe.dll ^
E:\Builds\Dynamo_master\Dynamo\src\Tools\Md2Html\Md2Html.csproj(43,9): error MSB3073: 		System.Text.Encoding.CodePages.dll " exited with code 9009.

    392 Warning(s)
    1 Error(s)

potentially we remove this step for now and resolve in the other ILMerge update task.

@JimmySanchezGlobant do local builds work for net48 and net6? Do you see the same errors?

I expect there to be test failures on this PR once parallel tests run.
at least temporarily (2.19) I believe we'll need to add a copy of ndesk.options to the extern folder and copy it to the bin folder using the legacy_remove_me infrastructure.

@mjkkirschner where are you seeing these errors, which job? Are these failing for .net6 or because of commandlineparser?

mjkkirschner · 2023-06-27T21:28:41Z

@aparajit-pratap multiple builds - see the PR status checks, probably because of .net6 - but not sure.

src/Tools/Md2Html/Md2Html.csproj

into DYN-5944

JimmySanchezGlobant · 2023-06-29T14:35:29Z

@JimmySanchezGlobant please try to build the md2html project without ilmerge. Try to add the dependencies directly to project and use <PublishSingleFile>true</PublishSingleFile> and <SelfContained>true</SelfContained> https://learn.microsoft.com/en-us/dotnet/core/deploying/single-file/overview?tabs=cli

@pinzart90 Solved, it's building correctly, could you please review it again to know if anything is missing to do here?

github-actions · 2023-06-29T14:45:23Z

":white_check_mark: Bin-Diff Issue Resolved."
(Updated: 2023-07-05-13:56:39)

pinzart90 · 2023-06-29T15:37:12Z

src/Tools/Md2Html/Md2Html.csproj

+        <Exec Condition=" '$([MSBuild]::IsOSPlatform(`Windows`))' == 'True'" Command="if not exist $(SharedOutputPath) mkdir $(SharedOutputPath)" />
+        <Exec Condition=" '!$([MSBuild]::IsOSPlatform(`Windows`))' == 'True' " Command="mkdir -p $(SharedOutputPath)" />
+        <Exec Condition=" '!$([MSBuild]::IsOSPlatform(`Windows`))' == 'True' " Command="cp $(OutputPath)*.* $(SharedOutputPath)" />
+    </Target>
 </Project>


we need this to run on net48 too. Please verify if the new way works for net48. If not you will have to bring back the old way with (ilmerge) under the net48 condition.

Net6 : SelfContained + PublishSingleFile

net48 - ilmerge (old way)

@pinzart90 As I told you before, this was built it correctly on .NET 4.8 and also I built it on .NET 6, and there wasn't any error.

@JimmySanchezGlobant please see the comment above from the bin diff job - the exe is not being produced:
#14106 (comment), so even if there are no errors, something seems off.

@JimmySanchezGlobant the new flags do not work for net48 target.
You need to use the old way (with ILMerge) for net48
And the new flags + Add a new publish target (the flags work on publish only I think)
ex

<Project Sdk="Microsoft.NET.Sdk" DefaultTargets="Build;Publish">

But this should be added only under net6 target somehow. Also SelfContained should be set to false

@pinzart90 The old way with ILMerge is failing, when I build the project it's returning the error code (missing file), so, it's not clear to me how this can solve this issue.

ILMerge should only work under net48. If you try to build from master it still works right ?

@pinzart90 Yes, it's working well under . Net 4.8 from the master branch, it's weird, yesterday I did exactly the same test and it was failing on the ILMerge command, so, I don't know what to do to solve this, to be honest, I'm still very confused.

sm6srw

LGTM!

mjkkirschner · 2023-07-05T23:20:09Z

src/Tools/Md2Html/Md2Html.csproj

+        <PackageReference Include="System.Buffers" Version="4.5.1" />
+        <PackageReference Include="System.Memory" Version="4.5.5" />
+        <PackageReference Include="System.Numerics.Vectors" Version="4.5.0" />
+        <PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="6.0.0" />


why do we need direct references to these?

all these references need to be bundled in a single executable. From what I can tell, these dependencies were needed for net48 (when using ilmerge). I assume they are still needed on net6 (I did not check though)

Oh, I see. These are actually transitive dependencies from one of the other packageRefs.
I can remove these

into pr/14106

github-actions · 2023-07-12T04:45:37Z

⚠️ [run-bin-diff-net60-windows] - Files Added/Deleted::1 new file(s) have been added
(Updated: 2023-07-21-13:05:43)

mjkkirschner · 2023-07-15T01:18:30Z

src/Dynamo.All.Net6.sln

@@ -137,6 +137,7 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DynamoSandbox", "DynamoSand
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "AssemblyRenamerCLI", "Tools\AssemblyRenamerCLI\AssemblyRenamerCLI.csproj", "{F382C3F8-C55C-4350-800A-3D13A94F8E13}"
 EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Md2Html", "Tools\Md2Html\Md2Html.csproj", "{0893F745-CB1A-427A-8E87-CA927273039A}"


@pinzart90 looks like a missing EndProject tag maybe messing up the build somehow?

mjkkirschner · 2023-07-15T01:24:02Z

src/Tools/Md2Html/Program.cs


 namespace Md2Html
 {
+    internal class CMDLineOptions
+    {
+        [Option('h', "help", Required = false, HelpText = "Show help and exit")]


@pinzart90 ... two thoughts - I don't see that this PR updates any of the tests. I think that ndesk options and command line parser require setting flags differently... ie / vs - or --, I can't remember for certain.

Have you tried running the docs browser tests locally? Perhaps the markdown converter tool is never shutting down causing some kind of deadlock in the test runner etc.

@mjkkirschner good catch, the md2html tests do seem to hang locally. Hopefully it is the same cause on the build machine

mjkkirschner · 2023-07-15T01:26:18Z

src/Tools/Md2Html/Program.cs

+                options.IgnoreUnknownArguments = true; options.HelpWriter = Console.Out;
+                options.CaseSensitive = false;
+            });
+            var results = parser.ParseArguments<CMDLineOptions>(args);


guess it's also possible that this is somehow messing with Console.Out or modifying the input argument list so it never makes its way to the rest of the Main function?

pinzart90 · 2023-07-21T12:53:05Z

Looks like a single test is failing
https://master-5.jenkins.autodesk.com/job/Dynamo/job/DynamoSelfServe/job/pullRequestValidation/10037/testReport/DynamoCoreWpfTests/CoreUserInterfaceTests/WorkspaceContextMenu_TestIfOpenOnRightClick

Removed NDesk.Options Library

cb3e0ea

JimmySanchezGlobant requested review from mjkkirschner, aparajit-pratap, sm6srw, pinzart, pinzart90, Cristhian-Marin-asdk, marbelarrietaGlobant and GuillermoRCADSK June 22, 2023 23:00

aparajit-pratap reviewed Jun 23, 2023

View reviewed changes

src/Tools/Md2Html/Md2Html.csproj Outdated Show resolved Hide resolved

JimmySanchezGlobant and others added 2 commits June 23, 2023 14:15

Merge branch 'DynamoDS:master' into DYN-5944

0f9174d

Removed Ndesk.Options from License and About Box

4bb7b70

pinzart90 reviewed Jun 26, 2023

View reviewed changes

src/Tools/Md2Html/Md2Html.csproj Outdated Show resolved Hide resolved

pinzart90 reviewed Jun 26, 2023

View reviewed changes

src/Tools/SignDynamo/SignDynamo.csproj Show resolved Hide resolved

JimmySanchezGlobant and others added 2 commits June 27, 2023 09:28

Merge branch 'DynamoDS:master' into DYN-5944

d0b8b8d

Fixed formatting and included CommandLine.dll

91fafcd

JimmySanchezGlobant requested review from pinzart90 and aparajit-pratap June 27, 2023 14:36

pinzart90 reviewed Jun 27, 2023

View reviewed changes

src/Tools/Md2Html/Md2Html.csproj Show resolved Hide resolved

Moved references under .NET 4.8 ItemGroup

2fb22c2

JimmySanchezGlobant requested a review from pinzart90 June 27, 2023 14:54

aparajit-pratap approved these changes Jun 27, 2023

View reviewed changes

avidit marked this pull request as draft June 28, 2023 00:55

avidit changed the title ~~Removed NDesk.Options Library~~ DYN-5944 Removed NDesk.Options Library Jun 28, 2023

pinzart90 reviewed Jun 28, 2023

View reviewed changes

src/Tools/Md2Html/Md2Html.csproj Outdated Show resolved Hide resolved

Moved more references under .NET 4.8 ItemGroup

61b2630

JimmySanchezGlobant added 2 commits June 29, 2023 09:31

Removed ILMerge and added dependencies directly to project

e855f59

Merge branch 'DYN-5944' of https://github.com/JimmySanchezGlobant/Dynamo

324bb3e

into DYN-5944

pinzart90 reviewed Jun 29, 2023

View reviewed changes

update

fc1a09f

pinzart90 marked this pull request as ready for review July 5, 2023 13:53

sm6srw approved these changes Jul 5, 2023

View reviewed changes

Merge branch 'DynamoDS:master' into DYN-5944

2e0af94

mjkkirschner reviewed Jul 5, 2023

View reviewed changes

mjkkirschner approved these changes Jul 5, 2023

View reviewed changes

pinzart added 7 commits July 6, 2023 10:22

Update Md2Html.csproj

33a1310

Merge branch 'master' into pr/14106

de6659d

Merge branch 'DYN-5944' of https://github.com/JimmySanchezGlobant/Dynamo

4e5216a

into pr/14106

Update License.rtf

fc78c58

Update License.rtf

c819987

Merge branch 'master' into pr/14106

8e44108

Update License.rtf

ad9c4ef

pinzart90 marked this pull request as draft July 12, 2023 13:53

mjkkirschner reviewed Jul 15, 2023

View reviewed changes

pinzart added 3 commits July 18, 2023 12:43

Merge branch 'master' into pr/14106

2cf6b72

Update Md2Html.csproj

bda2f98

Merge branch 'master' into pr/14106

3d73767

pinzart90 marked this pull request as ready for review July 20, 2023 02:27

Update License.rtf

4d6b5cb

pinzart90 merged commit ae24bb2 into DynamoDS:master Jul 21, 2023
19 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DYN-5944 Removed NDesk.Options Library #14106

DYN-5944 Removed NDesk.Options Library #14106

JimmySanchezGlobant commented Jun 22, 2023

mjkkirschner commented Jun 27, 2023

mjkkirschner commented Jun 27, 2023 •

edited

Loading

aparajit-pratap commented Jun 27, 2023

mjkkirschner commented Jun 27, 2023

JimmySanchezGlobant commented Jun 29, 2023

github-actions bot commented Jun 29, 2023 •

edited

Loading

pinzart90 Jun 29, 2023

JimmySanchezGlobant Jun 29, 2023

mjkkirschner Jun 29, 2023

pinzart90 Jun 29, 2023

JimmySanchezGlobant Jun 29, 2023

pinzart90 Jun 29, 2023

JimmySanchezGlobant Jun 29, 2023

sm6srw left a comment

mjkkirschner Jul 5, 2023

pinzart90 Jul 6, 2023

pinzart90 Jul 6, 2023

github-actions bot commented Jul 12, 2023 •

edited

Loading

mjkkirschner Jul 15, 2023

mjkkirschner Jul 15, 2023

pinzart90 Jul 19, 2023

mjkkirschner Jul 15, 2023

pinzart90 commented Jul 21, 2023

DYN-5944 Removed NDesk.Options Library #14106

DYN-5944 Removed NDesk.Options Library #14106

Conversation

JimmySanchezGlobant commented Jun 22, 2023

Purpose

Declarations

Release Notes

Reviewers

FYIs

mjkkirschner commented Jun 27, 2023

mjkkirschner commented Jun 27, 2023 • edited Loading

aparajit-pratap commented Jun 27, 2023

mjkkirschner commented Jun 27, 2023

JimmySanchezGlobant commented Jun 29, 2023

github-actions bot commented Jun 29, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sm6srw left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jul 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pinzart90 commented Jul 21, 2023

mjkkirschner commented Jun 27, 2023 •

edited

Loading

github-actions bot commented Jun 29, 2023 •

edited

Loading

github-actions bot commented Jul 12, 2023 •

edited

Loading