Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run on our infrastructure #1933

Merged
merged 8 commits into from
Jun 21, 2024
Merged

Run on our infrastructure #1933

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a3f74b0
Run on our infrastructure
sebastien-marichal Jun 10, 2024
e96209a
Use PowerShell not Pwsh
sebastien-marichal Jun 13, 2024
71fb1b3
Fix NuGet pack Global Tool
sebastien-marichal Jun 13, 2024
953b92b
Fix rebase issue with formatting
costin-zaharia-sonarsource Jun 18, 2024
f280c0f
Fix CycloneDX execution and update failing UT
zsolt-kolbay-sonarsource Jun 20, 2024
bf12559
Review
zsolt-kolbay-sonarsource Jun 20, 2024
179bbd5
Revert NuGetCommand
zsolt-kolbay-sonarsource Jun 20, 2024
a78561c
Remove signing tool setup tasks
zsolt-kolbay-sonarsource Jun 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion Tests/SonarScanner.MSBuild.PreProcessor.Test/TargetsInstallerTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,11 @@ public void InstallTargetsFile_Overwrite()
var msBuildPathSettings = new MsBuildPathSettings();

InstallTargetsFileAndAssert(sourceTargetsContent1, expectCopy: true);
msBuildPathSettings.GetImportBeforePaths().Should().HaveCount(7, "Expecting six destination directories");
// If the current user account is LocalSystem, then the local application data folder is inside %windir%\system32.
// When a 32-bit process tries to use this folder on a 64-bit machine, it is redirected to %windir%\SysWOW64.
// In that case the scanner needs to deploy ImportBefore.targets to both locations, doubling the number of destination directories (14 instead of 7).
var hasCorrectCount = msBuildPathSettings.GetImportBeforePaths().Count() is 7 or 14;
hasCorrectCount.Should().BeTrue("Expecting 7 destination directories (or 14 if the local application data folder is located in %windir%)");
Tim-Pohlmann marked this conversation as resolved.
Show resolved Hide resolved

var path = Path.Combine(msBuildPathSettings.GetImportBeforePaths().First(), FileConstants.ImportBeforeTargetsName);
File.Delete(path);
Expand Down
45 changes: 27 additions & 18 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ trigger:
- master
- branch-*

pool:
vmImage: 'windows-latest'
pool: .net-bubble-aws-re-team-prod

variables:
- group: sonarsource-build-variables
Expand Down Expand Up @@ -45,6 +44,8 @@ variables:
value: "Any CPU"
- name: SOLUTION
value: "SonarScanner.MSBuild.sln"
- name: NUGET_VERSION
value: "6.10.0"

resources:
repositories:
Expand All @@ -67,8 +68,12 @@ stages:
commonMavenArguments: -B -Pdeploy-sonarsource -Dmaven.test.skip=true
steps:
- checkout: self

- task: NuGetToolInstaller@1
displayName: "Install NuGet"
inputs:
versionSpec: $(NUGET_VERSION)
Tim-Pohlmann marked this conversation as resolved.
Show resolved Hide resolved
Comment on lines 72 to +75
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it runs on our infrastructure, we don't need to install NuGet.
Why is this needed?
I remembered, I removed it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1979

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the Nuget installed on the CI image doesn't work, the Package dotnet global tool task fails:

NuGet Version: 4.9.6.8
Attempting to build package from 'dotnet-sonarscanner.nuspec'.
System.InvalidOperationException: The element 'metadata' in namespace 'http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd' has invalid child element 'readme' in namespace 'http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd'. List of possible elements expected: 'serviceable, dependencies, repository, licenseUrl, copyright, references, packageTypes, contentFiles, releaseNotes, tags, language, developmentDependency, frameworkAssemblies, owners' in namespace 'http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd'. This validation error occurred in a 'readme' element.
   at NuGet.Packaging.Manifest.<>c.<ValidateManifestSchema>b__20_0(Object sender, ValidationEventArgs e)
   at System.Xml.Schema.XmlSchemaValidator.ValidateElementContext(XmlQualifiedName elementName, Boolean& invalidElementInContext)
   at System.Xml.Schema.XmlSchemaValidator.ValidateElement(String localName, String namespaceUri, XmlSchemaInfo schemaInfo, String xsiType, String xsiNil, String xsiSchemaLocation, String xsiNoNamespaceSchemaLocation)
   at System.Xml.Schema.XNodeValidator.ValidateElement(XElement e)
   at System.Xml.Schema.XNodeValidator.ValidateNodes(XElement e)
   at System.Xml.Schema.XNodeValidator.ValidateElement(XElement e)
   at System.Xml.Schema.XNodeValidator.ValidateNodes(XElement e)
   at System.Xml.Schema.XNodeValidator.ValidateElement(XElement e)
   at System.Xml.Schema.XNodeValidator.Validate(XObject source, XmlSchemaObject partialValidationType, Boolean addSchemaInfo)
   at NuGet.Packaging.Manifest.ValidateManifestSchema(XDocument document, String schemaNamespace)
   at NuGet.Packaging.Manifest.ReadFrom(Stream stream, Func`2 propertyProvider, Boolean validateSchema)
   at NuGet.Packaging.PackageBuilder.ReadManifest(Stream stream, String basePath, Func`2 propertyProvider)
   at NuGet.Packaging.PackageBuilder..ctor(String path, String basePath, Func`2 propertyProvider, Boolean includeEmptyDirectories)
   at NuGet.Commands.PackCommandRunner.CreatePackageBuilderFromNuspec(String path)
   at NuGet.Commands.PackCommandRunner.BuildFromNuspec(String path)
   at NuGet.Commands.PackCommandRunner.BuildPackage()
   at NuGet.CommandLine.PackCommand.ExecuteCommand()
   at NuGet.CommandLine.Command.ExecuteCommandAsync()
   at NuGet.CommandLine.Command.Execute()
   at NuGet.CommandLine.Program.MainCore(String workingDirectory, String[] args)
##[error]The nuget command failed with exit code(1) and error(The element 'metadata' in namespace 'http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd' has invalid child element 'readme' in namespace 'http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd'. List of possible elements expected: 'serviceable, dependencies, repository, licenseUrl, copyright, references, packageTypes, contentFiles, releaseNotes, tags, language, developmentDependency, frameworkAssemblies, owners' in namespace 'http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd'. This validation error occurred in a 'readme' element.)
##[error]An error occurred while trying to pack the files.

We'll have to update the Nuget version on the image first before we can remove the Nuget installer task.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's why I made this change: 71fb1b3

The NuGetCommand@2 task does not seem to use the NuGet we installed.
I didn't check if there is a way to tell to use the one we use.


- task: Cache@2
displayName: Cache Maven local repo
inputs:
Expand Down Expand Up @@ -152,14 +157,23 @@ stages:
includeNuGetOrg: true
arguments: 'install --global CycloneDX'

- task: DotNetCoreCLI@2
- powershell: |
# It's not possible to use the tool using only its name (dotnet CycloneDX [args]), because of this message in the pipeline:
# "Since you just installed the .NET Core SDK, you will need to reopen the Command Prompt window before running the tool you installed."
# Instead, we need to find the installed tool and execute it directly using its absolute path.
Tim-Pohlmann marked this conversation as resolved.
Show resolved Hide resolved

# Find the latest tool version with the latest .NET runtime version
$toolVersionRegex = "^\d+(\.\d+)+$"
$netVersionRegex = "^net\d+(\.\d+)*$"

$latestToolVersionPath = Get-ChildItem -Path "$env:userprofile\.dotnet\tools\.store\cyclonedx" -Directory | Where-Object { $_.Name -match $toolVersionRegex } | Sort-Object Name -Descending | Select-Object -First 1
Tim-Pohlmann marked this conversation as resolved.
Show resolved Hide resolved
$netVersionsDirectory = [IO.Path]::Combine($latestToolVersionPath.FullName, "cyclonedx", $latestToolVersionPath.Name, "tools")
$latestNetVersionPath = Get-ChildItem -Path $netVersionsDirectory -Directory | Where-Object { $_.Name -match $netVersionRegex } | Sort-Object Name -Descending | Select-Object -First 1
Tim-Pohlmann marked this conversation as resolved.
Show resolved Hide resolved
$toolFullPath = Join-Path $latestNetVersionPath.FullName "any\CycloneDX.dll"

# Execute CycloneDX tool to generate the SBOM
dotnet $toolFullPath "$(SOLUTION)" -t -j -o build
displayName: Dotnet generate SBOM
# https://sonarsource.atlassian.net/browse/BUILD-1303
Tim-Pohlmann marked this conversation as resolved.
Show resolved Hide resolved
inputs:
command: custom
custom: CycloneDX
projects: '$(SOLUTION)'
arguments: '-t -j -o build'

- task: DotNetCoreCLI@2
env:
Expand Down Expand Up @@ -199,15 +213,11 @@ stages:
$signAssemblies = [System.Convert]::ToBoolean("$(IS_RELEASE_BRANCH)") # the variable is a string, we need a boolean
Package-NetFrameworkScanner -SignAssemblies $signAssemblies
Package-NetScanner -SignAssemblies $signAssemblies
pwsh: true
Tim-Pohlmann marked this conversation as resolved.
Show resolved Hide resolved

- task: NuGetCommand@2
- powershell: |
nuget help | Select-String "NuGet Version"
Tim-Pohlmann marked this conversation as resolved.
Show resolved Hide resolved
nuget pack 'nuspec\netcoreglobaltool\dotnet-sonarscanner.nuspec' -NonInteractive -OutputDirectory build -Verbosity Detailed
displayName: 'Package dotnet global tool'
inputs:
command: 'pack'
packagesToPack: 'nuspec\netcoreglobaltool\dotnet-sonarscanner.nuspec'
packDestination: 'build'
versioningScheme: 'off'

- powershell: |
nuget sign "$env:PACKAGES_PATH" -Overwrite -HashAlgorithm SHA256 -CertificateFingerprint $(SM_CERT_FP) -Timestamper http://timestamp.digicert.com -TimestampHashAlgorithm SHA256
Expand Down Expand Up @@ -427,7 +437,6 @@ stages:
WINDOWSSDKTARGET: '10.0.17763.0'
MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository
MAVEN_OPTS: '-Xmx3072m -Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'
pool: .net-bubble-aws-re-team-prod
steps:
- checkout: self
fetchDepth: 1
Expand All @@ -452,7 +461,7 @@ stages:

- task: NuGetToolInstaller@1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this has been added?
It was not required.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1979

inputs:
versionSpec: '5.8.0'
versionSpec: $(NUGET_VERSION)

- powershell: |
$projectVersion = Get-Content "$(Build.SourcesDirectory)\build\version.txt"
Expand Down
Loading