Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to add arbitrary Command Line Options for options missing in tasks #463

Open
carstencodes opened this issue Dec 8, 2023 · 0 comments
Labels
feature request New feature or request

Comments

@carstencodes
Copy link

Is your feature request related to a problem? Please describe.
At our company, there are sometimes glitches when uploading artifacts from a self-hosted ADOS agent to artifactory. While the name resolution works, there might be a temporary link failure. When using the generic artifact upload task, I am possible to set the number of retries, but not the wait duration between these steps. So it defaults to 0s.

If I read the documentation correctly, this option is not available for all commands and cannot be set via an environmental variable.

Hence, after three retries with a timing duration of 0s, the upload fails.

As we found out, the upload might be successful if repeated within a time-period of 5 to 40 seconds. So it would be good for us, to set the retry wait time of 5s with a retry count of 12.

Describe the solution you'd like to see

It would be good to be able to add a list of arbitrary commands as list of strings like this:

        # Upload Debian package to Artifactory
        - task: ArtifactoryGenericUpload@2
          displayName: 'Artifactory Generic Upload (My Package)'
          inputs:
            artifactoryService: 'Artifactory My Product'
            fileSpec: |
              {
                "files": [
                  {
                    "pattern": "$(Build.SourcesDirectory)/binaries/package/company_executable_*_jammy_amd64.deb",
                    "target": "productfeed/pool/main/c/company_executable/"
                  }
                ]
              }
            collectBuildInfo: true
            buildName: '$(ArtifactoryBuildName)'
            buildNumber: '$(ArtifactoryBuildNumberDebian)'
            setDebianProps: true
            debDistribution: jammy
            debComponent: main
            debArchitecture: amd64
            additionalCliCommands:   # new
                - --retry-wait-time
                - 5s

The resulting command line arguments should be prepended to the generated list of CLI arguments, as the go flags package overwrites the first item with the later item. (I am not sure, how the cli context solves this issue).

This way, missing parameters can be added to the jfrog cli without having the urge to add a parameter to the task library, that might not apply to all commands.

If a command is not present in all versions of the cli, a min version could also be specified.

        # Upload Debian package to Artifactory
        - task: JFrogGenericArtifacts@2
          displayName: 'Artifactory Generic Upload (My Package)'
          inputs:
            connection: 'Artifactory My Product'
            specSource: 'taskConfiguration'
            fileSpec: |
              {
                "files": [
                  {
                    "pattern": "$(Build.SourcesDirectory)/binaries/package/company_executable_*_jammy_amd64.deb",
                    "target": "productfeed/pool/main/c/company_executable/"
                  }
                ]
              }
            collectBuildInfo: true
            buildName: '$(ArtifactoryBuildName)'
            buildNumber: '$(ArtifactoryBuildNumberDebian)'
            setDebianProps: true
            debDistribution: jammy
            debComponent: main
            debArchitecture: amd64
            minCliVersion: 2.1.7   #new
            minCliVersionNotFullfilled: ExitError   #new ExitError or ContinueWithWarning, Defaulting to ???
            additionalCliCommands:    #new
                - --retry-wait-time
                - 5s

The task could then check the version check the minimum version before the task is executed and fail, if the CLI version is not fulfilled. The default value could be latest, so the check could be skipped.

Describe alternatives you've considered

  • Just add the missing parameter to the task library. This might actually solve the issue for us, but will require more involvment, if the next step is missing.
  • Make all parameters in the jfrog cli accessible via env parameters. This way, the value could be specified via the generic environment variables of the job or agent.

Additional context

Log file from the build.

´´´
Executing JFrog CLI Command: /opt/microsoft/vsts-agent/_build/_work/_tool/_jfrog/current/jfrog rt u --url="https://jfrog-instance.company/artifactory" [...] --retries=3 [...]
09:07:21 [Info] [Thread 0] Uploading artifact: [...]/company_executable_2.1.0.123456_jammy_amd64.deb
09:08:01 [Warn] [Thread 0] (Attempt 1) - Failure occurred while uploading to [...]: Put [...]": write tcp 10.redacted:47352->10. redacted:443: write: connection reset by peer
09:08:01 [Warn] [Thread 0] (Attempt 2) - Failure occurred while uploading to [...]: Put [...]: dial tcp 10.redacted:443: connect: connection refused
09:08:01 [Warn] [Thread 0] (Attempt 3) - Failure occurred while uploading to [...]: Put [...]: dial tcp 10.redacted:443: connect: connection refused
09:08:01 [Warn] [Thread 0] (Attempt 4) - Failure occurred while uploading to [...]: Put [...]: dial tcp 10.redacted:443: connect: connection refused
09:08:01 [Info] [Thread 0] executor timeout after 3 attempts with 0 milliseconds wait intervals
09:08:01 [Error] Failed uploading 1 artifacts.
´´´

@carstencodes carstencodes added the feature request New feature or request label Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant