Plugin executes commands in a wrong container in case of k8s pod agent

[sotona-]

Jenkins and plugins versions report

Jenkins: 2.332.3
OS: Linux - 4.18.0-305.25.1.el8_4.x86_64
Agent: jnlp 4.11 in debian based container
Plugin: fortify:22.1.38

Reproduction steps

1. setup an agent pod template with two containers: jnlp and with Fortify SCA
2. run next code in that agent:

container('fortify') {
    fortifyRemoteAnalysis remoteAnalysisProjectType: fortifyMaven(), 
                    remoteOptionalConfig: [notifyEmail: '[email protected]'],
                    uploadSSC: [appName: 'myapp', appVersion: '0.0.1']
}

Expected Results

it should run a /opt/fortify/bin/scancentral ..... command inside a fortify container, which actually has the scancentral binary

Actual Results

plugin tries to run the scancentral command inside a jnlp container and fails:

Running Fortify remote analysis step
Fortify Jenkins plugin v 22.1.38
Performing Fortify remote analysis
WARNING: Cannot find scancentral executable
Checking for cloudscan executable
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
java.lang.RuntimeException: Cannot find cloudscan executable
	at com.fortify.plugin.jenkins.steps.CloudScanStart.perform(CloudScanStart.java:242)
	at com.fortify.plugin.jenkins.steps.CloudScanStart$Execution.run(CloudScanStart.java:414)
	at com.fortify.plugin.jenkins.steps.CloudScanStart$Execution.run(CloudScanStart.java:399)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Finished: FAILURE

Anything else?

I found a temporary way to avoid it: to create a universal agent image with both Fortify client and JNLP, and use it in a pod template, but I hope this bug will be fixed. Thank you.

[akaryakina]

@sotona- Are you sure that it's failing because it's looking in the wrong container? Could it be a side effect of #61 where it just didn't take the environment variables into account? Unfortunately, it's very difficult for me to reproduce it...

[sotona-]

Just checked with latest plugin version and this pipeline:

def fortifyContainer = [
    name: 'fortify',
    image: 'xxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/jenkins/fortify:22.2.2', // an image with fortify client installed
    resources: [
        limits: [
            cpu: "4",
            memory: "2200Mi"
        ],
        requests: [
            cpu: "2",
            memory: "2Gi"
        ]
    ],
    command: ['cat'],
    tty: true
]


def agentContainers = [
    fortifyContainer
]

def customAgent = [
    spec: [
        containers: agentContainers,
    ]
]

pipeline {
    agent {
        kubernetes {
            cloud 'kubernetes'
            inheritFrom 'default'
            yaml(writeYaml(returnText: true, data: customAgent))
        }
    }
    
    environment {
        FORTIFY_HOME='/opt/fortify'
    }

    options {
        skipDefaultCheckout(true)
    }
    stages {
        stage('fortifyTest') {
            steps {
                container('fortify') {
                    sh 'printenv | grep FORT'
                    sh 'echo $PATH'
                    sh 'sourceanalyzer --help'
                    fortifyClean(buildID: "test")
                }
            }
        }
    }
}

The sourceanalyzer binary is in $PATH, it can be executed using the sh step, but the plugin can't run it.

19:56:15  Running on [fortify-test-6-2k9kz-ptsw4-2qjld](https://my-jenkins-domain.com/tscore/computer/fortify%2Dtest%2D6%2D2k9kz%2Dptsw4%2D2qjld/) in /home/jenkins/agent/workspace/fortify_test
19:56:15  [Pipeline] {
19:56:15  [Pipeline] withEnv
19:56:15  [Pipeline] {
19:56:15  [Pipeline] stage
19:56:15  [Pipeline] { (fortifyTest)
19:56:15  [Pipeline] container
19:56:15  [Pipeline] {
19:56:15  [Pipeline] sh
19:56:16  + printenv
19:56:16  + grep FORT
19:56:16  FORTIFY_HOME=/opt/fortify
19:56:16  [Pipeline] sh
19:56:17  + echo /opt/fortify/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
19:56:17  /opt/fortify/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
19:56:17  [Pipeline] sh
19:56:17  + sourceanalyzer --help
19:56:20  Fortify Static Code Analyzer 22.2.2.0004
19:56:20  Copyright (c) 2003-2023 Micro Focus or one of its affiliates
19:56:20  
19:56:20  Usage:

...here is a lot of help output

19:56:20  [Pipeline] fortifyClean
19:56:20  Running FortifyClean step
19:56:20  Fortify Jenkins plugin v 22.2.39
19:56:20  Launching Fortify SCA clean command
19:56:20  [Pipeline] }
19:56:20  [Pipeline] // container
19:56:20  [Pipeline] }
19:56:20  [Pipeline] // stage
19:56:20  [Pipeline] }
19:56:20  [Pipeline] // withEnv
19:56:20  [Pipeline] }
19:56:20  [Pipeline] // node
19:56:20  [Pipeline] }
19:56:21  [Pipeline] // podTemplate
19:56:21  [Pipeline] End of Pipeline
19:56:21  Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 87779c2a-47e6-438e-8e21-e907e7115dc6
19:56:21  java.io.FileNotFoundException: ERROR: executable not found: sourceanalyzer; make sure that either FORTIFY_HOME environment variable is set or sourceanalyzer is on the PATH or in workspace
19:56:21  	at com.fortify.plugin.jenkins.steps.FortifyStep.findExecutablePath(FortifyStep.java:104)
19:56:21  	at com.fortify.plugin.jenkins.steps.FortifyStep.getExecutable(FortifyStep.java:93)
19:56:21  	at com.fortify.plugin.jenkins.steps.FortifySCAStep.getSourceAnalyzerExecutable(FortifySCAStep.java:94)
19:56:21  	at com.fortify.plugin.jenkins.steps.FortifyClean.perform(FortifyClean.java:67)
19:56:21  	at com.fortify.plugin.jenkins.steps.FortifyClean$Execution.run(FortifyClean.java:149)
19:56:21  	at com.fortify.plugin.jenkins.steps.FortifyClean$Execution.run(FortifyClean.java:134)
19:56:21  	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
19:56:21  	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
19:56:21  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
19:56:21  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
19:56:21  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
19:56:21  	at java.base/java.lang.Thread.run(Thread.java:829)
19:56:21  Finished: FAILURE