Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47770

Sometimes the node() step allocates a workspace that is already in use by another executor on the same slave.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • pipeline, workflow-durable-task-step-plugin
    • None

      I have the ability to run several of the same build concurrently on the same slave. Each build uses the node() step to access a slave and create a workspace. I have noticed that when I do this, Jenkins often assigns the same workspace directory to multiple nodes.  These node blocks might be executing in parallel as part of the same build (in a "parallel" block) – or they might belong to two discrete builds that happen to be running concurrently. Either way, I often see two node blocks executing at the same time on the same slave with the same workspace directory.

      When this happens, unexpected build results occur, or file access errors cause checkout to fail.

      The workaround I am using is to manually assign my workspaces, using the executor number to uniquely identify the workspace:

      ws ("workspace/" + env.EXECUTOR_NUMBER) { ... }

      This ensures that no two executors on the same slave will ever use the same workspace at the same time. Currently, the Jenkins node() step does not provide that guarantee for me.

      Here is an example of a pipeline script that will cause the problem (here, each node in the pipeline has 2-4 executors on it). The checkout step is done using the gitSCM plugin.

      pipeline {
    agent none
    stages {
        stage('SCM Checkout') {
            steps {
                node('ls620||ls623||ls638||ls629||ls722') {
                    script {
                        doCheckout()
                    }

                    stashSourceFiles()
                    deleteDir()
                }
            }
        }
        stage('Parallel Builds') {
            steps {
                parallel(
                    B1: {
                        node('ls620||ls623||ls638||ls629||ls722') {
                            loadSourceFiles()
                            performBuild(1)
                            deleteDir()
                        }
                    },

                    B2: {
                        node('ls620||ls623||ls638||ls629||ls722') {
                            loadSourceFiles()
                            performBuild(2)
                            deleteDir()
                        }
                    },

                    B3: {
                        node('ls620||ls623||ls638||ls629||ls722') {
                            loadSourceFiles()
                            performBuild(3)
                            deleteDir()
                        }
                    },

                    B4: {
                        node('ls722') {
                            loadSourceFiles()
                            performBuild(4)
                            deleteDir()
                        }
                    },

                    B5: {
                        node('ls620||ls623||ls638||ls629||ls722') {
                            loadSourceFiles()
                            performBuild(5)
                            deleteDir()
                        }
                    },

                    // If any of the parallel branches fails, stop execution.
                    failFast: true
                )
            }
        }
    }
    post {
    }
}

            Unassigned Unassigned
            jzylkin Jack Zylkin
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: