Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-36547

Queue.Task.getFullDisplayName is a poor choice of key for LoadBalancer.CONSISTENT_HASH

    Details

    • Similar Issues:
    • Released As:
      jenkins 2.145, workflow-durable-task-step 2.29

      Description

      The default LoadBalancer, CONSISTENT_HASH, tries to reassign the same node to repeated builds of a given project, so as to reuse workspaces whenever possible. However to identify the project it uses Queue.Task.getFullDisplayName as seen here. For an AbstractProject this works OK, since this method will be implemented in AbstractItem and remains pretty stable across builds—at least so long as you do not rename the project or a containing folder!

      For Pipeline builds it does not work at all, since the PlaceholderTask implementation encodes the build number. Thus, the key to the load balancer changes with every build, and there is no affinity.

      Queue.Task.getName used to be documented to serve this function but it is no different in the case of PlaceholderTask, and worse in the case of AbstractProject (since it throws out the project path, though it is more resilient to display name changes). getUrl sounds like a better key, but again for PlaceholderTask it is intentionally sensitive to the Run's build number, so that you can click on a queue item and see which Pipeline build it was coming from, not just which job. And Task has no getFullName, oddly.

      The load balancer could perhaps call getOwnerTask before using getFullDisplayName. This would have no effect on AbstractProject, but for PlaceholderTask would let it delegate to WorkflowJob, so the affinity would work tolerably well, at least so long as your build has only a single node block—for multiple blocks it will tend to break down. There is also some risk of mangling affinity calculations for other SubTask implementations with different needs.

      What we really need is an extended API for a Task which can specify a dedicated affinity key directly, not overloaded by UI elements. AbstractProject should implement it to return getFullName. PlaceholderTask could implement this at a minimum to return WorkflowJob.getFullName, but could also do better by looking up an enclosing label for the node as in JENKINS-26132 and concatenating that with the job name, so that for example a script like

      stage('Build') {
        node('unix') {
          git '.../main-sources'
          sh './build.sh'
          stash 'binaries'
        }
      }
      stage('Test') {
        parallel unixTests: {
          node('unix') {
            git '.../tests'
            unstash 'binaries'
            sh './test.sh'
          }
        }, windowsTests: {
          node('windows') {
            git '.../tests'
            unstash 'binaries'
            bat 'test.bat'
          }
        }
      }
      stage('Deploy') {
        node('unix') {
          git '.../tools'
          unstash 'binaries'
          sh './deploy.sh'
        }
      }
      

      would result in four affinity keys being used: prjname#Build, prhname#unixTests, prjname#windowsTests, prjname#Deploy, each resulting in distinct persistent workspaces.

        Attachments

          Issue Links

            Activity

            jglick Jesse Glick created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-26132 [ JENKINS-26132 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 173260 ] JNJira + In-Review [ 185024 ]
            abayer Andrew Bayer made changes -
            Labels api queue workflow api pipeline queue workflow
            abayer Andrew Bayer made changes -
            Labels api pipeline queue workflow api pipeline queue
            jglick Jesse Glick made changes -
            Link This issue is blocking JENKINS-45553 [ JENKINS-45553 ]
            jglick Jesse Glick made changes -
            Labels api pipeline queue api performance pipeline queue
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-48348 [ JENKINS-48348 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-51100 [ JENKINS-51100 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-52529 [ JENKINS-52529 ]
            oleg_nenashev Oleg Nenashev made changes -
            Issue Type Bug [ 1 ] Improvement [ 4 ]
            dnusbaum Devin Nusbaum made changes -
            Remote Link This issue links to "jenkinsci/workflow-durable-task-step-plugin#78 (Web Link)" [ 22332 ]
            dnusbaum Devin Nusbaum made changes -
            Remote Link This issue links to "jenkinsci/jenkins#3611 (Web Link)" [ 22333 ]
            dnusbaum Devin Nusbaum made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]
            Released As jenkins 2.145, workflow-durable-task-step 2.29
            dnusbaum Devin Nusbaum made changes -
            Assignee Joerg Schwaerzler [ macdrega ]
            jglick Jesse Glick made changes -
            Link This issue causes JENKINS-57805 [ JENKINS-57805 ]
            jglick Jesse Glick made changes -
            Link This issue causes JENKINS-60389 [ JENKINS-60389 ]

              People

              • Assignee:
                macdrega Joerg Schwaerzler
                Reporter:
                jglick Jesse Glick
              • Votes:
                7 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: