Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42635

Passing Closures to NonCPS Library Function give strange results

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Duplicate
    • Component/s: pipeline
    • Labels:
      None
    • Environment:
      Jenkins 2.19.3 with pipeline 2.4
    • Similar Issues:

      Description

      We are trying to write some administrative jobs using pipeline that need to iterate through some subset of slaves and run maintenance tasks on them.  To this end we wrote the following Global Library function a while ago, and it works fine:

      @NonCPS
      def static getSlaveMetadata() {
        Hudson.instance.slaves.collect { slave ->
          def isOnline = slave.computer.isOnline()
          [
            name: slave.getNodeName(),
            description: slave.getNodeDescription(),
            // getOSDescription() throw null pointer error if slave is off line.
            os: isOnline ? slave.computer.getOSDescription() : null,
            isUnix: slave.computer.isUnix(),
            remoteHomeDir: slave.getRemoteFS(),
            labels: slave.getAssignedLabels().collect { l -> l.toString() },
            isOnline: isOnline
          ] as Map
        }
      }

      Recently we tried to add a filter to function as follows:

      @NonCPS
      def static getSlaveMetadata(Closure filter = { s -> true } ) {
        Hudson.instance.slaves.collect { slave ->
          ...
        }.findAll(filter)
      }

      This failed to work if a non-default filter was used (even if { s -> true } was passed as the filter). In such cases, we got the behavior described in -JENKINS-26307. That is, the function returned either `true` or `false` depending how the filter evaluated the first time it executed.

      So remembering this bug was there, we tried this:

      @NonCPS
      static List getSlaveMetadataFiltered(Closure filter = { s -> true}) {
        List slaves = Hudson.instance.slaves.collect { slave ->
          ...
        }
      
        List filtered = []
        for(slave : slaves){
          if (filter(slave)){
            filtered << slave
          }
        }
        filtered
      }

      This returned the exact same results as the first attempt!

      Ultimately we found that this worked:

      def static getSlaveMetadata(Closure filter) {
        List slaves = getSlaveMetadata()
        List filtered = []
        for(slave in slaves){
          if (filter(slave)){
            filtered << slave
          }
        }
        filtered
      }
      
      @NonCPS
      def static getSlaveMetadata() {
        ... original implementation ...
      }

      And this only works because the overload does not require @NonCPS.  If we decorated it with @NonCPS it again fails in the manner described.

      In all cases, our test pipeline script was the following:

      @Library('kb-testing')
      List justSomethingForLibraryToDecorate(){}
      
      def slv = jenkins.SlaveUtils.getSlaveMetadata({Map s -> s['labels'].contains('docker')})
      println slv
      

        Attachments

          Issue Links

            Activity

            Hide
            abayer Andrew Bayer added a comment -

            Like I said over on JENKINS-42769, this is our old hated friend JENKINS-26481 in yet another form. Quoting from what I said there in case people stumble across this:

            In the end, this (and the issue you linked to) is a symptom of JENKINS-26481 - when you call a CPS-transformed method within (or pass a CPS-transformed closure to) non-CPS-transformed code, it gets...weird. As you saw, what you end up getting is often the return value from the first closure to "complete" in that scope. Basically, @NonCPS code can't call functions defined in CPS-transformed code and can't do anything with a Closure from CPS-transformed code. You'll need to find another way to pass the data you need into the @NonCPS function as parameters, rather than it calling a CPS-transformed function.

            Show
            abayer Andrew Bayer added a comment - Like I said over on JENKINS-42769 , this is our old hated friend JENKINS-26481 in yet another form. Quoting from what I said there in case people stumble across this: In the end, this (and the issue you linked to) is a symptom of  JENKINS-26481 - when you call a CPS-transformed method within (or pass a CPS-transformed closure to) non-CPS-transformed code, it gets...weird. As you saw, what you end up getting is often the return value from the first closure to "complete" in that scope. Basically, @NonCPS code can't call functions defined in CPS-transformed code and can't do anything with a Closure from CPS-transformed code. You'll need to find another way to pass the data you need into the @NonCPS  function as parameters, rather than it calling a CPS-transformed function.

              People

              • Assignee:
                Unassigned
                Reporter:
                kbaltrinic Kenneth Baltrinic
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: