Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44924

pipeline groovy script - Sort a list with custom comparator or closure not sorting

    Details

    • Similar Issues:

      Description

       

      When using groovy script in the Pipeline Plugin, sorting a list using closure or a custom comparator does not work anymore.

       

      Steps to reproduce:

      1. create new item of type Pipeline
      2. In the Pipeline script add the following code
      #!groovy
      
      assert ["aa","bb","cc"] == ["aa","cc","bb"].sort { a, b -> a <=> b }
      1. Click Save
      2. Click Build Now
      3. Check the failed build:

      [Pipeline] End of Pipeline
      hudson.remoting.ProxyException: Assertion failed:

      assert ["aa","bb","cc"] == ["aa","cc","bb"].sort { a, b -> a <=> b }

      at org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:404)
      at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.assertFailed(ScriptBytecodeAdapter.java:650)
      at com.cloudbees.groovy.cps.impl.AssertBlock$ContinuationImpl.fail(AssertBlock.java:47)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
      at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
      at com.cloudbees.groovy.cps.Next.step(Next.java:83)
      at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:173)
      at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:162)
      at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
      at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
      at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:162)
      at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:19)
      at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:35)
      at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:32)
      at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
      at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:32)
      at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:330)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230)
      at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
      at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)
      Finished: FAILURE

      Expected result is a sorted list:

      [aa, bb, cc]

       

       

        Attachments

          Issue Links

            Activity

            Hide
            joerg_baeuerle Joerg Baeuerle added a comment - - edited

            Yes, I also agree. This has to be fixed. I've just spent two days of my life trying to figure out what was wrong with my pipeline script. My Groovy code worked perfectly in Jenkin's Script Console, but once put in a Jenkinsfile strange things happened. In my case it was not sort() but toSorted() which has the same problem, e.g. with the following code snippet

            def foo = ["hello","hi","hey"].toSorted { a, b -> a.length() <=> b.length() }
            println foo.toString()
            

            taken straight from the toSorted() specification in the Groovy Docs.

            In Jenkin's Script Console the output was

            [hi, hey, hello]
            

            which is correct. When run from a Jenkinsfile the build's output, however, was

            -1
            

             This is not acceptable and certainly not a "Minor" issue, because it costs a lot of time and money to hunt down these kinds of bugs!

            Show
            joerg_baeuerle Joerg Baeuerle added a comment - - edited Yes, I also agree. This has to be fixed. I've just spent two days of my life trying to figure out what was wrong with my pipeline script. My Groovy code worked perfectly in Jenkin's Script Console, but once put in a Jenkinsfile strange things happened. In my case it was not sort() but toSorted() which has the same problem, e.g. with the following code snippet def foo = [ "hello" , "hi" , "hey" ].toSorted { a, b -> a.length() <=> b.length() } println foo.toString() taken straight from the toSorted() specification in the Groovy Docs . In Jenkin's Script Console the output was [hi, hey, hello] which is correct. When run from a Jenkinsfile the build's output, however, was -1  This is not acceptable and certainly not a "Minor" issue, because it costs a lot of time and money to hunt down these kinds of bugs!
            Hide
            intrepidpat Pat Butkiewicz added a comment - - edited

            Just want to make sure I'm understanding this correctly. Jenkins has broken custom sorting in the Groovy language, expects people to write their Jenkinsfiles and Pipeline libraries in Groovy, and then marks this as a minor issue? Everyone on this thread agrees that's unacceptable, yes?

            Show
            intrepidpat Pat Butkiewicz added a comment - - edited Just want to make sure I'm understanding this correctly. Jenkins has broken custom sorting in the Groovy language, expects people to write their Jenkinsfiles and Pipeline libraries in Groovy, and then marks this as a minor issue? Everyone on this thread agrees that's unacceptable, yes?
            Hide
            abayer Andrew Bayer added a comment -

            If I could have viably fixed this, I would have already - the transformation done to Groovy code to be able to be durable makes a lot of things more or less impossible to get working correctly. We've dealt with as many of them as we could without having to basically write our own implementation of all of Groovy, but there are still edge cases (sorry, but this is a comparative edge case) where we just can't fix them. We're sorry, but this is what it is.

            Show
            abayer Andrew Bayer added a comment - If I could have viably fixed this, I would have already - the transformation done to Groovy code to be able to be durable makes a lot of things more or less impossible to get working correctly. We've dealt with as many of them as we could without having to basically write our own implementation of all of Groovy, but there are still edge cases (sorry, but this is a comparative edge case) where we just can't fix them. We're sorry, but this is what it is.
            Hide
            nealmac01 Neal Macdonald added a comment - - edited

            I couldn't get a pipeline script to work with map sort() at all until I came across this

            https://stackoverflow.com/questions/52946282/jenkins-sort-method-behaviour-not-as-expected. This might be a workaround for the issue above.

            Calling sort from a function with @NonCPS annotation eg:

            @NonCPS
            def sortExample(items) 
            {
                def itemsSorted = items.sort{ it['val'] }
                println(itemsSorted) 
            }
            
            node('targetnode')
            {
                def m = [ [name: 'abc', val: '123'], [name: 'zwe', val: '934'], [name: 'wxc', val: '789']]
                println m
                sortExample(m)
            }
            

            resulted in a full map sort dependent on the value of a specific key:

            [Pipeline] echo
            [{name=abc, val=123}, {name=zwe, val=934}, {name=wxc, val=789}]
            [Pipeline] echo
            [{name=abc, val=123}, {name=wxc, val=789}, {name=zwe, val=934}]
            
            Show
            nealmac01 Neal Macdonald added a comment - - edited I couldn't get a pipeline script to work with map sort() at all until I came across this https://stackoverflow.com/questions/52946282/jenkins-sort-method-behaviour-not-as-expected . This might be a workaround for the issue above. Calling sort from a function with @NonCPS annotation eg: @NonCPS def sortExample(items) { def itemsSorted = items.sort{ it['val'] } println(itemsSorted) } node('targetnode') { def m = [ [name: 'abc', val: '123'], [name: 'zwe', val: '934'], [name: 'wxc', val: '789']] println m sortExample(m) } resulted in a full map sort dependent on the value of a specific key: [Pipeline] echo [{name=abc, val=123}, {name=zwe, val=934}, {name=wxc, val=789}] [Pipeline] echo [{name=abc, val=123}, {name=wxc, val=789}, {name=zwe, val=934}]
            Hide
            lucasc Lucas Cimon added a comment - - edited

            I found bit of helpful context about this issue : https://wiki.jenkins.io/display/JENKINS/Pipeline+CPS+method+mismatches#PipelineCPSmethodmismatches-Callingnon-CPS-transformedmethodswithCPS-transformedarguments

            Edit: captain obvious... I did not notice the link to this exact issue on the wiki

            Here is the workaround I used to sort a list of Map objects, based on a String field:

            pipeline {
                stages {
                    stage("Test") {
                        steps {
                            script {
                                def propList = [[name: 'B'], [name: 'A']]
                                sortByField(propList, 'name')
                                propList.eachWithIndex { entry, i ->
                                    println "${i}: ${entry}"
                                }
                            }
                        }
                    }
                }
            }
            @NonCPS
            static sortByField(list, fieldName) {
                list.sort{ it[fieldName] }
            }
             
            Show
            lucasc Lucas Cimon added a comment - - edited I found bit of helpful context about this issue : https://wiki.jenkins.io/display/JENKINS/Pipeline+CPS+method+mismatches#PipelineCPSmethodmismatches-Callingnon-CPS-transformedmethodswithCPS-transformedarguments Edit: captain obvious... I did not notice the link to this exact issue on the wiki Here is the workaround I used to sort a list of Map objects, based on a String field: pipeline { stages { stage( "Test" ) { steps { script { def propList = [[name: 'B' ], [name: 'A' ]] sortByField(propList, 'name' ) propList.eachWithIndex { entry, i -> println "${i}: ${entry}" } } } } } } @NonCPS static sortByField(list, fieldName) { list.sort{ it[fieldName] } }

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                persal79 Per Salomonsson
              • Votes:
                26 Vote for this issue
                Watchers:
                40 Start watching this issue

                Dates

                • Created:
                  Updated: