Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54982

Using Java8 Streams evaluates whole expression to Boolean

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I'm trying to write a common library building Docker images from Dockerfile. I wrote a reusable global pipeline, where I'm iterating over Dockerfile lines. I tried to use Java 8 Streams, however I ended up with issue potentially related to JENKINS-26307 and JENKINS-42769, which is not related not to Groovy functional features, but Java 8 feature.

       An example code used:

      def getMissingLabels(Map<String, String> labels) {
       ['label1', 'label2', 'label3', 'label4'].stream()
              .filter({ label -> !isLabelDefined(label, labels) })
              .collect(Collectors.toList())
      }
      

      Whole stream execution results in "false" boolean result, instead of a list. Even complex streams are passing through, but evaluates to "false". When using more complex stream like:

       

      Arrays.stream(splitted)
                  .filter { it.startsWith('LABEL') }
                  .map { it.replaceAll('(LABEL)|"', '') }
                  .map { it.trim() }
                  .map { it.split('=', 2) }
                  .collect(Collectors.toMap(
                  { val -> val[0].toLowerCase() },
                  { val -> val[1] }))
      

      it enters only first filter expression and does not evaluate the rest.

      When replaced functional style code with typical imperative style (for loop) it works.

       

        Attachments

          Issue Links

            Activity

            Hide
            dnusbaum Devin Nusbaum added a comment -

            I reproduced this with some quick testing. I think this is very similar to JENKINS-26481, for which I am not familiar with the full fix, but I think it involved rewriting DefaultGroovyMethods directly in groovy-cps, which I guess we can't really do here as this involves Java Platform code rather than groovy code, but I'm not completely sure. CC Andrew Bayer who worked on the fix for JENKINS-26481 and might have a better idea.

            You might be able to replace this with something equivalent that uses Groovy language features instead of the Stream API as a workaround.

            Show
            dnusbaum Devin Nusbaum added a comment - I reproduced this with some quick testing. I think this is very similar to JENKINS-26481 , for which I am not familiar with the full fix, but I think it involved rewriting DefaultGroovyMethods directly in groovy-cps, which I guess we can't really do here as this involves Java Platform code rather than groovy code, but I'm not completely sure. CC Andrew Bayer who worked on the fix for JENKINS-26481 and might have a better idea. You might be able to replace this with something equivalent that uses Groovy language features instead of the Stream API as a workaround.
            Hide
            fischer Thomas Fox added a comment -

            I have a similar problem when trying to use a Java8 parallelStream() in a Jenkins shared library. The stream processes only a single item in the list. The problem is also with stream() but not with the groovy each(which I cannot use because it does not execute in parallel). The shared library code is

            def streamtest()
            {
              def myList = ["1","2","3","4"]
              
              echo "each:"
              myList.each{ entry -> echo "$entry" }
              
              echo "parallelStream:"
              myList.parallelStream().forEach{ entry -> echo "$entry" }
              
              echo "stream:"
              myList.stream().forEach{ entry -> echo "$entry" }
            }
            

            Sample output is

             

             

            [Pipeline] stage
            [Pipeline] { (streamtest)
            [Pipeline] echo
            each:
            [Pipeline] echo
            1
            [Pipeline] echo
            2
            [Pipeline] echo
            3
            [Pipeline] echo
            4
            [Pipeline] echo
            parallelStream:
            [Pipeline] echo
            3
            [Pipeline] echo
            stream:
            [Pipeline] echo
            1
            [Pipeline] }
            [Pipeline] // stage
            

            As an aside, I have also tried to use GPars for parallelism but I could not get it to work with CPS. I also have tried an ThreadPoolExecutor which I also could not get to work (it did not seem to start any threads but I did not do a thorough investigation).

             

            My hopes are

            • If a java language feature does not work, it would be cool if it would fail fast instead of doing unexpected things
            • Is there any working method to use parallelism with threads (not with nodes) in a shared library?
            Show
            fischer Thomas Fox added a comment - I have a similar problem when trying to use a Java8 parallelStream() in a Jenkins shared library. The stream processes only a single item in the list. The problem is also with stream() but not with the groovy each(which I cannot use because it does not execute in parallel). The shared library code is def streamtest() { def myList = [ "1" , "2" , "3" , "4" ] echo "each:" myList.each{ entry -> echo "$entry" } echo "parallelStream:" myList.parallelStream().forEach{ entry -> echo "$entry" } echo "stream:" myList.stream().forEach{ entry -> echo "$entry" } } Sample output is     [Pipeline] stage [Pipeline] { (streamtest) [Pipeline] echo each: [Pipeline] echo 1 [Pipeline] echo 2 [Pipeline] echo 3 [Pipeline] echo 4 [Pipeline] echo parallelStream: [Pipeline] echo 3 [Pipeline] echo stream: [Pipeline] echo 1 [Pipeline] } [Pipeline] // stage As an aside, I have also tried to use GPars for parallelism but I could not get it to work with CPS. I also have tried an ThreadPoolExecutor which I also could not get to work (it did not seem to start any threads but I did not do a thorough investigation).   My hopes are If a java language feature does not work, it would be cool if it would fail fast instead of doing unexpected things Is there any working method to use parallelism with threads (not with nodes) in a shared library?

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                filipowm Mateusz Filipowicz
              • Votes:
                2 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: