Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39353

Include filtering by class name when splitting tests

    Details

    • Similar Issues:

      Description

      Starting with an scenario such as N test classes run:

      com.parallelizable.one (duration 10 mins)
      com.parallelizable.two (duration 10 mins)
      com.notparallelizable.one (duration 1 mins)
      com.notparallelizable.two (duration 1 mins)

      I am interested in parallelizing classes with packaging com.parallelizable, ending with an pseudo-pipeline such as:

      node

      { test (com.notparallelizable) splits = splitTests parallelism: 2 parallel ( "one": test(splits[0].list.join('\n')), "two": test(splits[1].list.join('\n'))) }

      My current workaround is filtering with grep:

      parallel ( "one": test(splits[0].list.grep(~/com.parallelizable.*/).join('\n')),
      "two": test(splits[1].list.grep(~/com.parallelizable.*/).join('\n')))

      but obviously, the resulting tests wont be ballanced in number of elements. I was thinking in filtering at the plugin level, so the returned groups would be properly ballanced.

      Im not sure if this improvement would be interesting for freestyle projects (i dont use them anymore), but for sure it will for pipelines.

        Attachments

          Issue Links

            Activity

            Hide
            dcendents Daniel Beland added a comment -

             

            I'd like to have the same functionality but for a different reason.

            I have a maven project with multiple fast unit tests run by the surefire plugin and a few longer integration tests run by the failsafe plugin.

            When I split my test I always en up with one split that contains only unit tests or only integration tests so effectively neither the execution of surefire or failsafe is optimal.

            It would be nice if I could specify test includes/excludes patterns and split the tests twice for my 2 test execution context.

             

            Show
            dcendents Daniel Beland added a comment -   I'd like to have the same functionality but for a different reason. I have a maven project with multiple fast unit tests run by the surefire plugin and a few longer integration tests run by the failsafe plugin. When I split my test I always en up with one split that contains only unit tests or only integration tests so effectively neither the execution of surefire or failsafe is optimal. It would be nice if I could specify test includes/excludes patterns and split the tests twice for my 2 test execution context.  
            Hide
            jglick Jesse Glick added a comment -

            To the extent I understand the reporter’s original use case, it sounds like it would be better addressed by JENKINS-27395, under study.

            Show
            jglick Jesse Glick added a comment - To the extent I understand the reporter’s original use case, it sounds like it would be better addressed by  JENKINS-27395 , under study.
            Hide
            witokondoria Javier Delgado added a comment -

            From what I read, for JENKINS-27395 to help with this use case, several invocations to junit would be needed to setup each archiveId and later reuse that ids for the filtering. Couldnt both functionalities coexist (filtering via regexp and via archiveId)? The mentioned ticket doesnt seem to be receiving work recently

            Show
            witokondoria Javier Delgado added a comment - From what I read, for JENKINS-27395 to help with this use case, several invocations to junit would be needed to setup each archiveId and later reuse that ids for the filtering. Couldnt both functionalities coexist (filtering via regexp and via archiveId)? The mentioned ticket doesnt seem to be receiving work recently
            Hide
            jglick Jesse Glick added a comment -

            several invocations to junit would be needed to setup each archiveId and later reuse that ids for the filtering

            Yes, which seems the natural way to extend parallel-test-executor. Integration in this plugin is IMO a critical use case for JENKINS-27395. I cannot speak to the status of that issue currently but I am not keen on piling on features to work around lack of proper design.

            W.r.t. to the maven-project use case, best to just avoid this plugin (for many reasons). If it happens to work with parallel-test-executor currently, that is more by luck than anything else. The better integration is via Pipeline.

            Show
            jglick Jesse Glick added a comment - several invocations to junit would be needed to setup each archiveId and later reuse that ids for the filtering Yes, which seems the natural way to extend parallel-test-executor . Integration in this plugin is IMO a critical use case for  JENKINS-27395 . I cannot speak to the status of that issue currently but I am not keen on piling on features to work around lack of proper design. W.r.t. to the maven-project use case, best to just avoid this plugin (for many reasons). If it happens to work with parallel-test-executor currently, that is more by luck than anything else. The better integration is via Pipeline.
            Hide
            witokondoria Javier Delgado added a comment -

            Wont do as the linked ticket will eventually allow accessing test reports archives selectively (and parallelizing over them)

            Show
            witokondoria Javier Delgado added a comment - Wont do as the linked ticket will eventually allow accessing test reports archives selectively (and parallelizing over them)

              People

              • Assignee:
                witokondoria Javier Delgado
                Reporter:
                witokondoria Javier Delgado
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: