Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40695

Allow lightweight or remote checkout for Groovy libraries

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      New feature suggestion.

      Currently, all shared library functions seem to execute on the master.

      Allow defining some sort of mechanism to allow specifying an executor for the code execution.

      Some ideas:

      • by defining this in the library code (something similar to what node(...) definitions do)
      • via a shared library form field that allows defining the agent to run on
      • checkbox on the shared library form that toggles whether code should be run same node as calling Pipeline block

        Attachments

          Activity

          Hide
          dkder3k Nikita Kunets added a comment -

          Same problem.

          I have to write monstrous pipelines (instead of just encapsulate similar functionality in shared library methods and use them) to be able to run them on slaves in kubernetes cluster.

          Show
          dkder3k Nikita Kunets added a comment - Same problem. I have to write monstrous pipelines (instead of just encapsulate similar functionality in shared library methods and use them) to be able to run them on slaves in kubernetes cluster.
          Hide
          steve_dev Steven G added a comment -

          Nikita Kunets

          We found a workaround that may help you out. We've written Functions.groovy and at the start of each pipeline, we pull the groovy files from a separate git repo and run the following:

          def functions = load "./scripts/Functions.groovy"
          

          This allows you to call the functions defined in Functions.groovy using functions.functionName(). The only requirement is to write the functions in the groovy file and 

          return this
          

          at the very end of the script.

          At this point, we're basically re-creating the functionality of shared libraries. I'd like to hear how Jesse Glick imagines we should be doing this, we've got a few thousand lines of code spread across 6 external groovy files now, on top of our 500 line jenkinsfile that runs the CI flow. If there's a better way, I'm all ears.

          Show
          steve_dev Steven G added a comment - Nikita Kunets We found a workaround that may help you out. We've written Functions.groovy and at the start of each pipeline, we pull the groovy files from a separate git repo and run the following: def functions = load "./scripts/Functions.groovy" This allows you to call the functions defined in Functions.groovy using functions.functionName(). The only requirement is to write the functions in the groovy file and  return this at the very end of the script. At this point, we're basically re-creating the functionality of shared libraries. I'd like to hear how Jesse Glick imagines we should be doing this, we've got a few thousand lines of code spread across 6 external groovy files now, on top of our 500 line jenkinsfile that runs the CI flow. If there's a better way, I'm all ears.
          Hide
          dkder3k Nikita Kunets added a comment -

          Steven G thank you!
          I know this workaround, but in my book if there is such functionality as shared libraries I'd prefer to use it instead of writing workarounds.
          I suppose, in master-slave system it is strange for the master to do something except job management and shared libraries are the way to

          share parts of Pipelines between various projects to reduce redundancies and keep code "DRY"

          So I can't understand why this code should be executed only on the master

          Show
          dkder3k Nikita Kunets added a comment - Steven G thank you! I know this workaround, but in my book if there is such functionality as shared libraries I'd prefer to use it instead of writing workarounds. I suppose, in master-slave system it is strange for the master to do something except job management and shared libraries are the way to share parts of Pipelines between various projects to reduce redundancies and keep code "DRY" So I can't understand why this code should be executed only on the master
          Hide
          jglick Jesse Glick added a comment -

          The issue title is misleading. Yes you can use the load step.

          The standard library retrievers could probably be made to use lightweight checkouts with supported SCMs like GitHub, thus avoiding the need to maintain a local copy anywhere. Not sure if retrieval of source files could be made lazy, though.

          Making the library step use a contextual node might work, with some API changes, though it would introduce security issues if not done carefully—could not be offered for trusted libraries. Also would not work for @Library which must be done prior to script parsing.

          Show
          jglick Jesse Glick added a comment - The issue title is misleading. Yes you can use the load step. The standard library retrievers could probably be made to use lightweight checkouts with supported SCMs like GitHub, thus avoiding the need to maintain a local copy anywhere. Not sure if retrieval of source files could be made lazy, though. Making the library step use a contextual node might work, with some API changes, though it would introduce security issues if not done carefully—could not be offered for trusted libraries. Also would not work for @Library which must be done prior to script parsing.
          Hide
          ordiel Cesar Martinez added a comment -

          Similar use case here; I am trying to identify the files changed on my last (git scm) changeset, and perform some greps on them, but they are on a different node.

          Would it be possible to get an ETA for this? (in case it will be implemented) 

          Show
          ordiel Cesar Martinez added a comment - Similar use case here; I am trying to identify the files changed on my last (git scm) changeset, and perform some greps on them, but they are on a different node. Would it be possible to get an ETA for this? (in case it will be implemented) 

            People

            • Assignee:
              Unassigned
              Reporter:
              owood Owen Wood
            • Votes:
              8 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated: