Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41396

Add method definition section to root pipeline block

    Details

    • Similar Issues:

      Description

      Had a discussion about this on #jenkins IRC with Andrew Bayer and R. Tyler Croy.

      Summary
      Currently, there are two ways to declare helper methods to use in declarative pipeline: importing a shared library or putting Groovy method definitions before of the pipeline block.

      This JIRA is aimed at the later case. I understand that we do not want to encourage people to use script blocks in Declarative Pipeline, but there is a difference between providing guidance that says, "Avoid use script blocks, consider using shared libraries, here's how you migrate from script to library" and requiring users to go outside of Declarative Pipeline entirely to do something (but saying they can do it). Further by doing this we make it harder for users to leverage code reuse, one of the key advantages of Pipeline as code.

      This is a akin to the exposing of currentBuild.rawBuild. We tell people they can use it but probably shouldn't, while at the same time not exposing basic build information such as build causes anywhere but on currentBuild.rawBuild.

      We should provide a clear progression for users from raw steps, to helper methods, to shared library. Right now, Scripted Pipeline makes that second step easy, while Declarative Pipeline make it harder and ugly. I think it would be better to have the Declarative model include a section that supports that second step, rather than something that people have to do outside of the model.

      As part of the model, we can reason over it and provide better feedback. For that matter we could even enforce rules on it such as "only declare variables and methods in this section".

      The basic idea would be to have method (or similarly named) section added directly under pipeline.

      At it's most basic, it would look like this:

      pipeline {
          agent any
          stages {
              stage ('Example') {
                  steps {
                      myHelper()
                  }
              }
          }
      
          method {
              def myHelper() {
                  echo "Some complex set of steps"
              }
          }
      }
      

      Or consider this Scripted Pipeline: https://github.com/bitwiseman/hermann/blob/blog/notifications/Jenkinsfile

      stage ('Build') {
      
        node {
          try {
          notifyBuild('STARTED')
      
            // Checkout
            checkout scm
      
            // install required bundles
            sh 'bundle install'
      
            // build and run tests with coverage
            sh 'bundle exec rake build spec'
      
            // Archive the built artifacts
            archive (includes: 'pkg/*.gem')
      
            // publish html
            publishHTML ([
                allowMissing: false,
                alwaysLinkToLastBuild: false,
                keepAll: true,
                reportDir: 'coverage',
                reportFiles: 'index.html',
                reportName: "RCov Report"
              ])
          } catch (Exception e) {
              currentBuild.result = "FAILED"
              throw e
          } finally {
            notifyBuild(currentBuild.result)
          }
        }
      }
      
      def notifyBuild(String buildStatus = 'STARTED') {
        // build status of null means successful
        buildStatus =  buildStatus ?: 'SUCCESSFUL'
      
        // Default values
        def colorName = 'RED'
        def colorCode = '#FF0000'
        def subject = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"
        def summary = "${subject} (${env.BUILD_URL})"
        def details = """<p>STARTED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]':</p>
          <p>Check console output at &QUOT;<a href='${env.BUILD_URL}'>${env.JOB_NAME} [${env.BUILD_NUMBER}]</a>&QUOT;</p>"""
      
        // Override default values based on build status
        if (buildStatus == 'STARTED') {
          color = 'YELLOW'
          colorCode = '#FFFF00'
        } else if (buildStatus == 'SUCCESSFUL') {
          color = 'GREEN'
          colorCode = '#00FF00'
        } else {
          color = 'RED'
          colorCode = '#FF0000'
        }
      
        // Send notifications
        slackSend (color: colorCode, message: summary)
      
        hipchatSend (color: color, notify: true, message: summary)
      
        emailext (
            to: 'bitwiseman@bitwiseman.com',
            subject: subject,
            body: details,
            recipientProviders: [[$class: 'DevelopersRecipientProvider']]
          )
      }
      

      The current guidance for migrating this to Declarative would be to keep notifyBuild outside of pipeline. This is not bad, but to anyone paying attention is a glaring gap in the Declarative model. We support script blocks under steps, but for cross-stage helpers we have to switch to shared libraries. I'll add more detail to this example shortly.

        Attachments

          Issue Links

            Activity

            bitwiseman Liam Newman created issue -
            bitwiseman Liam Newman made changes -
            Field Original Value New Value
            Link This issue relates to JENKINS-38110 [ JENKINS-38110 ]
            rtyler R. Tyler Croy made changes -
            Description Had a discussion about this on #jenkins IRC with [~abayer] and [~201604291_tyler].

            Summary
            Currently, there are two ways to declare helper methods to use in declarative pipeline: importing a shared library or putting Groovy method definitions before of the {{pipeline}} block.

            This JIRA is aimed at the later case. I understand that we do not want to encourage people to use script blocks in Declarative Pipeline, but there is a difference between providing guidance that says, "Avoid use script blocks, consider using shared libraries, here's how you migrate from script to library" and requiring users to go outside of Declarative Pipeline entirely to do something (but saying they can do it). Further by doing this we make it harder for users to leverage code reuse, one of the key advantages of Pipeline as code.

            This is a akin to the exposing of {{currentBuild.rawBuild}}. We tell people they can use it but probably shouldn't, while at the same time not exposing basic build information such as build causes anywhere but on {{currentBuild.rawBuild}}.

            We should provide a clear progression for users from raw steps, to helper methods, to shared library. Right now, Scripted Pipeline makes that second step easy, while Declarative Pipeline make it harder and ugly. I think it would be better to have the Declarative model include a section that supports that second step, rather than something that people have to do outside of the model.

            As part of the model, we can reason over it and provide better feedback. For that matter we could even enforce rules on it such as "only declare variables and methods in this section".

            The basic idea would be to have {{script}} (or similarly named) section added directly under {{pipeline}}.

            At it's most basic, it would look like this:
            {code}
            pipeline {
                agent any
                stages {
                    stage ('Example') {
                        steps {
                            myHelper()
                        }
                    }
                }

                script {
                    def myHelper() {
                        echo "Some complex set of steps"
                    }
                }
            }
            {code}

            Or consider this Scripted Pipeline: https://github.com/bitwiseman/hermann/blob/blog/notifications/Jenkinsfile

            {code}
            stage ('Build') {

              node {
                try {
                notifyBuild('STARTED')

                  // Checkout
                  checkout scm

                  // install required bundles
                  sh 'bundle install'

                  // build and run tests with coverage
                  sh 'bundle exec rake build spec'

                  // Archive the built artifacts
                  archive (includes: 'pkg/*.gem')

                  // publish html
                  publishHTML ([
                      allowMissing: false,
                      alwaysLinkToLastBuild: false,
                      keepAll: true,
                      reportDir: 'coverage',
                      reportFiles: 'index.html',
                      reportName: "RCov Report"
                    ])
                } catch (Exception e) {
                    currentBuild.result = "FAILED"
                    throw e
                } finally {
                  notifyBuild(currentBuild.result)
                }
              }
            }

            def notifyBuild(String buildStatus = 'STARTED') {
              // build status of null means successful
              buildStatus = buildStatus ?: 'SUCCESSFUL'

              // Default values
              def colorName = 'RED'
              def colorCode = '#FF0000'
              def subject = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"
              def summary = "${subject} (${env.BUILD_URL})"
              def details = """<p>STARTED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]':</p>
                <p>Check console output at &QUOT;<a href='${env.BUILD_URL}'>${env.JOB_NAME} [${env.BUILD_NUMBER}]</a>&QUOT;</p>"""

              // Override default values based on build status
              if (buildStatus == 'STARTED') {
                color = 'YELLOW'
                colorCode = '#FFFF00'
              } else if (buildStatus == 'SUCCESSFUL') {
                color = 'GREEN'
                colorCode = '#00FF00'
              } else {
                color = 'RED'
                colorCode = '#FF0000'
              }

              // Send notifications
              slackSend (color: colorCode, message: summary)

              hipchatSend (color: color, notify: true, message: summary)

              emailext (
                  to: 'bitwiseman@bitwiseman.com',
                  subject: subject,
                  body: details,
                  recipientProviders: [[$class: 'DevelopersRecipientProvider']]
                )
            }
            {code}

            The current guidance for migrating this to Declarative would be to keep {{notifyBuild}} outside of {{pipeline}}. This is not bad, but to anyone paying attention is a glaring gap in the Declarative model. We support {{script}} blocks under steps, but for cross-stage helpers we have to switch to shared libraries. I'll add more detail to this example shortly.


            Had a discussion about this on #jenkins IRC with [~abayer] and [~rtyler].

            Summary
            Currently, there are two ways to declare helper methods to use in declarative pipeline: importing a shared library or putting Groovy method definitions before of the {{pipeline}} block.

            This JIRA is aimed at the later case. I understand that we do not want to encourage people to use script blocks in Declarative Pipeline, but there is a difference between providing guidance that says, "Avoid use script blocks, consider using shared libraries, here's how you migrate from script to library" and requiring users to go outside of Declarative Pipeline entirely to do something (but saying they can do it). Further by doing this we make it harder for users to leverage code reuse, one of the key advantages of Pipeline as code.

            This is a akin to the exposing of {{currentBuild.rawBuild}}. We tell people they can use it but probably shouldn't, while at the same time not exposing basic build information such as build causes anywhere but on {{currentBuild.rawBuild}}.

            We should provide a clear progression for users from raw steps, to helper methods, to shared library. Right now, Scripted Pipeline makes that second step easy, while Declarative Pipeline make it harder and ugly. I think it would be better to have the Declarative model include a section that supports that second step, rather than something that people have to do outside of the model.

            As part of the model, we can reason over it and provide better feedback. For that matter we could even enforce rules on it such as "only declare variables and methods in this section".

            The basic idea would be to have {{script}} (or similarly named) section added directly under {{pipeline}}.

            At it's most basic, it would look like this:
            {code}
            pipeline {
                agent any
                stages {
                    stage ('Example') {
                        steps {
                            myHelper()
                        }
                    }
                }

                script {
                    def myHelper() {
                        echo "Some complex set of steps"
                    }
                }
            }
            {code}

            Or consider this Scripted Pipeline: https://github.com/bitwiseman/hermann/blob/blog/notifications/Jenkinsfile

            {code}
            stage ('Build') {

              node {
                try {
                notifyBuild('STARTED')

                  // Checkout
                  checkout scm

                  // install required bundles
                  sh 'bundle install'

                  // build and run tests with coverage
                  sh 'bundle exec rake build spec'

                  // Archive the built artifacts
                  archive (includes: 'pkg/*.gem')

                  // publish html
                  publishHTML ([
                      allowMissing: false,
                      alwaysLinkToLastBuild: false,
                      keepAll: true,
                      reportDir: 'coverage',
                      reportFiles: 'index.html',
                      reportName: "RCov Report"
                    ])
                } catch (Exception e) {
                    currentBuild.result = "FAILED"
                    throw e
                } finally {
                  notifyBuild(currentBuild.result)
                }
              }
            }

            def notifyBuild(String buildStatus = 'STARTED') {
              // build status of null means successful
              buildStatus = buildStatus ?: 'SUCCESSFUL'

              // Default values
              def colorName = 'RED'
              def colorCode = '#FF0000'
              def subject = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"
              def summary = "${subject} (${env.BUILD_URL})"
              def details = """<p>STARTED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]':</p>
                <p>Check console output at &QUOT;<a href='${env.BUILD_URL}'>${env.JOB_NAME} [${env.BUILD_NUMBER}]</a>&QUOT;</p>"""

              // Override default values based on build status
              if (buildStatus == 'STARTED') {
                color = 'YELLOW'
                colorCode = '#FFFF00'
              } else if (buildStatus == 'SUCCESSFUL') {
                color = 'GREEN'
                colorCode = '#00FF00'
              } else {
                color = 'RED'
                colorCode = '#FF0000'
              }

              // Send notifications
              slackSend (color: colorCode, message: summary)

              hipchatSend (color: color, notify: true, message: summary)

              emailext (
                  to: 'bitwiseman@bitwiseman.com',
                  subject: subject,
                  body: details,
                  recipientProviders: [[$class: 'DevelopersRecipientProvider']]
                )
            }
            {code}

            The current guidance for migrating this to Declarative would be to keep {{notifyBuild}} outside of {{pipeline}}. This is not bad, but to anyone paying attention is a glaring gap in the Declarative model. We support {{script}} blocks under steps, but for cross-stage helpers we have to switch to shared libraries. I'll add more detail to this example shortly.


            bitwiseman Liam Newman made changes -
            Link This issue relates to JENKINS-41335 [ JENKINS-41335 ]
            bitwiseman Liam Newman made changes -
            Summary Declarative: Add "script" section to root pipeline block Declarative: Add script definition section to root pipeline block
            abayer Andrew Bayer made changes -
            Link This issue duplicates JENKINS-41335 [ JENKINS-41335 ]
            abayer Andrew Bayer made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Duplicate [ 3 ]
            abayer Andrew Bayer made changes -
            Resolution Duplicate [ 3 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            abayer Andrew Bayer made changes -
            Link This issue duplicates JENKINS-41335 [ JENKINS-41335 ]
            bitwiseman Liam Newman made changes -
            Summary Declarative: Add script definition section to root pipeline block Declarative: Add method definition section to root pipeline block
            bitwiseman Liam Newman made changes -
            Link This issue relates to JENKINS-42079 [ JENKINS-42079 ]
            bitwiseman Liam Newman made changes -
            Description Had a discussion about this on #jenkins IRC with [~abayer] and [~rtyler].

            Summary
            Currently, there are two ways to declare helper methods to use in declarative pipeline: importing a shared library or putting Groovy method definitions before of the {{pipeline}} block.

            This JIRA is aimed at the later case. I understand that we do not want to encourage people to use script blocks in Declarative Pipeline, but there is a difference between providing guidance that says, "Avoid use script blocks, consider using shared libraries, here's how you migrate from script to library" and requiring users to go outside of Declarative Pipeline entirely to do something (but saying they can do it). Further by doing this we make it harder for users to leverage code reuse, one of the key advantages of Pipeline as code.

            This is a akin to the exposing of {{currentBuild.rawBuild}}. We tell people they can use it but probably shouldn't, while at the same time not exposing basic build information such as build causes anywhere but on {{currentBuild.rawBuild}}.

            We should provide a clear progression for users from raw steps, to helper methods, to shared library. Right now, Scripted Pipeline makes that second step easy, while Declarative Pipeline make it harder and ugly. I think it would be better to have the Declarative model include a section that supports that second step, rather than something that people have to do outside of the model.

            As part of the model, we can reason over it and provide better feedback. For that matter we could even enforce rules on it such as "only declare variables and methods in this section".

            The basic idea would be to have {{script}} (or similarly named) section added directly under {{pipeline}}.

            At it's most basic, it would look like this:
            {code}
            pipeline {
                agent any
                stages {
                    stage ('Example') {
                        steps {
                            myHelper()
                        }
                    }
                }

                script {
                    def myHelper() {
                        echo "Some complex set of steps"
                    }
                }
            }
            {code}

            Or consider this Scripted Pipeline: https://github.com/bitwiseman/hermann/blob/blog/notifications/Jenkinsfile

            {code}
            stage ('Build') {

              node {
                try {
                notifyBuild('STARTED')

                  // Checkout
                  checkout scm

                  // install required bundles
                  sh 'bundle install'

                  // build and run tests with coverage
                  sh 'bundle exec rake build spec'

                  // Archive the built artifacts
                  archive (includes: 'pkg/*.gem')

                  // publish html
                  publishHTML ([
                      allowMissing: false,
                      alwaysLinkToLastBuild: false,
                      keepAll: true,
                      reportDir: 'coverage',
                      reportFiles: 'index.html',
                      reportName: "RCov Report"
                    ])
                } catch (Exception e) {
                    currentBuild.result = "FAILED"
                    throw e
                } finally {
                  notifyBuild(currentBuild.result)
                }
              }
            }

            def notifyBuild(String buildStatus = 'STARTED') {
              // build status of null means successful
              buildStatus = buildStatus ?: 'SUCCESSFUL'

              // Default values
              def colorName = 'RED'
              def colorCode = '#FF0000'
              def subject = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"
              def summary = "${subject} (${env.BUILD_URL})"
              def details = """<p>STARTED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]':</p>
                <p>Check console output at &QUOT;<a href='${env.BUILD_URL}'>${env.JOB_NAME} [${env.BUILD_NUMBER}]</a>&QUOT;</p>"""

              // Override default values based on build status
              if (buildStatus == 'STARTED') {
                color = 'YELLOW'
                colorCode = '#FFFF00'
              } else if (buildStatus == 'SUCCESSFUL') {
                color = 'GREEN'
                colorCode = '#00FF00'
              } else {
                color = 'RED'
                colorCode = '#FF0000'
              }

              // Send notifications
              slackSend (color: colorCode, message: summary)

              hipchatSend (color: color, notify: true, message: summary)

              emailext (
                  to: 'bitwiseman@bitwiseman.com',
                  subject: subject,
                  body: details,
                  recipientProviders: [[$class: 'DevelopersRecipientProvider']]
                )
            }
            {code}

            The current guidance for migrating this to Declarative would be to keep {{notifyBuild}} outside of {{pipeline}}. This is not bad, but to anyone paying attention is a glaring gap in the Declarative model. We support {{script}} blocks under steps, but for cross-stage helpers we have to switch to shared libraries. I'll add more detail to this example shortly.


            Had a discussion about this on #jenkins IRC with [~abayer] and [~rtyler].

            Summary
            Currently, there are two ways to declare helper methods to use in declarative pipeline: importing a shared library or putting Groovy method definitions before of the {{pipeline}} block.

            This JIRA is aimed at the later case. I understand that we do not want to encourage people to use script blocks in Declarative Pipeline, but there is a difference between providing guidance that says, "Avoid use script blocks, consider using shared libraries, here's how you migrate from script to library" and requiring users to go outside of Declarative Pipeline entirely to do something (but saying they can do it). Further by doing this we make it harder for users to leverage code reuse, one of the key advantages of Pipeline as code.

            This is a akin to the exposing of {{currentBuild.rawBuild}}. We tell people they can use it but probably shouldn't, while at the same time not exposing basic build information such as build causes anywhere but on {{currentBuild.rawBuild}}.

            We should provide a clear progression for users from raw steps, to helper methods, to shared library. Right now, Scripted Pipeline makes that second step easy, while Declarative Pipeline make it harder and ugly. I think it would be better to have the Declarative model include a section that supports that second step, rather than something that people have to do outside of the model.

            As part of the model, we can reason over it and provide better feedback. For that matter we could even enforce rules on it such as "only declare variables and methods in this section".

            The basic idea would be to have {{method}} (or similarly named) section added directly under {{pipeline}}.

            At it's most basic, it would look like this:
            {code}
            pipeline {
                agent any
                stages {
                    stage ('Example') {
                        steps {
                            myHelper()
                        }
                    }
                }

                method {
                    def myHelper() {
                        echo "Some complex set of steps"
                    }
                }
            }
            {code}

            Or consider this Scripted Pipeline: https://github.com/bitwiseman/hermann/blob/blog/notifications/Jenkinsfile

            {code}
            stage ('Build') {

              node {
                try {
                notifyBuild('STARTED')

                  // Checkout
                  checkout scm

                  // install required bundles
                  sh 'bundle install'

                  // build and run tests with coverage
                  sh 'bundle exec rake build spec'

                  // Archive the built artifacts
                  archive (includes: 'pkg/*.gem')

                  // publish html
                  publishHTML ([
                      allowMissing: false,
                      alwaysLinkToLastBuild: false,
                      keepAll: true,
                      reportDir: 'coverage',
                      reportFiles: 'index.html',
                      reportName: "RCov Report"
                    ])
                } catch (Exception e) {
                    currentBuild.result = "FAILED"
                    throw e
                } finally {
                  notifyBuild(currentBuild.result)
                }
              }
            }

            def notifyBuild(String buildStatus = 'STARTED') {
              // build status of null means successful
              buildStatus = buildStatus ?: 'SUCCESSFUL'

              // Default values
              def colorName = 'RED'
              def colorCode = '#FF0000'
              def subject = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"
              def summary = "${subject} (${env.BUILD_URL})"
              def details = """<p>STARTED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]':</p>
                <p>Check console output at &QUOT;<a href='${env.BUILD_URL}'>${env.JOB_NAME} [${env.BUILD_NUMBER}]</a>&QUOT;</p>"""

              // Override default values based on build status
              if (buildStatus == 'STARTED') {
                color = 'YELLOW'
                colorCode = '#FFFF00'
              } else if (buildStatus == 'SUCCESSFUL') {
                color = 'GREEN'
                colorCode = '#00FF00'
              } else {
                color = 'RED'
                colorCode = '#FF0000'
              }

              // Send notifications
              slackSend (color: colorCode, message: summary)

              hipchatSend (color: color, notify: true, message: summary)

              emailext (
                  to: 'bitwiseman@bitwiseman.com',
                  subject: subject,
                  body: details,
                  recipientProviders: [[$class: 'DevelopersRecipientProvider']]
                )
            }
            {code}

            The current guidance for migrating this to Declarative would be to keep {{notifyBuild}} outside of {{pipeline}}. This is not bad, but to anyone paying attention is a glaring gap in the Declarative model. We support {{script}} blocks under steps, but for cross-stage helpers we have to switch to shared libraries. I'll add more detail to this example shortly.


            jamesdumay James Dumay made changes -
            Summary Declarative: Add method definition section to root pipeline block -Add method definition section to root pipeline block
            jamesdumay James Dumay made changes -
            Summary -Add method definition section to root pipeline block Add method definition section to root pipeline block
            abayer Andrew Bayer made changes -
            Status Reopened [ 4 ] Resolved [ 5 ]
            Resolution Won't Fix [ 2 ]
            bitwiseman Liam Newman made changes -
            Status Resolved [ 5 ] Closed [ 6 ]

              People

              • Assignee:
                abayer Andrew Bayer
                Reporter:
                bitwiseman Liam Newman
              • Votes:
                12 Vote for this issue
                Watchers:
                19 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: