Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41146

SCM Trigger configuration being overwritten by SYSTEM user in Pipeline

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Not A Defect
    • Component/s: pipeline
    • Environment:
      Jenkins 2.8
      BlueOcean 1.0.0-b17
      CentOS 6.7
      java version "1.7.0_101"
      OpenJDK Runtime Environment (rhel-2.6.6.4.el6_8-x86_64 u101-b00)
      OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)
    • Similar Issues:

      Description

      I setup a Pipeline job using a Pipeline script written in the job (not in SCM) and enabled SCM polling as a Build Trigger.

      The polling mechanism works and triggers a build after detecting changes, however whenever it polls there is a high chance that the config.xml will be modified, removing the SCM Trigger entirely.

      I used the Job Configuration History plugin and it shows the changes happening from SYSTEM user.

      Looking through jenkins.log doesn't provide much information either, as this is the only entry around the same time as the SYSTEM user updates the config.xml

      Jan 17, 2017 2:17:24 PM hudson.triggers.SCMTrigger$Runner run
      INFO: SCM changes detected in unified-trunk. Triggering #84

        Attachments

          Issue Links

            Activity

            jonathandyer Jonathan Dyer created issue -
            jonathandyer Jonathan Dyer made changes -
            Field Original Value New Value
            Description I setup a Pipeline job using a Pipeline script written in the job (not in SCM) and enabled SCM polling as a Build Trigger.

            The polling mechanism works and triggers a build after detecting changes, however whenever it polls there is a high chance that the config.xml will be modified, removing the SCM Trigger entirely.
             !image-2017-01-17-14-43-37-557.png|thumbnail!

            I used the Job Configuration History plugin and it shows the changes happening from SYSTEM user.
             !image-2017-01-17-14-41-29-596.png|thumbnail!





            I setup a Pipeline job using a Pipeline script written in the job (not in SCM) and enabled SCM polling as a Build Trigger.

            The polling mechanism works and triggers a build after detecting changes, however whenever it polls there is a high chance that the config.xml will be modified, removing the SCM Trigger entirely.
             !image-2017-01-17-14-43-37-557.png|thumbnail!

            I used the Job Configuration History plugin and it shows the changes happening from SYSTEM user.
             !image-2017-01-17-14-41-29-596.png|thumbnail!

            Looking through jenkins.log doesn't provide much information either, as this is the only entry around the same time as the SYSTEM user updates the config.xml
            {quote}
            Jan 17, 2017 2:17:24 PM hudson.triggers.SCMTrigger$Runner run
            INFO: SCM changes detected in unified-trunk. Triggering #84
            {quote}





            Hide
            jonathandyer Jonathan Dyer added a comment -

            I've discovered the issue, in my pipeline script I was defining job properties and it had an empty pipelineTriggers section.

            It wasn't clear to me that this script would actually modify the config.xml on the master, and that not using a build trigger initially in the snippet generator it would default to being empty thus removing any build triggers within the config.xml.

            Show
            jonathandyer Jonathan Dyer added a comment - I've discovered the issue, in my pipeline script I was defining job properties and it had an empty pipelineTriggers section. It wasn't clear to me that this script would actually modify the config.xml on the master, and that not using a build trigger initially in the snippet generator it would default to being empty thus removing any build triggers within the config.xml.
            Hide
            jamesdumay James Dumay added a comment -

            Jonathan Dyer would you be able to post the Pipeline here that demonstrated the original problem? We might be able to make the error handling here a lot better.

            Show
            jamesdumay James Dumay added a comment - Jonathan Dyer would you be able to post the Pipeline here that demonstrated the original problem? We might be able to make the error handling here a lot better.
            jamesdumay James Dumay made changes -
            Component/s blueocean-plugin [ 21481 ]
            Hide
            jonathandyer Jonathan Dyer added a comment -

            James Dumay,

            // Setup Job properties
                properties([buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '30', numToKeepStr: '')), disableConcurrentBuilds(), pipelineTriggers([])])
            

            The offending bit is "pipelineTriggers([])".

            It's now my understanding that any properties defined within the pipeline script will supersede and overwrite on disk the properties manually set on the Job itself.

            I have tested this theory and created a job with a single build parameter, then used the exact same properties I noted above which has no mention of build parameters. The initial run allowed me to select a build parameter, but after that the config changed and it was no longer an option.

            So it appears that pipeline properties will override all configurations it is able to, regardless if specified within the pipeline script or not.

            Show
            jonathandyer Jonathan Dyer added a comment - James Dumay , // Setup Job properties properties([buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: ' ', daysToKeepStr: ' 30 ', numToKeepStr: ' ')), disableConcurrentBuilds(), pipelineTriggers([])]) The offending bit is "pipelineTriggers([])". It's now my understanding that any properties defined within the pipeline script will supersede and overwrite on disk the properties manually set on the Job itself. I have tested this theory and created a job with a single build parameter, then used the exact same properties I noted above which has no mention of build parameters. The initial run allowed me to select a build parameter, but after that the config changed and it was no longer an option. So it appears that pipeline properties will override all configurations it is able to, regardless if specified within the pipeline script or not.
            Hide
            jonathandyer Jonathan Dyer added a comment -

            An even more simple test of a pipeline job with just

            properties([])

            will cause all properties to be overwritten/removed with defaults.

            Show
            jonathandyer Jonathan Dyer added a comment - An even more simple test of a pipeline job with just properties([]) will cause all properties to be overwritten/removed with defaults.
            hrmpw Patrick Wolf made changes -
            Link This issue is related to JENKINS-41072 [ JENKINS-41072 ]
            hrmpw Patrick Wolf made changes -
            Link This issue is related to JENKINS-41073 [ JENKINS-41073 ]
            hrmpw Patrick Wolf made changes -
            Link This issue is related to JENKINS-41074 [ JENKINS-41074 ]
            Hide
            abayer Andrew Bayer added a comment -

            Yup, that's, for good or for ill, by design. If you check your build's logs, you'll see it warning you that you're overwriting your properties.

            Show
            abayer Andrew Bayer added a comment - Yup, that's, for good or for ill, by design. If you check your build's logs, you'll see it warning you that you're overwriting your properties.
            Hide
            jonathandyer Jonathan Dyer added a comment -

            You're absolutely right, it does mention that in the build logs.

            WARNING: The properties step will remove all JobPropertys currently configured in this job, either from the UI or from an earlier properties step.
            This includes configuration for discarding old builds, parameters, concurrent builds and build triggers.
            WARNING: Removing existing job property 'Discard old builds'
            WARNING: Removing existing job property 'Do not allow concurrent builds'
            WARNING: Removing existing job property 'This project is parameterized'
            WARNING: Removing existing job property 'Build triggers'

            I'm still not a fan of this behaviour

            Show
            jonathandyer Jonathan Dyer added a comment - You're absolutely right, it does mention that in the build logs. WARNING: The properties step will remove all JobPropertys currently configured in this job, either from the UI or from an earlier properties step. This includes configuration for discarding old builds, parameters, concurrent builds and build triggers. WARNING: Removing existing job property 'Discard old builds' WARNING: Removing existing job property 'Do not allow concurrent builds' WARNING: Removing existing job property 'This project is parameterized' WARNING: Removing existing job property 'Build triggers' I'm still not a fan of this behaviour
            Hide
            jamesdumay James Dumay added a comment -

            What do you want it to do instead?

            Show
            jamesdumay James Dumay added a comment - What do you want it to do instead?
            Hide
            hrmpw Patrick Wolf added a comment -

            We couldn't find an easy way to do some properties but not all in settings in the current configuration.The best way forward is to use everything as config as code. We are trying to push Jenkins in that direction.

            The good news is that the new Declarative Syntax makes it much easier to define these properties in your Jenkinsfile. On top of that the new Pipeline Editor being developed with Blue Ocean will let you configure these settings in your Jenkinsfile with a GUI.

            Show
            hrmpw Patrick Wolf added a comment - We couldn't find an easy way to do some properties but not all in settings in the current configuration.The best way forward is to use everything as config as code. We are trying to push Jenkins in that direction. The good news is that the new Declarative Syntax makes it much easier to define these properties in your Jenkinsfile. On top of that the new Pipeline Editor being developed with Blue Ocean will let you configure these settings in your Jenkinsfile with a GUI.
            Hide
            jonathandyer Jonathan Dyer added a comment -

            James Dumay, I wouldn't have expected using it to first, update the jobs configuration on disk and second, to touch properties not explicitly set in the pipeline.

            Maybe just make it more clear that this is the default behaviour rather than just spitting out warnings in the build log.

            Show
            jonathandyer Jonathan Dyer added a comment - James Dumay , I wouldn't have expected using it to first, update the jobs configuration on disk and second, to touch properties not explicitly set in the pipeline. Maybe just make it more clear that this is the default behaviour rather than just spitting out warnings in the build log.
            Hide
            jonathandyer Jonathan Dyer added a comment -

            Patrick Wolf, the declarative syntax looks very promising!

            I agree with the sentiment of "config as code" however JENKINS-33273 and our large codebase forced me to rely upon a simple pipeline job and I tried to leverage both the pipeline properties and the ease of changing job config on a whim and got unexpected results.

            I have since switched to just using the pipeline properties exclusively.

            Show
            jonathandyer Jonathan Dyer added a comment - Patrick Wolf , the declarative syntax looks very promising! I agree with the sentiment of "config as code" however JENKINS-33273 and our large codebase forced me to rely upon a simple pipeline job and I tried to leverage both the pipeline properties and the ease of changing job config on a whim and got unexpected results. I have since switched to just using the pipeline properties exclusively.
            Hide
            jglick Jesse Glick added a comment -

            As designed. I will emphasize this behavior in the step documentation though.

            Show
            jglick Jesse Glick added a comment - As designed. I will emphasize this behavior in the step documentation though.
            jglick Jesse Glick made changes -
            Status Open [ 1 ] Resolved [ 5 ]
            Resolution Not A Defect [ 7 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "workflow-multibranch PR 52 (Web Link)" [ 15353 ]
            Hide
            as_kumlali Ali Sadik Kumlali added a comment -

            workflow-multibranch's documentation for properties step is not clear in that manner. It only says properties step "updates" the properties. It is not clear that it will override all properties configured in this job, either from the UI or from an earlier properties step.

            properties: Set job properties

            Updates the properties of the job which runs this step. Mainly useful from multibranch Pipelines, so that Jenkinsfile itself can encode what would otherwise be static job configuration.

            Show
            as_kumlali Ali Sadik Kumlali added a comment - workflow-multibranch's documentation for properties step is not clear in that manner. It only says properties step "updates" the properties. It is not clear that it will override all properties configured in this job, either from the UI or from an earlier properties step. properties: Set job properties Updates the properties of the job which runs this step. Mainly useful from multibranch Pipelines, so that Jenkinsfile itself can encode what would otherwise be static job configuration.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            src/main/resources/org/jenkinsci/plugins/workflow/multibranch/JobPropertyStep/help.html
            http://jenkins-ci.org/commit/workflow-multibranch-plugin/5c7f723cd8b807a8c7ff10d349c55c531a2860ea
            Log:
            Emphasize JENKINS-41146.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/resources/org/jenkinsci/plugins/workflow/multibranch/JobPropertyStep/help.html http://jenkins-ci.org/commit/workflow-multibranch-plugin/5c7f723cd8b807a8c7ff10d349c55c531a2860ea Log: Emphasize JENKINS-41146 .
            tknerr Torben Knerr made changes -
            Link This issue is related to JENKINS-45053 [ JENKINS-45053 ]
            Hide
            tknerr Torben Knerr added a comment -

            Imho it would be really useful if the `properties` step would provide an additive behaviour as an optional alternative to the current destructive behaviour.

            For now I'm working around this by using the Jenkins API directly instead of the Pipeline DSL's `properties` step:
            https://issues.jenkins-ci.org/browse/JENKINS-45053?focusedCommentId=304390&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-304390

            Not sure if this is a good idea really, but the best workaround I could find so far.

            Jesse Glick opinions?

            Show
            tknerr Torben Knerr added a comment - Imho it would be really useful if the `properties` step would provide an additive behaviour as an optional alternative to the current destructive behaviour. For now I'm working around this by using the Jenkins API directly instead of the Pipeline DSL's `properties` step: https://issues.jenkins-ci.org/browse/JENKINS-45053?focusedCommentId=304390&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-304390 Not sure if this is a good idea really, but the best workaround I could find so far. Jesse Glick opinions?
            Show
            jglick Jesse Glick added a comment - See JENKINS-44848 . https://wiki.jenkins.io/display/JENKINS/Pipeline+Multibranch+Plugin

              People

              • Assignee:
                Unassigned
                Reporter:
                jonathandyer Jonathan Dyer
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: