Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50810

postbuildscript not running on master

    Details

    • Type: Bug
    • Status: Fixed but Unreleased (View Workflow)
    • Priority: Major
    • Resolution: Won't Fix
    • Component/s: postbuildscript-plugin
    • Labels:
      None
    • Environment:
      Linux, Ubuntu 16 LTS
      Jenkins 2.107.2
      postbuildscript plugin 2.6.0
    • Similar Issues:

      Description

      Jenkins setup is master plus a bunch of slaves

      • Master node has no label verify
      • All slaves have label verify

      Job (jjb attached below) is three parts:

      1. pre-part to run only in master (using wrappers/env-script)
      2. Then different parts running in slaves as part of a matrix and copy their build output back to master with a postbuildscript / copy-to-master
      3. postbuildscript to postprocess the data in master

      I cannot get step 3 to work, with seemingly all the values set correctly. Step (3) is never executed in master (nor in the slaves).

       

      I can only get it to work if I label the master as verify, then the step (1) will run there, maybe a (2) slave and then immediately (3). However, this does not work (among other things) because I need all the build products of all (2) phases in the master before (3) runs.

      JJB with jenkins-jobs 2.0.5-16-g82f2ab44 from https://git.openstack.org/openstack-infra/jenkins-job-builder (needed for wrappers/env-script/run-only-on-parent to work):

      - project:
          name: prj1
          jobs:
              - job-fake-{param_prj_name}:
                  param_prj_name: prj1
      
      - job-template:
          id: job-fake-{param_prj_name}
          name: fake-{param_prj_name}
          concurrent: false
      
          # Need to run on master to get all the stuff from the raw builds
          node: master
          
          project-type: matrix
          axes:
            # Run with each toolchain
            - axis:
                type: user-defined
                name: AXIS1
                values:
                  - 1a
                  - 1b
      
            - axis:
                type: user-defined
                name: AXIS2
                values:
                  - 2a
                  - 2b
      
            - axis:
                type: slave
                name: LABEL
                values:
                  - verify
          
          logrotate:
              numToKeep: 2
              artifactNumToKeep: 10
      
          wrappers:
      
              - env-script:
                  # Run this only on parent to generate a unique RUNID for
                  # the whole job
                  run-only-on-parent: true
                  only-run-on-parent: true
                  script-type: unix-script
                  script-content: |
                    echo $(TZ=US/Pacific date +%y%m%d-%H%M) > yymmdd_hhmm.txt
                    echo PROP_YYMMDD_HHMM=$(cat yymmdd_hhmm.txt)
                    echo DEBUG_the_prescript_is_running_in=$NODE_NAME
      
          builders:
            # The builders are supposed to be running on the slaves
            - inject:
                properties-file: /etc/environment
                properties-content: |
                  RUNID: ci-${{PROP_YYMMDD_HHMM}}-$BUILD_NUMBER
            - shell: |
                echo "$NODE_NAME: running BUILD script $RUNID ($AXIS1 $AXIS2 $LABEL)"
                echo $AXIS1 $AXIS2 $LABEL > build-$SHARD-$ZEPHYR_GCC_VARIANT
          
          publishers:
            - copy-to-master:
                includes:
                  # copy all the build output to the master for
                  # post-processing from the slaves
                  - build-*
       
            - archive:
                artifacts: build-*
                allow-empty: true
                only-if-success: false
      
            - postbuildscript:
      
                builders:
                  - role: MASTER
                    # if aborted, do not run it
                    build-on:
                      - SUCCESS
                      - FAILURE
                      - UNSTABLE
                    build-steps:
                      - shell: |
                          # postprocess the output of all the slaves
                          ls -l build-*
                          echo $NODE_NAME: running POSTBUILD script runid $RUNID    
      

       

       

       

        Attachments

          Activity

          Hide
          dheid Daniel Heid added a comment -

          Thanks for your bug report and sorry for responding so late.

          For matrix jobs you need to use the MatrixPostBuildScript. The resulting publishers section in the config.xml must look like this:

          <publishers>
              <org.jenkinsci.plugins.postbuildscript.MatrixPostBuildScript plugin="postbuildscript@2.7.0">
                <config>
                  <scriptFiles/>
                  <groovyScripts/>
                  <buildSteps>
                    <org.jenkinsci.plugins.postbuildscript.model.PostBuildStep>
                      <results>
                        <string>SUCCESS</string>
                        <string>NOT_BUILT</string>
                        <string>ABORTED</string>
                        <string>FAILURE</string>
                        <string>UNSTABLE</string>
                      </results>
                      <role>BOTH</role>
                      <executeOn>BOTH</executeOn>
                      <buildSteps>
                        <hudson.tasks.Shell>
                          <command>echo hello</command>
                        </hudson.tasks.Shell>
                      </buildSteps>
                    </org.jenkinsci.plugins.postbuildscript.model.PostBuildStep>
                  </buildSteps>
                  <markBuildUnstable>false</markBuildUnstable>
                </config>
              </org.jenkinsci.plugins.postbuildscript.MatrixPostBuildScript>
            </publishers>
          

          Important is the "executeOn" section, that is only available for the MatrixPostBuildScript.

          It takes more time for me to install additional tools to create the jobs. Would you please just provide me the config.xml of the job?

          Please try the newest version. There were some problems with failing build steps.

          Show
          dheid Daniel Heid added a comment - Thanks for your bug report and sorry for responding so late. For matrix jobs you need to use the MatrixPostBuildScript. The resulting publishers section in the config.xml must look like this: <publishers> <org.jenkinsci.plugins.postbuildscript.MatrixPostBuildScript plugin= "postbuildscript@2.7.0" > <config> <scriptFiles/> <groovyScripts/> <buildSteps> <org.jenkinsci.plugins.postbuildscript.model.PostBuildStep> <results> <string>SUCCESS</string> <string>NOT_BUILT</string> <string>ABORTED</string> <string>FAILURE</string> <string>UNSTABLE</string> </results> <role>BOTH</role> <executeOn>BOTH</executeOn> <buildSteps> <hudson.tasks.Shell> <command>echo hello</command> </hudson.tasks.Shell> </buildSteps> </org.jenkinsci.plugins.postbuildscript.model.PostBuildStep> </buildSteps> <markBuildUnstable> false </markBuildUnstable> </config> </org.jenkinsci.plugins.postbuildscript.MatrixPostBuildScript> </publishers> Important is the "executeOn" section, that is only available for the MatrixPostBuildScript. It takes more time for me to install additional tools to create the jobs. Would you please just provide me the config.xml of the job? Please try the newest version. There were some problems with failing build steps.
          Hide
          inakypg Inaky Perez-Gonzalez added a comment -

          Thank you, Daniel Heid; something important you mentioned set off this: we use jenkins-job-builder, I am not seeing how can I force it to report a MatrixPostBuildScript vs PostBuildSscript (using 2.0.10), so this'd be the first thing to figure out; currently the xml it generates is (for my sample job):

              <org.jenkinsci.plugins.postbuildscript.PostBuildScript>
                <config>
                  <markBuildUnstable>false</markBuildUnstable>
                  <scriptFiles/>
                  <groovyScripts/>
                  <buildSteps>
                    <org.jenkinsci.plugins.postbuildscript.model.PostBuildStep>
                      <results>
                        <string>SUCCESS</string>
                        <string>FAILURE</string>
                        <string>UNSTABLE</string>
                      </results>
                      <role>MASTER</role>
                      <buildSteps>
                        <hudson.tasks.Shell>
                          <command># postprocess the output of all the slaves
          ls -l build-*
          echo $NODE_NAME: running POSTBUILD script runid $RUNID    
          </command>
                        </hudson.tasks.Shell>
                      </buildSteps>
                    </org.jenkinsci.plugins.postbuildscript.model.PostBuildStep>
                  </buildSteps>
                </config>
              </org.jenkinsci.plugins.postbuildscript.PostBuildScript>
          

          And then of course, it is missing the executeOn – Looking at the latest jenkins-jobs-builder (2.0.10-12-g561081fe), seems jenkins-jobs considers it to be a deprecated option (for what it was postbuildscript).

          And certainly, no mention of MAtrixPostBuildScript

           

          Maybe

          Show
          inakypg Inaky Perez-Gonzalez added a comment - Thank you, Daniel Heid ; something important you mentioned set off this: we use jenkins-job-builder, I am not seeing how can I force it to report a MatrixPostBuildScript vs PostBuildSscript (using 2.0.10), so this'd be the first thing to figure out; currently the xml it generates is (for my sample job):     <org.jenkinsci.plugins.postbuildscript.PostBuildScript>       <config>         <markBuildUnstable>false</markBuildUnstable>         <scriptFiles/>         <groovyScripts/>         <buildSteps>           <org.jenkinsci.plugins.postbuildscript.model.PostBuildStep>             <results>               <string>SUCCESS</string>               <string>FAILURE</string>               <string>UNSTABLE</string>             </results>             <role>MASTER</role>             <buildSteps>               <hudson.tasks.Shell>                 <command># postprocess the output of all the slaves ls -l build-* echo $NODE_NAME: running POSTBUILD script runid $RUNID     </command>               </hudson.tasks.Shell>             </buildSteps>           </org.jenkinsci.plugins.postbuildscript.model.PostBuildStep>         </buildSteps>       </config>     </org.jenkinsci.plugins.postbuildscript.PostBuildScript> And then of course, it is missing the executeOn – Looking at the latest jenkins-jobs-builder (2.0.10-12-g561081fe), seems jenkins-jobs considers it to be a deprecated option (for what it was postbuildscript). And certainly, no mention of MAtrixPostBuildScript   Maybe

            People

            • Assignee:
              dheid Daniel Heid
              Reporter:
              inakypg Inaky Perez-Gonzalez
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: