Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13348

EnvInject overriding WORKSPACE variable

    Details

    • Similar Issues:

      Description

      I upgraded Jenkins to 1.458 and envinject from 1.36 to 1.44. After the upgrade all my jobs that did not use envinject were getting their WORKSPACE variable set to another jobs that did use envinject WORKSPACE. Downgraded envinject to 1.36 and the problem went away.

      Here's an edited log that shows initially the workspace is correct, even after EnvInject line, but when the shell script runs, it is wrong.

      Also, I don't know why EnvInject is even being run for this job since it is not enabled anywhere...

      [EnvInject] - Preparing an environment for the build.
      Building on master in workspace -correct-workspace-
      Updating http://svn....
      At revision 36652
      no change for http://svn.... since the previous build
      No emails were triggered.
      [bronze-bin] $ /bin/sh -xe /tmp/hudson6983282044770433158.sh
      + echo -some-other-jobs-workspace-

        Attachments

          Activity

          jimsearle Jim Searle created issue -
          Hide
          gbois Gregory Boissinot added a comment -

          Strange behavior.
          I don't know where it is from and I can't reproduce it.
          Could you try to isolate it?
          Are your sure?

          Show
          gbois Gregory Boissinot added a comment - Strange behavior. I don't know where it is from and I can't reproduce it. Could you try to isolate it? Are your sure?
          gbois Gregory Boissinot made changes -
          Field Original Value New Value
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          jimsearle Jim Searle added a comment -

          Thanks. I am sure... I will try something on our test server and get back to you.

          Show
          jimsearle Jim Searle added a comment - Thanks. I am sure... I will try something on our test server and get back to you.
          Hide
          gbois Gregory Boissinot added a comment -

          Did you have time to test it?
          Thanks in advance.

          Show
          gbois Gregory Boissinot added a comment - Did you have time to test it? Thanks in advance.
          Hide
          gbois Gregory Boissinot added a comment -

          Any update?

          Show
          gbois Gregory Boissinot added a comment - Any update?
          Hide
          gbois Gregory Boissinot added a comment -

          I'm closing the issue at the moment because I cannot reproduce it without more inputs.
          Do not hesitate to reopen it when you will have more information.
          Thanks in advance.

          Show
          gbois Gregory Boissinot added a comment - I'm closing the issue at the moment because I cannot reproduce it without more inputs. Do not hesitate to reopen it when you will have more information. Thanks in advance.
          gbois Gregory Boissinot made changes -
          Status In Progress [ 3 ] Closed [ 6 ]
          Resolution Cannot Reproduce [ 5 ]
          Hide
          mixu Mikko Mattila added a comment -

          I'm seeing this same issue with Jenkins 1.454 & the EnvInject plugin 1.74-SNAPSHOT from yesterday. I had 1.72 when I initially encountered this. Not reopening the bug yet, because I have no further useful information.

          Show
          mixu Mikko Mattila added a comment - I'm seeing this same issue with Jenkins 1.454 & the EnvInject plugin 1.74-SNAPSHOT from yesterday. I had 1.72 when I initially encountered this. Not reopening the bug yet, because I have no further useful information.
          Hide
          mixu Mikko Mattila added a comment -

          The problem appears to get fixed by upgrading Jenkins core to 1.489.

          Show
          mixu Mikko Mattila added a comment - The problem appears to get fixed by upgrading Jenkins core to 1.489.
          Hide
          gbois Gregory Boissinot added a comment -

          Ok thanks Mikko for your last update.

          Show
          gbois Gregory Boissinot added a comment - Ok thanks Mikko for your last update.
          Hide
          byronbrummer Byron Brummer added a comment -

          The issue still exists. Jenkins 1.513, EnvInject 1.85

          Same story as the original reporter: Downgrading EnvInject to 1.36 fixes it.

          Show
          byronbrummer Byron Brummer added a comment - The issue still exists. Jenkins 1.513, EnvInject 1.85 Same story as the original reporter: Downgrading EnvInject to 1.36 fixes it.
          Hide
          nnau Natalia Naumova added a comment - - edited

          Strongly reproduced with scripttrigger plugin
          jenkins core: 1.492
          evinject: 1.82
          scripttrigger: 0.26, 0.23

          steps to reproduce:

          • JOB1. create any job that polls using scripttrigger plugin with poling node = master
          • JOB2. create job which is restricted to run on master and uses envinject
          • (restart jenkins - looks like it is needed)
          • run JOB2

          The results is that $WORKSPACE of JOB2 is defined correctly before envinject, but then it is re-defined with $WORKSPACE of job which use scripttrigger (JOB1).

          ==========
          Started by user ...
          [EnvInject] - Loading node environment variables.
          Building on master in workspace /jenkins/jobs/JOB2/workspace
          [EnvInject] - Executing scripts and injecting environment variables after the SCM step.
          [EnvInject] - Executing and processing the following script content:
          A=aaaa

          [workspace] $ /bin/sh -xe /tmp/hudson5302730654246338218.sh
          + A=aaaa
          [EnvInject] - Script executed successfully.
          [workspace] $ /bin/sh -xe /tmp/hudson6529723899012472418.sh
          + echo /jenkins/jobs/JOB1/workspace
          /jenkins/jobs/JOB1/workspace
          Notifying upstream projects of job completion
          Finished: SUCCESS
          ===========

          Can it be related that envinject and scripttrigger use different envinject-lib ?
          envinject uses envinject-lib-1.15.jar while scripttrigger uses envinject-lib-1.16.jar ?

          Show
          nnau Natalia Naumova added a comment - - edited Strongly reproduced with scripttrigger plugin jenkins core: 1.492 evinject: 1.82 scripttrigger: 0.26, 0.23 steps to reproduce: JOB1. create any job that polls using scripttrigger plugin with poling node = master JOB2. create job which is restricted to run on master and uses envinject (restart jenkins - looks like it is needed) run JOB2 The results is that $WORKSPACE of JOB2 is defined correctly before envinject, but then it is re-defined with $WORKSPACE of job which use scripttrigger (JOB1). ========== Started by user ... [EnvInject] - Loading node environment variables. Building on master in workspace /jenkins/jobs/JOB2/workspace [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Executing and processing the following script content: A=aaaa [workspace] $ /bin/sh -xe /tmp/hudson5302730654246338218.sh + A=aaaa [EnvInject] - Script executed successfully. [workspace] $ /bin/sh -xe /tmp/hudson6529723899012472418.sh + echo /jenkins/jobs/JOB1/workspace /jenkins/jobs/JOB1/workspace Notifying upstream projects of job completion Finished: SUCCESS =========== Can it be related that envinject and scripttrigger use different envinject-lib ? envinject uses envinject-lib-1.15.jar while scripttrigger uses envinject-lib-1.16.jar ?
          nnau Natalia Naumova made changes -
          Resolution Cannot Reproduce [ 5 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          Hide
          wgreven Wilco Greven added a comment -

          I can reproduce the issue using Natalia's steps, but not consistently. To check if WORKSPACE has been modified you can check the value of hudson.EnvVars.masterEnvVars using the script console.

          Could it be that the cause is in EnvVarsResolver.gatherEnvVarsNode? It looks like EnvVars.masterEnvVars is updated directly in that method when the code runs on master.

          Show
          wgreven Wilco Greven added a comment - I can reproduce the issue using Natalia's steps, but not consistently. To check if WORKSPACE has been modified you can check the value of hudson.EnvVars.masterEnvVars using the script console. Could it be that the cause is in EnvVarsResolver.gatherEnvVarsNode? It looks like EnvVars.masterEnvVars is updated directly in that method when the code runs on master.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Gregory Boissinot
          Path:
          src/main/java/org/jenkinsci/lib/envinject/service/EnvVarsResolver.java
          http://jenkins-ci.org/commit/envinject-lib/ad78b8f1833861e40d4c9d7a56fa7f72ede33bbc
          Log:
          Merge pull request #3 from wgreven/JENKINS-13348

          Don't overwrite masterEnvVars.

          Compare: https://github.com/jenkinsci/envinject-lib/compare/1a9def04d4d1...ad78b8f18338

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Gregory Boissinot Path: src/main/java/org/jenkinsci/lib/envinject/service/EnvVarsResolver.java http://jenkins-ci.org/commit/envinject-lib/ad78b8f1833861e40d4c9d7a56fa7f72ede33bbc Log: Merge pull request #3 from wgreven/ JENKINS-13348 Don't overwrite masterEnvVars. Compare: https://github.com/jenkinsci/envinject-lib/compare/1a9def04d4d1...ad78b8f18338
          gbois Gregory Boissinot made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          byronbrummer Byron Brummer added a comment -

          Gregory's patch didn't fix it, the problem still exists.

          EnvInject 1.87, Jenkins 1.519

          Downgrading to 1.36 to work around it. If I get some time I'll walk through releases to narrow down where the problem started.

          Is it possible to just hardcode 'WORKSPACE' as an untouchable variable? Or perhaps an "Advanced" check box to allow overriding core Jenkins variables of any kind?

          Show
          byronbrummer Byron Brummer added a comment - Gregory's patch didn't fix it, the problem still exists. EnvInject 1.87, Jenkins 1.519 Downgrading to 1.36 to work around it. If I get some time I'll walk through releases to narrow down where the problem started. Is it possible to just hardcode 'WORKSPACE' as an untouchable variable? Or perhaps an "Advanced" check box to allow overriding core Jenkins variables of any kind?
          Hide
          wgreven Wilco Greven added a comment -

          I don't think all plugins have already been rebuild with the new EnvInject version.

          What we did is creating a job which periodically runs the following Groovy system script:

          hudson.EnvVars.masterEnvVars.remove('WORKSPACE')

          It's not nice, but it minimizes the damage caused by this bug.

          Show
          wgreven Wilco Greven added a comment - I don't think all plugins have already been rebuild with the new EnvInject version. What we did is creating a job which periodically runs the following Groovy system script: hudson.EnvVars.masterEnvVars.remove( 'WORKSPACE' ) It's not nice, but it minimizes the damage caused by this bug.
          Hide
          adityachs aditya ch added a comment -

          The problem is still appearing with "Environment Injector Plugin" 1.89 and Jenkins ver. "1.509.4". Many of the jenkins variables are getting overridden with this plugin. Due to this all the working jobs getting failed.
          the configured workspace in job is /disc/c/new-artifacts/${JOB_NAME}
          the at the start of the job workspace was ok as defined except JOB_NAME variable
          the expected job name is "test1" where it shows some number generated:
          (just printed all the env varibales defined in jenkins)
          [EnvInject] - Loading node environment variables.
          [EnvInject] - Preparing an environment for the build.
          [EnvInject] - Keeping Jenkins system variables.
          [EnvInject] - Keeping Jenkins build variables.
          [EnvInject] - Injecting contributions.
          Building remotely on Compute Farm 1 in workspace /disc/c/new-artifacts/jk4-40146
          [jk4-40146] $ /bin/sh -xe /tmp/843283.1.jenkins.q/hudson7322359360021153706.sh
          + echo build_tagjenkins-test1-5
          build_tagjenkins-test1-5
          + echo 5
          5
          + echo 2014-05-07_13-25-10
          2014-05-07_13-25-10
          + echo jk4-40146
          jk4-40146
          + echo jenkins-test1-5
          jenkins-test1-5
          + echo 0
          0
          + echo Compute Farm 1
          Compute Farm 1
          + echo Compute Farm 1
          Compute Farm 1
          + echo /disc/d/tools/ci-jenkins/classic-ws/workspace-cf1
          /disc/d/tools/tools/ci-jenkins/classic-ws/workspace-cf1
          + echo /disc/d/tools/ci-jenkins/classic-ws
          /disc/d/tools/ci-jenkins/classic-ws
          + echo https://jenkins.mydomain.com/classic/
          https://jenkins.mydomain.com/classic/
          + echo https://jenkins.mydomain.com/classic/job/test1/5/
          https://jenkins.mydomain.com/classic/job/test1/5/
          + echo https://jenkins.mydomain.com/classic/job/test1/
          https://jenkins.mydomain.com/classic/job/test1/
          Finished: SUCCESS

          Show
          adityachs aditya ch added a comment - The problem is still appearing with "Environment Injector Plugin" 1.89 and Jenkins ver. "1.509.4". Many of the jenkins variables are getting overridden with this plugin. Due to this all the working jobs getting failed. the configured workspace in job is /disc/c/new-artifacts/${JOB_NAME} the at the start of the job workspace was ok as defined except JOB_NAME variable the expected job name is "test1" where it shows some number generated: (just printed all the env varibales defined in jenkins) [EnvInject] - Loading node environment variables. [EnvInject] - Preparing an environment for the build. [EnvInject] - Keeping Jenkins system variables. [EnvInject] - Keeping Jenkins build variables. [EnvInject] - Injecting contributions. Building remotely on Compute Farm 1 in workspace /disc/c/new-artifacts/jk4-40146 [jk4-40146] $ /bin/sh -xe /tmp/843283.1.jenkins.q/hudson7322359360021153706.sh + echo build_tagjenkins-test1-5 build_tagjenkins-test1-5 + echo 5 5 + echo 2014-05-07_13-25-10 2014-05-07_13-25-10 + echo jk4-40146 jk4-40146 + echo jenkins-test1-5 jenkins-test1-5 + echo 0 0 + echo Compute Farm 1 Compute Farm 1 + echo Compute Farm 1 Compute Farm 1 + echo /disc/d/tools/ci-jenkins/classic-ws/workspace-cf1 /disc/d/tools/tools/ci-jenkins/classic-ws/workspace-cf1 + echo /disc/d/tools/ci-jenkins/classic-ws /disc/d/tools/ci-jenkins/classic-ws + echo https://jenkins.mydomain.com/classic/ https://jenkins.mydomain.com/classic/ + echo https://jenkins.mydomain.com/classic/job/test1/5/ https://jenkins.mydomain.com/classic/job/test1/5/ + echo https://jenkins.mydomain.com/classic/job/test1/ https://jenkins.mydomain.com/classic/job/test1/ Finished: SUCCESS
          adityachs aditya ch made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          lacostej lacostej added a comment -

          I am hitting the same issue. EnvInject 1.89 and jenkins 1.532.3 and WORKSPACE got set into hudson.EnvVars.masterEnvVars which caused lots of strange build issues. This is really a bummer.

          Show
          lacostej lacostej added a comment - I am hitting the same issue. EnvInject 1.89 and jenkins 1.532.3 and WORKSPACE got set into hudson.EnvVars.masterEnvVars which caused lots of strange build issues. This is really a bummer.
          Hide
          ceilfors Wisen Tanasa added a comment -

          I'm hitting the same issue with EnvInject 1.89 and Jenkins 1.553.

          Show
          ceilfors Wisen Tanasa added a comment - I'm hitting the same issue with EnvInject 1.89 and Jenkins 1.553.
          Hide
          david_rubio David Rubio added a comment - - edited

          Similar problem detected with EnvInject 1.91.1 and Jenkins 1.609. Version 1.90 does not have the problem

          For some reason the Path env variable is not updated but java.library.path is. When you try to build, it looks in the old path

          Show
          david_rubio David Rubio added a comment - - edited Similar problem detected with EnvInject 1.91.1 and Jenkins 1.609. Version 1.90 does not have the problem For some reason the Path env variable is not updated but java.library.path is. When you try to build, it looks in the old path
          Hide
          deepchip Martin d'Anjou added a comment -

          Do you still see the problem with the 1.92.1 released on Aug 21 2015?

          Show
          deepchip Martin d'Anjou added a comment - Do you still see the problem with the 1.92.1 released on Aug 21 2015?
          Hide
          lautou Laurent TOURREAU added a comment - - edited

          I have installed 1.92.1 the problem is present.
          I have downgraded until 1.38 which is the latest version that doesn't have this defect.

          Here a log extract:

          Building remotely on quality in workspace /apps/homes/jenkins/node_quality/workspace/sonar_build
          [sonar_build] $ /bin/sh -xe /tmp/hudson4867033689412661438.sh
          + echo now Launching the build workspace...
          now Launching the build workspace...
          + /apps/tools/misc-tools/build_test.sh
          WORKSPACE=/apps/tools/homes/jenkins/node_quality
          

          As you can see WORKSPACE variable is not set correctly

          Show
          lautou Laurent TOURREAU added a comment - - edited I have installed 1.92.1 the problem is present. I have downgraded until 1.38 which is the latest version that doesn't have this defect. Here a log extract: Building remotely on quality in workspace /apps/homes/jenkins/node_quality/workspace/sonar_build [sonar_build] $ /bin/sh -xe /tmp/hudson4867033689412661438.sh + echo now Launching the build workspace... now Launching the build workspace... + /apps/tools/misc-tools/build_test.sh WORKSPACE=/apps/tools/homes/jenkins/node_quality As you can see WORKSPACE variable is not set correctly
          Hide
          ajbarber Andrew Barber added a comment -

          I see this issue as well. EnvInject 1.92.1, jenkins 1.642.1. I originally posted this bug
          https://issues.jenkins-ci.org/browse/JENKINS-19754

          The work around to get WORKSPACE set correctly is to
          ○ Select "Prepare jobs environment" on the slave
          ○ Select "Unset System Environment Variables" on the slave

          After recent upgrades on jenkins and plugins I now see intermittent env inject of WORKSPACE to the slave root. I think the "unset system environment variables" is being ignored intermittently. Passing builds have a small set of environment variables set which appear to be only jenkins related. Failing builds have many more environment variables, especially system ones like HOST and LSF environment variables. The passing cases do not have WORKSPACE set.
          This is causing a lot of grief for our work. I hope someone can look into it.

          Show
          ajbarber Andrew Barber added a comment - I see this issue as well. EnvInject 1.92.1, jenkins 1.642.1. I originally posted this bug https://issues.jenkins-ci.org/browse/JENKINS-19754 The work around to get WORKSPACE set correctly is to ○ Select "Prepare jobs environment" on the slave ○ Select "Unset System Environment Variables" on the slave After recent upgrades on jenkins and plugins I now see intermittent env inject of WORKSPACE to the slave root. I think the "unset system environment variables" is being ignored intermittently. Passing builds have a small set of environment variables set which appear to be only jenkins related. Failing builds have many more environment variables, especially system ones like HOST and LSF environment variables. The passing cases do not have WORKSPACE set. This is causing a lot of grief for our work. I hope someone can look into it.
          Hide
          ajbarber Andrew Barber added a comment -

          I should also note that the job affected by this does not have "Prepare an environment for the run" selected. EnvInject is still lurking though as this message is always present in stdout:
          [EnvInject] - Loading node environment variables.

          Show
          ajbarber Andrew Barber added a comment - I should also note that the job affected by this does not have "Prepare an environment for the run" selected. EnvInject is still lurking though as this message is always present in stdout: [EnvInject] - Loading node environment variables.
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 143812 ] JNJira + In-Review [ 186055 ]

            People

            • Assignee:
              gbois Gregory Boissinot
              Reporter:
              jimsearle Jim Searle
            • Votes:
              8 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

              • Created:
                Updated: