Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46120

OOM: MultiJob and adding value to end of environment variable, eg. path

    Details

    • Similar Issues:

      Description

      Multijob fails to handle (global) environment variable that is adding value to end of variable eg. for path.

      Finally the issue seems to be "minor" but I lefted it open because Multijob was only "project model" to fail on this.

      Global properties in system preference:

      LD_LIBRARY_PATH
      /opt/oracle/instantclient:$LD_LIBRARY_PATH

       

      This is from end to change build name which also fails for OOM heap.

      Set build name.
      13:36:46 FATAL: Java heap space
      13:36:46 java.lang.OutOfMemoryError: Java heap space
      13:36:46 	at java.util.Arrays.copyOf(Arrays.java:3332)
      13:36:46 	at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
      13:36:46 	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
      13:36:46 	at java.lang.StringBuilder.append(StringBuilder.java:136)
      13:36:46 	at hudson.Util.replaceMacro(Util.java:180)
      13:36:46 	at hudson.Util.replaceMacro(Util.java:146)
      13:36:46 	at hudson.EnvVars.resolve(EnvVars.java:343)
      13:36:46 	at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:886)
      13:36:46 	at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:225)
      13:36:46 	at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:216)
      13:36:46 	at org.jenkinsci.plugins.tokenmacro.TokenMacro.expandAll(TokenMacro.java:208)
      13:36:46 	at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.setDisplayName(BuildNameSetter.java:74)
      13:36:46 	at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter.access$000(BuildNameSetter.java:31)
      13:36:46 	at org.jenkinsci.plugins.buildnamesetter.BuildNameSetter$1.tearDown(BuildNameSetter.java:64)
      13:36:46 	at hudson.model.Build$BuildExecution.doRun(Build.java:174)
      13:36:46 	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:490)
      13:36:46 	at com.tikal.jenkins.plugins.multijob.MultiJobBuild$MultiJobRunnerImpl.run(MultiJobBuild.java:136)
      13:36:46 	at hudson.model.Run.execute(Run.java:1737)
      13:36:46 	at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:73)
      13:36:46 	at hudson.model.ResourceController.execute(ResourceController.java:97)
      13:36:46 	at hudson.model.Executor.run(Executor.java:419)
      13:36:59 Notifying upstream projects of job completion
      

        Attachments

        1. Reachability table of heap dump.png
          Reachability table of heap dump.png
          410 kB
        2. job2.xml
          24 kB
        3. job.xml
          31 kB
        4. Class based heap usage.png
          Class based heap usage.png
          175 kB
        5. Biggest objects 2,2GB for MultijobBuild.png
          Biggest objects 2,2GB for MultijobBuild.png
          315 kB

          Issue Links

            Activity

            Hide
            heikkisi Heikki Simperi added a comment - - edited

            Added as attachment second plain multijob project xml file with same result. This is console printout:

             

            12:56:55 Started by user Name
            12:56:55 [EnvInject] - Loading node environment variables.
            12:56:55 Building remotely on hara mini (REMOTE_MINI_JOBS JENKINS_MINI) in workspace /mnt/jenkins/mini/workspace/31-PROJ-CUST-deploy-all
            12:56:55 [WS-CLEANUP] Deleting project workspace...
            12:56:55 [WS-CLEANUP] Done
            12:56:55 Copied 2 artifacts from "00-PROJ-build" build number 7957
            12:56:55 [EnvInject] - Injecting environment variables from a build step.
            12:56:55 [EnvInject] - Injecting as environment variables the properties file path 'version.properties'
            12:56:55 [EnvInject] - Variables injected successfully.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:55 Starting build job 31-PROJ-CUST-deploy.
            12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:56 Starting build job 31-PROJ-CUST-deploy.
            12:56:56 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:56:57 Starting build job 31-PROJ-CUST-deploy.
            12:56:57 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:57:00 Starting build job 31-PROJ-CUST-deploy.
            12:57:01 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:57:10 Starting build job 31-PROJ-CUST-deploy.
            12:57:11 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:57:40 Starting build job 31-PROJ-CUST-deploy.
            12:57:44 >> Job status: Warning: [31-PROJ-CUST-deploy] subjob in the phase configuration is disabled.
            12:59:17 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled.
            12:59:29 [MultiJob] - [ERROR] - Problems occurs on injecting env vars as a build step: Java heap space
            12:59:29 Starting build job 31-PROJ-CUST-deploy.
            12:59:39 FATAL: Java heap space
            12:59:39 java.lang.OutOfMemoryError: Java heap space
            12:59:40 Notifying upstream projects of job completion
            12:59:40 Finished: FAILURE
            Show
            heikkisi Heikki Simperi added a comment - - edited Added as attachment second plain multijob project xml file with same result. This is console printout:   12:56:55 Started by user Name 12:56:55 [EnvInject] - Loading node environment variables. 12:56:55 Building remotely on hara mini (REMOTE_MINI_JOBS JENKINS_MINI) in workspace /mnt/jenkins/mini/workspace/31-PROJ-CUST-deploy-all 12:56:55 [WS-CLEANUP] Deleting project workspace... 12:56:55 [WS-CLEANUP] Done 12:56:55 Copied 2 artifacts from "00-PROJ-build" build number 7957 12:56:55 [EnvInject] - Injecting environment variables from a build step. 12:56:55 [EnvInject] - Injecting as environment variables the properties file path 'version.properties' 12:56:55 [EnvInject] - Variables injected successfully. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:55 Starting build job 31-PROJ-CUST-deploy. 12:56:55 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:56 Starting build job 31-PROJ-CUST-deploy. 12:56:56 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:56:57 Starting build job 31-PROJ-CUST-deploy. 12:56:57 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:57:00 Starting build job 31-PROJ-CUST-deploy. 12:57:01 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:57:10 Starting build job 31-PROJ-CUST-deploy. 12:57:11 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:57:40 Starting build job 31-PROJ-CUST-deploy. 12:57:44 >> Job status: Warning: [31-PROJ-CUST-deploy] subjob in the phase configuration is disabled. 12:59:17 >> Job status: [31-PROJ-CUST-deploy] the 'build only if scm changes' feature is disabled. 12:59:29 [MultiJob] - [ERROR] - Problems occurs on injecting env vars as a build step: Java heap space 12:59:29 Starting build job 31-PROJ-CUST-deploy. 12:59:39 FATAL: Java heap space 12:59:39 java.lang.OutOfMemoryError: Java heap space 12:59:40 Notifying upstream projects of job completion 12:59:40 Finished: FAILURE
            Hide
            heikkisi Heikki Simperi added a comment -

            It seems that master node is running out of Heap

            Show
            heikkisi Heikki Simperi added a comment - It seems that master node is running out of Heap
            Hide
            heikkisi Heikki Simperi added a comment -

            Added multiple screenshots from heap dump

            Show
            heikkisi Heikki Simperi added a comment - Added multiple screenshots from heap dump
            Hide
            heikkisi Heikki Simperi added a comment -

            Finally the heap dump shown me the problem. Late I had added global new environment variable for Oracle libraries which caused the problem by "referring" to it self again and again. Multijob plugin "job" failed to handle situation while others worked fine.

            Show
            heikkisi Heikki Simperi added a comment - Finally the heap dump shown me the problem. Late I had added global new environment variable for Oracle libraries which caused the problem by "referring" to it self again and again. Multijob plugin "job" failed to handle situation while others worked fine.
            Hide
            yorammi Yoram Michaeli added a comment -

            Closing issue as part of tikal-multijob-plugin issues cleanup.
            If still relevant, please open a matching issue in https://github.com/jenkinsci/tikal-multijob-plugin/issues (you can refer to this issue in its description)

            Show
            yorammi Yoram Michaeli added a comment - Closing issue as part of tikal-multijob-plugin issues cleanup. If still relevant, please open a matching issue in https://github.com/jenkinsci/tikal-multijob-plugin/issues (you can refer to this issue in its description)

              People

              • Assignee:
                Unassigned
                Reporter:
                heikkisi Heikki Simperi
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: