Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38211

JenkinsFiles cannot run on Linux agent of Windows master

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Component/s: pipeline
    • Environment:
    • Similar Issues:

      Description

      In a multi-branch pipeline using a JenkinsFile in the SCM it does not appear to be possible to execute 'sh' commands on a Linux node/agent. As per the example below, when using the 'sh' command in the JenkinsFile it causes a 'bad interpreter' error.

      JenkinsFile Snippit:

      node('linux-agent-1') {
          String jdktool = tool name: "1.7", type: 'hudson.model.JDK'
          withEnv(["PATH+MVN=${tool 'Maven 2'}/bin", "PATH+JDK=${jdktool}/bin", "JAVA_HOME=${jdktool}"]) {
              stage 'Checkout'
              checkout scm
      
              stage 'Build'
              try {
                  sh "mvn test"
              } catch (Exception e) {
                  //Notification here!
              }
      }
      

      Error from console log:

      [Pipeline] node
      Running on linux-agent-1 in /opt/jenkins/workspace/ProjectName - TestMultiBranchPipeline/deveoper-branch-1
      [Pipeline] {
      [Pipeline] stage (Build)
      Entering stage Build
      Proceeding
      [Pipeline] sh
      [developer-branch-1] Running shell script
      sh: /opt/jenkins/workspace/ProjectName - TestMultiBranchPipeline/developer-branch-1@tmp/durable-6f9c6a38/script.sh: C:\Program: bad interpreter: No such file or directory
      [Pipeline] }
      

        Attachments

          Issue Links

            Activity

            Hide
            bobthemagicman Justen Britain added a comment - - edited

            Some additional information that I didn't feel was appropriate for the description as it is a bit speculative. Looking into the "bad interpreter: No such file or directory" output, it seems that most of the time this is related to windows style EOL characters in the script that is attempting to run. I verified that the JenkinsFile did not have the windows specific ^M line endings, to no avail. My hunch as to what is happening is that the command, "sh mvn test" is getting converted to a script, due to this line of output "sh: /opt/jenkins/workspace/ProjectName - TestMultiBranchPipeline/developer-branch-1@tmp/durable-6f9c6a38/script.sh:" and that script is either:

            1. Getting created on the windows machine and copied to the node causing it to have windows style line endings
            2. Getting created by the windows master on the node causing the windows line endings

            Again this is speculative but seemed worth adding to the comments.

            Show
            bobthemagicman Justen Britain added a comment - - edited Some additional information that I didn't feel was appropriate for the description as it is a bit speculative. Looking into the "bad interpreter: No such file or directory" output, it seems that most of the time this is related to windows style EOL characters in the script that is attempting to run. I verified that the JenkinsFile did not have the windows specific ^M line endings, to no avail. My hunch as to what is happening is that the command, "sh mvn test" is getting converted to a script, due to this line of output "sh: /opt/jenkins/workspace/ProjectName - TestMultiBranchPipeline/developer-branch-1@tmp/durable-6f9c6a38/script.sh:" and that script is either: 1. Getting created on the windows machine and copied to the node causing it to have windows style line endings 2. Getting created by the windows master on the node causing the windows line endings Again this is speculative but seemed worth adding to the comments.
            Hide
            mrichar2 Mark R added a comment -

            Any progress on this? This is a showstopper for us as we are unable to run linux shell scripts.

            Show
            mrichar2 Mark R added a comment - Any progress on this? This is a showstopper for us as we are unable to run linux shell scripts.
            Hide
            mrichar2 Mark R added a comment - - edited

            A coworker was able to grab the temporary script being run on the Linux agent. The line endings are fine (LF-only):

            [agent].../durable-3ca81cbf:
            $ od -cxv script.sh
            0000000   #   !   s   h       -   x   e  \n   e   n   v
                       2123    6873    2d20    6578    650a    766e
            0000014
            

            But he still sees the same error:

            [agent].../durable-3ca81cbf:
            $ ./script.sh
            bash: ./script.sh: sh: bad interpreter: No such file or directory
            

            It looks to be a malformed shebang. Modifying the shebang to be an absolute path allowed the script to run successfully.

            [agent].../durable-3ca81cbf:
            $ od -cxv mod.sh
            0000000   #   !   /   b   i   n   /   s   h       -   x   e  \n   e   n
                       2123    622f    6e69    732f    2068    782d    0a65    6e65
            0000020   v
                       0076
            0000021
            

            I'm now looking into if that is configurable.

            Show
            mrichar2 Mark R added a comment - - edited A coworker was able to grab the temporary script being run on the Linux agent. The line endings are fine (LF-only): [agent].../durable-3ca81cbf: $ od -cxv script.sh 0000000 # ! s h - x e \n e n v 2123 6873 2d20 6578 650a 766e 0000014 But he still sees the same error: [agent].../durable-3ca81cbf: $ ./script.sh bash: ./script.sh: sh: bad interpreter: No such file or directory It looks to be a malformed shebang. Modifying the shebang to be an absolute path allowed the script to run successfully. [agent].../durable-3ca81cbf: $ od -cxv mod.sh 0000000 # ! / b i n / s h - x e \n e n 2123 622f 6e69 732f 2068 782d 0a65 6e65 0000020 v 0076 0000021 I'm now looking into if that is configurable.
            Hide
            mrichar2 Mark R added a comment -

            It comes from this:

            https://github.com/jenkinsci/durable-task-plugin/blob/master/src/main/java/org/jenkinsci/plugins/durabletask/BourneShellScript.java

            String s = script;
                    final Jenkins jenkins = Jenkins.getInstance();
                    if (!s.startsWith("#!") && jenkins != null) {
                        String defaultShell = jenkins.getInjector().getInstance(Shell.DescriptorImpl.class).getShellOrDefault(ws.getChannel());
                        s = "#!"+defaultShell+" -xe\n" + s;
                    }
                    shf.write(s, "UTF-8");
                    shf.chmod(0755);
            

            The default shell for the node must be 'sh'. It should be configurable somewhere... I'll ask our build guys if that can be done. For some reason it doesn't like '#!sh'.

            In the mean time I am able to get it to work by manually setting the shell:

            sh '#!/usr/bin/sh -xe\n env'
            
            Show
            mrichar2 Mark R added a comment - It comes from this: https://github.com/jenkinsci/durable-task-plugin/blob/master/src/main/java/org/jenkinsci/plugins/durabletask/BourneShellScript.java String s = script; final Jenkins jenkins = Jenkins.getInstance(); if (!s.startsWith( "#!" ) && jenkins != null ) { String defaultShell = jenkins.getInjector().getInstance(Shell.DescriptorImpl.class).getShellOrDefault(ws.getChannel()); s = "#!" +defaultShell+ " -xe\n" + s; } shf.write(s, "UTF-8" ); shf.chmod(0755); The default shell for the node must be 'sh'. It should be configurable somewhere... I'll ask our build guys if that can be done. For some reason it doesn't like '#!sh'. In the mean time I am able to get it to work by manually setting the shell: sh '#!/usr/bin/sh -xe\n env'
            Hide
            bitwiseman Liam Newman added a comment -

            I discussed this with Justen Britain at CD Summit yesterday.

            This is at least partly being caused by a strange configuration on that specific Windows-based Jenkins Master. The "Shell Executable" has been manually set to "C:\Program Files (x86)\Git\git-bash.exe". This means that the sh step on all nodes (master and agents) will add #! C:\Program Files (x86)\Git\git-bash.exe -xe to the front of their shell scripts. On windows, I believe this gets ignored, but on linux it blows up.

            If we treat this as a feature request it would be that there should be a "Shell Executable" configuration available on each node agent that inherits from global configuration by default. Or treating "shell" as a tool that one can configure (like ant, maven, JDK, etc).

            But I'd like let's talk about work-arounds that Justen Britain can use (aside from switching to a linux master):
            1. Set the "Shell executable" configuration value to empty. That is what is by default, and generally is correct. Then make sure you have Windows Git installed, and add C:\Program Files\Git\bin to the path on all windows machines that run Jenkins agents (including master).
            (based on http://stackoverflow.com/a/35045651/1637252)

            2. If you cannot change the shell executable setting, you can get around this by manually adding #! to the start of all your sh steps, but this really is a distant second.

            The third point is that this and other difficulties and edge cases around running jenkins (either master or agents) need to be documented. I'll get a page started on jenkins.io and discuss this.

            Show
            bitwiseman Liam Newman added a comment - I discussed this with Justen Britain at CD Summit yesterday. This is at least partly being caused by a strange configuration on that specific Windows-based Jenkins Master. The "Shell Executable" has been manually set to "C:\Program Files (x86)\Git\git-bash.exe". This means that the sh step on all nodes (master and agents) will add #! C:\Program Files (x86)\Git\git-bash.exe -xe to the front of their shell scripts. On windows, I believe this gets ignored, but on linux it blows up. If we treat this as a feature request it would be that there should be a "Shell Executable" configuration available on each node agent that inherits from global configuration by default. Or treating "shell" as a tool that one can configure (like ant, maven, JDK, etc). But I'd like let's talk about work-arounds that Justen Britain can use (aside from switching to a linux master): 1. Set the "Shell executable" configuration value to empty. That is what is by default, and generally is correct. Then make sure you have Windows Git installed, and add C:\Program Files\Git\bin to the path on all windows machines that run Jenkins agents (including master). (based on http://stackoverflow.com/a/35045651/1637252 ) 2. If you cannot change the shell executable setting, you can get around this by manually adding #! to the start of all your sh steps, but this really is a distant second. The third point is that this and other difficulties and edge cases around running jenkins (either master or agents) need to be documented. I'll get a page started on jenkins.io and discuss this.
            Show
            bitwiseman Liam Newman added a comment - - edited This is where the default gets provided: https://github.com/jenkinsci/jenkins/blob/720b75f096e573b102323b529ec952e2106d4a57/core/src/main/java/hudson/tasks/Shell.java#L155
            Hide
            abayer Andrew Bayer added a comment -

            Is this still relevant?

            Show
            abayer Andrew Bayer added a comment - Is this still relevant?

              People

              • Assignee:
                jglick Jesse Glick
                Reporter:
                bobthemagicman Justen Britain
              • Votes:
                2 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: