Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50902

shell pipeline step cannot use relative path to shell

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      durable-task 1.28

      Description

      There is some difference between free-style project shell build step and the pipeline `sh` step. If I set in jenkins global configuration `bash` as the "Shell Executable", then free-style project works and calls `bash` from the PATH on the slave. The pipeline `sh` step fails like this though:

      [pipeline-test-job] Running shell script
      sh: /home/jenkins/workspace/pipeline-test-akostadi@tmp/durable-2ff6e6ae/script.sh: bash: bad interpreter: No such file or directory
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      ERROR: script returned exit code 126
      Finished: FAILURE
      

      The use case is that some systems like MacOS ship with an old bash version under /bin/sh. And we want to not be limited in shell scripts by that version. Since proper bash should be installed in other locations on such slaves, we need to run `bash` from PATH. But also can't hardcode `bash` absolute path because it is still `/bin/bash` on normal linux slaves.

      I'm reading through ShellStep.java and BourneShellScript.java.

      It looks like it launches `sh -c cmd` but I don't see how the global setting is applied, even less I understand why it fails. At the same time free-style projects work fine with this setting on all slaves.

      I think this difference between free-style and pipeline might be related to JENKINS-44341 also.

      Any clue what exactly might be the difference and how it could be fixed?

        Attachments

          Issue Links

            Activity

            Hide
            olivergondza Oliver Gondža added a comment -

            The problem here seems to be that BourneShellScript is using shebang approach to choose interpreter while ShellStep calls it directly. I have not found a reason why pipeline impl has chosen such approach in 2014 so I am attempting to unify this.

            A number of customers have run into situation of combining freestyle/matrix with pipeline jobs while utilizing windows/linux/unix hosts and we failed to identify a setting what would work here (unless one wants to compromise on subset of shell features). An alternative is to hardcode shebangs in all script invocations and use different absolute paths for windows.

            Show
            olivergondza Oliver Gondža added a comment - The problem here seems to be that BourneShellScript is using shebang approach to choose interpreter while ShellStep calls it directly. I have not found a reason why pipeline impl has chosen such approach in 2014 so I am attempting to unify this. A number of customers have run into situation of combining freestyle/matrix with pipeline jobs while utilizing windows/linux/unix hosts and we failed to identify a setting what would work here (unless one wants to compromise on subset of shell features). An alternative is to hardcode shebangs in all script invocations and use different absolute paths for windows.
            Hide
            olivergondza Oliver Gondža added a comment -

            Fix proposed

            Show
            olivergondza Oliver Gondža added a comment - Fix proposed
            Hide
            dnusbaum Devin Nusbaum added a comment -

            The fix for this was just released in version 1.28 of the Durable Task plugin. Thanks Oliver Gondža!

            Show
            dnusbaum Devin Nusbaum added a comment - The fix for this was just released in version 1.28 of the Durable Task plugin. Thanks  Oliver Gondža !
            Hide
            aluvian Amir O. Khosrowshahi added a comment -

            I am getting the same issue after updating durable-task to 1.28 and restart Jenkins.

            We have a shell script which takes a couple of arguments (the environment, the browser, test type, tag, etc.). The command is as follows:

            sh /root/runtest.sh $DEVICE $ENVIRONMENT $TYPE $IS_GROUP $PRODUCT $TAG
            

            This invokes codeception tool to trigger the following command to run :

             

            /root/test/vendor/bin/codecept run acceptance $PRODUCT\\${TAG}Cest.php --steps --html='index.html'
            

            This works fine when we use a Freestyle Project, but when I started to add the same step to my Pipeline, it cannot identify the arguments, especially the relative path used for testing.

            Let me know if any more info is needed.

             

            Show
            aluvian Amir O. Khosrowshahi added a comment - I am getting the same issue after updating durable-task to 1.28 and restart Jenkins. We have a shell script which takes a couple of arguments (the environment, the browser, test type, tag, etc.). The command is as follows: sh /root/runtest.sh $DEVICE $ENVIRONMENT $TYPE $IS_GROUP $PRODUCT $TAG This invokes codeception tool to trigger the following command to run :   /root/test/vendor/bin/codecept run acceptance $PRODUCT\\${TAG}Cest.php --steps --html= 'index.html' This works fine when we use a Freestyle Project, but when I started to add the same step to my Pipeline, it cannot identify the arguments, especially the relative path used for testing. Let me know if any more info is needed.  
            Hide
            olivergondza Oliver Gondža added a comment - - edited

            Amir O. Khosrowshahi, your issue appears unrelated. This was never intended to change the argument resolution (that is supposed to be relative to pwd of the pipeline) but the hardwired interpreter resolution.

            Show
            olivergondza Oliver Gondža added a comment - - edited Amir O. Khosrowshahi , your issue appears unrelated. This was never intended to change the argument resolution (that is supposed to be relative to pwd of the pipeline) but the hardwired interpreter resolution.

              People

              • Assignee:
                olivergondza Oliver Gondža
                Reporter:
                akostadinov akostadinov
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: