Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-16176

ShiningPanda PATH update uses wrong separator on Windows

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: shiningpanda-plugin
    • Labels:
      None
    • Environment:
      Linux master, Windows XP slave with Python 2.7
    • Similar Issues:

      Description

      When running ShiningPanda on a Windows slave, the plugin (or virtualenv?) updates the PATH to include the virtualenv directory.

      0. Install ShiningPanda plugin (this result is for v1.15)
      1. Create a build with a Virtualenv build step
      2. Use the Shell or XShell command with a minimal script:

      set 
      pip.exe install -r pkg_requirements.txt
      

      The build fails because the pip command can not be found. The output shows the Path environment variable uses the : (colon) separator for the virtualenv Scripts directory. On Windows, the PATH separator should be ; (semicolon).

      [1-SMSDK-apps-venv] $ cmd.exe /c call C:\DOCUME~1\builder\LOCALS~1\Temp\shiningpanda4030889806584404855.bat
      
      c:\jenkins-dev\workspace\1-SMSDK-apps-venv>set
      ... [edited for relevance] ...
      Path=c:\jenkins-dev\workspace\1-SMSDK-apps-venv:c:\jenkins-dev\shiningpanda\deb23754\virtualenvs\d41d8cd9\Scripts:C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\FLEXlm;C:\Program Files\CollabNet\Subversion Client;c:\cygwin\bin;c:\cygwin\usr\bin;C:\Program Files\Java\jre6\bin
      PYTHON_EXE=c:\jenkins-dev\shiningpanda\deb23754\virtualenvs\d41d8cd9\Scripts\python.exe
      VIRTUAL_ENV=c:\jenkins-dev\shiningpanda\deb23754\virtualenvs\d41d8cd9
      WORKSPACE=c:\jenkins-dev\workspace\1-SMSDK-apps-venv
      
      c:\jenkins-dev\workspace\1-SMSDK-apps-venv>pip.exe install -r pkg_requirements.pip 
      'pip.exe' is not recognized as an internal or external command,
      operable program or batch file.
      

        Attachments

          Issue Links

            Activity

            Hide
            haney David Haney added a comment -

            For both of the workarounds involving for, it appears that these only work if the build environment is Shell. In the XShell environment, the for /f will be replaced by for \f, resulting in the for call failing. It appears that the plugin is interpreting the /f as a path and converting it to use Windows separators.

            Show
            haney David Haney added a comment - For both of the workarounds involving for , it appears that these only work if the build environment is Shell . In the XShell environment, the for /f will be replaced by for \f , resulting in the for call failing. It appears that the plugin is interpreting the /f as a path and converting it to use Windows separators.
            Hide
            cbennett Colin Bennett added a comment -

            Weird – I am seeing an incorrect path in the converse way – there are semicolons getting inserted into the Linux slave's PATH. I have a Windows master and Linux slave. The PATH variable is set like this

            /usr/lib/jvm/java-7-openjdk-amd64/bin:/var/jenkins/workspace/testproject;/var/jenkins/shiningpanda/jobs/ef9d92b6/virtualenvs/4a2c09ff/bin;/var/jenkins/bin:/var/jenkins/bin/cmake-3.5.0-rc2-Linux-x86_64/bin:/usr/local/bin:/usr/bin:/bin

            Show
            cbennett Colin Bennett added a comment - Weird – I am seeing an incorrect path in the converse way – there are semicolons getting inserted into the Linux slave's PATH. I have a Windows master and Linux slave. The PATH variable is set like this /usr/lib/jvm/java-7-openjdk-amd64/bin:/var/jenkins/workspace/testproject;/var/jenkins/shiningpanda/jobs/ef9d92b6/virtualenvs/4a2c09ff/bin;/var/jenkins/bin:/var/jenkins/bin/cmake-3.5.0-rc2-Linux-x86_64/bin:/usr/local/bin:/usr/bin:/bin
            Hide
            cbennett Colin Bennett added a comment -

            Look at this very similar bug JENKINS-14515, especially this code change that uses toComputer(): 8b09f8bbff45...6f337351a745:

            -                EnvVars vars = new EnvVars();
            +                EnvVars vars = node.toComputer().getEnvironment();
            
            Show
            cbennett Colin Bennett added a comment - Look at this very similar bug JENKINS-14515 , especially this code change that uses toComputer() : 8b09f8bbff45...6f337351a745 : - EnvVars vars = new EnvVars(); + EnvVars vars = node.toComputer().getEnvironment();
            Hide
            cbennett Colin Bennett added a comment -

            Here's a possible way to use Python itself for a cross-platform PATH workaround hack. Call this script with the name of the command to run with a corrected PATH:

            python -c "import os; import sys; import subprocess; env=os.environ.copy(); badsep = {':': ';', ';': ':'}[os.pathsep]; env['PATH'] = env['PATH'].replace(badsep, os.pathsep); subprocess.call(sys.argv[1:], env=env)" python --version
            # Replace "python --version" with any other command to run
            

            This can be run on either Windows cmd.exe or a bash shell on Windows or Linux.

            Show
            cbennett Colin Bennett added a comment - Here's a possible way to use Python itself for a cross-platform PATH workaround hack. Call this script with the name of the command to run with a corrected PATH: python -c "import os; import sys; import subprocess; env=os.environ.copy(); badsep = {':': ';', ';': ':'}[os.pathsep]; env['PATH'] = env['PATH'].replace(badsep, os.pathsep); subprocess.call(sys.argv[1:], env=env)" python --version # Replace "python --version" with any other command to run This can be run on either Windows cmd.exe or a bash shell on Windows or Linux.
            Hide
            plondino Paul Londino added a comment - - edited

            I have encountered this bug as well. It's not trivial to troubleshoot unless you stumble upon this issue, and it's not an uncommon problem to want to test Python code across Windows and Linux, which necessitates a mismatch in the OS between the master and the slave.

            Show
            plondino Paul Londino added a comment - - edited I have encountered this bug as well. It's not trivial to troubleshoot unless you stumble upon this issue, and it's not an uncommon problem to want to test Python code across Windows and Linux, which necessitates a mismatch in the OS between the master and the slave.

              People

              • Assignee:
                Unassigned
                Reporter:
                dbacher Dave Bacher
              • Votes:
                5 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: