Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58725

Ansible-Tower Plugin: Ansible Tower will hang when Jenkins job is cancelled on the web portal. And now the latter jobs will all be pending.

    Details

    • Similar Issues:
    • Released As:
      0.11.0

      Description

        Attachments

          Activity

          Hide
          johnwestcottiv John Westcott added a comment -

          You are seeing jobs in Tower which were launched by Jenkins stuck in a pending state?
          Can you add more details to this? i.e. what version of Tower are you on. What version of the plugin are you using? Also, can you manually invoke a job in Tower and get it past the pending state?

          The pending state in Tower means that the job was requested but its waiting to be scheduled (actually run) within the Tower cluster.
          Once the jobs is scheduled in Jenkins, nothing else is done on the Jenkins side other than monitor the job.

          Jobs getting stuck in a pending state would indicate (to me) that there is some kind of issue with your Tower installation.

          Show
          johnwestcottiv John Westcott added a comment - You are seeing jobs in Tower which were launched by Jenkins stuck in a pending state? Can you add more details to this? i.e. what version of Tower are you on. What version of the plugin are you using? Also, can you manually invoke a job in Tower and get it past the pending state? The pending state in Tower means that the job was requested but its waiting to be scheduled (actually run) within the Tower cluster. Once the jobs is scheduled in Jenkins, nothing else is done on the Jenkins side other than monitor the job. Jobs getting stuck in a pending state would indicate (to me) that there is some kind of issue with your Tower installation.
          Hide
          fen_zhou fen zhou added a comment -

          4.0.0 AWX.

          0.9.1 Ansible Tower Plugin.

          When I invoke Jenkins Job which has already called a Tower job, the Jenkins job gets stopped with Tower job still running. But actually, if the Tower job is stucked in one step like running a long-term progress, it will still remain running, which makes it impossible for me to call another Tower job in the same Tower project.

          I think one problem is I can't parallel the Tower jobs which is a normal feature of Tower (no way to change this). On the other hand, Ansible Tower Plugin doesn't notify Tower when it stops. These two features make me feel uncomfortable, and the page shows all the latter jobs wait to be scheduled. Then it will be one-century waiting without any feedback.

          Now, I can only stop the world with manually invoking on the Tower. But I hope there is any way to stop the Tower job without handly stopping.

           

          Show
          fen_zhou fen zhou added a comment - 4.0.0 AWX. 0.9.1 Ansible Tower Plugin. When I invoke Jenkins Job which has already called a Tower job, the Jenkins job gets stopped with Tower job still running. But actually, if the Tower job is stucked in one step like running a long-term progress, it will still remain running, which makes it impossible for me to call another Tower job in the same Tower project. I think one problem is I can't parallel the Tower jobs which is a normal feature of Tower (no way to change this). On the other hand, Ansible Tower Plugin doesn't notify Tower when it stops. These two features make me feel uncomfortable, and the page shows all the latter jobs wait to be scheduled. Then it will be one-century waiting without any feedback. Now, I can only stop the world with manually invoking on the Tower. But I hope there is any way to stop the Tower job without handly stopping.  
          Hide
          fen_zhou fen zhou added a comment -

          Is there any comment about this?

          Show
          fen_zhou fen zhou added a comment - Is there any comment about this?
          Hide
          johnwestcottiv John Westcott added a comment -

          I am either unable to replicate this problem or I am not understanding what is going on exactly in your case. When Jenkins executes a job on Tower is calling the Tower API and Tower will run the job asynchronously from Jenkins but Jenkins will wait for that job to complete before Jenkins moves on to the next step. If you are running two Jenkins jobs concurrently and they both call the same Tower instance as long as the Tower job is configured to run the job concurrently both Jenkins job should run in parallel. Is that what you are seeing as not working? If so can you confirm that your Tower job is correctly set for concurrent execution?

          Also, if you stop the Jenkins job any further jobs it would have run in Tower will not be run but the currently active Tower job will continue to run until Tower is complete. Are you looking for a feature where a cancelled job in Jenkins would attempt to cancel the Tower job? if so, I can look into the possibility of a feature like this.

          Also, for async jobs between Tower and Jenkins (i.e. having Jenkins not wait for a job to complete in Tower) this can now be done in the latest version of the plugin so long as your Jenkins job is a pipeline and not a freestyle job. See the "Async Execution" section on https://wiki.jenkins.io/display/JENKINS/Ansible+Tower+Plugin and let me know if that helps this situation at all.

          If none of this helps can you please try to describe further what is currently happening and what you expect to happen?

          Show
          johnwestcottiv John Westcott added a comment - I am either unable to replicate this problem or I am not understanding what is going on exactly in your case. When Jenkins executes a job on Tower is calling the Tower API and Tower will run the job asynchronously from Jenkins but Jenkins will wait for that job to complete before Jenkins moves on to the next step. If you are running two Jenkins jobs concurrently and they both call the same Tower instance as long as the Tower job is configured to run the job concurrently both Jenkins job should run in parallel. Is that what you are seeing as not working? If so can you confirm that your Tower job is correctly set for concurrent execution? Also, if you stop the Jenkins job any further jobs it would have run in Tower will not be run but the currently active Tower job will continue to run until Tower is complete. Are you looking for a feature where a cancelled job in Jenkins would attempt to cancel the Tower job? if so, I can look into the possibility of a feature like this. Also, for async jobs between Tower and Jenkins (i.e. having Jenkins not wait for a job to complete in Tower) this can now be done in the latest version of the plugin so long as your Jenkins job is a pipeline and not a freestyle job. See the "Async Execution" section on  https://wiki.jenkins.io/display/JENKINS/Ansible+Tower+Plugin  and let me know if that helps this situation at all. If none of this helps can you please try to describe further what is currently happening and what you expect to happen?
          Hide
          fen_zhou fen zhou added a comment - - edited

           

          1. Tower job is correctly set for concurrent execution.

          We use Tower job for real case in CD, so we want to keep it running one by one instead of parallelism, so we will not change the settings. Anyway, thanks for ur reminder.

          2.  A cancelled job in Jenkins attempt to cancel the Tower job.

          This is exactly what I want. It's more natural to cancel the Tower job when Jenkins job is cancelled.

          3. Tower will run the job asynchronously.

          I checked this awesome new feature, and thanks for ur help. But I think it can't help with the situation directly.

           

          Thanks for your support

          Show
          fen_zhou fen zhou added a comment - - edited   1. Tower job is correctly set for concurrent execution. We use Tower job for real case in CD, so we want to keep it running one by one instead of parallelism, so we will not change the settings. Anyway, thanks for ur reminder. 2.   A cancelled job in Jenkins attempt to cancel the Tower job. This is exactly what I want. It's more natural to cancel the Tower job when Jenkins job is cancelled. 3. Tower will run the job asynchronously. I checked this awesome new feature, and thanks for ur help. But I think it can't help with the situation directly.   Thanks for your support
          Hide
          fen_zhou fen zhou added a comment -

          BTW, I have just tested the parallelism of the same Tower job template, Tower jobs of the same Tower job template will run one by one when I click the launch button twice in a short time. But never mind, it's accessible in my situation.

          Show
          fen_zhou fen zhou added a comment - BTW, I have just tested the parallelism of the same Tower job template , Tower jobs of the same Tower job template will run one by one when I click the launch button twice in a short time. But never mind, it's accessible in my situation.
          Hide
          johnwestcottiv John Westcott added a comment -

          Please try v0.11.0 of the plugin. It will now attempt to cancel a job running in Tower if the Jenkins job is canceled.

          Show
          johnwestcottiv John Westcott added a comment - Please try v0.11.0 of the plugin. It will now attempt to cancel a job running in Tower if the Jenkins job is canceled.

            People

            • Assignee:
              fen_zhou fen zhou
              Reporter:
              fen_zhou fen zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: