Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45571

"likely stuck" job is not actually stuck.

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I doubt this one is reproductible. 

      Going to yourjiraurlhere/computer/api/json?pretty=true&tree=computer[oneOffExecutors[likelyStuck,currentExecutable[result,url]]]{0}

      gives you the jobs currently running.

      One of my job is marked as "likely stuck", but his state is result is "SUCCESS" (and has been "SUCCESS" since 2h30, making me doubt about the veracity of the "likely stuck". 

      The job isn't running either. It's completed, but is still somehow showing as "likely stuck". 

        Attachments

          Issue Links

            Activity

            Hide
            atikhonova Anna Tikhonova added a comment - - edited

            Devin Nusbaum unfortunately, I don't. I've got a few 1000+ LOC pipelines running continuously. I do not know how to tell which one leaves executors and when.

            Pipeline build that has such "likelyStuck" executor looks completed on its build page (no progress bars, build status is set). But I still can see a matching OneOffExecutor on master:

                  "_class" : "hudson.model.Hudson$MasterComputer",
                  "oneOffExecutors" : [
                    {
                      "currentExecutable" : {
                        "_class" : "org.jenkinsci.plugins.workflow.job.WorkflowRun",
                        "building" : false,    // always false for these lost executors
                        "result" : "SUCCESS",    // always set to some valid build status != null
                        "url" : "JENKINS/job/PIPELINE/BUILD_NUMBER/"
                      },
                      "likelyStuck" : false    // can be true or false
                    }, ...
            
            Show
            atikhonova Anna Tikhonova added a comment - - edited Devin Nusbaum unfortunately, I don't. I've got a few 1000+ LOC pipelines running continuously. I do not know how to tell which one leaves executors and when. Pipeline build that has such "likelyStuck" executor looks completed on its build page (no progress bars, build status is set). But I still can see a matching OneOffExecutor on master: "_class" : "hudson.model.Hudson$MasterComputer" , "oneOffExecutors" : [ { "currentExecutable" : { "_class" : "org.jenkinsci.plugins.workflow.job.WorkflowRun" , "building" : false , // always false for these lost executors "result" : "SUCCESS" , // always set to some valid build status != null "url" : "JENKINS/job/PIPELINE/BUILD_NUMBER/" }, "likelyStuck" : false // can be true or false }, ...
            Hide
            dnusbaum Devin Nusbaum added a comment - - edited

            Anna Tikhonova Are you able upload the build directory of the build matching the stuck executor? Specifically, it would be helpful to see build.xml and the xml file(s) in the workflow directory. EDIT: I see now that you can't easily tell which are stuck and which are good. If you can find an executor with likelyStuck: true, and whose build looks like it has otherwise completed or is suck, that would be a great candidate.

            Another note: JENKINS-38381 will change the control flow here significantly.

            Show
            dnusbaum Devin Nusbaum added a comment - - edited Anna Tikhonova  Are you able upload the build directory of the build matching the stuck executor? Specifically, it would be helpful to see build.xml and the xml file(s) in the workflow directory. EDIT: I see now that you can't easily tell which are stuck and which are good. If you can find an executor with likelyStuck: true , and whose build looks like it has otherwise completed or is suck, that would be a great candidate. Another note:  JENKINS-38381 will change the control flow here significantly.
            Hide
            jglick Jesse Glick added a comment -

            gives you the jobs currently running

            This is not really an appropriate API query to use for that question. If your interest is limited to all Pipeline builds, FlowExecutionList is likely to be more useful. If you are looking at builds of a particular job (Pipeline or not), I think that information is available from the endpoint for that job.

            Show
            jglick Jesse Glick added a comment - gives you the jobs currently running This is not really an appropriate API query to use for that question. If your interest is limited to all Pipeline builds, FlowExecutionList is likely to be more useful. If you are looking at builds of a particular job (Pipeline or not), I think that information is available from the endpoint for that job.
            Hide
            jglick Jesse Glick added a comment -

            TCP prevents scheduling more builds because it considers those hanging executors.

            Offhand this sounds like a flaw in TCB. This PR introduced that behavior, purportedly to support the build-flow plugin (a conceptual predecessor of Pipeline née Workflow). If TCB intends to throttle builds per se (rather than work done by those builds—typically node blocks for Pipeline), then there are more direct ways of doing this than counting Executor slots.

            Show
            jglick Jesse Glick added a comment - TCP prevents scheduling more builds because it considers those hanging executors. Offhand this sounds like a flaw in TCB. This PR introduced that behavior, purportedly to support the build-flow plugin (a conceptual predecessor of Pipeline née Workflow). If TCB intends to throttle builds per se (rather than work done by those builds—typically node blocks for Pipeline), then there are more direct ways of doing this than counting Executor slots.
            Hide
            basil Basil Crow added a comment -

            Offhand this sounds like a flaw in TCB.

            I am attempting to fix this flaw in jenkinsci/throttle-concurrent-builds-plugin#57.

            Show
            basil Basil Crow added a comment - Offhand this sounds like a flaw in TCB. I am attempting to fix this flaw in jenkinsci/throttle-concurrent-builds-plugin#57 .

              People

              • Assignee:
                svanoort Sam Van Oort
                Reporter:
                zeal_iskander Stark Gabriel
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: