Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55220

Multibranch Pipeline jobs get randomly lost

    Details

    • Similar Issues:

      Description

      We are running different Multibranch Pipelines with git (Bitbucket) since almost 2 years now.

      Unfortunately jobs (aka branches) do disappear randomly. It feels like the the number of losses is increasing and our workaround (see first comment) is too complicated to keep up.

      Even worse it looks like develop and master are the most likely to disappear.

      We are not sure what causes the losses. It feels pretty randomly.

      The Scan Multibranch Pipeline Log says Detected unsupported subitem:

      [Mon Dec 17 10:28:18 CET 2018] Starting branch indexing...

      Connecting to https://bitbucket.org using *****

      Repository type: Git

      Looking up sj/sj-mobile-ios for pull requests

       

        0 pull requests were processed

      Connecting to https://bitbucket.org using ******

      Repository type: Git

      Looking up sj/sj-mobile-ios for branches

      Checking branch develop from sj/sj-mobile-ios

            'Jenkinsfile' found

          Met criteria

      Detected unsupported subitem sj-mobile-ios » develop, skipping

      Checking branch master from sj/sj-mobile-ios

            'Jenkinsfile' found

          Met criteria

      Detected unsupported subitem sj-mobile-ios » master, skipping

      Checking branch stash/iapPromotion from sj/sj-mobile-ios

            'Jenkinsfile' not found

          Skipped

       

        3 branches were processed

      Looking up sj/sj-mobile-ios for pull requests

       

        0 pull requests were processed

      [Mon Dec 17 10:28:21 CET 2018] Finished branch indexing. Indexing took 3.4 sec

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@65a9d08a[sj-ios/ffffeeeea.j2i0s0od1eao.innnntttt], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@4f2f2fc0[sj-ios/rrrreeeel.vled0mqc8i7t.2----3333], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@72d388bb[sj-ios/rrrreeeel.r1m7pv7ujr6o.5----1111], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@29a3274d[sj-ios/rrrreeeel.tgai4liiqai5.3----1111], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@63eb56a6[sj-ios/ffffeeeea.fqa397lnq84t.aggggyyyy], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@2548a24e[sj-ios/rrrreeeel.0d0a3oaeksil.5----2222], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@52e05031[sj-ios/PR-130], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@69b325d6[sj-ios/rrrreeeel.2s6jniopvdin.1----7777], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@1ffba7d3[sj-ios/rrrreeeel.c8v76qkpj7f5.1----4444], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@1c6e3ff6[sj-ios/rrrreeeel.tndhtl2k8ai1.1----6666], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@734db299[sj-ios/rrrreeeel.9dgfoipjl376.2----2222], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@f3b27ec[sj-ios/PR-129], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@22f40536[sj-ios/rrrreeeel.i2j4g40joneo.1----5555], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@2928d02b[sj-ios/ffffeeeea.i5k5i2srbpsp.nkkkkssss], skipping

      Detected unsupported subitem org.jenkinsci.plugins.workflow.job.WorkflowJob@5795381d[sj-ios/rrrreeeel.akouhq90ivkl.1----3333], skipping

      Finished: SUCCESS

       

      You can still see the job if you navigate to /project/job/develop/ but its not working anymore. Somehow the config.xml gets reseted to

      <?xml version='1.1' encoding='UTF-8'?>
      <flow-definition plugin="workflow-job@2.30">
      <actions/>
      <keepDependencies>false</keepDependencies>
      <properties/>
      <triggers/>
      <disabled>false</disabled>
      </flow-definition>

       

      We are thankful for your help.

       

       

      We are running Jenkins on AWS Bitnami Installation on Apache Tomcat Version 8.0.47

      See also screenshots of Project Configuration and Bitbuckets Webhooks.

      OS: 

      os.name Linux
      os.version 4.9.0-5-amd64

      VM:

      java.vendor Oracle Corporation
      java.vendor.url http://java.oracle.com/
      java.vendor.url.bug http://bugreport.sun.com/bugreport/
      java.version 1.8.0_151
      java.vm.info mixed mode
      java.vm.name Java HotSpot(TM) 64-Bit Server VM
      java.vm.specification.name Java Virtual Machine Specification
      java.vm.specification.vendor Oracle Corporation
      java.vm.specification.version 1.8
      java.vm.vendor Oracle Corporation
      java.vm.version 25.151-b12

       

      Installed Plugins:

      ace-editor 1.1 true
      amazon-ecr 1.6 true
      amazon-ecs 1.18 true
      analysis-core 1.95 true
      ansicolor 0.5.3 true
      ant 1.9 true
      antisamy-markup-formatter 1.5 true
      apache-httpcomponents-client-4-api 4.5.5-3.0 true
      authentication-tokens 1.3 true
      aws-credentials 1.24 true
      aws-java-sdk 1.11.457 true
      bitbucket 1.1.8 true
      bitbucket-build-status-notifier 1.4.1 true
      bitbucket-scm-trait-commit-skip 0.1.1 true
      bouncycastle-api 2.17 true
      branch-api 2.1.2 true
      build-monitor-plugin 1.12+build.201809061734 true
      build-timeout 1.19 true
      checkstyle 3.50 true
      cloudbees-bitbucket-branch-source 2.2.15 true
      cloudbees-folder 6.7 true
      cobertura 1.13 true
      code-coverage-api 1.0.5 true
      command-launcher 1.2 true
      copyartifact 1.41 true
      credentials 2.1.18 true
      credentials-binding 1.17 true
      display-url-api 2.3.0 true
      docker-build-publish 1.3.2 true
      docker-commons 1.13 true
      docker-workflow 1.17 true
      dry 2.50 true
      durable-task 1.28 true
      email-ext 2.63 true
      external-monitor-job 1.7 true
      git 3.9.1 true
      git-client 2.7.4 true
      git-parameter 0.9.6 true
      git-server 1.7 true
      github 1.29.3 true
      github-api 1.95 true
      github-branch-source 2.4.1 true
      github-organization-folder 1.6 true
      gradle 1.29 true
      handlebars 1.1.1 true
      handy-uri-templates-2-api 2.1.6-1.0 true
      icon-shim 2.0.3 true
      jackson2-api 2.9.7.1 true
      javadoc 1.4 true
      jdk-tool 1.2 true
      jquery 1.12.4-0 true
      jquery-detached 1.2.1 true
      jsch 0.1.54.2 true
      junit 1.26.1 true
      ldap 1.20 true
      locale 1.4 true
      lockable-resources 2.3 true
      log-parser 2.0 true
      mailer 1.22 true
      mapdb-api 1.0.9.0 true
      matrix-auth 2.3 true
      matrix-project 1.13 true
      maven-plugin 3.2 true
      mercurial 2.4 true
      momentjs 1.1.1 true
      multiple-scms 0.6 true
      notification 1.13 true
      pam-auth 1.4 true
      pipeline-aws 1.35 true
      pipeline-build-step 2.7 true
      pipeline-github-lib 1.0 true
      pipeline-graph-analysis 1.9 true
      pipeline-input-step 2.8 true
      pipeline-milestone-step 1.3.1 true
      pipeline-model-api 1.3.3 true
      pipeline-model-declarative-agent 1.1.1 true
      pipeline-model-definition 1.3.3 true
      pipeline-model-extensions 1.3.3 true
      pipeline-rest-api 2.10 true
      pipeline-stage-step 2.3 true
      pipeline-stage-tags-metadata 1.3.3 true
      pipeline-stage-view 2.10 true
      pipeline-utility-steps 2.2.0 true
      plain-credentials 1.4 true
      pmd 3.50 true
      pyenv-pipeline 2.0.1 true
      resource-disposer 0.12 true
      s3 0.11.2 true
      scm-api 2.3.0 true
      script-security 1.49 true
      sloccount 1.24 true
      ssh-agent 1.17 true
      ssh-credentials 1.14 true
      ssh-slaves 1.29.1 true
      structs 1.17 true
      subversion 2.12.1 true
      timestamper 1.8.10 true
      token-macro 2.5 true
      trilead-api 1.0.1 true
      warnings 4.68 true
      windows-slaves 1.3.1 true
      workflow-aggregator 2.6 true
      workflow-api 2.33 true
      workflow-basic-steps 2.13 true
      workflow-cps 2.61 true
      workflow-cps-global-lib 2.12 true
      workflow-durable-task-step 2.26 true
      workflow-job 2.30 true
      workflow-multibranch 2.20 true
      workflow-scm-step 2.7 true
      workflow-step-api 2.17 true
      workflow-support 2.23 true
      ws-cleanup 0.36 true

        Attachments

          Activity

          Hide
          esmaras Eric Maras added a comment -

          I am experiencing issues as well. I have been able to reproduce this. Every time I change a multibranch config file all of my branch pipelines seem to disappear. This was not happening every time before but now it is consistently happening. Deleting a lot of branches in Bitbucket seems to cause this issue for me as well. 

          Show
          esmaras Eric Maras added a comment - I am experiencing issues as well. I have been able to reproduce this. Every time I change a multibranch config file all of my branch pipelines seem to disappear. This was not happening every time before but now it is consistently happening. Deleting a lot of branches in Bitbucket seems to cause this issue for me as well. 
          Hide
          esmaras Eric Maras added a comment -

          I think I may have found the issue. I am not sure why my browser auto-fill thought the "Marker File" field is a username field, but it has been putting my username in that field every time I edited a configuration file. I am thinking this may be the case for others as well and could be an issue. How this was causing all branch pipelines to disappear for me I still do not understand. This issue could be different from the one this ticket is meant to address. Apologies if that is the case.

          Show
          esmaras Eric Maras added a comment - I think I may have found the issue. I am not sure why my browser auto-fill thought the "Marker File" field is a username field, but it has been putting my username in that field every time I edited a configuration file. I am thinking this may be the case for others as well and could be an issue. How this was causing all branch pipelines to disappear for me I still do not understand. This issue could be different from the one this ticket is meant to address. Apologies if that is the case.
          Hide
          matthieubrucher Matthieu Brucher added a comment -

          I see this happening with github as well, so not just related to bitbucket.

          Could be indeed related to unauthorized user and then the branches get cleaned.

          Show
          matthieubrucher Matthieu Brucher added a comment - I see this happening with github as well, so not just related to bitbucket. Could be indeed related to unauthorized user and then the branches get cleaned.
          Hide
          cornr Jonas Büth added a comment -

          I think we might found an answer (or problem causer). In our case we automatically restarted Jenkins once a day in either in the shut down or boot up process branches went missing.
          The amount of missing branches declined significantly after keeping the Jenkins running all the time.

          Show
          cornr Jonas Büth added a comment - I think we might found an answer (or problem causer). In our case we automatically restarted Jenkins once a day in either in the shut down or boot up process branches went missing. The amount of missing branches declined significantly after keeping the Jenkins running all the time.
          Hide
          ncosta Nuno Costa added a comment - - edited

          This is happening to me today, although is not showing any errors. The Scan log shows that is not detecting jenkinsfile on a specific branch but the file is there. Other branches with jenkinsfile are been shown in the log.

          I had this happening a few weeks ago and without any change from our part, it reenabled the branch.

          We are using bitbucket.org Team.

          Using jenkins 2.164.2 and nginx, both as containers.

          Show
          ncosta Nuno Costa added a comment - - edited This is happening to me today, although is not showing any errors. The Scan log shows that is not detecting jenkinsfile on a specific branch but the file is there. Other branches with jenkinsfile are been shown in the log. I had this happening a few weeks ago and without any change from our part, it reenabled the branch. We are using bitbucket.org Team. Using jenkins 2.164.2 and nginx, both as containers.

            People

            • Assignee:
              Unassigned
              Reporter:
              cornr Jonas Büth
            • Votes:
              5 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated: