Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20078

email-ext plugin takes a huge amount of time to send out final status emails (30min)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: email-ext-plugin
    • Labels:
    • Environment:
      This issue is platform independent and happens for any kind of job which sends out final status emails.
    • Similar Issues:

      Description

      When you have a job configured to send out status emails for a job (like success, fail, still failing) the jobs will block the node for up to 15 minutes or even more, only to send out the email. Given that some of our jobs only run 2min this is an absolutely large overhead we have to get rid of soon. It's mainly blocking our internal QA team from signing off of new releases.

      Here an example of the last lines of a job which has debug mode enabled for email-ext:

      00:09:31.082 Archiving artifacts
      00:09:31.093 Recording test results
      00:09:31.218 Checking for post-build
      00:09:31.218 Performing post-build step
      00:09:31.219 Checking if email needs to be generated
      00:09:31.219 Email was triggered for: Success
      00:09:31.219 Sending email for trigger: Success
      00:09:31.219 NOT overriding default server settings, using Mailer to create session
      00:24:25.439 messageContentType = text/plain; charset=UTF-8
      00:24:25.446 Adding recipients from recipient list
      00:31:41.324 Successfully created MimeMessage
      00:31:41.324 Sending email to: mozmill-ci@mozilla.org
      00:31:41.523 Finished: SUCCESS

      As you can see the job is running 9:31 minutes. Then when trying to add the content (i hope that's right here) it takes about 15 minutes, and again 6 more minutes to create the MimeMessage.

      During all that time Jenkins has a dramatically high cpu load and it consuming nearly all CPU power with 99% load. As more jobs are running concurrently as worse the situation is becoming.

      Right now we are using version 2.32, and we haven't upgraded to the latest yet given that we don't see features or fixes included we would benefit.

      Detailed information for our current problems can be found in our own issue tracker: https://github.com/mozilla/mozmill-ci/issues/301

      We would appreciate a quick fix if possible. I would be around if you need more information. You can also reach me via IRC. My nickname is whimboo.

      Thanks

        Attachments

        1. cloudbees-support(1).zip
          12 kB
        2. cloudbees-support(10).zip
          11 kB
        3. cloudbees-support(2).zip
          12 kB
        4. cloudbees-support(3).zip
          12 kB
        5. cloudbees-support(4).zip
          12 kB
        6. cloudbees-support(5).zip
          11 kB
        7. cloudbees-support(6).zip
          11 kB
        8. cloudbees-support(7).zip
          12 kB
        9. cloudbees-support(8).zip
          11 kB
        10. cloudbees-support(9).zip
          11 kB
        11. hotFix Methods.png
          hotFix Methods.png
          84 kB
        12. PLUGIN-STATE-cloudbees-support.zip
          58 kB
        13. preHotFix Methods.png
          preHotFix Methods.png
          74 kB
        14. SPEEDY-support.zip
          17 kB

          Activity

          Hide
          imakowski Ireneusz Makowski added a comment -

          Mailer plugin is in version 1.8, but I believe i use override section in configuration, then this plugin is not used (Override Global Settings <- this setting is used)

          Show
          imakowski Ireneusz Makowski added a comment - Mailer plugin is in version 1.8, but I believe i use override section in configuration, then this plugin is not used (Override Global Settings <- this setting is used)
          Hide
          slide_o_mix Alex Earl added a comment -

          The MailAddressResolver is part of the Mailer plugin. email-ext uses this to resolve an address for a username. MailAddressResolver is an extension point that allows other plugins to provide an address resolver, SCM systems generally will provide one to map a committer username to an email address. In the case of the slowdown you are seeing, the BlameSubversionMailAddressResolverImpl is what is taking a long time, not email-ext itself.

          Show
          slide_o_mix Alex Earl added a comment - The MailAddressResolver is part of the Mailer plugin. email-ext uses this to resolve an address for a username. MailAddressResolver is an extension point that allows other plugins to provide an address resolver, SCM systems generally will provide one to map a committer username to an email address. In the case of the slowdown you are seeing, the BlameSubversionMailAddressResolverImpl is what is taking a long time, not email-ext itself.
          Hide
          slide_o_mix Alex Earl added a comment -

          The reopened issue is not with email-ext.

          Show
          slide_o_mix Alex Earl added a comment - The reopened issue is not with email-ext.
          Hide
          whimboo Henrik Skupin added a comment -

          I think the resolution of this issue should be fixed, and not 'not a defect'.

          Show
          whimboo Henrik Skupin added a comment - I think the resolution of this issue should be fixed, and not 'not a defect'.
          Hide
          slide_o_mix Alex Earl added a comment -

          Original issue was fixed, so setting the state back to fixed.

          Show
          slide_o_mix Alex Earl added a comment - Original issue was fixed, so setting the state back to fixed.

            People

            • Assignee:
              slide_o_mix Alex Earl
              Reporter:
              whimboo Henrik Skupin
            • Votes:
              5 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: