Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20078

email-ext plugin takes a huge amount of time to send out final status emails (30min)

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: email-ext-plugin
    • Labels:
    • Environment:
      This issue is platform independent and happens for any kind of job which sends out final status emails.
    • Similar Issues:

      Description

      When you have a job configured to send out status emails for a job (like success, fail, still failing) the jobs will block the node for up to 15 minutes or even more, only to send out the email. Given that some of our jobs only run 2min this is an absolutely large overhead we have to get rid of soon. It's mainly blocking our internal QA team from signing off of new releases.

      Here an example of the last lines of a job which has debug mode enabled for email-ext:

      00:09:31.082 Archiving artifacts
      00:09:31.093 Recording test results
      00:09:31.218 Checking for post-build
      00:09:31.218 Performing post-build step
      00:09:31.219 Checking if email needs to be generated
      00:09:31.219 Email was triggered for: Success
      00:09:31.219 Sending email for trigger: Success
      00:09:31.219 NOT overriding default server settings, using Mailer to create session
      00:24:25.439 messageContentType = text/plain; charset=UTF-8
      00:24:25.446 Adding recipients from recipient list
      00:31:41.324 Successfully created MimeMessage
      00:31:41.324 Sending email to: mozmill-ci@mozilla.org
      00:31:41.523 Finished: SUCCESS

      As you can see the job is running 9:31 minutes. Then when trying to add the content (i hope that's right here) it takes about 15 minutes, and again 6 more minutes to create the MimeMessage.

      During all that time Jenkins has a dramatically high cpu load and it consuming nearly all CPU power with 99% load. As more jobs are running concurrently as worse the situation is becoming.

      Right now we are using version 2.32, and we haven't upgraded to the latest yet given that we don't see features or fixes included we would benefit.

      Detailed information for our current problems can be found in our own issue tracker: https://github.com/mozilla/mozmill-ci/issues/301

      We would appreciate a quick fix if possible. I would be around if you need more information. You can also reach me via IRC. My nickname is whimboo.

      Thanks

        Attachments

        1. cloudbees-support(1).zip
          12 kB
        2. cloudbees-support(10).zip
          11 kB
        3. cloudbees-support(2).zip
          12 kB
        4. cloudbees-support(3).zip
          12 kB
        5. cloudbees-support(4).zip
          12 kB
        6. cloudbees-support(5).zip
          11 kB
        7. cloudbees-support(6).zip
          11 kB
        8. cloudbees-support(7).zip
          12 kB
        9. cloudbees-support(8).zip
          11 kB
        10. cloudbees-support(9).zip
          11 kB
        11. hotFix Methods.png
          hotFix Methods.png
          84 kB
        12. PLUGIN-STATE-cloudbees-support.zip
          58 kB
        13. preHotFix Methods.png
          preHotFix Methods.png
          74 kB
        14. SPEEDY-support.zip
          17 kB

          Activity

          Hide
          slide_o_mix Alex Earl added a comment -

          How is your recipient list generated? Is it a static list, or do you use tokens? Same for the content of the email, please add those items to the JENKINS ticket at a minimum. Also, 2.35.1 is the latest version of the plugin.

          Show
          slide_o_mix Alex Earl added a comment - How is your recipient list generated? Is it a static list, or do you use tokens? Same for the content of the email, please add those items to the JENKINS ticket at a minimum. Also, 2.35.1 is the latest version of the plugin.
          Hide
          whimboo Henrik Skupin added a comment -

          Sure, so as recipient we have a static entry:
          https://github.com/mozilla/mozmill-ci/blob/master/config/production/jenkins.patch#L1300

          As content we have this:
          https://github.com/mozilla/mozmill-ci/blob/master/jenkins-master/jobs/mozilla-aurora_update/config.xml#L197

          As it has been shown about 2 hours ago, a restart of Jenkins temporarily made the problem go away:

          02:06:08.483 Checking for post-build
          02:06:08.484 Performing post-build step
          02:06:08.484 Checking if email needs to be generated
          02:06:08.484 Email was triggered for: Success
          02:06:08.484 Sending email for trigger: Success
          02:06:08.485 NOT overriding default server settings, using Mailer to create session
          02:06:25.838 messageContentType = text/plain; charset=UTF-8
          02:06:25.841 Adding recipients from recipient list
          02:06:34.349 Successfully created MimeMessage
          02:06:34.350 Sending email to: mozmill-ci@mozilla.org
          02:06:34.571 Finished: SUCCESS

          It's not the first time we have seen this problem. Sadly I haven't reported it earlier.

          Regarding version 2.35.1, was there any change since 2.32 I'm not aware of, and which could have changed that?

          Show
          whimboo Henrik Skupin added a comment - Sure, so as recipient we have a static entry: https://github.com/mozilla/mozmill-ci/blob/master/config/production/jenkins.patch#L1300 As content we have this: https://github.com/mozilla/mozmill-ci/blob/master/jenkins-master/jobs/mozilla-aurora_update/config.xml#L197 As it has been shown about 2 hours ago, a restart of Jenkins temporarily made the problem go away: 02:06:08.483 Checking for post-build 02:06:08.484 Performing post-build step 02:06:08.484 Checking if email needs to be generated 02:06:08.484 Email was triggered for: Success 02:06:08.484 Sending email for trigger: Success 02:06:08.485 NOT overriding default server settings, using Mailer to create session 02:06:25.838 messageContentType = text/plain; charset=UTF-8 02:06:25.841 Adding recipients from recipient list 02:06:34.349 Successfully created MimeMessage 02:06:34.350 Sending email to: mozmill-ci@mozilla.org 02:06:34.571 Finished: SUCCESS It's not the first time we have seen this problem. Sadly I haven't reported it earlier. Regarding version 2.35.1, was there any change since 2.32 I'm not aware of, and which could have changed that?
          Hide
          whimboo Henrik Skupin added a comment -

          Just to add, the times in my last comment is not the duration but the real time. So it took about 26s, which probably also not that fast.

          Show
          whimboo Henrik Skupin added a comment - Just to add, the times in my last comment is not the duration but the real time. So it took about 26s, which probably also not that fast.
          Hide
          slide_o_mix Alex Earl added a comment -

          Can you show me your global config as well?

          Show
          slide_o_mix Alex Earl added a comment - Can you show me your global config as well?
          Hide
          whimboo Henrik Skupin added a comment -

          Sure. It's here:
          https://github.com/mozilla/mozmill-ci/blob/master/jenkins-master/config.xml

          For the various instances (staging, production) we simply patch this global config with the changes in the jenkins.patch file, which I have referenced above.

          Show
          whimboo Henrik Skupin added a comment - Sure. It's here: https://github.com/mozilla/mozmill-ci/blob/master/jenkins-master/config.xml For the various instances (staging, production) we simply patch this global config with the changes in the jenkins.patch file, which I have referenced above.
          Hide
          slide_o_mix Alex Earl added a comment -

          Also, can you increase the level for logging of Jenkins stuff to FINE and see if there is anything in there regarding email stuff.

          Show
          slide_o_mix Alex Earl added a comment - Also, can you increase the level for logging of Jenkins stuff to FINE and see if there is anything in there regarding email stuff.
          Hide
          slide_o_mix Alex Earl added a comment -

          It doesn't look like that includes anything related to email-ext, usually there is another file for the global config of each plugin. hudson.plugins.emailext.SOMETHING.xml in the same directory as the global config.

          Show
          slide_o_mix Alex Earl added a comment - It doesn't look like that includes anything related to email-ext, usually there is another file for the global config of each plugin. hudson.plugins.emailext.SOMETHING.xml in the same directory as the global config.
          Hide
          whimboo Henrik Skupin added a comment -

          Yes it is 'hudson.plugins.emailext.ExtendedEmailPublisher.xml' but we do not include it in our repository because it will be auto-generated when Jenkins is getting started.

          I will check how to set Jenkins logging into FINE mode and we might hopefully see some results soon. Thanks for letting me know that option.

          Show
          whimboo Henrik Skupin added a comment - Yes it is 'hudson.plugins.emailext.ExtendedEmailPublisher.xml' but we do not include it in our repository because it will be auto-generated when Jenkins is getting started. I will check how to set Jenkins logging into FINE mode and we might hopefully see some results soon. Thanks for letting me know that option.
          Hide
          slide_o_mix Alex Earl added a comment -

          By auto-generated, you mean it has the default values?

          Show
          slide_o_mix Alex Earl added a comment - By auto-generated, you mean it has the default values?
          Hide
          whimboo Henrik Skupin added a comment -

          That's correct. It gets the settings from the global configuration. We don't make any other changes to that file.

          Show
          whimboo Henrik Skupin added a comment - That's correct. It gets the settings from the global configuration. We don't make any other changes to that file.
          Hide
          slide_o_mix Alex Earl added a comment -

          Then how does the recipient list get set to: $NOTIFICATION_ADDRESS?

          Show
          slide_o_mix Alex Earl added a comment - Then how does the recipient list get set to: $NOTIFICATION_ADDRESS?
          Hide
          whimboo Henrik Skupin added a comment -

          This is via an environment variable which gets set in our jenkins.patch file:
          https://github.com/mozilla/mozmill-ci/blob/master/config/production/jenkins.patch#L1300

          As mentioned earlier this modifies the global config with instance specific settings. Above for our production instance.

          Show
          whimboo Henrik Skupin added a comment - This is via an environment variable which gets set in our jenkins.patch file: https://github.com/mozilla/mozmill-ci/blob/master/config/production/jenkins.patch#L1300 As mentioned earlier this modifies the global config with instance specific settings. Above for our production instance.
          Hide
          slide_o_mix Alex Earl added a comment -

          Can you show me the 'hudson.plugins.emailext.ExtendedEmailPublisher.xml' file that is generated?

          Show
          slide_o_mix Alex Earl added a comment - Can you show me the 'hudson.plugins.emailext.ExtendedEmailPublisher.xml' file that is generated?
          Hide
          whimboo Henrik Skupin added a comment -

          Sure. Here it is:

          $ cat jenkins-master/hudson.plugins.emailext.ExtendedEmailPublisher.xml
          <?xml version='1.0' encoding='UTF-8'?>
          <hudson.plugins.emailext.ExtendedEmailPublisherDescriptor plugin="email-ext@2.32">
          <hudsonUrl>http://mm-ci-master.qa.scl3.mozilla.com:8080/</hudsonUrl>
          <adminAddress>mozmill-ci@mozilla.com</adminAddress>
          <useSsl>false</useSsl>
          <defaultContentType>text/plain</defaultContentType>
          <defaultSubject>$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!</defaultSubject>
          <defaultBody>$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS:

          Check console output at $BUILD_URL to view the results.</defaultBody>
          <defaultPresendScript></defaultPresendScript>
          <maxAttachmentSize>-1</maxAttachmentSize>
          <recipientList>$NOTIFICATION_ADDRESS</recipientList>
          <defaultReplyTo></defaultReplyTo>
          <excludedCommitters></excludedCommitters>
          <overrideGlobalSettings>false</overrideGlobalSettings>
          <listId>Notifications from the Mozmill-CI system <mozmill-ci.mozilla.org></listId>
          <precedenceBulk>false</precedenceBulk>
          <debugMode>true</debugMode>
          <enableSecurity>false</enableSecurity>
          <enableWatching>false</enableWatching>
          </hudson.plugins.emailext.ExtendedEmailPublisherDescriptor>

          Show
          whimboo Henrik Skupin added a comment - Sure. Here it is: $ cat jenkins-master/hudson.plugins.emailext.ExtendedEmailPublisher.xml <?xml version='1.0' encoding='UTF-8'?> <hudson.plugins.emailext.ExtendedEmailPublisherDescriptor plugin="email-ext@2.32"> <hudsonUrl> http://mm-ci-master.qa.scl3.mozilla.com:8080/ </hudsonUrl> <adminAddress>mozmill-ci@mozilla.com</adminAddress> <useSsl>false</useSsl> <defaultContentType>text/plain</defaultContentType> <defaultSubject>$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!</defaultSubject> <defaultBody>$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS: Check console output at $BUILD_URL to view the results.</defaultBody> <defaultPresendScript></defaultPresendScript> <maxAttachmentSize>-1</maxAttachmentSize> <recipientList>$NOTIFICATION_ADDRESS</recipientList> <defaultReplyTo></defaultReplyTo> <excludedCommitters></excludedCommitters> <overrideGlobalSettings>false</overrideGlobalSettings> <listId>Notifications from the Mozmill-CI system <mozmill-ci.mozilla.org></listId> <precedenceBulk>false</precedenceBulk> <debugMode>true</debugMode> <enableSecurity>false</enableSecurity> <enableWatching>false</enableWatching> </hudson.plugins.emailext.ExtendedEmailPublisherDescriptor>
          Hide
          slide_o_mix Alex Earl added a comment -
          Show
          slide_o_mix Alex Earl added a comment - Please try the hpi located at http://earl-of-code.com/email-ext-2.32-debug.hpi
          Hide
          kt Kevin Thieling added a comment -

          This is happening to me as well. We updated from 2.33 to the latest (2.36) last week and now we are noticing builds taking a lot longer than normal. When we check the output they are just sitting there trying to send the email. All recipients are added in to the recipients box, not to a variable of any kind. The content does include some variables.

          Show
          kt Kevin Thieling added a comment - This is happening to me as well. We updated from 2.33 to the latest (2.36) last week and now we are noticing builds taking a lot longer than normal. When we check the output they are just sitting there trying to send the email. All recipients are added in to the recipients box, not to a variable of any kind. The content does include some variables.
          Hide
          slide_o_mix Alex Earl added a comment -

          Can you enable debug mode in the global config?

          Show
          slide_o_mix Alex Earl added a comment - Can you enable debug mode in the global config?
          Hide
          walterk82 Walter Kacynski added a comment -

          Interesting, I'm similar issues after implementing LTS 1.532.1.2 in Jenkins Enterprise. In my OSS 1.532.1 installation, I don't see this problem. I'm wondering if there is a plug-in conflict of some sort. I enabled the debug mode the majority of time seems to be spent between this two operations:

          14:16:34 NOT overriding default server settings, using Mailer to create session
          14:16:40 messageContentType = text/html; charset=UTF-8

          Show
          walterk82 Walter Kacynski added a comment - Interesting, I'm similar issues after implementing LTS 1.532.1.2 in Jenkins Enterprise. In my OSS 1.532.1 installation, I don't see this problem. I'm wondering if there is a plug-in conflict of some sort. I enabled the debug mode the majority of time seems to be spent between this two operations: 14:16:34 NOT overriding default server settings, using Mailer to create session 14:16:40 messageContentType = text/html; charset=UTF-8
          Hide
          slide_o_mix Alex Earl added a comment -

          Isn't that in seconds though? 6 seconds?

          Show
          slide_o_mix Alex Earl added a comment - Isn't that in seconds though? 6 seconds?
          Hide
          walterk82 Walter Kacynski added a comment -

          Agreed, in my case the difference is much less extreme, however there is a noticeable difference between my master installations.

          Show
          walterk82 Walter Kacynski added a comment - Agreed, in my case the difference is much less extreme, however there is a noticeable difference between my master installations.
          Hide
          slide_o_mix Alex Earl added a comment -

          It's possible the Mailer plugin is different between the two installations? What version of the Mailer is on each?

          Show
          slide_o_mix Alex Earl added a comment - It's possible the Mailer plugin is different between the two installations? What version of the Mailer is on each?
          Hide
          walterk82 Walter Kacynski added a comment -

          I have tried mailer 1.6 and mailer 1.8 on my "slow" installation and both given the same timing effects. I have Mailer 1.6 on my "speedy" installation.

          Show
          walterk82 Walter Kacynski added a comment - I have tried mailer 1.6 and mailer 1.8 on my "slow" installation and both given the same timing effects. I have Mailer 1.6 on my "speedy" installation.
          Hide
          slide_o_mix Alex Earl added a comment -

          Can you provide a dump of all the plugins and their versions?

          Show
          slide_o_mix Alex Earl added a comment - Can you provide a dump of all the plugins and their versions?
          Hide
          walterk82 Walter Kacynski added a comment -

          I have captured a number of thread dumps for my problem.

          It looks like:
          at hudson.plugins.emailext.plugins.ContentBuilder.getPrivateMacros(ContentBuilder.java:76)

          is causing a search of a bunch of classes to occur.

          Show
          walterk82 Walter Kacynski added a comment - I have captured a number of thread dumps for my problem. It looks like: at hudson.plugins.emailext.plugins.ContentBuilder.getPrivateMacros(ContentBuilder.java:76) is causing a search of a bunch of classes to occur.
          Hide
          walterk82 Walter Kacynski added a comment -

          I have attached a support bundle of our current configuration.

          Show
          walterk82 Walter Kacynski added a comment - I have attached a support bundle of our current configuration.
          Hide
          jglick Jesse Glick added a comment -

          Looks like the plugin is loading @EmailToken-annotated macros over and over without any caching. It is unclear to me why SezPoz is being used here directly, when the usual Jenkins @Extension would be easy to use (and would be cached).

          Show
          jglick Jesse Glick added a comment - Looks like the plugin is loading @EmailToken -annotated macros over and over without any caching. It is unclear to me why SezPoz is being used here directly, when the usual Jenkins @Extension would be easy to use (and would be cached).
          Hide
          slide_o_mix Alex Earl added a comment -

          There is a reason that I didn't use @Extension...but I don't remember what it is without looking at the code. I do need to add some caching for sure if there is a valid reason I'm not using @Extension.

          Show
          slide_o_mix Alex Earl added a comment - There is a reason that I didn't use @Extension...but I don't remember what it is without looking at the code. I do need to add some caching for sure if there is a valid reason I'm not using @Extension.
          Hide
          slide_o_mix Alex Earl added a comment -

          Ah, the reason I didn't use @Extension was because I didn't want those macros to show up in the help for Token Macro, they are specific to email-ext. I'll add some caching.

          Show
          slide_o_mix Alex Earl added a comment - Ah, the reason I didn't use @Extension was because I didn't want those macros to show up in the help for Token Macro, they are specific to email-ext. I'll add some caching.
          Hide
          walterk82 Walter Kacynski added a comment -

          Is there a particular reason why this is recently a problem? Some of my other masters don't experiencing this problem.

          Show
          walterk82 Walter Kacynski added a comment - Is there a particular reason why this is recently a problem? Some of my other masters don't experiencing this problem.
          Hide
          slide_o_mix Alex Earl added a comment - - edited

          @Walter are they all on the same version of email-ext? Also, do they all have the same plugins installed and so forth?

          Show
          slide_o_mix Alex Earl added a comment - - edited @Walter are they all on the same version of email-ext? Also, do they all have the same plugins installed and so forth?
          Hide
          walterk82 Walter Kacynski added a comment -

          Yes, unfortunately, I believe there to be a number of differences including the Jenkins core. Attached is "SPEEDY-support.zip".

          Show
          walterk82 Walter Kacynski added a comment - Yes, unfortunately, I believe there to be a number of differences including the Jenkins core. Attached is "SPEEDY-support.zip".
          Hide
          slide_o_mix Alex Earl added a comment -

          I'll try and diff the SPEEDY vs others and see if anything pops out at me.

          Show
          slide_o_mix Alex Earl added a comment - I'll try and diff the SPEEDY vs others and see if anything pops out at me.
          Hide
          xiaopan3322 Bao Xiaopan(Bob) added a comment - - edited

          Also met this problem for a long time, wish to see a final solution:

          Overriding default server settings, creating our own session
          Overriding charset UTF-8 //very slow, hang
          messageContentType = text/html; charset=UTF-8
          Adding recipients from recipient list  //very slow, hang
          Successfully created MimeMessage  //very slow, hang
          Sending email to: 
          
          Show
          xiaopan3322 Bao Xiaopan(Bob) added a comment - - edited Also met this problem for a long time, wish to see a final solution: Overriding default server settings, creating our own session Overriding charset UTF-8 //very slow, hang messageContentType = text/html; charset=UTF-8 Adding recipients from recipient list //very slow, hang Successfully created MimeMessage //very slow, hang Sending email to:
          Hide
          slide_o_mix Alex Earl added a comment -

          Is anyone willing to try a test version based on 2.37.2 to see if the fix I am putting in actually fixes the issue?

          Show
          slide_o_mix Alex Earl added a comment - Is anyone willing to try a test version based on 2.37.2 to see if the fix I am putting in actually fixes the issue?
          Hide
          slide_o_mix Alex Earl added a comment -
          Show
          slide_o_mix Alex Earl added a comment - http://earl-of-code.com/email-ext-2.37.2.1.hpi is the test version.
          Hide
          walterk82 Walter Kacynski added a comment -

          I have tested the new version and I do see an improvement. So much so that I am unable to capture a thread dump during the run of the email generation. When monitoring the process CPU, I still see a large spike in CPU time, but the duration of this spike is much smaller.

          Show
          walterk82 Walter Kacynski added a comment - I have tested the new version and I do see an improvement. So much so that I am unable to capture a thread dump during the run of the email generation. When monitoring the process CPU, I still see a large spike in CPU time, but the duration of this spike is much smaller.
          Hide
          walterk82 Walter Kacynski added a comment -
          08:23:13 Deleting old workspace snapshot from #19.
          08:23:13 Archiving artifacts
          08:23:13 Sending artifact delta relative to Deploy » SND » Build_WAS_Environment #19
          08:23:14 Archived 418 artifacts
          08:23:14 Archive block size is 32768
          08:23:14 Received 215 blocks and 142533 bytes
          08:23:14 Compression is 98.0%
          08:23:14 Checking for post-build
          08:23:14 Performing post-build step
          08:23:14 Checking if email needs to be generated
          08:23:14 Email was triggered for: Success
          08:23:14 Sending email for trigger: Success
          08:23:14 NOT overriding default server settings, using Mailer to create session
          08:23:17 messageContentType = text/html; charset=UTF-8
          08:23:17 Adding developers
          08:23:17 Sending to requester
          08:23:17 Successfully created MimeMessage
          08:23:17 Sending email to: ...........
          08:23:18 Finished: SUCCESS
          
          Show
          walterk82 Walter Kacynski added a comment - 08:23:13 Deleting old workspace snapshot from #19. 08:23:13 Archiving artifacts 08:23:13 Sending artifact delta relative to Deploy » SND » Build_WAS_Environment #19 08:23:14 Archived 418 artifacts 08:23:14 Archive block size is 32768 08:23:14 Received 215 blocks and 142533 bytes 08:23:14 Compression is 98.0% 08:23:14 Checking for post-build 08:23:14 Performing post-build step 08:23:14 Checking if email needs to be generated 08:23:14 Email was triggered for: Success 08:23:14 Sending email for trigger: Success 08:23:14 NOT overriding default server settings, using Mailer to create session 08:23:17 messageContentType = text/html; charset=UTF-8 08:23:17 Adding developers 08:23:17 Sending to requester 08:23:17 Successfully created MimeMessage 08:23:17 Sending email to: ........... 08:23:18 Finished: SUCCESS
          Hide
          walterk82 Walter Kacynski added a comment -

          On additional comment, after the first job that builds an email the process is much slower like the original implementation. The above log is from a second run so it appears that the caching has improved response time.

          Show
          walterk82 Walter Kacynski added a comment - On additional comment, after the first job that builds an email the process is much slower like the original implementation. The above log is from a second run so it appears that the caching has improved response time.
          Hide
          slide_o_mix Alex Earl added a comment -

          @Walter, just to clarify, are you saying the issue is "fixed" or that there is still an issue?

          Show
          slide_o_mix Alex Earl added a comment - @Walter, just to clarify, are you saying the issue is "fixed" or that there is still an issue?
          Hide
          walterk82 Walter Kacynski added a comment -

          I compared the Master CPU from "SPEEDY" and this fix and I would say that the problem is NOT fixed. When generating emails from "SPEEDY" I hardly notice a master CPU spike. With this fix, I see a 10% spike every-time an email is generated. This spike is much shorter than without the recent fix.

          Show
          walterk82 Walter Kacynski added a comment - I compared the Master CPU from "SPEEDY" and this fix and I would say that the problem is NOT fixed. When generating emails from "SPEEDY" I hardly notice a master CPU spike. With this fix, I see a 10% spike every-time an email is generated. This spike is much shorter than without the recent fix.
          Hide
          walterk82 Walter Kacynski added a comment -

          I have Wily/Introscope available to perform a method level trace of the application. Please let me know the package names that I can instrument that would be helpful to debug the slow areas of code if you are interested in this information. Thanks.

          Show
          walterk82 Walter Kacynski added a comment - I have Wily/Introscope available to perform a method level trace of the application. Please let me know the package names that I can instrument that would be helpful to debug the slow areas of code if you are interested in this information. Thanks.
          Hide
          slide_o_mix Alex Earl added a comment -

          hudson.plugins.emailext.* (mainly ExtendedEmailPublisher and EmailRecipientUtils)
          hudson.plugins.emailext.plugins.* (mainly ContentBuilder)
          org.jenkinsci.plugins.tokenmacro.*

          The only thing I changed in the release was adding caching. Perhaps my caching implementation wasn't a good one.

          public class ContentBuilder {
          
              @CopyOnWrite
              private static volatile List<TokenMacro> privateMacros;
              ...
          
              public String transformText(String origText, ExtendedEmailPublisherContext context, List<TokenMacro> additionalMacros) {
              ....
                 try {
                     List<TokenMacro> privMacros = getPrivateMacros();
                     if(additionalMacros != null)
                         privateMacros.addAll(additionalMacros);
                     newText = TokenMacro.expandAll(context.getBuild(), context.getListener(), newText, false, privMacros);
                 } catch (MacroEvaluationException e) {
                     context.getListener().getLogger().println("Error evaluating token: " + e.getMessage());
                 } catch (Exception e) {
                     Logger.getLogger(ContentBuilder.class.getName()).log(Level.SEVERE, null, e);
                 }
              ...
              }
          
              public static List<TokenMacro> getPrivateMacros() {
                  if(privateMacros != null)
                      return privateMacros;
          
                  privateMacros = new ArrayList<TokenMacro>();
                  ClassLoader cl = Jenkins.getInstance().pluginManager.uberClassLoader;
                  for (final IndexItem<EmailToken, TokenMacro> item : Index.load(EmailToken.class, TokenMacro.class, cl)) {
                      try {
                          privateMacros.add(item.instance());
                      } catch (Exception e) {
                          // ignore errors loading tokens
                      }
                  }
                  return privateMacros;
              }
          }
          
          

          I'm not a Java guy really, so if there is a better way to do it in Java, please give me some pointers.

          Show
          slide_o_mix Alex Earl added a comment - hudson.plugins.emailext.* (mainly ExtendedEmailPublisher and EmailRecipientUtils) hudson.plugins.emailext.plugins.* (mainly ContentBuilder) org.jenkinsci.plugins.tokenmacro.* The only thing I changed in the release was adding caching. Perhaps my caching implementation wasn't a good one. public class ContentBuilder { @CopyOnWrite private static volatile List<TokenMacro> privateMacros; ... public String transformText( String origText, ExtendedEmailPublisherContext context, List<TokenMacro> additionalMacros) { .... try { List<TokenMacro> privMacros = getPrivateMacros(); if (additionalMacros != null ) privateMacros.addAll(additionalMacros); newText = TokenMacro.expandAll(context.getBuild(), context.getListener(), newText, false , privMacros); } catch (MacroEvaluationException e) { context.getListener().getLogger().println( "Error evaluating token: " + e.getMessage()); } catch (Exception e) { Logger.getLogger(ContentBuilder. class. getName()).log(Level.SEVERE, null , e); } ... } public static List<TokenMacro> getPrivateMacros() { if (privateMacros != null ) return privateMacros; privateMacros = new ArrayList<TokenMacro>(); ClassLoader cl = Jenkins.getInstance().pluginManager.uberClassLoader; for ( final IndexItem<EmailToken, TokenMacro> item : Index.load(EmailToken.class, TokenMacro.class, cl)) { try { privateMacros.add(item.instance()); } catch (Exception e) { // ignore errors loading tokens } } return privateMacros; } } I'm not a Java guy really, so if there is a better way to do it in Java, please give me some pointers.
          Hide
          walterk82 Walter Kacynski added a comment -

          Attached are the method traces for before and after the hot Fix. The hot fix eliminated 4 seconds of time on the getPrivateMacros method.

          Show
          walterk82 Walter Kacynski added a comment - Attached are the method traces for before and after the hot Fix. The hot fix eliminated 4 seconds of time on the getPrivateMacros method.
          Hide
          walterk82 Walter Kacynski added a comment -

          So, the next question, how do I dig down into the methods calls to compromise Scriptcontent|evaluate

          Show
          walterk82 Walter Kacynski added a comment - So, the next question, how do I dig down into the methods calls to compromise Scriptcontent|evaluate
          Hide
          slide_o_mix Alex Earl added a comment -

          ScriptContent.evaluate will use GroovyShell to evaluate the template. Not sure if you are currently looking at hudson.plugins.emailext.plugins.content in your traces.

          Show
          slide_o_mix Alex Earl added a comment - ScriptContent.evaluate will use GroovyShell to evaluate the template. Not sure if you are currently looking at hudson.plugins.emailext.plugins.content in your traces.
          Hide
          walterk82 Walter Kacynski added a comment -

          I did a more detailed trace and the most time is spent in ScriptContent.renderTemplate for all my test, I'm using the same groovy based transfer template.

          In plugin version 2.36 render time is about 1.1 seconds
          In plugin version 2.37.2.1 render time is about 2.2 seconds for script / managed script based

          Show
          walterk82 Walter Kacynski added a comment - I did a more detailed trace and the most time is spent in ScriptContent.renderTemplate for all my test, I'm using the same groovy based transfer template. In plugin version 2.36 render time is about 1.1 seconds In plugin version 2.37.2.1 render time is about 2.2 seconds for script / managed script based
          Hide
          walterk82 Walter Kacynski added a comment -

          I'm not familiar at all with Groovy and how it works within the JVM. I could try switching to Java 6 to see if there is any performance change.

          Show
          walterk82 Walter Kacynski added a comment - I'm not familiar at all with Groovy and how it works within the JVM. I could try switching to Java 6 to see if there is any performance change.
          Hide
          walterk82 Walter Kacynski added a comment -

          Overall, I think that the latest caching changes can be pushed up, and I can investigate the rending performance as a separate item.

          Show
          walterk82 Walter Kacynski added a comment - Overall, I think that the latest caching changes can be pushed up, and I can investigate the rending performance as a separate item.
          Hide
          slide_o_mix Alex Earl added a comment -

          I'll look into the script rendering stuff as well, there is some magic happening in that part of the code too.

          Show
          slide_o_mix Alex Earl added a comment - I'll look into the script rendering stuff as well, there is some magic happening in that part of the code too.
          Hide
          slide_o_mix Alex Earl added a comment -

          Can you monitor hudson.plugins.emailext.plugins.content.EmailExtScript? There is some stuff that goes on in there to allow for token macros to be used in the groovy templates. It shouldn't need additional caching because TokenMacro.all caches and getPrivateMacros now caches, but it would be interesting to see if methodMissing of that class is getting called and how long it is taking.

          Show
          slide_o_mix Alex Earl added a comment - Can you monitor hudson.plugins.emailext.plugins.content.EmailExtScript? There is some stuff that goes on in there to allow for token macros to be used in the groovy templates. It shouldn't need additional caching because TokenMacro.all caches and getPrivateMacros now caches, but it would be interesting to see if methodMissing of that class is getting called and how long it is taking.
          Hide
          walterk82 Walter Kacynski added a comment -

          That class seems to be an abstract class. What or who implements a concrete version of it?

          Show
          walterk82 Walter Kacynski added a comment - That class seems to be an abstract class. What or who implements a concrete version of it?
          Hide
          slide_o_mix Alex Earl added a comment -

          The Groovy CompilerConfiguration will subclass it when it creates the script scope, so its somewhere internal to the Groovy classes.

          Show
          slide_o_mix Alex Earl added a comment - The Groovy CompilerConfiguration will subclass it when it creates the script scope, so its somewhere internal to the Groovy classes.
          Hide
          walterk82 Walter Kacynski added a comment -

          Hmm, I don't know how to instrument the groovy compiled class then. "methodMissing" disappears for some reason.

          Show
          walterk82 Walter Kacynski added a comment - Hmm, I don't know how to instrument the groovy compiled class then. "methodMissing" disappears for some reason.
          Hide
          slide_o_mix Alex Earl added a comment -

          I'm not sure either, I've never done it. I guess I could instrument in the code itself...

          Show
          slide_o_mix Alex Earl added a comment - I'm not sure either, I've never done it. I guess I could instrument in the code itself...
          Hide
          walterk82 Walter Kacynski added a comment -

          When will you be publishing 2.37.2.1 as an official fix? I can help pursue the groovy script at a later point in time?

          Show
          walterk82 Walter Kacynski added a comment - When will you be publishing 2.37.2.1 as an official fix? I can help pursue the groovy script at a later point in time?
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Alex Earl
          Path:
          src/main/java/hudson/plugins/emailext/plugins/ContentBuilder.java
          http://jenkins-ci.org/commit/email-ext-plugin/6dfefca4438cd5d15f2d3c5cd88a29b2ae2f8556
          Log:
          Fix JENKINS-20078

          Added caching to the private macro lookup

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Alex Earl Path: src/main/java/hudson/plugins/emailext/plugins/ContentBuilder.java http://jenkins-ci.org/commit/email-ext-plugin/6dfefca4438cd5d15f2d3c5cd88a29b2ae2f8556 Log: Fix JENKINS-20078 Added caching to the private macro lookup
          Hide
          imakowski Ireneusz Makowski added a comment -

          We still have an issue on latest 2.38 version with huge amount of time to send email....

          This is the output from Debug mode:
          Checking for post-build
          Performing post-build step
          Checking if email needs to be generated
          Email was triggered for: Failure - Any
          Email was triggered for: Failure - Still
          Trigger Failure - Any was overridden by another trigger and will not send an email.
          Sending email for trigger: Failure - Still
          Overriding default server settings, creating our own session
          messageContentType = text/html; charset=UTF-8

          It hangs after messageContentType for 20 minutes

          Show
          imakowski Ireneusz Makowski added a comment - We still have an issue on latest 2.38 version with huge amount of time to send email.... This is the output from Debug mode: Checking for post-build Performing post-build step Checking if email needs to be generated Email was triggered for: Failure - Any Email was triggered for: Failure - Still Trigger Failure - Any was overridden by another trigger and will not send an email. Sending email for trigger: Failure - Still Overriding default server settings, creating our own session messageContentType = text/html; charset=UTF-8 It hangs after messageContentType for 20 minutes
          Hide
          imakowski Ireneusz Makowski added a comment -

          See my comment below. Sometimes this job hangs over 40 minutes to send email.

          Show
          imakowski Ireneusz Makowski added a comment - See my comment below. Sometimes this job hangs over 40 minutes to send email.
          Hide
          slide_o_mix Alex Earl added a comment -

          Please provide a stack trace of what is happening.

          Show
          slide_o_mix Alex Earl added a comment - Please provide a stack trace of what is happening.
          Hide
          imakowski Ireneusz Makowski added a comment -

          How to do this?

          Show
          imakowski Ireneusz Makowski added a comment - How to do this?
          Hide
          slide_o_mix Alex Earl added a comment - - edited
          Show
          slide_o_mix Alex Earl added a comment - - edited https://wiki.jenkins-ci.org/display/JENKINS/Obtaining+a+thread+dump . Please do this during the long wait.
          Hide
          imakowski Ireneusz Makowski added a comment -

          One stack trace:

          at java.lang.reflect.Method.copy(Unknown Source)
          at java.lang.reflect.ReflectAccess.copyMethod(Unknown Source)
          at sun.reflect.ReflectionFactory.copyMethod(Unknown Source)
          at java.lang.Class.copyMethods(Unknown Source)
          at java.lang.Class.getMethods(Unknown Source)
          at org.apache.commons.beanutils.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:973)
          at org.apache.commons.beanutils.MappedPropertyDescriptor.getMethod(MappedPropertyDescriptor.java:409)
          at org.apache.commons.beanutils.MappedPropertyDescriptor.<init>(MappedPropertyDescriptor.java:111)
          at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:934)
          at org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:935)
          at org.apache.commons.beanutils.BeanUtilsBean.populate(BeanUtilsBean.java:830)
          at org.apache.commons.beanutils.BeanUtils.populate(BeanUtils.java:433)
          at org.apache.commons.digester.SetPropertiesRule.begin(SetPropertiesRule.java:254)
          at org.apache.commons.digester.Rule.begin(Rule.java:177)
          at org.apache.commons.digester.Digester.startElement(Digester.java:1583)
          at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
          at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
          at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
          at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
          at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
          at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
          at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
          at org.apache.commons.digester.Digester.parse(Digester.java:1871)
          at hudson.scm.BlameSubversionChangeLogParser.parse(BlameSubversionChangeLogParser.java:68)
          at hudson.scm.BlameSubversionChangeLogParser.parse(BlameSubversionChangeLogParser.java:47)
          at hudson.model.AbstractBuild.calcChangeSet(AbstractBuild.java:922)
          at hudson.model.AbstractBuild.getChangeSet(AbstractBuild.java:896)
          at hudson.model.AbstractBuild.hasParticipant(AbstractBuild.java:454)
          at hudson.model.AbstractProject.hasParticipant(AbstractProject.java:1644)
          at hudson.model.User.getProjects(User.java:479)
          at hudson.scm.BlameSubversionMailAddressResolverImpl.findMailAddressFor(BlameSubversionMailAddressResolverImpl.java:23)
          at hudson.tasks.MailAddressResolver.resolve(MailAddressResolver.java:112)
          at hudson.tasks.Mailer$UserProperty.getAddress(Mailer.java:547)
          at hudson.plugins.emailext.EmailRecipientUtils.getUserConfiguredEmail(EmailRecipientUtils.java:110)
          at hudson.plugins.emailext.plugins.recipients.CulpritsRecipientProvider.addRecipients(CulpritsRecipientProvider.java:41)
          at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:516)
          at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:290)
          at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:281)
          at hudson.plugins.emailext.ExtendedEmailPublisher.access$100(ExtendedEmailPublisher.java:79)
          at hudson.plugins.emailext.ExtendedEmailPublisher$1.endBuild(ExtendedEmailPublisher.java:689)
          at hudson.matrix.MatrixBuild$MatrixBuildExecution.post2(MatrixBuild.java:403)
          at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:725)
          at hudson.model.Run.execute(Run.java:1709)
          at hudson.matrix.MatrixBuild.run(MatrixBuild.java:304)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:231)
          at hudson.model.OneOffExecutor.run(OneOffExecutor.java:43)

          Show
          imakowski Ireneusz Makowski added a comment - One stack trace: at java.lang.reflect.Method.copy(Unknown Source) at java.lang.reflect.ReflectAccess.copyMethod(Unknown Source) at sun.reflect.ReflectionFactory.copyMethod(Unknown Source) at java.lang.Class.copyMethods(Unknown Source) at java.lang.Class.getMethods(Unknown Source) at org.apache.commons.beanutils.MethodUtils.getMatchingAccessibleMethod(MethodUtils.java:973) at org.apache.commons.beanutils.MappedPropertyDescriptor.getMethod(MappedPropertyDescriptor.java:409) at org.apache.commons.beanutils.MappedPropertyDescriptor.<init>(MappedPropertyDescriptor.java:111) at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:934) at org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:935) at org.apache.commons.beanutils.BeanUtilsBean.populate(BeanUtilsBean.java:830) at org.apache.commons.beanutils.BeanUtils.populate(BeanUtils.java:433) at org.apache.commons.digester.SetPropertiesRule.begin(SetPropertiesRule.java:254) at org.apache.commons.digester.Rule.begin(Rule.java:177) at org.apache.commons.digester.Digester.startElement(Digester.java:1583) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at org.apache.commons.digester.Digester.parse(Digester.java:1871) at hudson.scm.BlameSubversionChangeLogParser.parse(BlameSubversionChangeLogParser.java:68) at hudson.scm.BlameSubversionChangeLogParser.parse(BlameSubversionChangeLogParser.java:47) at hudson.model.AbstractBuild.calcChangeSet(AbstractBuild.java:922) at hudson.model.AbstractBuild.getChangeSet(AbstractBuild.java:896) at hudson.model.AbstractBuild.hasParticipant(AbstractBuild.java:454) at hudson.model.AbstractProject.hasParticipant(AbstractProject.java:1644) at hudson.model.User.getProjects(User.java:479) at hudson.scm.BlameSubversionMailAddressResolverImpl.findMailAddressFor(BlameSubversionMailAddressResolverImpl.java:23) at hudson.tasks.MailAddressResolver.resolve(MailAddressResolver.java:112) at hudson.tasks.Mailer$UserProperty.getAddress(Mailer.java:547) at hudson.plugins.emailext.EmailRecipientUtils.getUserConfiguredEmail(EmailRecipientUtils.java:110) at hudson.plugins.emailext.plugins.recipients.CulpritsRecipientProvider.addRecipients(CulpritsRecipientProvider.java:41) at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:516) at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:290) at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:281) at hudson.plugins.emailext.ExtendedEmailPublisher.access$100(ExtendedEmailPublisher.java:79) at hudson.plugins.emailext.ExtendedEmailPublisher$1.endBuild(ExtendedEmailPublisher.java:689) at hudson.matrix.MatrixBuild$MatrixBuildExecution.post2(MatrixBuild.java:403) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:725) at hudson.model.Run.execute(Run.java:1709) at hudson.matrix.MatrixBuild.run(MatrixBuild.java:304) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) at hudson.model.OneOffExecutor.run(OneOffExecutor.java:43)
          Hide
          imakowski Ireneusz Makowski added a comment -

          Second stack trace:

          owned by "Executor #-1 for plkrbuilddb : executing FlexNet 10 - TeamAUTO » 2 Database Tests » DBU #97" Id=41963
          at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:648)

          • blocked on hudson.model.RunMap@231579
            at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:381)
            at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:219)
            at hudson.model.AbstractProject.hasParticipant(AbstractProject.java:1643)
            at hudson.model.User.getProjects(User.java:479)
            at hudson.scm.BlameSubversionMailAddressResolverImpl.findMailAddressFor(BlameSubversionMailAddressResolverImpl.java:23)
            at hudson.tasks.MailAddressResolver.resolve(MailAddressResolver.java:112)
            at hudson.tasks.Mailer$UserProperty.getAddress(Mailer.java:547)
            at hudson.plugins.emailext.EmailRecipientUtils.getUserConfiguredEmail(EmailRecipientUtils.java:110)
            at hudson.plugins.emailext.plugins.recipients.CulpritsRecipientProvider.addRecipients(CulpritsRecipientProvider.java:41)
            at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:516)
            at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:290)
            at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:281)
            at hudson.plugins.emailext.ExtendedEmailPublisher.access$100(ExtendedEmailPublisher.java:79)
            at hudson.plugins.emailext.ExtendedEmailPublisher$1.endBuild(ExtendedEmailPublisher.java:689)
            at hudson.matrix.MatrixBuild$MatrixBuildExecution.post2(MatrixBuild.java:403)
            at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:725)
            at hudson.model.Run.execute(Run.java:1709)
            at hudson.matrix.MatrixBuild.run(MatrixBuild.java:304)
            at hudson.model.ResourceController.execute(ResourceController.java:88)
            at hudson.model.Executor.run(Executor.java:231)
            at hudson.model.OneOffExecutor.run(OneOffExecutor.java:43)
          Show
          imakowski Ireneusz Makowski added a comment - Second stack trace: owned by "Executor #-1 for plkrbuilddb : executing FlexNet 10 - TeamAUTO » 2 Database Tests » DBU #97" Id=41963 at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:648) blocked on hudson.model.RunMap@231579 at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:381) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:219) at hudson.model.AbstractProject.hasParticipant(AbstractProject.java:1643) at hudson.model.User.getProjects(User.java:479) at hudson.scm.BlameSubversionMailAddressResolverImpl.findMailAddressFor(BlameSubversionMailAddressResolverImpl.java:23) at hudson.tasks.MailAddressResolver.resolve(MailAddressResolver.java:112) at hudson.tasks.Mailer$UserProperty.getAddress(Mailer.java:547) at hudson.plugins.emailext.EmailRecipientUtils.getUserConfiguredEmail(EmailRecipientUtils.java:110) at hudson.plugins.emailext.plugins.recipients.CulpritsRecipientProvider.addRecipients(CulpritsRecipientProvider.java:41) at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:516) at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:290) at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:281) at hudson.plugins.emailext.ExtendedEmailPublisher.access$100(ExtendedEmailPublisher.java:79) at hudson.plugins.emailext.ExtendedEmailPublisher$1.endBuild(ExtendedEmailPublisher.java:689) at hudson.matrix.MatrixBuild$MatrixBuildExecution.post2(MatrixBuild.java:403) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:725) at hudson.model.Run.execute(Run.java:1709) at hudson.matrix.MatrixBuild.run(MatrixBuild.java:304) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) at hudson.model.OneOffExecutor.run(OneOffExecutor.java:43)
          Hide
          slide_o_mix Alex Earl added a comment -

          Ok, so its the Subversion mail address resolution that is taking a long time. That is not the email-ext plugin's fault.

          Show
          slide_o_mix Alex Earl added a comment - Ok, so its the Subversion mail address resolution that is taking a long time. That is not the email-ext plugin's fault.
          Hide
          slide_o_mix Alex Earl added a comment -

          What version of the Mailer plugin do you have installed?

          Show
          slide_o_mix Alex Earl added a comment - What version of the Mailer plugin do you have installed?
          Hide
          imakowski Ireneusz Makowski added a comment -

          Mailer plugin is in version 1.8, but I believe i use override section in configuration, then this plugin is not used (Override Global Settings <- this setting is used)

          Show
          imakowski Ireneusz Makowski added a comment - Mailer plugin is in version 1.8, but I believe i use override section in configuration, then this plugin is not used (Override Global Settings <- this setting is used)
          Hide
          slide_o_mix Alex Earl added a comment -

          The MailAddressResolver is part of the Mailer plugin. email-ext uses this to resolve an address for a username. MailAddressResolver is an extension point that allows other plugins to provide an address resolver, SCM systems generally will provide one to map a committer username to an email address. In the case of the slowdown you are seeing, the BlameSubversionMailAddressResolverImpl is what is taking a long time, not email-ext itself.

          Show
          slide_o_mix Alex Earl added a comment - The MailAddressResolver is part of the Mailer plugin. email-ext uses this to resolve an address for a username. MailAddressResolver is an extension point that allows other plugins to provide an address resolver, SCM systems generally will provide one to map a committer username to an email address. In the case of the slowdown you are seeing, the BlameSubversionMailAddressResolverImpl is what is taking a long time, not email-ext itself.
          Hide
          slide_o_mix Alex Earl added a comment -

          The reopened issue is not with email-ext.

          Show
          slide_o_mix Alex Earl added a comment - The reopened issue is not with email-ext.
          Hide
          whimboo Henrik Skupin added a comment -

          I think the resolution of this issue should be fixed, and not 'not a defect'.

          Show
          whimboo Henrik Skupin added a comment - I think the resolution of this issue should be fixed, and not 'not a defect'.
          Hide
          slide_o_mix Alex Earl added a comment -

          Original issue was fixed, so setting the state back to fixed.

          Show
          slide_o_mix Alex Earl added a comment - Original issue was fixed, so setting the state back to fixed.

            People

            • Assignee:
              slide_o_mix Alex Earl
              Reporter:
              whimboo Henrik Skupin
            • Votes:
              5 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: