Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18912

Token macro help text is duplicated many times

    Details

    • Similar Issues:

      Description

      When expanding the help sections in Editable Email notification or Conditional Run action, the sections for the Token Macro help text are duplicate many times. This makes the help almost unusable as there is a mass amount of scrolling that is required to figure out which options are available for use.

      Jenkins 1.510
      Token Macro 1.8
      Email Extension 2.30.2
      Run condition .10
      conditional-buildstep 1.3

        Attachments

          Activity

          walterk82 Walter Kacynski created issue -
          walterk82 Walter Kacynski made changes -
          Field Original Value New Value
          Assignee domi [ domi ] Alex Earl [ slide_o_mix ]
          Hide
          slide_o_mix Alex Earl added a comment -

          Please post a screenshot.

          Show
          slide_o_mix Alex Earl added a comment - Please post a screenshot.
          walterk82 Walter Kacynski made changes -
          Attachment repeatingtokens.jpg [ 24120 ]
          Hide
          walterk82 Walter Kacynski added a comment -

          The problem still occurs for with with token-macro 1.7 However, reverting to version 1.6 fixes this problem the caveat being that some of the token options are no longer displayed.

          Show
          walterk82 Walter Kacynski added a comment - The problem still occurs for with with token-macro 1.7 However, reverting to version 1.6 fixes this problem the caveat being that some of the token options are no longer displayed.
          Hide
          slide_o_mix Alex Earl added a comment -

          In looking at the source for run-condition and conditional-buildstep, I am not seeing where they are showing the token help. Which plugin did you take the screenshot of?

          Show
          slide_o_mix Alex Earl added a comment - In looking at the source for run-condition and conditional-buildstep, I am not seeing where they are showing the token help. Which plugin did you take the screenshot of?
          Hide
          slide_o_mix Alex Earl added a comment -

          Also, here is the diff between 1.6 and 1.7, nothing major there, so I'm not sure how 1.6 would not show the issue.

          https://github.com/jenkinsci/token-macro-plugin/compare/token-macro-1.6...token-macro-1.7

          Show
          slide_o_mix Alex Earl added a comment - Also, here is the diff between 1.6 and 1.7, nothing major there, so I'm not sure how 1.6 would not show the issue. https://github.com/jenkinsci/token-macro-plugin/compare/token-macro-1.6...token-macro-1.7
          Hide
          slide_o_mix Alex Earl added a comment -

          Can you see if this still happens with the latest email-ext?

          Show
          slide_o_mix Alex Earl added a comment - Can you see if this still happens with the latest email-ext?
          Hide
          danielbeck Daniel Beck added a comment -

          I also experience this issue. Jenkins 1.509.2 with Email-ext 2.30.2 and Token Macro 1.8.1.

          There's also significant additional page load time when viewing the config page and adding Email-ext as publisher.

          Show
          danielbeck Daniel Beck added a comment - I also experience this issue. Jenkins 1.509.2 with Email-ext 2.30.2 and Token Macro 1.8.1. There's also significant additional page load time when viewing the config page and adding Email-ext as publisher.
          Hide
          walterk82 Walter Kacynski added a comment -

          Using these version of plugins has corrected this problem and the job configuration page loads are MUCH faster.
          Token Macro 1.8.1
          Email Extension 2.32

          Show
          walterk82 Walter Kacynski added a comment - Using these version of plugins has corrected this problem and the job configuration page loads are MUCH faster. Token Macro 1.8.1 Email Extension 2.32
          slide_o_mix Alex Earl made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          walterk82 Walter Kacynski made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          walterk82 Walter Kacynski added a comment -

          OK. Apparently I was too quick to close this problem. The problem is fixed for the help text in Run condition. However, if I click on "Content Token Reference" then it takes a very long time to load and the same duplicate entries are present.

          Show
          walterk82 Walter Kacynski added a comment - OK. Apparently I was too quick to close this problem. The problem is fixed for the help text in Run condition. However, if I click on "Content Token Reference" then it takes a very long time to load and the same duplicate entries are present.
          walterk82 Walter Kacynski made changes -
          Resolution Fixed [ 1 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          Hide
          slide_o_mix Alex Earl added a comment -

          Can you post another screenshot? Also, is there anything in the Jenkins logs?

          Show
          slide_o_mix Alex Earl added a comment - Can you post another screenshot? Also, is there anything in the Jenkins logs?
          Hide
          corneil_duplessis Corneil du Plessis added a comment -

          Jenkins 1.526
          Email Ext 2.32
          Token Macro 1.8.1

          Nothing special in logfile

          The list of duplicates is very long. Notice the scrollbar on the right is nearly at the top while the email-ext appears at the bottom of my pages.
          Loading after clicking Token Reference takes a few minutes and scrolling all the way using arrows takes minutes.

          Show
          corneil_duplessis Corneil du Plessis added a comment - Jenkins 1.526 Email Ext 2.32 Token Macro 1.8.1 Nothing special in logfile The list of duplicates is very long. Notice the scrollbar on the right is nearly at the top while the email-ext appears at the bottom of my pages. Loading after clicking Token Reference takes a few minutes and scrolling all the way using arrows takes minutes.
          corneil_duplessis Corneil du Plessis made changes -
          Attachment jenkins-email-ext-dups.png [ 24228 ]
          Hide
          danielbeck Daniel Beck added a comment -

          The output of the following in the script console might be interesting (this is basically how the plugin builds the list of entries to the help dialog). Planning to run this when back at work.

                 ClassLoader cl = Jenkins.getInstance().pluginManager.uberClassLoader;
                 for (final net.java.sezpoz.IndexItem<hudson.plugins.emailext.plugins.EmailToken, org.jenkinsci.plugins.tokenmacro.TokenMacro> item : 
                      net.java.sezpoz.Index.load(hudson.plugins.emailext.plugins.EmailToken.class, org.jenkinsci.plugins.tokenmacro.TokenMacro.class, cl)) {
                     try {
                         println item
                     } catch (Exception e) {
                         // ignore errors loading tokens
                     }
                 }
          
          Show
          danielbeck Daniel Beck added a comment - The output of the following in the script console might be interesting (this is basically how the plugin builds the list of entries to the help dialog). Planning to run this when back at work. ClassLoader cl = Jenkins.getInstance().pluginManager.uberClassLoader; for ( final net.java.sezpoz.IndexItem<hudson.plugins.emailext.plugins.EmailToken, org.jenkinsci.plugins.tokenmacro.TokenMacro> item : net.java.sezpoz.Index.load(hudson.plugins.emailext.plugins.EmailToken.class, org.jenkinsci.plugins.tokenmacro.TokenMacro.class, cl)) { try { println item } catch (Exception e) { // ignore errors loading tokens } }
          Hide
          slide_o_mix Alex Earl added a comment -

          Ok, I see this on my local server. I didn't see it before with the LTS release of Jenkins, but do with 1.528. I'll see what I can dig up. Also, it only looks like its the ones provided by email-ext, tokens provided by other plugins don't seem to have this issue, so I think its definitely related to the email-ext itself and not Token Macro.

          Show
          slide_o_mix Alex Earl added a comment - Ok, I see this on my local server. I didn't see it before with the LTS release of Jenkins, but do with 1.528. I'll see what I can dig up. Also, it only looks like its the ones provided by email-ext, tokens provided by other plugins don't seem to have this issue, so I think its definitely related to the email-ext itself and not Token Macro.
          Hide
          oleg_nenashev Oleg Nenashev added a comment -

          Alex, issue can be reproduced in 1.509.3

          Show
          oleg_nenashev Oleg Nenashev added a comment - Alex, issue can be reproduced in 1.509.3
          Hide
          slide_o_mix Alex Earl added a comment -

          Yes, the problem is that I can't reproduce it when running the plugin in debug mode, so its very difficult to figure out why its occuring.

          Show
          slide_o_mix Alex Earl added a comment - Yes, the problem is that I can't reproduce it when running the plugin in debug mode, so its very difficult to figure out why its occuring.
          Hide
          domi Dominik Bartholdi added a comment -

          does this problem still exist if you disable/remove the email-ext plugin?

          Show
          domi Dominik Bartholdi added a comment - does this problem still exist if you disable/remove the email-ext plugin?
          Hide
          danielbeck Daniel Beck added a comment -

          Dominik: How'd you determine whether the issue exists without access to the affected help text that is provided by email-ext?

          Show
          danielbeck Daniel Beck added a comment - Dominik: How'd you determine whether the issue exists without access to the affected help text that is provided by email-ext?
          Hide
          domi Dominik Bartholdi added a comment -

          you'r right, sorry.
          but what if you create a "boolean" conditional-buildstep and open the help for it. This will also expand all the token-macro definitions in the help context.

          Show
          domi Dominik Bartholdi added a comment - you'r right, sorry. but what if you create a "boolean" conditional-buildstep and open the help for it. This will also expand all the token-macro definitions in the help context.
          Hide
          slide_o_mix Alex Earl added a comment -

          @Dominik - the email-ext plugin uses a "private tokens" feature of token-macro so it doesn't pollute the macros for things that would only really be useful in email generation. It seems from the pictures posted, only those "private tokens" are repeating, so I think this problem would go away if email-ext is not there. I will try out your suggestion though and see if it is indeed reproducible without email-ext.

          Show
          slide_o_mix Alex Earl added a comment - @Dominik - the email-ext plugin uses a "private tokens" feature of token-macro so it doesn't pollute the macros for things that would only really be useful in email generation. It seems from the pictures posted, only those "private tokens" are repeating, so I think this problem would go away if email-ext is not there. I will try out your suggestion though and see if it is indeed reproducible without email-ext.
          Hide
          slide_o_mix Alex Earl added a comment -

          Any additional information that could help me track down the issue would be great. I can't replicate it when I'm running the plugin in debug mode, so its going to take some more information and sleuthing to get this figured out.

          Show
          slide_o_mix Alex Earl added a comment - Any additional information that could help me track down the issue would be great. I can't replicate it when I'm running the plugin in debug mode, so its going to take some more information and sleuthing to get this figured out.
          Hide
          corneil_duplessis Corneil du Plessis added a comment -

          Still happens in 1.535 and mail-ext 2.35.1

          The mail-ext does not seems to remove duplicates when determining the list of token expansions available.

          Show
          corneil_duplessis Corneil du Plessis added a comment - Still happens in 1.535 and mail-ext 2.35.1 The mail-ext does not seems to remove duplicates when determining the list of token expansions available.
          corneil_duplessis Corneil du Plessis made changes -
          Priority Minor [ 4 ] Major [ 3 ]
          Hide
          slide_o_mix Alex Earl added a comment -

          There should be no duplicates in the list. When I've tried debugging, the list returned does not contain any duplicates. I have not been able to reproduce this in a debug case.

          Show
          slide_o_mix Alex Earl added a comment - There should be no duplicates in the list. When I've tried debugging, the list returned does not contain any duplicates. I have not been able to reproduce this in a debug case.
          Hide
          slide_o_mix Alex Earl added a comment -

          When I click on the token help for conditional-build-plugin, I don't even see BUILD_ID or BUILD_LOG in the list of available tokens. Can someone post their config.xml for the job they are seeing this on?

          Show
          slide_o_mix Alex Earl added a comment - When I click on the token help for conditional-build-plugin, I don't even see BUILD_ID or BUILD_LOG in the list of available tokens. Can someone post their config.xml for the job they are seeing this on?
          Hide
          abarychev Andrei Barychev added a comment -

          Maybe this will help.

          I looked at the number of BUILD_LOG tokens on the help screen, and found 366 of them, with the size of the frame about 2.3MB.

          After Jenkins is restarted yesterday evening, there was exactly 1 BUILD_LOG token.
          After running 8 overnight jobs, there were 30 BUILD_LOG tokens.
          After running 2 more jobs, there were 37 tokens.
          One more, there were 41. One more, and it is 45. One more, it is 49.

          A run of a job adds 4 more sets of tokens to the help screen. However, the 8 overnight jobs produced 29 new tokens, which appears more like 7*4+1. Also, the first 2 jobs in the morning produced 3+4 tokens. It might be worth to notice that the morning project was brand new, with the first job being the very first run of it.

          As you asked, I'll upload the config.xml for one of the jobs that ran overnight.

          The relevant installed plugins are:

          build-blocker-plugin 1.4.1
          build-pipeline-plugin 1.3.5
          conditional-buildstep 1.3.1
          email-ext 2.32
          envinject 1.88
          mailer 1.5
          parameterized-trigger 2.2
          run-condition 0.1
          token-macro 1.8.1

          Show
          abarychev Andrei Barychev added a comment - Maybe this will help. I looked at the number of BUILD_LOG tokens on the help screen, and found 366 of them, with the size of the frame about 2.3MB. After Jenkins is restarted yesterday evening, there was exactly 1 BUILD_LOG token. After running 8 overnight jobs, there were 30 BUILD_LOG tokens. After running 2 more jobs, there were 37 tokens. One more, there were 41. One more, and it is 45. One more, it is 49. A run of a job adds 4 more sets of tokens to the help screen. However, the 8 overnight jobs produced 29 new tokens, which appears more like 7*4+1. Also, the first 2 jobs in the morning produced 3+4 tokens. It might be worth to notice that the morning project was brand new, with the first job being the very first run of it. As you asked, I'll upload the config.xml for one of the jobs that ran overnight. The relevant installed plugins are: build-blocker-plugin 1.4.1 build-pipeline-plugin 1.3.5 conditional-buildstep 1.3.1 email-ext 2.32 envinject 1.88 mailer 1.5 parameterized-trigger 2.2 run-condition 0.1 token-macro 1.8.1
          abarychev Andrei Barychev made changes -
          Attachment config.xml [ 24580 ]
          Hide
          slide_o_mix Alex Earl added a comment -

          Are you looking at the help for email-ext or conditional-build?

          Show
          slide_o_mix Alex Earl added a comment - Are you looking at the help for email-ext or conditional-build?
          Hide
          abarychev Andrei Barychev added a comment - - edited

          The duplicate tokens are in both, ext-mail and cond-build, help frames. Updated: with equal number of duplicates.

          If that matters, the conditional-step plugin is installed, but no projects use any conditional build steps.

          Show
          abarychev Andrei Barychev added a comment - - edited The duplicate tokens are in both, ext-mail and cond-build, help frames. Updated: with equal number of duplicates. If that matters, the conditional-step plugin is installed, but no projects use any conditional build steps.
          Hide
          slide_o_mix Alex Earl added a comment -

          What I don't understand is those tokens shouldn't appear for cond-build AT ALL. They are not marked with the annotation that token-macro should show them, so I don't understand how they are being shown. For email-ext, they should only be shown under the Email Ext Tokens header and not in the Token Macro tokens.

          Show
          slide_o_mix Alex Earl added a comment - What I don't understand is those tokens shouldn't appear for cond-build AT ALL. They are not marked with the annotation that token-macro should show them, so I don't understand how they are being shown. For email-ext, they should only be shown under the Email Ext Tokens header and not in the Token Macro tokens.
          Hide
          abarychev Andrei Barychev added a comment -

          I had a project that never ever ran, with ext-mail configured. I dropped ext-mail from the project – that added 1 token to the duplicates. I added ext-mail back to the project – no new duplicates. I then ran it: 3 more tokens after the first job, 2 more after the second job, and then 2 more tokens after each subsequent run. The other project (mentioned above) still adds 4 tokens with each run. And I can't see anything between the two projects that would make sense for the difference in duplicates. Adding/dropping triggers does not affect the number of duplicates added with each run.

          Show
          abarychev Andrei Barychev added a comment - I had a project that never ever ran, with ext-mail configured. I dropped ext-mail from the project – that added 1 token to the duplicates. I added ext-mail back to the project – no new duplicates. I then ran it: 3 more tokens after the first job, 2 more after the second job, and then 2 more tokens after each subsequent run. The other project (mentioned above) still adds 4 tokens with each run. And I can't see anything between the two projects that would make sense for the difference in duplicates. Adding/dropping triggers does not affect the number of duplicates added with each run.
          Hide
          slide_o_mix Alex Earl added a comment -

          Can you post screenshots please?

          Show
          slide_o_mix Alex Earl added a comment - Can you post screenshots please?
          Hide
          abarychev Andrei Barychev added a comment -

          Sure. Anything that may help. What screenshots would you like to see? I can also upload the configs for the two projects that add 2 and 4 duplicates respectively to the screen with each run. If you come up with any test scenarios, I could run them too.

          Show
          abarychev Andrei Barychev added a comment - Sure. Anything that may help. What screenshots would you like to see? I can also upload the configs for the two projects that add 2 and 4 duplicates respectively to the screen with each run. If you come up with any test scenarios, I could run them too.
          Hide
          slide_o_mix Alex Earl added a comment -

          I'd like to see the context in which you are seeing the issue (e.g., is it only in the project level or does it show up in the global config as well?) Also, seeing how it looks in the conditional-build step area vs. email-ext area.

          Show
          slide_o_mix Alex Earl added a comment - I'd like to see the context in which you are seeing the issue (e.g., is it only in the project level or does it show up in the global config as well?) Also, seeing how it looks in the conditional-build step area vs. email-ext area.
          Hide
          abarychev Andrei Barychev added a comment -

          Alex, the screenshots are so huge now, it's impractical to take them. You're interested in a full screenshot, right? I'll restart the server on weekend and take the screenshots while they are still manageable. In the meantime – the duplicate tokens are visible across all projects as well as in the global config.

          Show
          abarychev Andrei Barychev added a comment - Alex, the screenshots are so huge now, it's impractical to take them. You're interested in a full screenshot, right? I'll restart the server on weekend and take the screenshots while they are still manageable. In the meantime – the duplicate tokens are visible across all projects as well as in the global config.
          Hide
          slide_o_mix Alex Earl added a comment -

          I guess I'm not so interested in a full screenshot, just something that gives me a little more context of where to look. I need to understand why the private tokens from email-ext are showing up in conditional-build plugin at all, they shouldn't be, so I want to see if I can understand from the context (a job's config.xml might help as well).

          Show
          slide_o_mix Alex Earl added a comment - I guess I'm not so interested in a full screenshot, just something that gives me a little more context of where to look. I need to understand why the private tokens from email-ext are showing up in conditional-build plugin at all, they shouldn't be, so I want to see if I can understand from the context (a job's config.xml might help as well).
          Hide
          abarychev Andrei Barychev added a comment -

          As per your request: Please take a look at the set of Dups-* files. There are screenshots of:

          • Jenkins global config after restart
          • Jenkins global config after a run of a project
          • Project after restart (mail-ext and cond-step tokens)
          • Project after 1 run (mail-ext and cond-step tokens)
            I have also attached the project's xml config.
          Show
          abarychev Andrei Barychev added a comment - As per your request: Please take a look at the set of Dups-* files. There are screenshots of: Jenkins global config after restart Jenkins global config after a run of a project Project after restart (mail-ext and cond-step tokens) Project after 1 run (mail-ext and cond-step tokens) I have also attached the project's xml config.
          Hide
          abarychev Andrei Barychev added a comment -

          As you can see in the screenshots, the private tokens are not present in cond-step's token list after Jenkins restart. They appear there only after I run a job.

          Show
          abarychev Andrei Barychev added a comment - As you can see in the screenshots, the private tokens are not present in cond-step's token list after Jenkins restart. They appear there only after I run a job.
          Hide
          slide_o_mix Alex Earl added a comment -

          Aha! I think I have a fix for this. I'll push the changes and post the URL for the build server so you can try a new hpi for token-macro.

          Show
          slide_o_mix Alex Earl added a comment - Aha! I think I have a fix for this. I'll push the changes and post the URL for the build server so you can try a new hpi for token-macro.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Alex Earl
          Path:
          src/main/java/org/jenkinsci/plugins/tokenmacro/TokenMacro.java
          http://jenkins-ci.org/commit/token-macro-plugin/95b946e160434368fb381031af1879d246520580
          Log:
          Fix JENKINS-18912

          Created new list to hold the macros instead of using that returned by all(). If we use the one returned by all(), then all the private macros get added to the ExtensionList for TokenMacro!

          Compare: https://github.com/jenkinsci/token-macro-plugin/compare/c08666135eb6...95b946e16043

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Alex Earl Path: src/main/java/org/jenkinsci/plugins/tokenmacro/TokenMacro.java http://jenkins-ci.org/commit/token-macro-plugin/95b946e160434368fb381031af1879d246520580 Log: Fix JENKINS-18912 Created new list to hold the macros instead of using that returned by all(). If we use the one returned by all(), then all the private macros get added to the ExtensionList for TokenMacro! Compare: https://github.com/jenkinsci/token-macro-plugin/compare/c08666135eb6...95b946e16043
          Show
          slide_o_mix Alex Earl added a comment - https://jenkins.ci.cloudbees.com/job/plugins/job/token-macro-plugin/lastBuild/org.jenkins-ci.plugins$token-macro/artifact/org.jenkins-ci.plugins/token-macro/1.9-SNAPSHOT/token-macro-1.9-SNAPSHOT.hpi
          Hide
          slide_o_mix Alex Earl added a comment -

          Made a new List<TokenMacro> so that the private tokens were not added to the ExtensionList<TokenMacro> each time.

          Show
          slide_o_mix Alex Earl added a comment - Made a new List<TokenMacro> so that the private tokens were not added to the ExtensionList<TokenMacro> each time.
          slide_o_mix Alex Earl made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 150359 ] JNJira + In-Review [ 193490 ]

            People

            • Assignee:
              slide_o_mix Alex Earl
              Reporter:
              walterk82 Walter Kacynski
            • Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: