Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-5282

unstable state -> diff substate triggers

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Component/s: email-ext-plugin
    • Labels:
      None

      Description

      reason:
      Trigger Still-Unstable is too heavy. We are mostly on unstable state and monitor change in Test results trend. Some test take time to fix. Some are forgotten to be erased/ingored (after you find that you are not able to fix them due some wierd error in environment). Meanwhile you commit and commit and the test results graph go up and down and you are still on unstable state.

      So you constantly fire Still-Unstable trigger on email-ext. Unstable is synonymum for "Unknown" or "Unusable", because you still need to check manually Hudson dashboard for test results. Emails, Jabber are worhless.

      Yes, it is for some part problem of company policy and developers experience, but anywhere you need time for fixing the test and meanwhile new changes are coming from SCM...

      expected solution:

      • new triggers for email configuration, based on Test result change, not on build status change.

      further analysis:
      a) Still-Unstable is good enough for someone. So must be preserved.
      b) New triggers are substates of Still-Unstable state, so must be available only on this trigger selected and triggered only when "master" Still-Unstable is triggered. For UI - when I select Still-Unstable trigger, new selectbox with these substates & default "no substate"
      appear.
      c) New triggers are based on diff, so not only Test Results, but in general anything that can produce "Hudson Result table with diff" must be taken to consideration, from core or another plugins (Task scanner, PMD, CheckStyle, Emma etc). For UI - checkboxs

      • 3 new substates:
        • Still-Unstable-Increased ( "Test failed", "Checkstyle violation", "TODO tasks" whatever) - when there is +X diff on Result table
        • Still-Unstable-Decreased - when there is -X diff on Result table
        • Still-Unstable-Changed - when there is both -X diff and +X on Result table

      – note_1: Still-Unstable-Unchanged substate (0 diff) is in fact the same as master Still-Unstable state. Can be added as a forth substate to complete the list of substates for selectbox and avoid ambiguity.

      – note_2 The "Still-Unstable-Changed" is needed, it is not good for usability to have only +X and -X state and fire them both. One change can be "fix this, but screw another" and of cource one build can be triggerd by many SCM change, so this substate is valid.

      d) As in general any diff can trigger email and any change can produce diff in some results, for usability is better to have two tresholds:

      – change treshold:
      if (+X) or (-Y) <= change treshold than trigger is not fired.

      for example, in mostly any commit you change the Checkstyle results. You want to be emailed only if build violate more than for example 50 Checkstyle rules. For Test result the usual treshold from my perspective is 1; every new test failed is wrong.

      – total treshold: If you are not interested in movements in some results, but wanted to be informed when it is "too bad" (mostly it is not change from STABLE to UNSTABLE build but a long slipping regression in something, so on UNSTABLE build).

      FINAL THOUGHTS

      => with both substates and tresholds very strong Verify + Notify functionm for example when Email-ext plugin combined with Violations plugin.

      => call for this to be a new plugin / core function and not only improvement of email-ext plugin , because this kind of plugable configuration is I think usefull for not only anything that produce diff (any kind of verifier), but also anything, that notify somebody somehow (email, jabber plugin, etc.).

      So not only to better select WHEN and email it, but select WHEN do it & HOW do it. Content for each notifier must be separated, it of course difference between email and jabber.

        Activity

        Hide
        hanys hanys added a comment - - edited

        See also issue JENKINS-279.

        final thought is as I see it now basically idea presented by kohsuke there, only generalized (+ separeta call for some substates, of course)

        cit. from JENKINS-279:
        kohsuke added a comment - 12/Jun/07 06:43 AM
        It would be desirable for various facets of the e-mail notification to be
        separate out, and each made extensible.

        For example, there's "when" configuration, which determines which build should
        trigger an e-mail notification. There could be "what" configuration, which
        determines the e-mail contents. There's also "who", which determines who will
        receive e-mails. Those aspects are orthogonal.

        Making them extensible allows plugins to contribute somewhat unusual
        implementation, thereby allowing us to keep the core options relatively simple.

        Show
        hanys hanys added a comment - - edited See also issue JENKINS-279 . final thought is as I see it now basically idea presented by kohsuke there, only generalized (+ separeta call for some substates, of course) cit. from JENKINS-279 : kohsuke added a comment - 12/Jun/07 06:43 AM It would be desirable for various facets of the e-mail notification to be separate out, and each made extensible. For example, there's "when" configuration, which determines which build should trigger an e-mail notification. There could be "what" configuration, which determines the e-mail contents. There's also "who", which determines who will receive e-mails. Those aspects are orthogonal. Making them extensible allows plugins to contribute somewhat unusual implementation, thereby allowing us to keep the core options relatively simple.
        Hide
        ashlux ashlux added a comment -

        Similar to JENKINS-3430 (and it has a patch).

        Show
        ashlux ashlux added a comment - Similar to JENKINS-3430 (and it has a patch).
        Hide
        hanys hanys added a comment -

        ashlux: thanks, thiss issue I somehow missed. But my issue is more generalized (JENKINS-3430 is only about email-ext plugin), so I´m keepin it for now.

        Show
        hanys hanys added a comment - ashlux: thanks, thiss issue I somehow missed. But my issue is more generalized ( JENKINS-3430 is only about email-ext plugin), so I´m keepin it for now.
        Hide
        slide_o_mix Alex Earl added a comment -

        Is this still an issue?

        Show
        slide_o_mix Alex Earl added a comment - Is this still an issue?
        Hide
        slide_o_mix Alex Earl added a comment -

        Need more information from submitted. Reopen with more information.

        Show
        slide_o_mix Alex Earl added a comment - Need more information from submitted. Reopen with more information.

          People

          • Assignee:
            Unassigned
            Reporter:
            hanys hanys
          • Votes:
            4 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: