Details

    • Similar Issues:

      Description

      The plugin should be able to synchronize the configuration in two directions. That means When changes are detected in the SCM, they should be applied to Hudson. In addition, there is the issue, that the config was changed in the file system, what should happen then?

      config changed in SCM:

      • when to detect?
      • The plugin checks the scm periodically (configurable interval) or manually (button in "Manage Hudson")
      • On Hudson start up
      • When committing a new config
      • What action when change detected (should be configurable)
      • SCM always rules (update local config, trigger a reload of config). If a change requires a restart notify admin or display a message (automatically restart ??)
      • local always rules -> Revert/update the scm
      • notify admin or display a message in Hudson

      config on disc changed:

      • When to detect?
      • NO periodic check required. Change would have needed to be manually loaded anyway (restart or load config).
      • When reloading the config was triggered
      • When Hudson starts up
      • What action when change detected (should be configurable)
      • SCM always rules (revert the local changes.
      • local always rules (commit changes to SCM)
      • newer copy rules (only one changed)
      • newer copy rules (even if both changed)
      • if scm & local changed - ask (email to admin or message in Hudson)
      • always ask (email to admin or message in Hudson)

      The nice-to-have bonus:
      when both configs changed and asking whether to use local or scm config use

      • local
      • scm
      • newest version (timestamp)
        until the changes are confirmed.
        If the local version changed and asking whether to accept the changes, use the scm config until the change is confirmed.

        Attachments

          Issue Links

            Activity

            Hide
            fcamblor Frédéric Camblor added a comment - - edited

            To begin, let's focus on "what is done" in the current version "0.0.3".
            Once configured, SCM Sync configuration plugin will :
            (A) Checkout files in the HUDSON_HOME/scm-sync-configuration/checkoutConfiguration folder.
            (B) Then, to initialize the SCM repository, every sync'ed files will be copied from HUDSON_HOME/ hierarchy to HUDSON_HOME/scm-sync-configuration/checkoutConfiguration/ (we'll call it Hudson_SSC in the next occurences of this comment) and, if necessary (files doesn't exist / are different between HUDSON_HOME and Hudson_SSC), will be commited to the SCM.

            Hence, here is my point of view for the 2 way synchronization.

            I don't like the SCM => Hudson sync would be made "automatically".
            Except for a "mirroring" Hudson instance, I think the SCM => Hudson should be made by hand, via the Hudson UI.
            I was thinking it this way :

            • A new entry in the administration panel, leading to a dashboard displaying the state of every sync'ed files
            • When displaying the view, a check will be performed to eventually revert pending changes in Hudson_SSC (file modified in Hudson_SSC and not commited for unknown reasons). Thus, files in Hudson_SSC will always corresponds to a revision (eventually not the HEAD).
            • Several states are possible, with UI actions allowed for every state :
              (1) Sync'ed : file in HUDSON_HOME/ and Hudson_SSC/ are the same. File in Hudson_SSC is up to date (last revision) => Administrator won't be able to do anything
              (2) Not SCM up to date : file in HUDSON_HOME/ and Hudson_SSC/ are the same. But File in Hudson_SSC is not the last revision => Administrator will have a button allowing to "update configuration" (svn update will be performed, updated file will copied in HUDSON_HOME)
              (3) Created on SCM : here, we must check if this created file on SCM doesn't exist on Hudson. If it doesn't exist on Hudson, allow to retrieve file from SCM and populate HUDSON_HOME with it. If file were already existing in Hudson, we should prompt the user if he prefers to overwrite SCM or Hudson file
              (4) Deleted on SCM : we must see if this file exist on Hudson. If this is the case, we should allow the user to "delete the file on Hudson too". If file doesn't exist on Hudson, just ask the user to update Hudson_SSC.
              So, it was the simpler part ... Now the harder one : files in HUDSON_HOME & Hudson_SSC differ.
              (5) Synchronization not up to date : file in HUDSON_HOME differs from HUDSON_SSC, and HUDSON_SSC is svn up to date. This points a synchronization problem in the past (config file edited without being commited on SCM). We should prompt the user either to commit the change on the SCM, or retrieve the SCM file.
              (6) Time failure : both file in HUDSON_HOME & Hudson_SSC differ AND file Hudson_SSC is not SCM up to date => we should prompt the user for a "update configuration without applying it in Hudson". Then SCM file will be up to date and we'll be either in configuration (1) or (5). This step could be done automatically (it will only alter Hudson_SSC and not HUDSON_HOME)

            I hope I covered every possible cases..
            To sum up, from a UI point of view, user will only have, for each file, following buttons :

            • Nothing to do, file is green ! (1)
            • "Refresh Hudson configuration with SCM news" (2) (4)
            • Import new SCM file (3)
            • Overwrite existing Hudson file with the SCM one (3) (5)
            • Overwrite SCM file with Hudson one (3) (5)
            • Delete not-synchronized-anymore file (4)
              Each time we'll have a difference, I planned to provide some nice UI comparison tool based on prettydiff (http://prettydiff.com/ – javascript based diff tool )

            We'll have to wonder about some things :

            • What triggers a Hudson reboot ? I'm not sure that if we change a config.xml, underlying object in Hudson is refreshed (I'm really not sure about that ! Will have to test it !). If it is what I'm thinking, every time we performed a SCM => Hudson synchronization, we should propose the Administrator a "reboot Hudson" button.
            Show
            fcamblor Frédéric Camblor added a comment - - edited To begin, let's focus on "what is done" in the current version "0.0.3". Once configured, SCM Sync configuration plugin will : (A) Checkout files in the HUDSON_HOME/scm-sync-configuration/checkoutConfiguration folder. (B) Then, to initialize the SCM repository, every sync'ed files will be copied from HUDSON_HOME/ hierarchy to HUDSON_HOME/scm-sync-configuration/checkoutConfiguration/ (we'll call it Hudson_SSC in the next occurences of this comment) and, if necessary (files doesn't exist / are different between HUDSON_HOME and Hudson_SSC), will be commited to the SCM. Hence, here is my point of view for the 2 way synchronization. I don't like the SCM => Hudson sync would be made "automatically". Except for a "mirroring" Hudson instance, I think the SCM => Hudson should be made by hand, via the Hudson UI. I was thinking it this way : A new entry in the administration panel, leading to a dashboard displaying the state of every sync'ed files When displaying the view, a check will be performed to eventually revert pending changes in Hudson_SSC (file modified in Hudson_SSC and not commited for unknown reasons). Thus, files in Hudson_SSC will always corresponds to a revision (eventually not the HEAD). Several states are possible, with UI actions allowed for every state : (1) Sync'ed : file in HUDSON_HOME/ and Hudson_SSC/ are the same. File in Hudson_SSC is up to date (last revision) => Administrator won't be able to do anything (2) Not SCM up to date : file in HUDSON_HOME/ and Hudson_SSC/ are the same. But File in Hudson_SSC is not the last revision => Administrator will have a button allowing to "update configuration" (svn update will be performed, updated file will copied in HUDSON_HOME) (3) Created on SCM : here, we must check if this created file on SCM doesn't exist on Hudson. If it doesn't exist on Hudson, allow to retrieve file from SCM and populate HUDSON_HOME with it. If file were already existing in Hudson, we should prompt the user if he prefers to overwrite SCM or Hudson file (4) Deleted on SCM : we must see if this file exist on Hudson. If this is the case, we should allow the user to "delete the file on Hudson too". If file doesn't exist on Hudson, just ask the user to update Hudson_SSC. So, it was the simpler part ... Now the harder one : files in HUDSON_HOME & Hudson_SSC differ. (5) Synchronization not up to date : file in HUDSON_HOME differs from HUDSON_SSC, and HUDSON_SSC is svn up to date. This points a synchronization problem in the past (config file edited without being commited on SCM). We should prompt the user either to commit the change on the SCM, or retrieve the SCM file. (6) Time failure : both file in HUDSON_HOME & Hudson_SSC differ AND file Hudson_SSC is not SCM up to date => we should prompt the user for a "update configuration without applying it in Hudson". Then SCM file will be up to date and we'll be either in configuration (1) or (5). This step could be done automatically (it will only alter Hudson_SSC and not HUDSON_HOME) I hope I covered every possible cases.. To sum up, from a UI point of view, user will only have, for each file, following buttons : Nothing to do, file is green ! (1) "Refresh Hudson configuration with SCM news" (2) (4) Import new SCM file (3) Overwrite existing Hudson file with the SCM one (3) (5) Overwrite SCM file with Hudson one (3) (5) Delete not-synchronized-anymore file (4) Each time we'll have a difference, I planned to provide some nice UI comparison tool based on prettydiff ( http://prettydiff.com/ – javascript based diff tool ) We'll have to wonder about some things : What triggers a Hudson reboot ? I'm not sure that if we change a config.xml, underlying object in Hudson is refreshed (I'm really not sure about that ! Will have to test it !). If it is what I'm thinking, every time we performed a SCM => Hudson synchronization, we should propose the Administrator a "reboot Hudson" button.
            Hide
            peter_schuetze peter_schuetze added a comment -

            Thanks fcamblor, for that great writeup. Of course I disagree with some points of your analysis. (If you know me better you would expect that from me - I like to play the devils advocate.)

            Strategies
            ----------
            Currently the plugin only supports one strategy - HUDSON_HOME rules. I think the plugin should support more strategies.
            (S-1) HUDSON_HOME rules
            (S-2) SVN rules
            (S-3) HUDSON_HOME and SVN rules
            In case S-1 and S-2 the synchronization can be performed automatically. For S-3, the automation needs additional criteria or default to asking the admin . In all three cases there should be a ask admin option (default).

            Use cases
            ---------

            • All jobs will be tested on a Hudson-Dev instance. Hudson_Dev pushes the changes to its own configuration branch. When the job does what it is supposed to do, the changes will be copied from the dev location in SCM to the prod location in SCM. The prod Hudson will pull the changes and apply them without additional user iteraction. (S-2)
            • Some job changes will automatically be performed using the API or by changing the configuration on disk. The desire is to automatically update SCM to have a history of the file changes. The assumption is here that the plugin does not intercept (re)loading the config or config changes through the API to automatically commit them. (S-1)
            • A combination of the above. The big changes are developed and tested on the Dev-Hudson and pushed to the prod instance via SCM. The smaller changes will be performed directly on the Prod-Hudson. (S-3)
            • Jobs will be update and created in Hudson. But certain jobs are created automatically and pushed to SCM instead of using the remote API from Hudson. (S-3)

            Relevance of Hudson_SSC
            or
            How to compare
            -----------------------
            We don't need to change the behavior of the current checkout behavior. As long as you make sure that files that are not in the SCM are eliminated (svn revert + svn update should do the trick). Special cases:
            (C-1) The location in SCM does not exists. Delete the contents of HUDSON_SSC and initialize HUDSON_SSC with contents of HODSON_HOME hierarchy. Initialize the SCM with contents of HUDSON_SSC.
            (C-2) HUDSON_SSC is not attached to any SCM. Delete the contents of HUDSON_SSC and checkout a fresh copy from SCM. Synchronize according to compare strategy.
            If you follow the above step every time you synchronize with the SCM, the HUDSON_SSC folder is up to date. This makes it easier for the plugin too, since it only has to worry about the UI point of view.

            Ideas for Implementation
            ------------------------

            • Hudson offers the remote API, to change a job on the fly. It can always be called from inside or outside of Hudson. I am not sure what the implications are, if the Hudson instance is secured by username and password.
            • The configuration slicing plugin can update the job config on the fly, so there must be a functionality to change the job-config from a plugin without reloading the config.

            Summery
            -------
            I hope that I properly communicated the motivation behind this request and that I have shown that the scenarios are possible (may be even used) in the real world. For me the first use case (Hudson-Dev -> Hudson-Prod) is appealing and I would be interested to implement this in my environment.

            Legend
            ------

            explanation of the numbering scheme:

            S ... Strategy
            I ... Initialization
            C ... Compare Process

            Show
            peter_schuetze peter_schuetze added a comment - Thanks fcamblor, for that great writeup. Of course I disagree with some points of your analysis. (If you know me better you would expect that from me - I like to play the devils advocate.) Strategies ---------- Currently the plugin only supports one strategy - HUDSON_HOME rules. I think the plugin should support more strategies. (S-1) HUDSON_HOME rules (S-2) SVN rules (S-3) HUDSON_HOME and SVN rules In case S-1 and S-2 the synchronization can be performed automatically. For S-3, the automation needs additional criteria or default to asking the admin . In all three cases there should be a ask admin option (default). Use cases --------- All jobs will be tested on a Hudson-Dev instance. Hudson_Dev pushes the changes to its own configuration branch. When the job does what it is supposed to do, the changes will be copied from the dev location in SCM to the prod location in SCM. The prod Hudson will pull the changes and apply them without additional user iteraction. (S-2) Some job changes will automatically be performed using the API or by changing the configuration on disk. The desire is to automatically update SCM to have a history of the file changes. The assumption is here that the plugin does not intercept (re)loading the config or config changes through the API to automatically commit them. (S-1) A combination of the above. The big changes are developed and tested on the Dev-Hudson and pushed to the prod instance via SCM. The smaller changes will be performed directly on the Prod-Hudson. (S-3) Jobs will be update and created in Hudson. But certain jobs are created automatically and pushed to SCM instead of using the remote API from Hudson. (S-3) Relevance of Hudson_SSC or How to compare ----------------------- We don't need to change the behavior of the current checkout behavior. As long as you make sure that files that are not in the SCM are eliminated (svn revert + svn update should do the trick). Special cases: (C-1) The location in SCM does not exists. Delete the contents of HUDSON_SSC and initialize HUDSON_SSC with contents of HODSON_HOME hierarchy. Initialize the SCM with contents of HUDSON_SSC. (C-2) HUDSON_SSC is not attached to any SCM. Delete the contents of HUDSON_SSC and checkout a fresh copy from SCM. Synchronize according to compare strategy. If you follow the above step every time you synchronize with the SCM, the HUDSON_SSC folder is up to date. This makes it easier for the plugin too, since it only has to worry about the UI point of view. Ideas for Implementation ------------------------ Hudson offers the remote API, to change a job on the fly. It can always be called from inside or outside of Hudson. I am not sure what the implications are, if the Hudson instance is secured by username and password. The configuration slicing plugin can update the job config on the fly, so there must be a functionality to change the job-config from a plugin without reloading the config. Summery ------- I hope that I properly communicated the motivation behind this request and that I have shown that the scenarios are possible (may be even used) in the real world. For me the first use case (Hudson-Dev -> Hudson-Prod) is appealing and I would be interested to implement this in my environment. Legend ------ explanation of the numbering scheme: S ... Strategy I ... Initialization C ... Compare Process
            Hide
            wo_hauser Wolfgang Hauser added a comment -

            As we often change the jenkins configuration directly on disk. Especially for bulk changes or adds. The plugin should synchronize the scm while reloading the configuration from disk.

            I agree with the strategies peter_schuetze suggested. This will be our strategies too. Since our system isn't very huge, we do changes according (S-1) normally.

            So a synchronization where JENKINS_HOME (HUDSON_HOME) rules the scm is what I would prefer when the configuration is reloaded from disk by JENKINS maintenance command.

            Show
            wo_hauser Wolfgang Hauser added a comment - As we often change the jenkins configuration directly on disk. Especially for bulk changes or adds. The plugin should synchronize the scm while reloading the configuration from disk. I agree with the strategies peter_schuetze suggested. This will be our strategies too. Since our system isn't very huge, we do changes according (S-1) normally. So a synchronization where JENKINS_HOME (HUDSON_HOME) rules the scm is what I would prefer when the configuration is reloaded from disk by JENKINS maintenance command.

              People

              • Assignee:
                fcamblor Frédéric Camblor
                Reporter:
                peter_schuetze peter_schuetze
              • Votes:
                5 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: