-
Improvement
-
Resolution: Won't Fix
-
Major
-
None
-
Hudson 1.382, see description
After watching our Hudson instance working too much again I want to start another try to solve our problem.
Short description:
We have a lot of CVS projects registered in our Hudson (around 400 projects). Most of them have a configuration to check CVS for changes every 20 minutes. From the instantiation of Hudson we recognized that the logic does not fit our needs because Hudson built the projects far too often. It detects SCM changes although nothing has been committed to the projects.
The configuration:
We run the Hudson Server without a master build processor and have currently added 5 nodes on which we build mostly maven projects. Because two of the nodes have limited resources we run a script periodically to remove all project workspaces on these nodes.
Most projects ought to be build on every node. And most of them, as I mentioned before, are SCM checked periodically for changes.
The problem:
- If I register a new CVS project with the mentioned configuration it gets build first, for example, on node number 1. So we have the workspace of that project on node 1.
- When the next SCM check occures Hudson checks the last build of the project and tries to get the workspace (see hudson.model.AbstractProject.poll(TaskListener)). In this case it would try to reach the workspace on node 1. The logic works correct if the workspace on that node is available, because Hudson then checks the SCM changes against the last known checkout.
- But if in between a script ran and removed the workspace on node 1, then there is no workspace to check for and Hudson says: "Build this project because we do not have a workspace yet".
Now imagine what happens if you really have all your nodes getting a clean workspace periodically and you have some hundred CVS projects: the result is that your Hudson instance will never have a break and the queue is a long long list. Every SCM check would return the need to build the according project.
The proposition:
Because I think it is not a missarranged configuration when nodes get a clean workspace periodically something has to be done for the polling to be more flexible:
- When a project gets registered and is based on a SCM project it should be checked out into a special folder in the HUDSON_HOME/jobs/PROJECTXY/ configuration folder.
- This should then be the only referencing workspace on which SCM checks will be performed against.
- When there is a SCM change occuring poll it to the reference workspace and trigger the build on any appropriate node (meaning another checkout on that node).
- This of course means a double checkout but it will decrease the overall building of a Hudson instance with the above mentioned configuration.
- To keep it generic it would probably be a good decision to make this an optional setting in Hudson.
Your opinion:
What do you think of the idea? I triggered an issue long before (JENKINS-6903) that is related to the arguments posted here and I hope there is a way to start the discussion leading to a better handling of SCM polls at all.
- is related to
-
JENKINS-6903 SCM Update leads to builds on every build processor
- Open