Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52296

DependencyGraph persistence make jobs unreadable

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: job-fan-in-plugin
    • Labels:
      None
    • Environment:
      job-fan-in:1.1.3
    • Similar Issues:

      Description

      The dependency graph is persisted in the XML in few scenarios and this makes a job unreadable and could also cause performance issue.

      The DependencyGraph can be huge. In some cases, I have seen the config.xml containing >12000 lines just for the dependencyGraph attribute. Jenkins is not able to load the job, trying to read an XML attribute that is just too much.

      How to Reproduce

      The simplest scenario is as follow:

      • Create an upstream job
      • Create a downstream job with a "Fan In" trigger that takes the upstream job as "Upstream Projects to watch"

      You'll see that the config.xml does not contain the dependencyGraph:

      [...]
      
      	<triggers>
      		<org.lonkar.jobfanin.FanInReverseBuildTrigger plugin="job-fan-in@1.1.3">
      			<spec/>
      			<upstreamProjects>upstream</upstreamProjects>
      			<watchUpstreamRecursively>false</watchUpstreamRecursively>
      			<threshold>
      				<name>SUCCESS</name>
      				<ordinal>0</ordinal>
      				<color>BLUE</color>
      				<completeBuild>true</completeBuild>
      			</threshold>
      			<upsteamProjects/>
      		</org.lonkar.jobfanin.FanInReverseBuildTrigger>
      	</triggers>
      [...]
      
      • Move the upstream job (for example inside a folder)

      This causes the dependencyGraph to be persisted:

      [...]
      <triggers>
      	<org.lonkar.jobfanin.FanInReverseBuildTrigger plugin="job-fan-in@1.1.3">
      		<spec/>
      		<upstreamProjects>FolderA/upstream</upstreamProjects>
      		<watchUpstreamRecursively>false</watchUpstreamRecursively>
      		<threshold>
      			<name>SUCCESS</name>
      			<ordinal>0</ordinal>
      			<color>BLUE</color>
      			<completeBuild>true</completeBuild>
      		</threshold>
      		<upsteamProjects/>
      		<dependencyGraph>
      			<forward class="java.util.Collections$UnmodifiableMap">
      				<m>
      					<entry>
      						<project resolves-to="hudson.model.AbstractItem$Replacer">
      							<fullName>upstream</fullName>
      						</project>
      						<java.util.Collections_-UnmodifiableRandomAccessList resolves-to="java.util.Collections$UnmodifiableList">
      							<c class="list">
      								<hudson.model.DependencyGraph_-DependencyGroup>
      									<group class="linked-hash-set">
      										<org.lonkar.jobfanin.FanInReverseBuildTrigger_-1>
      											<upstream class="project" reference="../../../../../../project"/>
      											<downstream class="project" reference="../../../../../../../../../../../.."/>
      											<description/>
      											<val_-downstream class="project" reference="../../../../../../../../../../../.."/>
      											<outer-class reference="../../../../../../../../../.."/>
      										</org.lonkar.jobfanin.FanInReverseBuildTrigger_-1>
      									</group>
      									<upstream class="project" reference="../../../../project"/>
      									<downstream class="project" reference="../../../../../../../../../.."/>
      								</hudson.model.DependencyGraph_-DependencyGroup>
      							</c>
      							<list reference="../c"/>
      						</java.util.Collections_-UnmodifiableRandomAccessList>
      					</entry>
      				</m>
      			</forward>
      			<backward class="java.util.Collections$UnmodifiableMap">
      				<m>
      					<entry>
      						<project reference="../../../../../../.."/>
      						<java.util.Collections_-UnmodifiableRandomAccessList resolves-to="java.util.Collections$UnmodifiableList">
      							<c class="list">
      								<hudson.model.DependencyGraph_-DependencyGroup>
      									<group class="linked-hash-set">
      										<org.lonkar.jobfanin.FanInReverseBuildTrigger_-1 reference="../../../../../../../../forward/m/entry/java.util.Collections_-UnmodifiableRandomAccessList/c/hudson.model.DependencyGraph_-DependencyGroup/group/org.lonkar.jobfanin.FanInReverseBuildTrigger_-1"/>
      									</group>
      									<upstream class="project" reference="../../../../../../../forward/m/entry/project"/>
      									<downstream class="project" reference="../../../../../../../../../.."/>
      								</hudson.model.DependencyGraph_-DependencyGroup>
      							</c>
      							<list reference="../c"/>
      						</java.util.Collections_-UnmodifiableRandomAccessList>
      					</entry>
      				</m>
      			</backward>
      			<built>true</built>
      			<topologicalOrder class="hudson.model.DependencyGraph$2">
      				<val_-topoOrder>
      					<entry>
      						<project reference="../../../../forward/m/entry/project"/>
      						<int>0</int>
      					</entry>
      					<entry>
      						<project reference="../../../../../../.."/>
      						<int>1</int>
      					</entry>
      				</val_-topoOrder>
      				<outer-class reference="../.."/>
      			</topologicalOrder>
      			<topologicallySorted class="java.util.Collections$UnmodifiableRandomAccessList" resolves-to="java.util.Collections$UnmodifiableList">
      				<c class="list">
      					<project reference="../../../forward/m/entry/project"/>
      					<project reference="../../../../../.."/>
      				</c>
      				<list reference="../c"/>
      			</topologicallySorted>
      		</dependencyGraph>
      	</org.lonkar.jobfanin.FanInReverseBuildTrigger>
      </triggers>
      [...]
      

      With recent version of core, you would also see a WARNING:

      Jul 02, 2018 10:05:41 AM hudson.XmlFile replaceIfNotAtTopLevel
      WARNING: JENKINS-45892: reference to hudson.model.FreeStyleProject@773618d6[upstream] being saved from unexpected /tmp/je-1-home/jobs/downstream/config.xml
      java.lang.IllegalStateException
      	at hudson.XmlFile.replaceIfNotAtTopLevel(XmlFile.java:222)
              [...]
      

        Attachments

          Issue Links

            Activity

            allan_burdajewicz Allan BURDAJEWICZ created issue -
            allan_burdajewicz Allan BURDAJEWICZ made changes -
            Field Original Value New Value
            Link This issue relates to JENKINS-45892 [ JENKINS-45892 ]
            allan_burdajewicz Allan BURDAJEWICZ made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            allan_burdajewicz Allan BURDAJEWICZ made changes -
            Remote Link This issue links to "PR#3 (Web Link)" [ 21130 ]
            allan_burdajewicz Allan BURDAJEWICZ made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            Hide
            lonkaryogeshr Yogesh Lonkar added a comment -

            Fixed in 1.1.4 release.

            Show
            lonkaryogeshr Yogesh Lonkar added a comment - Fixed in 1.1.4 release.
            lonkaryogeshr Yogesh Lonkar made changes -
            Status In Review [ 10005 ] Resolved [ 5 ]
            Resolution Fixed [ 1 ]

              People

              • Assignee:
                lonkaryogeshr Yogesh Lonkar
                Reporter:
                allan_burdajewicz Allan BURDAJEWICZ
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: