Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48188

Broken Symlinks with External Build Record Root Directory and Maven Modules

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: core, maven-plugin
    • Environment:
      Jenkins 2.74
      Maven Integration Plugin 3.0
      Maven 3.5.2
      Java 1.8.0_151
      * Also replicates with Jenkins 2.7.4, Maven Integration Plugin 2.14, Maven 3.3.9, Java 1.8.0_77
    • Similar Issues:

      Description

      Assume that we have a job named this_job which is Maven project consisting of two modules, module1, module2 and so on, with single groupId. The project is large (>100 modules), so I want to keep builds separate from configuration. My default location is /jenkins/jobs (where /jenkins is ${JENKINS_HOME}) and I want to use network drive mounted at /data/jobs.

      When using Jenkins with default configuration we have directory structure that looks something like this: 

      /jenkins/jobs/this_job
      ├── builds
      ├── config.xml
      ├── disk-usage.xml
      ├── lastStable -> builds/lastStableBuild
      ├── lastSuccessful -> builds/lastSuccessfulBuild
      ├── modules
      └── nextBuildNumber
      

      Where modules and builds are separate directories. In builds directory we have symlinks to appropriate modules in modules directory: 

      $ ll /jenkins/jobs/this_job/builds/1
      total 576
      drwxr-xr-x 2 jenkins jenkins 4096 Nov 23 08:22 ./
      drwxr-xr-x 4 jenkins jenkins 4096 Nov 23 08:22 ../
      -rw-r--r-- 1 jenkins jenkins 17251 Nov 23 08:22 build.xml
      -rw-r--r-- 1 jenkins jenkins 0 Nov 23 08:21 changelog.xml
      lrwxrwxrwx 1 jenkins jenkins 51 Nov 23 08:21 groupId$module1 -> ../../modules/groupId$module1/builds/12/
      lrwxrwxrwx 1 jenkins jenkins 55 Nov 23 08:22 groupId$module2 -> ../../modules/groupId$module2/builds/12/
      

      Now if we change build record directory in Jenkins Configuration to /jenkins-jobs/${ITEM_FULL_NAME}/builds, the symlinks will still contain relative path to modules directory (../../modules), but the modules directory will not be created. In fact, it will continue to exist (and be updated) at the old location:

      /jenkins/jobs/this_job
      ├── config.xml
      ├── disk-usage.xml
      ├── lastStable -> builds/lastStableBuild
      ├── lastSuccessful -> builds/lastSuccessfulBuild
      ├── modules
      └── nextBuildNumber
      

       

      The modules will be put directly at the root of new directory. So instead of having new directory look like this: 

      /data/jobs/this_job
      ├── builds
      └── modules
        ├── groupId$module1
        └── groupId$module2
      

      I end with that:

      /data/jobs/this_job
      ├── builds
      ├── groupId$module1
      └── groupId$module2
      

      Which does not look that bad, until you have a projects that consist of >100 modules which all lie at root directory.

      Now if I go to builds directory, the symlinks will still exists, but they will point to modules directory, which does not exist in new location. So effectively we have hundreds of broken symlinks:

      mamisiak@kworkstation /data/jobs/this_job/builds/2 $ ll
      total 576
      drwxr-xr-x 2 jenkins jenkins 4096 Nov 23 08:47 ./
      drwxr-xr-x 28 jenkins jenkins 4096 Nov 23 08:47 ../
      -rw-r--r-- 1 jenkins jenkins 17251 Nov 23 08:47 build.xml
      -rw-r--r-- 1 jenkins jenkins 0 Nov 23 08:46 changelog.xml
      lrwxrwxrwx 1 jenkins jenkins 51 Nov 23 08:46 groupId$module1 -> ../../modules/groupId$module1/builds/2
      lrwxrwxrwx 1 jenkins jenkins 55 Nov 23 08:46 groupId$module2 -> ../../modules/groupId$module2/builds/2
      
      

      So I tried to use the second variable, ITEM_FULLNAME. This didn't turn out great, as it replaced $ to : in files names in new directory but not in the old. So I have got now ambiguous links: groupId$module1 -> /data/jobs/groupId:module2 and new data directory keeps everything, so I also have this:

      /data/jobs/this_job
      ├── builds
      ├── groupId:module1
      ├── groupId:module2
      ├── groupId$module1
      └── groupId$module2
      

      The should be configuration option for linking 'builds' and 'modules' directory. Clearly, changing build also changes modules, but does so incorrectly.

      How to replicate

      Create vanilla Jenkins with version as above. Go to Manage Jenkins / System Configuration and change Build Record Root Directory to any directory outside $JENKINS_HOME. Create Maven project and build.

      Is it an issue?

      This may be configuration problem, but if so it misses documentation. This is unclear why there are two variables, ITEM_FULLNAME and ITEM_FULL_NAME, which behave differently. I am missing documentation, which variable is for what purpose. With neither of two I can get the anticipated reasons. It creates mess in new Jenkins build directory. No builds are broken because of that, but it is possible that it affects the build times for incremental Maven builds, if the modules can't be found.

        Attachments

          Issue Links

            Activity

            tadek Tadeusz Kosciuszko created issue -
            tadek Tadeusz Kosciuszko made changes -
            Field Original Value New Value
            Description Assume that we have a job named this_job which is Maven project consisting of two modules, module1, module2 and so on, with single groupId. The project is large (>100 modules), so I want to keep builds separate from configuration. My default location is /jenkins/jobs (where /jenkins is $\{JENKINS_HOME}) and I want to use network drive mounted at /data/jobs.

            When using Jenkins with default configuration we have directory structure that looks something like this:

             
            {code:java}
            /jenkins/jobs/this_job
            ├── builds
            ├── config.xml
            ├── disk-usage.xml
            ├── lastStable -> builds/lastStableBuild
            ├── lastSuccessful -> builds/lastSuccessfulBuild
            ├── modules
            └── nextBuildNumber
            {code}
            Where modules and builds are separate directories. In builds directory we have symlinks to appropriate modules in modules directory:

             

             
            {code:java}
            $ ll /jenkins/jobs/this_job/builds/1
            total 576
            drwxr-xr-x 2 jenkins jenkins 4096 Nov 23 08:22 ./
            drwxr-xr-x 4 jenkins jenkins 4096 Nov 23 08:22 ../
            -rw-r--r-- 1 jenkins jenkins 17251 Nov 23 08:22 build.xml
            -rw-r--r-- 1 jenkins jenkins 0 Nov 23 08:21 changelog.xml
            lrwxrwxrwx 1 jenkins jenkins 51 Nov 23 08:21 groupId$module1 -> ../../modules/groupId$module1/builds/12/
            lrwxrwxrwx 1 jenkins jenkins 55 Nov 23 08:22 groupId$module2 -> ../../modules/groupId$module2/builds/12/
            {code}
            Now if we change build record directory in Jenkins Configuration to /jenkins-jobs/$\{ITEM_FULL_NAME}/builds, the symlinks will still contain relative path to modules directory (../../modules), but the modules directory will not be created. In fact, it will continue to exist (and be updated) at the old location:
            {code:java}
            /jenkins/jobs/this_job
            ├── config.xml
            ├── disk-usage.xml
            ├── lastStable -> builds/lastStableBuild
            ├── lastSuccessful -> builds/lastSuccessfulBuild
            ├── modules
            └── nextBuildNumber
            {code}
            And the modules will be put directly at the root of new directory. So instead of having new directory look like this:

             
            {code:java}
            /data/jobs/this_job
            ├── builds
            └── modules
              ├── groupId$module1
              └── groupId$module2
            {code}
            I end with that:
            {code:java}
            /data/jobs/this_job
            ├── builds
            ├── groupId$module1
            └── groupId$module2
            {code}
            Which does not look that bad, until you have a projects that consist of >100 modules which all lie at root directory.

            Now if I go to builds directory, the symlinks will still exists, but they will point to modules directory, which does not exist in new location. So effectively we have hundreds of broken symlinks:
            {code:java}
            mamisiak@kworkstation /data/jobs/this_job/builds/2 $ ll
            total 576
            drwxr-xr-x 2 jenkins jenkins 4096 Nov 23 08:47 ./
            drwxr-xr-x 28 jenkins jenkins 4096 Nov 23 08:47 ../
            -rw-r--r-- 1 jenkins jenkins 17251 Nov 23 08:47 build.xml
            -rw-r--r-- 1 jenkins jenkins 0 Nov 23 08:46 changelog.xml
            lrwxrwxrwx 1 jenkins jenkins 51 Nov 23 08:46 groupId$module1 -> ../../modules/groupId$module1/builds/2
            lrwxrwxrwx 1 jenkins jenkins 55 Nov 23 08:46 groupId$module2 -> ../../modules/groupId$module2/builds/2

            {code}
            So I tried to use the second variable, ITEM_FULLNAME. This didn't turn out great, as it replaced $ to : in files names in new directory but not in the old. So I have got now ambiguous links: groupId$module1 -> /data/jobs/groupId:module2 and new data directory keeps everything, so I also have this:
            {code:java}
            /data/jobs/this_job
            ├── builds
            ├── groupId:module1
            ├── groupId:module2
            ├── groupId$module1
            └── groupId$module2
            {code}
            The should be configuration option for linking 'builds' and 'modules' directory. Clearly, changing build also changes modules, but does so incorrectly.

            *How to replicate*

            Create vanilla Jenkins with version as above. Go to Manage Jenkins / System Configuration and change Build Record Root Directory to any directory outside $JENKINS_HOME. Create Maven project and build.

            *Is it an issue?*

            This may be configuration problem, but if so it misses documentation. This is unclear why there are two variables, ITEM_FULLNAME and ITEM_FULL_NAME, which behave differently. I am missing documentation, which variable is for what purpose. With neither of two I can get the anticipated reasons. It creates mess in new Jenkins build directory. No builds are broken because of that, but it is possible that it affects the build times for incremental Maven builds, if the modules can't be found.
            Assume that we have a job named this_job which is Maven project consisting of two modules, module1, module2 and so on, with single groupId. The project is large (>100 modules), so I want to keep builds separate from configuration. My default location is /jenkins/jobs (where /jenkins is $\{JENKINS_HOME}) and I want to use network drive mounted at /data/jobs.

            When using Jenkins with default configuration we have directory structure that looks something like this: 
            {code:java}
            /jenkins/jobs/this_job
            ├── builds
            ├── config.xml
            ├── disk-usage.xml
            ├── lastStable -> builds/lastStableBuild
            ├── lastSuccessful -> builds/lastSuccessfulBuild
            ├── modules
            └── nextBuildNumber
            {code}
            Where modules and builds are separate directories. In builds directory we have symlinks to appropriate modules in modules directory: 
            {code:java}
            $ ll /jenkins/jobs/this_job/builds/1
            total 576
            drwxr-xr-x 2 jenkins jenkins 4096 Nov 23 08:22 ./
            drwxr-xr-x 4 jenkins jenkins 4096 Nov 23 08:22 ../
            -rw-r--r-- 1 jenkins jenkins 17251 Nov 23 08:22 build.xml
            -rw-r--r-- 1 jenkins jenkins 0 Nov 23 08:21 changelog.xml
            lrwxrwxrwx 1 jenkins jenkins 51 Nov 23 08:21 groupId$module1 -> ../../modules/groupId$module1/builds/12/
            lrwxrwxrwx 1 jenkins jenkins 55 Nov 23 08:22 groupId$module2 -> ../../modules/groupId$module2/builds/12/
            {code}
            Now if we change build record directory in Jenkins Configuration to /jenkins-jobs/$\{ITEM_FULL_NAME}/builds, the symlinks will still contain relative path to modules directory (../../modules), but the modules directory will not be created. In fact, it will continue to exist (and be updated) at the old location:
            {code:java}
            /jenkins/jobs/this_job
            ├── config.xml
            ├── disk-usage.xml
            ├── lastStable -> builds/lastStableBuild
            ├── lastSuccessful -> builds/lastSuccessfulBuild
            ├── modules
            └── nextBuildNumber
            {code}
             

            The modules will be put directly at the root of new directory. So instead of having new directory look like this: 
            {code:java}
            /data/jobs/this_job
            ├── builds
            └── modules
              ├── groupId$module1
              └── groupId$module2
            {code}
            I end with that:
            {code:java}
            /data/jobs/this_job
            ├── builds
            ├── groupId$module1
            └── groupId$module2
            {code}
            Which does not look that bad, until you have a projects that consist of >100 modules which all lie at root directory.

            Now if I go to builds directory, the symlinks will still exists, but they will point to modules directory, which does not exist in new location. So effectively we have hundreds of broken symlinks:
            {code:java}
            mamisiak@kworkstation /data/jobs/this_job/builds/2 $ ll
            total 576
            drwxr-xr-x 2 jenkins jenkins 4096 Nov 23 08:47 ./
            drwxr-xr-x 28 jenkins jenkins 4096 Nov 23 08:47 ../
            -rw-r--r-- 1 jenkins jenkins 17251 Nov 23 08:47 build.xml
            -rw-r--r-- 1 jenkins jenkins 0 Nov 23 08:46 changelog.xml
            lrwxrwxrwx 1 jenkins jenkins 51 Nov 23 08:46 groupId$module1 -> ../../modules/groupId$module1/builds/2
            lrwxrwxrwx 1 jenkins jenkins 55 Nov 23 08:46 groupId$module2 -> ../../modules/groupId$module2/builds/2

            {code}
            So I tried to use the second variable, ITEM_FULLNAME. This didn't turn out great, as it replaced $ to : in files names in new directory but not in the old. So I have got now ambiguous links: groupId$module1 -> /data/jobs/groupId:module2 and new data directory keeps everything, so I also have this:
            {code:java}
            /data/jobs/this_job
            ├── builds
            ├── groupId:module1
            ├── groupId:module2
            ├── groupId$module1
            └── groupId$module2
            {code}
            The should be configuration option for linking 'builds' and 'modules' directory. Clearly, changing build also changes modules, but does so incorrectly.

            *How to replicate*

            Create vanilla Jenkins with version as above. Go to Manage Jenkins / System Configuration and change Build Record Root Directory to any directory outside $JENKINS_HOME. Create Maven project and build.

            *Is it an issue?*

            This may be configuration problem, but if so it misses documentation. This is unclear why there are two variables, ITEM_FULLNAME and ITEM_FULL_NAME, which behave differently. I am missing documentation, which variable is for what purpose. With neither of two I can get the anticipated reasons. It creates mess in new Jenkins build directory. No builds are broken because of that, but it is possible that it affects the build times for incremental Maven builds, if the modules can't be found.
            tadek Tadeusz Kosciuszko made changes -
            Link This issue is related to JENKINS-12251 [ JENKINS-12251 ]
            tadek Tadeusz Kosciuszko made changes -
            Link This issue depends on JENKINS-8446 [ JENKINS-8446 ]
            tadek Tadeusz Kosciuszko made changes -
            Link This issue is related to JENKINS-19764 [ JENKINS-19764 ]
            tadek Tadeusz Kosciuszko made changes -
            Link This issue is related to JENKINS-11716 [ JENKINS-11716 ]
            tadek Tadeusz Kosciuszko made changes -
            Link This issue is related to JENKINS-17138 [ JENKINS-17138 ]
            tadek Tadeusz Kosciuszko made changes -
            Priority Minor [ 4 ] Major [ 3 ]
            Hide
            danielbeck Daniel Beck added a comment -

            This is unclear why there are two variables, ITEM_FULLNAME and ITEM_FULL_NAME, which behave differently. I am missing documentation, which variable is for what purpose

            One existed first, then we found out about its problem with filesystem unsafe names. Hence the second was added. But you're correct, https://github.com/jenkinsci/jenkins/blob/master/core/src/main/resources/jenkins/model/Jenkins/help-rawBuildsDir.html does not mention the old one, or the reason for the change.


            While well written, this issue seems to be about two mostly unrelated problems: The documentation issue above, and https://github.com/jenkinsci/maven-plugin/blob/f6112cb4e7fece3c5dd48c93a68f61649bad0b7b/src/main/java/hudson/maven/MavenModuleSetBuild.java#L607 making invalid assumptions about directory structure (and perhaps maven-plugin not supporting external build record storage at all).

            Show
            danielbeck Daniel Beck added a comment - This is unclear why there are two variables, ITEM_FULLNAME and ITEM_FULL_NAME, which behave differently. I am missing documentation, which variable is for what purpose One existed first, then we found out about its problem with filesystem unsafe names. Hence the second was added. But you're correct, https://github.com/jenkinsci/jenkins/blob/master/core/src/main/resources/jenkins/model/Jenkins/help-rawBuildsDir.html does not mention the old one, or the reason for the change. While well written, this issue seems to be about two mostly unrelated problems: The documentation issue above, and https://github.com/jenkinsci/maven-plugin/blob/f6112cb4e7fece3c5dd48c93a68f61649bad0b7b/src/main/java/hudson/maven/MavenModuleSetBuild.java#L607 making invalid assumptions about directory structure (and perhaps maven-plugin not supporting external build record storage at all).

              People

              • Assignee:
                Unassigned
                Reporter:
                tadek Tadeusz Kosciuszko
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: