Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-5784

Usage of kill in logrotate script is non-portable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Component/s: other
    • Labels:
      None

      Description

      The logrotate script uses

      kill -SIGALRM `cat /var/run/hudson.pid`

      which works fine for the bash builtin, but fails for /bin/kill, which only accepts

      usage: kill [ -s signal | -p ] [ -a ] pid ...
             kill -l [ signal ]

      on CentOS 5.

      Using kill -s SIGALRM would make both variants happy and increase portability.

        Activity

        Hide
        pdurbin pdurbin added a comment -

        We are also seeing this problem on RHEL5.5, using the RPM from http://jenkins-ci.org/redhat/ (hudson-1.381-1.1 as this writing):

        /etc/cron.daily/logrotate:
        kill -SIGALRM `cat /var/run/hudson.pid`
        : line 1: kill: SIGALRM: invalid signal specification
        error: error running postrotate script for /var/log/hudson/hudson.log /var/log/hudson/access_log

        The recommendation above (using "kill -s SIGALRM") should fix the problem, but you could also consider calling /bin/kill explicitly and using -ALRM, as in the attached patch. Using "kill [-signal_name] pid" as noted in http://www.opengroup.org/onlinepubs/009695399/utilities/kill.html seems to be the most popular approach on RHEL, at least. Compare /etc/logrotate.d/syslog, for example.

        Show
        pdurbin pdurbin added a comment - We are also seeing this problem on RHEL5.5, using the RPM from http://jenkins-ci.org/redhat/ (hudson-1.381-1.1 as this writing): /etc/cron.daily/logrotate: kill -SIGALRM `cat /var/run/hudson.pid` : line 1: kill: SIGALRM: invalid signal specification error: error running postrotate script for /var/log/hudson/hudson.log /var/log/hudson/access_log The recommendation above (using "kill -s SIGALRM") should fix the problem, but you could also consider calling /bin/kill explicitly and using -ALRM, as in the attached patch. Using "kill [-signal_name] pid" as noted in http://www.opengroup.org/onlinepubs/009695399/utilities/kill.html seems to be the most popular approach on RHEL, at least. Compare /etc/logrotate.d/syslog, for example.
        Hide
        sannies sannies added a comment - - edited

        The patch works fine on our CentOS 5.5 hudson installation. Please apply! Changed priority to major since our guys maintaining the don't like manual steps after updating to a new version

        Show
        sannies sannies added a comment - - edited The patch works fine on our CentOS 5.5 hudson installation. Please apply! Changed priority to major since our guys maintaining the don't like manual steps after updating to a new version
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        changelog.html
        opensuse/SOURCES/jenkins.logrotate
        rpm/SOURCES/jenkins.logrotate
        http://jenkins-ci.org/commit/core/9c01038ab5cbd1a92e0a172a1834028b7862099f
        Log:
        [FIXED JENKINS-5784] logrotate script for RPM/openSUSE bug fix

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html opensuse/SOURCES/jenkins.logrotate rpm/SOURCES/jenkins.logrotate http://jenkins-ci.org/commit/core/9c01038ab5cbd1a92e0a172a1834028b7862099f Log: [FIXED JENKINS-5784] logrotate script for RPM/openSUSE bug fix
        Hide
        oreinert Olav Reinert added a comment -

        The applied patch is incorrect, the signal name is wrong.

        Show
        oreinert Olav Reinert added a comment - The applied patch is incorrect, the signal name is wrong.
        Hide
        oreinert Olav Reinert added a comment -

        New patch to fix logrotate script for openSUSE. It works also when Jenkins is not running. Also, it checks that the ALRM signal really is sent to Jenkins and not some other process.

        Show
        oreinert Olav Reinert added a comment - New patch to fix logrotate script for openSUSE. It works also when Jenkins is not running. Also, it checks that the ALRM signal really is sent to Jenkins and not some other process.
        Hide
        salsa Sami Salonen added a comment -

        Unfortunately, the current solution does not work for me on RHEL 5.5 where log rotation is executed under /bin/sh. I'm facing the same problem as pdurbin, i.e. signal name SIGALRM is not recognised.

        The following do not work, because of invalid signal specification:
        kill -SIGALRM
        kill -s SIGALRM # This is the current solution

        The following do work:
        kill -ALRM
        kill -s ALRM
        /bin/kill -ALRM
        /bin/kill -s ALRM
        /bin/kill -SIGALRM
        /bin/kill -s SIGALRM

        Would it be possible to pick one working alternative above that would play well in other distros also?

        Jenkins 1.425

        $ uname -a
        Linux ... 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
        $ cat /etc/redhat-release
        Red Hat Enterprise Linux Server release 5.5 (Tikanga)
        $ cat /proc/version
        Linux version 2.6.18-194.el5 (mockbuild@x86-005.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Tue Mar 16 21:52:39 EDT 2010

        Show
        salsa Sami Salonen added a comment - Unfortunately, the current solution does not work for me on RHEL 5.5 where log rotation is executed under /bin/sh. I'm facing the same problem as pdurbin, i.e. signal name SIGALRM is not recognised. The following do not work, because of invalid signal specification: kill -SIGALRM kill -s SIGALRM # This is the current solution The following do work: kill -ALRM kill -s ALRM /bin/kill -ALRM /bin/kill -s ALRM /bin/kill -SIGALRM /bin/kill -s SIGALRM Would it be possible to pick one working alternative above that would play well in other distros also? Jenkins 1.425 $ uname -a Linux ... 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux $ cat /etc/redhat-release Red Hat Enterprise Linux Server release 5.5 (Tikanga) $ cat /proc/version Linux version 2.6.18-194.el5 (mockbuild@x86-005.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Tue Mar 16 21:52:39 EDT 2010
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        changelog.html
        rpm/SOURCES/jenkins.logrotate
        http://jenkins-ci.org/commit/jenkins/8fb2f5162e499b5c0c4dc8f88af761b7c490bc38
        Log:
        [FIXED JENKINS-5784]

        Applied the change from Olav Reinert.

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html rpm/SOURCES/jenkins.logrotate http://jenkins-ci.org/commit/jenkins/8fb2f5162e499b5c0c4dc8f88af761b7c490bc38 Log: [FIXED JENKINS-5784] Applied the change from Olav Reinert.
        Hide
        dogfood dogfood added a comment -

        Integrated in jenkins_main_trunk #1606
        [FIXED JENKINS-5784] (Revision 8fb2f5162e499b5c0c4dc8f88af761b7c490bc38)

        Result = SUCCESS
        Kohsuke Kawaguchi : 8fb2f5162e499b5c0c4dc8f88af761b7c490bc38
        Files :

        • changelog.html
        • rpm/SOURCES/jenkins.logrotate
        Show
        dogfood dogfood added a comment - Integrated in jenkins_main_trunk #1606 [FIXED JENKINS-5784] (Revision 8fb2f5162e499b5c0c4dc8f88af761b7c490bc38) Result = SUCCESS Kohsuke Kawaguchi : 8fb2f5162e499b5c0c4dc8f88af761b7c490bc38 Files : changelog.html rpm/SOURCES/jenkins.logrotate
        Hide
        dogfood dogfood added a comment -

        Integrated in jenkins_ui-changes_branch #21
        [FIXED JENKINS-5784] (Revision 8fb2f5162e499b5c0c4dc8f88af761b7c490bc38)

        Result = SUCCESS
        Kohsuke Kawaguchi : 8fb2f5162e499b5c0c4dc8f88af761b7c490bc38
        Files :

        • rpm/SOURCES/jenkins.logrotate
        • changelog.html
        Show
        dogfood dogfood added a comment - Integrated in jenkins_ui-changes_branch #21 [FIXED JENKINS-5784] (Revision 8fb2f5162e499b5c0c4dc8f88af761b7c490bc38) Result = SUCCESS Kohsuke Kawaguchi : 8fb2f5162e499b5c0c4dc8f88af761b7c490bc38 Files : rpm/SOURCES/jenkins.logrotate changelog.html
        Hide
        guyshaanan Guy Shaanan added a comment -

        Still having this problem.
        Jenkins: 1.533
        OS: CentOS 6.5
        We are using the patch (/bin/kill -s ALRM).
        We see the same results as describe in JENKINS-8881.

        After rotation, old file descriptor is not released, eating up disk space, jenkins.log remains 0KB. Only after 'service jenkins restart' things are working fine, until next log rotation.

        Will be happy to provide any other needed information.

        Show
        guyshaanan Guy Shaanan added a comment - Still having this problem. Jenkins: 1.533 OS: CentOS 6.5 We are using the patch (/bin/kill -s ALRM). We see the same results as describe in JENKINS-8881 . After rotation, old file descriptor is not released, eating up disk space, jenkins.log remains 0KB. Only after 'service jenkins restart' things are working fine, until next log rotation. Will be happy to provide any other needed information.
        Hide
        oleg_nenashev Oleg Nenashev added a comment -

        I see the similar issue on RHEL6.4

        BTW, the issue appeared after the service restart with misconfigured user /etc/logrotate.d/jenkins
        After the fixing of the username Jenkins does not rotate logs at all. I have not restarted the server yet...

        Show
        oleg_nenashev Oleg Nenashev added a comment - I see the similar issue on RHEL6.4 BTW, the issue appeared after the service restart with misconfigured user /etc/logrotate.d/jenkins After the fixing of the username Jenkins does not rotate logs at all. I have not restarted the server yet...
        Hide
        guyshaanan Guy Shaanan added a comment -

        With response to Oleg Nenashev:
        I have another Jenkins machine, with same Jenkins version (1.533) but CentOS 6.4.

        On the CentOS 6.4:

        [root@dev ~]# ps aux | grep jenkins
        jenkins   1932  1.8  4.7 5510192 477720 ?      Ssl  May01 562:03 /etc/alternatives/java -Djava.awt.headless=true -DJENKINS_HOME=/workspace/jenkins -XX:PermSize=512m -jar /usr/lib/jenkins/jenkins.war --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=8009 --debug=5 --handlerCountMax=100 --handlerCountMaxIdle=20
        

        On the CentOS 6.5:

        [root@jenkins ~]# ps aux | grep jenkins
        root     26770 1969 60.9 9436916 4913964 ?     Ssl  08:08 8460:14 /opt/jdk_6u31/bin/java -Xmx6144m -Duser.timezone=Europe/Athens -DJENKINS_HOME=/opt/jenkins -XX:PermSize=512m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -XX:+HeapDumpOnOutOfMemoryError -jar /usr/lib/jenkins/jenkins.war --logfile=/var/log/jenkins/jenkins.log --httpPort=80 --debug=5 --handlerCountMax=100 --handlerCountMaxIdle=20
        

        The difference I see is that the 6.4 runs with user "jenkins" and the 6.5 with user "root".
        Could this be the problem?
        Should I change the "root" to "jenkins"? Where?

        Show
        guyshaanan Guy Shaanan added a comment - With response to Oleg Nenashev: I have another Jenkins machine, with same Jenkins version (1.533) but CentOS 6.4. On the CentOS 6.4: [root@dev ~]# ps aux | grep jenkins jenkins 1932 1.8 4.7 5510192 477720 ? Ssl May01 562:03 /etc/alternatives/java -Djava.awt.headless= true -DJENKINS_HOME=/workspace/jenkins -XX:PermSize=512m -jar /usr/lib/jenkins/jenkins.war --logfile=/ var /log/jenkins/jenkins.log --webroot=/ var /cache/jenkins/war --httpPort=8080 --ajp13Port=8009 --debug=5 --handlerCountMax=100 --handlerCountMaxIdle=20 On the CentOS 6.5: [root@jenkins ~]# ps aux | grep jenkins root 26770 1969 60.9 9436916 4913964 ? Ssl 08:08 8460:14 /opt/jdk_6u31/bin/java -Xmx6144m -Duser.timezone=Europe/Athens -DJENKINS_HOME=/opt/jenkins -XX:PermSize=512m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate= false -Dcom.sun.management.jmxremote.ssl= false -Dcom.sun.management.jmxremote.local.only= false -XX:+HeapDumpOnOutOfMemoryError -jar /usr/lib/jenkins/jenkins.war --logfile=/ var /log/jenkins/jenkins.log --httpPort=80 --debug=5 --handlerCountMax=100 --handlerCountMaxIdle=20 The difference I see is that the 6.4 runs with user "jenkins" and the 6.5 with user "root". Could this be the problem? Should I change the "root" to "jenkins"? Where?
        Hide
        oleg_nenashev Oleg Nenashev added a comment -

        /etc/sysconfig/jenkins provides all configurations including the user
        There's an additional bug in JIRA regarding the integration between this config file and the logrotate configuration

        Show
        oleg_nenashev Oleg Nenashev added a comment - /etc/sysconfig/jenkins provides all configurations including the user There's an additional bug in JIRA regarding the integration between this config file and the logrotate configuration
        Hide
        guyshaanan Guy Shaanan added a comment -

        Fixed for me.
        Our Jenkins process was running with user "root".
        The log rotate script had "-user jenkins" inside it. So changed to "-user root" and restarted Jenkins.
        Now everything works as expected.
        Thanks @Oleg Nenashev for your help locating all the different configuration files.

        Show
        guyshaanan Guy Shaanan added a comment - Fixed for me. Our Jenkins process was running with user "root". The log rotate script had "-user jenkins" inside it. So changed to "-user root" and restarted Jenkins. Now everything works as expected. Thanks @Oleg Nenashev for your help locating all the different configuration files.
        Hide
        guyshaanan Guy Shaanan added a comment -

        Just make sure the user running Jenkins and the user found in the /etc/logrotate.d/jenkins are the same user.

        Show
        guyshaanan Guy Shaanan added a comment - Just make sure the user running Jenkins and the user found in the /etc/logrotate.d/jenkins are the same user.
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        SOURCES/jenkins.logrotate
        http://jenkins-ci.org/commit/packaging/3a80ebec8548f4c8c1093aff7003043effb10aac
        Log:
        [FIXED JENKINS-5784] logrotate script for RPM/openSUSE bug fix

        Originally-From: jenkins-ci.org/commit/core/9c01038ab5cbd1a92e0a172a1834028b7862099f

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: SOURCES/jenkins.logrotate http://jenkins-ci.org/commit/packaging/3a80ebec8548f4c8c1093aff7003043effb10aac Log: [FIXED JENKINS-5784] logrotate script for RPM/openSUSE bug fix Originally-From: jenkins-ci.org/commit/core/9c01038ab5cbd1a92e0a172a1834028b7862099f
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        SOURCES/jenkins.logrotate
        http://jenkins-ci.org/commit/packaging/f714817bff7fd132d018d4e9690064498c213eeb
        Log:
        [FIXED JENKINS-5784] logrotate script for RPM/openSUSE bug fix

        Originally-From: jenkins-ci.org/commit/core/9c01038ab5cbd1a92e0a172a1834028b7862099f

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: SOURCES/jenkins.logrotate http://jenkins-ci.org/commit/packaging/f714817bff7fd132d018d4e9690064498c213eeb Log: [FIXED JENKINS-5784] logrotate script for RPM/openSUSE bug fix Originally-From: jenkins-ci.org/commit/core/9c01038ab5cbd1a92e0a172a1834028b7862099f
        Hide
        scm_issue_link SCM/JIRA link daemon added a comment -

        Code changed in jenkins
        User: Kohsuke Kawaguchi
        Path:
        SOURCES/jenkins.logrotate
        http://jenkins-ci.org/commit/packaging/3027ec69285fb765650089b4fb6863906f67ed83
        Log:
        [FIXED JENKINS-5784]

        Applied the change from Olav Reinert.

        Originally-From: jenkins-ci.org/commit/core/8fb2f5162e499b5c0c4dc8f88af761b7c490bc38

        Show
        scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: SOURCES/jenkins.logrotate http://jenkins-ci.org/commit/packaging/3027ec69285fb765650089b4fb6863906f67ed83 Log: [FIXED JENKINS-5784] Applied the change from Olav Reinert. Originally-From: jenkins-ci.org/commit/core/8fb2f5162e499b5c0c4dc8f88af761b7c490bc38

          People

          • Assignee:
            Unassigned
            Reporter:
            rombert rombert
          • Votes:
            6 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: