← Back to team overview

papercuts-ninja team mailing list archive

[Bug 1630516] [NEW] Logrotate fails repeatedly with /etc/cron.daily/logrotate: error: error creating output file /var/log/... File exists run-parts: /etc/cron.daily/logrotate exited with return code 1

 

You have been subscribed to a public bug by Thomas A. F. Thorne (tafthorne):

Good afternoon.  
I have started seeing something very similar to Debian Dug 734688 "Logs are not rotated for a month" but in the latest Ubuntu LTS (16.04).  I seem to have 
$ logrotate --version
logrotate 3.8.7
bundled in it.  A few weeks ago I started getting root emails such as this:
> Subject: Cron <root@warden> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
>
> /etc/cron.daily/logrotate:
> error: error creating output file /var/log/munin/munin-node.log.1: File exists
> run-parts: /etc/cron.daily/logrotate exited with return code 1
When I inspected the area of concern I was able to see that there was an existing .1 file.  
manager@warden:/var/log/munin$ ll
total 580
drwxr-xr-x  2 munin adm      4096 Sep 27 06:31 ./
drwxr-xr-x 13 root  syslog   4096 Oct  5 06:26 ../
-rw-r--r--  1 root  root     3440 Sep 26 13:39 munin-node-configure.log
-rw-r--r--  1 root  root   490251 Oct  5 10:25 munin-node.log
-rw-r--r--  1 root  root    56598 Sep 21 02:01 munin-node.log.1
-rw-r--r--  1 root  root    24576 Aug 31 02:01 munin-node.log.2
-rw-r--r--  1 root  root     1906 Sep 19 06:25 munin-node.log.8.gz
The contents of the munin-node.log file seem to run from the 19th September until today.  Unlike other parts of this bug the .1 and .2 files do not seem to be already compressed.  

I deleted all but the munin-node.log file to see if it would resolve the problem and was going to leave it at that.  Then I noticed that I have had another Ubuntu machine which has been sending similar emails for the past week:
> Subject: Cron <root@trac> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
>
> /etc/cron.daily/logrotate:
> error: error creating output file /var/log/syslog.1.gz: File exists
> run-parts: /etc/cron.daily/logrotate exited with return code 1
Different file, different machine but a very similar error message.  

Checking on the syslog file I can see that it better fits with other reports on this bug as my duplicated .1 files has a corresponding .1.gz file.  
manager@trac:/var/log$ ll syslog*
-rw-r----- 1 syslog adm 918492 Oct  5 10:30 syslog
-rw-r----- 1 syslog adm 125819 Sep 30 06:25 syslog.1
-rw-r----- 1 syslog adm  20638 Oct  2 02:01 syslog.1.gz
-rw-r----- 1 syslog adm  41989 Sep 30 02:00 syslog.2.gz
-rw-r----- 1 syslog adm  18654 Sep 28 02:01 syslog.3.gz
-rw-r----- 1 syslog adm  31720 Sep 26 06:40 syslog.4.gz
-rw-r----- 1 syslog adm  33151 Sep 25 02:01 syslog.5.gz
-rw-r----- 1 syslog adm  17290 Sep 23 02:01 syslog.6.gz
-rw-r----- 1 syslog adm  39275 Sep 21 06:35 syslog.7.gz

There has been some speculation that a full or nearly full /var
partition would cause this issue.  I can confirm that /var is part of /
on my systems and that presently both of them have several gigabytes of
space.  I run Munin an Icinga to monitor system state.  Neither of these
show / being completely full in the past month.  They have both had
/boot fill significantly.  Trac had a highest use value of / being
99.28% full in the past year  but warden has only had a peak of 33% in
the past year.

A quick search of the internet suggests a couple of other people reporting similar issues: 
https://github.com/gitlabhq/gitlabhq/issues/6894
http://raspberrypi.stackexchange.com/questions/22545/why-are-system-logs-not-rotating 

I do not believe that I have altered by logrotate configuration files but here is a copy of the ones I know about:
$ cat /etc/logrotate.conf 
# see "man logrotate" for details
# rotate log files weekly
weekly

# use the syslog group by default, since this is the owning group
# of /var/log/syslog.
su root syslog

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

$ cat /etc/logrotate.d/rsyslog 
/var/log/syslog
{
            rotate 7
            daily
            missingok
            notifempty
            delaycompress
            compress
            postrotate
                        reload rsyslog >/dev/null 2>&1 || true
            endscript
}

/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
            rotate 4
            weekly
            missingok
            notifempty
            compress
            delaycompress
            sharedscripts
            postrotate
                        reload rsyslog >/dev/null 2>&1 || true
            endscript
}

manager@trac:/var/log$ cat /etc/logrotate.d/munin-node 
/var/log/munin/munin-node.log {
            daily
            missingok
            rotate 7
            compress
            copytruncate
            notifempty
            create 640 root root
}

I am confident that is all the files which are relative to the configuration:
manager@trac:/var/log$ rgrep syslog /etc/logrotate.*
/etc/logrotate.conf:# use the syslog group by default, since this is the owning group
/etc/logrotate.conf:# of /var/log/syslog.
/etc/logrotate.conf:su root syslog
/etc/logrotate.d/ufw:                invoke-rc.d rsyslog reload >/dev/null 2>&1 || true
/etc/logrotate.d/rsyslog:/var/log/syslog
/etc/logrotate.d/rsyslog:                       reload rsyslog >/dev/null 2>&1 || true
/etc/logrotate.d/rsyslog:                       reload rsyslog >/dev/null 2>&1 || true
manager@trac:/var/log$ rgrep munin /etc/logrotate.*
/etc/logrotate.d/munin-node:/var/log/munin/munin-node.log

If there is anything else I can do to help diagnose the issue please let
me know.

Usual boiler plate bits: 
"trac" machine:
$ lsb_release -rd
Description:	Ubuntu 14.04.5 LTS
Release:	14.04

$ apt-cache policy logrotate
logrotate:
  Installed: 3.8.7-1ubuntu1
  Candidate: 3.8.7-1ubuntu1
  Version table:
 *** 3.8.7-1ubuntu1 0
        500 http://gb.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
        100 /var/lib/dpkg/status

"warden" machine:
$ lsb_release -rd
Description:	Ubuntu 16.04.1 LTS
Release:	16.04

$ apt-cache policy logrotate 
logrotate:
  Installed: 3.8.7-2ubuntu2
  Candidate: 3.8.7-2ubuntu2
  Version table:
 *** 3.8.7-2ubuntu2 500
        500 http://gb.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status
N: Ignoring file '50unattended-upgrades.ucf-dist' in directory '/etc/apt/apt.conf.d/' as it has an invalid filename extension

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: logrotate 3.8.7-1ubuntu1
ProcVersionSignature: Ubuntu 4.2.0-42.49~14.04.1-generic 4.2.8-ckt12
Uname: Linux 4.2.0-42-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.21
Architecture: amd64
Date: Wed Oct  5 11:14:53 2016
InstallationDate: Installed on 2015-12-14 (295 days ago)
InstallationMedia: Ubuntu-Server 14.04.3 LTS "Trusty Tahr" - Beta amd64 (20150805)
SourcePackage: logrotate
UpgradeStatus: No upgrade log present (probably fresh install)

** Affects: logrotate (Ubuntu)
     Importance: Undecided
         Status: Confirmed

** Affects: logrotate (Debian)
     Importance: Unknown
         Status: Fix Released


** Tags: amd64 apport-bug trusty
-- 
Logrotate fails repeatedly with /etc/cron.daily/logrotate: error: error creating output file /var/log/... File exists run-parts: /etc/cron.daily/logrotate exited with return code 1
https://bugs.launchpad.net/bugs/1630516
You received this bug notification because you are a member of Papercuts Ninjas, which is subscribed to the bug report.