Debian Bug report logs - #773538
systemd: journal is quite big compared to rsyslog output

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <Martin@Lichtvoll.de>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: systemd: journal is quite big compared to rsyslog output

Date: Fri, 19 Dec 2014 17:32:33 +0100

Package: systemd
Version: 218-2
Severity: normal

Dear Maintainer,

I have this here:

merkaba:~> du -sch /var/log/* | sort -rh | head -10
1,3G    insgesamt
1,1G    /var/log/journal
143M    /var/log/atop
53M     /var/log/collectl
13M     /var/log/installer
6,0M    /var/log/kern.log.3.gz
5,9M    /var/log/debug.3.gz
3,9M    /var/log/atop.log.8
3,6M    /var/log/atop.log.10
3,5M    /var/log/atop.log.14

I think the journal takes quite a bit of space compared to what rsyslog
produces with its standard logrotate settings (especially if you
substract the 143M atop performance data):

merkaba:~> cat /etc/logrotate.d/rsyslog
/var/log/syslog
{
        rotate 7
        daily
        missingok
        notifempty
        delaycompress
        compress
        postrotate
                invoke-rc.d rsyslog rotate > /dev/null
        endscript
}

/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
        rotate 4
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                invoke-rc.d rsyslog rotate > /dev/null
        endscript
}



Do you have any intention to reduce this at least a bit?

Hmm, I see this defaults to 10% (SystemMaxUse) of the filesystem. This can
get pretty large with a larger root filesystem.

Ah, it seems I can set a "MaxRetentionSec=" to "14day" or so.

Okay, that will cover my needs I think.

Still, I send this as you raise the question whether the current default
is good for systems with SSD or embedded systems. At least it can easily get
much larger than what rsyslog produces by default and may be an unexpected
additional size allocation for administrators of systems.

Thanks,
Martin


-- Package-specific info:

-- System Information:
Debian Release: 8.0
  APT prefers unstable
  APT policy: (500, 'unstable'), (200, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.18.0-tp520 (SMP w/4 CPU cores; PREEMPT)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages systemd depends on:
ii  acl             2.2.52-2
ii  adduser         3.113+nmu3
ii  initscripts     2.88dsf-58
ii  libacl1         2.2.52-2
ii  libaudit1       1:2.4-1+b1
ii  libblkid1       2.25.2-4
ii  libc6           2.19-13
ii  libcap2         1:2.24-6
ii  libcap2-bin     1:2.24-6
ii  libcryptsetup4  2:1.6.6-4
ii  libgcrypt20     1.6.2-4+b1
ii  libkmod2        18-3
ii  liblzma5        5.1.1alpha+20120614-2+b3
ii  libmount1       2.25.2-4
ii  libpam0g        1.1.8-3.1
ii  libselinux1     2.3-2
ii  libsystemd0     218-2
ii  mount           2.25.2-4
ii  sysv-rc         2.88dsf-58
ii  udev            215-8
ii  util-linux      2.25.2-4

Versions of packages systemd recommends:
ii  dbus            1.8.12-1
ii  libpam-systemd  218-2

Versions of packages systemd suggests:
ii  systemd-ui  3-2

-- no debconf information

Message #10 received at 773538@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <Martin@lichtvoll.de>

To: 773538@bugs.debian.org

Subject: systemd: journal is quite big compared to rsyslog output

Date: Fri, 19 Dec 2014 17:38:42 +0100

Hi!

So this works nice. After setting "MaxRetentionSec=14day" and restarting 
"systemd-journald" I get:

merkaba:~> du -sch /var/log/* | sort -rh | head -10
348M    insgesamt
143M    /var/log/atop
76M     /var/log/journal
53M     /var/log/collectl
13M     /var/log/installer
6,0M    /var/log/kern.log.3.gz
5,9M    /var/log/debug.3.gz
3,9M    /var/log/atop.log.8
3,6M    /var/log/atop.log.10
3,5M    /var/log/atop.log.14

Thats nice. :)

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

Message #17 received at 773538@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>

To: Martin Pitt <martin.pitt@ubuntu.com>

Cc: pkg-systemd-maintainers@lists.alioth.debian.org, 773538@bugs.debian.org

Subject: Re: Processed: tagging 773538

Date: Tue, 30 Dec 2014 17:15:07 +0100

[Message part 1 (text/plain, inline)]

Hi Martin,

Am 30.12.2014 um 16:39 schrieb Debian Bug Tracking System:
> Processing commands for control@bugs.debian.org:
> 
>> tags 773538 + pending
> Bug #773538 [systemd] systemd: journal is quite big compared to rsyslog output
> Added tag(s) pending.
>> thanks
> Stopping processing here.
> 
> Please contact me if you need assistance.
> 


I see, that you committed a change which limits the journal to 7 days
via "MaxRetentionSec=7day".

First, it has to be noted, that the persistent journal is not enabled by
default.
Second, I'm not convinced that the current defaults, which are
size-based, are buggy and actually need a "fix". By limiting the journal
to 7 days, we imho unnecessarily make it less useful. I don't see a good
reason to not utilize available free disk space.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

[signature.asc (application/pgp-signature, attachment)]

Message #22 received at 773538@bugs.debian.org (full text, mbox, reply):

From: Martin Pitt <mpitt@debian.org>

To: Michael Biebl <biebl@debian.org>

Cc: pkg-systemd-maintainers@lists.alioth.debian.org, 773538@bugs.debian.org

Subject: Re: Processed: tagging 773538

Date: Wed, 31 Dec 2014 16:52:59 +0100

[Message part 1 (text/plain, inline)]

Hey Michael,

Michael Biebl [2014-12-30 17:15 +0100]:
> Second, I'm not convinced that the current defaults, which are
> size-based, are buggy and actually need a "fix". By limiting the journal
> to 7 days, we imho unnecessarily make it less useful. I don't see a good
> reason to not utilize available free disk space.

Will journald shrink the journal dynamically when disk space gets low
while it's running? It sounded like you needed to restart it for that.
On long-running servers the journal would be restarted very seldomly.
But then again, the same would probably be true for time based
truncation.

If journald DTRT while running, I'm fine with reverting the commit and
closing this bug as wontfix.

Guten Rutsch!

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

[signature.asc (application/pgp-signature, inline)]

Message #31 received at 773538@bugs.debian.org (full text, mbox, reply):

From: Yuri D'Elia <wavexx@thregr.org>

To: Debian Bug Tracking System <773538@bugs.debian.org>

Subject: Re: systemd: journal is quite big compared to rsyslog output

Date: Thu, 03 Dec 2015 01:34:33 +0100

Package: systemd
Version: 228-2
Followup-For: Bug #773538

I'd also like to add that journalctl logging format is not just a bit, but a
*lot* bigger than a regular syslog daemon.

On two similar systems that I have, one configured with rsyslog and one with
journalctl, I have ~200MB for the entire /var/log tree with rsyslog+logrotate
for one year of retention.
  
Meanwhile, I'm at ~420MB for a 2 month retention *just* for /var/log/journal on
the other. I used to have MaxRetentionSec=6month, until logs grew beyond 2GB
and I started to notice. I never thought it would grow so fast (in fact, I
never had a reason to reduce retention below 1Y before).

I don't think SystemMaxUse/KeepFree make sense for a logging daemon. I'm
interested in retention time first. Increasing retention if additional space is
available would be nice, but not at this cost unfortunately.

I like the journalctl interface in general, so working on a more efficient
storage format would be important.

Otherwise, it's actually smarter to remove the retention limit, and just dump
the log to a regular text file before vacuuming. That feels backward.

Send a report that this bug log contains spam.